So here we are in Google Colab for our first collaborative session on the cloud using a GPU box.
On that note, I've actually connected to a T4 box, one of the lower spec boxes.
You can see my resources here.
You can see that I just ran this and managed to fill up most of the GPU, but then I've just gone to
runtime and done a restart session, which is why the GPU memory has come slamming down to nothing.
Anyway, we'll now, uh, remove this screen so that it's not in our way.
There we go.
Uh, and look at our colab.
So I begin by introducing you to the world of pipelines, a reminder that the Transformers library with
hugging face has these two API levels the pipelines.
And then later we will come to Tokenizers and models for today.
It's all about pipelines, high level API that allows you to run inference for common tasks in just
a couple of lines of code, which makes it frictionless and easy to be using open source models in production.
The way that we use it is, as I say, super simple.
You first call pipeline and you pass in a string with the task that you want to do.
So there's various things you can pass in here, various, uh, strings, and you will see what they
are in just a moment.
You get back a pipeline object and you can then call it with your input, and you get back the result.
That's all there is to it.
Simple as that.
So I'm going to start by doing the installs the Transformers library of course, which is the heart
of everything.
Datasets is a library that gives us access to hugging faces, datasets.
Diffusers actually are sort of companion library to transformers when you're talking about diffusion
models that generate images.
Often when I say transformers, I'm really referring to both transformers and its sibling library diffusers
as well for any times that we're generating images.
So I'm doing a pip install.
When you have an exclamation mark like that in front of a of a cell, it means that you want to run
that as a terminal command.
And so that's going to run.
And if you didn't know that minus Q is it puts it on quiet mode so that we don't get all of the outputs
of installing all of the packages, but it installs now installed particularly quickly for me because
I already ran this notebook earlier.
It may take 30s for you while those packages install.
And then we're going to do some imports.
And then once we've done that we're going to get started with pipelines.
Get ready for this.
So first of all sentiment analysis is something positive or negative a bit of text.
And we're going to start by saying I am super excited to be on the way to LM mastery.
I wonder if that's positive or negative.
So a few things to note immediately.
First of all, it warns us that no model was supplied and it's defaulted to this particular model.
What it means there is that you can say model equals and tell it which of the models from the hugging
face hub you would like to use as part of this pipeline.
If you don't supply one, it just picks the default for that task, which is great for our purposes
today.
The other thing it's telling us is that the GPU is available in the environment, but we didn't tell
it to use a GPU, which it correctly thinks is a bit strange of us.
And we can make it do that by saying device equals Cuda like that, and that command will tell it that
we would like to use this pipeline, and we would like to take advantage of the GPU that we have.
And so now we've run it, it's run on our GPU.
And maybe we should also look at the results.
The result is that it's considered a positive statement and its score, which is its level of confidence,
is very high indeed.
So that sounds good.
I think that's probably a good interpretation of the sentence.
Let's just try putting in the word not.
I'm not super excited to be on the way to LLM mastery.
Perish the thought.
I hope you're not thinking that for a moment, but we should just check that.
If we say that, it clearly identifies, then that the label is negative and it's pretty confident that
that is indeed a negative statement to be making.
So that works well, let's leave it on an enthusiastic note.
We would want it to think otherwise.
Named entity recognition is the next task.
I'm just going to be rattling through these pipelines and you should try all of this yourself.
Of course, named entity recognition is when you provide some text and you ask the model to identify
what kinds of things are being referred to.
This is a standard one that that I took.
Barack Obama was the 44th president of the United States.
Uh, and we ask it to analyze that, And it responds here that the there's it responds with, um, two
different named entities.
If you can see it in this text right here, the first named entity is of type per as in person.
Uh, it's got high confidence and the word is Barack Obama.
And the second one is a lock for a location and the word is United States.
And of course, it shows you tells you where that is in the input.
It's a very common use case in data science, a great thing to have at your disposal and to be able
to do so quickly.
Question answering with context, you can create a question answering pipeline.
And again, I'm using this device's Cuda to run it on the GPU and say, who was the 44th president of
the US and provide it with some context here.
So it has something to look up against and ask it to print the result there, and it's simply no surprise
answers that result.
I think.
I'm not trying to show off the power of this model right now.
I'm trying to show off the simplicity of the pipeline API.
You can play with more sophisticated context and better questions, and I'd also encourage you to try
passing in different models to explore some of the different models available on the Hugging Face hub.
Text summarization is just as easy.
You then of course the pipeline.
The type is summarization and you can put in a ton of text.
Here I'm talking about I'm generally gushing about the hugging face Transformers library.
I ask for the summarizer, I give it a min and a max length, and I get back a nice short and sharp
sentence that summarizes that text.
Um, and I think it's mostly just a sort of chop of what I already put in.
So it didn't do a wonderful job, but it's a pretty simple model.
Again, you can explore better summarizations from better models.
When you have a moment, we can translate.
Translation on to English.
To French.
The data scientists were truly amazed by the power and simplicity of the hugging Face pipeline API.
And let's see how it performs for all of you French speakers out there.
I'm not going to try and say that my high school French is good enough, but you can say, astonished
at the power and simplicity of the API of the hugging face pipeline.
And as far as my limited French skills can tell, that seems like it's a pretty robust translation.
Very easy classification, or what's called zero shot classification, when we just give it an example
and ask it to label it with some labels without giving it any prior examples.
So we're giving it the text hugging face Transformers library is amazing, and asking it to classify
technology or sports or politics.
And let's see how it performs.
Um, and it says labels, uh, for technology.
It gives it a score of 95% and then a tiny score for sports and politics.
Politics.
Low of them all.
Uh, and that seems especially as we didn't particularly have any words that were directly tech related.
That's not bad at all.
And then last on this series of the really simple ones is text generation.
Let's say if there's one thing I want you to remember about using Huggingface pipelines, it's and let's
see.
Obviously we're using a vanilla model, but let's see how it handles it.
It's that any application that runs on Nautilus will generate the hugging face package as the target,
just as if it had been compiled.
So that's a bit random.
It did better in some of my, uh, uh, prior tests.
Uh, it's how good and resilient they are.
As with any project, you'll need to remember your risk tolerance.
Remember when to push it to the point.
So of course, it's rambling based on how it thinks that begins.
You can try some more amusing starts and see how it performs.
You could also try bigger, beefier models and you'll get something.
Of course, that is more accurate in terms of the text generation.
Okay.
And then of course, since you are now all experts in Multi-modality, let's just show you some image
generation.
Everything up to this point has been using the Transformers library, and we're now flipping to the
diffusers or diffusion style models, which is the architecture that generates images.
And we are using the well-known stable diffusion model, the parameters a few more need to be passed
in.
As you'll see here.
We need to tell it the kind of data type that we're using.
And then once we've done that and and put it on the GPU, we can give it some text.
And I'm saying here a class of data scientists learning about AI in the surreal style of Salvador Dali,
and let's generate that image.
This will take a little bit longer.
It's a bit more of a meaty task.
Um, it might take even longer for you because I have the benefit of having already run this once.
And so the model has been downloaded from Huggingface and stored in its local directory.
So it may take more like a minute when you run it.
Um, but here we go.
Here are here is the room full of, uh, uh, class of data scientists learning about AI in the surreal
style of Dali.
I love it, I love it.
Do you look anything like these guys?
Uh, I hope not only in your worst nightmare, but there we go.
This is the surreal Dali.
Scary, strange world.
Uh, for a class of data scientists, I love it.
Now, you can substitute this for the flux model that I mentioned before, and you may remember, I
flashed up the code at the last, uh, lectures.
Uh, by all means, use that.
You'll find it does take longer, and you will need to be running on a beefier box than the T4.
If you do it on an A100, it'll take a couple of minutes and you will get a breathtakingly good image
like the one that I showed in the last lecture.
Uh, but, um, this seems pretty good for a quick, cheap model to me.
And last but not least, some audio generation.
Uh, we are going to use the text to speech pipeline now, and we are going to tell it which model we
want.
We want Microsoft's Speech five TTS.
There is a little bit more that you have to provide into the model, something about the type of voice
that it should use.
And so there's a couple of lines to load that data set from hugging face.
Um, and, and get it into the right shape.
But once you've done that, we can just call our pipeline and we're going to say hi to an artificial
intelligence engineer on the way to mastery and pass in this this speech voice.
Um, then this is some code to then, uh, write that to, uh, to a wav file.
Um, and uh, we will then be able to play it within the, um, this, uh, Colab notebook.
So it ran pretty fast.
Let's see how it sounds.
Hi.
To an artificial intelligence engineer on the way to mastery.
Seems pretty good to me.
That would be an example of using pipeline for text to speech generation.
And that wraps up this colab walkthrough of pipeline APIs.
I, of course, will share this colab so that you can have access to this and very much encourage you
to go through and try these yourself, experiment with different inputs and also experiment with different
models that you'll find on the Huggingface hub.
Enjoy.