Welcome to the world of Google Colab.
You may already be very familiar with Google Colab, even if so, I hope I'll show you a couple of things
here and there.
But if not, uh, prepare for a great tool.
Um, and as I say, there's other competitors to Google Colab that are pretty similar.
Um, but this is where I suggest you start or do do the same sort of thing in your, uh, cloud compute
platform of choice.
So the first thing you'll need to do is have a Google account if you don't already have one.
So when you go to this URL colab.research.google.com, uh, if you don't already have a Google account,
it will prompt you to, to, to to create one.
And it's worth it.
There's going to be tons that we can do with it.
So uh, go ahead and do that if you need to.
Um, but uh, for everybody else that has one, you will see this, um, which will give you some information
about Colab.
There's a free tier and there is a paid tier.
There's an awful lot you can do just with the free tier.
Um, and I think in theory, you should be able to do almost everything in our class with the free tier.
It just might take longer.
And the paid tier, you can control how much you spend, and it can be relatively small in a matter
of a few dollars.
So it's certainly something that I'd recommend you consider because it will allow you to get deeper
into training and it will be very satisfying.
So, uh, when you come up with a new Colab notebook, it looks a bit like what gets served up right
away.
It looks very much like a Jupyter notebook.
You have cells that can be code, or they can be text, and you can run code just by clicking in it
and running it.
And this is a kind of default one that comes up.
What we can do is we can go file new notebook in drive.
And it says in drive like that, because this notebook is created in your Google Drive, which is so
convenient.
It has the same kind of construct as making Google Docs, making Google Sheets, and it's done in a
way that you can share it just as you would share anything else.
So here we are.
And the first thing that we see, what looks like a Jupiter notebook over here is a connect button.
And I'm going to show you we can start with change runtime type because it shows you the different kinds
of runtime, the different kinds of VM that we can run a CPU.
In other words, a normal box that doesn't have one of these GPUs, graphics processing units that are
so good at running, uh, parallel matrix maths that sits behind neural networks.
So we can just choose a CPU box, which is very much available on the free tier.
There is a low end GPU box called a T4, which has a smaller GPU attached to it.
This is available on the free plan with some rate limits in terms of how much you can use it, and it's
also very cheap on the paid plan.
Um, there's an L4, which is a bit higher spec, and an A100, which is the strongest one and which
we will use when we want to do things quickly.
It does cost a little bit more, but still we're talking about dollars.
Not not not massive amounts.
$10 will get you.
I think it's it's, uh, with $10, you'd be able to to keep training for about 24 to 48 hours, uh,
using that box constantly.
So it's, uh, it's not not still not going to break the bank, but it is on the radar when you start
using a100s a lot.
Uh, so, um, and of course, you always get to see how much you're spending, and you can always choose
to to go with the cheaper option or go with a free option as you wish.
And when you pick a box, you can have a high Ram version of it.
And that's talking about the CPU, Ram, not the GPU.
The GPU Ram is associated with which instance you pick, but you can choose whether you want a high
CPU, Ram, or not.
So let's just go with a CPU box with normal amount of Ram and connect to that box by pressing the connect
button.
It does take a little while to connect, because it has to hunt down a box and connect to it, but there
we go.
We are now attached to a box.
You go to this dropdown and say View Resources to see what you're working with.
You can see the system Ram.
We've got like almost 13 gigs on this box, and we've got 225 gigs of disk space there.
And I can go over here and I can type something like print hello Data Science World and run that.
And shockingly, we get that message printed.
Uh, so, uh, hopefully no surprises there.
It's a Jupyter notebook running in the cloud on a CPU.
A couple of other things to mention.
If you look down here, there's some useful stuff.
This one here opens up your sort of browser, a file browser, onto your local disk.
This local disk is ephemeral, and then it gets completely wiped once you finished using this box.
So consider it temporary and you can use it to be writing files there that you maybe are then going
to upload your model or data to the Huggingface hub, um, which you will later download somewhere else.
But this is temporary.
This is very important.
This key is for what's called the secrets associated with your notebook.
And this is where you can put in the environment variables that you'll be able to access within your
notebook.
That should not be included in the code of the notebook.
And what you'll see here is I have my anthropic API key.
I have my OpenAI API key and my hugging face token.
That's the thing we created in the last video, and I've got them associated with this notebook.
You can just press Add New Secret to do that.
And it comes associated with all of my notebooks.
Um, because I've got that set up as my colab secrets.
And you can create a new one by pressing Add New Secret there.
You can switch notebook access on here.
I've just seen that there's a Create Gemini key option there.
They're obviously a cross-selling to Gemini, and I know that I that I say that creating Gemini Keys
is is hard.
Maybe they've got an easier path to creating Gemini API keys right there.
So that would be worth trying.
If you haven't already gone through the rigmarole of setting up a Gemini API key.
Uh, so, um, and it's even I was going to say later we'll find out how to access your key from within
the Jupyter notebook.
But wonderfully, they've given you the little, little scriptlet of code just there.
That's what we'll be doing later to be accessing our secrets within the code on the right.
So you should set these up when you get a chance.
When you're working with an actual notebook in particular, you flip this switch on to make sure that
when you execute this code in a cell, it will have access to that secret.
And of course, as you can imagine, the sort of powerful thing about these secrets is that if you share
this notebook with others, then they get all of your code.
But of course, they don't get your secrets shared.
They will have to enter in their own secrets in order to be able to run that code.
And similarly, of course, when I share notebooks for you to use, the same thing will apply.
You'll need to put in your own tokens in order to make take advantage of the code and run it against
the frontier models or use your hugging face hub, um, or whatever.
Okay, let's close that down.
So let me just show you some of the more powerful boxes.
So you remember we can go here and go change runtime type.
Click on T4 to to use that box.
And I did that earlier.
And I did that because uh, it can take a little while to connect to some of these boxes.
And with the really high spec boxes like A100, sometimes it just won't be available and you'll have
to come back and try again two minutes later, and then it will be available.
Invariably it becomes available after a couple of tries, but sometimes they are oversubscribed and
it takes a few attempts.
So this is a T4 box.
If I do view resources, we'll see that we have again 12 and a bit of system Ram.
We have the same a slightly smaller hard drive, I think.
I think it was two, two five before, but it's 200 whatever.
That's plenty of disk space.
And we have a GPU with 15GB of Ram, and 15GB might sound like a huge amount of Ram to have for a GPU.
But as you'll quickly discover when it comes to training deep neural networks, that is a kind of puny
GPU, but it's good enough for our purposes.
We'll be able to use this for this class.
Um, uh, but but it just some things might take a long time.
Uh, this is a bit of code that I just copied from the original colab that Google prompted us with,
which gives us a nice little, uh, printout of details behind this GPU, including how much memory
we're using out of the 15GB.
Although, of course, you can always watch it happening over here.
Uh, so this is the T4 box.
I'm now going to show you the A100 box.
This is the super powered one, and I may splash out and use this from time to time.
Just in the spirit of keeping this class moving fast and showing you, uh, great results really quickly.
Uh, if we view the resources, you'll see what's going on.
Now, we've got a 40 gigabyte Ram, GPU and that that is a beefy GPU.
That is something which will be able to use to do some hefty training.
Um, and we can use this to print more details.
You can see that we are using two megabytes by, uh, when we're not doing anything out of the 40GB
of available memory.
So that's the quick tour of what's going on with Colab.
The one other thing I'll mention is the share button up here.
Uh, if you press the share button, then you will see a very familiar interface, because if you use
Google Drive at all, it looks just like everything else in Google Drive.
You can share these notebooks and with different levels of permission with different groups, and use
that as a way to collaborate really effectively.
Uh, with friends, colleagues, coworkers on the, uh, the AI Jen AI projects that you're working
on.
And it's a super effective way to collaborate, of course.
And that's one of the great benefits of using the Google Colab setup.
All right.
I'll see you back for the next lecture.