You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 

472 lines
14 KiB

WEBVTT
00:00.740 --> 00:03.140
Welcome to the world of Google Colab.
00:03.140 --> 00:07.730
You may already be very familiar with Google Colab, even if so, I hope I'll show you a couple of things
00:07.730 --> 00:08.660
here and there.
00:08.780 --> 00:13.340
But if not, uh, prepare for a great tool.
00:13.610 --> 00:17.630
Um, and as I say, there's other competitors to Google Colab that are pretty similar.
00:17.750 --> 00:25.490
Um, but this is where I suggest you start or do do the same sort of thing in your, uh, cloud compute
00:25.490 --> 00:26.990
platform of choice.
00:26.990 --> 00:33.140
So the first thing you'll need to do is have a Google account if you don't already have one.
00:33.170 --> 00:40.970
So when you go to this URL colab.research.google.com, uh, if you don't already have a Google account,
00:40.970 --> 00:43.850
it will prompt you to, to, to to create one.
00:43.850 --> 00:44.840
And it's worth it.
00:44.840 --> 00:46.880
There's going to be tons that we can do with it.
00:46.880 --> 00:50.240
So uh, go ahead and do that if you need to.
00:50.510 --> 00:56.990
Um, but uh, for everybody else that has one, you will see this, um, which will give you some information
00:56.990 --> 00:58.010
about Colab.
00:58.010 --> 01:00.650
There's a free tier and there is a paid tier.
01:00.650 --> 01:02.900
There's an awful lot you can do just with the free tier.
01:02.960 --> 01:07.320
Um, and I think in theory, you should be able to do almost everything in our class with the free tier.
01:07.320 --> 01:08.820
It just might take longer.
01:09.060 --> 01:13.680
And the paid tier, you can control how much you spend, and it can be relatively small in a matter
01:13.680 --> 01:14.430
of a few dollars.
01:14.430 --> 01:20.490
So it's certainly something that I'd recommend you consider because it will allow you to get deeper
01:20.490 --> 01:23.670
into training and it will be very satisfying.
01:23.760 --> 01:31.980
So, uh, when you come up with a new Colab notebook, it looks a bit like what gets served up right
01:31.980 --> 01:32.460
away.
01:32.460 --> 01:34.890
It looks very much like a Jupyter notebook.
01:34.920 --> 01:41.820
You have cells that can be code, or they can be text, and you can run code just by clicking in it
01:41.820 --> 01:42.840
and running it.
01:42.840 --> 01:45.990
And this is a kind of default one that comes up.
01:46.020 --> 01:49.950
What we can do is we can go file new notebook in drive.
01:49.950 --> 01:55.650
And it says in drive like that, because this notebook is created in your Google Drive, which is so
01:55.650 --> 01:56.280
convenient.
01:56.280 --> 02:03.360
It has the same kind of construct as making Google Docs, making Google Sheets, and it's done in a
02:03.360 --> 02:06.780
way that you can share it just as you would share anything else.
02:06.780 --> 02:08.270
So here we are.
02:09.020 --> 02:14.840
And the first thing that we see, what looks like a Jupiter notebook over here is a connect button.
02:14.840 --> 02:19.400
And I'm going to show you we can start with change runtime type because it shows you the different kinds
02:19.400 --> 02:23.690
of runtime, the different kinds of VM that we can run a CPU.
02:23.690 --> 02:29.480
In other words, a normal box that doesn't have one of these GPUs, graphics processing units that are
02:29.480 --> 02:35.900
so good at running, uh, parallel matrix maths that sits behind neural networks.
02:35.900 --> 02:41.300
So we can just choose a CPU box, which is very much available on the free tier.
02:41.330 --> 02:50.420
There is a low end GPU box called a T4, which has a smaller GPU attached to it.
02:50.420 --> 02:56.390
This is available on the free plan with some rate limits in terms of how much you can use it, and it's
02:56.390 --> 02:58.340
also very cheap on the paid plan.
02:58.550 --> 03:05.630
Um, there's an L4, which is a bit higher spec, and an A100, which is the strongest one and which
03:05.630 --> 03:08.930
we will use when we want to do things quickly.
03:08.960 --> 03:12.150
It does cost a little bit more, but still we're talking about dollars.
03:12.180 --> 03:14.940
Not not not massive amounts.
03:14.940 --> 03:16.050
$10 will get you.
03:16.050 --> 03:23.850
I think it's it's, uh, with $10, you'd be able to to keep training for about 24 to 48 hours, uh,
03:23.850 --> 03:26.580
using that box constantly.
03:26.580 --> 03:32.370
So it's, uh, it's not not still not going to break the bank, but it is on the radar when you start
03:32.370 --> 03:34.170
using a100s a lot.
03:34.830 --> 03:40.020
Uh, so, um, and of course, you always get to see how much you're spending, and you can always choose
03:40.020 --> 03:43.500
to to go with the cheaper option or go with a free option as you wish.
03:43.530 --> 03:46.650
And when you pick a box, you can have a high Ram version of it.
03:46.650 --> 03:48.870
And that's talking about the CPU, Ram, not the GPU.
03:48.900 --> 03:54.030
The GPU Ram is associated with which instance you pick, but you can choose whether you want a high
03:54.030 --> 03:55.290
CPU, Ram, or not.
03:55.290 --> 04:01.770
So let's just go with a CPU box with normal amount of Ram and connect to that box by pressing the connect
04:01.770 --> 04:02.730
button.
04:02.790 --> 04:07.320
It does take a little while to connect, because it has to hunt down a box and connect to it, but there
04:07.320 --> 04:07.680
we go.
04:07.710 --> 04:09.480
We are now attached to a box.
04:09.480 --> 04:15.750
You go to this dropdown and say View Resources to see what you're working with.
04:15.780 --> 04:17.040
You can see the system Ram.
04:17.040 --> 04:24.840
We've got like almost 13 gigs on this box, and we've got 225 gigs of disk space there.
04:25.290 --> 04:35.910
And I can go over here and I can type something like print hello Data Science World and run that.
04:35.910 --> 04:39.150
And shockingly, we get that message printed.
04:39.330 --> 04:42.990
Uh, so, uh, hopefully no surprises there.
04:42.990 --> 04:46.530
It's a Jupyter notebook running in the cloud on a CPU.
04:46.560 --> 04:48.210
A couple of other things to mention.
04:48.210 --> 04:50.370
If you look down here, there's some useful stuff.
04:50.370 --> 04:56.520
This one here opens up your sort of browser, a file browser, onto your local disk.
04:56.550 --> 05:01.380
This local disk is ephemeral, and then it gets completely wiped once you finished using this box.
05:01.380 --> 05:06.900
So consider it temporary and you can use it to be writing files there that you maybe are then going
05:06.900 --> 05:13.290
to upload your model or data to the Huggingface hub, um, which you will later download somewhere else.
05:13.290 --> 05:14.880
But this is temporary.
05:14.910 --> 05:16.290
This is very important.
05:16.290 --> 05:21.000
This key is for what's called the secrets associated with your notebook.
05:21.000 --> 05:26.520
And this is where you can put in the environment variables that you'll be able to access within your
05:26.520 --> 05:27.090
notebook.
05:27.120 --> 05:31.020
That should not be included in the code of the notebook.
05:31.050 --> 05:33.930
And what you'll see here is I have my anthropic API key.
05:33.960 --> 05:37.530
I have my OpenAI API key and my hugging face token.
05:37.530 --> 05:43.890
That's the thing we created in the last video, and I've got them associated with this notebook.
05:43.920 --> 05:46.020
You can just press Add New Secret to do that.
05:46.020 --> 05:48.270
And it comes associated with all of my notebooks.
05:48.450 --> 05:51.870
Um, because I've got that set up as my colab secrets.
05:51.870 --> 05:56.880
And you can create a new one by pressing Add New Secret there.
05:57.270 --> 06:01.590
You can switch notebook access on here.
06:01.860 --> 06:05.280
I've just seen that there's a Create Gemini key option there.
06:05.280 --> 06:10.500
They're obviously a cross-selling to Gemini, and I know that I that I say that creating Gemini Keys
06:10.530 --> 06:11.370
is is hard.
06:11.370 --> 06:15.300
Maybe they've got an easier path to creating Gemini API keys right there.
06:15.300 --> 06:16.740
So that would be worth trying.
06:16.770 --> 06:20.460
If you haven't already gone through the rigmarole of setting up a Gemini API key.
06:20.670 --> 06:26.340
Uh, so, um, and it's even I was going to say later we'll find out how to access your key from within
06:26.340 --> 06:27.180
the Jupyter notebook.
06:27.180 --> 06:30.540
But wonderfully, they've given you the little, little scriptlet of code just there.
06:30.540 --> 06:36.690
That's what we'll be doing later to be accessing our secrets within the code on the right.
06:36.690 --> 06:40.020
So you should set these up when you get a chance.
06:40.050 --> 06:45.630
When you're working with an actual notebook in particular, you flip this switch on to make sure that
06:45.630 --> 06:50.820
when you execute this code in a cell, it will have access to that secret.
06:51.120 --> 06:56.100
And of course, as you can imagine, the sort of powerful thing about these secrets is that if you share
06:56.100 --> 06:59.490
this notebook with others, then they get all of your code.
06:59.490 --> 07:02.100
But of course, they don't get your secrets shared.
07:02.100 --> 07:07.380
They will have to enter in their own secrets in order to be able to run that code.
07:07.380 --> 07:12.240
And similarly, of course, when I share notebooks for you to use, the same thing will apply.
07:12.240 --> 07:18.180
You'll need to put in your own tokens in order to make take advantage of the code and run it against
07:18.180 --> 07:23.850
the frontier models or use your hugging face hub, um, or whatever.
07:24.600 --> 07:26.700
Okay, let's close that down.
07:26.700 --> 07:30.930
So let me just show you some of the more powerful boxes.
07:30.930 --> 07:34.500
So you remember we can go here and go change runtime type.
07:34.500 --> 07:38.040
Click on T4 to to use that box.
07:38.040 --> 07:40.080
And I did that earlier.
07:40.230 --> 07:45.150
And I did that because uh, it can take a little while to connect to some of these boxes.
07:45.150 --> 07:50.700
And with the really high spec boxes like A100, sometimes it just won't be available and you'll have
07:50.700 --> 07:54.180
to come back and try again two minutes later, and then it will be available.
07:54.180 --> 07:58.710
Invariably it becomes available after a couple of tries, but sometimes they are oversubscribed and
07:58.710 --> 08:00.660
it takes a few attempts.
08:00.660 --> 08:02.580
So this is a T4 box.
08:02.580 --> 08:09.210
If I do view resources, we'll see that we have again 12 and a bit of system Ram.
08:09.210 --> 08:12.780
We have the same a slightly smaller hard drive, I think.
08:12.960 --> 08:15.690
I think it was two, two five before, but it's 200 whatever.
08:15.690 --> 08:16.980
That's plenty of disk space.
08:16.980 --> 08:24.000
And we have a GPU with 15GB of Ram, and 15GB might sound like a huge amount of Ram to have for a GPU.
08:24.000 --> 08:28.240
But as you'll quickly discover when it comes to training deep neural networks, that is a kind of puny
08:28.270 --> 08:30.490
GPU, but it's good enough for our purposes.
08:30.490 --> 08:32.920
We'll be able to use this for this class.
08:33.130 --> 08:38.110
Um, uh, but but it just some things might take a long time.
08:38.260 --> 08:45.100
Uh, this is a bit of code that I just copied from the original colab that Google prompted us with,
08:45.100 --> 08:51.970
which gives us a nice little, uh, printout of details behind this GPU, including how much memory
08:52.000 --> 08:54.250
we're using out of the 15GB.
08:54.280 --> 08:57.040
Although, of course, you can always watch it happening over here.
08:58.000 --> 09:02.110
Uh, so this is the T4 box.
09:02.110 --> 09:05.410
I'm now going to show you the A100 box.
09:05.410 --> 09:11.290
This is the super powered one, and I may splash out and use this from time to time.
09:11.290 --> 09:17.440
Just in the spirit of keeping this class moving fast and showing you, uh, great results really quickly.
09:17.590 --> 09:21.700
Uh, if we view the resources, you'll see what's going on.
09:21.700 --> 09:29.380
Now, we've got a 40 gigabyte Ram, GPU and that that is a beefy GPU.
09:29.380 --> 09:34.240
That is something which will be able to use to do some hefty training.
09:34.480 --> 09:37.750
Um, and we can use this to print more details.
09:37.840 --> 09:46.930
You can see that we are using two megabytes by, uh, when we're not doing anything out of the 40GB
09:46.930 --> 09:49.270
of available memory.
09:49.870 --> 09:53.200
So that's the quick tour of what's going on with Colab.
09:53.200 --> 09:57.040
The one other thing I'll mention is the share button up here.
09:57.070 --> 10:03.880
Uh, if you press the share button, then you will see a very familiar interface, because if you use
10:03.880 --> 10:07.600
Google Drive at all, it looks just like everything else in Google Drive.
10:07.630 --> 10:13.600
You can share these notebooks and with different levels of permission with different groups, and use
10:13.600 --> 10:16.330
that as a way to collaborate really effectively.
10:16.330 --> 10:25.810
Uh, with friends, colleagues, coworkers on the, uh, the AI Jen AI projects that you're working
10:25.810 --> 10:26.110
on.
10:26.110 --> 10:29.380
And it's a super effective way to collaborate, of course.
10:29.410 --> 10:32.980
And that's one of the great benefits of using the Google Colab setup.
10:33.220 --> 10:33.910
All right.
10:33.910 --> 10:35.500
I'll see you back for the next lecture.