You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 

454 lines
13 KiB

WEBVTT
00:00.620 --> 00:02.630
It's time for action, everybody.
00:02.660 --> 00:04.670
We've set up our colab.
00:04.670 --> 00:06.380
Here we are, week seven, day three.
00:06.410 --> 00:08.240
We've got our constant setup.
00:08.540 --> 00:12.920
We've checked our model name and it's time to get into things.
00:13.250 --> 00:19.940
Um, so the first thing we're going to do is log in to hugging Face, which you know, well.
00:19.940 --> 00:23.810
And then after we've logged into hugging face here, we're going to do something that you don't know
00:23.810 --> 00:29.480
so well, which is new, which is logging into the wonderful weights and biases platform.
00:29.690 --> 00:34.940
So you may have already set this up as part of last week's, uh, foray into weights and biases, but
00:34.940 --> 00:36.830
if not, it's very simple to do.
00:36.830 --> 00:38.150
And of course it is free.
00:38.150 --> 00:43.550
You go to the, uh, weights and biases W and B dot I right linked here.
00:43.550 --> 00:50.630
Uh, you set up a free account and then you can find your, your wand API key, your weights and biases
00:50.660 --> 00:51.530
API key.
00:51.530 --> 00:58.160
And then you have to go to this padlock symbol here and add it in as a secret to your Google Colab.
00:58.220 --> 01:03.090
Um, or if you don't want to do that, you can also just type it directly into the notebook right here
01:03.090 --> 01:03.840
instead as well.
01:03.840 --> 01:05.850
But it's recommended to use the secrets.
01:05.850 --> 01:09.990
So once you've set up weights and biases on we go.
01:10.020 --> 01:15.420
We then load in the data set, and we can just quickly check that the data set is what we expect.
01:15.450 --> 01:18.750
We expect there to be and we expect.
01:18.780 --> 01:24.870
Let's have a look at how many training data points we're looking for 400,000 of them there they are
01:24.870 --> 01:27.840
400,000 training data points a lot of them.
01:27.840 --> 01:29.040
Let's look at the first one.
01:29.040 --> 01:32.760
Just make sure it's exactly what we expect it should be.
01:32.760 --> 01:39.570
The text in this training data point is the prompt we're expecting followed by the price right here.
01:39.570 --> 01:44.340
And it's going to be this that the model is going to be desperate to predict this next token and get
01:44.340 --> 01:45.120
it right.
01:45.510 --> 01:47.460
And we also have the actual price here.
01:47.460 --> 01:49.950
You can see we're rounding it to the nearest whole token.
01:49.950 --> 01:51.660
And we tell it that we're looking.
01:51.660 --> 01:56.550
How much does this cost to the nearest dollar, the nearest token the nearest dollar.
01:57.100 --> 01:58.360
And there is the price.
01:58.390 --> 01:59.440
The real price.
01:59.440 --> 02:03.940
If we look at the test data set and not that we'll be using that today.
02:03.940 --> 02:06.400
But you will remember the test data set is very similar.
02:06.430 --> 02:08.020
Just one tiny difference.
02:08.020 --> 02:10.480
There is not a price after here.
02:10.480 --> 02:12.070
It just says price is dollar.
02:12.280 --> 02:16.360
And that's that is what we're training our model to be good at doing.
02:16.750 --> 02:17.530
Okay.
02:17.560 --> 02:23.890
So um, if we're logged in, if we're asking to log to weights and biases, it then sets up the weights
02:23.890 --> 02:24.880
and Biases project.
02:24.880 --> 02:30.190
This is in fact not required to have this line of code, because actually hugging face will do this
02:30.190 --> 02:33.160
for you if you start to run and you don't, you haven't run this line.
02:33.160 --> 02:39.700
But I sometimes add this because you can also, if you wish, pass in a config config a config with
02:39.700 --> 02:45.070
some parameters, and that lets you set up some extra um, attributes that will appear in weights and
02:45.070 --> 02:45.580
biases.
02:45.580 --> 02:47.470
That's a useful trick to know.
02:47.560 --> 02:50.290
Um, but but anyways, you can run this or not.
02:50.320 --> 02:51.940
It doesn't make much difference.
02:52.570 --> 02:55.310
Okay, so then this we saw before.
02:55.310 --> 03:01.310
We're just going to take this hyper parameter and decide whether we're going to use the bits and bytes
03:01.340 --> 03:04.280
config for four bit quantization or for eight bit.
03:04.280 --> 03:06.320
And we are of course going with four bits.
03:06.350 --> 03:08.390
We're going all the way down.
03:08.390 --> 03:13.100
And then it is time to load in the tokenizer and the model.
03:13.160 --> 03:16.220
And so usual code to load the tokenizer.
03:16.220 --> 03:22.670
I mentioned to you that this is boilerplate stuff that you do that tells the trainer that we want to
03:22.700 --> 03:30.950
pad every data point so that it fills up our maximum sequence length, and it should pad it all up with
03:30.950 --> 03:39.110
end of sentence tokens just to fill up that that set of characters, the 182, uh, length of our of
03:39.110 --> 03:41.630
our tokens in our, in our sequences.
03:41.630 --> 03:44.570
So we'll pad them to the right with these tokens.
03:44.960 --> 03:52.490
And then there's this one line here, which is just another boilerplate thing to set the pad token ID
03:52.520 --> 03:54.690
on this thing called the generation config.
03:54.690 --> 04:00.930
And the reason that put this in there is it stops it saying a warning later that it that it doesn't
04:00.930 --> 04:05.490
see this, but I actually think it would take this anyway.
04:05.490 --> 04:09.420
So so this avoids it printing an unnecessary warning.
04:10.290 --> 04:17.730
Um, and then we end by printing the memory footprint as usual, and we see that it is the 5.6GB that
04:17.730 --> 04:24.840
we expect for our four bit version of the llama, 3.18 billion parameter model.
04:25.560 --> 04:27.300
Okay, one more technicality here.
04:27.300 --> 04:29.610
This thing called the data collator.
04:29.670 --> 04:37.020
So there is this, um, this cunning thing that when we're doing our training, we're going to be passing
04:37.020 --> 04:43.380
in the the full sequence, including the price, and the model will see lots of examples of this and
04:43.380 --> 04:46.230
get better and better at predicting these tokens.
04:46.230 --> 04:47.940
But there's a bit of a catch.
04:47.940 --> 04:55.260
We actually don't care for the model to learn about how to predict all of the tokens in the prompt up
04:55.290 --> 04:57.000
until the dollar sign.
04:57.000 --> 05:00.060
We want it to learn how to predict that token right there.
05:00.060 --> 05:05.790
So we don't want it to spend lots of time seeing how good it is at writing descriptions of products,
05:05.790 --> 05:10.200
and then also learn the price we want it to focus on that price.
05:10.380 --> 05:15.120
Um, and, and doing that involves setting up something called a mask.
05:15.270 --> 05:21.300
When you tell the trainer that you don't need it to learn about the prompt, you just want it to take
05:21.300 --> 05:23.310
this into account, to give it context.
05:23.310 --> 05:27.960
But learn how to predict this token right here after the dollar sign.
05:27.960 --> 05:34.860
Setting up masks is a bit fiddly and involves a bit of uh um uh, yeah, yeah, a bit of, uh, messing
05:34.860 --> 05:36.390
around with with tokens.
05:36.390 --> 05:42.000
But luckily hugging face has made it very easy indeed with a nice little utility that does it all for
05:42.000 --> 05:42.330
you.
05:42.330 --> 05:46.500
And it's called the data Collator for completion only.
05:46.830 --> 05:49.420
LM, uh, which is what we've got.
05:49.450 --> 05:51.970
We've got something where all we want to do is have it complete.
05:51.970 --> 05:53.380
This particular sentence.
05:53.380 --> 05:59.800
How it works is you come up with something that's called the response template, which sounds very fancy,
05:59.800 --> 06:05.440
but it just simply means what is the chunk of text which is going to indicate that I want you to predict
06:05.440 --> 06:06.700
whatever comes next.
06:06.700 --> 06:10.480
And in our case, it is quite simply price is dollar sign.
06:10.480 --> 06:13.120
We want it to predict the next thing after that.
06:13.120 --> 06:18.490
So you put that into something called response templates and then you create something, an instance
06:18.490 --> 06:23.800
of data collator for completion, only passing in the response templates.
06:23.800 --> 06:30.550
And you also have to tell it your tokenizer, and it will use that to build the final set of masked
06:30.550 --> 06:33.730
data that goes into the the trainer.
06:33.730 --> 06:35.230
And you'll see that in just a second.
06:35.230 --> 06:40.090
So this is super simple and it's doing something quite sophisticated behind the scenes and making it
06:40.090 --> 06:41.320
easy for us.
06:42.010 --> 06:42.910
All right.
06:43.270 --> 06:46.750
And finally the last the last big moment for us.
06:46.750 --> 06:50.090
This is going to look like lots of code, but it's not actually that much to it.
06:50.270 --> 06:56.090
We are about to call our trainer, and we're going to have to pass in two sets of parameters.
06:56.090 --> 06:58.460
One of them is called the Laura config.
06:58.460 --> 07:02.960
It's the hyperparameters that relate to how we're doing our Laura training.
07:02.960 --> 07:08.390
And the other, the psft config is the supervised fine tuning.
07:08.390 --> 07:15.140
Config is just general stuff about how training should work, and what I've done is I've basically set
07:15.140 --> 07:20.060
these things up to pull from the constants that we defined right at the top of the notebook, like alpha
07:20.060 --> 07:24.560
and dropout, and then the things that don't matter so much, I've just put put in here what to use.
07:24.560 --> 07:28.610
And these are not important hyperparameters or they're or they're just things where we should set it
07:28.610 --> 07:30.110
to a to a certain value.
07:30.140 --> 07:34.910
So you can see that for the Laura parameters, we of course pass in the alpha, which we know well,
07:34.940 --> 07:40.610
I won't define it yet again, the dropout and the target modules that we're targeting.
07:40.610 --> 07:46.350
So so that goes into the Laura config, the training parameters, the SFT config had stuff like the
07:46.350 --> 07:46.950
run name.
07:46.950 --> 07:50.160
How many epochs are we running the batch size?
07:50.310 --> 07:52.170
Uh, super important 16.
07:52.200 --> 07:56.610
In my case, it might be one for you if you're on a on a the T4 box.
07:56.850 --> 08:00.660
Uh, the gradient accumulation, that is one for us.
08:00.750 --> 08:06.360
And then things like learning rate the cosine scheduler that we decided to use.
08:06.510 --> 08:12.960
Um, and then just down at the bottom here, we're saying that we want this to be pushing to the hub.
08:13.050 --> 08:19.050
Um, every time that, that it's doing a save, which is every 5000 of the, of the batch steps, we
08:19.050 --> 08:23.520
want it to push this model to the hub so that we'll get a series of saves to the hub.
08:23.520 --> 08:26.220
And we can see our model at each of those points.
08:26.520 --> 08:32.700
Um, and we ask to give it this name, and we say we want it to be a private, uh, repo so that it's
08:32.700 --> 08:36.570
not public yet until we're satisfied that we've got great results from it.
08:37.710 --> 08:40.740
Uh, and then this is the final line.
08:40.740 --> 08:43.990
We set up something called an SFT trainer.
08:43.990 --> 08:47.230
We pass in just a few simple things.
08:47.260 --> 08:49.630
Number one, it needs to know the base model.
08:49.630 --> 08:54.940
What is the underlying model that we want to fine tune, which is llama 3.1.
08:55.330 --> 08:57.490
Number two what is our training data?
08:57.520 --> 08:58.510
We pass that in.
08:58.510 --> 09:00.430
What are the Lora parameters.
09:00.430 --> 09:07.750
This has as you know it has the alpha, the r, the dropout, the tokenizer, the training parameters
09:07.750 --> 09:09.430
that we just set up right here.
09:09.430 --> 09:16.120
And finally we pass in that collator, which is that sneaky object which knows to look for the prices
09:16.120 --> 09:17.770
dollar sign and it's telling it.
09:17.800 --> 09:20.110
Don't bother predicting what comes before that.
09:20.440 --> 09:22.570
Worry about what comes next.
09:22.990 --> 09:27.400
Um, and so this then will have set up our trainer.
09:27.400 --> 09:35.590
And in the next video we'll run this single line fine tuning train, which is the line that kicks it
09:35.590 --> 09:36.190
off.
09:36.310 --> 09:37.990
Uh, that will be quite the moment.
09:37.990 --> 09:42.070
So without further ado, let's go do that.