You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 

280 lines
7.6 KiB

WEBVTT
00:01.340 --> 00:02.180
Hey, gang.
00:02.210 --> 00:04.250
Look, I know what you're thinking.
00:04.250 --> 00:07.130
This week was supposed to be training week.
00:07.160 --> 00:11.900
I set it all up to be all about fine tuning frontier models.
00:11.900 --> 00:13.250
And what haven't we done?
00:13.250 --> 00:15.110
We haven't fine tuned frontier models.
00:15.110 --> 00:16.490
But I have good news.
00:16.490 --> 00:18.140
Today is the day.
00:18.170 --> 00:19.460
Today is the day.
00:19.460 --> 00:25.550
But I should prepare you that today may also disappoint in some ways as well as we will.
00:25.580 --> 00:26.720
We will find out.
00:26.900 --> 00:35.720
Um, but be prepared for that as we embark upon a whole brave new world, we're getting finally to training.
00:35.720 --> 00:42.710
And as a quick recap, you can already generate text and code with frontier models, with APIs and using
00:42.710 --> 00:51.140
open source models hugging face through both the pipelines and the the lower level transformers APIs.
00:51.140 --> 00:58.640
Like using the models directly, you can create advanced Rag pipelines using Lange chain and not using
00:58.640 --> 00:59.390
Lange chain.
00:59.390 --> 01:06.620
And most importantly, you can now follow a five step strategy for problem solving that includes a lot
01:06.650 --> 01:09.110
of time we seem to spend curating data.
01:09.140 --> 01:13.910
It turns out that lots of time is spent curating data and then making a baseline model.
01:13.910 --> 01:16.760
We did some training, but it was training of a baseline model.
01:16.790 --> 01:22.970
Traditional ML or we've not yet done is trained, fine tuned, a frontier model.
01:22.970 --> 01:24.500
And that's what we're going to do today.
01:24.530 --> 01:30.170
Going to understand the process for fine tuning in front of your model, create a data set for it,
01:30.200 --> 01:35.360
run fine tuning, and then test our new fine tuned model.
01:35.930 --> 01:42.290
And just as a point of order to explain when you're talking about training, when in the context of
01:42.290 --> 01:48.290
these kinds of models, fine tuning is synonymous with with training, we never, of course, train
01:48.290 --> 01:54.020
one of these things from scratch, because that would cost north of hundreds of millions of dollars.
01:54.020 --> 02:00.680
So we're always taking an existing model that's been trained, a pre-trained model, and we are doing
02:00.680 --> 02:05.670
more training, taking advantage of transfer learning, which is this theory that says that you can
02:05.670 --> 02:10.410
just take an existing pre-trained model, do a bit more training, and it will be better at the new
02:10.410 --> 02:12.030
task you're training it for.
02:12.120 --> 02:14.760
And that's also known as fine tuning.
02:15.450 --> 02:24.030
So with that vocabulary out of the way, let's just talk about the three steps to fine tuning with OpenAI.
02:24.180 --> 02:30.930
There's three things we need to follow in order to take GPT four zero or GPT four mini that we will
02:30.930 --> 02:33.390
take and fine tune it.
02:33.420 --> 02:39.270
The first step is you need to prepare training data that it will use for training.
02:39.270 --> 02:44.820
We obviously use training data in the context of the traditional models, linear regression and so on.
02:44.820 --> 02:49.170
We got some training examples and we pumped it through a linear regression model.
02:49.170 --> 02:50.790
So we have to create training data.
02:50.820 --> 02:54.720
Now um, and then we have to upload it to OpenAI.
02:54.720 --> 03:02.520
And it expects that training data in a particular format called JSON L which stands for JSON lines,
03:02.670 --> 03:07.160
which is subtly different as I will show you to normal JSON.
03:07.640 --> 03:13.880
We are then going to run our training, our fine tuning and these charts all pointing downwards.
03:13.880 --> 03:14.900
Might might trouble you.
03:14.930 --> 03:20.270
It looks like things are going wrong, but au contraire, when it comes to training, your one is watching.
03:20.300 --> 03:21.590
Training loss.
03:21.710 --> 03:23.960
And of course you want loss to go down.
03:23.960 --> 03:25.880
That means that things are getting better.
03:25.970 --> 03:31.610
And so we will be watching our charts like a hawk and trying to make sure that our losses are coming
03:31.610 --> 03:32.480
down.
03:33.200 --> 03:39.560
Uh, and most importantly, you look at training loss during the course of a batch, and you also look
03:39.590 --> 03:44.090
at validation loss, which is on a held out data set.
03:44.090 --> 03:45.740
Uh, is that coming down, too?
03:45.770 --> 03:49.940
Because you may be overfitting to your training data if you just watch training loss.
03:49.970 --> 03:55.940
And that actually isn't a problem in our case, because we're only going to be running one epoch through
03:55.940 --> 03:56.990
our training data.
03:56.990 --> 03:58.850
And epoch is what you call it.
03:58.940 --> 04:04.340
When you go, you take take a complete training run all the way through your training data, and then
04:04.340 --> 04:07.370
you repeat and do it all a second time with the same data.
04:07.400 --> 04:10.070
That would be called a second epoch of training.
04:10.490 --> 04:16.130
And we are not going to do that because we have so much training data that we don't need to do that.
04:16.130 --> 04:19.580
We might as well just use a bit more training data and do one epoch.
04:19.580 --> 04:26.360
And since all of the data will always be new data, the training loss is just as useful for us as validation
04:26.360 --> 04:27.230
loss.
04:28.100 --> 04:31.760
And then finally you evaluate your results.
04:31.760 --> 04:38.240
And then based on what you see, you tweak and you repeat and keep going.
04:38.390 --> 04:40.310
So those are the stages.
04:41.060 --> 04:44.870
And as I say, the first of them is to prepare the data.
04:45.050 --> 04:52.970
So OpenAI expects it in this format called JSON L, which means that it is a series of lines of JSON
04:52.970 --> 04:53.750
data.
04:53.900 --> 04:56.030
And you may think, isn't that just the same as JSON data?
04:56.030 --> 04:56.870
It's not.
04:56.870 --> 04:59.000
It's not in a in a collection.
04:59.000 --> 04:59.900
So it's not in a list.
04:59.900 --> 05:02.770
It doesn't start with a square bracket with with commas.
05:02.770 --> 05:10.720
It's just each row, each line in this file is a separate JSON object starting and ending with curly
05:10.720 --> 05:11.410
braces.
05:11.410 --> 05:16.270
It's a subtle distinction, but it can catch you out if you're not expecting that you're not writing
05:16.270 --> 05:19.390
a JSON object, because that would have a list around it.
05:19.390 --> 05:26.320
You're writing rows of JSON to this file, and then each row is going to be something that is mostly
05:26.320 --> 05:27.550
very familiar to us.
05:27.580 --> 05:30.730
It will have one attribute called messages.
05:30.730 --> 05:38.890
And what goes in there is the thing that we know so well, the list of dictionaries where each dictionary
05:38.920 --> 05:40.750
has a role and a content.
05:40.750 --> 05:42.100
It's a conversation.
05:42.100 --> 05:45.460
So that is what is going in each row.
05:45.610 --> 05:51.040
As you will see, we will craft this particular type of data set for uploading.
05:52.240 --> 05:53.260
All right.
05:53.680 --> 05:57.130
With that enough enough chit chat.
05:57.160 --> 05:58.900
Let's go to Jupyter Lab.
05:58.900 --> 06:01.000
Let's actually run this thing.
06:01.000 --> 06:05.230
And for the first time we will train a frontier model.
06:05.260 --> 06:06.370
Let's do it.