Hey, gang.
Look, I know what you're thinking.
This week was supposed to be training week.
I set it all up to be all about fine tuning frontier models.
And what haven't we done?
We haven't fine tuned frontier models.
But I have good news.
Today is the day.
But I should prepare you that today may also disappoint in some ways as well as we will.
We will find out.
Um, but be prepared for that as we embark upon a whole brave new world, we're getting finally to training.
And as a quick recap, you can already generate text and code with frontier models, with APIs and using
open source models hugging face through both the pipelines and the the lower level transformers APIs.
Like using the models directly, you can create advanced Rag pipelines using Lange chain and not using
Lange chain.
And most importantly, you can now follow a five step strategy for problem solving that includes a lot
of time we seem to spend curating data.
It turns out that lots of time is spent curating data and then making a baseline model.
We did some training, but it was training of a baseline model.
Traditional ML or we've not yet done is trained, fine tuned, a frontier model.
And that's what we're going to do today.
Going to understand the process for fine tuning in front of your model, create a data set for it,
run fine tuning, and then test our new fine tuned model.
And just as a point of order to explain when you're talking about training, when in the context of
these kinds of models, fine tuning is synonymous with with training, we never, of course, train
one of these things from scratch, because that would cost north of hundreds of millions of dollars.
So we're always taking an existing model that's been trained, a pre-trained model, and we are doing
more training, taking advantage of transfer learning, which is this theory that says that you can
just take an existing pre-trained model, do a bit more training, and it will be better at the new
task you're training it for.
And that's also known as fine tuning.
So with that vocabulary out of the way, let's just talk about the three steps to fine tuning with OpenAI.
There's three things we need to follow in order to take GPT four zero or GPT four mini that we will
take and fine tune it.
The first step is you need to prepare training data that it will use for training.
We obviously use training data in the context of the traditional models, linear regression and so on.
We got some training examples and we pumped it through a linear regression model.
So we have to create training data.
Now um, and then we have to upload it to OpenAI.
And it expects that training data in a particular format called JSON L which stands for JSON lines,
which is subtly different as I will show you to normal JSON.
We are then going to run our training, our fine tuning and these charts all pointing downwards.
Might might trouble you.
It looks like things are going wrong, but au contraire, when it comes to training, your one is watching.
Training loss.
And of course you want loss to go down.
That means that things are getting better.
And so we will be watching our charts like a hawk and trying to make sure that our losses are coming
down.
Uh, and most importantly, you look at training loss during the course of a batch, and you also look
at validation loss, which is on a held out data set.
Uh, is that coming down, too?
Because you may be overfitting to your training data if you just watch training loss.
And that actually isn't a problem in our case, because we're only going to be running one epoch through
our training data.
And epoch is what you call it.
When you go, you take take a complete training run all the way through your training data, and then
you repeat and do it all a second time with the same data.
That would be called a second epoch of training.
And we are not going to do that because we have so much training data that we don't need to do that.
We might as well just use a bit more training data and do one epoch.
And since all of the data will always be new data, the training loss is just as useful for us as validation
loss.
And then finally you evaluate your results.
And then based on what you see, you tweak and you repeat and keep going.
So those are the stages.
And as I say, the first of them is to prepare the data.
So OpenAI expects it in this format called JSON L, which means that it is a series of lines of JSON
data.
And you may think, isn't that just the same as JSON data?
It's not.
It's not in a in a collection.
So it's not in a list.
It doesn't start with a square bracket with with commas.
It's just each row, each line in this file is a separate JSON object starting and ending with curly
braces.
It's a subtle distinction, but it can catch you out if you're not expecting that you're not writing
a JSON object, because that would have a list around it.
You're writing rows of JSON to this file, and then each row is going to be something that is mostly
very familiar to us.
It will have one attribute called messages.
And what goes in there is the thing that we know so well, the list of dictionaries where each dictionary
has a role and a content.
It's a conversation.
So that is what is going in each row.
As you will see, we will craft this particular type of data set for uploading.
All right.
With that enough enough chit chat.
Let's go to Jupyter Lab.
Let's actually run this thing.
And for the first time we will train a frontier model.
Let's do it.