And you thought we'd never get here.
Here we are in Jupyter Lab, running our fine tuning for a frontier model.
So we start with a bunch of imports.
We also import the test data.
If you remember, this is that nifty piece of code that is able to run through 250 test examples and
give us a beautiful chart.
At the end of it, we load in our environment, we load in hugging face token, which we're not going
to use this time, but like before, why not always like hugging and hugging face?
And we are going to use OpenAI.
So we very much need to run that line there and here.
And again we're going to do this where we open the training data and the test data from a pickle file
so that we don't have to recreate everything from scratch in it comes.
All right.
So let's talk about what we're going to do now.
So OpenAI recommends that when you're doing training you use somewhere between 50 and 100 example points
that you use for training.
And really the the main intention of fine tuning for a frontier model is about adapting its tone and
style, and correcting for errors and improving accuracy in some circumstances.
Um, there's not a massive point in, in putting in enormous numbers of examples, because a model like
the GPT four series is trained on so much data that what you're really trying to do is just give it
enough examples of something specific you want it to do, so it can learn from that.
Um, so yeah, there's um, not not a recommendation to go to a very large number, but I'm at the very
least going to pick 500 here, which is more than they recommend.
Uh, and I've tested it and it does better than, than smaller numbers.
Um, and so I'm picking 500 of our examples.
Now our examples are very small.
Our text is very small and I think typically there they are thinking about much bigger training documents.
So because of that, I don't feel bad about this.
And at the moment this is actually fine tuning.
Um, is is free for a period of time until I think it's September the 23rd, but later in September.
But even when it stops being free, the cost you pay is similar to the cost to actually just run inference
on 500 of these, which is measured in a few cents again.
So at this point, I imagine we're talking about about $0.05 to do this, uh, for, for or equivalent
in, in your currency.
Um, so it's still small pennies.
And as I say, it's free at least until late September.
So with that, I'm dividing into a training set of 500 from the actual training set that we've got,
which is 400,000.
Uh, and I'm going to take 50 as validation.
I mentioned a moment ago, we don't actually need to do validation because our training set, we're
only going to do one epoch through it.
Um, but I thought it'd be useful to show it to you so that you know how to do this for the future in
in your projects.
Because all of this can be replicated for your projects.
So we run this.
So I mentioned to you that the first step is preparing the Jsonl JSON lines data, converting our training
data into this format.
So first of all, I wrote a method that a function that you know well messages for uh, which is taken
exactly from what we did last time.
Uh, it says you estimate prices of items, reply only with the price, no explanation.
And then for the user prompt, I take the test prompt from the item.
Um, and I strip out to the nearest dollar and just replace that with uh, and with with empty.
So it's not it's not directing it to only go to the nearest dollar, The frontier Labs need no such
approximation.
And I also take out that, um, and that's what goes in the user prompt.
And then I reply with the assistant saying price is and then giving the price.
So let's run that.
And just in case it's not clear what's going on, let's just give you an example.
Messages for train zero, which will be the first one that the model sees.
And this is what you get roll system.
And that's the system prompt.
Check you're happy with that and then roll user.
And this is the user prompt.
Uh it's as if we have asked this question how much does this cost question mark.
And then this spiel about a Delphi or Delphi, Delphi, uh, fuel pump module.
Um, and then the This is the assistant's response.
The price is $226.
I would never have I well, I remember, I didn't guess that it was anything like that.
Uh, so there you go.
You learn something every day.
Anyway, this is the format of the messages, which is something that should be very, very familiar
to you at this stage.
And you can see how this is a perfectly crafted test, sorry, training data point that we will be providing
to the model.
Okay, so then here is a function make JSON L that is going to do just what you would think it will
take in a bunch of items.
It will iterate through those items.
It will create this text, this, this, uh, object for each one.
And then it will use Json.dumps dump string to convert that into a simple string.
And then look, it just simply adds that to this one string with a carriage return at the end of it.
And then I return that back and I strip out that last carriage return.
So let's see this in action.
So let's say let's run it first.
Don't make my usual blunder.
There we go.
And now say make JSON.
L uh, and let's pass in uh, some of our training data.
Let's just pass in the first three of that so we don't crowd everything out.
So here we get back a string, of course.
And it's a string which has, uh.
It might be easier if we print it.
Let's print it so that we get empty lines clearly showing through.
Okay, so it's a string and you can see one, two, three lines in the string.
Um, it's sort of wrapping around.
And you can see that each row has in it, um, the, the full messages exchanged for the represents
that training data point.
Okay.
So far so good.
Now we have this function just building on that.
Right?
Jsonl take items and takes a file name.
And this is super simple stuff.
Opens that file name and calls the function above and writes it out.
So I don't think I need to to give you a demo of that one.
I do need to execute it, but we can actually run it.
So we're going to take our training data set, which you remember the fine tuned train which is 500
items.
Let's check.
There it is, 500 items from the training data set overall, which is 400,000.
We're not going to we're not going to write all of them to a file and upload them to GPT four.
Uh, so, um, we we write that out to a file called Fine Tune train dot JSON.
L let's run that and then we'll take the validation set and do exactly the same run that.
So we've written those two files and you can see they just wrote a couple of seconds ago.
So if I open this up, I can open it.
There is actually a fancy JSON lines editor in, uh, in JupyterLab, but we're just going to go over
a normal editor.
And here you can see, just as you'd expect, we're expecting 500 rows.
Here we go, all the way down to the end, 500 rows it is.
And they all have exactly the structure that you would hope.
Um, and you can see this actually isn't well-formed JSON because, uh, each line is a well-formed
JSON document.
I know I'm belaboring that point, but it is it is important you wouldn't be able to read this in and
parse it as a JSON document, because it's a not well-formed JSON.
It's separate lines.
Um, and the validation file open with editor is just, I think 50.
We said 50 lines much the same way.
Uh, I can I'll just show you what it looks like in the JSON lines.
Editor.
It's a fancy editor that looks like this.
And you can open up each one and it's like a JSON object.
Uh.
Look at that.
That's how I should have started.
Probably.
It gives you a very good sense of what's going on.
Uh, it's a very intuitive sense of the way the reason why messages are packaged.
The way they're packaged.
All right, so that are those are the files.
Uh, that's the last step of this.
This part.
Um, it will be time for us to upload these files to OpenAI.
And to do that, we call OpenAI dot files dot create.
And we pass in the file and we tell it the purpose is fine tune.
Uh, and just one tiny thing to watch out for.
When you pass in this file, you have to pass it in.
You have to open it as a binary file, because it's just going to be the binary bytes in that file that
will get streamed up to OpenAI.
So you don't want this to be an R, you want it to be an RB.
Uh, so just just a small nuance to watch out for.
We're just sending the entire contents of the file as is to OpenAI.
So we execute that line takes a second.
If I just inspect what came back, I get back a file object.
It's got a certain number of bytes.
Object is file, purpose is fine tuned, status is processed.
So already OpenAI is taking that file and has processed it.
And we will do the same thing for the validation.
We'll run it and there we go.
And once again it is processed.
So at this point we have created two JSON files.
One for our fine tuned training set, one for our fine tuned validation set.
We've written them out to our file system.
And then we have uploaded them to OpenAI, where they are now sitting as file objects in OpenAI.
In the next session, we will actually do some fine tuning.
See you there.