It's time for action, everybody.
We've set up our colab.
Here we are, week seven, day three.
We've got our constant setup.
We've checked our model name and it's time to get into things.
Um, so the first thing we're going to do is log in to hugging Face, which you know, well.
And then after we've logged into hugging face here, we're going to do something that you don't know
so well, which is new, which is logging into the wonderful weights and biases platform.
So you may have already set this up as part of last week's, uh, foray into weights and biases, but
if not, it's very simple to do.
And of course it is free.
You go to the, uh, weights and biases W and B dot I right linked here.
Uh, you set up a free account and then you can find your, your wand API key, your weights and biases
API key.
And then you have to go to this padlock symbol here and add it in as a secret to your Google Colab.
Um, or if you don't want to do that, you can also just type it directly into the notebook right here
instead as well.
But it's recommended to use the secrets.
So once you've set up weights and biases on we go.
We then load in the data set, and we can just quickly check that the data set is what we expect.
We expect there to be and we expect.
Let's have a look at how many training data points we're looking for 400,000 of them there they are
400,000 training data points a lot of them.
Let's look at the first one.
Just make sure it's exactly what we expect it should be.
The text in this training data point is the prompt we're expecting followed by the price right here.
And it's going to be this that the model is going to be desperate to predict this next token and get
it right.
And we also have the actual price here.
You can see we're rounding it to the nearest whole token.
And we tell it that we're looking.
How much does this cost to the nearest dollar, the nearest token the nearest dollar.
And there is the price.
The real price.
If we look at the test data set and not that we'll be using that today.
But you will remember the test data set is very similar.
Just one tiny difference.
There is not a price after here.
It just says price is dollar.
And that's that is what we're training our model to be good at doing.
Okay.
So um, if we're logged in, if we're asking to log to weights and biases, it then sets up the weights
and Biases project.
This is in fact not required to have this line of code, because actually hugging face will do this
for you if you start to run and you don't, you haven't run this line.
But I sometimes add this because you can also, if you wish, pass in a config config a config with
some parameters, and that lets you set up some extra um, attributes that will appear in weights and
biases.
That's a useful trick to know.
Um, but but anyways, you can run this or not.
It doesn't make much difference.
Okay, so then this we saw before.
We're just going to take this hyper parameter and decide whether we're going to use the bits and bytes
config for four bit quantization or for eight bit.
And we are of course going with four bits.
We're going all the way down.
And then it is time to load in the tokenizer and the model.
And so usual code to load the tokenizer.
I mentioned to you that this is boilerplate stuff that you do that tells the trainer that we want to
pad every data point so that it fills up our maximum sequence length, and it should pad it all up with
end of sentence tokens just to fill up that that set of characters, the 182, uh, length of our of
our tokens in our, in our sequences.
So we'll pad them to the right with these tokens.
And then there's this one line here, which is just another boilerplate thing to set the pad token ID
on this thing called the generation config.
And the reason that put this in there is it stops it saying a warning later that it that it doesn't
see this, but I actually think it would take this anyway.
So so this avoids it printing an unnecessary warning.
Um, and then we end by printing the memory footprint as usual, and we see that it is the 5.6GB that
we expect for our four bit version of the llama, 3.18 billion parameter model.
Okay, one more technicality here.
This thing called the data collator.
So there is this, um, this cunning thing that when we're doing our training, we're going to be passing
in the the full sequence, including the price, and the model will see lots of examples of this and
get better and better at predicting these tokens.
But there's a bit of a catch.
We actually don't care for the model to learn about how to predict all of the tokens in the prompt up
until the dollar sign.
We want it to learn how to predict that token right there.
So we don't want it to spend lots of time seeing how good it is at writing descriptions of products,
and then also learn the price we want it to focus on that price.
Um, and, and doing that involves setting up something called a mask.
When you tell the trainer that you don't need it to learn about the prompt, you just want it to take
this into account, to give it context.
But learn how to predict this token right here after the dollar sign.
Setting up masks is a bit fiddly and involves a bit of uh um uh, yeah, yeah, a bit of, uh, messing
around with with tokens.
But luckily hugging face has made it very easy indeed with a nice little utility that does it all for
you.
And it's called the data Collator for completion only.
LM, uh, which is what we've got.
We've got something where all we want to do is have it complete.
This particular sentence.
How it works is you come up with something that's called the response template, which sounds very fancy,
but it just simply means what is the chunk of text which is going to indicate that I want you to predict
whatever comes next.
And in our case, it is quite simply price is dollar sign.
We want it to predict the next thing after that.
So you put that into something called response templates and then you create something, an instance
of data collator for completion, only passing in the response templates.
And you also have to tell it your tokenizer, and it will use that to build the final set of masked
data that goes into the the trainer.
And you'll see that in just a second.
So this is super simple and it's doing something quite sophisticated behind the scenes and making it
easy for us.
All right.
And finally the last the last big moment for us.
This is going to look like lots of code, but it's not actually that much to it.
We are about to call our trainer, and we're going to have to pass in two sets of parameters.
One of them is called the Laura config.
It's the hyperparameters that relate to how we're doing our Laura training.
And the other, the psft config is the supervised fine tuning.
Config is just general stuff about how training should work, and what I've done is I've basically set
these things up to pull from the constants that we defined right at the top of the notebook, like alpha
and dropout, and then the things that don't matter so much, I've just put put in here what to use.
And these are not important hyperparameters or they're or they're just things where we should set it
to a to a certain value.
So you can see that for the Laura parameters, we of course pass in the alpha, which we know well,
I won't define it yet again, the dropout and the target modules that we're targeting.
So so that goes into the Laura config, the training parameters, the SFT config had stuff like the
run name.
How many epochs are we running the batch size?
Uh, super important 16.
In my case, it might be one for you if you're on a on a the T4 box.
Uh, the gradient accumulation, that is one for us.
And then things like learning rate the cosine scheduler that we decided to use.
Um, and then just down at the bottom here, we're saying that we want this to be pushing to the hub.
Um, every time that, that it's doing a save, which is every 5000 of the, of the batch steps, we
want it to push this model to the hub so that we'll get a series of saves to the hub.
And we can see our model at each of those points.
Um, and we ask to give it this name, and we say we want it to be a private, uh, repo so that it's
not public yet until we're satisfied that we've got great results from it.
Uh, and then this is the final line.
We set up something called an SFT trainer.
We pass in just a few simple things.
Number one, it needs to know the base model.
What is the underlying model that we want to fine tune, which is llama 3.1.
Number two what is our training data?
We pass that in.
What are the Lora parameters.
This has as you know it has the alpha, the r, the dropout, the tokenizer, the training parameters
that we just set up right here.
And finally we pass in that collator, which is that sneaky object which knows to look for the prices
dollar sign and it's telling it.
Don't bother predicting what comes before that.
Worry about what comes next.
Um, and so this then will have set up our trainer.
And in the next video we'll run this single line fine tuning train, which is the line that kicks it
off.
Uh, that will be quite the moment.
So without further ado, let's go do that.