Continuing our adventure through hyperparameters for training.
The next one is pretty crucial and it is called learning Rate.
And again, many data scientists amongst you will know this one only too well.
So for very, very quickly, for those that are less familiar with this, again the purpose of training
is that you take your model, you take a training data point.
You do what they call a forward pass, which is an inference where you go through the model and say,
predict the next token that should come, and it gives a prediction the the, the predicted next token.
Or in fact, it gives a probabilities of all of the possible next tokens.
And you use that and you have the actual next token that it should have actually been.
And you can take these two, the prediction and the actual to come up with a loss.
How poorly did it do at predicting the actual and what you can then do is take that loss and you can
do what they call back propagation when you go back through the model and figure out how sensitive,
how much would I have to tweak each weight up or down in order to do a little bit better next time?
Uh, and then you have to take a step in the direction.
You have to shift your weights, a step in the direction to do better next time.
And that step, that amount that you shift your weights in a good direction so that it will do a little
bit better next time.
When faced with exactly that training data point, uh, is called the learning rate.
And it's typically it's it's either 0.0001 or 0.00001.
Uh, you will see some examples when we go through it.
And there's also an ability to do have what's called a learning rate scheduler, which is when you start
the learning rate at one number and during the course of your run over the period of several epochs,
you gradually lower it and lower it and lower it, because as your model gets more trained, you want
your learning rate, the amount of step that you take to get shorter and shorter and shorter until you're
only making tiny Any adjustments to your network.
Because you're pretty confident that you're in the right vicinity.
So that is learning rates.
Again, it will be old hat to many people who have a data science background.
It might be new to others.
Gradient accumulation is a way.
It's it's a technique that allows you to improve speed of going through training where you say, okay,
so what we're going to do is we're going to we normally do a forward pass.
We get the, the, the, the, the loss as I just described it.
We then work out the gradients going backwards and then we take a step in the right direction.
And then we repeat gradient accumulation says well perhaps what we can do is we can do a forward pass
and get the gradients and don't take a step, just do a second forward pass and get the gradients and
add up those gradients and do that a few more times.
Just keep accumulating these gradients and then take a step and then optimize the network.
And that just means that you do these steps less frequently, which means it can run a bit faster.
Um, in some ways it's a bit similar to batch size.
It has some some there's some sort of a conceptual similarities there, because you're sort of grouping
things together and just taking one slightly bigger step.
Um, in the hyperparameters that I've set up, I'm not using gradient accumulation.
I've got that set to one.
But I did try it in the past, and I see how it speeds things up.
And so you might well be interested in experimenting with that and see what it does.
So that is gradient accumulation.
And then last but not least the optimizer.
The optimizer is the formula that's used when it's time, when you've got the gradients you've got your
learning rate.
And it's time to now make an update to your neural network to shift everything a little bit in a good
direction, so that next time it's that little bit more likely to predict the right next token.
And the process for doing that is called the optimizer.
And there are a bunch of well-known formulae for how you could do that, each with pros and cons, you'll
see we pick one in particular.
That's one that is a little bit more expensive in terms of the performance.
It's a it's a bit harder work, but it leads to good outcomes.
So it's the one that I would recommend starting with.
And then if you do end up having any kind of memory problems, there are alternatives that are, um,
that consume less memory.
But that that process is called optimization.
And the algorithm that you pick to do it is called the optimizer.
And it's another hyperparameter, and that you can try different ones and see how they do.
So I realize it's an awful lot of talking.
And I've also used the the conversation about hyperparameters to explain a bit about the training process.
Uh, but hopefully this was good foundational background that's prepared you for what is just about
to happen now, which is we're going back to Google Colab, where we are going to set up and kick off
our SFT trainer to fine tune our own, uh, specialized LM.