Well hi there everybody.
I'm not going to give you my usual song and dance about how excited you are, because I know how excited
you are, as am I.
No doubt overnight your run has been running as mine has, and you are eager to see the results.
But we have some content to do.
First of all, um, so already you can do so many things, uh, coding against frontier and open source
models using data set curation, baseline and fine tuning, frontier models, and now running.
Q Laura, uh, so today I had previously said they were going to be two bullets, but I have put in
an extra bullet here, beginning here with how training works.
It occurs to me that I've been quite hand-wavy about the training process itself.
And at this point, now that you've got some hands on experience running, training, seeing Laura in
action, it's worth me just taking a minute to explain it properly.
So you've got that basis.
You may know it all already.
It might be something that at this point you've either picked up or you already had had encountered.
Um, either way, I think it's super important that I do clarify that.
And so we'll take a couple of minutes to do it.
Um, and I think it's really nice that you've had some experience first running training so that this
will hopefully connect the dots and things will click in place.
We will then run inference for a fine tuned model, and then we will have the conclusion of week seven,
which is exciting indeed.
All right let's get started.
So I want to explain that the training process, the process of improving a model so that it's better
and better at performing a task, is something that has four steps to it.
The first step is what's known as the forward pass, which is just another name for a sort of running
inference.
You have a data, a point in your data set, you have a particular training data point, and you take
that that training prompt and you pass it through your neural network to get the prediction for the
next token.
And that is called the forward pass, because you're thinking of the input coming in, going through,
and the output popping out at the end of your transformer.
There is then what's called the loss calculation.
And we'll talk a little bit more about this later.
But this is saying okay.
So the network predicted that this would be the output.
And in fact this is the true next token because we're in training.
And so we've got real examples that include what actually did come next in the data.
And so now that you've got the prediction and the truth, you can come come up with some way of calculating
the loss or how wrong were you.
How bad was it loss being the sort of inverse of of of accuracy.
So a bigger loss number means things went worse.
So that's step two.
The loss calculation.
Step three is known as the backward pass.
And you hear different terms for this.
It's called backprop back propagation.
Um and the idea is that in in this backward pass you take this loss and you look back through the neural
network and you ask the question, if I were to tweak each of the parameters in this neural network
by a little tiny bit.
Would it have made this loss bigger or smaller?
How is the loss dependent?
How is this particular weight?
Uh, how does that vary the loss?
Um, what's the difference in loss based on on this weight?
Um, and that that sensitivity is called a gradient.
Of course.
Um, as, as, as it is generally in maths.
Uh, and so this is about calculating the gradients of all of your weights to see how the loss is affected
by a small tweak to those weights, so that, that calculating the gradients of all of your weights
is known as the backward pass.
Uh, and then finally, the fourth step optimization is, uh, and this is the thing that we selected
the Adam with weight decay, the Adam W optimizer for our particular training exercise.
Optimization is saying, okay, so now we've calculated all of our gradients.
What we now want to do is we want to tweak all of the weights a tiny tiny bit such that next time,
if it were given the same prompt, it would be more likely to do a little bit better.
The loss would be a little bit lower.
So we're going to tweak in the in the direction the opposite direction to the to the gradient so that
losses would be reduced.
And that small step which is based on the learning rate, how much of a step you take is based on on
how big the learning rate is, is designed to make things a little bit better.
And you always want to try and do it in a way that will generalize well.
You don't want to just be solving for exactly this input prompt.
You just want the model to be learning to get a little bit better with those kinds of prompts in the
future.
Um, and of course, all of this process happens with mini batches at the same time, and it happens
again and again and again all the way through your your data is one epoch and then potentially another
time and another time after that as it goes through multiple epochs.
So that repeated process is what is known as training.
And now in the next video, we will just step through a diagram to illustrate that.