Take one more moment to look at this very nice diagram that lays it all out, and we will move on.
Now to the technicality that I wanted to mention.
So I have been a bit loose in the past when I've said that the model predicts the next token, which
is one way to think about it.
One simple way to think about it.
And we often use that, that turn of phrase, but it's not what's actually going on.
It's not like you have a transformer architecture and the bottom layer, what comes out of that is a
single token, which is the predicted next token.
That's not how it works.
What actually comes out of the last layer is probabilities.
It has every single token, and for every token it has.
What is the probability that that is the next token.
It outputs that series of probabilities.
That's what comes out of the bottom of the neural network.
And it's you may remember there was the last layer that we saw when we printed.
The model is called LM Head.
And that is really the very final step that comes out.
It actually comes out with a vector of numbers that are known as the logits, which are the the that
represent the probabilities.
And you need to put that through a function.
Um, this is getting probably to too much detail now, but you may know it already.
There's a function called softmax that converts these numbers into what can be thought of as probabilities,
because each each token will then be somewhere between 0 and 1, and they will all add up to one.
So it gives you a way to interpret the results of the model as a probability of each possible next token.
So that is actually what comes out of the forward pass.
Um, when you're doing inference when you're running this in inference mode, um, what you get out
of it is then all of these probabilities.
So what do you do with that.
How do you how do you say what next token it's predicting?
Well, most of the time actually it's a very simple approach.
You simply take the most likely next token.
You take the token with the highest probability.
It gave you all these different probabilities.
Find out which one is the max.
Take that one as the next letter.
There are other techniques that are a little bit more sophisticated.
You can sample randomly using these probabilities as your as your weight for how you sample.
And that gives you a bit more variety.
And there are some other techniques you can use to sample a few letters in a row, and then decide whether
that's the path that you want to take.
So there are a bunch of different strategies during inference that you can use based on these probabilities
to do the best job.
And in fact, when we go and look in our project in a second, we are going to use a slightly non-standard
strategy, since we know that each that this token represents a cost, it represents a number.
We can do something a little bit smart with it, but but that's not necessary.
You can also always just simply pick pick the one with the highest probability.
So that's model output really explained.
I hope that's now crystal clear for you.
Um, and then the loss function.
So I slightly glossed over in the last video.
Like you calculate a loss.
How bad was it?
So what does that actually mean in practice?
It's wonderfully simple.
So we've got these probabilities of all of the possible next tokens.
So what you do is you say okay, well we actually know what the next token was supposed to be.
Let's say it was supposed to be 99.
So you look up in all of these probabilities and you say, what probability did the model give for 99
for the thing that actually was the right next token.
And that's all that matters here.
All that matters is what probability did it give to the thing that was actually correct.
Um, if it gives if it gave that a 100% probability, then it was perfect.
It was 100% confident in the right results.
And everything else would have to be zero because probabilities will add up to one.
So so that would be absolutely perfect.
If it's anything less than 100%, then it didn't do well.
And the you know, the lower probability it gave, the worse it did.
And so you take that probability and then it just turns out that the formula that seems to work well
is to take the log of that number and then negate it.
So you take minus one times the log of that number.
And if you work that out, that means that if that number is one, if it's 100% probability, then you
get zero.
And that's sounds good because you want zero loss.
Loss should be nothing if you were perfect.
And then the lower your probability is, the worse the higher that loss number will be.
So taking negative log of the probability, um, is a way of of having a good, well well-behaving loss
function.
And there's a fancy name for it.
This loss function is known as the cross entropy loss.
That's what they call it.
It's just negative log of the probability of the true next token.
And that's, that's what's used.
Um, and that's that's what's being used right now if your training is going on, it is calculating
the cross entropy loss for each of the predictions.
And there's a there's another side note.
There's an The interpretation of this for particularly for data scientists amongst you, this is the
calculation that's used for classification when you're trying to classify something into different bins.
If you've got like back in the day when there were times when we were trying to classify images to be
one of 4 or 5 different categories, cross-entropy loss, you come up with a probability and you use
cross-entropy loss to figure out whether or not you've done a good job of classification.
And in fact, that makes sense, because the whole process of predicting the next token is really a
classification problem.
You're just trying to say there are many possible categories that this next token could be.
In fact, there's the entire all of the possible next tokens.
And we're going to predict which one is the most likely bucket to to put for the next token.
And so the whole process of generative AI is really just a classification problem.
Classifying the next token, figuring out the probability that the next token is what it turns out to
be.
And so interestingly, for our particular project for predicting model prices, it's really, as I said
some time ago, it's really a regression problem.
You're trying to predict a number and we're treating it like a classification problem, which is okay,
because it's really going to turn out to be just a number between 0 and 999, which, sorry, between
1 and 999, which are just 999 different possible buckets.
And we're just trying to classify effectively every product into one of these 999 buckets.
And that's why it works quite well as a classification problem.
And it's why the frontier models are good at it and why we're hoping, fingers crossed, that our open
source model is going to be good at it too.
And with all of this, some good useful theory behind us.
But now it's time to get back to some practice.
So with that in mind, it's time for us to talk about how has our open source, uh, fine tuning been
going?