And it's this time again, when we look at the podium of how our models are performing across the board.
You already know the answer, but we should look at it anyway and and and have a giggle.
Uh, so we first look at the constant model that just guesses the average value from the training data
set.
And if we look at that number, it was 146.
It's looking distinctly lower in here right now.
And I think you probably know why.
Uh, then we looked at a traditional machine learning model that was able to beat the average, which
we would hope it would, but not by masses.
Uh, we looked at the random forest approach, which was the best of the traditional machine learning
approaches coming in at 97.
We did compare it with a human being and showed that human could beat the basic, simple features,
but was beaten by random forest model.
Uh, GPT four zero was the best of the various frontier models we looked at, and absolutely crushed
it with 76 way outperforming this human.
And then today we've looked at the, uh, fine tuned bass llama, 3.18 billion parameters quantized
all the way down to four bits.
Uh, and we saw that that was at 396, devastatingly bad.
Uh, and it was only slightly better when we quantized to eight bits, instead coming in at $301 of
error.
Uh, so clearly we're off to a bad start with llama 3.1, but in some ways that's exciting because the
challenge is on.
Uh, we've got the the model here ready for us to try and explore how we can make it better.
And our target is to try and be competitive with a model like GPT four, which is, you know, has trillions
of parameters.
If we can be competitive with that in this particular task, and we can do it with an open source model
that's for free, then we have achieved a great thing.
Okay.
And so in summary, at this point, at this juncture, you have reached 80% of your way in this journey
and that is absolutely fantastic.
I'm so happy you.
Are sticking at it.
I'm so happy you're here.
I got to tell you, the 20% that's still to come.
It's the best.
It is the best.
The juiciest content is what's to come.
As we look at training this model.
And then in the finale next week, we really put everything together.
I mean, it's just it's going to be a crescendo.
It's going to get better and better.
Hang in there.
All of the good stuff is to come.
So what we're doing next time, there's some more hyperparameters.
I know that these hyperparameters can be a bit grueling, but this is where the really important stuff
is learned.
So there's some hyperparameters that control training that I'll explain to you.
We're then going to set up something called a supervised fine tuning SFT trainer, which is where it
all happens.
And then we're going to kick off training.
You're going to be training your own proprietary large language model, uh, based on, uh, llama 3.1
based model.
And we're going to be doing it, uh, in the next session.
So I will see you there.