So before we try our new model and one more recap on the models so far and keep notes of this so we
can see how we do.
And your excitement can be there while we run our fine tuned model.
We started with a constant model.
We actually started with a random model, but I think we can put that one to bed.
That's that was that was silly.
So a constant model which just guesses the average from the training data set ends up with an error
of 146.
Uh, and we certainly hope that we can do better than 146.
Otherwise, we might as well stick with a constant.
When we used a very simplistic traditional machine learning with basic features, we got 139.
Remember that?
Well, I hope random forest, a more sophisticated algorithm that also that looked at the language,
the words um, got down to 97.
This human did a poor job at 127.
Uh, GPT four.
Oh, the big guy did very nicely indeed.
At 76 and the Bass Llama 3.1, untrained, quantized down to four bits, did an appalling $396 of error.
A much better off just sticking with the constant than using an untrained llama.
The poor thing did not do particularly well at all.
So I go through this one more time so that you have this nicely framed.
The question is, remember, GPT four is a model that has trillions of weights.
GPT four had 1.76 trillion GPT four.
It's not known, but it's considered to be much more than that.
So a huge number of weights.
Llama 3.1 base has 8 billion weights, and we have reduced them down to four bits.
And then we have used our color.
Sorry.
Our Lora adapters like 109MB worth of them to to put some extra weights that we can use to adapt.
Llama lemma 3.1 base, but these are still small numbers, and obviously this is an open source model,
which means it's free to run.
So I'm saying all this to set expectations that obviously it's a lot to ask to try and compete with
some of these models at the frontier.
The thing that you need to be looking out for is, can we do better than traditional machine learning?
Can we do better than a human can?
Certainly.
Can we do better than constant?
And how do we stack up when we compare ourselves to GPT four?
So the leading frontier model, and we can also compare it to GPT four or mini, um, as well, uh,
afterwards.
So that gives you the context.
I hope you have this in your mind.
Maybe write down the numbers so you're ready for for what's to come.
And it is time for us to head to Colab and to run inference on the the best, strongest checkpoint from
from the the training of our own verticalized specialized, uh, open source model.