And we return to the hugging face open LLM leaderboard.
The first place you go when selecting your base model for your training.
So first thing I'm going to do is I'm going to focus in on models with a parameter size at most, let's
say nine.
And we can we can afford to go down a bit and let's filter out everything but the base models to start
with.
Um, and let's have it show the number of parameters as well.
Here we go in this table down below.
So this is the results of the strongest models according to these various metrics.
And you will see that powerhouse clan that I mentioned many times, including this 2.52.5 is very new.
Uh 7 billion variant.
Uh is uh the highest scorer of this lot.
Um, and you'll see that Gemma is a 9 billion parameter variant of Gemma.
That's right up there to, uh, Mistral up here.
Uh, and the one that we like to talk about a lot.
Um, there was phi2, by the way, from Microsoft.
Uh, llama 3.1 is a little bit further down.
Uh, now it's worth mentioning that the numbers are all reasonably close at the top, although there's
somewhat something of a margin here.
Uh, so it might concern you that llama 3.1 that you happen to know is the one we're going to end up
picking because you've seen it in the code is further down.
Um, but there is there is something of a reason for that.
Uh, when you're looking at these different scores, really you do need to, to also bring in the,
the versions that have been trained.
Um, the what's called the instruct variants, which are the same models but then given uh, more trained
using various reinforcement learning techniques to to respond to that particular chat instruct style.
And when it's uh, given that kind of framework, it's more likely to be able to perform against these
various tests because it will respond to the instruction that's being given rather than being expected
to be trained to, to adapt to a different task.
Um, so really, you're getting a more realistic view of the capability, even of the base model, if
you look at how it performs with benchmarks, when you look at the instruct variation, if that makes
sense.
And when we do that, we see that llama 3.18 billion really is in the top grouping here.
Uh, we've got uh, Phi three is up there as well.
Uh, jammer.
And then um, the Meta Llama 3.1.
So it's doing it's doing very well when you look at the instruct variant.
And as I said, uh, somewhat perversely, I'm not suggesting that we actually use the instruct variant.
I'm suggesting that we stick with the the base version of it because we don't want it.
We don't want it necessarily to have used up lots of its thought process, lots of its sort of training
power, learning about things like system prompts and user prompts and so on.
I'm just saying that once you have been through that exercise, you can see it performs well in all
of these scores.
And that gives us a good sense that the base model is good at adapting to be able to address these different
benchmarks.
So that's a more nuanced way to interpret the results of the leaderboard.
You can look at the instruct variant and see how that performs, and it still gives you a good indication
of how the base model will perform as well.
Now, there is one other slightly subtle reason that that I'm picking llama, even though you might
say that either five three, Gemma or indeed Gwen would look like they are scoring higher in many fronts.
There is a convenience to llama.
That's just.
It only makes a small difference to everything, but it does make our code a bit simpler and it makes
the task a bit easier for llama, which is that when you look at the tokenizer for llama, you'll see
that for llama, every number between 0 and 999, every three digit number gets mapped to one token.
The same is actually not true for 3 or 4, or for Quan.
In all three of those other models, they have basically a kind of you can think of it as like a token
per digit.
So the number 999 ends up as three separate tokens.
Now, you might ask me, what difference does that make that it shouldn't make any difference at all.
So it's going to turn out that when we're doing training we are we're using a model to generate tokens
and we're trying to make it think in terms of more of a regression model.
We want it to be trying to solve for getting better and better at predicting the next token, and that
that should map to the price.
So it simplifies the problem.
If the price is reflected exactly in one token that the model has to generate.
So just in this particular situation, for the particular problem we're trying to solve, the tokenization
strategy for Lambda 3.1 works very well because the the, the single next token that it generates will
in itself reflect everything about the price.
So that, uh, it's not the case that it might predict that the next token should be nine, and that
could be $9 or $99 or $999.
And that will only transpire when it does the token.
After that.
No, it's going to be the case that the single token that it projects as the next token in its answer
will reflect the full price of the product in one token.
So it's a nuance, but it's a reason why we lean towards selecting llama 3.1 in this case.
But by all means, we will have the ability to choose other models and see how they perform.
But llama gets a bit of an edge because of this convenience with the way that it tokenizes.
So that gives you some color on some of the thought process that goes behind selecting a model, looking
at the leaderboards, looking a little bit more deeply at leaderboards, thinking about instruct variants
versus the base model's parameter sizes, and then also some nuances about things like the way that
the tokenization works.
And all of that together has allowed us to come to the decision that we are going to select llama 3.18
billion as the base model for our project.
And now with that, let's go to the Colab and give that base model a try.