Well, I realized that was a whole lot of theory, but I hope it gave you a good intuition that will
be a basis for what we're about to do.
And it's also very helpful when you encounter problems or if you're exploring hyperparameters, optimizations
that you have that sense of why we are playing with what we are and what it represents.
But to summarize, when we started out, we were talking about the smallest variant of Lambda 3.1,
which is an 8 billion parameter model, a 32GB of Ram that it takes up.
We realized that you can quantize it down so that the weights are an eight bits, and then it only uses
up nine gigabytes.
So only that's still a very big amount of Ram.
Uh, we could quantize it all the way down to four bits, uh, using the double Quant trick and get
it down to 5.6GB.
Uh, and then we also saw that we could use instead of trying to train the big guy, we could instead
fine tune these separate, uh, Laura matrices that get applied to the big model.
And if we do so, then we're looking at 100MB or so, 109MB of parameters, a far smaller number, a
little dot compared to the enormous base model.
So hopefully that gives you, again, a great sense of how it all fits together.
And with that, you have built some essential domain expertise.
Uh, this has been a really important week of knowledge building.
We're about to put it all into practice.
We're going to select an open source model that we'll be using for fine tuning.
We will look at some different variants of it, and then we will evaluate the base model out of the
box to see how it performs.
It's going to be a practical week next week and I'm looking forward to it.