Welcome everybody.
So in the past I've said quite a few times, I am excited to start this this week or this topic.
And I want to say that all of that has been nonsense, because the excitement that I've had has paled
in significance compared to the level of excitement I have today.
As we start week six and we embark upon the world of training, this is what it's all about.
Now it gets real.
Prepare yourself.
So up to this point, we've always been talking about what they call inference.
And inference is when you take a model that's been trained to against a ton of data, and it's then
used at runtime to predict the next token given an input.
And we've been looking at different ways to get that inference better and better.
We're now going to go and look at the models themselves and understand how can you train models so that
they are even better at runtime and inference.
And this this is where it gets advanced.
We start with the less glamorous part of it, which is about the data.
And whilst crafting a data set might might not sound as glamorous, it isn't as glamorous.
It happens to be absolutely essential and perhaps one of the most important parts.
And we're going to spend today and tomorrow spending, getting really deep into the data, understanding
it back to front, visualizing it, cleaning it up, curating it, getting it to a form where we really
like it a lot.
Um, and that's something that you have to do.
We're also going to spend some time understanding, how are we going to gauge success of this project?
What are we trying to achieve and how do we know if we've done it or not?
But first, let's talk for a second about the eight weeks that you've had, where you've come from and
where you are going.
We started some time ago now, six weeks ago, uh, right on the left where you're heading is an LM
engineering master.
Over on the right.
In week one, we talked about frontier models, and we tried some of them out.
In week two we were using multiple APIs.
We were building UIs with Gradio and Multi-modality.
Week three we explored hugging Face the Pipelines Tokenizers and then the models.
In week four, we were selecting models.
We were using code generation and we built something that was what, 60,000 times faster?
It was able to optimize code 60,000 times is remarkable.
Week five.
Last week, of course, was rag all about rag very hot topic.
And now that brings us to week six fine tuning a frontier model where we've arrived at training in week
seven.
We'll do this again.
But now.
Now we'll be dealing with open source models and basically building our own model.
And week eight is where it all comes together.
So with that, let's talk a bit about the transition that we are now making.
We are moving from inference to training as I say.
So let's just talk about when we when we've been working on inference, what have we been doing.
We've we've developed different techniques so that when we are running these models, we can try and
get them to perform better and better.
We've tried Multi-shot prompting when we give it lots of examples for it to work on.
We've tried prompt chaining when we send multiple different messages and build on top of each other
and combine the results.
We've used tools where we had the model almost be able to call back into our code, although it wasn't
quite that magical.
In order to do things like calculate the price of an airline ticket or the price of to travel to a different
city.
And then most recently, we worked on Rag injecting more relevant content context into the prompt.
So all of these, what they have in common is they're all about taking an existing trained model and
figuring out how we can best use it to take advantage of what it knows by calling it multiple times,
adding in context, and so on.
What we're going to do now is move on to training.
So in training, what we're trying to do is take a deep neural network, potentially with its billions
of parameters, and figure out how can we tweak those parameters, change those weights, optimize them
very slightly based on data so that it gets better and better at predicting future tokens, and whilst
adding in more context at inference and things like that is a is a bit of a broad brush stroke in terms
of how you can affect the outcomes with training.
It's a much more nuanced technique that allows you to gradually build up deeper, finer grained understanding
of the problem you're trying to solve.
Now, trying to train a multi-billion parameter LLM is a rather expensive proposition.
It's something that the Frontier Labs probably spend well, they do spend north of $100 million on training
their their best models, and that's outside my budget.
And I'm guessing it's outside your budget.
And so that's unfortunately not possible for us.
But luckily we could take advantage of something called transfer learning.
And transfer learning says that it's perfectly doable to take an existing trained LM, a model that's
already been pre-trained on a ton of data, and you can then just continue the training with a particular
data set.
Perhaps that solves a very specialized problem, and it will sort of transfer all of the knowledge that's
already accumulated, and you'll be able to add on some extra knowledge on top of that.
Um, and so you could take a pre-trained model space and then you can sort of, um, you can, you can
make it more precisely trained for your task.
And that process is known as fine tuning.
Um, as just, just as it sounds.
Um, and of course, we're going to be using some techniques that I've name dropped in the past, like
Q Laura, as ways to do it.
That will be manageable in terms of memory and so on.
So let me now introduce the the problem, the commercial problem that we are going to be working on
for most of the next few weeks.
So let's say we work at an e-commerce company or a marketplace company working in products, and we
want to build a model that can take a description of any of almost any product, a wide variety of products,
let's say electrical electronic products or computers, fridges, washing machines and other things
for the home and car and be able to estimate how much it costs based just on the description.
Um, now, uh, that's it's a it's a nice, easy to understand problem.
It's got a very easy to measure outcome.
You might if the data scientists amongst you might raise your hand and say, that doesn't particularly
sound like a problem that's designed for a generative AI solution that generates text.
Sounds like something that needs a model that creates a number.
And typically that's the domain of what people call regression models.
Types of models that produce a number that you can then try and fit to, um.
And you'd be right.
You'd be making a valid point.
It is typically more a regression kind of problem.
Um, but it turns out it's still is going to be a great problem for us to work with.
And there's a few reasons for that.
One of them is that turns out frontier models are actually great at solving this kind of problem to
to start with, they were intended just to generate text and just to be able to do things that involve
things like tasks like summarization and other and other text generation activities.
But as we've discovered, when we ask a model to respond in JSON and respond with with information,
it can be very effective at responding back with quantitative results.
And so actually the frontier models, um, perhaps because of the emergent intelligence we talked about
a long time ago, now have become highly effective at even these kinds of problems that were traditionally
the domain of a regression model.
So it is absolutely possible to use JNI for this.
And in fact, you'll see the frontier models are going to do spectacularly well at this better than
simple regression models that we'll build to.
So it turns out that it does work in this space.
Um, it's also going to be much more enjoyable for us when we try and build our own models.
And here's why.
The great thing about this problem is that it's very easy to to measure whether we're doing it well
or not.
If if we predict if we take a product and we know how much it costs, we can put that in and see how
well the product does.
See how well our model does in guessing the price of that product.
Where are some other text generation problems are harder to to measure in a very human understandable
way.
So if you're doing translation between two languages, say, then sure, you can tell whether something
is generating good Spanish from from English, but how good becomes a judgment call.
And there are of course scoring methodologies.
But then you get into a lot of complexity about how they work and, and whether you're actually doing
better or not.
So, you know, there are lots of other problems that are more text generation related, such as building
something that would actually write the description of a product, but they're not as easy to measure
as just saying, come up with a price.
Come up with the price is fabulously simple to measure.
If we're doing it well and and measure it in a very humanly understandable way.
Not with some fancy data science metric like perplexity and stuff, but with something that we'll all
understand.
How accurate is the price of this fridge that we've just told it?
We'll be able to tell that and see it and watch the improvement.
So for that reason and the other reasons, I actually think this is a really nice, well-defined challenge
for us to take on, and we're going to have some success with it so that that is the problem for you.
All right.
In the next video, we're going to start talking about data and I will see you there.