Welcome to week six, day three.
Today is going to be a day that you will either love or you will hate, depending on your particular
preference, but I'm sure there's going to be no middle ground.
It's either going to be a great day or a miserable day, depending on whether you like or are interested
in traditional machine learning, because today we're going to take a step back in time and look at
some foundational machine learning and get some practical examples of how things used to be.
So with that intro, and it's only for one day, I'm sure you can put up with it, even if it is something
that you find very unsavory.
Well, as a quick reminder, what you can already do is work with frontier models, building AI assistants
with tools, and with open source models like hugging face with pipelines and tokenizers and models.
You can use long chain to build a complete Rag pipeline.
And in fact, as you saw, it's actually quite, quite perfectly possible at least to do it without
long chain as well.
It's just quicker if you use line chain, but you know that there's not anything particularly magic
about rag.
And then we've talked about a five step strategy to solve commercial problems.
And we've got really, really deep with data.
I hope it wasn't too deep.
Hopefully you've survived the experience.
We did a lot of work with data.
We saw lots of charts.
You hopefully by now are very familiar with the item class and the item loader more than than perhaps
you'd ever intended to be.
But at this point we know our data back to front.
So today we talk about baselines.
I'm going to talk about what a baseline model is and why it's so important.
And then we are going to at least I'm going to have an absolute blast playing with some baseline models
and exploring more traditional machine learning to see how good we can do without using all this fancy
schmancy LM stuff.
Um, before, uh, tomorrow, we turn to the frontier.
So without further ado, let's talk a bit about a baseline.
So it's mostly common sense stuff that if you're looking to tackle a problem you start simple.
But in particular it's it's something which is fundamentally important in the world of data science
for really a couple of reasons.
The obvious one is that it gives us a sort of yardstick, which we can use to measure progress against.
If we start with something simple and traditional, then we know we're using, uh, sophisticated deep
neural networks properly.
When we see the needle moving and we see ourselves achieving far greater heights without that baseline,
we just don't know whether we're getting fabulous results or whether we're just making small steps in
a unpleasant direction.
So obviously it gives us that yardstick.
Uh, but there's another thing, too, which is that llms are not always the right solutions.
In fact, in the specific business problem we're setting out to solve around, uh, around predicting
prices of products.
It's not immediately obvious at all that llms are the right solution because typically, as I said before,
generating a price, a number from a description seems like it's more traditional NLP and linear regression,
so it feels like it belongs in the field of machine learning.
A traditional machine learning.
Um, and that makes it even more important to build a baseline, because for all we know, we'll do
that and then we'll try out frontier models and they won't do any better.
So it's obvious stuff, but it explains why we do this.
So what models are we going to be playing with today.
And it's only one day.
It's only one time that we're going to spend doing this.
And you know, it's really worth it if you're already super familiar with these models.
And it's just going to be an interesting quick experiment with our particular commercial problem if
you're new to them, I'm not going to go into tons of detail on them, but it will give you a good sense
of the perspective.
Um, so the first thing we're going to do is we're going to take our business problem.
We're going to do something that's very old school.
We're going to do what they call feature engineering.
When we understand the data and we say, okay, what do we think are going to be some of the important
factors which are likely to affect the price?
And we try and come up with these things that we will call features.
And we'll come up with some pretty obvious features, like how how do they rank in Amazon's best seller
rank, that kind of thing.
And we will then try and see whether some linear combination of these features does a good job of predicting
the price or not.
And that is often the place where you start when you're dealing with a machine learning model.
We're then going to do something called Bag of Words, which is one of the first our first forays into
natural language processing.
NLP Bag of Words is a particularly simplistic approach, where you quite literally count up the number
of words and you build yourself a little vector that consists of just how many times does each particular
word feature in this description?
So if you have a word, you one doesn't include what are known as stop words, which are words like
the which, which aren't going to make much difference to anything.
But if there's a word like Intel, uh, which may indicate that it's a laptop or a computer that would
have a certain value, Intel might be one of the words in our vocab.
And depending on whether that appears or not or if it does, how many times it appears that will affect
that location.
Uh, in our in in this bag of words, this list of counts of words in each product.
And then we're going to take that bag of words and again see if there's some linear combination of these
different words that, when combined together predicts the price of a product.
We then going to use something called word two vec, which I mentioned some time ago, which was one
of the first real sort of, uh, neural network, um, Encoding algorithms that could produce a vector
in a way that is rather smarter than a bag of words.
And we'll first use that with linear regression.
And then we're going to use that with random forests, which is a more sophisticated technique that
I'll talk about then.
But it involves taking random chunks of your data and your features in the form of bits of vectors,
and seeing whether and then creating an ensemble, a series of models that combines averages across
many of these little samples.
And then we're going to have something called support vector regression, a type of support vector machines,
which is another technique, a specific way of trying to separate out your data into different groups.
So we will try these different techniques.
We will see which one does best and see how that fares in solving our problem of predicting the price
of a product based only on its description.
With that, let's go to JupyterLab.