And welcome back to our final time in Jupyter Lab with traditional machine learning.
It's almost over.
Personally, I find it a lot of fun.
I hope, I hope it hasn't been too unbearable for you.
Uh, it's a great experience to have had, though, and I'm really hoping that you've been playing around
yourself, adding some more features, doing some more experiments, seeing if you can't get more out
of this.
This was the last chart we looked at, which was word two vec getting an error of 115 on average.
And you may remember that we did better with that with the original Bag of Words NLP model that got
us to, I think 114 113.6 or something.
So, um, what we're now going to do is unveil the last two models.
We're going to use support vector regression from Support Vector Machines, uh, which is a fancy schmancy
traditional machine learning technique.
When you take your, your, your data points and you try and fit a hyperplane that separates the data
using things called support vectors, which are the the vectors with the points that are closest to
the hyperplane.
This may be nonsense to you, or it may be stuff that you know back to front and that I'm not explaining
it well.
In either case, it doesn't matter.
We're just going to take the library as it is from scikit learn, which is so easy to use.
We are using a linear SVR.
There are other types with different kernels that maybe give better results, but they take ages to
run.
This one runs very quickly, almost too quickly, which makes me think maybe I'm not using it to to
its best.
Um, but I have already run it and it took about five seconds, but the one I used with a different
kernel I ran all night and still hadn't finished.
So maybe that's somewhere in the middle that that is something that you may be able to to, to explore.
But this was the the best that I could do.
Uh, and let's see how it performs.
Are you ready?
Put in your bets.
And now I will run it.
No I won't.
Oh, there we go.
That works.
Uh, okay.
So lots of yellows, lots of reds, lots of greens.
It's obviously not crushing it, but there's some, uh, looks not terrible.
Let's see how that does when we get to the charts.
Well, so, uh, it is a winner so far.
112.5.
It is a hair better than the, uh, the prior winner, which was the bag of words linear regression
model.
You can see visually that there's some good things going on, but obviously it's struggling to estimate,
um, much above the average point.
Uh, so you can see that there's some progress, but not tremendous progress.
That is our support vector regression model.
And now that brings us to our last one, our last model, which is a random forest regression random
forest.
A particular technique.
It's a type of ensemble technique that involves combining lots of smaller models.
The models that it combines, each of them take a random sample of your data points and a random sample
of your features, which in our case means different chunks of our vectors.
Uh, and trains many models based on that and then combines all of those models.
In the case of a regression, it takes the average across all of these mini models, and that is called
a random forest.
So we will see how that works.
These are generally known to perform well for all shapes and sizes of datasets.
And they're they're good in that they don't have a lot of hyper parameters.
Hyper parameters is what people call just extra knobs to tweak extra things.
You have to try lots of different values for.
Random forests don't have a lot of them.
You just use it as it is and see how it does.
So we've used it as it is and now we will see how it does.
Tester dot test and we pass in random forest processor.
And again put in your bets.
Uh, do you think the random forest is going to do better or worse?
112 is the number to beat.
Let's see how traditional machine learning performs.
We see some greens.
We see some reds, we see some greens.
It takes a little bit slower to to run.
We're seeing some greens, greens, greens, reds, lots of reds.
But generally there we have it.
There we have it.
So random forest for the win.
The error is $97.
It's come in under 100.
We have a nine handle.
We've come in under $100.
Our best so far.
34% of the dots are green.
Here is our line.
Here are the green dots.
It's it's also had a bit of a problem getting predicting above the average but not too bad.
You see how well it did with that guy there.
It came in green for the really expensive item.
Uh, and uh uh, it's generally it's generally fared pretty well I would say certainly are running winner.
Congratulations to Random Forest.
Uh, and of course, congratulations to you.
If you've beaten this, you can do things like you can use random forest, but put in not only the vectors
that we've just come up with, but you can add in features as well.
You can manufacture engineer some features and shove them in as well and use that to try and beat beat
this number, get do better than than 97.
Uh, and see how you do have fun with traditional machine learning, because this is going to be the
end of it before we move on to trying out LMS.
But first, a quick wrap up with the slides.