You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 

178 lines
5.2 KiB

WEBVTT
00:00.560 --> 00:04.880
Thank you for putting up with me during my foray into traditional machine learning.
00:04.880 --> 00:08.990
I think it was useful for us and I hope that you didn't mind it too much.
00:09.020 --> 00:14.330
Maybe you enjoyed yourself a little bit like I did and tried out your own models too.
00:14.360 --> 00:17.690
Let's just look at how they appear side by side.
00:17.690 --> 00:27.320
We started with a random model, which came in at a somewhat shocking $341 off from the from the reality.
00:27.320 --> 00:34.220
And then we tried a constant model that did a whole lot better, but was still $146 wrong.
00:34.520 --> 00:40.520
We then did some proper models, the features and and linear regression model.
00:40.550 --> 00:49.370
At 139 we did a whole lot better with a bag of words model, the Countvectorizer and $114.
00:50.150 --> 00:56.570
We were slightly disappointed that when we layered on the powerful word two vec, it came in at $115.
00:56.570 --> 01:02.480
You may have noticed that there were 400 dimensions of word two vec, whilst there were a thousand dimensions
01:02.480 --> 01:07.000
in the bag of words model, But still, you would expect that the 400 dimensions in the word two vec
01:07.030 --> 01:09.490
would be just so much.
01:09.520 --> 01:14.830
There would be so much more signal in those vectors that you would expect better results.
01:14.830 --> 01:18.070
So whilst that was a bit disappointing, we quickly made up for it.
01:18.100 --> 01:24.700
First of all, by getting a hair better when we use support vector machines, but then random forests
01:24.700 --> 01:29.980
save the day with a nice $97 error there.
01:29.980 --> 01:35.410
And you know, there's there's this potential school of thought that would be to say that $97 is still
01:35.410 --> 01:40.510
disappointing given just predicting the the price of a product.
01:40.510 --> 01:41.920
But I'll tell you something.
01:41.920 --> 01:47.860
I challenge you yourself to go in and pick some of those products and blindly try and price them.
01:47.860 --> 01:49.120
It ain't easy.
01:49.120 --> 01:50.890
It's surprisingly difficult.
01:50.890 --> 01:55.360
You saw when we were confronted with that LED light, that we looked at that example a moment ago,
01:55.390 --> 02:00.340
I think, and I don't know if I had seen that, I would have probably have guessed that's about $40
02:00.340 --> 02:00.820
or something.
02:00.820 --> 02:02.500
And it was 200 and something.
02:02.500 --> 02:09.270
So, you know, it's actually surprisingly hard just given a description of something to figure out.
02:09.270 --> 02:11.160
Where is this on a scale?
02:11.250 --> 02:19.320
And so getting within $97 based purely on a description of some product that could be electronics,
02:19.350 --> 02:23.970
it could be an appliance, it could be any of those other automotive, of course, any of the other
02:23.970 --> 02:28.290
things, the categories that we picked, it's it's not as easy as it sounds.
02:28.290 --> 02:35.610
And so getting within $97 of it on average across our test set is not bad at all.
02:35.610 --> 02:36.810
Not bad at all.
02:36.840 --> 02:39.210
But potentially we'll be able to do better.
02:39.210 --> 02:40.320
We will see.
02:40.590 --> 02:41.550
All right.
02:41.550 --> 02:44.520
So well done on getting to this point.
02:44.520 --> 02:46.590
It's been a lot of fun for me.
02:46.590 --> 02:47.220
Anyway.
02:47.520 --> 02:51.660
You've tolerated me and hopefully you didn't mind it.
02:51.720 --> 02:57.330
But fear not, the time has arrived for us to go to the frontier.
02:57.330 --> 03:04.980
So, uh, next time we're going to be talking about solving commercial problems using frontier models,
03:04.980 --> 03:12.430
we are then going to run that runner against GPT four mini and see how it fares.
03:12.460 --> 03:14.140
And then I'm going to be brave.
03:14.170 --> 03:21.040
I'm going to set our sights high, and we are going to run our test dataset against the big guy, against
03:21.040 --> 03:27.610
GPT four zero maxi, the full version, the frontier version from August.
03:27.820 --> 03:30.790
And that's going to be a big test for us.
03:30.790 --> 03:32.080
We'll see how it does.
03:32.380 --> 03:38.380
And yeah, remember, it's quite a challenge for an LLM because we're we're basically we're not going
03:38.380 --> 03:40.060
to give it any training data.
03:40.090 --> 03:45.130
Unlike these traditional models where we've given them training data, we're simply going to send the
03:45.130 --> 03:51.520
test data to the LLM and say, given all of your worldly knowledge, how much do you think this is going
03:51.520 --> 03:53.980
to get and how much do you think it's going to be worth?
03:53.980 --> 03:56.470
And that's not an easy problem to set.
03:56.470 --> 04:01.480
So in many ways, the traditional machine learning models have a big advantage that they've been trained
04:01.480 --> 04:03.190
based on a training data set.
04:03.190 --> 04:08.320
In the case of these frontier models, we're just going to give them the descriptions and say, okay,
04:08.320 --> 04:09.460
how much is this?
04:09.910 --> 04:13.120
We will see how they get on in the next video.