From the uDemy course on LLM engineering.
https://www.udemy.com/course/llm-engineering-master-ai-and-large-language-models
You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
178 lines
5.2 KiB
178 lines
5.2 KiB
WEBVTT |
|
|
|
00:00.560 --> 00:04.880 |
|
Thank you for putting up with me during my foray into traditional machine learning. |
|
|
|
00:04.880 --> 00:08.990 |
|
I think it was useful for us and I hope that you didn't mind it too much. |
|
|
|
00:09.020 --> 00:14.330 |
|
Maybe you enjoyed yourself a little bit like I did and tried out your own models too. |
|
|
|
00:14.360 --> 00:17.690 |
|
Let's just look at how they appear side by side. |
|
|
|
00:17.690 --> 00:27.320 |
|
We started with a random model, which came in at a somewhat shocking $341 off from the from the reality. |
|
|
|
00:27.320 --> 00:34.220 |
|
And then we tried a constant model that did a whole lot better, but was still $146 wrong. |
|
|
|
00:34.520 --> 00:40.520 |
|
We then did some proper models, the features and and linear regression model. |
|
|
|
00:40.550 --> 00:49.370 |
|
At 139 we did a whole lot better with a bag of words model, the Countvectorizer and $114. |
|
|
|
00:50.150 --> 00:56.570 |
|
We were slightly disappointed that when we layered on the powerful word two vec, it came in at $115. |
|
|
|
00:56.570 --> 01:02.480 |
|
You may have noticed that there were 400 dimensions of word two vec, whilst there were a thousand dimensions |
|
|
|
01:02.480 --> 01:07.000 |
|
in the bag of words model, But still, you would expect that the 400 dimensions in the word two vec |
|
|
|
01:07.030 --> 01:09.490 |
|
would be just so much. |
|
|
|
01:09.520 --> 01:14.830 |
|
There would be so much more signal in those vectors that you would expect better results. |
|
|
|
01:14.830 --> 01:18.070 |
|
So whilst that was a bit disappointing, we quickly made up for it. |
|
|
|
01:18.100 --> 01:24.700 |
|
First of all, by getting a hair better when we use support vector machines, but then random forests |
|
|
|
01:24.700 --> 01:29.980 |
|
save the day with a nice $97 error there. |
|
|
|
01:29.980 --> 01:35.410 |
|
And you know, there's there's this potential school of thought that would be to say that $97 is still |
|
|
|
01:35.410 --> 01:40.510 |
|
disappointing given just predicting the the price of a product. |
|
|
|
01:40.510 --> 01:41.920 |
|
But I'll tell you something. |
|
|
|
01:41.920 --> 01:47.860 |
|
I challenge you yourself to go in and pick some of those products and blindly try and price them. |
|
|
|
01:47.860 --> 01:49.120 |
|
It ain't easy. |
|
|
|
01:49.120 --> 01:50.890 |
|
It's surprisingly difficult. |
|
|
|
01:50.890 --> 01:55.360 |
|
You saw when we were confronted with that LED light, that we looked at that example a moment ago, |
|
|
|
01:55.390 --> 02:00.340 |
|
I think, and I don't know if I had seen that, I would have probably have guessed that's about $40 |
|
|
|
02:00.340 --> 02:00.820 |
|
or something. |
|
|
|
02:00.820 --> 02:02.500 |
|
And it was 200 and something. |
|
|
|
02:02.500 --> 02:09.270 |
|
So, you know, it's actually surprisingly hard just given a description of something to figure out. |
|
|
|
02:09.270 --> 02:11.160 |
|
Where is this on a scale? |
|
|
|
02:11.250 --> 02:19.320 |
|
And so getting within $97 based purely on a description of some product that could be electronics, |
|
|
|
02:19.350 --> 02:23.970 |
|
it could be an appliance, it could be any of those other automotive, of course, any of the other |
|
|
|
02:23.970 --> 02:28.290 |
|
things, the categories that we picked, it's it's not as easy as it sounds. |
|
|
|
02:28.290 --> 02:35.610 |
|
And so getting within $97 of it on average across our test set is not bad at all. |
|
|
|
02:35.610 --> 02:36.810 |
|
Not bad at all. |
|
|
|
02:36.840 --> 02:39.210 |
|
But potentially we'll be able to do better. |
|
|
|
02:39.210 --> 02:40.320 |
|
We will see. |
|
|
|
02:40.590 --> 02:41.550 |
|
All right. |
|
|
|
02:41.550 --> 02:44.520 |
|
So well done on getting to this point. |
|
|
|
02:44.520 --> 02:46.590 |
|
It's been a lot of fun for me. |
|
|
|
02:46.590 --> 02:47.220 |
|
Anyway. |
|
|
|
02:47.520 --> 02:51.660 |
|
You've tolerated me and hopefully you didn't mind it. |
|
|
|
02:51.720 --> 02:57.330 |
|
But fear not, the time has arrived for us to go to the frontier. |
|
|
|
02:57.330 --> 03:04.980 |
|
So, uh, next time we're going to be talking about solving commercial problems using frontier models, |
|
|
|
03:04.980 --> 03:12.430 |
|
we are then going to run that runner against GPT four mini and see how it fares. |
|
|
|
03:12.460 --> 03:14.140 |
|
And then I'm going to be brave. |
|
|
|
03:14.170 --> 03:21.040 |
|
I'm going to set our sights high, and we are going to run our test dataset against the big guy, against |
|
|
|
03:21.040 --> 03:27.610 |
|
GPT four zero maxi, the full version, the frontier version from August. |
|
|
|
03:27.820 --> 03:30.790 |
|
And that's going to be a big test for us. |
|
|
|
03:30.790 --> 03:32.080 |
|
We'll see how it does. |
|
|
|
03:32.380 --> 03:38.380 |
|
And yeah, remember, it's quite a challenge for an LLM because we're we're basically we're not going |
|
|
|
03:38.380 --> 03:40.060 |
|
to give it any training data. |
|
|
|
03:40.090 --> 03:45.130 |
|
Unlike these traditional models where we've given them training data, we're simply going to send the |
|
|
|
03:45.130 --> 03:51.520 |
|
test data to the LLM and say, given all of your worldly knowledge, how much do you think this is going |
|
|
|
03:51.520 --> 03:53.980 |
|
to get and how much do you think it's going to be worth? |
|
|
|
03:53.980 --> 03:56.470 |
|
And that's not an easy problem to set. |
|
|
|
03:56.470 --> 04:01.480 |
|
So in many ways, the traditional machine learning models have a big advantage that they've been trained |
|
|
|
04:01.480 --> 04:03.190 |
|
based on a training data set. |
|
|
|
04:03.190 --> 04:08.320 |
|
In the case of these frontier models, we're just going to give them the descriptions and say, okay, |
|
|
|
04:08.320 --> 04:09.460 |
|
how much is this? |
|
|
|
04:09.910 --> 04:13.120 |
|
We will see how they get on in the next video.
|
|
|