From the uDemy course on LLM engineering.
https://www.udemy.com/course/llm-engineering-master-ai-and-large-language-models
You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
469 lines
13 KiB
469 lines
13 KiB
WEBVTT |
|
|
|
00:01.040 --> 00:07.820 |
|
Okay, so here we are, back in Jupyter Lab, ready for the next use of a frontier model, and see this |
|
|
|
00:07.820 --> 00:10.460 |
|
now as somewhere where you're really building expertise. |
|
|
|
00:10.460 --> 00:12.320 |
|
This should be very natural to you. |
|
|
|
00:12.710 --> 00:17.900 |
|
As usual, we start by considering our system prompt and being thoughtful about how we will prompt the |
|
|
|
00:17.900 --> 00:18.470 |
|
model. |
|
|
|
00:18.800 --> 00:25.610 |
|
Um, what we're going to say to to the model is, look, you identify and summarize the five most detailed |
|
|
|
00:25.610 --> 00:32.990 |
|
items from a list by selecting deals that have the most detailed, high quality descriptions and the |
|
|
|
00:32.990 --> 00:37.820 |
|
most clear price, uh, respond strictly in JSON, blah, blah, blah. |
|
|
|
00:37.910 --> 00:42.740 |
|
Um, most important is you respond with five deals that have the most detailed product description. |
|
|
|
00:42.740 --> 00:44.810 |
|
It's not important to mention the terms. |
|
|
|
00:45.110 --> 00:49.940 |
|
Um, and now I also I have laid out here the JSON structure. |
|
|
|
00:49.940 --> 00:54.020 |
|
I should probably mention that this isn't required because we're going to be using structured outputs, |
|
|
|
00:54.020 --> 01:01.070 |
|
but I like to do this to use it as another opportunity to give more direction on exactly what's expected |
|
|
|
01:01.070 --> 01:01.940 |
|
in each one. |
|
|
|
01:02.120 --> 01:04.650 |
|
Uh, so this is something which you can experiment with. |
|
|
|
01:04.650 --> 01:08.700 |
|
But strictly speaking we shouldn't need to specify JSON like that. |
|
|
|
01:09.420 --> 01:11.640 |
|
So that's my system prompt. |
|
|
|
01:11.670 --> 01:16.740 |
|
The user prompt says respond with the most promising five deals from this list. |
|
|
|
01:16.740 --> 01:17.700 |
|
Selecting those. |
|
|
|
01:17.730 --> 01:18.510 |
|
Blah blah blah blah blah. |
|
|
|
01:18.540 --> 01:19.470 |
|
I will let you read it through. |
|
|
|
01:19.470 --> 01:20.610 |
|
But you know, this kind of stuff. |
|
|
|
01:20.610 --> 01:29.730 |
|
Well, uh, and um, uh, the deals and then it lists the deals and it calls deal dot describe for each |
|
|
|
01:29.730 --> 01:30.780 |
|
of those. |
|
|
|
01:31.410 --> 01:31.800 |
|
Okay. |
|
|
|
01:31.830 --> 01:32.310 |
|
That's simple. |
|
|
|
01:32.310 --> 01:36.660 |
|
So now let me just print the beginning of the user prompt is quite long. |
|
|
|
01:36.660 --> 01:39.720 |
|
So I'm just going to print the first 2000 characters of the user prompt. |
|
|
|
01:39.720 --> 01:41.310 |
|
So you get a sense for it. |
|
|
|
01:41.340 --> 01:42.270 |
|
Here it is. |
|
|
|
01:42.270 --> 01:43.290 |
|
Let's have a look. |
|
|
|
01:43.290 --> 01:48.270 |
|
So there's the uh user prompt respond with blah blah blah blah blah blah blah. |
|
|
|
01:48.270 --> 01:50.370 |
|
And then here are the deals. |
|
|
|
01:50.370 --> 01:58.500 |
|
And this is the description that we've taken from um, the, the, the description function that we |
|
|
|
01:58.500 --> 02:00.960 |
|
looked at before the describe method. |
|
|
|
02:00.960 --> 02:10.010 |
|
So essentially the task that we're assigning to GPT four is to digest this list of deals, select pluck |
|
|
|
02:10.040 --> 02:12.650 |
|
out from this list and there's going to be 50 of them. |
|
|
|
02:12.650 --> 02:19.220 |
|
Pick the five which have the clearest description and from which it can take the price. |
|
|
|
02:19.250 --> 02:24.950 |
|
Parse the price out of this in a way that is that is most clear. |
|
|
|
02:25.100 --> 02:29.240 |
|
And that is, of course, a fuzzy kind of problem that requires some thought. |
|
|
|
02:29.240 --> 02:33.680 |
|
It's the kind of thing that would be only a year or two ago would have been unthinkably hard. |
|
|
|
02:33.770 --> 02:38.180 |
|
How would you do that in a reliable way, using just engineering code. |
|
|
|
02:38.180 --> 02:44.990 |
|
And it's the kind of problem that's so perfect for frontier models, because that kind of nuanced understanding |
|
|
|
02:44.990 --> 02:51.260 |
|
and being able to realize that up to something means a price and that, but up to something off is not |
|
|
|
02:51.260 --> 02:52.190 |
|
the same as a price. |
|
|
|
02:52.190 --> 02:54.620 |
|
That kind of logic is really hard. |
|
|
|
02:54.620 --> 03:00.680 |
|
But frontier models are remarkably effective at understanding that, understanding the meaning behind |
|
|
|
03:00.680 --> 03:05.630 |
|
these expressions and using that intelligently to give us an output. |
|
|
|
03:05.630 --> 03:12.950 |
|
So just based on these prompts, we're able to now say, okay, let's get recommendations. |
|
|
|
03:12.950 --> 03:18.290 |
|
And now we're going to see a different API than usual because we're using structured output. |
|
|
|
03:18.710 --> 03:23.300 |
|
So my function get recommendations completion is OpenAI. |
|
|
|
03:23.300 --> 03:24.800 |
|
And look it's a bit different here. |
|
|
|
03:24.800 --> 03:31.160 |
|
The this is at least as of this moment uh is a beta offering from OpenAI. |
|
|
|
03:31.190 --> 03:34.310 |
|
But I imagine it's going to be fully productionized very, very soon. |
|
|
|
03:34.310 --> 03:39.110 |
|
So you say OpenAI beta or that's beta for Americans. |
|
|
|
03:39.170 --> 03:44.870 |
|
Uh, OpenAI beta, dot chat, dot completions, dot parse. |
|
|
|
03:44.960 --> 03:52.520 |
|
Uh, that's the, the the way that you call this, um, and you provide the model as, as usual. |
|
|
|
03:52.520 --> 03:57.620 |
|
And this is supported by GPT four and mini and GPT four, the frontier version two. |
|
|
|
03:57.770 --> 04:02.120 |
|
Uh, as always, the message is, you know, I'm just going to not even say it. |
|
|
|
04:02.120 --> 04:02.870 |
|
You know it. |
|
|
|
04:02.870 --> 04:04.100 |
|
You don't need me to tell you. |
|
|
|
04:04.130 --> 04:07.310 |
|
Surely, uh, this is the difference. |
|
|
|
04:07.310 --> 04:12.440 |
|
You remember, in the past, we've used this response format as a way of specifying that we require |
|
|
|
04:12.470 --> 04:14.000 |
|
JSON to come back. |
|
|
|
04:14.030 --> 04:16.700 |
|
Now, this time we don't say JSON. |
|
|
|
04:16.730 --> 04:20.900 |
|
What we do is we pass in the class the deal selection class. |
|
|
|
04:20.900 --> 04:25.490 |
|
That's the class we were just looking at here, the one that I did some scrappy JSON for. |
|
|
|
04:25.520 --> 04:30.020 |
|
We pass in that class in as our response format. |
|
|
|
04:30.500 --> 04:35.810 |
|
And then when we're getting the results we don't call our usual API calls. |
|
|
|
04:35.810 --> 04:43.340 |
|
We say completion is zero as usual dot message and then not content but instead dot parsed. |
|
|
|
04:43.340 --> 04:49.970 |
|
And that is going to take the response and parse it into an instance of deal selection. |
|
|
|
04:50.000 --> 04:52.100 |
|
And that is what it will return. |
|
|
|
04:52.400 --> 04:54.710 |
|
So with that build up it better return it. |
|
|
|
04:54.740 --> 04:59.780 |
|
Let's run that code and see what happens and what we get back. |
|
|
|
04:59.780 --> 05:05.150 |
|
So we are calling GPT for zero and we're sorry GPT for mini. |
|
|
|
05:05.180 --> 05:09.830 |
|
We are specifying a particular structured format of output. |
|
|
|
05:09.830 --> 05:10.820 |
|
It has returned. |
|
|
|
05:10.850 --> 05:17.390 |
|
Let's see why don't we print what type of thing has come back type results. |
|
|
|
05:17.390 --> 05:19.520 |
|
Let's see what kind of object has come back. |
|
|
|
05:19.520 --> 05:23.210 |
|
It is an agents deals deal selection. |
|
|
|
05:23.210 --> 05:26.150 |
|
That is indeed the kind of object that we were hoping to get back. |
|
|
|
05:26.150 --> 05:31.760 |
|
I think we better, uh, see what it actually looks like. |
|
|
|
05:31.790 --> 05:34.490 |
|
It's going to be a set. |
|
|
|
05:34.610 --> 05:38.780 |
|
Uh, it's a deal selection which has within it deals is the one field. |
|
|
|
05:38.780 --> 05:39.350 |
|
If you remember. |
|
|
|
05:39.380 --> 05:40.850 |
|
Deals is the one thing we set. |
|
|
|
05:40.850 --> 05:42.140 |
|
And here are the deals. |
|
|
|
05:42.140 --> 05:45.830 |
|
So let's let's find out how many deals we got back. |
|
|
|
05:46.460 --> 05:49.730 |
|
We are hoping of course, that we got back five deals. |
|
|
|
05:50.000 --> 05:50.870 |
|
Let's see. |
|
|
|
05:50.900 --> 05:53.870 |
|
We did indeed get back five deals. |
|
|
|
05:53.870 --> 05:57.020 |
|
And let's print the first of those deals. |
|
|
|
05:57.020 --> 06:02.780 |
|
And we get a Samsung Galaxy Watch Ultra, a top tier smartwatch. |
|
|
|
06:02.960 --> 06:06.710 |
|
Uh, and we've got a nice summary of this. |
|
|
|
06:06.710 --> 06:10.010 |
|
And you'll notice this is actually looking it's looking great for us. |
|
|
|
06:10.010 --> 06:14.150 |
|
It's looking like this is the kind of summary that we're going to want to use when we pass it into our |
|
|
|
06:14.150 --> 06:22.250 |
|
prices to estimate the price of this, and GPT four mini has pulled out a price and a URL. |
|
|
|
06:22.250 --> 06:26.060 |
|
And of course, if we click on that URL, it's going to open up that deal. |
|
|
|
06:26.060 --> 06:27.020 |
|
There it is. |
|
|
|
06:27.020 --> 06:28.160 |
|
Wonderful. |
|
|
|
06:28.310 --> 06:32.690 |
|
Uh, so it is working well for us. |
|
|
|
06:32.960 --> 06:34.790 |
|
Uh, okay. |
|
|
|
06:34.820 --> 06:46.850 |
|
Now, the last thing to show here is that, um, we are going to, uh, look to import our scanner agent. |
|
|
|
06:46.850 --> 06:50.600 |
|
So I'm going to do the thing again where I'm going to kick this off because it's going to take a minute |
|
|
|
06:50.600 --> 06:51.380 |
|
or two. |
|
|
|
06:51.770 --> 06:57.950 |
|
Um, and while it's running, I'm going to go over and show you what the scanner agent is. |
|
|
|
06:57.950 --> 07:01.370 |
|
So the scanner agent is over here. |
|
|
|
07:01.640 --> 07:08.180 |
|
Uh, this is a bunch of code that's going to look very similar to the code that we looked at before. |
|
|
|
07:08.540 --> 07:15.920 |
|
Uh, and so it's, uh, just been written now with comments and with better structure so that this is |
|
|
|
07:15.920 --> 07:18.620 |
|
something we can consider to be production quality. |
|
|
|
07:18.620 --> 07:18.620 |
|
see. |
|
|
|
07:18.890 --> 07:21.710 |
|
Um, but it has the system prompts that we just looked at. |
|
|
|
07:21.710 --> 07:28.880 |
|
It has a user prompt that we looked at and it says respond strictly with five deals, no more. |
|
|
|
07:28.880 --> 07:33.890 |
|
And what we get is an init method that sets it up and connects to OpenAI. |
|
|
|
07:33.920 --> 07:41.000 |
|
The fetch deals is the method that we were just looking at that does, uh, gets a list of scraped deal |
|
|
|
07:41.300 --> 07:45.020 |
|
by calling scrape deal fetch, just as you would expect. |
|
|
|
07:45.050 --> 07:52.820 |
|
What it also does here, though, is that potentially it gets a memory object, which is a list of URLs, |
|
|
|
07:52.820 --> 07:59.540 |
|
and it will remove from the results anything that it remembers has already been surfaced in a previous |
|
|
|
07:59.540 --> 08:00.200 |
|
search. |
|
|
|
08:00.200 --> 08:05.030 |
|
And this is how we make sure that we fetch new deals, not deals that we've already looked at before. |
|
|
|
08:05.030 --> 08:10.850 |
|
And of course, memory, which is one of the of the facets of building an agent framework, is something |
|
|
|
08:10.850 --> 08:12.860 |
|
we'll be looking at more in the future. |
|
|
|
08:13.790 --> 08:20.510 |
|
Uh, and then we make user prompt and then we do, uh, this is the same code as before. |
|
|
|
08:20.510 --> 08:25.100 |
|
And then scan is basically exactly the function that we just looked at. |
|
|
|
08:25.130 --> 08:30.920 |
|
We make a user prompt and then we call OpenAI beta dot chat, dot completions, dot parse. |
|
|
|
08:30.920 --> 08:32.390 |
|
We pass in the model. |
|
|
|
08:32.390 --> 08:39.260 |
|
We specify that the response format needs to be the deal selection class, so that we know that it's |
|
|
|
08:39.260 --> 08:43.250 |
|
going to return that and we parse it back into result. |
|
|
|
08:43.250 --> 08:46.130 |
|
And that is what we get to right here. |
|
|
|
08:46.340 --> 08:53.000 |
|
When we then, um, uh, we do first remove any deals. |
|
|
|
08:53.000 --> 08:57.860 |
|
If the price isn't greater than zero, we don't want to return deals which have zero price. |
|
|
|
08:58.010 --> 09:00.020 |
|
Um, and we return that result. |
|
|
|
09:00.020 --> 09:05.180 |
|
And if we got no deals back, um, because perhaps we'd already surfaced everything, then we return |
|
|
|
09:05.180 --> 09:05.750 |
|
none. |
|
|
|
09:05.750 --> 09:08.930 |
|
And that's indicated by that optional at the top there. |
|
|
|
09:08.990 --> 09:10.640 |
|
So that's the class. |
|
|
|
09:10.670 --> 09:13.340 |
|
And with that, let me go back to day three. |
|
|
|
09:13.340 --> 09:16.790 |
|
And we should see that the results have come back. |
|
|
|
09:17.900 --> 09:22.190 |
|
Now one of the things I'm aware of that you may have caught, you may have noticed this yourself when |
|
|
|
09:22.190 --> 09:27.020 |
|
I was doing this, but I was busy thinking about this while I was talking is that I think we might have |
|
|
|
09:27.020 --> 09:35.300 |
|
just seen a goof from the model that the price here it's saying, is $350 associated with this Samsung |
|
|
|
09:35.300 --> 09:36.230 |
|
Galaxy Watch. |
|
|
|
09:36.230 --> 09:45.530 |
|
But I think when this website came up, let's close this down that it says up to 350 off with trade |
|
|
|
09:45.530 --> 09:45.860 |
|
in. |
|
|
|
09:45.860 --> 09:48.830 |
|
If you have another device you can save up to 350. |
|
|
|
09:48.860 --> 09:55.280 |
|
So after I showed off about how frontier models are so incredibly good at this stuff, this is an example |
|
|
|
09:55.280 --> 10:01.850 |
|
of a mistake that the model has actually made, that it thinks that this is worth 350, when in fact |
|
|
|
10:01.850 --> 10:03.530 |
|
it's $350 off. |
|
|
|
10:03.530 --> 10:06.020 |
|
So I should eat my words. |
|
|
|
10:06.020 --> 10:07.940 |
|
And it did actually get that wrong. |
|
|
|
10:07.940 --> 10:09.620 |
|
And it's going to be interesting. |
|
|
|
10:09.650 --> 10:13.400 |
|
That is, it's useful to have learned that and to have seen that. |
|
|
|
10:13.670 --> 10:20.150 |
|
And I might try adding that in to the system prompt and see if we can improve on the performance of |
|
|
|
10:20.180 --> 10:20.630 |
|
that. |
|
|
|
10:20.630 --> 10:23.360 |
|
Now that we see that it can make those mistakes. |
|
|
|
10:23.360 --> 10:27.380 |
|
Anyway, I will see you next time for the slides.
|
|
|