From the uDemy course on LLM engineering.
https://www.udemy.com/course/llm-engineering-master-ai-and-large-language-models
You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
166 lines
5.2 KiB
166 lines
5.2 KiB
WEBVTT |
|
|
|
00:00.560 --> 00:06.710 |
|
It is terrific that you're hanging on in there and making such great progress with this course. |
|
|
|
00:06.710 --> 00:17.180 |
|
As we enter the final few days of building really important expertise on the topic for today, for week |
|
|
|
00:17.210 --> 00:17.630 |
|
eight. |
|
|
|
00:17.660 --> 00:25.910 |
|
Day three is about continuing to strengthen, to upskill, building one more skill, but also resummarize |
|
|
|
00:25.970 --> 00:28.700 |
|
revise some of the stuff that we've done in the past. |
|
|
|
00:28.700 --> 00:30.620 |
|
So today we're going to look at something new. |
|
|
|
00:30.650 --> 00:35.630 |
|
Structured outputs, which is something actually that is quite a recent innovation, a way that you |
|
|
|
00:35.630 --> 00:41.000 |
|
can tell frontier models that you want them to respond according to a particular specification. |
|
|
|
00:41.000 --> 00:42.500 |
|
So we'll be doing that today. |
|
|
|
00:42.500 --> 00:48.950 |
|
And we're also just going to be doing more work experience with frontier models, carrying out something |
|
|
|
00:48.950 --> 00:54.140 |
|
which is going to be a throwback, a callback to the first week, because we're going to be doing some |
|
|
|
00:54.140 --> 01:01.040 |
|
internet scrappage, as we had done in the past, and using frontier models to help synthesize data. |
|
|
|
01:01.070 --> 01:04.220 |
|
You remember we wrote a summarizer A way back. |
|
|
|
01:04.250 --> 01:07.610 |
|
The Reader's Digest of the internet was our week. |
|
|
|
01:07.610 --> 01:10.310 |
|
One day, one instant gratification. |
|
|
|
01:10.460 --> 01:13.070 |
|
Well, we're taking that a couple of notches further. |
|
|
|
01:13.400 --> 01:15.980 |
|
Um, so that's the plan for today. |
|
|
|
01:15.980 --> 01:21.740 |
|
And a lot of this, again, is going to be about revising and building and experimenting. |
|
|
|
01:21.980 --> 01:26.270 |
|
So let me just say a few words about structured outputs. |
|
|
|
01:26.300 --> 01:27.590 |
|
So structured outputs. |
|
|
|
01:27.590 --> 01:30.710 |
|
You remember in the past we've used JSON generation. |
|
|
|
01:30.710 --> 01:35.960 |
|
We've we've said that we want the model to respond with an output format in JSON. |
|
|
|
01:35.960 --> 01:40.070 |
|
And then in the prompt we describe exactly what that JSON should look like. |
|
|
|
01:40.070 --> 01:43.070 |
|
And it's not 100% reliable. |
|
|
|
01:43.100 --> 01:44.600 |
|
It's actually very good. |
|
|
|
01:44.600 --> 01:50.270 |
|
It's it's uh, frequently, uh, if not almost all the time will respond with JSON. |
|
|
|
01:50.540 --> 01:55.580 |
|
But where it starts to, to go wonky is if you've got really complicated objects that you need it to |
|
|
|
01:55.610 --> 02:03.680 |
|
respond in, um, it will after a while, potentially hallucinate in some parts or give back wrong formats. |
|
|
|
02:03.680 --> 02:10.620 |
|
So the idea of structured outputs was to be more directive about specifying exactly how the model should |
|
|
|
02:10.620 --> 02:11.490 |
|
respond. |
|
|
|
02:11.490 --> 02:19.500 |
|
And the way you do that is you define the response with a class, with a Python class, and it's actually |
|
|
|
02:19.500 --> 02:24.510 |
|
going to be a class which is going to be a subclass of something called base model from Pydantic, which |
|
|
|
02:24.510 --> 02:28.800 |
|
you may have already had experience with if you're from an engineering background, but don't worry. |
|
|
|
02:28.800 --> 02:30.240 |
|
If not, I will show you. |
|
|
|
02:30.510 --> 02:36.990 |
|
Uh, you make a subclass of base model, and you use that to describe exactly what you're looking for. |
|
|
|
02:37.110 --> 02:46.170 |
|
Um, and then you specify that class when you call OpenAI, and it will create an instance of that class |
|
|
|
02:46.170 --> 02:47.970 |
|
in what it sends back to you. |
|
|
|
02:48.480 --> 02:54.870 |
|
Um, and so that's the, the idea, it's useful, as I say, for generating data in precisely a structure |
|
|
|
02:54.870 --> 02:55.980 |
|
that you need. |
|
|
|
02:56.010 --> 03:03.210 |
|
It needs to be compared with an alternative approach, which is use of tools function calling that we |
|
|
|
03:03.210 --> 03:05.580 |
|
looked at again some time ago. |
|
|
|
03:05.760 --> 03:11.980 |
|
Um, and they, they, they both they're quite similar techniques for ensuring that a particular, um, |
|
|
|
03:11.980 --> 03:16.840 |
|
uh, type of, of structure comes back in the response from the model. |
|
|
|
03:16.840 --> 03:19.060 |
|
And there's pros and cons of both of them. |
|
|
|
03:19.390 --> 03:26.020 |
|
Generally speaking, the recommendation is that if you are going to be hooking up your model directly |
|
|
|
03:26.020 --> 03:31.930 |
|
to application code so that it's going to be making calls to functions which need to have a particular |
|
|
|
03:31.930 --> 03:35.980 |
|
method signature, then it's better to use function calling and tools. |
|
|
|
03:35.980 --> 03:41.800 |
|
That is the the better model, because then it will absolutely respond according to that JSON structure |
|
|
|
03:41.800 --> 03:46.390 |
|
that you've defined with the right parameters for calling your function. |
|
|
|
03:46.420 --> 03:53.140 |
|
If what you're looking at is to try and generate data in a particular format for downstream consumption |
|
|
|
03:53.140 --> 03:56.860 |
|
or something, uh, then structured outputs is the way. |
|
|
|
03:56.980 --> 04:01.900 |
|
Uh, so those are some of the pros and cons, and it's something that you get a feel for after you've |
|
|
|
04:01.930 --> 04:07.060 |
|
tried it for a while and get to appreciate when one performs better than the other. |
|
|
|
04:07.150 --> 04:10.480 |
|
But with that introduction, let's head over to JupyterLab. |
|
|
|
04:10.480 --> 04:12.700 |
|
We're going to try it out for ourselves.
|
|
|