From the uDemy course on LLM engineering.
https://www.udemy.com/course/llm-engineering-master-ai-and-large-language-models
You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
628 lines
17 KiB
628 lines
17 KiB
WEBVTT |
|
|
|
00:00.050 --> 00:03.890 |
|
It's time for our first LM experiment at this point. |
|
|
|
00:03.980 --> 00:08.330 |
|
So some of this you may know well, you may know very well already. |
|
|
|
00:08.360 --> 00:11.180 |
|
For some people this might be new, but let me just explain. |
|
|
|
00:11.240 --> 00:13.550 |
|
The models that we're going to be using. |
|
|
|
00:13.550 --> 00:17.090 |
|
These frontier models have been trained in a particular way. |
|
|
|
00:17.090 --> 00:23.090 |
|
That means that they expect two different types of instruction from us the user. |
|
|
|
00:23.120 --> 00:27.590 |
|
One of them is known as the system prompt, and one of them is known as the user prompt. |
|
|
|
00:27.620 --> 00:33.980 |
|
The system prompt is something which explains the context of this conversation. |
|
|
|
00:33.980 --> 00:39.440 |
|
It tells them what kind of task they're performing, what tone they should use, and we'll be experimenting |
|
|
|
00:39.440 --> 00:44.780 |
|
with what it means to to change a system prompt and what kind of information that you can include in |
|
|
|
00:44.780 --> 00:47.480 |
|
the system prompt throughout this course. |
|
|
|
00:47.570 --> 00:51.230 |
|
The user prompt is the actual conversation itself. |
|
|
|
00:51.230 --> 00:55.040 |
|
And in our case right now, it's going to just be the the conversation starter. |
|
|
|
00:55.040 --> 01:02.790 |
|
And the role of the LM of the large language model is to figure out what is the most likely way that |
|
|
|
01:02.790 --> 01:05.730 |
|
it should respond, given this user prompt. |
|
|
|
01:05.730 --> 01:11.580 |
|
If it's given this user prompt, and in the context of this system prompt, what is the most likely |
|
|
|
01:11.580 --> 01:13.410 |
|
next text that will come after it? |
|
|
|
01:13.410 --> 01:17.460 |
|
That would come from an assistant responding to this user. |
|
|
|
01:18.000 --> 01:22.950 |
|
So that's the difference between the system prompt that sets the context, the user prompt that is the |
|
|
|
01:22.950 --> 01:24.660 |
|
conversation starter. |
|
|
|
01:24.810 --> 01:26.790 |
|
So we're going to set a system prompt. |
|
|
|
01:26.790 --> 01:27.960 |
|
And this is what it's going to say. |
|
|
|
01:27.960 --> 01:33.870 |
|
It's going to say you are an assistant that analyzes the contents of a website and provides a short |
|
|
|
01:33.870 --> 01:37.590 |
|
summary, ignoring texts that might be navigation related. |
|
|
|
01:37.590 --> 01:39.630 |
|
Respond in markdown. |
|
|
|
01:39.660 --> 01:42.660 |
|
You'll see more of what that means in in just a second. |
|
|
|
01:42.660 --> 01:46.800 |
|
So that is our system prompt for the user prompt. |
|
|
|
01:46.800 --> 01:51.180 |
|
It's going to take as a we're going to write a function user prompt for. |
|
|
|
01:51.180 --> 01:54.990 |
|
And it's going to take a website as the argument to the function. |
|
|
|
01:54.990 --> 02:00.460 |
|
And it's going to say you are looking at a website titled The Website. |
|
|
|
02:01.000 --> 02:03.760 |
|
The contents of this website is as follows. |
|
|
|
02:03.760 --> 02:10.240 |
|
Please provide a short summary of the website in markdown if it includes news or announcements. |
|
|
|
02:10.240 --> 02:17.020 |
|
Summarize these two and we then take the text from the website object that Beautifulsoup plucked out |
|
|
|
02:17.020 --> 02:22.990 |
|
for us, and we add that into the user prompt and we return that user prompt. |
|
|
|
02:23.290 --> 02:28.870 |
|
So let's just quickly let's run that cell right now and let's just have a look now. |
|
|
|
02:28.870 --> 02:33.520 |
|
So after doing that, if I just look at what system Prompt has. |
|
|
|
02:35.560 --> 02:38.470 |
|
It has that text of course that we just said. |
|
|
|
02:38.650 --> 02:45.490 |
|
And now if you remember earlier on we created a new website object and we stored it in this variable |
|
|
|
02:45.490 --> 02:46.060 |
|
editor. |
|
|
|
02:46.060 --> 02:55.540 |
|
So if I come here I should be able to say user prompt for and then pass in the object Ed. |
|
|
|
02:56.260 --> 02:58.690 |
|
And what we'll get is a prompt. |
|
|
|
02:58.720 --> 03:03.340 |
|
It might be easier if I print this so that it prints out empty lines. |
|
|
|
03:05.440 --> 03:09.070 |
|
And here is the user prompt string that we've created. |
|
|
|
03:09.070 --> 03:11.440 |
|
It says you're looking at a website titled blah blah blah. |
|
|
|
03:11.470 --> 03:13.510 |
|
The contents of this website is as follows. |
|
|
|
03:13.510 --> 03:15.220 |
|
Please provide a short summary. |
|
|
|
03:15.250 --> 03:18.760 |
|
Look, it looks like we should have a space right here, otherwise it might be confusing. |
|
|
|
03:18.760 --> 03:19.810 |
|
Let's try that again. |
|
|
|
03:21.670 --> 03:25.960 |
|
That's always why it's worth printing things as you go, because you'll spot little inconsistencies |
|
|
|
03:25.960 --> 03:26.770 |
|
like that. |
|
|
|
03:28.000 --> 03:30.580 |
|
I think it'll be nicer, actually, now that I look at that. |
|
|
|
03:30.580 --> 03:34.540 |
|
If we have a carriage return there like so. |
|
|
|
03:38.080 --> 03:39.250 |
|
Let's have a look at this prompt. |
|
|
|
03:39.280 --> 03:45.280 |
|
Now you're looking at the website and there we go on a separate line that looks good okay. |
|
|
|
03:45.310 --> 03:49.780 |
|
So let's talk about the messages object. |
|
|
|
03:49.780 --> 03:55.800 |
|
So OpenAI expects to receive a conversation in a particular format. |
|
|
|
03:55.800 --> 04:01.920 |
|
It's a format that OpenAI came up with and they used for their APIs, and it became so well used that |
|
|
|
04:01.920 --> 04:07.200 |
|
all of the other major frontier models decided to adopt the same convention. |
|
|
|
04:07.200 --> 04:13.140 |
|
So this has gone from being originally OpenAI's way of using the API to being something of a standard |
|
|
|
04:13.140 --> 04:16.050 |
|
across many different models to use this approach. |
|
|
|
04:16.050 --> 04:17.520 |
|
And here's how it works. |
|
|
|
04:17.520 --> 04:25.590 |
|
When you're trying to describe a conversation, you describe it using a list a Python list of dictionaries. |
|
|
|
04:25.590 --> 04:29.640 |
|
So it's a list where each element in the list is a dictionary. |
|
|
|
04:29.640 --> 04:32.310 |
|
And that dictionary looks like this. |
|
|
|
04:32.340 --> 04:35.130 |
|
It's a dictionary with two elements. |
|
|
|
04:35.340 --> 04:43.170 |
|
One of them has a key of role, and here the value is either system or user, a key of role. |
|
|
|
04:43.170 --> 04:46.500 |
|
And the value is system a key of content. |
|
|
|
04:46.500 --> 04:49.770 |
|
And the value is of course the system message. |
|
|
|
04:49.800 --> 04:53.760 |
|
There's another Dictionary where there's a key of role. |
|
|
|
04:53.760 --> 04:56.700 |
|
The value is user because it's the user message. |
|
|
|
04:56.700 --> 05:01.680 |
|
The user prompt content is where the user message goes. |
|
|
|
05:02.130 --> 05:05.460 |
|
User message and user prompt are the same thing. |
|
|
|
05:06.120 --> 05:11.190 |
|
So hopefully I didn't explain it very well, but it makes sense when you see it visually like this. |
|
|
|
05:11.220 --> 05:18.120 |
|
It's just a dictionary which has role and content, system and system, message user and the user message. |
|
|
|
05:18.120 --> 05:21.630 |
|
And there are some other roles as well, but we're going to get to them in good time. |
|
|
|
05:21.630 --> 05:23.010 |
|
This is all we need for now. |
|
|
|
05:23.010 --> 05:26.130 |
|
So this is how messages are built. |
|
|
|
05:26.130 --> 05:33.300 |
|
And if you look at this next function def messages for hopefully it's super clear to you that this is |
|
|
|
05:33.300 --> 05:34.050 |
|
creating. |
|
|
|
05:34.050 --> 05:38.400 |
|
This here is creating exactly this construct using code. |
|
|
|
05:38.400 --> 05:42.000 |
|
It's going to do it's going to put in there the generic system prompt we came up with. |
|
|
|
05:42.000 --> 05:46.560 |
|
And it's going to create the user prompt for the website. |
|
|
|
05:46.980 --> 05:48.570 |
|
So let's run that. |
|
|
|
05:49.200 --> 05:58.220 |
|
And now, presumably it's clear that if I say messages for Ed, which is the object for my website, |
|
|
|
05:58.220 --> 06:02.300 |
|
let's print it so that we see empty lines and stuff. |
|
|
|
06:04.430 --> 06:07.370 |
|
Actually, sorry, in this case it might be better if we don't print it. |
|
|
|
06:07.640 --> 06:09.830 |
|
If we just do this, it might look a bit clearer. |
|
|
|
06:09.860 --> 06:10.370 |
|
There we go. |
|
|
|
06:10.400 --> 06:16.220 |
|
And now you can see that it is it's a list of two things role system. |
|
|
|
06:16.220 --> 06:18.170 |
|
And there's a system message role user. |
|
|
|
06:18.170 --> 06:20.270 |
|
And there is the user message. |
|
|
|
06:20.870 --> 06:21.710 |
|
Okay. |
|
|
|
06:21.710 --> 06:23.120 |
|
It's time to bring this together. |
|
|
|
06:23.120 --> 06:24.710 |
|
It's time to actually do it. |
|
|
|
06:24.710 --> 06:32.750 |
|
The API for OpenAI to make a call to a frontier model to do this for us is super simple, and we're |
|
|
|
06:32.750 --> 06:34.610 |
|
going to be using this API all the time. |
|
|
|
06:34.610 --> 06:37.700 |
|
So whereas now it might look like it's a few things to remember. |
|
|
|
06:37.700 --> 06:42.620 |
|
You're going to get so used to this, but we're going to make a function called summarize. |
|
|
|
06:42.620 --> 06:48.020 |
|
And that is that's going to do the business that's going to solve our problem and summarize a URL that's |
|
|
|
06:48.020 --> 06:48.860 |
|
passed in. |
|
|
|
06:48.860 --> 06:53.720 |
|
It will first create a website for that URL, just like we did for editor. |
|
|
|
06:53.720 --> 06:56.300 |
|
And this is where we call OpenAI. |
|
|
|
06:56.840 --> 06:59.840 |
|
We say OpenAI, which is the the OpenAI object. |
|
|
|
06:59.870 --> 07:05.480 |
|
We created OpenAI dot chat, dot completions, dot create. |
|
|
|
07:05.930 --> 07:08.390 |
|
And that for now you can just learn it by rote. |
|
|
|
07:08.390 --> 07:10.520 |
|
We'll understand a lot more about that later. |
|
|
|
07:10.820 --> 07:16.070 |
|
But as far as OpenAI is concerned, this is known as the completions API because we're asking it to |
|
|
|
07:16.100 --> 07:20.750 |
|
complete this conversation, predict what would be most likely to come next. |
|
|
|
07:20.900 --> 07:23.780 |
|
We pass in the name of the model we're going to use. |
|
|
|
07:23.780 --> 07:27.920 |
|
We're going to use a model called GPT four mini that you'll get very familiar with. |
|
|
|
07:27.920 --> 07:36.410 |
|
It is the light, cheap version of GPT four, the the one of the finest models on the planet, and this |
|
|
|
07:36.410 --> 07:39.350 |
|
will cost fractions of a cent to use. |
|
|
|
07:39.350 --> 07:45.620 |
|
This, um, you pass in the model and then you pass in the messages and the messages we pass in, use |
|
|
|
07:45.620 --> 07:49.910 |
|
this structure that we've just created and that is all it takes. |
|
|
|
07:50.660 --> 07:54.380 |
|
What comes back we put in this this object response. |
|
|
|
07:54.500 --> 08:01.910 |
|
And when we get back the response we call response dot choices zero dot message dot content. |
|
|
|
08:02.090 --> 08:05.270 |
|
Now I'm going to explain what this is another day we don't need to know. |
|
|
|
08:05.270 --> 08:05.810 |
|
For now. |
|
|
|
08:05.810 --> 08:10.820 |
|
We just need to know that we're going to do response dot choices zero dot message dot content. |
|
|
|
08:10.820 --> 08:11.930 |
|
That's going to be it. |
|
|
|
08:11.930 --> 08:14.330 |
|
That is our summarize function. |
|
|
|
08:14.330 --> 08:19.400 |
|
And with that let's try summarizing my website we're running. |
|
|
|
08:19.400 --> 08:22.070 |
|
It's now connecting to OpenAI in the cloud. |
|
|
|
08:22.070 --> 08:24.080 |
|
It's making the call and back. |
|
|
|
08:24.080 --> 08:26.870 |
|
Here is a summary of my website. |
|
|
|
08:26.870 --> 08:34.130 |
|
We have just uh, spent a fraction of a cent and we have just summarized my website. |
|
|
|
08:34.460 --> 08:39.110 |
|
We can do a little bit better because we can print this in a nice style. |
|
|
|
08:39.110 --> 08:44.300 |
|
Uh, GPT four, we've asked to respond in markdown, and that means that it's responded with various |
|
|
|
08:44.300 --> 08:48.910 |
|
characters to represent headings, things in bold and so on. |
|
|
|
08:49.270 --> 08:57.160 |
|
And we can use a feature of Jupyter Labs that we can ask it to actually show that in a nice markdown |
|
|
|
08:57.160 --> 08:57.880 |
|
format. |
|
|
|
08:57.910 --> 08:58.990 |
|
So let's do that. |
|
|
|
08:58.990 --> 09:02.440 |
|
Let's use this display summary function and try again. |
|
|
|
09:02.470 --> 09:05.770 |
|
Again we're going to GPT for a mini in the cloud. |
|
|
|
09:05.770 --> 09:08.800 |
|
And here is a summary of my website. |
|
|
|
09:08.980 --> 09:11.800 |
|
Uh, it says something about me. |
|
|
|
09:12.070 --> 09:17.350 |
|
Uh, and it's uh yeah, very nicely formatted, very nicely structured. |
|
|
|
09:17.350 --> 09:18.760 |
|
Pretty impressive. |
|
|
|
09:19.360 --> 09:24.970 |
|
And apparently it highlights my work with proprietary LMS, offers resources related to AI and LMS, |
|
|
|
09:25.150 --> 09:28.840 |
|
showcasing his commitment to advancing knowledge in this field. |
|
|
|
09:28.870 --> 09:30.550 |
|
Good for you, GPT for mini. |
|
|
|
09:30.580 --> 09:32.620 |
|
That's a very nice summary. |
|
|
|
09:33.070 --> 09:33.910 |
|
Okay. |
|
|
|
09:33.910 --> 09:36.490 |
|
And now we can try some more websites. |
|
|
|
09:36.520 --> 09:39.490 |
|
Let's try summarizing cnn.com. |
|
|
|
09:39.760 --> 09:42.490 |
|
Uh, we'll see what this happens. |
|
|
|
09:42.520 --> 09:47.470 |
|
Obviously, CNN is a much bigger, uh, result you've got here. |
|
|
|
09:47.650 --> 09:53.350 |
|
Uh, and, uh, we get some information about what's going on. |
|
|
|
09:53.440 --> 10:00.370 |
|
I'm actually recording this right now on the 5th of November at, uh, in the evening, which is the |
|
|
|
10:00.370 --> 10:02.800 |
|
date of the 2024 elections going on right now. |
|
|
|
10:02.800 --> 10:05.920 |
|
So that, of course, is featured on CNN's web page. |
|
|
|
10:05.920 --> 10:11.050 |
|
We can also summarize anthropic, which is the website for Claude. |
|
|
|
10:11.050 --> 10:12.940 |
|
And they have a nice page. |
|
|
|
10:12.940 --> 10:13.990 |
|
And here you go. |
|
|
|
10:13.990 --> 10:18.100 |
|
And you can read more about it in this nice little summary of their web page. |
|
|
|
10:19.030 --> 10:20.020 |
|
All right. |
|
|
|
10:20.020 --> 10:24.010 |
|
And that wraps up our first instant gratification. |
|
|
|
10:24.010 --> 10:25.750 |
|
It's it's juicy. |
|
|
|
10:25.750 --> 10:27.700 |
|
It's something where we've actually done something useful. |
|
|
|
10:27.700 --> 10:28.780 |
|
We've scraped the web. |
|
|
|
10:28.780 --> 10:33.640 |
|
We've summarized summarization is one of the most common AI use cases. |
|
|
|
10:33.640 --> 10:36.430 |
|
So common it's useful for all sorts of purposes. |
|
|
|
10:36.430 --> 10:41.850 |
|
We'll be doing it a few different ways during during this course, even in our week eight a sticky solution |
|
|
|
10:41.850 --> 10:44.160 |
|
will be using something that will do some summarization. |
|
|
|
10:44.160 --> 10:48.030 |
|
So it's a great, uh, thing to have experimented with already. |
|
|
|
10:48.360 --> 10:53.640 |
|
So there are so many other business applications of summarization. |
|
|
|
10:53.640 --> 10:56.340 |
|
This is something you should be able to put to good use. |
|
|
|
10:56.340 --> 11:01.020 |
|
You should be able to think of some ways you could apply this to your day job right away, or be building |
|
|
|
11:01.020 --> 11:05.190 |
|
a couple of example projects in GitHub that show summarization in action. |
|
|
|
11:05.190 --> 11:10.710 |
|
You could apply it to summarizing the news, summarizing financial performance from a financial report, |
|
|
|
11:10.740 --> 11:12.300 |
|
a resume, and a cover letter. |
|
|
|
11:12.300 --> 11:14.820 |
|
You could you could take a resume and generate a cover letter. |
|
|
|
11:14.970 --> 11:19.470 |
|
Uh, there are so many different things you can do with summarization of of documents. |
|
|
|
11:19.470 --> 11:23.220 |
|
And also adding on to that the scraping the web angle of it. |
|
|
|
11:23.220 --> 11:30.030 |
|
So have a think about how you would apply summarization to your business and try extending this to do |
|
|
|
11:30.030 --> 11:31.440 |
|
some summarization. |
|
|
|
11:32.280 --> 11:37.320 |
|
There's also uh, for for the more technically inclined, uh, one of the things that you'll discover |
|
|
|
11:37.350 --> 11:42.480 |
|
quite quickly when you use this is that there are many websites that cannot be summarized with this |
|
|
|
11:42.480 --> 11:49.560 |
|
approach, and that's because they use JavaScript to render the web page and are rather simplistic. |
|
|
|
11:49.560 --> 11:55.710 |
|
Approach has just taken the the just just made the requests the server call and taken what we get back. |
|
|
|
11:55.710 --> 11:56.970 |
|
But there's a solution. |
|
|
|
11:56.970 --> 12:03.450 |
|
And the solution is to use a platform like selenium or others like it, or playwright, which would |
|
|
|
12:03.450 --> 12:07.230 |
|
allow you to render the page and and do it that way. |
|
|
|
12:07.230 --> 12:12.720 |
|
So if you're technically inclined and have some background with that kind of thing, then a really interesting |
|
|
|
12:12.720 --> 12:19.200 |
|
challenge is to turn this into something that's a bit beefier and add selenium to the mix. |
|
|
|
12:19.380 --> 12:22.470 |
|
Um, as it happens, someone has already done that. |
|
|
|
12:22.470 --> 12:24.600 |
|
Uh, one of the students, thank you very much. |
|
|
|
12:24.690 --> 12:29.490 |
|
And if you go into this folder community contributions, you'll see a few different solutions. |
|
|
|
12:29.490 --> 12:32.520 |
|
And one of them is a selenium based solution. |
|
|
|
12:32.520 --> 12:34.860 |
|
So you can always go in and just just look at that yourself. |
|
|
|
12:34.860 --> 12:36.840 |
|
Or you can have a shot at doing it too. |
|
|
|
12:36.840 --> 12:39.380 |
|
And you'll find the solution in there. |
|
|
|
12:40.370 --> 12:44.900 |
|
And if you do come up with a solution to that or to anything, I would love it if you were willing to |
|
|
|
12:44.900 --> 12:47.420 |
|
share your code so that others can benefit from it. |
|
|
|
12:47.420 --> 12:53.240 |
|
Ideally, put it in the community contributions folder and be sure to clear the output. |
|
|
|
12:53.240 --> 12:58.190 |
|
So you go to kernel restart kernel and clear outputs of all cells. |
|
|
|
12:58.310 --> 13:03.170 |
|
Otherwise, everything that you've got in your output would also get checked into code which which would |
|
|
|
13:03.320 --> 13:04.730 |
|
just clutter things up a bit. |
|
|
|
13:04.730 --> 13:06.200 |
|
So so do that. |
|
|
|
13:06.470 --> 13:12.440 |
|
And then if you could submit a PR, a pull request, I can then merge that into the code. |
|
|
|
13:12.440 --> 13:16.550 |
|
And if that's a new thing for you, it is a bit of a process. |
|
|
|
13:16.580 --> 13:21.620 |
|
There is a write up here for exactly what you need to do to make that work. |
|
|
|
13:21.830 --> 13:25.700 |
|
Anyways, this was the first project, the first of many. |
|
|
|
13:25.730 --> 13:27.920 |
|
It's a simple project, but it's an important one. |
|
|
|
13:27.950 --> 13:29.960 |
|
A very important business use case. |
|
|
|
13:29.960 --> 13:31.610 |
|
I hope you found it worthwhile. |
|
|
|
13:31.610 --> 13:34.700 |
|
I will see you for the next video when we wrap up. |
|
|
|
13:34.700 --> 13:35.570 |
|
Week one. |
|
|
|
13:35.570 --> 13:36.200 |
|
Day one.
|
|
|