From the uDemy course on LLM engineering.
https://www.udemy.com/course/llm-engineering-master-ai-and-large-language-models
You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
529 lines
14 KiB
529 lines
14 KiB
WEBVTT |
|
|
|
00:00.680 --> 00:05.900 |
|
Let's keep going with our project to equip our LM with a tool. |
|
|
|
00:05.900 --> 00:12.200 |
|
We just created this piece of code to describe our function, which I'll now execute. |
|
|
|
00:12.200 --> 00:19.760 |
|
And then the next step is we just create something called tools, which now has a single it's a it's |
|
|
|
00:19.790 --> 00:20.930 |
|
a list of tools. |
|
|
|
00:20.960 --> 00:22.340 |
|
And in our case we only have one. |
|
|
|
00:22.340 --> 00:24.560 |
|
It's one element type is function. |
|
|
|
00:24.560 --> 00:28.460 |
|
And the function is this thing right here. |
|
|
|
00:28.460 --> 00:31.880 |
|
So that is now in a list that's called tools. |
|
|
|
00:32.840 --> 00:38.630 |
|
And now the moment it's time to give an LM in our case GPT. |
|
|
|
00:38.660 --> 00:41.570 |
|
For many the power to use this tool. |
|
|
|
00:41.570 --> 00:43.190 |
|
How is that going to work. |
|
|
|
00:43.190 --> 00:48.710 |
|
So it's a bit fiddly I gotta warn you, be be mentally prepared for something a bit fiddly. |
|
|
|
00:48.710 --> 00:51.980 |
|
It starts with something that will look very familiar to you. |
|
|
|
00:51.980 --> 00:54.680 |
|
This is our usual chat function. |
|
|
|
00:54.680 --> 00:55.760 |
|
You know it well. |
|
|
|
00:55.760 --> 01:03.290 |
|
It has a message in history that you know and then it uses that to craft the messages that go to OpenAI. |
|
|
|
01:03.320 --> 01:05.930 |
|
Now there is a small, subtle difference here. |
|
|
|
01:05.930 --> 01:06.950 |
|
Something has changed. |
|
|
|
01:06.950 --> 01:08.360 |
|
Something has been added. |
|
|
|
01:08.570 --> 01:10.880 |
|
I'm going to wait a moment for you to spot it. |
|
|
|
01:10.910 --> 01:15.860 |
|
Once you've spotted it, it's super obvious, but there is a little addition that's been slipped in |
|
|
|
01:15.890 --> 01:16.220 |
|
there. |
|
|
|
01:16.220 --> 01:19.070 |
|
And if you saw it, very nice. |
|
|
|
01:19.070 --> 01:20.720 |
|
If you didn't, it's very obvious. |
|
|
|
01:20.720 --> 01:21.500 |
|
Here it is. |
|
|
|
01:22.370 --> 01:23.870 |
|
We pass in the tools. |
|
|
|
01:23.870 --> 01:27.350 |
|
The API is exactly the same chat dot completions, dot create. |
|
|
|
01:27.380 --> 01:31.520 |
|
We give it a model, we give it the messages and then we also give it the tools. |
|
|
|
01:31.520 --> 01:34.970 |
|
In other words, we give it this object right here. |
|
|
|
01:35.270 --> 01:44.420 |
|
Uh, and when ChatGPT sorry when chat when OpenAI is taking that and building the prompt, the tokens |
|
|
|
01:44.420 --> 01:52.250 |
|
that get sent to the to the LLM, to the GPT four LLM, it's going to take this and convert it into |
|
|
|
01:52.250 --> 01:55.700 |
|
a series of tokens that are going to describe our function. |
|
|
|
01:55.700 --> 02:02.810 |
|
It's going to use this English to say what it does, and it's going to inform the LLM that this is a |
|
|
|
02:02.810 --> 02:05.180 |
|
function that it can call. |
|
|
|
02:05.210 --> 02:10.910 |
|
Uh, and the reason that works is because it's been trained with lots of examples that use tokens in |
|
|
|
02:10.940 --> 02:14.120 |
|
that way to give the LLM that ability. |
|
|
|
02:14.690 --> 02:21.080 |
|
And so the some of the magic is about to be lost when you see what's going to happen next. |
|
|
|
02:22.430 --> 02:28.430 |
|
What comes back from the LLM you remember is in response choices zero. |
|
|
|
02:28.430 --> 02:35.450 |
|
And what we do is we find out whether or not the finish reason is this thing tool calls. |
|
|
|
02:35.450 --> 02:41.300 |
|
That happens when GPT four is telling us, I don't have an answer for you yet. |
|
|
|
02:41.300 --> 02:48.410 |
|
Instead, I'm going to stop because I want you to call one of your tools and provide me with its output. |
|
|
|
02:48.470 --> 02:56.240 |
|
So if that happens, what we then need to do is collect the message from GPT four. |
|
|
|
02:56.450 --> 03:00.800 |
|
So this collects that response choice zero dot message. |
|
|
|
03:00.860 --> 03:07.970 |
|
It contains the whatever it's sent back, which will in fact be a request for us to run a tool. |
|
|
|
03:08.180 --> 03:13.390 |
|
And then there's a bit of work to do to unpack that message. |
|
|
|
03:13.450 --> 03:15.820 |
|
Figure out what it wants to do and do it. |
|
|
|
03:15.820 --> 03:17.710 |
|
And I've put all of that in a separate function. |
|
|
|
03:17.710 --> 03:18.370 |
|
I've cheated. |
|
|
|
03:18.370 --> 03:21.400 |
|
I could put it all in here, but it becomes quite messy. |
|
|
|
03:21.610 --> 03:23.350 |
|
So instead I've separated that out. |
|
|
|
03:23.350 --> 03:25.810 |
|
So at this point we'll see it in a second. |
|
|
|
03:25.810 --> 03:32.440 |
|
Believe me that this function will unpack the message back from GPT four zero and if necessary, well, |
|
|
|
03:32.440 --> 03:33.250 |
|
it will be necessary. |
|
|
|
03:33.250 --> 03:37.060 |
|
It will call our tool and it will return the result of that tool. |
|
|
|
03:37.060 --> 03:44.530 |
|
The response to go back to GPT four zero and the city that was called, uh, what we then do is we have |
|
|
|
03:44.530 --> 03:49.180 |
|
to add two more rows to our list of messages. |
|
|
|
03:49.210 --> 03:53.410 |
|
Our messages, you remember, have like user assistant, user assistant. |
|
|
|
03:53.410 --> 03:56.320 |
|
We're now going to add two new rows. |
|
|
|
03:56.320 --> 04:04.840 |
|
One of those rows is this thing message, which is nothing more than what we got back from GPT four |
|
|
|
04:04.870 --> 04:07.600 |
|
zero asking us to call a tool. |
|
|
|
04:07.600 --> 04:12.400 |
|
So the first thing we add in is the assistant asking us to run a tool. |
|
|
|
04:12.400 --> 04:19.840 |
|
We put that in the list of messages, and then after that, what we put next in the list of messages |
|
|
|
04:19.870 --> 04:23.740 |
|
is our result of calling the function. |
|
|
|
04:23.950 --> 04:32.620 |
|
So what's now in this list of messages is user assistant, user assistant, user assistant says run |
|
|
|
04:32.620 --> 04:33.340 |
|
tool. |
|
|
|
04:33.340 --> 04:35.710 |
|
We say this is the tool result. |
|
|
|
04:35.980 --> 04:40.420 |
|
And then that is what we're now sending back to OpenAI. |
|
|
|
04:40.450 --> 04:41.920 |
|
I hope that made sense. |
|
|
|
04:41.950 --> 04:46.990 |
|
If not, you simply put some print statements in here when you run it and you'll see it's exactly what |
|
|
|
04:46.990 --> 04:47.470 |
|
I said. |
|
|
|
04:47.470 --> 04:51.400 |
|
And once you print the whole messages thing, I think it will be crystal clear to you. |
|
|
|
04:51.430 --> 04:56.170 |
|
You'll see that full exchange appearing in those messages. |
|
|
|
04:57.160 --> 05:04.060 |
|
And then at the end here, as usual, we just return the ultimate answer from the LM. |
|
|
|
05:04.270 --> 05:09.670 |
|
Uh, what I should point out probably is that I've decided I don't pass in the tools a second time, |
|
|
|
05:09.670 --> 05:14.500 |
|
because we wouldn't expect it to run our tool twice, but there'd be no harm in putting it in there. |
|
|
|
05:14.500 --> 05:20.370 |
|
But it obviously wouldn't use it, So the final missing ingredient, of course, is that I now have |
|
|
|
05:20.370 --> 05:25.950 |
|
to write this function here, which is again a little bit more involved than than one might like it, |
|
|
|
05:25.950 --> 05:28.740 |
|
but you can just use it verbatim in your own projects. |
|
|
|
05:28.770 --> 05:30.780 |
|
It's just some stuff to know. |
|
|
|
05:30.930 --> 05:36.510 |
|
Um, so we've got this, this thing, this message that's come back from GPT four. |
|
|
|
05:36.930 --> 05:43.740 |
|
And what we have to do is unpack it to find out which tool was it wanting to call. |
|
|
|
05:43.770 --> 05:48.330 |
|
Now, in our case, we know what tool it wants to call because we only have one tool and it's the tool |
|
|
|
05:48.360 --> 05:49.380 |
|
to get prices. |
|
|
|
05:49.410 --> 05:53.370 |
|
Um, but I'm keeping this in here anyway so that you can see how it works. |
|
|
|
05:53.430 --> 05:59.850 |
|
Um, and what we should really do here is say if tool call equals, uh, get ticket price, then. |
|
|
|
05:59.880 --> 06:05.700 |
|
So there should really be a sort of like, like if or some sort of, uh, a series of, uh, like a, |
|
|
|
06:05.730 --> 06:10.290 |
|
like a, like a switch of all the possible, uh, things that it could ask to, to call. |
|
|
|
06:10.290 --> 06:12.330 |
|
But in our case, we know there's only one tool available. |
|
|
|
06:12.330 --> 06:13.560 |
|
So that's what it is. |
|
|
|
06:13.830 --> 06:18.570 |
|
What we can then do is load the arguments that it wants to call. |
|
|
|
06:18.570 --> 06:23.520 |
|
So in this this tool call there is a tool called dot function. |
|
|
|
06:23.550 --> 06:28.710 |
|
Dot arguments that tells us what parameters it's chosen. |
|
|
|
06:28.920 --> 06:31.770 |
|
Um, and that comes back in the form of JSON. |
|
|
|
06:31.770 --> 06:32.880 |
|
Uh, just a string. |
|
|
|
06:32.880 --> 06:38.670 |
|
So we have to use the Json.loads Loadstring function to convert that into a dictionary. |
|
|
|
06:38.670 --> 06:44.880 |
|
And then we look up the only actual argument that we have, which is destination city. |
|
|
|
06:44.910 --> 06:48.420 |
|
We look that up and put that in a variable city. |
|
|
|
06:48.420 --> 06:52.020 |
|
So we have now unpacked the tool and the argument. |
|
|
|
06:52.020 --> 06:56.190 |
|
And what remains to be done is this line here. |
|
|
|
06:56.190 --> 06:59.670 |
|
And that line obviously is nothing very clever. |
|
|
|
06:59.670 --> 07:04.890 |
|
That line is quite simply calling our get ticket price function. |
|
|
|
07:04.890 --> 07:10.890 |
|
We've established that's the tool it wants to run, and we've plucked out the city and we now call that |
|
|
|
07:10.890 --> 07:11.790 |
|
function. |
|
|
|
07:12.000 --> 07:14.340 |
|
Um, we then build a response. |
|
|
|
07:14.340 --> 07:15.990 |
|
And here is the response. |
|
|
|
07:15.990 --> 07:20.910 |
|
This is the thing that's going to get shoved at the bottom of the the messages. |
|
|
|
07:20.910 --> 07:21.420 |
|
There it goes. |
|
|
|
07:21.420 --> 07:24.920 |
|
It's going to get shoved in there and what their response looked like is this. |
|
|
|
07:24.950 --> 07:29.060 |
|
Now, you know, each of these rows has a role and content. |
|
|
|
07:29.390 --> 07:35.600 |
|
And in the past we've seen that role can be user or it can be system user or assistant system user assistant, |
|
|
|
07:35.600 --> 07:36.470 |
|
user assistant. |
|
|
|
07:36.470 --> 07:38.090 |
|
Well, there's something else it can be too. |
|
|
|
07:38.120 --> 07:39.590 |
|
It can be this word tool. |
|
|
|
07:39.590 --> 07:43.100 |
|
So in this case we put into the response tool. |
|
|
|
07:43.100 --> 07:47.630 |
|
And for the content we put in a string which is this. |
|
|
|
07:47.660 --> 07:52.280 |
|
This dictionary turned into a string using the JSON dump string function. |
|
|
|
07:52.280 --> 07:55.220 |
|
So we put in the destination city and the price. |
|
|
|
07:55.220 --> 08:04.670 |
|
We do also need to add this tool call ID into the the message, which is a way that it links this response |
|
|
|
08:04.670 --> 08:08.180 |
|
to the request that came right before it. |
|
|
|
08:08.180 --> 08:17.270 |
|
So we're putting message.tool.id into this guy so that when it sees these two, it fully understands |
|
|
|
08:17.270 --> 08:20.990 |
|
that that is associated with this request. |
|
|
|
08:20.990 --> 08:23.990 |
|
And that's all there is to it. |
|
|
|
08:24.260 --> 08:27.480 |
|
Although this time This time it was a fair amount. |
|
|
|
08:27.510 --> 08:29.370 |
|
There's quite a lot to take on board there. |
|
|
|
08:29.580 --> 08:34.710 |
|
But as I say, step through it, put some print statements, see that working. |
|
|
|
08:34.710 --> 08:38.070 |
|
But let's see now what happens if we bring this up in a chat. |
|
|
|
08:39.720 --> 08:40.800 |
|
Hi there. |
|
|
|
08:44.550 --> 08:45.270 |
|
Hello. |
|
|
|
08:45.270 --> 08:46.500 |
|
How can I assist you today? |
|
|
|
08:46.530 --> 08:49.920 |
|
I'd like to go to London. |
|
|
|
08:51.600 --> 08:52.170 |
|
Sure. |
|
|
|
08:52.170 --> 08:53.460 |
|
Would you like to know the price? |
|
|
|
08:53.490 --> 08:54.390 |
|
Yes. |
|
|
|
08:55.770 --> 08:59.520 |
|
The ticket price for a return trip to London is $7.99. |
|
|
|
08:59.520 --> 09:00.900 |
|
So it got the right price. |
|
|
|
09:00.900 --> 09:05.850 |
|
So what we're hoping is if we turn back to our Jupyter lab, what we should see is that it's printed |
|
|
|
09:05.850 --> 09:08.490 |
|
that our tool was called, uh. |
|
|
|
09:08.490 --> 09:11.370 |
|
And I think we can be pretty confident that it will, but let's have a look. |
|
|
|
09:11.400 --> 09:15.210 |
|
It is indeed tool get ticket price called for London. |
|
|
|
09:15.240 --> 09:16.740 |
|
Let's keep going. |
|
|
|
09:17.220 --> 09:22.620 |
|
Uh, and how about, uh, Paris? |
|
|
|
09:25.320 --> 09:27.870 |
|
899 and Tokyo. |
|
|
|
09:30.710 --> 09:34.130 |
|
The ticket price for a return trip to Tokyo is 1400. |
|
|
|
09:34.760 --> 09:35.930 |
|
And Berlin. |
|
|
|
09:37.280 --> 09:38.660 |
|
Berlin was the one we just added in. |
|
|
|
09:38.690 --> 09:39.530 |
|
Let's see if it's got that. |
|
|
|
09:39.560 --> 09:41.480 |
|
Yes, 499. |
|
|
|
09:41.510 --> 09:46.850 |
|
It certainly seems to work and timbuk2. |
|
|
|
09:48.710 --> 09:53.420 |
|
I'm sorry, but I don't have information on ticket prices to timbuk2. |
|
|
|
09:53.810 --> 09:55.520 |
|
Uh, let's have a look here. |
|
|
|
09:55.520 --> 09:58.940 |
|
And you can see the series of tools called. |
|
|
|
09:58.940 --> 10:02.720 |
|
And we know that when it was called for Timbuk2, it would replied unknown. |
|
|
|
10:02.720 --> 10:07.910 |
|
And as a result, because we gave it the system prompt to say when it doesn't know, it was quite clear |
|
|
|
10:07.910 --> 10:08.960 |
|
that it didn't know. |
|
|
|
10:09.680 --> 10:12.530 |
|
And with that, there was a lot going on. |
|
|
|
10:12.530 --> 10:13.910 |
|
I hope you got a sense. |
|
|
|
10:13.910 --> 10:18.560 |
|
This is in fact, a powerful, uh, piece of functionality. |
|
|
|
10:18.560 --> 10:27.650 |
|
It's a powerful technique to allow you to give more powers to your LM, but but under the covers, it's |
|
|
|
10:27.650 --> 10:28.580 |
|
not magical. |
|
|
|
10:28.580 --> 10:34.990 |
|
It's really a bunch of if statements and some complicated messages to and fro, so that we can allow |
|
|
|
10:34.990 --> 10:38.770 |
|
the LLM to inform us that it needs more information about something. |
|
|
|
10:38.770 --> 10:40.990 |
|
And that's how it works under the covers. |
|
|
|
10:40.990 --> 10:42.850 |
|
And I hope that makes sense to you. |
|
|
|
10:42.850 --> 10:45.610 |
|
And I hope you're able to use this in your own projects. |
|
|
|
10:45.610 --> 10:47.350 |
|
As ways to extend this. |
|
|
|
10:47.350 --> 10:54.340 |
|
You can look at adding more kinds of tools and tools that might say something about the availability |
|
|
|
10:54.340 --> 10:56.710 |
|
of that flight or something like that. |
|
|
|
10:56.710 --> 10:58.510 |
|
So you can use more tools. |
|
|
|
10:58.510 --> 11:03.640 |
|
Or if you want to be really bold, you can add a tool to actually book the flight. |
|
|
|
11:03.640 --> 11:07.120 |
|
It's a tool that, when it's called again, could just print something or could write something to a |
|
|
|
11:07.120 --> 11:08.170 |
|
file or whatever. |
|
|
|
11:08.170 --> 11:14.890 |
|
And that would allow the LLM to call back into your second tool and actually book a flight when the |
|
|
|
11:14.890 --> 11:16.390 |
|
user asks for it. |
|
|
|
11:16.390 --> 11:21.100 |
|
So that would be a fun one, and you'd have a bunch of arguments about the dates and the like. |
|
|
|
11:21.100 --> 11:28.810 |
|
So give that a shot, and by the end of that, you will be very proficient in how to write tools and |
|
|
|
11:28.810 --> 11:35.560 |
|
how to equip your LLM to carry out actions that run in your software.
|
|
|