From the uDemy course on LLM engineering.
https://www.udemy.com/course/llm-engineering-master-ai-and-large-language-models
You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
313 lines
9.2 KiB
313 lines
9.2 KiB
WEBVTT |
|
|
|
00:01.040 --> 00:04.340 |
|
Well, thank you for coming along for week two, day four. |
|
|
|
00:04.370 --> 00:06.920 |
|
We have lots of good stuff in store today. |
|
|
|
00:06.920 --> 00:15.770 |
|
It's another day of levelling up, of building new skills that adds to your capabilities of using Llms |
|
|
|
00:15.770 --> 00:19.490 |
|
for generating important business value. |
|
|
|
00:19.940 --> 00:26.240 |
|
As always, a quick recap of what you can do already describing Transformers and the terminology involved. |
|
|
|
00:26.240 --> 00:27.290 |
|
You know it well. |
|
|
|
00:27.290 --> 00:33.920 |
|
Confidently coding with the APIs for the top three frontier models, and now most recently, building |
|
|
|
00:33.920 --> 00:37.670 |
|
a chatbot assistant, an AI chatbot including an interactive UI. |
|
|
|
00:37.670 --> 00:43.400 |
|
And you're very familiar now with that messages structure going into OpenAI and with the way that the |
|
|
|
00:43.400 --> 00:46.130 |
|
chat function works for Gradio. |
|
|
|
00:46.310 --> 00:49.880 |
|
So today is about these things called tools. |
|
|
|
00:49.880 --> 00:54.080 |
|
By the end, you'll be able to define them, you'll have common use cases for them, and you'll be able |
|
|
|
00:54.080 --> 00:57.650 |
|
to code an AI assistant that uses tools. |
|
|
|
00:57.650 --> 00:58.910 |
|
Let's get to it. |
|
|
|
00:59.570 --> 01:02.420 |
|
So what are tools? |
|
|
|
01:02.900 --> 01:11.300 |
|
So it allows frontier models to connect with external functions with functionality outside the frontier |
|
|
|
01:11.330 --> 01:11.780 |
|
model. |
|
|
|
01:11.810 --> 01:14.570 |
|
In fact, tools can mean something broader than that. |
|
|
|
01:14.570 --> 01:15.650 |
|
It can be other things too. |
|
|
|
01:15.680 --> 01:21.110 |
|
But most commonly, when you hear people talk about tools, it's in the context of giving frontier models |
|
|
|
01:21.140 --> 01:23.870 |
|
access to external functions. |
|
|
|
01:23.870 --> 01:32.300 |
|
It allows for a richer replies from an LLM by extending its knowledge, um, it can carry out advanced |
|
|
|
01:32.330 --> 01:39.380 |
|
actions within your application, and it can enhance its abilities by, for example, giving it a calculator. |
|
|
|
01:39.530 --> 01:44.510 |
|
So as I said at the end of the last time, this might sound very mysterious. |
|
|
|
01:44.510 --> 01:47.090 |
|
How exactly what exactly is going on here? |
|
|
|
01:47.090 --> 01:52.430 |
|
We're going to build something like a calculator, like a function that can do calculations, uh, even |
|
|
|
01:52.430 --> 01:54.860 |
|
go as far as to do a sort of exact of Python code. |
|
|
|
01:54.860 --> 01:59.960 |
|
And then we're going to sort of give that to the LLM and say, okay, you can use this. |
|
|
|
01:59.960 --> 02:03.140 |
|
You can run this software on my computer in some way. |
|
|
|
02:03.140 --> 02:05.210 |
|
It sounds sounds mysterious. |
|
|
|
02:05.210 --> 02:06.860 |
|
It sounds a bit spooky, really. |
|
|
|
02:06.920 --> 02:09.790 |
|
Uh, but alas, it is not. |
|
|
|
02:09.790 --> 02:10.930 |
|
Not that clever. |
|
|
|
02:10.930 --> 02:13.840 |
|
It's a pretty simple workflow around it. |
|
|
|
02:13.870 --> 02:15.130 |
|
Here's the scoop. |
|
|
|
02:15.130 --> 02:24.220 |
|
What we do is we start by defining what functions we have available that the LM is allowed to call. |
|
|
|
02:24.220 --> 02:25.900 |
|
So we define these functions. |
|
|
|
02:25.900 --> 02:27.100 |
|
Let's say we have a calculator. |
|
|
|
02:27.100 --> 02:28.390 |
|
We define the calculator. |
|
|
|
02:28.390 --> 02:32.920 |
|
We say what are the inputs, what kind of outputs and when should the LM use it. |
|
|
|
02:33.010 --> 02:36.550 |
|
And then we tell the LM about that. |
|
|
|
02:36.550 --> 02:42.430 |
|
When we make a call to do something we say to it, hey, can you can you respond to this user. |
|
|
|
02:42.430 --> 02:45.760 |
|
And by the way, you have access to this tool. |
|
|
|
02:45.790 --> 02:51.970 |
|
When the LM replies to us, it can either just respond with a prompt or it can respond with something |
|
|
|
02:51.970 --> 02:57.490 |
|
like, hey, if I'm going to to to generate you a response, first I'm going to need to ask you to run |
|
|
|
02:57.490 --> 03:03.310 |
|
that tool you told me about and run it with these inputs and then provide me back with the outputs. |
|
|
|
03:03.310 --> 03:12.280 |
|
And then you take that, you run the tool and then you provide the responses back to the LM, and it |
|
|
|
03:12.280 --> 03:14.770 |
|
then uses it to generate its response. |
|
|
|
03:14.860 --> 03:20.140 |
|
So if you follow my drift there, it's not actually particularly amazing. |
|
|
|
03:20.140 --> 03:24.850 |
|
It's that you call an LM and it responds and says, hey, I need you to call the tool that you told |
|
|
|
03:24.850 --> 03:25.570 |
|
me you have. |
|
|
|
03:25.600 --> 03:31.630 |
|
You do that, you provide it back to the LM, and then it's able to give you richer responses. |
|
|
|
03:32.320 --> 03:38.620 |
|
And if you're really following along, you'll realize that that's not massively different to the kind |
|
|
|
03:38.650 --> 03:45.970 |
|
of thing we did in the last lab when we just looked for a string and we just inserted extra context |
|
|
|
03:45.970 --> 03:52.330 |
|
in the prompt that goes to the LM, it's just about really inserting extra context in prompts. |
|
|
|
03:52.360 --> 03:52.930 |
|
All right. |
|
|
|
03:52.960 --> 03:56.860 |
|
Anyway, hopefully I didn't muddle you there, but it's going to come together when you see the code, |
|
|
|
03:56.860 --> 03:57.820 |
|
I promise you. |
|
|
|
03:58.630 --> 04:00.550 |
|
But first, what are the use cases. |
|
|
|
04:00.550 --> 04:02.500 |
|
When when do we typically do this. |
|
|
|
04:02.500 --> 04:07.150 |
|
There are four ones that really that you come across a lot. |
|
|
|
04:07.330 --> 04:15.100 |
|
Um, you can use tools to fetch extra data, like look something up in a database, um, add knowledge. |
|
|
|
04:15.100 --> 04:19.720 |
|
Uh, and again, you can think of it that's rather similar to what we did with with belts in the last |
|
|
|
04:19.720 --> 04:23.800 |
|
lab, but you can do that using tools instead. |
|
|
|
04:24.370 --> 04:30.940 |
|
Uh, you can use it as a way that the LM can take an action, like booking a meeting, so you can tell |
|
|
|
04:30.970 --> 04:34.120 |
|
it as part of your, uh, you have access. |
|
|
|
04:34.120 --> 04:40.240 |
|
You have the ability to actually, uh, to carry out these, these, these items to buy a plane ticket |
|
|
|
04:40.240 --> 04:41.710 |
|
to do, do the following. |
|
|
|
04:41.860 --> 04:47.050 |
|
Um, and essentially in its response back, it will tell you that that's the tool that wants to use, |
|
|
|
04:48.580 --> 04:51.880 |
|
as I just mentioned, a use case would be a calculator. |
|
|
|
04:51.880 --> 04:58.510 |
|
Uh, LMS are famously not great at calculations because all they're trying to do is predict, uh, tokens |
|
|
|
04:58.510 --> 04:59.530 |
|
in English language. |
|
|
|
04:59.530 --> 05:04.360 |
|
They don't have, like, a calculator built in to a to a deep neural network. |
|
|
|
05:04.360 --> 05:07.090 |
|
But you can provide that as a tool. |
|
|
|
05:07.270 --> 05:13.240 |
|
And you can notice that, uh, GPT four is very good at calculations these days. |
|
|
|
05:13.240 --> 05:17.800 |
|
And one wonders whether something that's going on behind the scenes might be something like this, that |
|
|
|
05:17.800 --> 05:22.020 |
|
it might have its own tool made available in order to run calculations. |
|
|
|
05:22.020 --> 05:26.010 |
|
Perhaps just speculation, but it seems very reasonable. |
|
|
|
05:27.090 --> 05:34.260 |
|
Another thing it can do is modify the UI so you could tell it, hey, here's some tools. |
|
|
|
05:34.260 --> 05:39.390 |
|
You can use, some functions you can call that will update different things on my user interface. |
|
|
|
05:39.390 --> 05:46.980 |
|
And that would give the LLM the direct ability to trigger changes in the UI, which is a pretty cool |
|
|
|
05:46.980 --> 05:51.600 |
|
idea to have sort of tighter integration between the LLM and the UI. |
|
|
|
05:52.740 --> 06:00.090 |
|
Again, one thing worth pointing out for the second one here, and for the fourth one for taking actions |
|
|
|
06:00.090 --> 06:03.660 |
|
and modifying the UI, there will be another way to achieve this. |
|
|
|
06:03.660 --> 06:09.090 |
|
That would be perhaps a simpler approach if that's all you wanted to do. |
|
|
|
06:09.210 --> 06:14.460 |
|
See if you can, based on something we've already done before, uh, give you a moment to pause, to |
|
|
|
06:14.490 --> 06:16.470 |
|
think about what I might be getting at. |
|
|
|
06:17.070 --> 06:24.740 |
|
The answer is, you remember, uh, in one of the earlier labs we had the model respond in JSON to respond |
|
|
|
06:24.740 --> 06:29.900 |
|
with a structured response, and its response had JSON to tell us bits of information. |
|
|
|
06:29.900 --> 06:35.180 |
|
In our case, it was about links and uh, giving us more information about fully qualified links and |
|
|
|
06:35.180 --> 06:36.440 |
|
which links to collect. |
|
|
|
06:36.470 --> 06:41.900 |
|
Well, similarly, we could just ask the model to respond in JSON with what actions need to be taken |
|
|
|
06:41.900 --> 06:46.790 |
|
to book a meeting or respond in JSON based on how it wants the user interface modified. |
|
|
|
06:46.790 --> 06:50.690 |
|
So there are other ways other than using tools to accomplish this. |
|
|
|
06:50.690 --> 06:55.430 |
|
But if you want to be able to give it tools in addition to streaming back text, then this is a good |
|
|
|
06:55.430 --> 06:55.970 |
|
solution. |
|
|
|
06:55.970 --> 07:00.740 |
|
That's the that's the best time to use this when it's in conjunction with a number of other things that |
|
|
|
07:00.740 --> 07:01.730 |
|
the LM is doing. |
|
|
|
07:01.730 --> 07:05.600 |
|
So these tools are sort of adding to its capabilities. |
|
|
|
07:06.800 --> 07:13.250 |
|
So what we're going to do now is build an informed airline customer support agent. |
|
|
|
07:13.250 --> 07:19.190 |
|
We're going to want to be able to tell it that we're traveling to Paris and then have it respond with |
|
|
|
07:19.190 --> 07:21.170 |
|
a ticket price to Paris. |
|
|
|
07:21.170 --> 07:22.310 |
|
That's the idea. |
|
|
|
07:22.310 --> 07:26.960 |
|
We're going to do it with tools, and I will see you over in the lab to find out how.
|
|
|