You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 

313 lines
9.2 KiB

WEBVTT
00:01.040 --> 00:04.340
Well, thank you for coming along for week two, day four.
00:04.370 --> 00:06.920
We have lots of good stuff in store today.
00:06.920 --> 00:15.770
It's another day of levelling up, of building new skills that adds to your capabilities of using Llms
00:15.770 --> 00:19.490
for generating important business value.
00:19.940 --> 00:26.240
As always, a quick recap of what you can do already describing Transformers and the terminology involved.
00:26.240 --> 00:27.290
You know it well.
00:27.290 --> 00:33.920
Confidently coding with the APIs for the top three frontier models, and now most recently, building
00:33.920 --> 00:37.670
a chatbot assistant, an AI chatbot including an interactive UI.
00:37.670 --> 00:43.400
And you're very familiar now with that messages structure going into OpenAI and with the way that the
00:43.400 --> 00:46.130
chat function works for Gradio.
00:46.310 --> 00:49.880
So today is about these things called tools.
00:49.880 --> 00:54.080
By the end, you'll be able to define them, you'll have common use cases for them, and you'll be able
00:54.080 --> 00:57.650
to code an AI assistant that uses tools.
00:57.650 --> 00:58.910
Let's get to it.
00:59.570 --> 01:02.420
So what are tools?
01:02.900 --> 01:11.300
So it allows frontier models to connect with external functions with functionality outside the frontier
01:11.330 --> 01:11.780
model.
01:11.810 --> 01:14.570
In fact, tools can mean something broader than that.
01:14.570 --> 01:15.650
It can be other things too.
01:15.680 --> 01:21.110
But most commonly, when you hear people talk about tools, it's in the context of giving frontier models
01:21.140 --> 01:23.870
access to external functions.
01:23.870 --> 01:32.300
It allows for a richer replies from an LLM by extending its knowledge, um, it can carry out advanced
01:32.330 --> 01:39.380
actions within your application, and it can enhance its abilities by, for example, giving it a calculator.
01:39.530 --> 01:44.510
So as I said at the end of the last time, this might sound very mysterious.
01:44.510 --> 01:47.090
How exactly what exactly is going on here?
01:47.090 --> 01:52.430
We're going to build something like a calculator, like a function that can do calculations, uh, even
01:52.430 --> 01:54.860
go as far as to do a sort of exact of Python code.
01:54.860 --> 01:59.960
And then we're going to sort of give that to the LLM and say, okay, you can use this.
01:59.960 --> 02:03.140
You can run this software on my computer in some way.
02:03.140 --> 02:05.210
It sounds sounds mysterious.
02:05.210 --> 02:06.860
It sounds a bit spooky, really.
02:06.920 --> 02:09.790
Uh, but alas, it is not.
02:09.790 --> 02:10.930
Not that clever.
02:10.930 --> 02:13.840
It's a pretty simple workflow around it.
02:13.870 --> 02:15.130
Here's the scoop.
02:15.130 --> 02:24.220
What we do is we start by defining what functions we have available that the LM is allowed to call.
02:24.220 --> 02:25.900
So we define these functions.
02:25.900 --> 02:27.100
Let's say we have a calculator.
02:27.100 --> 02:28.390
We define the calculator.
02:28.390 --> 02:32.920
We say what are the inputs, what kind of outputs and when should the LM use it.
02:33.010 --> 02:36.550
And then we tell the LM about that.
02:36.550 --> 02:42.430
When we make a call to do something we say to it, hey, can you can you respond to this user.
02:42.430 --> 02:45.760
And by the way, you have access to this tool.
02:45.790 --> 02:51.970
When the LM replies to us, it can either just respond with a prompt or it can respond with something
02:51.970 --> 02:57.490
like, hey, if I'm going to to to generate you a response, first I'm going to need to ask you to run
02:57.490 --> 03:03.310
that tool you told me about and run it with these inputs and then provide me back with the outputs.
03:03.310 --> 03:12.280
And then you take that, you run the tool and then you provide the responses back to the LM, and it
03:12.280 --> 03:14.770
then uses it to generate its response.
03:14.860 --> 03:20.140
So if you follow my drift there, it's not actually particularly amazing.
03:20.140 --> 03:24.850
It's that you call an LM and it responds and says, hey, I need you to call the tool that you told
03:24.850 --> 03:25.570
me you have.
03:25.600 --> 03:31.630
You do that, you provide it back to the LM, and then it's able to give you richer responses.
03:32.320 --> 03:38.620
And if you're really following along, you'll realize that that's not massively different to the kind
03:38.650 --> 03:45.970
of thing we did in the last lab when we just looked for a string and we just inserted extra context
03:45.970 --> 03:52.330
in the prompt that goes to the LM, it's just about really inserting extra context in prompts.
03:52.360 --> 03:52.930
All right.
03:52.960 --> 03:56.860
Anyway, hopefully I didn't muddle you there, but it's going to come together when you see the code,
03:56.860 --> 03:57.820
I promise you.
03:58.630 --> 04:00.550
But first, what are the use cases.
04:00.550 --> 04:02.500
When when do we typically do this.
04:02.500 --> 04:07.150
There are four ones that really that you come across a lot.
04:07.330 --> 04:15.100
Um, you can use tools to fetch extra data, like look something up in a database, um, add knowledge.
04:15.100 --> 04:19.720
Uh, and again, you can think of it that's rather similar to what we did with with belts in the last
04:19.720 --> 04:23.800
lab, but you can do that using tools instead.
04:24.370 --> 04:30.940
Uh, you can use it as a way that the LM can take an action, like booking a meeting, so you can tell
04:30.970 --> 04:34.120
it as part of your, uh, you have access.
04:34.120 --> 04:40.240
You have the ability to actually, uh, to carry out these, these, these items to buy a plane ticket
04:40.240 --> 04:41.710
to do, do the following.
04:41.860 --> 04:47.050
Um, and essentially in its response back, it will tell you that that's the tool that wants to use,
04:48.580 --> 04:51.880
as I just mentioned, a use case would be a calculator.
04:51.880 --> 04:58.510
Uh, LMS are famously not great at calculations because all they're trying to do is predict, uh, tokens
04:58.510 --> 04:59.530
in English language.
04:59.530 --> 05:04.360
They don't have, like, a calculator built in to a to a deep neural network.
05:04.360 --> 05:07.090
But you can provide that as a tool.
05:07.270 --> 05:13.240
And you can notice that, uh, GPT four is very good at calculations these days.
05:13.240 --> 05:17.800
And one wonders whether something that's going on behind the scenes might be something like this, that
05:17.800 --> 05:22.020
it might have its own tool made available in order to run calculations.
05:22.020 --> 05:26.010
Perhaps just speculation, but it seems very reasonable.
05:27.090 --> 05:34.260
Another thing it can do is modify the UI so you could tell it, hey, here's some tools.
05:34.260 --> 05:39.390
You can use, some functions you can call that will update different things on my user interface.
05:39.390 --> 05:46.980
And that would give the LLM the direct ability to trigger changes in the UI, which is a pretty cool
05:46.980 --> 05:51.600
idea to have sort of tighter integration between the LLM and the UI.
05:52.740 --> 06:00.090
Again, one thing worth pointing out for the second one here, and for the fourth one for taking actions
06:00.090 --> 06:03.660
and modifying the UI, there will be another way to achieve this.
06:03.660 --> 06:09.090
That would be perhaps a simpler approach if that's all you wanted to do.
06:09.210 --> 06:14.460
See if you can, based on something we've already done before, uh, give you a moment to pause, to
06:14.490 --> 06:16.470
think about what I might be getting at.
06:17.070 --> 06:24.740
The answer is, you remember, uh, in one of the earlier labs we had the model respond in JSON to respond
06:24.740 --> 06:29.900
with a structured response, and its response had JSON to tell us bits of information.
06:29.900 --> 06:35.180
In our case, it was about links and uh, giving us more information about fully qualified links and
06:35.180 --> 06:36.440
which links to collect.
06:36.470 --> 06:41.900
Well, similarly, we could just ask the model to respond in JSON with what actions need to be taken
06:41.900 --> 06:46.790
to book a meeting or respond in JSON based on how it wants the user interface modified.
06:46.790 --> 06:50.690
So there are other ways other than using tools to accomplish this.
06:50.690 --> 06:55.430
But if you want to be able to give it tools in addition to streaming back text, then this is a good
06:55.430 --> 06:55.970
solution.
06:55.970 --> 07:00.740
That's the that's the best time to use this when it's in conjunction with a number of other things that
07:00.740 --> 07:01.730
the LM is doing.
07:01.730 --> 07:05.600
So these tools are sort of adding to its capabilities.
07:06.800 --> 07:13.250
So what we're going to do now is build an informed airline customer support agent.
07:13.250 --> 07:19.190
We're going to want to be able to tell it that we're traveling to Paris and then have it respond with
07:19.190 --> 07:21.170
a ticket price to Paris.
07:21.170 --> 07:22.310
That's the idea.
07:22.310 --> 07:26.960
We're going to do it with tools, and I will see you over in the lab to find out how.