WEBVTT

00:01.040 --> 00:04.340
Well, thank you for coming along for week two, day four.

00:04.370 --> 00:06.920
We have lots of good stuff in store today.

00:06.920 --> 00:15.770
It's another day of levelling up, of building new skills that adds to your capabilities of using Llms

00:15.770 --> 00:19.490
for generating important business value.

00:19.940 --> 00:26.240
As always, a quick recap of what you can do already describing Transformers and the terminology involved.

00:26.240 --> 00:27.290
You know it well.

00:27.290 --> 00:33.920
Confidently coding with the APIs for the top three frontier models, and now most recently, building

00:33.920 --> 00:37.670
a chatbot assistant, an AI chatbot including an interactive UI.

00:37.670 --> 00:43.400
And you're very familiar now with that messages structure going into OpenAI and with the way that the

00:43.400 --> 00:46.130
chat function works for Gradio.

00:46.310 --> 00:49.880
So today is about these things called tools.

00:49.880 --> 00:54.080
By the end, you'll be able to define them, you'll have common use cases for them, and you'll be able

00:54.080 --> 00:57.650
to code an AI assistant that uses tools.

00:57.650 --> 00:58.910
Let's get to it.

00:59.570 --> 01:02.420
So what are tools?

01:02.900 --> 01:11.300
So it allows frontier models to connect with external functions with functionality outside the frontier

01:11.330 --> 01:11.780
model.

01:11.810 --> 01:14.570
In fact, tools can mean something broader than that.

01:14.570 --> 01:15.650
It can be other things too.

01:15.680 --> 01:21.110
But most commonly, when you hear people talk about tools, it's in the context of giving frontier models

01:21.140 --> 01:23.870
access to external functions.

01:23.870 --> 01:32.300
It allows for a richer replies from an LLM by extending its knowledge, um, it can carry out advanced

01:32.330 --> 01:39.380
actions within your application, and it can enhance its abilities by, for example, giving it a calculator.

01:39.530 --> 01:44.510
So as I said at the end of the last time, this might sound very mysterious.

01:44.510 --> 01:47.090
How exactly what exactly is going on here?

01:47.090 --> 01:52.430
We're going to build something like a calculator, like a function that can do calculations, uh, even

01:52.430 --> 01:54.860
go as far as to do a sort of exact of Python code.

01:54.860 --> 01:59.960
And then we're going to sort of give that to the LLM and say, okay, you can use this.

01:59.960 --> 02:03.140
You can run this software on my computer in some way.

02:03.140 --> 02:05.210
It sounds sounds mysterious.

02:05.210 --> 02:06.860
It sounds a bit spooky, really.

02:06.920 --> 02:09.790
Uh, but alas, it is not.

02:09.790 --> 02:10.930
Not that clever.

02:10.930 --> 02:13.840
It's a pretty simple workflow around it.

02:13.870 --> 02:15.130
Here's the scoop.

02:15.130 --> 02:24.220
What we do is we start by defining what functions we have available that the LM is allowed to call.

02:24.220 --> 02:25.900
So we define these functions.

02:25.900 --> 02:27.100
Let's say we have a calculator.

02:27.100 --> 02:28.390
We define the calculator.

02:28.390 --> 02:32.920
We say what are the inputs, what kind of outputs and when should the LM use it.

02:33.010 --> 02:36.550
And then we tell the LM about that.

02:36.550 --> 02:42.430
When we make a call to do something we say to it, hey, can you can you respond to this user.

02:42.430 --> 02:45.760
And by the way, you have access to this tool.

02:45.790 --> 02:51.970
When the LM replies to us, it can either just respond with a prompt or it can respond with something

02:51.970 --> 02:57.490
like, hey, if I'm going to to to generate you a response, first I'm going to need to ask you to run

02:57.490 --> 03:03.310
that tool you told me about and run it with these inputs and then provide me back with the outputs.

03:03.310 --> 03:12.280
And then you take that, you run the tool and then you provide the responses back to the LM, and it

03:12.280 --> 03:14.770
then uses it to generate its response.

03:14.860 --> 03:20.140
So if you follow my drift there, it's not actually particularly amazing.

03:20.140 --> 03:24.850
It's that you call an LM and it responds and says, hey, I need you to call the tool that you told

03:24.850 --> 03:25.570
me you have.

03:25.600 --> 03:31.630
You do that, you provide it back to the LM, and then it's able to give you richer responses.

03:32.320 --> 03:38.620
And if you're really following along, you'll realize that that's not massively different to the kind

03:38.650 --> 03:45.970
of thing we did in the last lab when we just looked for a string and we just inserted extra context

03:45.970 --> 03:52.330
in the prompt that goes to the LM, it's just about really inserting extra context in prompts.

03:52.360 --> 03:52.930
All right.

03:52.960 --> 03:56.860
Anyway, hopefully I didn't muddle you there, but it's going to come together when you see the code,

03:56.860 --> 03:57.820
I promise you.

03:58.630 --> 04:00.550
But first, what are the use cases.

04:00.550 --> 04:02.500
When when do we typically do this.

04:02.500 --> 04:07.150
There are four ones that really that you come across a lot.

04:07.330 --> 04:15.100
Um, you can use tools to fetch extra data, like look something up in a database, um, add knowledge.

04:15.100 --> 04:19.720
Uh, and again, you can think of it that's rather similar to what we did with with belts in the last

04:19.720 --> 04:23.800
lab, but you can do that using tools instead.

04:24.370 --> 04:30.940
Uh, you can use it as a way that the LM can take an action, like booking a meeting, so you can tell

04:30.970 --> 04:34.120
it as part of your, uh, you have access.

04:34.120 --> 04:40.240
You have the ability to actually, uh, to carry out these, these, these items to buy a plane ticket

04:40.240 --> 04:41.710
to do, do the following.

04:41.860 --> 04:47.050
Um, and essentially in its response back, it will tell you that that's the tool that wants to use,

04:48.580 --> 04:51.880
as I just mentioned, a use case would be a calculator.

04:51.880 --> 04:58.510
Uh, LMS are famously not great at calculations because all they're trying to do is predict, uh, tokens

04:58.510 --> 04:59.530
in English language.

04:59.530 --> 05:04.360
They don't have, like, a calculator built in to a to a deep neural network.

05:04.360 --> 05:07.090
But you can provide that as a tool.

05:07.270 --> 05:13.240
And you can notice that, uh, GPT four is very good at calculations these days.

05:13.240 --> 05:17.800
And one wonders whether something that's going on behind the scenes might be something like this, that

05:17.800 --> 05:22.020
it might have its own tool made available in order to run calculations.

05:22.020 --> 05:26.010
Perhaps just speculation, but it seems very reasonable.

05:27.090 --> 05:34.260
Another thing it can do is modify the UI so you could tell it, hey, here's some tools.

05:34.260 --> 05:39.390
You can use, some functions you can call that will update different things on my user interface.

05:39.390 --> 05:46.980
And that would give the LLM the direct ability to trigger changes in the UI, which is a pretty cool

05:46.980 --> 05:51.600
idea to have sort of tighter integration between the LLM and the UI.

05:52.740 --> 06:00.090
Again, one thing worth pointing out for the second one here, and for the fourth one for taking actions

06:00.090 --> 06:03.660
and modifying the UI, there will be another way to achieve this.

06:03.660 --> 06:09.090
That would be perhaps a simpler approach if that's all you wanted to do.

06:09.210 --> 06:14.460
See if you can, based on something we've already done before, uh, give you a moment to pause, to

06:14.490 --> 06:16.470
think about what I might be getting at.

06:17.070 --> 06:24.740
The answer is, you remember, uh, in one of the earlier labs we had the model respond in JSON to respond

06:24.740 --> 06:29.900
with a structured response, and its response had JSON to tell us bits of information.

06:29.900 --> 06:35.180
In our case, it was about links and uh, giving us more information about fully qualified links and

06:35.180 --> 06:36.440
which links to collect.

06:36.470 --> 06:41.900
Well, similarly, we could just ask the model to respond in JSON with what actions need to be taken

06:41.900 --> 06:46.790
to book a meeting or respond in JSON based on how it wants the user interface modified.

06:46.790 --> 06:50.690
So there are other ways other than using tools to accomplish this.

06:50.690 --> 06:55.430
But if you want to be able to give it tools in addition to streaming back text, then this is a good

06:55.430 --> 06:55.970
solution.

06:55.970 --> 07:00.740
That's the that's the best time to use this when it's in conjunction with a number of other things that

07:00.740 --> 07:01.730
the LM is doing.

07:01.730 --> 07:05.600
So these tools are sort of adding to its capabilities.

07:06.800 --> 07:13.250
So what we're going to do now is build an informed airline customer support agent.

07:13.250 --> 07:19.190
We're going to want to be able to tell it that we're traveling to Paris and then have it respond with

07:19.190 --> 07:21.170
a ticket price to Paris.

07:21.170 --> 07:22.310
That's the idea.

07:22.310 --> 07:26.960
We're going to do it with tools, and I will see you over in the lab to find out how.