Well, thank you for coming along for week two, day four.
We have lots of good stuff in store today.
It's another day of levelling up, of building new skills that adds to your capabilities of using Llms
for generating important business value.
As always, a quick recap of what you can do already describing Transformers and the terminology involved.
You know it well.
Confidently coding with the APIs for the top three frontier models, and now most recently, building
a chatbot assistant, an AI chatbot including an interactive UI.
And you're very familiar now with that messages structure going into OpenAI and with the way that the
chat function works for Gradio.
So today is about these things called tools.
By the end, you'll be able to define them, you'll have common use cases for them, and you'll be able
to code an AI assistant that uses tools.
Let's get to it.
So what are tools?
So it allows frontier models to connect with external functions with functionality outside the frontier
model.
In fact, tools can mean something broader than that.
It can be other things too.
But most commonly, when you hear people talk about tools, it's in the context of giving frontier models
access to external functions.
It allows for a richer replies from an LLM by extending its knowledge, um, it can carry out advanced
actions within your application, and it can enhance its abilities by, for example, giving it a calculator.
So as I said at the end of the last time, this might sound very mysterious.
How exactly what exactly is going on here?
We're going to build something like a calculator, like a function that can do calculations, uh, even
go as far as to do a sort of exact of Python code.
And then we're going to sort of give that to the LLM and say, okay, you can use this.
You can run this software on my computer in some way.
It sounds sounds mysterious.
It sounds a bit spooky, really.
Uh, but alas, it is not.
Not that clever.
It's a pretty simple workflow around it.
Here's the scoop.
What we do is we start by defining what functions we have available that the LM is allowed to call.
So we define these functions.
Let's say we have a calculator.
We define the calculator.
We say what are the inputs, what kind of outputs and when should the LM use it.
And then we tell the LM about that.
When we make a call to do something we say to it, hey, can you can you respond to this user.
And by the way, you have access to this tool.
When the LM replies to us, it can either just respond with a prompt or it can respond with something
like, hey, if I'm going to to to generate you a response, first I'm going to need to ask you to run
that tool you told me about and run it with these inputs and then provide me back with the outputs.
And then you take that, you run the tool and then you provide the responses back to the LM, and it
then uses it to generate its response.
So if you follow my drift there, it's not actually particularly amazing.
It's that you call an LM and it responds and says, hey, I need you to call the tool that you told
me you have.
You do that, you provide it back to the LM, and then it's able to give you richer responses.
And if you're really following along, you'll realize that that's not massively different to the kind
of thing we did in the last lab when we just looked for a string and we just inserted extra context
in the prompt that goes to the LM, it's just about really inserting extra context in prompts.
All right.
Anyway, hopefully I didn't muddle you there, but it's going to come together when you see the code,
I promise you.
But first, what are the use cases.
When when do we typically do this.
There are four ones that really that you come across a lot.
Um, you can use tools to fetch extra data, like look something up in a database, um, add knowledge.
Uh, and again, you can think of it that's rather similar to what we did with with belts in the last
lab, but you can do that using tools instead.
Uh, you can use it as a way that the LM can take an action, like booking a meeting, so you can tell
it as part of your, uh, you have access.
You have the ability to actually, uh, to carry out these, these, these items to buy a plane ticket
to do, do the following.
Um, and essentially in its response back, it will tell you that that's the tool that wants to use,
as I just mentioned, a use case would be a calculator.
Uh, LMS are famously not great at calculations because all they're trying to do is predict, uh, tokens
in English language.
They don't have, like, a calculator built in to a to a deep neural network.
But you can provide that as a tool.
And you can notice that, uh, GPT four is very good at calculations these days.
And one wonders whether something that's going on behind the scenes might be something like this, that
it might have its own tool made available in order to run calculations.
Perhaps just speculation, but it seems very reasonable.
Another thing it can do is modify the UI so you could tell it, hey, here's some tools.
You can use, some functions you can call that will update different things on my user interface.
And that would give the LLM the direct ability to trigger changes in the UI, which is a pretty cool
idea to have sort of tighter integration between the LLM and the UI.
Again, one thing worth pointing out for the second one here, and for the fourth one for taking actions
and modifying the UI, there will be another way to achieve this.
That would be perhaps a simpler approach if that's all you wanted to do.
See if you can, based on something we've already done before, uh, give you a moment to pause, to
think about what I might be getting at.
The answer is, you remember, uh, in one of the earlier labs we had the model respond in JSON to respond
with a structured response, and its response had JSON to tell us bits of information.
In our case, it was about links and uh, giving us more information about fully qualified links and
which links to collect.
Well, similarly, we could just ask the model to respond in JSON with what actions need to be taken
to book a meeting or respond in JSON based on how it wants the user interface modified.
So there are other ways other than using tools to accomplish this.
But if you want to be able to give it tools in addition to streaming back text, then this is a good
solution.
That's the that's the best time to use this when it's in conjunction with a number of other things that
the LM is doing.
So these tools are sort of adding to its capabilities.
So what we're going to do now is build an informed airline customer support agent.
We're going to want to be able to tell it that we're traveling to Paris and then have it respond with
a ticket price to Paris.
That's the idea.
We're going to do it with tools, and I will see you over in the lab to find out how.