Welcome back.
It's time to make our full agent framework.
I'm super excited about this.
It's pulling everything together that we've been doing before, and I think you'll be very happy with
the outcome.
Uh, so just a quick recap.
An agent framework.
The term agent I as I said, it's an umbrella term.
It can refer to a bunch of different techniques.
Um, for example, it can be any of these five.
It can be about breaking a complex problem into smaller steps with multiple models carrying out different
specialized tasks.
It can be the ability for an LLM to have tools to give them extra capabilities.
It can be, uh, talking about the agent environment, which is the setup or the agent framework that
allows agents to collaborate.
Um, it can be the idea that one LLM can act as a planner, dividing tasks into smaller ones, that
specialists that can themselves be llms or bits of software can carry out.
Um, and then there is another point here, which is that people talk about agentic AI when you're thinking
about an agent having its own autonomy agency, uh, beyond necessarily just responding to a prompt,
such as having memory, being able to sort of, uh, I don't know, do something like, uh, scrape
the web for news information and using that to make decisions about buying or selling stocks, something
like that.
That is a kind of, uh, something that that exists outside the context of just say, a request response
chat.
So these are all the kinds of ways that that these are the kinds of things people are referring to when
they talk about agentic AI and the use of agents.
And what we're really doing here is we're talking about, uh, definitely number one and two there and
to a certain extent, numbers three and five.
But we're not we're not building an LLM that does the planning.
That's not something we'll be doing in this session.
So, uh, this should be somewhat familiar to you because this is the chat method that's quite close
to what we had before.
So you'll recognize a few things about this.
This section here is the usual chat radio function that we know really well.
It takes a message and a history, and it, uh, sort of unpacks that history into the format that OpenAI
will expect and then calls the response.
This part here will also look familiar to you because it's our use of tools.
It's where we find out if the model wants to call a tool, and if so, we handle that tool.
Uh, but there's one little extra line just inserted in there, and it's that line there that what we're
going to say is if the person does, if the model decides it needs to, to run the tool to find the
price of a ticket, then we will also have the, um, artist generate an image to represent that city
that's being looked up.
So there we have it.
Uh, that's, uh that's nice.
And also, now, if you remember before I told you there was a reason I passed back city that you're
going to find out.
Here it is.
That's why I needed the city to pass it to the artist.
Um, and then, uh, this is all exactly the same.
There's one more tiny change.
Which is this here, which is that, uh, once I've collected the response from the model, I then call
talker to make sure that we speak the response.
So that is our chat.
Uh, let's run that.
Now, this, I should say, since I've always showed off about how easy Gradio is, this code is a little
bit more involved.
You may notice the reason is because we're now because we want to do a little bit more and show images.
We're going outside the default, the sort of off the shelf, uh, chat user interface that Gradio provides
for us.
And we have to then build the interface ourselves.
And as a result, I've had to put together this interface that kind of puts together the various components
like the input and the buttons.
But what I'll say is this is still actually super straightforward.
It still reads like English.
It's very clear what's going on.
You'll see everything that's happening here, and hopefully this will be quite readable for you.
And you can use this to build more sophisticated chats, more sophisticated UIs yourself.
So with that background, we now are going to run this to it's running and we'll bring that up.
And here we have our chat with our new assistant.
Let's give it a try.
Hello.
How can I assist you today.
You like that?
It spoke to us.
There we go.
That's the first use of an agent.
We had a specialist model that's able to create, uh, audio, and we integrated that with our chatbot
so that it was able to speak back to us.
Great choice.
Would you like to know the to the ticket price for a return trip to London?
There we go.
That's entertaining, let's say.
We know there's a pause.
Here we go.
A return ticket to London is priced at 799.
And there we have it.
A return ticket to London is priced at $7.99.
And there is the image.
And that image looks spectacular.
A London bus in the middle.
It's got Big Ben.
It's got the bridge.
It's got, uh.
Yeah, I can see taxi there.
It's just a great montage of images.
Uh, and so I find this to be very compelling indeed, a wonderful example of what we're able to achieve
with just a little bit of code.
And so I present to you a multimodal app, complete with audio and some images running as part of what
is a in a, in a small way, a multimodal agentic framework for talking to an airline AI assistant.
Great work.
I'll see you for the challenge of the week and the wrap up.