And here, once more we find ourselves in our favorite place, the Jupyter Lab.
Ready to go with weeks.
Week two's exercises.
So we go into week two folder and we open up week two, day one.
Uh, and here we go.
So a reminder that in week one we used uh multiple frontier LMS through the chat user interface, a
way to use it through the web, uh, and then through the API, we connect it to OpenAI's API.
So today we're going to add to the mix the APIs for Anthropic and Google to join with our skills of
using OpenAI.
Uh, so as a one more reminder, you're going to kill me for keeping going on about this.
This is where you set up your keys.
Uh, you can set up keys for OpenAI, which presumably you already did last week, uh, for anthropic
and for Gemini for Google.
Uh, but, uh, bearing in mind that there is more of an adventure to be had in setting up your Google
Keys.
Once you've set them up, you create.
You should already have created the file called.
Env and make sure that your keys are in there in that form.
If you wish, instead of doing that, you can do it by typing your keys in these cells.
It's it's possible to do it that way.
It's not recommended for security reasons.
In case you one day make this public and then other people will see your keys.
All right, enough preamble.
Let's run some imports.
Let's run this block of code here that sets the environment variables.
You're pretty familiar with this.
And now in this cell, you can see that I make the same call to OpenAI to, to establish that connection
to the OpenAI API that you're familiar with now.
But then I have something pretty similar for Claude and then something a little bit different for Google
for Gemini.
So these are the sort of, uh, semi somewhat analogous commands that we're using for those three.
Okay.
So we've seen a bunch of things that Llms are pretty good at, and then just a few things where it tripped
up, but mostly things that it's very good at.
One of the things that it's not so good at as it would happen, is telling jokes.
When you give it a very tight, uh, context in which it has to try and form that joke.
Uh, and so, you know, this is clearly not a very commercial example, but it's a way of having some
fun and getting some experience with the APIs.
Uh, we are going to ask some llms to tell jokes over the API.
Um, so what information do you do?
You send over an API.
Typically, you always specify the name of the model that you want to use.
You typically give the system message and the user message.
You're super familiar with this now system message giving the overall context.
The user message is the actual prompt.
Um, and there are some other characteristics.
There are some other things that you can do.
You can pass in something called the temperature which is between 0 and 1, usually where one means
I want a more random creative output Outputs, and zero would be the lowest possible focused, deterministic
repeatable setting.
So that is another parameter that you can often provide.
So in this case we're going to set a system message to be you are an assistant that is great at telling
jokes.
And the user prompt will be tell a light hearted joke for an audience of data scientists.
That would be you and also me.
Okay, so then this structure here is hopefully something very familiar to you.
This is where we put the prompts into a list to elements you a system and a user as the as the role
in these two elements.
Going into this list, I hopefully don't need to explain it because you're now quite familiar with this.
Uh, as I say, this, this, this, um, value here, the role can be system or user.
You're going to find out later.
It can also be assistant.
So it can be system user or assistant.
And then later this week you're going to find some other thing that can go in there as well.
So.
But for now, all you need to remember is system and user as the two roles we're going to be using.
So we put that into the list of prompts.
And I should remember to execute the cell before it.
Before I do that did I execute the cell here.
Yes I did all right.
Here we go.
Let's try that one again.
Execute that cell.
Execute this cell.
Very good okay.
Let's start with one of the older GPT models GPT 3.5 turbo, which quite recently was like the the latest
and greatest frontier model.
But it's already old news.
But we will use this.
And so the API, which now you're quite familiar with for OpenAI is OpenAI dot chat, dot completions,
dot create completions, um, being the name of this, this API, the one that basically takes an existing
set of prompts and then tries to complete generate text to complete the conversation.
Um, and as we call create, we pass in a model and we pass in the messages in the format that you're
familiar with.
So let's see.
And you remember when we get back the response, what we do is we take completion dot choices, which
is a list of possible choices.
But there will only be one element in there.
There is a way that you can specify that you want it to return multiple choices.
But since we haven't done that, we just get back one and it's in location zero of course.
So completion dot choices zero dot message gives us back the message and content returns it in a string.
So that is what we get back and we print it.
And now let's see what kind of joke.
For data scientists GPT 3.5 turbo can come up with.
Here we go.
Why did the data scientists break up with their computer?
It just couldn't handle their complex relationship.
Okay, okay.
You know, I get it, I see it.
It's not the world's funniest joke, but it's not terrible.
You know, the data scientists model relationships between things and couldn't handle their complex
relationship.
Fair enough.
I'd say that's a perfectly acceptable, acceptable joke coming from GPT 3.5 turbo.
So let's see if GPT four mini can do better.
This time, we're going to just slightly expand our use of the API.
I'm including temperature, so this is where you can pass in this number between 0 and 1.
One for the most creative, zero for the least.
Um, and uh, out of this I have completion choices zero message content.
Again, you should be very familiar with this.
Let's see how it performs.
Why did the data scientist break up with a statistician?
Because she found him too mean.
I'd say that's a pretty good joke.
I'd say that's fine.
That's that's, uh, that's an acceptable joke.
Maybe I was harsh when I said that llms aren't very good at this, because that's a perfectly decent
joke.
Uh, and, uh, I think we will give GPT four a mini, uh, a round of applause for that.
Okay, let's try GPT four Minis, uh, bigger cousin, GPT four.
Oh, the maxi version of GPT four.
Oh, the big guy.
And we will ask it.
Let's give it the same temperature so we're not messing with things as we go.
We'll ask it for it for a joke.
Two and let's see how it does.
Why did the data scientist go broke?
Because they couldn't find any cache in their array.
If it hadn't put on in their array, I might have found that better.
I don't, uh, couldn't find any cache.
Would be okay.
Maybe I'm missing something here.
I I'm not sure I get it.
Uh, let's try another one.
Let's do what I had in there before and start pulling the temperature down a bit, see what we get.
Why did scientists break up with the logistic regression model?
Because it couldn't find the right fit.
Uh, you know, that's perfectly decent.
That's acceptable.
That's that's maybe, uh, I'm not sure which I prefer between Mini and Maxi, but, uh, that's a that's
a pretty solid, solid gag there.
I think we will we will say that that that's a pass for sure.
All right.
Let's move on to clause 3.5.
Sonnet.
Uh, so the API looks strikingly similar.
That's the good news.
It's basically very, very similar indeed.
A couple of differences.
You do have to pass in the system message as its own separate attribute.
And then the messages is again this this list of decks.
But of course it doesn't have that first entry for the system message because you've already passed
that in separately.
Um, so that's a slight difference.
Um, also, Max tokens is something which is optional for the OpenAI API to to specify the, the maximum
number of tokens.
And I believe it's actually required for Claude.
So that's why it's in here.
But otherwise everything should look very similar.
The API itself is a little bit easier to memorize.
It's just Claude dot messages dot create.
It's slightly shorter, but it's otherwise quite similar to OpenAI ChatGPT completions create.
Uh, so there it is.
And then when we get back a response, it's message content zero.
Again, you're asking for the the first one, but we're only going to get back one because we've only
asked for one dot text gives us that's the equivalent of dot content for OpenAI.
So let's see.
This is a useful hopefully for you for for the API framework for Claude.
Let's see now how Claude does with a joke.
Sure.
Here's a lighthearted joke for data scientists.
Why do data scientists break up with their significant other?
They just was too much variance in the relationship, and they couldn't find a good way to normalize
it.
Uh, yeah, that's all right.
I'd say it's a nerdier it's a slightly more, uh, um, data sciency.
It's perhaps just a tiny bit less funny, but it's not bad at all.
I don't know, I think whether you prefer that to GPT four is probably a matter of taste.
They're perfectly solid jokes.
They're not explosively funny, but I'd say perfectly solid.
Not terrible.
Um.
Anyway, the point of this is more about APIs and about jokes, although it always keeps it entertaining.
What I want to show you now is about streaming.
Um, you remember we talked briefly about streaming before the streaming example?
We did before, uh, looked a bit complicated because we had to deal with the fact that we were bringing
back markdown and we had to to handle that markdown.
This looks a bit simpler because we're not dealing with with a markdown response.
We're going to ask the same model, cloud 3.5 again for a joke, but this time we're going to stream
back results.
So you may remember when we asked OpenAI to stream the way we did it is we just added another attribute
stream equals true.
And that meant that it was in streaming mode.
For Claude, it's slightly different.
There is no extra attribute.
Instead, you call the dot stream method instead of the dot create method.
So slightly different approach there.
That's a nuance of difference between anthropic and OpenAI for streaming.
So we call Claude messages stream.
Otherwise it's the same.
And then with what comes back, we use a context manager with results as stream.
Um, and then it's for text in stream text stream.
And you remember OpenAI was was for chunk in response.
So OpenAI was a bit different again in the way that you read back results.
But there it is.
We get each little chunk back and just going to print that chunk.
Um, and the reason for this is to make sure that it doesn't print each chunk on a separate line.
Otherwise it'd be very hard to read.
So this should look better.
Let's see how Claude 3.5 sonnet does with a joke that it will then stream back to us in JupyterLab.
There we go.
You see?
It's streaming.
Sure, here's a light hearted joke for Data Scientist.
Why did that same joke?
It seems exactly the same joke, but it's added in a Brahms little drum.
Uh, explosion at the end, which is nice.
I wonder why did I ask for more tokens than before?
Let's see.
No.
The same.
Um, it's.
And it gives a little explanation.
This joke plays on statistical concepts which are common to data science.
It's a bit nerdy, but should get a chuckle from data savvy audience.
Well, I would say you guys are a data savvy audience, so you can be the judge of that.
Did it get a chuckle from you?
Moving on.
Gemini has a different structure.
It's it's quite a bit different, actually.
Um, and I'd probably say to Google's credit, their ability to set up tokens is much more complicated,
but the API is a bit simpler.
Uh, you can see here you create a generative model object and you pass in the name of the model, we'll
use the Gemini 1.5 flash.
You remember how many how large the context window is for Gemini 1.5 flash.
Can you remember that?
It was top of the table that we had before?
It was a remarkable 1 million tokens.
A million tokens.
750,000 words.
So, Gemini 1.5 flash.
We pass in the system instruction when we create this object, and then we call Gemini dot.
Generate content with the user prompt.
And it's just response dot text.
So a little bit less futzing around with both the request and the response here it's a bit of a simpler
API, but let's see the quality of joke.
Importantly, why did the data scientists break up with a statistician?
Because they couldn't see eye to eye on the p value.
Ah.
Well, uh, I see the data science side of it.
I'm not sure I get it.
Hahaha.
Uh, maybe you do get it.
And I'm being being, uh, being dozy.
Uh, in which case, by all means pointed out to me.
But I don't particularly get the funny aspect of that joke.
So for me, I would say that, uh, Gemini certainly lags in terms of its, uh, Gemini 1.5 flash in
terms of its humor value.
All right.
Anyways, to get serious for a moment, let's go back to GPT four.
Many with the original question.
You're a helpful assistant.
How do I decide if a business problem is suitable for an LLM solution?
Remember, that was the very first question we asked through the chat interface.
Um, and we can now bring this together again.
This should be pretty familiar to you.
We're going to stream back the results in markdown.
So it's OpenAI chat dot completions dot create.
We pass in the model.
We're going to go for the big guy.
Um we use the prompts.
We set a temperature.
We say stream equals true.
That's the way that you do it with OpenAI.
Um, and then this is the way that we stream back the results again.
It's a little bit more involved because we're dealing with markdown.
And so we have to do some, some sort of, uh, special stuff here to basically refresh the markdown
with each iteration.
If you're not sure we have to do it this way, try taking that out and doing it differently, and you'll
immediately see what what what happens.
It it won't look good.
Uh, and let's run that.
And here we get the results, and you can see that it looks great.
You can see some of the flicking happening when the markdown has only partially come through.
And so it's interpreting things like when there's perhaps multiple hashes representing a subheading.
And it's only received one hash and it thinks there's a big heading coming.
Uh, at least I think that's what we were seeing there briefly, with some of that flickering as the
markdown appeared.
But at the end of it we get back, of course, a very nicely constructed response, well structured,
and it's formatted perfectly in markdown streams back.
All right.
So that has given you a sense of the different APIs and a bit of messing around with some, some fun
questions.
And what we're going to do next in the next video is actually have a couple of llms talk to each other,
which should be fun.
I will see you then.