Well, I hope you found that both educational and enjoyable.
As we went through and learned so much about these models.
And I think if there's one takeaway to have from it, it's that all six of these LMS are just unbelievably
powerful.
They're so good, particularly at this kind of building, structured, reasoned responses to difficult
questions.
Claude tends to be the favorite.
As I mentioned, it's the leader on most of the leaderboards and most of the benchmarks.
The it's got a slightly more humorous side to it, more charismatic.
It's a little bit more pithy.
It gives more succinct answers, typically, and it has more attention to safety and alignment.
Uh, and I think something that is worth appreciating is that what we're really seeing is that at the
frontier, these models are converging in terms of how good they are at answering questions.
And if you take the very first question I asked about, uh, how do you know whether to apply a, whether
a business problem is suitable for an LM solution?
If you ask that to all of the models, you'll get back answers that are universally excellent and quite
consistent.
And we're increasingly seeing that.
And as that happens, the differentiator is likely to become price.
Which is why we're seeing this this gradual decrease in API costs.
And we see models like GPT four mini, the small version of GPT four, which is largely very similar
in power to GPT four and is many times cheaper.
And so we see that cost and and other things like, like rate limits are going to become more and more
the factor as the performance of these models starts to converge.
So what we're going to do now, to end this day is leave you with something a bit fun.
This is thoroughly unscientific, and it's just so that we can get our own little experience with working
with these models.
Uh, what I've done is I have teed up GPT four.
I've actually used Claude three opus.
Uh, and I actually ran this about, uh, I think it was about a month or two ago.
So.
So it's a it's been run.
I use Claude three opus, the really big version of Claude and Gemini 1.5 Pro, and I gave them each
a name, GPT four.
I called Alex Claude three opus, I called Blake and Gemini.
I called Charlie and I gave them all a similar prompt.
I said, look, we're going to play a game.
You are in a chat with two other chat bots.
Your name is blah and their names are blah and blah together.
You need to elect one of the three of you to be the leader of the pack, the leader of the three of
you.
You will each get to make a short pitch for why you should be the leader, and then make your pitch,
and then afterwards you will need to vote.
And of course they won't be allowed to vote for themselves.
They'll have to vote for somebody else.
Um, and uh, now I will go through their pitches.
I will let you consider it.
And then next time I will reveal the winner.
So this was Alex's pitch.
And I got to tell you, it's really very compelling.
Uh, this is, of course, GPT four, uh, saying why it should be the leader, giving its strengths
highly adaptable.
Adjust strategies.
Um, thank you for considering me a nice a nice ending there.
Blake, this is Blake.
This is Claude three opus.
It's classic.
Uh, for for anthropic.
It's a little bit witty.
It's shorter.
Uh, and then there are some things here that I think are just are magical.
There is in here.
Perhaps most importantly, I truly care about both of you and want to foster an environment where we
can work together effectively, have fun, and bring out the best in each other.
Uh, really, really incredible.
Gemini, uh, gives a this is Charlie as Gemini gives a shorter, more matter of fact, more business
like response.
But it's perfectly precise and compelling.
Uh, so there are the three pitches, and in the next time I will reveal the votes and the winner of
our thoroughly unscientific, but fun leadership challenge.
And with that, that brings us to the conclusion of this day three.
You are now 7.5% of the way on the journey.
I really hope that what you've got from this exploration we did today is a deeper appreciation for how
to compare the different models.
Also, we of course we've seen some of the latest.
We've seen one preview and we've seen, uh, canvas and artifacts.
And so hopefully you've got both a sense of all the things that these models are capable of and also
where they are strongest and some of their vulnerabilities, like in many cases, counting the number
of letters, which is in some ways a silly example, but just does that does demonstrate something about
the way they work internally.
Uh, so next time we're going to be talking about Transformers, we're going to be talking about various
different aspects of of the way that LM technology has taken the world by storm.
And then we're going to talk about things like tokens, context, windows, parameters, API costs.
It might be old hat to some of you, but I do hope that I'll be filling in some gaps and that there'll
be something for everybody to learn in the next lecture.
It's a really important one and I will see you there.