WEBVTT

00:00.170 --> 00:03.470
Well, I hope you found that both educational and enjoyable.

00:03.470 --> 00:06.740
As we went through and learned so much about these models.

00:06.740 --> 00:13.520
And I think if there's one takeaway to have from it, it's that all six of these LMS are just unbelievably

00:13.520 --> 00:14.210
powerful.

00:14.210 --> 00:20.480
They're so good, particularly at this kind of building, structured, reasoned responses to difficult

00:20.480 --> 00:21.380
questions.

00:21.380 --> 00:23.330
Claude tends to be the favorite.

00:23.330 --> 00:27.890
As I mentioned, it's the leader on most of the leaderboards and most of the benchmarks.

00:27.890 --> 00:32.270
The it's got a slightly more humorous side to it, more charismatic.

00:32.360 --> 00:33.950
It's a little bit more pithy.

00:33.980 --> 00:40.280
It gives more succinct answers, typically, and it has more attention to safety and alignment.

00:40.490 --> 00:48.290
Uh, and I think something that is worth appreciating is that what we're really seeing is that at the

00:48.290 --> 00:54.080
frontier, these models are converging in terms of how good they are at answering questions.

00:54.080 --> 01:01.520
And if you take the very first question I asked about, uh, how do you know whether to apply a, whether

01:01.520 --> 01:03.590
a business problem is suitable for an LM solution?

01:03.590 --> 01:08.480
If you ask that to all of the models, you'll get back answers that are universally excellent and quite

01:08.480 --> 01:09.410
consistent.

01:09.680 --> 01:11.330
And we're increasingly seeing that.

01:11.330 --> 01:15.860
And as that happens, the differentiator is likely to become price.

01:15.920 --> 01:20.060
Which is why we're seeing this this gradual decrease in API costs.

01:20.060 --> 01:27.740
And we see models like GPT four mini, the small version of GPT four, which is largely very similar

01:27.740 --> 01:32.570
in power to GPT four and is many times cheaper.

01:32.570 --> 01:39.290
And so we see that cost and and other things like, like rate limits are going to become more and more

01:39.320 --> 01:44.300
the factor as the performance of these models starts to converge.

01:45.440 --> 01:52.430
So what we're going to do now, to end this day is leave you with something a bit fun.

01:52.430 --> 02:00.230
This is thoroughly unscientific, and it's just so that we can get our own little experience with working

02:00.230 --> 02:01.670
with these models.

02:01.790 --> 02:06.020
Uh, what I've done is I have teed up GPT four.

02:06.320 --> 02:09.230
I've actually used Claude three opus.

02:09.290 --> 02:14.660
Uh, and I actually ran this about, uh, I think it was about a month or two ago.

02:14.660 --> 02:14.960
So.

02:14.960 --> 02:16.940
So it's a it's been run.

02:16.970 --> 02:23.900
I use Claude three opus, the really big version of Claude and Gemini 1.5 Pro, and I gave them each

02:23.930 --> 02:25.370
a name, GPT four.

02:25.580 --> 02:29.600
I called Alex Claude three opus, I called Blake and Gemini.

02:29.630 --> 02:34.130
I called Charlie and I gave them all a similar prompt.

02:34.130 --> 02:36.170
I said, look, we're going to play a game.

02:36.170 --> 02:39.710
You are in a chat with two other chat bots.

02:39.710 --> 02:44.390
Your name is blah and their names are blah and blah together.

02:44.390 --> 02:50.000
You need to elect one of the three of you to be the leader of the pack, the leader of the three of

02:50.000 --> 02:50.300
you.

02:50.330 --> 02:57.650
You will each get to make a short pitch for why you should be the leader, and then make your pitch,

02:57.650 --> 03:00.140
and then afterwards you will need to vote.

03:00.140 --> 03:02.420
And of course they won't be allowed to vote for themselves.

03:02.450 --> 03:04.460
They'll have to vote for somebody else.

03:04.790 --> 03:09.050
Um, and uh, now I will go through their pitches.

03:09.050 --> 03:10.640
I will let you consider it.

03:10.640 --> 03:13.850
And then next time I will reveal the winner.

03:14.720 --> 03:17.480
So this was Alex's pitch.

03:17.480 --> 03:20.810
And I got to tell you, it's really very compelling.

03:20.930 --> 03:27.750
Uh, this is, of course, GPT four, uh, saying why it should be the leader, giving its strengths

03:27.750 --> 03:28.890
highly adaptable.

03:28.920 --> 03:30.510
Adjust strategies.

03:30.750 --> 03:34.470
Um, thank you for considering me a nice a nice ending there.

03:35.100 --> 03:36.900
Blake, this is Blake.

03:36.930 --> 03:38.430
This is Claude three opus.

03:38.460 --> 03:40.590
It's classic.

03:40.590 --> 03:41.790
Uh, for for anthropic.

03:41.820 --> 03:42.780
It's a little bit witty.

03:42.810 --> 03:43.980
It's shorter.

03:44.190 --> 03:47.910
Uh, and then there are some things here that I think are just are magical.

03:47.910 --> 03:49.290
There is in here.

03:49.320 --> 03:55.590
Perhaps most importantly, I truly care about both of you and want to foster an environment where we

03:55.590 --> 04:01.080
can work together effectively, have fun, and bring out the best in each other.

04:01.530 --> 04:03.930
Uh, really, really incredible.

04:04.080 --> 04:11.250
Gemini, uh, gives a this is Charlie as Gemini gives a shorter, more matter of fact, more business

04:11.280 --> 04:12.120
like response.

04:12.120 --> 04:15.300
But it's perfectly precise and compelling.

04:15.390 --> 04:25.350
Uh, so there are the three pitches, and in the next time I will reveal the votes and the winner of

04:25.350 --> 04:29.550
our thoroughly unscientific, but fun leadership challenge.

04:30.000 --> 04:35.730
And with that, that brings us to the conclusion of this day three.

04:36.030 --> 04:38.850
You are now 7.5% of the way on the journey.

04:38.850 --> 04:46.170
I really hope that what you've got from this exploration we did today is a deeper appreciation for how

04:46.170 --> 04:47.790
to compare the different models.

04:47.820 --> 04:50.430
Also, we of course we've seen some of the latest.

04:50.460 --> 04:56.100
We've seen one preview and we've seen, uh, canvas and artifacts.

04:56.190 --> 05:01.350
And so hopefully you've got both a sense of all the things that these models are capable of and also

05:01.350 --> 05:06.240
where they are strongest and some of their vulnerabilities, like in many cases, counting the number

05:06.240 --> 05:11.400
of letters, which is in some ways a silly example, but just does that does demonstrate something about

05:11.400 --> 05:12.870
the way they work internally.

05:13.230 --> 05:19.830
Uh, so next time we're going to be talking about Transformers, we're going to be talking about various

05:19.830 --> 05:25.200
different aspects of of the way that LM technology has taken the world by storm.

05:25.200 --> 05:30.210
And then we're going to talk about things like tokens, context, windows, parameters, API costs.

05:30.210 --> 05:35.040
It might be old hat to some of you, but I do hope that I'll be filling in some gaps and that there'll

05:35.070 --> 05:37.890
be something for everybody to learn in the next lecture.

05:37.890 --> 05:40.890
It's a really important one and I will see you there.