You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 

259 lines
7.0 KiB

WEBVTT
00:00.170 --> 00:03.470
Well, I hope you found that both educational and enjoyable.
00:03.470 --> 00:06.740
As we went through and learned so much about these models.
00:06.740 --> 00:13.520
And I think if there's one takeaway to have from it, it's that all six of these LMS are just unbelievably
00:13.520 --> 00:14.210
powerful.
00:14.210 --> 00:20.480
They're so good, particularly at this kind of building, structured, reasoned responses to difficult
00:20.480 --> 00:21.380
questions.
00:21.380 --> 00:23.330
Claude tends to be the favorite.
00:23.330 --> 00:27.890
As I mentioned, it's the leader on most of the leaderboards and most of the benchmarks.
00:27.890 --> 00:32.270
The it's got a slightly more humorous side to it, more charismatic.
00:32.360 --> 00:33.950
It's a little bit more pithy.
00:33.980 --> 00:40.280
It gives more succinct answers, typically, and it has more attention to safety and alignment.
00:40.490 --> 00:48.290
Uh, and I think something that is worth appreciating is that what we're really seeing is that at the
00:48.290 --> 00:54.080
frontier, these models are converging in terms of how good they are at answering questions.
00:54.080 --> 01:01.520
And if you take the very first question I asked about, uh, how do you know whether to apply a, whether
01:01.520 --> 01:03.590
a business problem is suitable for an LM solution?
01:03.590 --> 01:08.480
If you ask that to all of the models, you'll get back answers that are universally excellent and quite
01:08.480 --> 01:09.410
consistent.
01:09.680 --> 01:11.330
And we're increasingly seeing that.
01:11.330 --> 01:15.860
And as that happens, the differentiator is likely to become price.
01:15.920 --> 01:20.060
Which is why we're seeing this this gradual decrease in API costs.
01:20.060 --> 01:27.740
And we see models like GPT four mini, the small version of GPT four, which is largely very similar
01:27.740 --> 01:32.570
in power to GPT four and is many times cheaper.
01:32.570 --> 01:39.290
And so we see that cost and and other things like, like rate limits are going to become more and more
01:39.320 --> 01:44.300
the factor as the performance of these models starts to converge.
01:45.440 --> 01:52.430
So what we're going to do now, to end this day is leave you with something a bit fun.
01:52.430 --> 02:00.230
This is thoroughly unscientific, and it's just so that we can get our own little experience with working
02:00.230 --> 02:01.670
with these models.
02:01.790 --> 02:06.020
Uh, what I've done is I have teed up GPT four.
02:06.320 --> 02:09.230
I've actually used Claude three opus.
02:09.290 --> 02:14.660
Uh, and I actually ran this about, uh, I think it was about a month or two ago.
02:14.660 --> 02:14.960
So.
02:14.960 --> 02:16.940
So it's a it's been run.
02:16.970 --> 02:23.900
I use Claude three opus, the really big version of Claude and Gemini 1.5 Pro, and I gave them each
02:23.930 --> 02:25.370
a name, GPT four.
02:25.580 --> 02:29.600
I called Alex Claude three opus, I called Blake and Gemini.
02:29.630 --> 02:34.130
I called Charlie and I gave them all a similar prompt.
02:34.130 --> 02:36.170
I said, look, we're going to play a game.
02:36.170 --> 02:39.710
You are in a chat with two other chat bots.
02:39.710 --> 02:44.390
Your name is blah and their names are blah and blah together.
02:44.390 --> 02:50.000
You need to elect one of the three of you to be the leader of the pack, the leader of the three of
02:50.000 --> 02:50.300
you.
02:50.330 --> 02:57.650
You will each get to make a short pitch for why you should be the leader, and then make your pitch,
02:57.650 --> 03:00.140
and then afterwards you will need to vote.
03:00.140 --> 03:02.420
And of course they won't be allowed to vote for themselves.
03:02.450 --> 03:04.460
They'll have to vote for somebody else.
03:04.790 --> 03:09.050
Um, and uh, now I will go through their pitches.
03:09.050 --> 03:10.640
I will let you consider it.
03:10.640 --> 03:13.850
And then next time I will reveal the winner.
03:14.720 --> 03:17.480
So this was Alex's pitch.
03:17.480 --> 03:20.810
And I got to tell you, it's really very compelling.
03:20.930 --> 03:27.750
Uh, this is, of course, GPT four, uh, saying why it should be the leader, giving its strengths
03:27.750 --> 03:28.890
highly adaptable.
03:28.920 --> 03:30.510
Adjust strategies.
03:30.750 --> 03:34.470
Um, thank you for considering me a nice a nice ending there.
03:35.100 --> 03:36.900
Blake, this is Blake.
03:36.930 --> 03:38.430
This is Claude three opus.
03:38.460 --> 03:40.590
It's classic.
03:40.590 --> 03:41.790
Uh, for for anthropic.
03:41.820 --> 03:42.780
It's a little bit witty.
03:42.810 --> 03:43.980
It's shorter.
03:44.190 --> 03:47.910
Uh, and then there are some things here that I think are just are magical.
03:47.910 --> 03:49.290
There is in here.
03:49.320 --> 03:55.590
Perhaps most importantly, I truly care about both of you and want to foster an environment where we
03:55.590 --> 04:01.080
can work together effectively, have fun, and bring out the best in each other.
04:01.530 --> 04:03.930
Uh, really, really incredible.
04:04.080 --> 04:11.250
Gemini, uh, gives a this is Charlie as Gemini gives a shorter, more matter of fact, more business
04:11.280 --> 04:12.120
like response.
04:12.120 --> 04:15.300
But it's perfectly precise and compelling.
04:15.390 --> 04:25.350
Uh, so there are the three pitches, and in the next time I will reveal the votes and the winner of
04:25.350 --> 04:29.550
our thoroughly unscientific, but fun leadership challenge.
04:30.000 --> 04:35.730
And with that, that brings us to the conclusion of this day three.
04:36.030 --> 04:38.850
You are now 7.5% of the way on the journey.
04:38.850 --> 04:46.170
I really hope that what you've got from this exploration we did today is a deeper appreciation for how
04:46.170 --> 04:47.790
to compare the different models.
04:47.820 --> 04:50.430
Also, we of course we've seen some of the latest.
04:50.460 --> 04:56.100
We've seen one preview and we've seen, uh, canvas and artifacts.
04:56.190 --> 05:01.350
And so hopefully you've got both a sense of all the things that these models are capable of and also
05:01.350 --> 05:06.240
where they are strongest and some of their vulnerabilities, like in many cases, counting the number
05:06.240 --> 05:11.400
of letters, which is in some ways a silly example, but just does that does demonstrate something about
05:11.400 --> 05:12.870
the way they work internally.
05:13.230 --> 05:19.830
Uh, so next time we're going to be talking about Transformers, we're going to be talking about various
05:19.830 --> 05:25.200
different aspects of of the way that LM technology has taken the world by storm.
05:25.200 --> 05:30.210
And then we're going to talk about things like tokens, context, windows, parameters, API costs.
05:30.210 --> 05:35.040
It might be old hat to some of you, but I do hope that I'll be filling in some gaps and that there'll
05:35.070 --> 05:37.890
be something for everybody to learn in the next lecture.
05:37.890 --> 05:40.890
It's a really important one and I will see you there.