From the uDemy course on LLM engineering.
https://www.udemy.com/course/llm-engineering-master-ai-and-large-language-models
You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
259 lines
7.0 KiB
259 lines
7.0 KiB
WEBVTT |
|
|
|
00:00.170 --> 00:03.470 |
|
Well, I hope you found that both educational and enjoyable. |
|
|
|
00:03.470 --> 00:06.740 |
|
As we went through and learned so much about these models. |
|
|
|
00:06.740 --> 00:13.520 |
|
And I think if there's one takeaway to have from it, it's that all six of these LMS are just unbelievably |
|
|
|
00:13.520 --> 00:14.210 |
|
powerful. |
|
|
|
00:14.210 --> 00:20.480 |
|
They're so good, particularly at this kind of building, structured, reasoned responses to difficult |
|
|
|
00:20.480 --> 00:21.380 |
|
questions. |
|
|
|
00:21.380 --> 00:23.330 |
|
Claude tends to be the favorite. |
|
|
|
00:23.330 --> 00:27.890 |
|
As I mentioned, it's the leader on most of the leaderboards and most of the benchmarks. |
|
|
|
00:27.890 --> 00:32.270 |
|
The it's got a slightly more humorous side to it, more charismatic. |
|
|
|
00:32.360 --> 00:33.950 |
|
It's a little bit more pithy. |
|
|
|
00:33.980 --> 00:40.280 |
|
It gives more succinct answers, typically, and it has more attention to safety and alignment. |
|
|
|
00:40.490 --> 00:48.290 |
|
Uh, and I think something that is worth appreciating is that what we're really seeing is that at the |
|
|
|
00:48.290 --> 00:54.080 |
|
frontier, these models are converging in terms of how good they are at answering questions. |
|
|
|
00:54.080 --> 01:01.520 |
|
And if you take the very first question I asked about, uh, how do you know whether to apply a, whether |
|
|
|
01:01.520 --> 01:03.590 |
|
a business problem is suitable for an LM solution? |
|
|
|
01:03.590 --> 01:08.480 |
|
If you ask that to all of the models, you'll get back answers that are universally excellent and quite |
|
|
|
01:08.480 --> 01:09.410 |
|
consistent. |
|
|
|
01:09.680 --> 01:11.330 |
|
And we're increasingly seeing that. |
|
|
|
01:11.330 --> 01:15.860 |
|
And as that happens, the differentiator is likely to become price. |
|
|
|
01:15.920 --> 01:20.060 |
|
Which is why we're seeing this this gradual decrease in API costs. |
|
|
|
01:20.060 --> 01:27.740 |
|
And we see models like GPT four mini, the small version of GPT four, which is largely very similar |
|
|
|
01:27.740 --> 01:32.570 |
|
in power to GPT four and is many times cheaper. |
|
|
|
01:32.570 --> 01:39.290 |
|
And so we see that cost and and other things like, like rate limits are going to become more and more |
|
|
|
01:39.320 --> 01:44.300 |
|
the factor as the performance of these models starts to converge. |
|
|
|
01:45.440 --> 01:52.430 |
|
So what we're going to do now, to end this day is leave you with something a bit fun. |
|
|
|
01:52.430 --> 02:00.230 |
|
This is thoroughly unscientific, and it's just so that we can get our own little experience with working |
|
|
|
02:00.230 --> 02:01.670 |
|
with these models. |
|
|
|
02:01.790 --> 02:06.020 |
|
Uh, what I've done is I have teed up GPT four. |
|
|
|
02:06.320 --> 02:09.230 |
|
I've actually used Claude three opus. |
|
|
|
02:09.290 --> 02:14.660 |
|
Uh, and I actually ran this about, uh, I think it was about a month or two ago. |
|
|
|
02:14.660 --> 02:14.960 |
|
So. |
|
|
|
02:14.960 --> 02:16.940 |
|
So it's a it's been run. |
|
|
|
02:16.970 --> 02:23.900 |
|
I use Claude three opus, the really big version of Claude and Gemini 1.5 Pro, and I gave them each |
|
|
|
02:23.930 --> 02:25.370 |
|
a name, GPT four. |
|
|
|
02:25.580 --> 02:29.600 |
|
I called Alex Claude three opus, I called Blake and Gemini. |
|
|
|
02:29.630 --> 02:34.130 |
|
I called Charlie and I gave them all a similar prompt. |
|
|
|
02:34.130 --> 02:36.170 |
|
I said, look, we're going to play a game. |
|
|
|
02:36.170 --> 02:39.710 |
|
You are in a chat with two other chat bots. |
|
|
|
02:39.710 --> 02:44.390 |
|
Your name is blah and their names are blah and blah together. |
|
|
|
02:44.390 --> 02:50.000 |
|
You need to elect one of the three of you to be the leader of the pack, the leader of the three of |
|
|
|
02:50.000 --> 02:50.300 |
|
you. |
|
|
|
02:50.330 --> 02:57.650 |
|
You will each get to make a short pitch for why you should be the leader, and then make your pitch, |
|
|
|
02:57.650 --> 03:00.140 |
|
and then afterwards you will need to vote. |
|
|
|
03:00.140 --> 03:02.420 |
|
And of course they won't be allowed to vote for themselves. |
|
|
|
03:02.450 --> 03:04.460 |
|
They'll have to vote for somebody else. |
|
|
|
03:04.790 --> 03:09.050 |
|
Um, and uh, now I will go through their pitches. |
|
|
|
03:09.050 --> 03:10.640 |
|
I will let you consider it. |
|
|
|
03:10.640 --> 03:13.850 |
|
And then next time I will reveal the winner. |
|
|
|
03:14.720 --> 03:17.480 |
|
So this was Alex's pitch. |
|
|
|
03:17.480 --> 03:20.810 |
|
And I got to tell you, it's really very compelling. |
|
|
|
03:20.930 --> 03:27.750 |
|
Uh, this is, of course, GPT four, uh, saying why it should be the leader, giving its strengths |
|
|
|
03:27.750 --> 03:28.890 |
|
highly adaptable. |
|
|
|
03:28.920 --> 03:30.510 |
|
Adjust strategies. |
|
|
|
03:30.750 --> 03:34.470 |
|
Um, thank you for considering me a nice a nice ending there. |
|
|
|
03:35.100 --> 03:36.900 |
|
Blake, this is Blake. |
|
|
|
03:36.930 --> 03:38.430 |
|
This is Claude three opus. |
|
|
|
03:38.460 --> 03:40.590 |
|
It's classic. |
|
|
|
03:40.590 --> 03:41.790 |
|
Uh, for for anthropic. |
|
|
|
03:41.820 --> 03:42.780 |
|
It's a little bit witty. |
|
|
|
03:42.810 --> 03:43.980 |
|
It's shorter. |
|
|
|
03:44.190 --> 03:47.910 |
|
Uh, and then there are some things here that I think are just are magical. |
|
|
|
03:47.910 --> 03:49.290 |
|
There is in here. |
|
|
|
03:49.320 --> 03:55.590 |
|
Perhaps most importantly, I truly care about both of you and want to foster an environment where we |
|
|
|
03:55.590 --> 04:01.080 |
|
can work together effectively, have fun, and bring out the best in each other. |
|
|
|
04:01.530 --> 04:03.930 |
|
Uh, really, really incredible. |
|
|
|
04:04.080 --> 04:11.250 |
|
Gemini, uh, gives a this is Charlie as Gemini gives a shorter, more matter of fact, more business |
|
|
|
04:11.280 --> 04:12.120 |
|
like response. |
|
|
|
04:12.120 --> 04:15.300 |
|
But it's perfectly precise and compelling. |
|
|
|
04:15.390 --> 04:25.350 |
|
Uh, so there are the three pitches, and in the next time I will reveal the votes and the winner of |
|
|
|
04:25.350 --> 04:29.550 |
|
our thoroughly unscientific, but fun leadership challenge. |
|
|
|
04:30.000 --> 04:35.730 |
|
And with that, that brings us to the conclusion of this day three. |
|
|
|
04:36.030 --> 04:38.850 |
|
You are now 7.5% of the way on the journey. |
|
|
|
04:38.850 --> 04:46.170 |
|
I really hope that what you've got from this exploration we did today is a deeper appreciation for how |
|
|
|
04:46.170 --> 04:47.790 |
|
to compare the different models. |
|
|
|
04:47.820 --> 04:50.430 |
|
Also, we of course we've seen some of the latest. |
|
|
|
04:50.460 --> 04:56.100 |
|
We've seen one preview and we've seen, uh, canvas and artifacts. |
|
|
|
04:56.190 --> 05:01.350 |
|
And so hopefully you've got both a sense of all the things that these models are capable of and also |
|
|
|
05:01.350 --> 05:06.240 |
|
where they are strongest and some of their vulnerabilities, like in many cases, counting the number |
|
|
|
05:06.240 --> 05:11.400 |
|
of letters, which is in some ways a silly example, but just does that does demonstrate something about |
|
|
|
05:11.400 --> 05:12.870 |
|
the way they work internally. |
|
|
|
05:13.230 --> 05:19.830 |
|
Uh, so next time we're going to be talking about Transformers, we're going to be talking about various |
|
|
|
05:19.830 --> 05:25.200 |
|
different aspects of of the way that LM technology has taken the world by storm. |
|
|
|
05:25.200 --> 05:30.210 |
|
And then we're going to talk about things like tokens, context, windows, parameters, API costs. |
|
|
|
05:30.210 --> 05:35.040 |
|
It might be old hat to some of you, but I do hope that I'll be filling in some gaps and that there'll |
|
|
|
05:35.070 --> 05:37.890 |
|
be something for everybody to learn in the next lecture. |
|
|
|
05:37.890 --> 05:40.890 |
|
It's a really important one and I will see you there.
|
|
|