From the uDemy course on LLM engineering.
https://www.udemy.com/course/llm-engineering-master-ai-and-large-language-models
You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
199 lines
5.3 KiB
199 lines
5.3 KiB
WEBVTT |
|
|
|
00:00.050 --> 00:02.870 |
|
And now we'll go a bit faster through the other models. |
|
|
|
00:02.900 --> 00:05.510 |
|
We'll start with Google's Gemini. |
|
|
|
00:05.750 --> 00:11.030 |
|
I have the Pro plan and I can pick between Gemini Advanced and and flash. |
|
|
|
00:11.330 --> 00:12.560 |
|
And let's what should we do. |
|
|
|
00:12.590 --> 00:19.280 |
|
Let's let's ask it the, the uh, first of all, the question about the whimsical question about how |
|
|
|
00:19.280 --> 00:28.850 |
|
many rainbows does it take to jump all the way from Hawaii? |
|
|
|
00:28.850 --> 00:30.020 |
|
17. |
|
|
|
00:30.020 --> 00:38.870 |
|
And let's see, uh, let's see how it, uh, handles the the answer to this. |
|
|
|
00:39.500 --> 00:51.440 |
|
Uh, so, uh, you can see, um, that it's given an answer which, whilst it's fine, um, it's, um, |
|
|
|
00:53.090 --> 00:55.520 |
|
it's definitely overly literal. |
|
|
|
00:55.520 --> 01:01.700 |
|
It's certainly not got the kind of response that we got from GPT that's so clearly understood. |
|
|
|
01:01.700 --> 01:06.560 |
|
The, the, the fact that we were being humorous with the question and was able to work with it in a |
|
|
|
01:06.560 --> 01:07.700 |
|
playful way. |
|
|
|
01:07.850 --> 01:16.300 |
|
So it to me, this sort of shows the slight lack of of nuance, ability to understand the meaning behind |
|
|
|
01:16.330 --> 01:19.750 |
|
something like this, but it is still a thorough answer from Gemini. |
|
|
|
01:19.780 --> 01:23.350 |
|
Let's ask it how many times have a new chat? |
|
|
|
01:23.380 --> 01:32.230 |
|
How many times does the letter A appear in this sentence? |
|
|
|
01:33.100 --> 01:35.500 |
|
And let's see how it can handle that. |
|
|
|
01:36.370 --> 01:37.690 |
|
It's thinking. |
|
|
|
01:40.750 --> 01:43.300 |
|
So that's that wrong? |
|
|
|
01:43.450 --> 01:49.660 |
|
Uh, I mean, arguably there is a school of thought that would be to say that that a in inverted commas |
|
|
|
01:49.660 --> 01:50.710 |
|
shouldn't count. |
|
|
|
01:50.740 --> 01:55.660 |
|
I mean, it's a stretch, but if we give it that, then we might say that saying that there are three |
|
|
|
01:55.690 --> 01:58.720 |
|
A's, uh, is uh, is not terrible. |
|
|
|
01:58.720 --> 02:03.490 |
|
But then you'll see that it believes that one of those A's has come from the word sentence, which seems |
|
|
|
02:03.490 --> 02:04.870 |
|
like a bit of a gaffe. |
|
|
|
02:04.930 --> 02:10.960 |
|
Uh, so, uh, no, it's sadly oh, one preview is still our winner in this regard. |
|
|
|
02:11.080 --> 02:15.100 |
|
Uh, so we can do more, more experiments. |
|
|
|
02:15.100 --> 02:20.050 |
|
But I think we should move on to cohere from, uh, the. |
|
|
|
02:20.140 --> 02:23.760 |
|
Sorry, this is Command Plus from cohere, Canadian AI company. |
|
|
|
02:23.790 --> 02:31.590 |
|
It really focuses a lot on the knowledge that it has specific knowledge in different areas. |
|
|
|
02:31.680 --> 02:33.120 |
|
Let's ask it that question. |
|
|
|
02:33.120 --> 02:38.490 |
|
Compared to other frontier llms, what kinds of questions are you best at answering and compare it to |
|
|
|
02:38.520 --> 02:39.150 |
|
others? |
|
|
|
02:39.150 --> 02:41.520 |
|
So, um, what we get back? |
|
|
|
02:41.550 --> 02:45.900 |
|
First of all, it says as an AI language model I'm designed to assist blah blah blah. |
|
|
|
02:45.900 --> 02:51.150 |
|
It gives some strengths, it gives some challenges, complimentary llms. |
|
|
|
02:51.240 --> 03:00.360 |
|
Uh, so it gives a number of types of model, but it doesn't actually list the names of the models. |
|
|
|
03:00.360 --> 03:07.470 |
|
But it's still a perfectly decent answer, making making it clear that it doesn't have multimodal abilities. |
|
|
|
03:07.560 --> 03:09.090 |
|
Uh, and so on. |
|
|
|
03:09.090 --> 03:16.740 |
|
So it's a pretty good, thorough answer that demonstrates that it has resource to more information about |
|
|
|
03:16.740 --> 03:17.820 |
|
this kind of thing. |
|
|
|
03:18.090 --> 03:19.620 |
|
Uh, why don't we ask it? |
|
|
|
03:19.650 --> 03:21.090 |
|
What does it feel like? |
|
|
|
03:23.100 --> 03:24.510 |
|
To be jealous. |
|
|
|
03:26.010 --> 03:27.720 |
|
See how that compares? |
|
|
|
03:28.110 --> 03:33.000 |
|
Uh, and again, you can see it's a really thorough. |
|
|
|
03:33.030 --> 03:33.840 |
|
Really. |
|
|
|
03:33.870 --> 03:39.560 |
|
You can you can get that sense that it's able to draw on a wealth of knowledge. |
|
|
|
03:39.560 --> 03:44.690 |
|
It's perhaps less of an expressive answer than we got from Claude, but I'd say that it appears to have |
|
|
|
03:44.690 --> 03:50.570 |
|
more sort of structure and detail and substance to it, uh, just based on a on a cursory look through. |
|
|
|
03:50.570 --> 03:53.150 |
|
But but you should take some more time and see what you think. |
|
|
|
03:53.390 --> 03:56.570 |
|
Uh, but it's, it's, it's clear that it's, it's good at this stuff. |
|
|
|
03:56.570 --> 03:59.120 |
|
And, you know, we might as well we're having some fun with this. |
|
|
|
03:59.120 --> 04:00.680 |
|
Why don't we ask her, hear the question. |
|
|
|
04:00.680 --> 04:08.600 |
|
How many times does the letter A appear in this sentence? |
|
|
|
04:11.060 --> 04:14.210 |
|
And letter A appears 11 times in this sentence. |
|
|
|
04:14.210 --> 04:14.930 |
|
There we go. |
|
|
|
04:14.930 --> 04:16.400 |
|
Now we now we know. |
|
|
|
04:17.420 --> 04:25.730 |
|
So again, uh, the the fact that that this platform is able to draw on a wealth of background knowledge |
|
|
|
04:25.730 --> 04:27.950 |
|
doesn't help it with that kind of task. |
|
|
|
04:27.950 --> 04:33.950 |
|
And again, this is particularly picking on something which Llms can struggle with due to the way that |
|
|
|
04:33.950 --> 04:37.880 |
|
they are trained, uh, and, and the way that they run in inference mode. |
|
|
|
04:37.910 --> 04:38.630 |
|
All right. |
|
|
|
04:38.630 --> 04:41.810 |
|
So that's a quick look at uh Gemini and Co here. |
|
|
|
04:41.810 --> 04:46.610 |
|
And then the next time we'll, we'll look at the last two meta AI and perplexity.
|
|
|