From the uDemy course on LLM engineering.
https://www.udemy.com/course/llm-engineering-master-ai-and-large-language-models
You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
154 lines
4.3 KiB
154 lines
4.3 KiB
WEBVTT |
|
|
|
00:00.830 --> 00:01.940 |
|
Welcome. |
|
|
|
00:01.940 --> 00:02.870 |
|
It's week three. |
|
|
|
00:02.870 --> 00:03.800 |
|
It's day four. |
|
|
|
00:03.830 --> 00:11.720 |
|
We are back on the adventure in open source land, back investigating how to run inference over open |
|
|
|
00:11.720 --> 00:12.890 |
|
source models. |
|
|
|
00:13.130 --> 00:17.120 |
|
And today it is time to look at the model class in Hugging Face. |
|
|
|
00:17.120 --> 00:20.390 |
|
We talked originally about pipeline API, the high level API. |
|
|
|
00:20.420 --> 00:26.090 |
|
Then we started talking about the low level API, beginning with Tokenizers and now onto the model. |
|
|
|
00:26.150 --> 00:28.580 |
|
So what can you already do? |
|
|
|
00:28.610 --> 00:33.290 |
|
Of course, in addition to coding with frontier models, building multimodal AI assistants or you can |
|
|
|
00:33.290 --> 00:38.270 |
|
now do is use hugging faces, pipelines and tokenizers today. |
|
|
|
00:38.300 --> 00:41.270 |
|
New skills, new classes. |
|
|
|
00:41.270 --> 00:49.010 |
|
We're going to get into the models part of hugging face, which is when you actually create a transformer |
|
|
|
00:49.010 --> 00:51.860 |
|
and run it to generate text. |
|
|
|
00:51.860 --> 00:56.300 |
|
And we'll be comparing results across five different models. |
|
|
|
00:56.300 --> 01:02.090 |
|
I'm actually going to be doing three of them with you and leaving you to experiment with the other two, |
|
|
|
01:02.210 --> 01:07.910 |
|
uh, so that you can have an extra exercise, but I'll have all of the code ready for you. |
|
|
|
01:08.300 --> 01:10.700 |
|
Um, so it should be a lot of fun. |
|
|
|
01:10.970 --> 01:13.380 |
|
So the models then to introduce them. |
|
|
|
01:13.380 --> 01:21.330 |
|
We are going to again be working with llama 3.1 from meta, their flagship and groundbreaking model. |
|
|
|
01:21.330 --> 01:29.670 |
|
We are going to be looking at Phi three, which is Microsoft's open source model, and Gemma from Google. |
|
|
|
01:29.670 --> 01:32.190 |
|
It's a the small. |
|
|
|
01:32.190 --> 01:36.450 |
|
The small cousin of Gemini is Google's Gemma. |
|
|
|
01:36.510 --> 01:41.880 |
|
There are two other models that I'll be leaving you with to experiment with on your own. |
|
|
|
01:41.910 --> 01:49.830 |
|
One of them is Mistral from Mistral and the other, the other is the powerhouse that is Quinn two. |
|
|
|
01:50.040 --> 01:53.190 |
|
And I hope that you will enjoy using Quantu. |
|
|
|
01:54.270 --> 02:01.320 |
|
So we're also going to be covering three aspects of working with open source models in the hugging face |
|
|
|
02:01.320 --> 02:02.160 |
|
framework. |
|
|
|
02:02.430 --> 02:05.610 |
|
Um, the first of them is called quantization. |
|
|
|
02:05.640 --> 02:13.080 |
|
And this is about reducing the precision of the weights in the model so that it is easier to fit into |
|
|
|
02:13.080 --> 02:16.860 |
|
memory and loads in and also can run faster. |
|
|
|
02:16.860 --> 02:23.820 |
|
So quantization, a very important technique that allows us to work with, say, a one of the lower |
|
|
|
02:23.820 --> 02:25.290 |
|
end GPU boxes. |
|
|
|
02:25.330 --> 02:29.650 |
|
and when we get to training, it's going to be absolutely critical to be able to use quantization, |
|
|
|
02:29.650 --> 02:34.480 |
|
to be able to train large open source models. |
|
|
|
02:34.510 --> 02:39.160 |
|
In fact, you've heard me saying, now the Q Laura, that is the name of the technique that we're going |
|
|
|
02:39.190 --> 02:42.040 |
|
to be using in a couple of weeks time. |
|
|
|
02:42.040 --> 02:45.460 |
|
And the Q and Q, Laura stands for quantization. |
|
|
|
02:45.460 --> 02:49.960 |
|
So we will be coming up against quantization a few times on this journey. |
|
|
|
02:50.650 --> 02:54.580 |
|
Today we're also going to be looking inside a model. |
|
|
|
02:54.580 --> 03:00.310 |
|
So generally again this is a class that is more practical than theoretical. |
|
|
|
03:00.460 --> 03:02.050 |
|
But this will be one of those moments. |
|
|
|
03:02.050 --> 03:10.750 |
|
And we'll just take a peek inside at what what do the PyTorch layers look like that sit behind the hugging |
|
|
|
03:10.780 --> 03:12.970 |
|
face Transformers library. |
|
|
|
03:13.720 --> 03:20.050 |
|
And then also, we're so familiar with streaming at this point that it hardly needs to be said that |
|
|
|
03:20.050 --> 03:21.520 |
|
we want to be able to stream results. |
|
|
|
03:21.520 --> 03:26.290 |
|
So I will show you how you can work with open source models to stream results as well. |
|
|
|
03:26.290 --> 03:32.440 |
|
So these are some of the little extra bits that we're going to look into in our voyage into running |
|
|
|
03:32.440 --> 03:35.710 |
|
inference over the lower level APIs for hugging face. |
|
|
|
03:35.740 --> 03:36.940 |
|
There's quite enough talk. |
|
|
|
03:36.940 --> 03:38.350 |
|
Let's get to it.
|
|
|