You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 

154 lines
4.3 KiB

WEBVTT
00:00.830 --> 00:01.940
Welcome.
00:01.940 --> 00:02.870
It's week three.
00:02.870 --> 00:03.800
It's day four.
00:03.830 --> 00:11.720
We are back on the adventure in open source land, back investigating how to run inference over open
00:11.720 --> 00:12.890
source models.
00:13.130 --> 00:17.120
And today it is time to look at the model class in Hugging Face.
00:17.120 --> 00:20.390
We talked originally about pipeline API, the high level API.
00:20.420 --> 00:26.090
Then we started talking about the low level API, beginning with Tokenizers and now onto the model.
00:26.150 --> 00:28.580
So what can you already do?
00:28.610 --> 00:33.290
Of course, in addition to coding with frontier models, building multimodal AI assistants or you can
00:33.290 --> 00:38.270
now do is use hugging faces, pipelines and tokenizers today.
00:38.300 --> 00:41.270
New skills, new classes.
00:41.270 --> 00:49.010
We're going to get into the models part of hugging face, which is when you actually create a transformer
00:49.010 --> 00:51.860
and run it to generate text.
00:51.860 --> 00:56.300
And we'll be comparing results across five different models.
00:56.300 --> 01:02.090
I'm actually going to be doing three of them with you and leaving you to experiment with the other two,
01:02.210 --> 01:07.910
uh, so that you can have an extra exercise, but I'll have all of the code ready for you.
01:08.300 --> 01:10.700
Um, so it should be a lot of fun.
01:10.970 --> 01:13.380
So the models then to introduce them.
01:13.380 --> 01:21.330
We are going to again be working with llama 3.1 from meta, their flagship and groundbreaking model.
01:21.330 --> 01:29.670
We are going to be looking at Phi three, which is Microsoft's open source model, and Gemma from Google.
01:29.670 --> 01:32.190
It's a the small.
01:32.190 --> 01:36.450
The small cousin of Gemini is Google's Gemma.
01:36.510 --> 01:41.880
There are two other models that I'll be leaving you with to experiment with on your own.
01:41.910 --> 01:49.830
One of them is Mistral from Mistral and the other, the other is the powerhouse that is Quinn two.
01:50.040 --> 01:53.190
And I hope that you will enjoy using Quantu.
01:54.270 --> 02:01.320
So we're also going to be covering three aspects of working with open source models in the hugging face
02:01.320 --> 02:02.160
framework.
02:02.430 --> 02:05.610
Um, the first of them is called quantization.
02:05.640 --> 02:13.080
And this is about reducing the precision of the weights in the model so that it is easier to fit into
02:13.080 --> 02:16.860
memory and loads in and also can run faster.
02:16.860 --> 02:23.820
So quantization, a very important technique that allows us to work with, say, a one of the lower
02:23.820 --> 02:25.290
end GPU boxes.
02:25.330 --> 02:29.650
and when we get to training, it's going to be absolutely critical to be able to use quantization,
02:29.650 --> 02:34.480
to be able to train large open source models.
02:34.510 --> 02:39.160
In fact, you've heard me saying, now the Q Laura, that is the name of the technique that we're going
02:39.190 --> 02:42.040
to be using in a couple of weeks time.
02:42.040 --> 02:45.460
And the Q and Q, Laura stands for quantization.
02:45.460 --> 02:49.960
So we will be coming up against quantization a few times on this journey.
02:50.650 --> 02:54.580
Today we're also going to be looking inside a model.
02:54.580 --> 03:00.310
So generally again this is a class that is more practical than theoretical.
03:00.460 --> 03:02.050
But this will be one of those moments.
03:02.050 --> 03:10.750
And we'll just take a peek inside at what what do the PyTorch layers look like that sit behind the hugging
03:10.780 --> 03:12.970
face Transformers library.
03:13.720 --> 03:20.050
And then also, we're so familiar with streaming at this point that it hardly needs to be said that
03:20.050 --> 03:21.520
we want to be able to stream results.
03:21.520 --> 03:26.290
So I will show you how you can work with open source models to stream results as well.
03:26.290 --> 03:32.440
So these are some of the little extra bits that we're going to look into in our voyage into running
03:32.440 --> 03:35.710
inference over the lower level APIs for hugging face.
03:35.740 --> 03:36.940
There's quite enough talk.
03:36.940 --> 03:38.350
Let's get to it.