From the uDemy course on LLM engineering.
https://www.udemy.com/course/llm-engineering-master-ai-and-large-language-models
You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
271 lines
8.1 KiB
271 lines
8.1 KiB
WEBVTT |
|
|
|
00:00.680 --> 00:01.790 |
|
Hi everyone. |
|
|
|
00:01.790 --> 00:08.060 |
|
So the reason I'm so fired up about week seven is that this is the time when we actually start to build |
|
|
|
00:08.060 --> 00:10.790 |
|
our own proprietary model. |
|
|
|
00:10.790 --> 00:16.910 |
|
It's when we start to build our own AI, and we are looking to build something that we're going to try |
|
|
|
00:16.910 --> 00:23.780 |
|
and make to compete with frontier models or yes, even we're going to try and see if we can outperform |
|
|
|
00:23.780 --> 00:25.550 |
|
at least some frontier models. |
|
|
|
00:25.580 --> 00:32.300 |
|
Now, the reason that that's even possible is that we are trying to solve one particular specific problem |
|
|
|
00:32.300 --> 00:37.910 |
|
where we have a lot of data, and so we can take potentially a model that is significantly smaller than |
|
|
|
00:37.910 --> 00:38.960 |
|
a frontier model. |
|
|
|
00:38.960 --> 00:41.990 |
|
We can take something like an 8 billion parameter model. |
|
|
|
00:41.990 --> 00:50.810 |
|
And by making it very good at one focused task, we can compete against the, the, the enormous the |
|
|
|
00:50.810 --> 00:53.840 |
|
Giants, the Giants like GPT four zero. |
|
|
|
00:54.290 --> 01:00.560 |
|
Because even though GPT four has trillions of parameters, it's designed to do so many different things, |
|
|
|
01:00.560 --> 01:04.790 |
|
including writing witty poems about presidential Candidates. |
|
|
|
01:05.060 --> 01:09.980 |
|
We are not going to be able to do any of those things, but we are going to try and get really, really |
|
|
|
01:09.980 --> 01:12.620 |
|
good at predicting prices of products. |
|
|
|
01:12.620 --> 01:13.970 |
|
That's going to be our thing. |
|
|
|
01:13.970 --> 01:15.350 |
|
That's our business problem. |
|
|
|
01:15.350 --> 01:17.210 |
|
That's what we're setting out to solve. |
|
|
|
01:17.240 --> 01:24.170 |
|
The first step in this is to decide which model we're going to use as our base model for fine tuning, |
|
|
|
01:24.170 --> 01:30.050 |
|
and that's something that's worth taking time on, because it's going to be so obviously pivotal to |
|
|
|
01:30.080 --> 01:31.280 |
|
what we end up building. |
|
|
|
01:31.670 --> 01:34.880 |
|
You can think of it in itself as like a one massive great hyper parameter. |
|
|
|
01:34.880 --> 01:38.120 |
|
We can try a different base model and see how that performs. |
|
|
|
01:38.120 --> 01:45.020 |
|
And as part of that, we're going to have to decide whether we take the original pre-trained version |
|
|
|
01:45.020 --> 01:47.360 |
|
of that model, the very base model. |
|
|
|
01:47.360 --> 01:53.390 |
|
Sometimes people use the word foundation model to to describe that, as opposed to the one that's been |
|
|
|
01:53.390 --> 02:00.860 |
|
fine tuned for chat purposes that sometimes called the instruct variant, which is the one that's expecting |
|
|
|
02:00.950 --> 02:08.360 |
|
a dictionary or a list of dictionaries with a user prompt system, prompt user assistant, and so on. |
|
|
|
02:08.370 --> 02:15.120 |
|
That style of input, which you remember, gets turned into just a series of tokens separating out different |
|
|
|
02:15.120 --> 02:16.530 |
|
sections of the prompt. |
|
|
|
02:16.530 --> 02:18.540 |
|
So we've got to make a decision about that. |
|
|
|
02:18.540 --> 02:20.520 |
|
And there are of course, pros and cons. |
|
|
|
02:20.700 --> 02:26.550 |
|
Um, and then once we've done that, we should then take the base model as it is off the shelf and see |
|
|
|
02:26.550 --> 02:31.920 |
|
how that does against our challenge, because, you know, we see whether we're already in a position |
|
|
|
02:31.920 --> 02:34.950 |
|
where we are competing with the frontier models. |
|
|
|
02:34.950 --> 02:40.320 |
|
And even if we're not competing with frontier models, remember that this is going to be free of charge, |
|
|
|
02:40.320 --> 02:42.030 |
|
or at least it will be the cost of us. |
|
|
|
02:42.030 --> 02:48.330 |
|
Running computes so we won't have to pay API costs when we're running our own open source version of |
|
|
|
02:48.330 --> 02:48.840 |
|
the model. |
|
|
|
02:48.840 --> 02:54.210 |
|
So even if we're in the right territory, there's still many reasons why we should use open source. |
|
|
|
02:54.210 --> 02:59.910 |
|
Even before we talk about making it into a proprietary model specialized for this particular business |
|
|
|
02:59.910 --> 03:00.600 |
|
problem. |
|
|
|
03:00.600 --> 03:05.910 |
|
So anyways, with that in mind, it's time for us to talk about which model to use. |
|
|
|
03:06.060 --> 03:09.420 |
|
Uh, first of all, we need to decide how many parameters to use. |
|
|
|
03:09.420 --> 03:13.830 |
|
And, you know, it is probably a case that more is more. |
|
|
|
03:13.860 --> 03:19.140 |
|
More parameters is going to give us a better shot at, uh, solving the problem. |
|
|
|
03:19.140 --> 03:25.800 |
|
Well, particularly in a world as we have it, where we have a lot of training data, we have 400,000 |
|
|
|
03:25.830 --> 03:26.670 |
|
examples. |
|
|
|
03:26.670 --> 03:28.980 |
|
So we've got tons of training data. |
|
|
|
03:28.980 --> 03:31.020 |
|
It's not like we're limited in that regard. |
|
|
|
03:31.020 --> 03:33.180 |
|
And we could always generate more if we wanted to. |
|
|
|
03:33.210 --> 03:38.910 |
|
So really our constraint is going to be, uh, it's going to be the amount of memory capacity that we |
|
|
|
03:38.910 --> 03:39.270 |
|
have. |
|
|
|
03:39.300 --> 03:42.270 |
|
We're going to want to fit it in one smaller box. |
|
|
|
03:42.300 --> 03:49.320 |
|
And so we pretty much already know at this point that we can we can get in about an 8 billion parameter |
|
|
|
03:49.320 --> 03:49.740 |
|
model. |
|
|
|
03:49.770 --> 03:51.150 |
|
7 to 8 billion parameter. |
|
|
|
03:51.150 --> 03:53.430 |
|
We won't be able to go much more than that. |
|
|
|
03:53.460 --> 03:59.850 |
|
And whilst there are some small parameter models, very small uh, like like Gemma goes down to, I |
|
|
|
03:59.850 --> 04:02.400 |
|
think a 2 or 3 billion parameter version. |
|
|
|
04:02.400 --> 04:04.080 |
|
Uh, we'll take a look in a second. |
|
|
|
04:04.230 --> 04:12.660 |
|
Um, the, uh, we're probably going to want to, to take eight as the, uh, the largest possible model |
|
|
|
04:12.660 --> 04:16.510 |
|
that will fit into the boxes that we can afford. |
|
|
|
04:16.660 --> 04:22.930 |
|
So that's going to be our approach when it comes to the base or the instruct variants. |
|
|
|
04:22.930 --> 04:26.710 |
|
There are pros and cons and it's something to experiment with. |
|
|
|
04:26.740 --> 04:35.500 |
|
Generally speaking, if you're fine tuning specifically for one problem where you have a particular |
|
|
|
04:35.500 --> 04:41.230 |
|
prompt that you'll be using and expecting a response in a particular way, then you might as well start |
|
|
|
04:41.230 --> 04:48.640 |
|
with a base model, not an instruct variant, because you don't need to apply things like system prompts |
|
|
|
04:48.640 --> 04:53.200 |
|
and user prompts because you're just going to be training it for one purpose anyway. |
|
|
|
04:53.230 --> 04:59.620 |
|
So you might as well just take a base one and fine tune it, so that all it's really good at is that |
|
|
|
04:59.620 --> 05:00.490 |
|
one task. |
|
|
|
05:00.490 --> 05:02.680 |
|
So that's the kind of default answer. |
|
|
|
05:02.710 --> 05:06.850 |
|
Now there are some benefits to starting with the instruct variant. |
|
|
|
05:06.850 --> 05:08.890 |
|
One of them is that it's very good. |
|
|
|
05:08.950 --> 05:15.220 |
|
It's already been trained to recognize things like a system prompt and the user assistant interaction. |
|
|
|
05:15.220 --> 05:20.710 |
|
And you can use that to your advantage by using the system prompt to sort of tee it up in a certain |
|
|
|
05:20.710 --> 05:28.150 |
|
way to to frame it, to be working, uh, to be, uh, taking a persona, uh, rather than having to |
|
|
|
05:28.150 --> 05:31.210 |
|
rely on it, learning that persona through all of your training data. |
|
|
|
05:31.210 --> 05:37.300 |
|
So there are some situations where the instruct variant is a better starting point. |
|
|
|
05:37.450 --> 05:46.450 |
|
Uh, now, for us, my instinct is that base is the better starting point, because we've got such a |
|
|
|
05:46.450 --> 05:51.610 |
|
set structure that we're going to be using for all of our training data and for our objective. |
|
|
|
05:51.880 --> 05:57.070 |
|
Um, and I've in fact tried both and base did do slightly better than the instruct variant, but they |
|
|
|
05:57.070 --> 05:58.120 |
|
were very close. |
|
|
|
05:58.180 --> 06:02.770 |
|
Uh, you could also try both and see if you get the same results as me. |
|
|
|
06:02.770 --> 06:09.070 |
|
Uh, but uh, I would think it would be more normal in our kind of situation, when you have one particular |
|
|
|
06:09.070 --> 06:15.040 |
|
task and you don't need to be applying system prompts and the like that you would start with the base |
|
|
|
06:15.040 --> 06:16.090 |
|
variant. |
|
|
|
06:16.600 --> 06:23.470 |
|
But with that introduction, let's head over to the Hugging face, uh, open leaderboard and take a |
|
|
|
06:23.470 --> 06:24.670 |
|
look at some models.
|
|
|