From the uDemy course on LLM engineering.
https://www.udemy.com/course/llm-engineering-master-ai-and-large-language-models
You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
421 lines
11 KiB
421 lines
11 KiB
WEBVTT |
|
|
|
00:00.800 --> 00:02.300 |
|
So I know what you're thinking. |
|
|
|
00:02.300 --> 00:03.800 |
|
You're thinking, what's going on here? |
|
|
|
00:03.830 --> 00:05.000 |
|
We're on day five. |
|
|
|
00:05.030 --> 00:06.740 |
|
We're on day five of week five. |
|
|
|
00:06.770 --> 00:08.840 |
|
Why does he have a Jupyter notebook open? |
|
|
|
00:08.840 --> 00:10.730 |
|
That's called day 4.5. |
|
|
|
00:10.730 --> 00:18.020 |
|
And it's because it's just a little quick deviation on day four to show you genuinely how straightforward |
|
|
|
00:18.020 --> 00:21.110 |
|
it is to switch out the vector data store for another one. |
|
|
|
00:21.110 --> 00:27.020 |
|
And since this is super common out there, used a lot, I thought it was a great one to show you and |
|
|
|
00:27.020 --> 00:32.930 |
|
to use it both to illustrate a different vector data store, and also to show you how easy it is to |
|
|
|
00:32.960 --> 00:34.310 |
|
work with these abstractions. |
|
|
|
00:34.340 --> 00:39.140 |
|
Now, something to to explain is that this isn't really a vector data store. |
|
|
|
00:39.140 --> 00:47.810 |
|
It stands for Facebook AI Similarity Search, and it's a highly used, very common open source library |
|
|
|
00:47.840 --> 00:54.860 |
|
put out by Facebook that lets you do quick search for vectors close to other vectors, and as such, |
|
|
|
00:54.890 --> 00:58.370 |
|
it is often used as a kind of in-memory vector store. |
|
|
|
00:58.370 --> 01:04.160 |
|
It doesn't persist on the disk, but you can use it to plonk in vectors and to retrieve other vectors |
|
|
|
01:04.160 --> 01:09.230 |
|
that are similar, and you can also just use it as a library to do similarity search, to be looking |
|
|
|
01:09.230 --> 01:16.520 |
|
for vectors close to other vectors, and Langshan has the right abstractions built around it, so it |
|
|
|
01:16.520 --> 01:19.220 |
|
can just be a drop in replacement for chroma. |
|
|
|
01:19.220 --> 01:22.310 |
|
So we use chroma before, which was persistent on disk. |
|
|
|
01:22.310 --> 01:26.090 |
|
So it's a more serious hardcore data store vice. |
|
|
|
01:26.510 --> 01:29.360 |
|
We can just use that in lieu of chroma. |
|
|
|
01:29.360 --> 01:34.070 |
|
And I'm going to show you how easy it is because the code is going to be almost identical, except when |
|
|
|
01:34.070 --> 01:37.610 |
|
we get to that abstraction and a couple of other things. |
|
|
|
01:38.240 --> 01:42.110 |
|
So we run the imports as before in this import. |
|
|
|
01:42.110 --> 01:45.980 |
|
You can see I have commented out the import for chroma, so I can't cheat. |
|
|
|
01:45.980 --> 01:52.160 |
|
Chroma will not be included here and instead we're importing face Facebook AI similarity search. |
|
|
|
01:52.190 --> 01:58.760 |
|
It's worth noting that face comes in two variations a CPU variant and a GPU variant, depending on which |
|
|
|
01:58.760 --> 01:59.930 |
|
you have Pip installed. |
|
|
|
01:59.930 --> 02:04.490 |
|
And with this environment, of course, I have Pip installed the CPU variant, but for high performance |
|
|
|
02:04.490 --> 02:11.750 |
|
projects running on GPU, you can run the GPU version to run at blazing speed, so we've imported vice |
|
|
|
02:11.750 --> 02:13.880 |
|
and no errors, which is great. |
|
|
|
02:14.150 --> 02:15.650 |
|
Set up some things. |
|
|
|
02:15.650 --> 02:17.300 |
|
This is all the same code. |
|
|
|
02:17.300 --> 02:19.220 |
|
We load everything in as before. |
|
|
|
02:19.220 --> 02:25.970 |
|
We're going to get hopefully 123 uh, to change that to be Len chunks. |
|
|
|
02:25.970 --> 02:28.310 |
|
So we see do we have 123. |
|
|
|
02:28.340 --> 02:29.570 |
|
Yes we do. |
|
|
|
02:29.600 --> 02:31.700 |
|
Let's check that we have the right metadata. |
|
|
|
02:31.700 --> 02:33.590 |
|
We have the four types. |
|
|
|
02:33.590 --> 02:35.840 |
|
So far nothing is different. |
|
|
|
02:35.840 --> 02:36.980 |
|
This is all the same. |
|
|
|
02:36.980 --> 02:38.000 |
|
But look at this. |
|
|
|
02:38.000 --> 02:40.550 |
|
One line has changed here cunningly. |
|
|
|
02:40.580 --> 02:44.060 |
|
Uh, I didn't need to draw quite so much attention to it. |
|
|
|
02:44.060 --> 02:45.860 |
|
I think you probably spotted it already. |
|
|
|
02:45.860 --> 02:49.850 |
|
So we used to say vector store is chroma dot from documents. |
|
|
|
02:49.880 --> 02:53.720 |
|
And we passed in the chunks embeddings and the persist directory. |
|
|
|
02:53.720 --> 02:58.580 |
|
And now we just say vice dot from documents same construct. |
|
|
|
02:58.610 --> 03:00.380 |
|
We also pass in the chunks. |
|
|
|
03:00.380 --> 03:01.970 |
|
We also pass in the embeddings. |
|
|
|
03:01.970 --> 03:06.320 |
|
And of course we don't pass in a persist directory because vice doesn't persist on disk. |
|
|
|
03:06.620 --> 03:09.980 |
|
Um, now these two lines here are also different. |
|
|
|
03:09.980 --> 03:14.750 |
|
So the lower level ways to ask questions of the data store are different. |
|
|
|
03:14.750 --> 03:20.300 |
|
So if you're going to be using Feist, it's useful to have this to hand so you know how you can query |
|
|
|
03:20.300 --> 03:23.600 |
|
for things like the number of vectors and the dimensionality. |
|
|
|
03:23.630 --> 03:27.530 |
|
I don't know if you remember how many dimensions that there were when we did this in chroma. |
|
|
|
03:27.650 --> 03:32.630 |
|
Um, but it's the same number of dimensions we're coming up with here. |
|
|
|
03:32.630 --> 03:36.590 |
|
We're using the same OpenAI embeddings to vectorize. |
|
|
|
03:36.590 --> 03:40.550 |
|
So we're still using the same LLM to create the vectors. |
|
|
|
03:40.550 --> 03:45.620 |
|
The thing that has changed is how we are storing those vectors, not in chroma but in Feis. |
|
|
|
03:46.130 --> 03:50.540 |
|
Um, and so uh, no surprise there 123 vectors, one for each chunk. |
|
|
|
03:50.540 --> 03:54.470 |
|
And they have these dimensions coming back from OpenAI. |
|
|
|
03:55.190 --> 04:01.370 |
|
Uh, so the other thing that's changed has been this pre-work section was different in day four. |
|
|
|
04:01.400 --> 04:04.010 |
|
I'll remind you in day four, the pre-work. |
|
|
|
04:04.010 --> 04:05.660 |
|
Let's go and take a look at that. |
|
|
|
04:06.080 --> 04:07.850 |
|
Uh, ba ba ba ba. |
|
|
|
04:08.450 --> 04:14.270 |
|
Uh, the pre-work actually looks a bit simpler just plucking out the vectors, documents, and doc types |
|
|
|
04:14.300 --> 04:15.890 |
|
from chroma. |
|
|
|
04:15.890 --> 04:19.970 |
|
And you can look at the ways that you query chroma to get this data. |
|
|
|
04:20.170 --> 04:22.810 |
|
Um, and this is how I did it for vice. |
|
|
|
04:22.840 --> 04:25.840 |
|
There might be a tighter way to do that, but this seems simple enough. |
|
|
|
04:25.840 --> 04:31.810 |
|
I just collect the same vectors, documents, doc types, colors, and I map it into the color that |
|
|
|
04:31.990 --> 04:38.410 |
|
Plotly is expecting so that we can just plot the same, uh, diagram so we can visualize our vectors |
|
|
|
04:38.410 --> 04:40.480 |
|
just as we did before in 2D. |
|
|
|
04:40.600 --> 04:44.290 |
|
This code is identical with one tiny exception. |
|
|
|
04:44.320 --> 04:45.670 |
|
See if you can spot it. |
|
|
|
04:45.670 --> 04:47.290 |
|
That's the only change. |
|
|
|
04:47.290 --> 04:52.570 |
|
I've changed the title from saying chroma to saying vice, but otherwise the code is the same. |
|
|
|
04:52.750 --> 04:55.390 |
|
Uh, we will visualize our data store. |
|
|
|
04:55.390 --> 05:00.250 |
|
And so here then are the vectors as they are represented in vice. |
|
|
|
05:00.250 --> 05:04.660 |
|
And of course, as you would expect, it's the same vectorization approach. |
|
|
|
05:04.660 --> 05:06.430 |
|
So it looks pretty similar. |
|
|
|
05:06.430 --> 05:11.020 |
|
We're just using a different underlying technology as our vector data store. |
|
|
|
05:11.260 --> 05:13.960 |
|
Uh, and we can of course represent that in 3D. |
|
|
|
05:13.990 --> 05:21.190 |
|
And we're now looking at the 3D representation in vice and nice to see that actually this time we've |
|
|
|
05:21.190 --> 05:24.370 |
|
got, it's uh nicely it's very nicely separated. |
|
|
|
05:24.400 --> 05:25.060 |
|
There we go. |
|
|
|
05:25.090 --> 05:26.560 |
|
Anyway, I could look at that all day. |
|
|
|
05:27.160 --> 05:28.510 |
|
So you get that sense. |
|
|
|
05:28.510 --> 05:31.930 |
|
And then the code to bring it all together is identical. |
|
|
|
05:31.930 --> 05:33.310 |
|
I haven't changed this at all. |
|
|
|
05:33.340 --> 05:38.020 |
|
Vector stored as retriever can be called on chroma, or it can be called on face. |
|
|
|
05:38.020 --> 05:39.190 |
|
And it is the same. |
|
|
|
05:39.190 --> 05:39.940 |
|
So there we go. |
|
|
|
05:39.970 --> 05:40.750 |
|
We run it. |
|
|
|
05:40.780 --> 05:41.740 |
|
No errors. |
|
|
|
05:41.740 --> 05:42.370 |
|
It's fine. |
|
|
|
05:42.370 --> 05:45.550 |
|
Let's go straight to bringing up Gradio. |
|
|
|
05:45.820 --> 05:49.180 |
|
Uh, and here we go. |
|
|
|
05:49.210 --> 05:51.820 |
|
Here is our Gradio interface. |
|
|
|
05:52.150 --> 05:58.690 |
|
Um, now, one of the things that I meant to show you last time that I now take this chance, and you |
|
|
|
05:58.690 --> 06:04.600 |
|
can see the same will apply in chroma as well, is I can take questions like, uh, like you remember |
|
|
|
06:04.600 --> 06:07.060 |
|
last time I asked, what did Avery do before? |
|
|
|
06:07.060 --> 06:11.710 |
|
And it was cunning enough to look up the right context, even though I didn't say Lancaster and I spelled |
|
|
|
06:11.710 --> 06:17.500 |
|
it with a lowercase a, we can take that even further, and I can say something like, what did? |
|
|
|
06:17.500 --> 06:21.370 |
|
And I can spell Avery's name wrong like that. |
|
|
|
06:21.400 --> 06:24.400 |
|
What did Avery do before? |
|
|
|
06:25.870 --> 06:27.670 |
|
Um, uh. |
|
|
|
06:27.670 --> 06:32.920 |
|
And I can run this piece of code like it is and have a look at what's come back. |
|
|
|
06:33.130 --> 06:40.390 |
|
It has correctly identified that I'm talking about Avery Lancaster, and it looked up her HR document |
|
|
|
06:40.390 --> 06:45.580 |
|
and it's again correctly identified that she worked at Innovate Insurance Solutions. |
|
|
|
06:45.670 --> 06:48.760 |
|
Uh, we can might as well go in and just quickly check. |
|
|
|
06:48.760 --> 06:51.700 |
|
Since it's set, it will go to the employees documents. |
|
|
|
06:51.700 --> 06:53.500 |
|
We find Avery Lancaster. |
|
|
|
06:53.500 --> 06:54.580 |
|
Here it is. |
|
|
|
06:54.580 --> 06:56.560 |
|
Here is her HR record. |
|
|
|
06:56.560 --> 06:57.700 |
|
Let's see what she did. |
|
|
|
06:57.700 --> 07:03.310 |
|
She was indeed at Innovate Insurance Solutions before founding insurer Elm. |
|
|
|
07:03.550 --> 07:06.010 |
|
Uh, it's good to see it didn't invent that. |
|
|
|
07:06.340 --> 07:16.270 |
|
Uh, so, um, uh, the bottom line is that it's this this gives you a real sense that it didn't just |
|
|
|
07:16.270 --> 07:17.530 |
|
do text matching. |
|
|
|
07:17.530 --> 07:23.230 |
|
It didn't even, uh, get fussed by the wrong case that it was in lowercase and uppercase. |
|
|
|
07:23.230 --> 07:30.160 |
|
It actually was able to identify that Avery spelt wrong, has the same meaning as Avery spelt with a |
|
|
|
07:30.190 --> 07:31.870 |
|
Y Avery Lancaster. |
|
|
|
07:31.870 --> 07:39.220 |
|
It has enough common sense to recognize that that is very plausibly what we were after. |
|
|
|
07:39.220 --> 07:44.890 |
|
And again, the reason is because when it turned this into a vector and it put it in the vector and |
|
|
|
07:44.890 --> 07:51.640 |
|
it looked in the vector data store for the vectors that were close to that, it found the Avery Lancaster's |
|
|
|
07:51.820 --> 07:56.050 |
|
HR record as being something that was close to that in the data store. |
|
|
|
07:56.050 --> 08:00.430 |
|
So I think it's fascinating to see that you can spell things wrong and it still works. |
|
|
|
08:00.430 --> 08:06.580 |
|
And it's such a clear example of how much better using the vector lookup approach is than using the |
|
|
|
08:06.580 --> 08:11.080 |
|
brute force technique that we used in the first session, which obviously would have failed hopelessly |
|
|
|
08:11.080 --> 08:12.130 |
|
with this test. |
|
|
|
08:12.190 --> 08:17.890 |
|
So I will let you prove to yourself that the same test will work if you use chroma, of course, and |
|
|
|
08:17.890 --> 08:19.720 |
|
you can try out both, but you will see. |
|
|
|
08:19.720 --> 08:23.350 |
|
I hope that chroma and vice have both worked very well indeed. |
|
|
|
08:23.440 --> 08:26.020 |
|
But most importantly, you've seen that. |
|
|
|
08:26.050 --> 08:33.220 |
|
True to what I explained before, Lang makes it super simple to switch out different vector data stores |
|
|
|
08:33.220 --> 08:37.510 |
|
behind the scenes and use the same plumbing for your Rag workflow. |
|
|
|
08:38.080 --> 08:39.760 |
|
All right, back to the slides.
|
|
|