You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 

421 lines
11 KiB

WEBVTT
00:00.800 --> 00:02.300
So I know what you're thinking.
00:02.300 --> 00:03.800
You're thinking, what's going on here?
00:03.830 --> 00:05.000
We're on day five.
00:05.030 --> 00:06.740
We're on day five of week five.
00:06.770 --> 00:08.840
Why does he have a Jupyter notebook open?
00:08.840 --> 00:10.730
That's called day 4.5.
00:10.730 --> 00:18.020
And it's because it's just a little quick deviation on day four to show you genuinely how straightforward
00:18.020 --> 00:21.110
it is to switch out the vector data store for another one.
00:21.110 --> 00:27.020
And since this is super common out there, used a lot, I thought it was a great one to show you and
00:27.020 --> 00:32.930
to use it both to illustrate a different vector data store, and also to show you how easy it is to
00:32.960 --> 00:34.310
work with these abstractions.
00:34.340 --> 00:39.140
Now, something to to explain is that this isn't really a vector data store.
00:39.140 --> 00:47.810
It stands for Facebook AI Similarity Search, and it's a highly used, very common open source library
00:47.840 --> 00:54.860
put out by Facebook that lets you do quick search for vectors close to other vectors, and as such,
00:54.890 --> 00:58.370
it is often used as a kind of in-memory vector store.
00:58.370 --> 01:04.160
It doesn't persist on the disk, but you can use it to plonk in vectors and to retrieve other vectors
01:04.160 --> 01:09.230
that are similar, and you can also just use it as a library to do similarity search, to be looking
01:09.230 --> 01:16.520
for vectors close to other vectors, and Langshan has the right abstractions built around it, so it
01:16.520 --> 01:19.220
can just be a drop in replacement for chroma.
01:19.220 --> 01:22.310
So we use chroma before, which was persistent on disk.
01:22.310 --> 01:26.090
So it's a more serious hardcore data store vice.
01:26.510 --> 01:29.360
We can just use that in lieu of chroma.
01:29.360 --> 01:34.070
And I'm going to show you how easy it is because the code is going to be almost identical, except when
01:34.070 --> 01:37.610
we get to that abstraction and a couple of other things.
01:38.240 --> 01:42.110
So we run the imports as before in this import.
01:42.110 --> 01:45.980
You can see I have commented out the import for chroma, so I can't cheat.
01:45.980 --> 01:52.160
Chroma will not be included here and instead we're importing face Facebook AI similarity search.
01:52.190 --> 01:58.760
It's worth noting that face comes in two variations a CPU variant and a GPU variant, depending on which
01:58.760 --> 01:59.930
you have Pip installed.
01:59.930 --> 02:04.490
And with this environment, of course, I have Pip installed the CPU variant, but for high performance
02:04.490 --> 02:11.750
projects running on GPU, you can run the GPU version to run at blazing speed, so we've imported vice
02:11.750 --> 02:13.880
and no errors, which is great.
02:14.150 --> 02:15.650
Set up some things.
02:15.650 --> 02:17.300
This is all the same code.
02:17.300 --> 02:19.220
We load everything in as before.
02:19.220 --> 02:25.970
We're going to get hopefully 123 uh, to change that to be Len chunks.
02:25.970 --> 02:28.310
So we see do we have 123.
02:28.340 --> 02:29.570
Yes we do.
02:29.600 --> 02:31.700
Let's check that we have the right metadata.
02:31.700 --> 02:33.590
We have the four types.
02:33.590 --> 02:35.840
So far nothing is different.
02:35.840 --> 02:36.980
This is all the same.
02:36.980 --> 02:38.000
But look at this.
02:38.000 --> 02:40.550
One line has changed here cunningly.
02:40.580 --> 02:44.060
Uh, I didn't need to draw quite so much attention to it.
02:44.060 --> 02:45.860
I think you probably spotted it already.
02:45.860 --> 02:49.850
So we used to say vector store is chroma dot from documents.
02:49.880 --> 02:53.720
And we passed in the chunks embeddings and the persist directory.
02:53.720 --> 02:58.580
And now we just say vice dot from documents same construct.
02:58.610 --> 03:00.380
We also pass in the chunks.
03:00.380 --> 03:01.970
We also pass in the embeddings.
03:01.970 --> 03:06.320
And of course we don't pass in a persist directory because vice doesn't persist on disk.
03:06.620 --> 03:09.980
Um, now these two lines here are also different.
03:09.980 --> 03:14.750
So the lower level ways to ask questions of the data store are different.
03:14.750 --> 03:20.300
So if you're going to be using Feist, it's useful to have this to hand so you know how you can query
03:20.300 --> 03:23.600
for things like the number of vectors and the dimensionality.
03:23.630 --> 03:27.530
I don't know if you remember how many dimensions that there were when we did this in chroma.
03:27.650 --> 03:32.630
Um, but it's the same number of dimensions we're coming up with here.
03:32.630 --> 03:36.590
We're using the same OpenAI embeddings to vectorize.
03:36.590 --> 03:40.550
So we're still using the same LLM to create the vectors.
03:40.550 --> 03:45.620
The thing that has changed is how we are storing those vectors, not in chroma but in Feis.
03:46.130 --> 03:50.540
Um, and so uh, no surprise there 123 vectors, one for each chunk.
03:50.540 --> 03:54.470
And they have these dimensions coming back from OpenAI.
03:55.190 --> 04:01.370
Uh, so the other thing that's changed has been this pre-work section was different in day four.
04:01.400 --> 04:04.010
I'll remind you in day four, the pre-work.
04:04.010 --> 04:05.660
Let's go and take a look at that.
04:06.080 --> 04:07.850
Uh, ba ba ba ba.
04:08.450 --> 04:14.270
Uh, the pre-work actually looks a bit simpler just plucking out the vectors, documents, and doc types
04:14.300 --> 04:15.890
from chroma.
04:15.890 --> 04:19.970
And you can look at the ways that you query chroma to get this data.
04:20.170 --> 04:22.810
Um, and this is how I did it for vice.
04:22.840 --> 04:25.840
There might be a tighter way to do that, but this seems simple enough.
04:25.840 --> 04:31.810
I just collect the same vectors, documents, doc types, colors, and I map it into the color that
04:31.990 --> 04:38.410
Plotly is expecting so that we can just plot the same, uh, diagram so we can visualize our vectors
04:38.410 --> 04:40.480
just as we did before in 2D.
04:40.600 --> 04:44.290
This code is identical with one tiny exception.
04:44.320 --> 04:45.670
See if you can spot it.
04:45.670 --> 04:47.290
That's the only change.
04:47.290 --> 04:52.570
I've changed the title from saying chroma to saying vice, but otherwise the code is the same.
04:52.750 --> 04:55.390
Uh, we will visualize our data store.
04:55.390 --> 05:00.250
And so here then are the vectors as they are represented in vice.
05:00.250 --> 05:04.660
And of course, as you would expect, it's the same vectorization approach.
05:04.660 --> 05:06.430
So it looks pretty similar.
05:06.430 --> 05:11.020
We're just using a different underlying technology as our vector data store.
05:11.260 --> 05:13.960
Uh, and we can of course represent that in 3D.
05:13.990 --> 05:21.190
And we're now looking at the 3D representation in vice and nice to see that actually this time we've
05:21.190 --> 05:24.370
got, it's uh nicely it's very nicely separated.
05:24.400 --> 05:25.060
There we go.
05:25.090 --> 05:26.560
Anyway, I could look at that all day.
05:27.160 --> 05:28.510
So you get that sense.
05:28.510 --> 05:31.930
And then the code to bring it all together is identical.
05:31.930 --> 05:33.310
I haven't changed this at all.
05:33.340 --> 05:38.020
Vector stored as retriever can be called on chroma, or it can be called on face.
05:38.020 --> 05:39.190
And it is the same.
05:39.190 --> 05:39.940
So there we go.
05:39.970 --> 05:40.750
We run it.
05:40.780 --> 05:41.740
No errors.
05:41.740 --> 05:42.370
It's fine.
05:42.370 --> 05:45.550
Let's go straight to bringing up Gradio.
05:45.820 --> 05:49.180
Uh, and here we go.
05:49.210 --> 05:51.820
Here is our Gradio interface.
05:52.150 --> 05:58.690
Um, now, one of the things that I meant to show you last time that I now take this chance, and you
05:58.690 --> 06:04.600
can see the same will apply in chroma as well, is I can take questions like, uh, like you remember
06:04.600 --> 06:07.060
last time I asked, what did Avery do before?
06:07.060 --> 06:11.710
And it was cunning enough to look up the right context, even though I didn't say Lancaster and I spelled
06:11.710 --> 06:17.500
it with a lowercase a, we can take that even further, and I can say something like, what did?
06:17.500 --> 06:21.370
And I can spell Avery's name wrong like that.
06:21.400 --> 06:24.400
What did Avery do before?
06:25.870 --> 06:27.670
Um, uh.
06:27.670 --> 06:32.920
And I can run this piece of code like it is and have a look at what's come back.
06:33.130 --> 06:40.390
It has correctly identified that I'm talking about Avery Lancaster, and it looked up her HR document
06:40.390 --> 06:45.580
and it's again correctly identified that she worked at Innovate Insurance Solutions.
06:45.670 --> 06:48.760
Uh, we can might as well go in and just quickly check.
06:48.760 --> 06:51.700
Since it's set, it will go to the employees documents.
06:51.700 --> 06:53.500
We find Avery Lancaster.
06:53.500 --> 06:54.580
Here it is.
06:54.580 --> 06:56.560
Here is her HR record.
06:56.560 --> 06:57.700
Let's see what she did.
06:57.700 --> 07:03.310
She was indeed at Innovate Insurance Solutions before founding insurer Elm.
07:03.550 --> 07:06.010
Uh, it's good to see it didn't invent that.
07:06.340 --> 07:16.270
Uh, so, um, uh, the bottom line is that it's this this gives you a real sense that it didn't just
07:16.270 --> 07:17.530
do text matching.
07:17.530 --> 07:23.230
It didn't even, uh, get fussed by the wrong case that it was in lowercase and uppercase.
07:23.230 --> 07:30.160
It actually was able to identify that Avery spelt wrong, has the same meaning as Avery spelt with a
07:30.190 --> 07:31.870
Y Avery Lancaster.
07:31.870 --> 07:39.220
It has enough common sense to recognize that that is very plausibly what we were after.
07:39.220 --> 07:44.890
And again, the reason is because when it turned this into a vector and it put it in the vector and
07:44.890 --> 07:51.640
it looked in the vector data store for the vectors that were close to that, it found the Avery Lancaster's
07:51.820 --> 07:56.050
HR record as being something that was close to that in the data store.
07:56.050 --> 08:00.430
So I think it's fascinating to see that you can spell things wrong and it still works.
08:00.430 --> 08:06.580
And it's such a clear example of how much better using the vector lookup approach is than using the
08:06.580 --> 08:11.080
brute force technique that we used in the first session, which obviously would have failed hopelessly
08:11.080 --> 08:12.130
with this test.
08:12.190 --> 08:17.890
So I will let you prove to yourself that the same test will work if you use chroma, of course, and
08:17.890 --> 08:19.720
you can try out both, but you will see.
08:19.720 --> 08:23.350
I hope that chroma and vice have both worked very well indeed.
08:23.440 --> 08:26.020
But most importantly, you've seen that.
08:26.050 --> 08:33.220
True to what I explained before, Lang makes it super simple to switch out different vector data stores
08:33.220 --> 08:37.510
behind the scenes and use the same plumbing for your Rag workflow.
08:38.080 --> 08:39.760
All right, back to the slides.