WEBVTT 00:00.800 --> 00:02.300 So I know what you're thinking. 00:02.300 --> 00:03.800 You're thinking, what's going on here? 00:03.830 --> 00:05.000 We're on day five. 00:05.030 --> 00:06.740 We're on day five of week five. 00:06.770 --> 00:08.840 Why does he have a Jupyter notebook open? 00:08.840 --> 00:10.730 That's called day 4.5. 00:10.730 --> 00:18.020 And it's because it's just a little quick deviation on day four to show you genuinely how straightforward 00:18.020 --> 00:21.110 it is to switch out the vector data store for another one. 00:21.110 --> 00:27.020 And since this is super common out there, used a lot, I thought it was a great one to show you and 00:27.020 --> 00:32.930 to use it both to illustrate a different vector data store, and also to show you how easy it is to 00:32.960 --> 00:34.310 work with these abstractions. 00:34.340 --> 00:39.140 Now, something to to explain is that this isn't really a vector data store. 00:39.140 --> 00:47.810 It stands for Facebook AI Similarity Search, and it's a highly used, very common open source library 00:47.840 --> 00:54.860 put out by Facebook that lets you do quick search for vectors close to other vectors, and as such, 00:54.890 --> 00:58.370 it is often used as a kind of in-memory vector store. 00:58.370 --> 01:04.160 It doesn't persist on the disk, but you can use it to plonk in vectors and to retrieve other vectors 01:04.160 --> 01:09.230 that are similar, and you can also just use it as a library to do similarity search, to be looking 01:09.230 --> 01:16.520 for vectors close to other vectors, and Langshan has the right abstractions built around it, so it 01:16.520 --> 01:19.220 can just be a drop in replacement for chroma. 01:19.220 --> 01:22.310 So we use chroma before, which was persistent on disk. 01:22.310 --> 01:26.090 So it's a more serious hardcore data store vice. 01:26.510 --> 01:29.360 We can just use that in lieu of chroma. 01:29.360 --> 01:34.070 And I'm going to show you how easy it is because the code is going to be almost identical, except when 01:34.070 --> 01:37.610 we get to that abstraction and a couple of other things. 01:38.240 --> 01:42.110 So we run the imports as before in this import. 01:42.110 --> 01:45.980 You can see I have commented out the import for chroma, so I can't cheat. 01:45.980 --> 01:52.160 Chroma will not be included here and instead we're importing face Facebook AI similarity search. 01:52.190 --> 01:58.760 It's worth noting that face comes in two variations a CPU variant and a GPU variant, depending on which 01:58.760 --> 01:59.930 you have Pip installed. 01:59.930 --> 02:04.490 And with this environment, of course, I have Pip installed the CPU variant, but for high performance 02:04.490 --> 02:11.750 projects running on GPU, you can run the GPU version to run at blazing speed, so we've imported vice 02:11.750 --> 02:13.880 and no errors, which is great. 02:14.150 --> 02:15.650 Set up some things. 02:15.650 --> 02:17.300 This is all the same code. 02:17.300 --> 02:19.220 We load everything in as before. 02:19.220 --> 02:25.970 We're going to get hopefully 123 uh, to change that to be Len chunks. 02:25.970 --> 02:28.310 So we see do we have 123. 02:28.340 --> 02:29.570 Yes we do. 02:29.600 --> 02:31.700 Let's check that we have the right metadata. 02:31.700 --> 02:33.590 We have the four types. 02:33.590 --> 02:35.840 So far nothing is different. 02:35.840 --> 02:36.980 This is all the same. 02:36.980 --> 02:38.000 But look at this. 02:38.000 --> 02:40.550 One line has changed here cunningly. 02:40.580 --> 02:44.060 Uh, I didn't need to draw quite so much attention to it. 02:44.060 --> 02:45.860 I think you probably spotted it already. 02:45.860 --> 02:49.850 So we used to say vector store is chroma dot from documents. 02:49.880 --> 02:53.720 And we passed in the chunks embeddings and the persist directory. 02:53.720 --> 02:58.580 And now we just say vice dot from documents same construct. 02:58.610 --> 03:00.380 We also pass in the chunks. 03:00.380 --> 03:01.970 We also pass in the embeddings. 03:01.970 --> 03:06.320 And of course we don't pass in a persist directory because vice doesn't persist on disk. 03:06.620 --> 03:09.980 Um, now these two lines here are also different. 03:09.980 --> 03:14.750 So the lower level ways to ask questions of the data store are different. 03:14.750 --> 03:20.300 So if you're going to be using Feist, it's useful to have this to hand so you know how you can query 03:20.300 --> 03:23.600 for things like the number of vectors and the dimensionality. 03:23.630 --> 03:27.530 I don't know if you remember how many dimensions that there were when we did this in chroma. 03:27.650 --> 03:32.630 Um, but it's the same number of dimensions we're coming up with here. 03:32.630 --> 03:36.590 We're using the same OpenAI embeddings to vectorize. 03:36.590 --> 03:40.550 So we're still using the same LLM to create the vectors. 03:40.550 --> 03:45.620 The thing that has changed is how we are storing those vectors, not in chroma but in Feis. 03:46.130 --> 03:50.540 Um, and so uh, no surprise there 123 vectors, one for each chunk. 03:50.540 --> 03:54.470 And they have these dimensions coming back from OpenAI. 03:55.190 --> 04:01.370 Uh, so the other thing that's changed has been this pre-work section was different in day four. 04:01.400 --> 04:04.010 I'll remind you in day four, the pre-work. 04:04.010 --> 04:05.660 Let's go and take a look at that. 04:06.080 --> 04:07.850 Uh, ba ba ba ba. 04:08.450 --> 04:14.270 Uh, the pre-work actually looks a bit simpler just plucking out the vectors, documents, and doc types 04:14.300 --> 04:15.890 from chroma. 04:15.890 --> 04:19.970 And you can look at the ways that you query chroma to get this data. 04:20.170 --> 04:22.810 Um, and this is how I did it for vice. 04:22.840 --> 04:25.840 There might be a tighter way to do that, but this seems simple enough. 04:25.840 --> 04:31.810 I just collect the same vectors, documents, doc types, colors, and I map it into the color that 04:31.990 --> 04:38.410 Plotly is expecting so that we can just plot the same, uh, diagram so we can visualize our vectors 04:38.410 --> 04:40.480 just as we did before in 2D. 04:40.600 --> 04:44.290 This code is identical with one tiny exception. 04:44.320 --> 04:45.670 See if you can spot it. 04:45.670 --> 04:47.290 That's the only change. 04:47.290 --> 04:52.570 I've changed the title from saying chroma to saying vice, but otherwise the code is the same. 04:52.750 --> 04:55.390 Uh, we will visualize our data store. 04:55.390 --> 05:00.250 And so here then are the vectors as they are represented in vice. 05:00.250 --> 05:04.660 And of course, as you would expect, it's the same vectorization approach. 05:04.660 --> 05:06.430 So it looks pretty similar. 05:06.430 --> 05:11.020 We're just using a different underlying technology as our vector data store. 05:11.260 --> 05:13.960 Uh, and we can of course represent that in 3D. 05:13.990 --> 05:21.190 And we're now looking at the 3D representation in vice and nice to see that actually this time we've 05:21.190 --> 05:24.370 got, it's uh nicely it's very nicely separated. 05:24.400 --> 05:25.060 There we go. 05:25.090 --> 05:26.560 Anyway, I could look at that all day. 05:27.160 --> 05:28.510 So you get that sense. 05:28.510 --> 05:31.930 And then the code to bring it all together is identical. 05:31.930 --> 05:33.310 I haven't changed this at all. 05:33.340 --> 05:38.020 Vector stored as retriever can be called on chroma, or it can be called on face. 05:38.020 --> 05:39.190 And it is the same. 05:39.190 --> 05:39.940 So there we go. 05:39.970 --> 05:40.750 We run it. 05:40.780 --> 05:41.740 No errors. 05:41.740 --> 05:42.370 It's fine. 05:42.370 --> 05:45.550 Let's go straight to bringing up Gradio. 05:45.820 --> 05:49.180 Uh, and here we go. 05:49.210 --> 05:51.820 Here is our Gradio interface. 05:52.150 --> 05:58.690 Um, now, one of the things that I meant to show you last time that I now take this chance, and you 05:58.690 --> 06:04.600 can see the same will apply in chroma as well, is I can take questions like, uh, like you remember 06:04.600 --> 06:07.060 last time I asked, what did Avery do before? 06:07.060 --> 06:11.710 And it was cunning enough to look up the right context, even though I didn't say Lancaster and I spelled 06:11.710 --> 06:17.500 it with a lowercase a, we can take that even further, and I can say something like, what did? 06:17.500 --> 06:21.370 And I can spell Avery's name wrong like that. 06:21.400 --> 06:24.400 What did Avery do before? 06:25.870 --> 06:27.670 Um, uh. 06:27.670 --> 06:32.920 And I can run this piece of code like it is and have a look at what's come back. 06:33.130 --> 06:40.390 It has correctly identified that I'm talking about Avery Lancaster, and it looked up her HR document 06:40.390 --> 06:45.580 and it's again correctly identified that she worked at Innovate Insurance Solutions. 06:45.670 --> 06:48.760 Uh, we can might as well go in and just quickly check. 06:48.760 --> 06:51.700 Since it's set, it will go to the employees documents. 06:51.700 --> 06:53.500 We find Avery Lancaster. 06:53.500 --> 06:54.580 Here it is. 06:54.580 --> 06:56.560 Here is her HR record. 06:56.560 --> 06:57.700 Let's see what she did. 06:57.700 --> 07:03.310 She was indeed at Innovate Insurance Solutions before founding insurer Elm. 07:03.550 --> 07:06.010 Uh, it's good to see it didn't invent that. 07:06.340 --> 07:16.270 Uh, so, um, uh, the bottom line is that it's this this gives you a real sense that it didn't just 07:16.270 --> 07:17.530 do text matching. 07:17.530 --> 07:23.230 It didn't even, uh, get fussed by the wrong case that it was in lowercase and uppercase. 07:23.230 --> 07:30.160 It actually was able to identify that Avery spelt wrong, has the same meaning as Avery spelt with a 07:30.190 --> 07:31.870 Y Avery Lancaster. 07:31.870 --> 07:39.220 It has enough common sense to recognize that that is very plausibly what we were after. 07:39.220 --> 07:44.890 And again, the reason is because when it turned this into a vector and it put it in the vector and 07:44.890 --> 07:51.640 it looked in the vector data store for the vectors that were close to that, it found the Avery Lancaster's 07:51.820 --> 07:56.050 HR record as being something that was close to that in the data store. 07:56.050 --> 08:00.430 So I think it's fascinating to see that you can spell things wrong and it still works. 08:00.430 --> 08:06.580 And it's such a clear example of how much better using the vector lookup approach is than using the 08:06.580 --> 08:11.080 brute force technique that we used in the first session, which obviously would have failed hopelessly 08:11.080 --> 08:12.130 with this test. 08:12.190 --> 08:17.890 So I will let you prove to yourself that the same test will work if you use chroma, of course, and 08:17.890 --> 08:19.720 you can try out both, but you will see. 08:19.720 --> 08:23.350 I hope that chroma and vice have both worked very well indeed. 08:23.440 --> 08:26.020 But most importantly, you've seen that. 08:26.050 --> 08:33.220 True to what I explained before, Lang makes it super simple to switch out different vector data stores 08:33.220 --> 08:37.510 behind the scenes and use the same plumbing for your Rag workflow. 08:38.080 --> 08:39.760 All right, back to the slides.