So I know what you're thinking.
You're thinking, what's going on here?
We're on day five.
We're on day five of week five.
Why does he have a Jupyter notebook open?
That's called day 4.5.
And it's because it's just a little quick deviation on day four to show you genuinely how straightforward
it is to switch out the vector data store for another one.
And since this is super common out there, used a lot, I thought it was a great one to show you and
to use it both to illustrate a different vector data store, and also to show you how easy it is to
work with these abstractions.
Now, something to to explain is that this isn't really a vector data store.
It stands for Facebook AI Similarity Search, and it's a highly used, very common open source library
put out by Facebook that lets you do quick search for vectors close to other vectors, and as such,
it is often used as a kind of in-memory vector store.
It doesn't persist on the disk, but you can use it to plonk in vectors and to retrieve other vectors
that are similar, and you can also just use it as a library to do similarity search, to be looking
for vectors close to other vectors, and Langshan has the right abstractions built around it, so it
can just be a drop in replacement for chroma.
So we use chroma before, which was persistent on disk.
So it's a more serious hardcore data store vice.
We can just use that in lieu of chroma.
And I'm going to show you how easy it is because the code is going to be almost identical, except when
we get to that abstraction and a couple of other things.
So we run the imports as before in this import.
You can see I have commented out the import for chroma, so I can't cheat.
Chroma will not be included here and instead we're importing face Facebook AI similarity search.
It's worth noting that face comes in two variations a CPU variant and a GPU variant, depending on which
you have Pip installed.
And with this environment, of course, I have Pip installed the CPU variant, but for high performance
projects running on GPU, you can run the GPU version to run at blazing speed, so we've imported vice
and no errors, which is great.
Set up some things.
This is all the same code.
We load everything in as before.
We're going to get hopefully 123 uh, to change that to be Len chunks.
So we see do we have 123.
Yes we do.
Let's check that we have the right metadata.
We have the four types.
So far nothing is different.
This is all the same.
But look at this.
One line has changed here cunningly.
Uh, I didn't need to draw quite so much attention to it.
I think you probably spotted it already.
So we used to say vector store is chroma dot from documents.
And we passed in the chunks embeddings and the persist directory.
And now we just say vice dot from documents same construct.
We also pass in the chunks.
We also pass in the embeddings.
And of course we don't pass in a persist directory because vice doesn't persist on disk.
Um, now these two lines here are also different.
So the lower level ways to ask questions of the data store are different.
So if you're going to be using Feist, it's useful to have this to hand so you know how you can query
for things like the number of vectors and the dimensionality.
I don't know if you remember how many dimensions that there were when we did this in chroma.
Um, but it's the same number of dimensions we're coming up with here.
We're using the same OpenAI embeddings to vectorize.
So we're still using the same LLM to create the vectors.
The thing that has changed is how we are storing those vectors, not in chroma but in Feis.
Um, and so uh, no surprise there 123 vectors, one for each chunk.
And they have these dimensions coming back from OpenAI.
Uh, so the other thing that's changed has been this pre-work section was different in day four.
I'll remind you in day four, the pre-work.
Let's go and take a look at that.
Uh, ba ba ba ba.
Uh, the pre-work actually looks a bit simpler just plucking out the vectors, documents, and doc types
from chroma.
And you can look at the ways that you query chroma to get this data.
Um, and this is how I did it for vice.
There might be a tighter way to do that, but this seems simple enough.
I just collect the same vectors, documents, doc types, colors, and I map it into the color that
Plotly is expecting so that we can just plot the same, uh, diagram so we can visualize our vectors
just as we did before in 2D.
This code is identical with one tiny exception.
See if you can spot it.
That's the only change.
I've changed the title from saying chroma to saying vice, but otherwise the code is the same.
Uh, we will visualize our data store.
And so here then are the vectors as they are represented in vice.
And of course, as you would expect, it's the same vectorization approach.
So it looks pretty similar.
We're just using a different underlying technology as our vector data store.
Uh, and we can of course represent that in 3D.
And we're now looking at the 3D representation in vice and nice to see that actually this time we've
got, it's uh nicely it's very nicely separated.
There we go.
Anyway, I could look at that all day.
So you get that sense.
And then the code to bring it all together is identical.
I haven't changed this at all.
Vector stored as retriever can be called on chroma, or it can be called on face.
And it is the same.
So there we go.
We run it.
No errors.
It's fine.
Let's go straight to bringing up Gradio.
Uh, and here we go.
Here is our Gradio interface.
Um, now, one of the things that I meant to show you last time that I now take this chance, and you
can see the same will apply in chroma as well, is I can take questions like, uh, like you remember
last time I asked, what did Avery do before?
And it was cunning enough to look up the right context, even though I didn't say Lancaster and I spelled
it with a lowercase a, we can take that even further, and I can say something like, what did?
And I can spell Avery's name wrong like that.
What did Avery do before?
Um, uh.
And I can run this piece of code like it is and have a look at what's come back.
It has correctly identified that I'm talking about Avery Lancaster, and it looked up her HR document
and it's again correctly identified that she worked at Innovate Insurance Solutions.
Uh, we can might as well go in and just quickly check.
Since it's set, it will go to the employees documents.
We find Avery Lancaster.
Here it is.
Here is her HR record.
Let's see what she did.
She was indeed at Innovate Insurance Solutions before founding insurer Elm.
Uh, it's good to see it didn't invent that.
Uh, so, um, uh, the bottom line is that it's this this gives you a real sense that it didn't just
do text matching.
It didn't even, uh, get fussed by the wrong case that it was in lowercase and uppercase.
It actually was able to identify that Avery spelt wrong, has the same meaning as Avery spelt with a
Y Avery Lancaster.
It has enough common sense to recognize that that is very plausibly what we were after.
And again, the reason is because when it turned this into a vector and it put it in the vector and
it looked in the vector data store for the vectors that were close to that, it found the Avery Lancaster's
HR record as being something that was close to that in the data store.
So I think it's fascinating to see that you can spell things wrong and it still works.
And it's such a clear example of how much better using the vector lookup approach is than using the
brute force technique that we used in the first session, which obviously would have failed hopelessly
with this test.
So I will let you prove to yourself that the same test will work if you use chroma, of course, and
you can try out both, but you will see.
I hope that chroma and vice have both worked very well indeed.
But most importantly, you've seen that.
True to what I explained before, Lang makes it super simple to switch out different vector data stores
behind the scenes and use the same plumbing for your Rag workflow.
All right, back to the slides.