Well, I hope you're eager with anticipation for this session in JupyterLab as we finally get to see
vectors firsthand.
So hopefully you are in the week five folder, you're in day three, and you're going to be following
along with me more than ever.
It's important for this one because it's going to be visual, and you're going to get a real sense of
what's going on.
So this is just a duplicate of the previous day with more added.
So we begin with some imports, and now we've added to the imports to include some special things that
we'll be working on.
Today.
We're going to be importing OpenAI embeddings, which is part of Lang Lang chain OpenAI package.
We're going to be importing chroma from lang chain chroma.
Um, and then there's a couple of things we're going to be importing so that we can have some fun and
actually visualize these vectors, something called t-SNE, which we'll talk about, and then something
from Plotly that is going to give us some nice diagrams.
Let's do all of that importing.
Okay, uh, let's run the constants.
Bring in our environment variables.
This, of course, is now super familiar to you.
We're using the loaders to load in our knowledge base.
We're using the text splitter to create our chunks, of which there are 123.
And there's chunks for each of our four folders our contracts, products, employees and company.
So I wanted to take one more moment to remind you about embeddings and auto encoding LMS that we've
been talking about auto regressive LMS in most of the of the course so far.
This is the first time we're going to look at auto encoding LMS that take an entire input and use it
to create one output, in our case, a vector.
And an example of an auto encoding alarm is Bert.
We're going to be using this one here.
OpenAI embeddings.
Let's run that.
Okay.
So chroma is we're about to create our chroma data store.
And I've just put in this little section here to delete or empty the database if it's already there.
Otherwise every time you run this code it adds in the vectors another time a sort of duplicate set of
vectors and things get confusing.
So this helps to refresh the database.
If you've already run this once and you can imagine I've already run this once or twice, so this is
a useful thing for me.
We will delete it.
It's sitting in this folder here vector db um, which is just a db name, a constant that I set at the
top to show you that I said DB name is vector DB.
It can be whatever you want and it's based on SQLite.
So if you go in here you'll see there's some SQLite stuff going on.
So it's just got a bunch of different files sitting within the name of whatever database name you chose
to give it.
In this case vector db.
Okay, so it's time to create our vector database and populate it with our vectors.
And I'm going to have another of those gradio moments here.
You might think that the process of vectorizing a bunch of documents, turning them into vectors and
storing them in a vector database is something that would be reasonably sophisticated and would be,
I mean, at least several cells.
But you'll find, of course, that we can do it in one cell and we can do it in one line of code.
And that is, of course, thanks to Lang Chain putting a lot of great glue code around this.
It wouldn't actually be that hard to do it ourselves manually to iterate through, to turn each of these
embeddings into each of these chunks into vectors, and then store it ourselves in chroma, it would
perhaps just be 4 or 5 lines.
Um, but it's that much simpler if we can just do it in one like this.
And the other great thing about this is that if you try different vector data store, it's the same
thing just with a different vector store imported from Lang Chain, and you might want to give that
a try yourself.
But let me run this one line and it has run.
And the vector store has been created with 123 documents.
And that's comforting to know, because 123 was the number of chunks that we had.
So it's good that there is indeed a same, uh, number of documents in our vector data store.
Um, and the way that we found that, by the way, is that the vector store object that came back from
creating Chrome, we passed in, of course, the chunks, the embeddings, which was the OpenAI embeddings,
and then just simply the name of the database.
Those were the three things we provided, which makes total sense.
And that's all that we needed to create our vector store.
And that vector store we can call underscore collection dot count to get the number of documents in
the vector store.
So let's just find out how many dimensions these vectors have so we can get a vector from the collection
with this.
So so so the collection is just what you get if you call this underscore collection um uh attribute
from Vector Store.
So we can then call dot get.
We can pass in a limit.
We only want one.
We want it to bring back the embeddings which is the vector itself.
Um, and then we will just have a look at that and let's have a look at how large it is.
It is 1536 dimensions.
So 1536 numbers make up this chunk.
So let's have a look at this ourselves so we can just just print it.
Let's just print sample embedding see what it looks like.
Wowza.
It's a lot of numbers.
So it's a bunch of numbers.
It's meaningless to to you and me.
Uh, but these numbers in some way.
We see these numbers in some way reflect the meaning of the chunk that they're associated with.
And we'll see.
We'll associate it with a chunk in just a second.
So it's a ton of numbers.
It's 1536 numbers, which we can interpret as representing a coordinate in 1536 dimensional space.
Um, and that that coordinate is chosen such that other other vectors that have a similar coordinate
that are close in vector space should have similar meaning.
That's the whole idea behind this.
So let's take a moment to visualize this.
That's going to be a nice, uh, get rid of some of these comments.
This will be some nice, uh, um, uh, a nice way to, to be able to really investigate what's going
on behind the scenes.
So you can call Collectiongetty and ask for the embeddings, the documents and the metadata, like this.
And once I've done that, I can put the vectors.
I can take the embeddings and put them into an array of vectors, and I can take the the doc types and
put that into something called doc types.
And then I'm going to make some some colors up as you will see.
So this is just some pre-work to get ready.
Now there's one problem that we have is that, uh, as human beings have a deficiency, that we have
a problem visualizing anything in more than three dimensions.
So visualizing something in 1536 dimensions is going to be very challenging indeed.
Um, but luckily there are various techniques we can use to do what they call projecting down or trying
to, uh, reduce the dimensions to only, let's say, two dimensions in a way that does the best possible
job at separating things out to stay faithful to the, the, uh, multi-dimensional representation.
So trying to do it in a way that is going to have things that are further apart in all of these dimensions
will still be fairly far apart, even when it's projected down to two dimensions.
That's the idea.
There are various techniques for doing it that are not going to go into, but the one that we're going
to use is extremely common.
It's called t-SNE, which stands for T-distributed Stochastic Neighbor embedding, and is one of those
ones that you can Google or ask ChatGPT if you want an explanation behind it.
Um, so we pass into it, how many dimensions do we want?
We want it projected down from the thousand and whatever to two dimensions.
And this random state is a way of setting its random seed so that each time we call this, we get the
same thing so that we can reproduce this.
Um, and then you can get your reduced vectors just by calling the fit transform method on this t-SNE
object.
Um, what I'm then doing is I'm using the fabulous library plotly to make a nice scatter diagram.
Um, And, you know, this is all sort of this kind of code that you can copy and paste and reuse as
you wish.
But I've got some nice things like I'm making the markers have different colors based on the colors
that we set here for the different document types, and also got some pop up text which is going to
actually have a fragment, the first 100 characters from the chunk itself.
So what we're hoping to see here is we're hoping to see how do the documents look in the vector database.
And then for each of these different vectors, we're going to color it by what kind of document is it.
And it's going to have hover over text.
So we can actually read a little bit of the fragment of text that this chunk represents.
So we'll see if this works.
That would be quite cool.
Of course I know it works.
I've already tried it.
Here it is.
Okay.
So what are we looking at.
So we're looking at a visualization of the multi-dimensional vectors projected down to 2D for us, there's
no particular meaning of the x axis and the y axis.
It's just the best possible way of spreading out the different points.
Now there's a few things to notice.
The green dots that you see here are representing employees.
The red dots are representing.
And when I say employees I mean they're representing chunks of text plucked out of employee documents.
The red things are chunks of text coming out of contracts.
The blue are from product documentation and the yellow is from about documents.
Now, there's something a bit magical that you have to to appreciate, to realize at this point when
the OpenAI embeddings, when when it was vectorizing these chunks of text, it did not know the document
type.
We didn't tell it the metadata.
We didn't we didn't tell it that summer employees, summer products and summer contracts.
We just gave it the chunk of text and said, turn this into a vector.
And so that's all it had.
Um, and so it's kind of magical that based on those chunks, these chunks have been separated out in
vector space and they occupy different distinct regions in space.
Um, because they're somewhat similar to each other in terms of the content, the meaning of what they
represent, the employees are all kind of in the same general territory.
The, the, the contracts are over here and the products are here.
You may notice that some of the contract information appears to be more a sort of a, uh, in the same
vicinity as the products.
And that might be surprising for a moment.
It might look like maybe something's gone wrong.
But no, if you hover over these chunks, you'll see that this is the particular chunk of text in a
contract, which describes the key features that that client has signed up for.
And you can see there it says type the text in there.
Sorry if I if I hover for a moment where you see that second line where it says text, that is the extract
from the chunk that has been put in that location in vector space, and you'll see that it starts features
one AI powered matching.
And let's find another one here.
Uh, AI powered risk assessment features.
It's the same thing.
Um, so what we're seeing here is that within contracts, there is some information that is more like
functionality.
And that functionality lives in vector space in a similar kind of space to other product information.
So, uh, hopefully that makes sense.
It's again, it's almost spooky how good it is at kind of associating the right location in space to
the meaning behind the document.
And the final point I'll mention, and I might be I might be putting, uh, putting taking this too
far, but it really seems to me like this is what's going on.
You'll notice that the three documents that are about the company.
Uh, these three here are kind of in the center.
They're right in the middle between all these different documents.
And I think that's because it kind of encompasses some information that pertains to the employees and
the contracts and the, the, the, the product as well.
Uh, it's kind of central information that that is related to everything.
And that's why it's somehow in a sort of place that is in the middle of all of the rest of the information.
Um, so I find that fascinating.
I might be reading too much into it, but it does seem that way to me.
Uh, and one of the things that you can do is just experiment with putting any chunks of text that you
want into the vector database.
You can just imagine you can just go back and add in other documents, uh, into the documents that
we load here.
Either you can just specify a file directly, or you can just put in some more text or just add text
into the existing documents and then see where they end up in vector space, and use that as a way of
really getting to appreciate, uh, the the way that OpenAI embeddings is able to understand the meaning
behind a different chunk of text and position that in a way that is, that is closest to other things
with similar meaning.
And that's that's the whole idea.
Well, this 2D representation was was really cool.
Uh, is there anything better than a 2D representation?
Of course there is.
There's a 3D representation and that's what we're going to do next.
Uh, we're going to try 3D because it's just as simple as turning that number two into a number three,
and we will then be able to visualize these vectors in 3D.
It wasn't quite that simple.
I also had to put a 3D in here, and I had to mess around with with some other vectors and all of that
stuff, but it's pretty similar.
And I had to change the title from 2D to 3D as well.
Anyway, let's see how this looks.
Here it is.
Um, so these are our vectors projected not all the way down to 2D, but now to 3D.
Um, and the first thing you might notice is that actually, this is one of those rare times, uh,
a bit like movies that, uh, 3D isn't necessarily better than 2D.
Uh, that, uh, it does look a little bit more of a of a jumble.
It's a bit harder to to see.
It does look again like the yellow is in the middle.
Uh, and green, red and blue are clearly separated out.
But there are some, um, that have combined, as you would expect, as we saw before.
But the nice thing about this is we can, uh, actively, uh, we can interactively play with this.
We can rotate it like this.
Isn't that something?
And we can use this to get a better sense of how they are laid out.
And it does give you a nice sense that there is a distinction between them.
Um, but I do have to admit that it's not perhaps as clear as the 2D representation.
Uh, but nonetheless, it looks great.
If we hover, you get to see the meaning behind the points.
Of course, as before, the yellow is the about, which does seem to be somewhat in the middle of everything,
uh, consistent with the point I was making before, uh, this blue one here is a bit of an outlier.
Um, and, uh, yeah, you, uh, you can certainly have fun with the 3D representation, although perhaps
the 2D one is a bit more clear in some ways.
Anyway, that concludes our playing with vectors, but there's a big takeaway, which is now it's over
to you, not just about looking at these diagrams and inspecting the different chunks of text, but
adding your own chunks of text.
Simple way to do it is just to put things documents into the knowledge, into the knowledge base directory.
You can just do that and it will just work right away.
But you can also play around with the text loader classes to add in chunks using lang chains code instead
of just dropping a document in there.
But when you've put your chunks in, see where they end up in vector space and get a sense of how that
works.
Or you can replace the whole knowledge base directory, rename that to something else, create a new
directory and just fill it with some some very simple things just so you can experiment with seeing
how documents get put into different locations in vector space.
And that's super important homework because it's going to give you a good basis for the next part,
which is when we get to rag for real.
First, let's go back to the slides.