And so now the time has come to talk about the most crucial aspect of Rag, which is the idea of vectors.
If you're already familiar with vectors and vector embeddings, hang on in there.
I'm going to go fairly quickly through this.
You may pick up a thing that you didn't know about as I as I explained this.
So first of all, there's an important bit of background information, which is that we've been talking
about a bunch of different LMS through this course, but almost all the LMS we've been talking about
have been one kind of LM called an autoregressive LM.
And there is, in fact a completely different category of LM known as Autoencoding.
So what's the difference?
Autoregressive LMS or LMS, which are given a past set of tokens and they are required to generate the
next token in the sequence a future token given the past.
And they keep doing that repeating creating the next token.
Given the history of tokens, that's an autoregressive LM.
And of course it's all the rage.
It's most of the ones that we work with.
And of course it's it's GPT four and Claude and Gemini and so on.
There's also these types called autoencoding, and they take a full input that represents both the past
and the present and the future.
It's a full bit of input, and they create one output that reflects the whole input.
And so to make that real, there's some obvious examples.
Sentiment analysis where you take in a sentence and say if it's positive or negative.
Classification where you take a sentence and put it into buckets.
Both things that we did actually explore briefly with the Huggingface pipelines a couple of weeks ago.
And those are both examples of Autoencoding llms.
There is another way that they are used as well, and it is to create something called a vector embedding.
And that's what we're going to be talking about today.
So a vector embedding is a way of taking a sentence of text or a bunch of different things, but usually
a sentence of text.
And turning that into a series of numbers, a series of numbers that in some way reflect the meaning
behind that text.
And we'll go through exactly what that means in just a second.
It sounds a bit abstract right now, but the idea is that take some text, convert it into numbers,
and those numbers you could think of as representing a point in space.
So if we took some text and we turned it into three numbers, you could think of that as being like
an X, Y, and z that would represent exactly where abouts something is located in space.
As it happens, usually when you do this, it gets converted into hundreds or thousands of numbers.
So it represents a point in like 1000 dimensional space.
And that's kind of hard for us to visualize because we can only think in three dimensions, but it's
the same idea.
It's reflecting a point in space, and that point is meant to represent in some way the meaning behind
the text that went in to generate that are examples of auto encoding.
Llms are Bert from Google.
You may remember we actually mentioned Bert right back, I think in the first week.
Uh, so so Bert's been around for a while.
Um, there's also open AI embeddings from OpenAI, and that's actually the auto autoencoder model that
we'll be using this week for our Rag projects.
So let me just talk a bit more about what we mean by meaning.
So first of all, you can use you can create one of these vectors for a single character, for a token
or a bunch of characters, for a word, for a sentence, for a paragraph, for an entire document,
or even for something abstract.
Like in my company, Nebula, we create vectors for things like talent and jobs and things like that.
Often when you're working with these vectors, they will have hundreds or even thousands of dimensions,
will be like a thousand numbers that represent this one block of text.
And now I've said a few times that these numbers reflect the meaning behind the inputs.
What exactly does that mean?
So to put simply, one of the things it means is that if you have a bunch of paragraphs of text that
all end up mapping to similar points in space that are close to each other.
That should mean that these blocks of text have similar meaning.
They don't necessarily need to contain the same words.
They could be completely different words, but their meaning is the same.
They will be close to each other in vector space, so things close to each other in when they're turned
into numbers should mean similar things.
And that's the basic idea behind this.
There's also some more sophisticated ideas behind this, including this point that you can do what's
what's sometimes called vector math behind the meanings of these things.
And there's this example that's very often given it's been around for a long time, this example.
And you may well have heard of it before, and it says, supposing that you have the word king, and
you took the word king, and you used one of these vector encodings to find the the point in space that
represents the word king.
And you also find the vector that reflects the word man.
And the vector that reflects the word woman.
And you take the word king and you subtract man from it, which means you kind of move backwards in
the direction of man and you add woman, which means that you move forwards in the direction of woman.
What you've effectively done is you've taken the concept, the meaning of king, and you've said, I
want to replace the man with woman in this meaning king.
And somewhat remarkably, if you do this, you do actually end up in the position in vector space,
which is the same position as the position for the word queen.
So it really does seem that if you take the word king, the meaning of the word king, and you replace
the man aspect of it with woman, you're then at something which reflects the meaning of the word queen.
And so it's in that sense that these vectors really seem to reflect the meaning behind the words they
represent, both in terms of similar words being close to each other and the ability to carry out this
kind of vector math that allows you to understand the relationship between concepts.
So what's this got to do with rag?
Here's where it all comes together.
This is the big idea behind Rag now.
So this is the same diagram we had before.
But there's going to be a little bit more going on.
At the top we've got a new box called encoding LM.
This is something which is able to take some text and turn it into a vector.
And at the bottom we have something called a vector data store.
It's like the data store we had before the knowledge base.
But now along with text, we can also store the vector that represents that text, the vector that represents
the meaning of that text.
All right.
So here's what we do.
In comes a question from the user.
The first thing we do is we take that question and we turn it into a vector sometimes called a vectorizing.
So supposing the question was who is Amy Me.
Lancaster.
We take who is Amy Lancaster, and we turn that into a vector that reflects the meaning of the question,
Who is Amy Lancaster?
You can probably imagine what I'm going to say next.
What we then do is we go to the vector database and we say, tell me what information is in this vector
database where the vectors are close to the vector for who is Amy Lancaster?
So look at all the different documents we've got in there.
We've turned them all into vectors.
Some of those vectors will be close to the question who is Amy Lancaster?
Give me those vectors and give me the the original information, the text that was turned into those
vectors.
And presumably it's extremely likely that the actual air document for Amy Lancaster is going to be located
somewhere close to the vector, who is Amy Lancaster?
And so when we get that information, we quite simply take that text.
And just like before with the toy example, we shove that in the prompt to the LLM, we get back the
response, presumably taking advantage of the extra context.
And that's what goes back to the user.
So it's just like the toy example, except we're using a much more powerful technique for looking up
the relevant data, using vectors as a way of understanding which of our bits of knowledge have the
most similar meaning to the meaning of the question.
Well, that's really all there is to it.
And that's a wrap for this week, because it's next time that we're going to put this into action and
actually see vectors in databases.
We're also next time going to start looking at something called Lang Chain, a wonderful, wonderful
framework which is designed to make it easy to build these kinds of applications.
We could do it all the manual way.
We could actually create vectors and store them in vector databases using various APIs.
But Lang Chain makes it super simple, as you will see.
And it's going to be a bit like the Gradio experience, where in just a couple of lines of code, we're
going to be doing very powerful things.
So I'm excited about it.
I hope you are too.
And I'll see you then.