And at last the time has come to see rag in action.
After all of this talk, and here we are.
We're in, of course, the week five folder in JupyterLab.
We're looking at day four, the notebook, and it is, of course a duplicate of day three with more
added on.
Since we're still solving the same problem of a knowledge worker for our fictitious insurance tech company
in serum.
Uh, and we will start with the usual imports as before.
And now we have some imports for Lang chain, and I have sneakily added in two new imports from Lang
chain memory.
We're importing conversational buffer memory.
And from Lang chain chains we're bringing in conversational retrieval chain.
And these are two of the abstractions that I mentioned before.
Now the astute amongst you will have noticed that the third abstraction is also lurking in here.
I just already imported it in one of our previous lectures without mentioning it, but here it is.
Chat OpenAI is already being imported as part of from Lang chain.
OpenAI, uh, we've only been using so far the OpenAI embeddings, but this time we're going to bring
chat OpenAI into the mix.
Okay, so I better run those imports.
Otherwise we're not going to get very far.
So then we do some constants.
We load our environment variables.
And now you're quite familiar with this.
But we go through and we bring in our documents from the knowledge base directory over there.
And now we're going to bring in the text chunks the let's see how many.
But I do believe it's 123.
Yes, 123 text chunks.
And they are employees, products companies and contracts.
And now we're going to again put them into our vector database.
We delete and recreate the vector database.
And we see that each vector has 1536 dimensions.
Hard for us to visualize, but we can handle it in 2D.
So that's what we do.
And there they are.
And we can also say let's see it in 3D as well.
This is a little bit gratuitous.
I didn't need to go through and rerun all of this, but I do love seeing these these diagrams.
All right, so here, I didn't lie to you.
It really is as simple as these four lines of code.
We first create the new LM abstraction.
We the chat OpenAI.
We're going to put that that thing we've imported for a while.
We're going to put it to use finally.
And you supply a temperature and a model name, uh, memory.
We create the conversational buffer memory passing in, as I mentioned before, the key and saying we
want it returned in the form of a, of a list.
Uh, we take our chroma vector store and we call as retriever to sort of wrap it in this abstraction,
the retriever, which is needed by Lang chain.
And that is where we now get to when we create the conversational retrieval chain.
And we simply pass in the LM, the retriever and the memory.
That's all it is.
Let's run that.
Okay.
So we ran it.
Perhaps a slight anticlimax.
I'm not sure what you're expecting, whether you thought maybe we were going to get suddenly rag appearing
in front of us.
We have to actually call it.
We have to do something to make use of it.
So what we're going to say is we're going to say, um, um, query calls.
Uh, can you describe in short film in a few sentences, a nice we will start simple.
All right.
And this is what you say.
You say result is conversation chain.
The thing that we've just created and we call the method invoke.
And invoke takes a dictionary which has question as a key.
Did I spell that right?
Yes.
Question.
And we have to put in our message query.
And there we have it.
And then we're going to print result.
This should be something under the key of answer in that result.
So this then is the final piece of code that we put together to try and make use of our Rag pipeline.
So what do we think is going to happen?
It's going to take that query.
It's going to turn that into a vector.
It's going to look that up in our Chrome data store.
It's going to find relevant chunks.
And I say chunks plural.
And we're going to come back to that.
So it's going to find relevant chunks.
And it's going to drop them into the prompt and send that to OpenAI.
Uh, it's going to send it to GPT four mini because we've specified that here.
And then with what comes back, it's going to package it up and put that in the answer key.
Let's see if this works.
There we go.
We've just run our first Rag pipeline front to back in Shoreham.
Innovative insurance tech firm founded by Avery Lancaster, a name we know well at this point and so
on.
And it's got bits of information.
And I will leave this as an exercise for you to play around with, but you'll see that it's got that
out from various documents, I think probably all from the about from the company section.
Uh, but, uh, I hopefully you'll see that it has retrieved that from various chunks of information.
All right.
Well, wouldn't it be nice?
Do you know where this is going?
Wouldn't it be nice if we could package that up into a beautiful user interface, so that we could actually
use it through a chat UI?
And of course, Gradio makes it super simple as well.
We know at this point all we have to do is create a chat function in the format that Gradio expects.
That takes a message and a history.
So what I've done is I've taken exactly the line that we just wrote, and I've put it here, and then
I return exactly what we're expecting.
Now, you might see something curious about that, give you a moment to look at it and see anything,
anything strike you as odd.
Well, the one thing that might potentially strike you as odd is that we don't actually do anything
with this history parameter.
Uh, we ignore it completely.
And the reason we ignore it is because, of course, Lang Chain already handles history for us.
So even though Gradio has this chat UI that calls that sort of maintains the history in the user interface,
and then calls back every time with the with the full chat history.
We don't need that because because Lang has already given us this, this memory, and it's already keeping
track of the conversation so far.
All it needs to know is the new message and the new answer.
So anyway, I'm rerunning the cell here.
I actually already reran it, but I'm rerunning it to clean out the memory.
So we're starting absolutely fresh.
We call that we bring this up and now we can chat.
Hi there.
Hello.
How can I assist you today?
What is insurance?
Um.
There we go.
No surprise.
And now we can do something sneaky.
We can say something like.
What did Avery do before?
And now the reason I'm, uh, I'm bringing this up is that there's a few things that I want to surface
in a question like this.
So first of all, to state the obvious, our brute force solution before our toy version of Rag was
able to look at Lancaster as a last name and search for that in documents, which was pretty hopeless.
And if we tried the word aviary, then it failed on us.
So it's of course interesting to try it here.
Secondly, I have intentionally put aviary with a lowercase a, because anything that's doing a kind
of text search is going to get that wrong because aviary is uh, is spelt differently.
So, uh, it'll be interesting to see whether it can handle the fact that we've not used the right case.
And then thirdly, I'm sort of taking advantage of this memory idea because I'm referring to what she
did before.
Meaning what did she do before she founded in Elm.
And we'll see whether the model is has a good enough sense of what's going on to be able to keep the
context, both retrieve relevant information about aviary and what she did before.
That will need to come from her employee record.
Uh, and also, uh, just answer the question in a coherent way.
Let's see.
Before founding in Shoreham, Avery Lancaster worked as a senior product manager at Innovate Insurance
Solutions, where she developed groundbreaking insurance.
Prior to that, business analyst focusing on market trends.
So I will leave it as an exercise for you to check her employee record and make sure that you're satisfied
that it is indeed correctly finding it and getting the right background on Avery, uh, as a fun thing
to try.
Um, and also, of course, try other difficult questions.
Uh, what does, um.
Uh.
Let's see.
Calm do that was the car, um, uh, product.
Um.
Or how about, um, let's ask it differently.
Let's say does insurance offer any products in the car in the auto Assurance space.
Let's give it a nice, tricky question.
And there we go.
Yes.
Insurance offers Calm, which is a portal for auto insurance companies.
So it's able to even though I didn't use the word calm or even the word car, it was able to find the
relevant document, the relevant chunk, and answer the question in an accurate way.
So that is your first experiment with Rag.
I hope you will now try this.
I hope you will investigate, ask difficult questions, find out if you can break it or get it to give
wrong information, or go off the rails and stretch it to its limits.
Next time, amongst a few other things, we'll talk about some of the ways, some of the common problems
that you can get with these kinds of prompts, and how you can debug and find out more about what's
going on under the covers.
But I hope you enjoyed your first end to end rag pipeline built for our fictional insurance tech company
in Shoreham.
See you next time.