It's always welcome back to JupyterLab, my favorite place to be.
And now we are, of course in the week five folder.
And I'm looking here at the day one Jupyter notebook in week five, which is going to be our home for
today, where we look at building our own DIY rag implementation in a very simplistic way, but it will
give you some real insight into how this works.
So first of all, let me tell you about our company in Shoreham, an insurance tech company.
We have got access to their company shared folder and it's called Knowledge Base and I have dropped
it right here in the same week five folder for your perusal.
If we open it up, you'll see that it contains four folders company contracts, employees and products
representing four different um divisions or four different areas of the company.
Um, now, one thing I will mention, as you may have guessed perhaps, is that you'll see a fair amount
of company data here, which is all completely fictitious.
This was, of course created by an LLM.
I took took my own exercise from a couple of weeks ago, wrote a data generator and used it to craft
some of this data and keep relationships between some of the documents and so on.
I did add in a little couple of tweaks here and there myself, but almost all of this is generated by
either GPT four or by Claude.
So just to have a quick peek at this in company, there's a few things here about the company.
I think these are quite short documents if these are in the form of markdown documents.
So if I can open it in this way, then we get to see some nice, um, write ups about the insurer Elm.
It was founded apparently by Avery Lancaster in 2015.
Remember, Avery's name will be.
We'll be meeting her a few times.
Um, and if you want to see the original documents, of course.
They look like the markdown that we are familiar with.
Um, then if we go back into knowledge base in contracts.
We've got some more substantive information about contracts, which includes the contract terms renewal,
the features that are included in that contract that's going to come back and be important later.
So so keep note of that.
Um, and some other things.
Uh, if we go back into knowledge base again and look at employees, these are the employee HR records
of the different employees.
And even the CEO, Avery Lancaster herself has an HR record that you'll see right here.
Uh, so all everyone has their their HR documents, um, and then products contains a write up of the
different offerings from insurer Elm, including car Elm, uh, for auto insurance, uh home Elm and
and so on.
You can look through these documents, but they are wonderful in how real they are.
All of course courtesy of Frontier Models.
A nice product summary right there with even with a roadmap.
Love it.
Uh, okay.
Back then to day one.
So what are we going to do?
We are going to write something now that is going to be asking questions about this company, and we
are going to artificially look up and insert context into the prompt.
So we do some imports.
We're going to use GPT four mini this time.
Um, and um uh, so now we get to the juice.
So what we do here this this is a Python function which will, uh, look in this folder and take all
of its contents and put it into a list.
Employees.
So employees is, in fact, now going to be a list of the file names in knowledge base slash employees.
And for each one, this is some janky code that that is just sort of hacked together, which basically
takes the name of the file, which is in fact an employee's Floyd's name and takes the their last name.
It just splits out their last name opens up that file and puts it into a dictionary where the key is
the employee name and the document is the document.
So let's just run this if we have a look at this.
So we're expecting to see a dictionary.
Uh sorry.
Um let's see it's called context or dictionary.
Let's just look at all of the keys in context.
So here are the keys.
The keys are indeed the last names of the employees, as you would expect.
And one of them is Lancaster, our CEO.
So let's just have a quick look at Lancaster.
So we should simply be able to look in our dictionary at Lancaster.
And we should just see bam the markdown document from her HR record.
Nothing magic at all going on here, reading in files, shoving them in a dictionary.
That's all there is to it.
So now let's just extend this And what we're going to do here is do exactly the same thing for the products.
Look in the products folder, iterate through each of the files, pluck out its name, and then shove
that in the same context dictionary.
So now if I look in the if I run that first, try not to forget that.
There we go.
And now look at the keys.
We should see a combination of employee last names and product names.
Here are the last names and their market.
L'm home.
L'm calm.
Are the product names okay?
Ready for action.
We'll have a system message.
You are an expert in answering accurate questions about ensure l'm the insurance tech company.
Give brief, accurate answers.
If you don't know the answer, say so.
Do not make anything up.
If you haven't been provided with relevant context.
It turns out that making these kinds of very authoritative instructions in the system prompt to tell
the model not to invent things is effective in stopping it from hallucinating and keeping accuracy high.
We're going to see that again later, too, but it's worth bearing in mind in keeping this one in your
in your notebook or in a list of useful things, because that kind of prompting is useful when accuracy
is essential.
So we will see that we will add that in to get we will make a function called get relevant context.
So what this function is going to do this is very important.
It's going to um uh it's going to take um a message, any kind of message.
Uh, and it's then going to iterate through the context, taking the title and the details out of each
item in the context.
And it's going to see whether or not the title, the, the, either the last name of the employee or
the product name, if it's a product, if that exists anywhere in the message.
If that text, like the word Lancaster, is anywhere in the message, then it's simply going to shove
that into this list of relevant context and return that list in the end.
So let me show you exactly what that means.
So if I say get relevant context and I pass in something completely irrelevant, like the cat, we'll
get nothing back.
The cat is not relevant to the questions about ensure.
And if I put who is Avery Lancaster then hooray!
We will in fact get a good robust, uh, answer in there, which is exactly what we would expect.
If I say, who is Avery Lancaster and what is calm, then we should hopefully let's have a look.
See that we'll get back a list of two things that will include.
Yes, you can see there's the second one beginning.
That's column in there.
We get back a list that contains Avery's document and the column document.
Um, but of course this is very brittle.
If I don't include Avery's last name, I just say Avery and I spell this without a capital C because
I haven't even done a case insensitive match there.
We're going to discover we get nothing back.
This is brittle.
It's only going to work in basic cases.
All right, so now I just build on this one step further by having a function add context that takes
a message.
Um, so the idea is that this will simply get any relevant context.
And if it finds some it's going to add that into the message.
It's going to say the following additional context might be relevant in answering this question.
So it's just sort of building on top of whatever it's passed.
So to give an example Let's just ask again the question.
Um, but yeah, let me start with who is Avery Lancaster, a positive example, and I'll give you a
bad example.
Who is Avery Lancaster and what, you have to run the cell.
Uh, so, uh, who is Avery Lancaster?
What do we see?
Uh, we see.
Here we go.
Uh, who is Avery Lancaster?
The following additional context might be relevant in answering this question.
And then there is the detail.
Uh, and then I can give a counter example.
Another kind of problem.
If I say who is Alex Lancaster, it will still see the word Lancaster in there.
And of course, it's still going to say here is relevant context.
Um, and it's of course missed the trick there.
So it is rough and ready.
Um, it really is just doing a string lookup.
But that's all we need, of course, to be writing our favorite chat function.
You remember, this is the function that Gradio expects.
If it's to build a quick user interface, it takes the current message and the history of messages,
and we hopefully this is now second nature to you.
We convert that history into the format that OpenAI expects, the list of dicts in that particular structure.
And then we make this lovely call here.
We added extra context to our message, and that's what we send to OpenAI.
And then this of course OpenAI chat completions create also second nature to you and we will stream
back the response.
So let's give this a run.
And now we know the last step is the one liner that brings this up in Gradio.
Here we go.
Let's see how this looks.
So let's say, uh.
Hi there.
Hello.
How can I assist you today?
So we'll say, who is Avery Lancaster?
That's right.
Yeah.
Lancaster.
And of course, we get back a nice good answer because we know full well that the Avery Lancaster information
was shoved into the prompt and sent to OpenAI.
Um, and now we can also say what is calm.
And innovative auto insurance product.
And there we go.
It's got a whole ton of information summarized from that big chunk of context that will have centered
about the product.
And it even mentions the roadmap for continuous improvement.
Lovely.
Uh, so you can see it's rag.
It's simplistic, but, you know, this is what it's all about.
Let's just break it for a second.
You can imagine what I'm going to do.
I'm just going to say, uh, who is Avery?
Well, it may it actually has the the conversation from the past, so we should probably start this
again.
Bring up a fresh chat.
Otherwise it will know who Avery it is because we just asked it before and it got the context.
So let's bring up a fresh new user interface and try it from scratch and not give it that benefit and
just ask it.
Who is Avery?
I'm sorry, but I don't have any information about Avery in relation to ensure.
So you can see that it's brittle.
It has to take the last name.
And even if I spell the last name wrong, who is Lancaster without, uh, a capital L, it's also going
to break, so you get the idea.
This is an effective way to add supplemental information into the prompt.
It gets back more accurate answers, but it is brittle and it requires exact text matching.
Not particularly flexible, not very scalable solution.
Uh, we can do better.
And that is what we are going to be doing in the in the next day.
But now back to the slides.