You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 

586 lines
15 KiB

WEBVTT
00:01.070 --> 00:04.640
It's always welcome back to JupyterLab, my favorite place to be.
00:04.640 --> 00:07.790
And now we are, of course in the week five folder.
00:07.790 --> 00:14.000
And I'm looking here at the day one Jupyter notebook in week five, which is going to be our home for
00:14.000 --> 00:20.240
today, where we look at building our own DIY rag implementation in a very simplistic way, but it will
00:20.240 --> 00:22.610
give you some real insight into how this works.
00:22.700 --> 00:29.870
So first of all, let me tell you about our company in Shoreham, an insurance tech company.
00:29.990 --> 00:37.400
We have got access to their company shared folder and it's called Knowledge Base and I have dropped
00:37.400 --> 00:41.690
it right here in the same week five folder for your perusal.
00:41.690 --> 00:48.260
If we open it up, you'll see that it contains four folders company contracts, employees and products
00:48.260 --> 00:53.660
representing four different um divisions or four different areas of the company.
00:54.020 --> 01:00.050
Um, now, one thing I will mention, as you may have guessed perhaps, is that you'll see a fair amount
01:00.050 --> 01:03.450
of company data here, which is all completely fictitious.
01:03.660 --> 01:07.440
This was, of course created by an LLM.
01:07.470 --> 01:14.940
I took took my own exercise from a couple of weeks ago, wrote a data generator and used it to craft
01:14.940 --> 01:19.020
some of this data and keep relationships between some of the documents and so on.
01:19.050 --> 01:24.450
I did add in a little couple of tweaks here and there myself, but almost all of this is generated by
01:24.450 --> 01:28.530
either GPT four or by Claude.
01:29.370 --> 01:34.470
So just to have a quick peek at this in company, there's a few things here about the company.
01:34.470 --> 01:39.180
I think these are quite short documents if these are in the form of markdown documents.
01:39.180 --> 01:46.440
So if I can open it in this way, then we get to see some nice, um, write ups about the insurer Elm.
01:46.440 --> 01:50.880
It was founded apparently by Avery Lancaster in 2015.
01:50.880 --> 01:52.230
Remember, Avery's name will be.
01:52.260 --> 01:53.910
We'll be meeting her a few times.
01:54.240 --> 01:57.030
Um, and if you want to see the original documents, of course.
01:57.030 --> 02:00.210
They look like the markdown that we are familiar with.
02:00.210 --> 02:03.850
Um, then if we go back into knowledge base in contracts.
02:03.850 --> 02:10.540
We've got some more substantive information about contracts, which includes the contract terms renewal,
02:10.570 --> 02:15.130
the features that are included in that contract that's going to come back and be important later.
02:15.130 --> 02:16.960
So so keep note of that.
02:17.260 --> 02:19.150
Um, and some other things.
02:19.510 --> 02:25.480
Uh, if we go back into knowledge base again and look at employees, these are the employee HR records
02:25.480 --> 02:26.770
of the different employees.
02:26.770 --> 02:32.440
And even the CEO, Avery Lancaster herself has an HR record that you'll see right here.
02:32.770 --> 02:41.050
Uh, so all everyone has their their HR documents, um, and then products contains a write up of the
02:41.050 --> 02:49.840
different offerings from insurer Elm, including car Elm, uh, for auto insurance, uh home Elm and
02:49.870 --> 02:50.530
and so on.
02:50.530 --> 02:55.600
You can look through these documents, but they are wonderful in how real they are.
02:55.630 --> 02:59.800
All of course courtesy of Frontier Models.
02:59.920 --> 03:03.480
A nice product summary right there with even with a roadmap.
03:03.480 --> 03:04.500
Love it.
03:04.980 --> 03:06.360
Uh, okay.
03:06.390 --> 03:08.940
Back then to day one.
03:08.940 --> 03:10.800
So what are we going to do?
03:10.830 --> 03:16.380
We are going to write something now that is going to be asking questions about this company, and we
03:16.410 --> 03:20.700
are going to artificially look up and insert context into the prompt.
03:20.700 --> 03:22.110
So we do some imports.
03:22.110 --> 03:25.470
We're going to use GPT four mini this time.
03:25.530 --> 03:31.170
Um, and um uh, so now we get to the juice.
03:31.170 --> 03:40.560
So what we do here this this is a Python function which will, uh, look in this folder and take all
03:40.560 --> 03:43.350
of its contents and put it into a list.
03:43.350 --> 03:44.100
Employees.
03:44.100 --> 03:51.600
So employees is, in fact, now going to be a list of the file names in knowledge base slash employees.
03:51.600 --> 03:59.400
And for each one, this is some janky code that that is just sort of hacked together, which basically
03:59.640 --> 04:06.490
takes the name of the file, which is in fact an employee's Floyd's name and takes the their last name.
04:06.520 --> 04:14.140
It just splits out their last name opens up that file and puts it into a dictionary where the key is
04:14.140 --> 04:17.080
the employee name and the document is the document.
04:17.080 --> 04:20.020
So let's just run this if we have a look at this.
04:20.020 --> 04:22.090
So we're expecting to see a dictionary.
04:23.080 --> 04:24.760
Uh sorry.
04:25.210 --> 04:28.570
Um let's see it's called context or dictionary.
04:28.570 --> 04:31.180
Let's just look at all of the keys in context.
04:31.390 --> 04:33.130
So here are the keys.
04:33.130 --> 04:38.410
The keys are indeed the last names of the employees, as you would expect.
04:38.410 --> 04:42.040
And one of them is Lancaster, our CEO.
04:42.040 --> 04:44.560
So let's just have a quick look at Lancaster.
04:44.560 --> 04:48.520
So we should simply be able to look in our dictionary at Lancaster.
04:48.520 --> 04:53.620
And we should just see bam the markdown document from her HR record.
04:53.650 --> 04:58.990
Nothing magic at all going on here, reading in files, shoving them in a dictionary.
04:58.990 --> 05:00.670
That's all there is to it.
05:01.480 --> 05:09.200
So now let's just extend this And what we're going to do here is do exactly the same thing for the products.
05:09.230 --> 05:16.520
Look in the products folder, iterate through each of the files, pluck out its name, and then shove
05:16.520 --> 05:18.800
that in the same context dictionary.
05:18.800 --> 05:22.520
So now if I look in the if I run that first, try not to forget that.
05:22.520 --> 05:23.180
There we go.
05:23.180 --> 05:24.650
And now look at the keys.
05:24.650 --> 05:28.580
We should see a combination of employee last names and product names.
05:28.940 --> 05:32.570
Here are the last names and their market.
05:32.750 --> 05:33.620
L'm home.
05:33.620 --> 05:34.910
L'm calm.
05:35.030 --> 05:38.150
Are the product names okay?
05:38.810 --> 05:40.130
Ready for action.
05:40.160 --> 05:41.780
We'll have a system message.
05:41.780 --> 05:47.180
You are an expert in answering accurate questions about ensure l'm the insurance tech company.
05:47.210 --> 05:49.100
Give brief, accurate answers.
05:49.100 --> 05:51.590
If you don't know the answer, say so.
05:51.590 --> 05:52.820
Do not make anything up.
05:52.820 --> 05:55.550
If you haven't been provided with relevant context.
05:56.060 --> 06:02.570
It turns out that making these kinds of very authoritative instructions in the system prompt to tell
06:02.570 --> 06:08.790
the model not to invent things is effective in stopping it from hallucinating and keeping accuracy high.
06:08.970 --> 06:13.500
We're going to see that again later, too, but it's worth bearing in mind in keeping this one in your
06:13.530 --> 06:19.500
in your notebook or in a list of useful things, because that kind of prompting is useful when accuracy
06:19.500 --> 06:20.550
is essential.
06:20.940 --> 06:32.610
So we will see that we will add that in to get we will make a function called get relevant context.
06:32.610 --> 06:35.490
So what this function is going to do this is very important.
06:35.490 --> 06:45.090
It's going to um uh it's going to take um a message, any kind of message.
06:45.450 --> 06:53.160
Uh, and it's then going to iterate through the context, taking the title and the details out of each
06:53.160 --> 06:55.020
item in the context.
06:55.020 --> 07:02.370
And it's going to see whether or not the title, the, the, either the last name of the employee or
07:02.370 --> 07:07.370
the product name, if it's a product, if that exists anywhere in the message.
07:07.370 --> 07:14.030
If that text, like the word Lancaster, is anywhere in the message, then it's simply going to shove
07:14.030 --> 07:18.920
that into this list of relevant context and return that list in the end.
07:18.950 --> 07:21.110
So let me show you exactly what that means.
07:21.140 --> 07:33.620
So if I say get relevant context and I pass in something completely irrelevant, like the cat, we'll
07:33.620 --> 07:34.670
get nothing back.
07:34.700 --> 07:38.870
The cat is not relevant to the questions about ensure.
07:38.900 --> 07:46.160
And if I put who is Avery Lancaster then hooray!
07:46.190 --> 07:53.690
We will in fact get a good robust, uh, answer in there, which is exactly what we would expect.
07:53.840 --> 08:04.310
If I say, who is Avery Lancaster and what is calm, then we should hopefully let's have a look.
08:04.370 --> 08:09.030
See that we'll get back a list of two things that will include.
08:09.060 --> 08:10.800
Yes, you can see there's the second one beginning.
08:10.800 --> 08:12.600
That's column in there.
08:12.600 --> 08:17.940
We get back a list that contains Avery's document and the column document.
08:18.450 --> 08:21.030
Um, but of course this is very brittle.
08:21.030 --> 08:28.650
If I don't include Avery's last name, I just say Avery and I spell this without a capital C because
08:28.650 --> 08:32.220
I haven't even done a case insensitive match there.
08:32.220 --> 08:34.950
We're going to discover we get nothing back.
08:34.950 --> 08:36.360
This is brittle.
08:36.360 --> 08:39.780
It's only going to work in basic cases.
08:40.620 --> 08:46.950
All right, so now I just build on this one step further by having a function add context that takes
08:46.950 --> 08:47.970
a message.
08:48.120 --> 08:53.190
Um, so the idea is that this will simply get any relevant context.
08:53.190 --> 08:56.370
And if it finds some it's going to add that into the message.
08:56.370 --> 09:01.290
It's going to say the following additional context might be relevant in answering this question.
09:01.290 --> 09:04.320
So it's just sort of building on top of whatever it's passed.
09:04.320 --> 09:07.900
So to give an example Let's just ask again the question.
09:07.960 --> 09:12.160
Um, but yeah, let me start with who is Avery Lancaster, a positive example, and I'll give you a
09:12.160 --> 09:12.880
bad example.
09:12.880 --> 09:17.170
Who is Avery Lancaster and what, you have to run the cell.
09:19.270 --> 09:23.410
Uh, so, uh, who is Avery Lancaster?
09:23.410 --> 09:24.700
What do we see?
09:24.940 --> 09:25.840
Uh, we see.
09:25.840 --> 09:26.830
Here we go.
09:26.830 --> 09:29.650
Uh, who is Avery Lancaster?
09:29.680 --> 09:32.620
The following additional context might be relevant in answering this question.
09:32.620 --> 09:34.540
And then there is the detail.
09:34.750 --> 09:37.870
Uh, and then I can give a counter example.
09:37.870 --> 09:39.340
Another kind of problem.
09:39.340 --> 09:44.290
If I say who is Alex Lancaster, it will still see the word Lancaster in there.
09:44.290 --> 09:47.530
And of course, it's still going to say here is relevant context.
09:47.740 --> 09:50.380
Um, and it's of course missed the trick there.
09:50.380 --> 09:53.320
So it is rough and ready.
09:53.440 --> 09:56.470
Um, it really is just doing a string lookup.
09:56.470 --> 10:01.570
But that's all we need, of course, to be writing our favorite chat function.
10:01.570 --> 10:04.510
You remember, this is the function that Gradio expects.
10:04.510 --> 10:09.230
If it's to build a quick user interface, it takes the current message and the history of messages,
10:09.230 --> 10:12.200
and we hopefully this is now second nature to you.
10:12.230 --> 10:20.510
We convert that history into the format that OpenAI expects, the list of dicts in that particular structure.
10:20.510 --> 10:23.660
And then we make this lovely call here.
10:23.660 --> 10:28.880
We added extra context to our message, and that's what we send to OpenAI.
10:28.880 --> 10:36.590
And then this of course OpenAI chat completions create also second nature to you and we will stream
10:36.590 --> 10:38.150
back the response.
10:38.150 --> 10:40.190
So let's give this a run.
10:40.280 --> 10:46.430
And now we know the last step is the one liner that brings this up in Gradio.
10:46.760 --> 10:47.510
Here we go.
10:47.540 --> 10:49.370
Let's see how this looks.
10:54.230 --> 10:56.000
So let's say, uh.
10:56.060 --> 10:56.780
Hi there.
10:59.510 --> 11:00.050
Hello.
11:00.050 --> 11:01.460
How can I assist you today?
11:01.490 --> 11:06.410
So we'll say, who is Avery Lancaster?
11:08.620 --> 11:09.190
That's right.
11:09.190 --> 11:09.460
Yeah.
11:09.490 --> 11:10.570
Lancaster.
11:11.950 --> 11:21.670
And of course, we get back a nice good answer because we know full well that the Avery Lancaster information
11:21.670 --> 11:24.940
was shoved into the prompt and sent to OpenAI.
11:25.300 --> 11:30.040
Um, and now we can also say what is calm.
11:32.350 --> 11:34.870
And innovative auto insurance product.
11:34.900 --> 11:35.770
And there we go.
11:35.770 --> 11:40.660
It's got a whole ton of information summarized from that big chunk of context that will have centered
11:40.660 --> 11:41.830
about the product.
11:41.950 --> 11:45.370
And it even mentions the roadmap for continuous improvement.
11:45.400 --> 11:46.390
Lovely.
11:46.990 --> 11:50.110
Uh, so you can see it's rag.
11:50.110 --> 11:54.130
It's simplistic, but, you know, this is what it's all about.
11:54.220 --> 11:56.440
Let's just break it for a second.
11:56.440 --> 11:57.760
You can imagine what I'm going to do.
11:57.790 --> 12:02.380
I'm just going to say, uh, who is Avery?
12:02.410 --> 12:09.170
Well, it may it actually has the the conversation from the past, so we should probably start this
12:09.170 --> 12:09.950
again.
12:10.040 --> 12:11.570
Bring up a fresh chat.
12:11.600 --> 12:15.470
Otherwise it will know who Avery it is because we just asked it before and it got the context.
12:15.470 --> 12:22.250
So let's bring up a fresh new user interface and try it from scratch and not give it that benefit and
12:22.250 --> 12:22.880
just ask it.
12:22.910 --> 12:24.320
Who is Avery?
12:25.760 --> 12:30.170
I'm sorry, but I don't have any information about Avery in relation to ensure.
12:30.470 --> 12:32.540
So you can see that it's brittle.
12:32.540 --> 12:34.430
It has to take the last name.
12:34.430 --> 12:46.040
And even if I spell the last name wrong, who is Lancaster without, uh, a capital L, it's also going
12:46.040 --> 12:48.200
to break, so you get the idea.
12:48.200 --> 12:51.620
This is an effective way to add supplemental information into the prompt.
12:51.650 --> 12:57.140
It gets back more accurate answers, but it is brittle and it requires exact text matching.
12:57.140 --> 13:01.550
Not particularly flexible, not very scalable solution.
13:01.670 --> 13:03.140
Uh, we can do better.
13:03.140 --> 13:06.710
And that is what we are going to be doing in the in the next day.
13:06.710 --> 13:09.350
But now back to the slides.