WEBVTT 00:01.070 --> 00:04.640 It's always welcome back to JupyterLab, my favorite place to be. 00:04.640 --> 00:07.790 And now we are, of course in the week five folder. 00:07.790 --> 00:14.000 And I'm looking here at the day one Jupyter notebook in week five, which is going to be our home for 00:14.000 --> 00:20.240 today, where we look at building our own DIY rag implementation in a very simplistic way, but it will 00:20.240 --> 00:22.610 give you some real insight into how this works. 00:22.700 --> 00:29.870 So first of all, let me tell you about our company in Shoreham, an insurance tech company. 00:29.990 --> 00:37.400 We have got access to their company shared folder and it's called Knowledge Base and I have dropped 00:37.400 --> 00:41.690 it right here in the same week five folder for your perusal. 00:41.690 --> 00:48.260 If we open it up, you'll see that it contains four folders company contracts, employees and products 00:48.260 --> 00:53.660 representing four different um divisions or four different areas of the company. 00:54.020 --> 01:00.050 Um, now, one thing I will mention, as you may have guessed perhaps, is that you'll see a fair amount 01:00.050 --> 01:03.450 of company data here, which is all completely fictitious. 01:03.660 --> 01:07.440 This was, of course created by an LLM. 01:07.470 --> 01:14.940 I took took my own exercise from a couple of weeks ago, wrote a data generator and used it to craft 01:14.940 --> 01:19.020 some of this data and keep relationships between some of the documents and so on. 01:19.050 --> 01:24.450 I did add in a little couple of tweaks here and there myself, but almost all of this is generated by 01:24.450 --> 01:28.530 either GPT four or by Claude. 01:29.370 --> 01:34.470 So just to have a quick peek at this in company, there's a few things here about the company. 01:34.470 --> 01:39.180 I think these are quite short documents if these are in the form of markdown documents. 01:39.180 --> 01:46.440 So if I can open it in this way, then we get to see some nice, um, write ups about the insurer Elm. 01:46.440 --> 01:50.880 It was founded apparently by Avery Lancaster in 2015. 01:50.880 --> 01:52.230 Remember, Avery's name will be. 01:52.260 --> 01:53.910 We'll be meeting her a few times. 01:54.240 --> 01:57.030 Um, and if you want to see the original documents, of course. 01:57.030 --> 02:00.210 They look like the markdown that we are familiar with. 02:00.210 --> 02:03.850 Um, then if we go back into knowledge base in contracts. 02:03.850 --> 02:10.540 We've got some more substantive information about contracts, which includes the contract terms renewal, 02:10.570 --> 02:15.130 the features that are included in that contract that's going to come back and be important later. 02:15.130 --> 02:16.960 So so keep note of that. 02:17.260 --> 02:19.150 Um, and some other things. 02:19.510 --> 02:25.480 Uh, if we go back into knowledge base again and look at employees, these are the employee HR records 02:25.480 --> 02:26.770 of the different employees. 02:26.770 --> 02:32.440 And even the CEO, Avery Lancaster herself has an HR record that you'll see right here. 02:32.770 --> 02:41.050 Uh, so all everyone has their their HR documents, um, and then products contains a write up of the 02:41.050 --> 02:49.840 different offerings from insurer Elm, including car Elm, uh, for auto insurance, uh home Elm and 02:49.870 --> 02:50.530 and so on. 02:50.530 --> 02:55.600 You can look through these documents, but they are wonderful in how real they are. 02:55.630 --> 02:59.800 All of course courtesy of Frontier Models. 02:59.920 --> 03:03.480 A nice product summary right there with even with a roadmap. 03:03.480 --> 03:04.500 Love it. 03:04.980 --> 03:06.360 Uh, okay. 03:06.390 --> 03:08.940 Back then to day one. 03:08.940 --> 03:10.800 So what are we going to do? 03:10.830 --> 03:16.380 We are going to write something now that is going to be asking questions about this company, and we 03:16.410 --> 03:20.700 are going to artificially look up and insert context into the prompt. 03:20.700 --> 03:22.110 So we do some imports. 03:22.110 --> 03:25.470 We're going to use GPT four mini this time. 03:25.530 --> 03:31.170 Um, and um uh, so now we get to the juice. 03:31.170 --> 03:40.560 So what we do here this this is a Python function which will, uh, look in this folder and take all 03:40.560 --> 03:43.350 of its contents and put it into a list. 03:43.350 --> 03:44.100 Employees. 03:44.100 --> 03:51.600 So employees is, in fact, now going to be a list of the file names in knowledge base slash employees. 03:51.600 --> 03:59.400 And for each one, this is some janky code that that is just sort of hacked together, which basically 03:59.640 --> 04:06.490 takes the name of the file, which is in fact an employee's Floyd's name and takes the their last name. 04:06.520 --> 04:14.140 It just splits out their last name opens up that file and puts it into a dictionary where the key is 04:14.140 --> 04:17.080 the employee name and the document is the document. 04:17.080 --> 04:20.020 So let's just run this if we have a look at this. 04:20.020 --> 04:22.090 So we're expecting to see a dictionary. 04:23.080 --> 04:24.760 Uh sorry. 04:25.210 --> 04:28.570 Um let's see it's called context or dictionary. 04:28.570 --> 04:31.180 Let's just look at all of the keys in context. 04:31.390 --> 04:33.130 So here are the keys. 04:33.130 --> 04:38.410 The keys are indeed the last names of the employees, as you would expect. 04:38.410 --> 04:42.040 And one of them is Lancaster, our CEO. 04:42.040 --> 04:44.560 So let's just have a quick look at Lancaster. 04:44.560 --> 04:48.520 So we should simply be able to look in our dictionary at Lancaster. 04:48.520 --> 04:53.620 And we should just see bam the markdown document from her HR record. 04:53.650 --> 04:58.990 Nothing magic at all going on here, reading in files, shoving them in a dictionary. 04:58.990 --> 05:00.670 That's all there is to it. 05:01.480 --> 05:09.200 So now let's just extend this And what we're going to do here is do exactly the same thing for the products. 05:09.230 --> 05:16.520 Look in the products folder, iterate through each of the files, pluck out its name, and then shove 05:16.520 --> 05:18.800 that in the same context dictionary. 05:18.800 --> 05:22.520 So now if I look in the if I run that first, try not to forget that. 05:22.520 --> 05:23.180 There we go. 05:23.180 --> 05:24.650 And now look at the keys. 05:24.650 --> 05:28.580 We should see a combination of employee last names and product names. 05:28.940 --> 05:32.570 Here are the last names and their market. 05:32.750 --> 05:33.620 L'm home. 05:33.620 --> 05:34.910 L'm calm. 05:35.030 --> 05:38.150 Are the product names okay? 05:38.810 --> 05:40.130 Ready for action. 05:40.160 --> 05:41.780 We'll have a system message. 05:41.780 --> 05:47.180 You are an expert in answering accurate questions about ensure l'm the insurance tech company. 05:47.210 --> 05:49.100 Give brief, accurate answers. 05:49.100 --> 05:51.590 If you don't know the answer, say so. 05:51.590 --> 05:52.820 Do not make anything up. 05:52.820 --> 05:55.550 If you haven't been provided with relevant context. 05:56.060 --> 06:02.570 It turns out that making these kinds of very authoritative instructions in the system prompt to tell 06:02.570 --> 06:08.790 the model not to invent things is effective in stopping it from hallucinating and keeping accuracy high. 06:08.970 --> 06:13.500 We're going to see that again later, too, but it's worth bearing in mind in keeping this one in your 06:13.530 --> 06:19.500 in your notebook or in a list of useful things, because that kind of prompting is useful when accuracy 06:19.500 --> 06:20.550 is essential. 06:20.940 --> 06:32.610 So we will see that we will add that in to get we will make a function called get relevant context. 06:32.610 --> 06:35.490 So what this function is going to do this is very important. 06:35.490 --> 06:45.090 It's going to um uh it's going to take um a message, any kind of message. 06:45.450 --> 06:53.160 Uh, and it's then going to iterate through the context, taking the title and the details out of each 06:53.160 --> 06:55.020 item in the context. 06:55.020 --> 07:02.370 And it's going to see whether or not the title, the, the, either the last name of the employee or 07:02.370 --> 07:07.370 the product name, if it's a product, if that exists anywhere in the message. 07:07.370 --> 07:14.030 If that text, like the word Lancaster, is anywhere in the message, then it's simply going to shove 07:14.030 --> 07:18.920 that into this list of relevant context and return that list in the end. 07:18.950 --> 07:21.110 So let me show you exactly what that means. 07:21.140 --> 07:33.620 So if I say get relevant context and I pass in something completely irrelevant, like the cat, we'll 07:33.620 --> 07:34.670 get nothing back. 07:34.700 --> 07:38.870 The cat is not relevant to the questions about ensure. 07:38.900 --> 07:46.160 And if I put who is Avery Lancaster then hooray! 07:46.190 --> 07:53.690 We will in fact get a good robust, uh, answer in there, which is exactly what we would expect. 07:53.840 --> 08:04.310 If I say, who is Avery Lancaster and what is calm, then we should hopefully let's have a look. 08:04.370 --> 08:09.030 See that we'll get back a list of two things that will include. 08:09.060 --> 08:10.800 Yes, you can see there's the second one beginning. 08:10.800 --> 08:12.600 That's column in there. 08:12.600 --> 08:17.940 We get back a list that contains Avery's document and the column document. 08:18.450 --> 08:21.030 Um, but of course this is very brittle. 08:21.030 --> 08:28.650 If I don't include Avery's last name, I just say Avery and I spell this without a capital C because 08:28.650 --> 08:32.220 I haven't even done a case insensitive match there. 08:32.220 --> 08:34.950 We're going to discover we get nothing back. 08:34.950 --> 08:36.360 This is brittle. 08:36.360 --> 08:39.780 It's only going to work in basic cases. 08:40.620 --> 08:46.950 All right, so now I just build on this one step further by having a function add context that takes 08:46.950 --> 08:47.970 a message. 08:48.120 --> 08:53.190 Um, so the idea is that this will simply get any relevant context. 08:53.190 --> 08:56.370 And if it finds some it's going to add that into the message. 08:56.370 --> 09:01.290 It's going to say the following additional context might be relevant in answering this question. 09:01.290 --> 09:04.320 So it's just sort of building on top of whatever it's passed. 09:04.320 --> 09:07.900 So to give an example Let's just ask again the question. 09:07.960 --> 09:12.160 Um, but yeah, let me start with who is Avery Lancaster, a positive example, and I'll give you a 09:12.160 --> 09:12.880 bad example. 09:12.880 --> 09:17.170 Who is Avery Lancaster and what, you have to run the cell. 09:19.270 --> 09:23.410 Uh, so, uh, who is Avery Lancaster? 09:23.410 --> 09:24.700 What do we see? 09:24.940 --> 09:25.840 Uh, we see. 09:25.840 --> 09:26.830 Here we go. 09:26.830 --> 09:29.650 Uh, who is Avery Lancaster? 09:29.680 --> 09:32.620 The following additional context might be relevant in answering this question. 09:32.620 --> 09:34.540 And then there is the detail. 09:34.750 --> 09:37.870 Uh, and then I can give a counter example. 09:37.870 --> 09:39.340 Another kind of problem. 09:39.340 --> 09:44.290 If I say who is Alex Lancaster, it will still see the word Lancaster in there. 09:44.290 --> 09:47.530 And of course, it's still going to say here is relevant context. 09:47.740 --> 09:50.380 Um, and it's of course missed the trick there. 09:50.380 --> 09:53.320 So it is rough and ready. 09:53.440 --> 09:56.470 Um, it really is just doing a string lookup. 09:56.470 --> 10:01.570 But that's all we need, of course, to be writing our favorite chat function. 10:01.570 --> 10:04.510 You remember, this is the function that Gradio expects. 10:04.510 --> 10:09.230 If it's to build a quick user interface, it takes the current message and the history of messages, 10:09.230 --> 10:12.200 and we hopefully this is now second nature to you. 10:12.230 --> 10:20.510 We convert that history into the format that OpenAI expects, the list of dicts in that particular structure. 10:20.510 --> 10:23.660 And then we make this lovely call here. 10:23.660 --> 10:28.880 We added extra context to our message, and that's what we send to OpenAI. 10:28.880 --> 10:36.590 And then this of course OpenAI chat completions create also second nature to you and we will stream 10:36.590 --> 10:38.150 back the response. 10:38.150 --> 10:40.190 So let's give this a run. 10:40.280 --> 10:46.430 And now we know the last step is the one liner that brings this up in Gradio. 10:46.760 --> 10:47.510 Here we go. 10:47.540 --> 10:49.370 Let's see how this looks. 10:54.230 --> 10:56.000 So let's say, uh. 10:56.060 --> 10:56.780 Hi there. 10:59.510 --> 11:00.050 Hello. 11:00.050 --> 11:01.460 How can I assist you today? 11:01.490 --> 11:06.410 So we'll say, who is Avery Lancaster? 11:08.620 --> 11:09.190 That's right. 11:09.190 --> 11:09.460 Yeah. 11:09.490 --> 11:10.570 Lancaster. 11:11.950 --> 11:21.670 And of course, we get back a nice good answer because we know full well that the Avery Lancaster information 11:21.670 --> 11:24.940 was shoved into the prompt and sent to OpenAI. 11:25.300 --> 11:30.040 Um, and now we can also say what is calm. 11:32.350 --> 11:34.870 And innovative auto insurance product. 11:34.900 --> 11:35.770 And there we go. 11:35.770 --> 11:40.660 It's got a whole ton of information summarized from that big chunk of context that will have centered 11:40.660 --> 11:41.830 about the product. 11:41.950 --> 11:45.370 And it even mentions the roadmap for continuous improvement. 11:45.400 --> 11:46.390 Lovely. 11:46.990 --> 11:50.110 Uh, so you can see it's rag. 11:50.110 --> 11:54.130 It's simplistic, but, you know, this is what it's all about. 11:54.220 --> 11:56.440 Let's just break it for a second. 11:56.440 --> 11:57.760 You can imagine what I'm going to do. 11:57.790 --> 12:02.380 I'm just going to say, uh, who is Avery? 12:02.410 --> 12:09.170 Well, it may it actually has the the conversation from the past, so we should probably start this 12:09.170 --> 12:09.950 again. 12:10.040 --> 12:11.570 Bring up a fresh chat. 12:11.600 --> 12:15.470 Otherwise it will know who Avery it is because we just asked it before and it got the context. 12:15.470 --> 12:22.250 So let's bring up a fresh new user interface and try it from scratch and not give it that benefit and 12:22.250 --> 12:22.880 just ask it. 12:22.910 --> 12:24.320 Who is Avery? 12:25.760 --> 12:30.170 I'm sorry, but I don't have any information about Avery in relation to ensure. 12:30.470 --> 12:32.540 So you can see that it's brittle. 12:32.540 --> 12:34.430 It has to take the last name. 12:34.430 --> 12:46.040 And even if I spell the last name wrong, who is Lancaster without, uh, a capital L, it's also going 12:46.040 --> 12:48.200 to break, so you get the idea. 12:48.200 --> 12:51.620 This is an effective way to add supplemental information into the prompt. 12:51.650 --> 12:57.140 It gets back more accurate answers, but it is brittle and it requires exact text matching. 12:57.140 --> 13:01.550 Not particularly flexible, not very scalable solution. 13:01.670 --> 13:03.140 Uh, we can do better. 13:03.140 --> 13:06.710 And that is what we are going to be doing in the in the next day. 13:06.710 --> 13:09.350 But now back to the slides.