Last week, we worked with models that were able to speed up code by a factor of 60,000 times, which
was outrageous.
Hopefully you were super impressed by that, because I certainly was, and I'm hoping you're going to
be even more impressed at the end of this week when you see what I have in store for you.
So it's all about rag retrieval.
Augmented generation, where you can already do, of course, is code with frontier models, code with
hugging face transformers.
Choose the right LLM for your project and now build solutions that generate code.
Today you're going to learn about the big idea behind Rag retrieval augmented generation.
And we're going to walk through some of the interactions with Rag before we get there though, we're
also going to talk about the little idea, the small idea behind rag, which is actually quite obvious.
In fact, you may have already thought about it yourself.
And then we're going to implement a toy version of Rag using the small idea.
So you get a really good feel for how it works.
Before next week, we go on to the real deal.
Let me start by giving you some of the intuition behind rag.
We've already seen that we can make the performance of models stronger by enriching the prompts, the
information that we send to the models.
And we've done that in several ways.
We've used Multi-shot prompting to send a series of example questions and answers to the model.
We've used tools so that the LLM can call back into our code almost kind of, and run some code that
then is used to supplement its answers or carry out actions.
And we've had other ways to provide additional context as part of what we send the LLM, including in
the system prompt.
So the thinking is, can we step up this idea and take it to a new level by supplying more concrete
information into the prompt?
That's going to be particularly relevant to the question at hand.
So the idea is could we put together a database of information sometimes because this is a database
of knowledge, it's known as a knowledge base, a knowledge base of information.
And every time that the user asks us a question, we'll first look up in that knowledge base whether
there's any relevant information that we can pluck out.
And if there is, we simply stuff that in the prompt and that is sent in the prompt to the model.
That's all there is to it.
It's actually a very simple idea, and you probably already thought of it yourself while we were doing
some of the earlier exercises.
So let's just show this small idea behind rag in a diagram, and I promise you later we'll get to the
bigger idea, which is where it becomes somewhat less less obvious and more meaningful.
But in the little idea, what we're saying is let's start by the user asking us a question.
It comes to our code, and normally we'd send that straight on to the LLM.
But this time before we do so, we do a query in our knowledge base to see if we've got any relevant
background information.
And if we do, we pluck out that information and we include it in the prompt.
We send the LM and of course the response comes back as always, but hopefully it takes into account
some of this extra context.
And that is what goes back to the user.
That's really all there is to the small idea behind Rag.
So we're now going to put this into action with a small example of the small idea.
Let's say we work for an insurance tech startup.
And it's going to be a fictional insurance tech startup called insurance, which happens to be the word
insurer.
And LM stuffed together, which is, I think, the limit of my creativity.
We have a knowledge base in the form of a folder taken from the company's shared drive.
It is the entire contents of their shared drive, and our task is to build an AI knowledge worker.
Sometimes this expression knowledge worker is used to mean a person that works for a firm and is the
expert and able to carry out analysis on information about the company and carry out questions and answers.
Well, that's something that we can do with with with an LLM.
And we can supplement it with information from the knowledge base.
So we're going to do a toy implementation blunt instrument.
Um, basically we're going to read in some of these files, products and employees.
And we're going to store it in like a dictionary.
And then anytime a question comes in, we're just going to look up whether or not the word, the name
of the employee appears somewhere in the question.
And if so, we're just going to shove that whole employee record into our prompt.
So it's a kind of manual, brute force implementation of Rag, but it will give you a good sense of
how this actually works behind the scenes.
And it will show you that there's there's no magic to it.
It just improves the performance of the model right away.
And then once we've done that, we'll get on to the more exciting stuff.
But for now, let's go to JupyterLab and build our own homemade rag.