And now the time has come to curate our data set.
And the way we're going to do this is we're going to take each of the data points that we got from hugging
face, and we're going to convert it into a Python object, an object, a class that we're going to
create called an item.
And it's so important that I've actually set up a different module items.py, where I have written this
class, and I've done it as Python code, not in a Jupyter notebook, but in its own own module, so
that it can be reused from different places, and so that we don't clutter our Jupyter notebook with
the code behind it.
And it contains some messy code to do some data munging, some unraveling of the data to clean it up,
I'm going to show you this item, this module now and talk through it.
But really there's an exercise for you to go and look through this in more detail and understand it
a little bit more closely.
So as I say it's in its own module items.py and it defines a class item.
And I should point out, before we even get going with it, that we start by setting a constant called
base model to be the llama 3.18 billion variant base model.
Now, you might say to me, what on earth has the llama model got to do with what we're doing at the
moment?
We're not going on to open source until next week.
This week it's all about using frontier models for for fine tuning.
And here's the answer.
We're going to be crafting our data set so that it fits within a certain fixed number of tokens as a
maximum tokens for the llama tokenizer.
And the reason we're going to do that is because that's going to make it cheaper and easier to train
when we end up using our open source model.
It's also, as I say, going to make it cheaper when we use the frontier model as well, and we want
everyone to be on the same playing field.
So when we craft our prompts and we fix them to a certain number of tokens, we want to make sure that
both the frontier model and the open source model gets the same amount of information.
If you have more budget and you have ability to train on bigger GPUs, or to use, uh, or more budget
with frontier models, then you can extend the cutoff so that we can have bigger and bigger amounts
of text.
But I think we've got we're going to you'll see, we'll have plenty of text in each of these data points.
And so it's perfectly sufficient for our frontier models and our open source models to be training against.
Anyway, that is why we're looking at the llama model, because we're going to be using its tokenizer
to when we check whether or not we have the right number of characters.
So the class item, then it is something which each item is going to have a title, a price of course,
a category which will be things like appliances, a token count, how many tokens does it contain.
And then most importantly, a prompt which is going to be the text which will be fed into an LLM, which
it will then use to either train or to test against.
So just very briefly, the the takeaway is that you must look through this code yourself and satisfy
yourself that I'm not doing anything evil and that all of this is just good, uh, wholesome housekeeping
and cleaning up of strings.
I have a function called scrub details, which removes stuff from the text that feels like it's going
to be distracting to the model.
Stuff like batteries included.
Um, and, uh, some other things you see in there, the word manufacturer by manufacturer.
So a bunch of things where it's not relevant or it's not massively relevant, and it seemed better to
remove it than to have it use up precious tokens by being in there.
There's this, uh, method scrub, which goes through and cleans out weird characters, and it also
turns multiple spaces into one space, using some regex for regex ninjas out there.
This is probably, uh, easy stuff for you.
For others, this is one of the kinds of bits of script that you can reuse as ways to remove different
problems in your in your strings.
And you can also test this out at a Jupyter notebook to satisfy yourself that it's doing what it says
on the tin.
I will mention this line here, because this is just a little extra trick I put in there that is useful
for our particular case, and it's an example of the kind of thing you only discover when you really
dig into the data.
I noticed that there were a lot of products on Amazon which quote part numbers in their description.
So they say this is compatible with part number, blah blah and blah.
And those part numbers are often eight digits, eight characters long or longer and contain letters
and numbers.
And the problem with that is that when that gets turned into tokens, it uses up a lot of tokens because
obviously it's not in the vocabulary in any way.
And so you end up cramming all of your, your token, your unlimited capacity for tokens with tokens
that represent part numbers that are going to be totally irrelevant for our model.
So what this line here does is it says if there's any, any word that has eight or more characters and
contains a number inside it, then scrap that word.
It's going to be a distraction.
Um, so the reason I highlight this is, is really, again, to show that, that you only come across
this kind of discovery when you dig into your data.
You look at lots of examples and you see this happening, and then you come across this.
You have the moment you try this out.
And when you rerun your model, uh, which you can imagine I've done once or twice in the last few weeks,
you find that you've made an improvement because your data is richer and has more, more accuracy to
it.
So that's an important part of the process.
And we will then use a method parse, which takes a data point, and then does all of the various scrubbing
and stripping and ends up turning it into a prompt.
And along with the prompt, it counts the number of tokens in that prompt.
And you're going to see the prompt in just a second.
But the prompt is the thing that's going to get passed into an LLM, and it will be asked to complete
it.
And it's going to say, how much does this cost?
And it's going to have a cost.
There's going to be an ability to look at a look at a prompt.
There's also going to be something called the test prompt, which is the same as the prompt, but it
doesn't reveal the answer.
The prompt will be used during training and it has the answer in there.
So during training, the model will get better and better at predicting the answer during test time.
We don't want to show it the answer.
We want to give it the text and see whether or not it gets the right answer.
So we have those two different prompts the training prompt and the test prompt.
Later we're going to talk about breaking down your data into a training set and a test set.
You'll see you'll see more.
It will become much more clear later on.
So this is the item class.
And I really suggest that you take more of a look through this.
But never fear, we're going to be spending a lot of time with these items and looking at them.
And so you're going to get a good handle for for what this functionality does.
So back here, what we're now going to do is create one of these items objects for everything in in
the data set that has a price.
So let's run that right now.
So the this is running through that code.
It's it's doing the scrubbing.
It's removing things like part numbers.
It's replacing weird characters with with with with spaces.
And it's creating a prompt and then making sure that the prompt will fit into a decent number of tokens.
So all of that is happening right now, and it's going to be doing that for the 40 odd thousand appliances,
home appliances that have a price.
So it should be just about wrapping up now.
While it's finishing that off I will prepare for us to look at.
It's done.
There we go.
So let's say let's just have a look at the first first one in there.
So the first one in there is a rack roller and stud assembly kit.
Full pack by Ami parts replaces blah blah blah blah blah.
So this is the the title of the item and that's how much it costs $9.
And you will indeed see that in the title of the item.
There are these part numbers, these long part numbers.
Let's see another one.
Let's see the first item in there.
Again, the first item which is.
A compatible A door pivot block compatible.
Kenmore KitchenAid.
Maytag whirlpool refrigerator.
Again, lots of part numbers in there.
So let's now look at what happens if I look at the prompt that that I say that this function created
items.
Even try that again.
Let's print that.
So it comes up formatted with nice empty lines.
So this is what the prompt says.
How much does this cost to the nearest dollar.
I'll talk more about that to the nearest dollar.
In a later times.
We'll talk about why I ended up going with that and and the pros and cons.
So how much does this cost to the nearest dollar.
And here then is there's actually one line for the, for the, for the heading and one line for the
description.
And what you'll see is that, sure enough, these part numbers have been plucked out from this description,
and you'll see that it has been truncated.
When we've got to the end of a certain number of tokens.
It's actually, uh, comes to just under 180 tokens, is what I've kept.
And that's what you can see here.
And you can tell from reading this that it's a rich description of the item itself.
That should be sufficient for training.
Let's take a look at the next one.
This of course, is our pivot block.
Our door pivot block.
Let's go for number 100 and Ice Maker mech.
This is a Samsung replacement part.
So you'll also notice there are a lot of things in here that are parts and replacement parts.
Again consistent with what we saw before that this space could be crowded out by some of the bits and
pieces like like replacement parts that are lower cost.
Although somewhat surprisingly, this this part is $118, so it's not not such a simple part.
I hope I never need this particular, uh, Samsung Assembly ice maker mech.
Um, okay, so this is looking at the training prompt.
This is what we'll be passing in during training time.
And so the model will be will be given this and it will start to learn how best to recreate this price
here during training time.
What about during test time.
What about when it's time to assess whether or not the model is doing any good?
So let's look at this guy at item number 100 and see what we will do when it comes to test time.
We will then.
Provide the model with this.
It's exactly the same but it ends here.
And of course the idea is that the model will have seen so many examples of this, covering such a wide
variety of different items that when it's shown this again at runtime, it will know how to complete
it will it will have a good nuanced understanding based on this description that will help it to complete
this price.
All right.
Let's look at how many tokens we typically have in these items by doing another of our diagrams.
And what you'll see is that the highest number of tokens is 178.
Never quite get to 180, and the average is 176.
It's really sort of crammed in there.
So we've generally generally selected and and crafted data sets that have about this much information.
And it comes to up to 180 tokens.
And as I say, this is going to be very helpful during training because we're going to know the maximum
number of tokens we need to be able to support in any item, and it's also going to keep costs lower
when we end up using frontier models for this.
Okay.
And then let's just have another look at the distribution of prices in these items that we have selected.
Here we go.
So the average price is $100.
Uh, over here the highest price is, uh, almost $11,000.
So in the process of doing some weeding out, we have actually removed that super expensive microwave
along the way.
But we've still got something that's fairly expensive.
You can figure out what that is by by by replicating what I had above.
Um, and you can still see that the distribution is very heavily skewed towards super cheap things that
are presumably replacement parts, as we have been seeing.
So that is another area for us to investigate next time.
Uh, and so I did want to mention something that that visualizing these data sets is something we'll
be doing a lot.
And you will be doing a lot in different ways.
Uh, it's it's nice to be able to take advantage of various features in matplotlib.
And one of them is that it allows you to produce charts with many a huge array of different colors.
And if you would like to know what those colors are, I've included a link that will take you to the
page, uh, in matplotlib, where it will describe the different color schemes that you can use, including,
uh, something called xkcd's colors.
And it's good to take a look at that.
Uh, so this is, uh, just a by the by as a little extra thing for you to bookmark.
Uh, another another little pro tip for today, but the real to do's, what you have to do now, please,
is go and look at the item class.
I realize I went through it quickly.
It's because it's got some of the more gruesome data A scrubbing the data munging that one does based
on real examples of data to make the data as high quality as possible.
And I haven't bored you with all of the details, but that's partly because I trust that you will now
go in and look at the details yourself.
Use JupyterLab to to investigate, try out and understand how these functions have cleaned up some of
the data and got us to a point where we have about 180 tokens of rich description, rich wording for
each of our data points, each of our items that will be used as training prompts and test prompts in
the future.
So next time we'll be expanding this to combine many, many other types of products.
And if you thought this data set was a large data set, you ain't seen nothing yet.
So prepare for that.
But first, a couple more slides to wrap up this day.