So the good news is that this is the very final video about data set curation.
You were probably fed up of data set curation.
Now there's just one more piece and then we are done.
So we have crafted an outstanding data set of which we should be very proud.
Let's do some final peeks at it.
Um, one question that you might ask.
Um, well, I'm going to ask it anyway.
Is is it possible that the price of an item is related?
It's correlated to how long the description is of that item.
You might imagine a situation where the higher price things tend to have more information.
And that would be worth us understanding.
Um, because yeah, that's something that the model would learn from quickly.
And it gives us a good sense, perhaps when we look at traditional approaches about how we might approach
it.
So this is a nice little scatter plot that is going to show us, um, each of the, the sizes on the
x axis.
It's going to show us the length of the description, and on the y axis it's going to show us the price.
Um, let's have a look at this across the full sample data set.
So here we go.
Here's a nice nice picture for you.
So there are 400,000 points on this picture.
Uh and it's something to look at.
You can see in it there's there's a lot to digest.
You can see this interesting pattern that's happening as prices tend to be more prevalent at these boundary
points.
The $799 priced items, um, and you can see, of course, that there are many more cheaper items.
And you can see that there is something of apparently a correlation that, uh, items which have longer
descriptions do appear, perhaps sometimes to have a trend of being the more expensive ones.
But it's not clear that there's a significant correlation in that regard.
So there's something there, but it's nothing major.
So we suspect that traditional machine learning, when trying to look at something like that will probably
not find any major correlation.
So just an example of the kind of, um, diagram that you can come up with to try and get insight into
different aspects of your data.
One other thing I want to talk about for a moment more is tokens.
Um, and uh, the, um, we're going to be working a lot more with tokens when we get to actually training
against an open source model, but it's worth looking at tokens right now.
So I've just written this function report, which takes an item and which will then print, uh, the
prompt, first of all, the full training prompt that will be used during training.
And then the last ten tokens in that prompt, and then it will decode those.
So we'll see the bits of text that map to the last ten tokens.
And if you're wondering why the last ten you're going to see in just a second.
So let's pick a random number, number 40,000 and run this.
Okay.
So this here sorry for all of this text here.
That is the prompt that's going to be sent to the LLM to learn from.
Um, it's going to be asked how much does this cost to the nearest dollar.
And then it's going to get a description.
And then price is and then this which is the price rounded to the nearest dollar.
You'll note if you look in the item code that when building the training prompt, it rounds this to
the nearest dollar.
So if we look at the last ten tokens you can see what's happening here I'm printing out underneath it.
What are those ten tokens.
And I just wanted to show you that in terms of the final few tokens, price gets mapped to one token
is gets a token with that start of of of word space before it dollars again with the start of word,
and then the number 34 is getting mapped to one specific token.
And this is, as I say, just a feature of the llama tokenizer that it does.
As with GPT, it does have a separate token for every three digit number.
Some of the other tokenizers the other models do not.
Um, and whilst this isn't required for our project, it does make things a bit simpler for us later.
And then the period gets one token and the .00 gets one token.
Let's do another sample.
Let's do something completely, uh, something in a different location altogether.
Number 10,000.
And this is a something that's rather cheap.
It costs $9 and price is 9000.
Let's go for something that's near the end of the data set 398,000.
And this is a, um, uh, coilover damper kit.
And this price is $765.
And you'll see once more that the 765 gets mapped to one token.
So you should satisfy yourself this this sample is of course sorted by cheapest first ish because we've
gone through sampling, uh, in each, each category.
So, so rounded to the nearest dollar.
It is sorted by cheapest in the lower, um, parts of the sample, and the most expensive in the higher
parts of the sample.
And you can satisfy yourself that we are getting this effect, that every number from 1 to 999 is getting
mapped to one token, just as it says here.
And as I say one more time, when, uh, look at the quantity or gamma or phi three tokenizers, you'll
see that that's not the case.
Um, it turns out to be a little bit handy for us later on, but it's not required and definitely later.
If you want to experiment with using other models like Quantum Gemini three, you can simply switch
it in and it will work.
You'll just find here that it will be mapped to multiple tokens, not to the one token for the three
digit number.
Okay, big sigh of relief.
We've made it through data curation.
The last part of it all is to finish things off and upload to the hub.
And what we're going to do to start with is shuffle up our data set, because it's no good at all if
it's sorted in order of cheapest.
First we need a nice jumbled data set.
Um, and first I, um, set the random seed because I want to make sure that we always are working with
exactly the same data set so that you can reproduce exactly the same stuff that I will and get the same
outcomes.
Um, we use Random.shuffle to shuffle things up, and then I take the first 400,000 as my training data
set.
And then the next 2000 as the test set.
Now I hear you.
You cry, you data scientists.
That one normally takes, like, at least a 5% or 10% test data set here.
Um, and you can absolutely feel free to do so because obviously we've got we've got 8000 data points
right, right here.
And you can also, of course, sample more to have a bigger data set.
It won't be necessary for us because we're going to find that we're only going to use a few hundred
for testing.
And that's going to give us very accurate results.
And we get diminishing returns if we keep testing against more and more.
So this is plenty for our purposes for this project, but it is a best practice.
I don't know if it's a best practice.
It's a common practice to reserve at least 5% of these other, uh, for the test data set, and sometimes
to to separately have 5% for test and a 5% for validation, as I talked about before.
Um, not required for this purpose, but by all means, you can do it if you wish and have that as an
extra data set that you manage.
Um, but anyway, we will do that.
We will jumble it up.
It's been divided into a training dataset of 400,000 and a test set of 2000.
Let's have a look at the first the the the test first element.
The test prompt that you remember is the prompt without revealing the answer.
This is the prompt that will be sent to sorry, I'm looking at the training prompt first, then we'll
look at the test prompt.
The training prompt is the one that does have the answer.
So the training prompt says how much does this cost to the nearest dollar.
It is a Delphi fuel pump module.
Um, and uh yeah.
How about that.
It costs $227.
I would have had no clue about that.
So this is an example of something that will be sent to an LM as part of training, because it contains
the description and it contains the price.
Um, so let's look at a test prompt.
Now the test prompt is going to show us something that will be used, which will have the description,
but it will not have the price.
And this is the first item in our test set.
So there we have it.
Uh, let's have a quick look at the distribution of prices for the first 250 test points, because these
are actually the points that we'll be using most of the time for actually testing our model.
And you can see there's a nice healthy spread of different prices here.
There's plenty of things in the higher area that will test whether the model can handle expensive things.
And then, you know, the majority are the cheaper priced with a good variety of prices in our test
data set.
Okay, finally, finally, we now turn this into a series of training prompts and test prompts, uh,
which is just simply plucking out the the prompt and the test prompt that we just looked at, along
with the prices.
This little piece of code here will upload it to the hugging face.
I will turn it into a data set object suitable for the hugging face hub, by calling the Fromdict for
a data set and then putting that into a data set dict.
And then finally this line here will upload your data set to the Hugging Face hub so that you can continue
to use it and download it for future.
Uh, when we get to to fine tuning, uh, but I'm not going to run it because I've already run it.
And this is for you to put in your username.
I have this uploaded to to my to to uh, um, sorry, I have it uploaded to my username here.
So you will be able to also just retrieve the data that way too.
If you wanted to short circuit all of this data curation, which hopefully you do not want to do.
Um, and then as a final thing here, um, I'm going to turn this train and test the collection into
a pickle file.
I'm going to pickle it and put it into a file so we can load it for future days so we don't have to
go through all of this rigmarole again of building our lists.
So if you're familiar with Python pickles, it's super easy way to take a Python object and dump it
out to a file.
And now that I've run that, there will be two new files here test dot pickle and train dot pickle that
will contain my training and test data set.
And with that we have completed our data curation work.
Please can I leave with you to investigate the data set more and to also confirm when you try out this,
this exercise of trying to tokenize, uh, different, um, uh, different data points that you always
get the case that three digit numbers tokenized to one token and get a sense for those tokens.
And with that, I will see you back with the slides for a wrap up.