The moment has arrived.
Here we go.
We're in fine tuning.
We do fine tuning.
Train.
There is also a line here to push it to the hub after it's finished.
Let's kick that off.
So while it's kicking off, we'll take a quick look.
The GPU is starting at six, which is what we expect.
It's about as much as it needs for the the model itself.
And it's going to be warming itself up and getting ready for this training while it's doing that.
Since we're waiting for it just for a second, I might as well just talk to some of these other training
parameters that we that we didn't mention before.
Um, eval strategy.
So it is typical when you're doing this kind of training that you would repeatedly have a held out data
set that you would use for evaluation, that you would use as a constant set that's not involved in
training, that you would use to get the model to, uh, validate that it's making progress.
Now, I've not done that here.
Um, partly because we've got so much training data that I think it's got plenty to be training on.
Um, just in one epoch alone and and partly because I just performance, the speed of training was so
important.
I didn't want it to be stopping to do validation repeatedly through this.
But it is a best practice to have a validation data set.
We've got plenty of data for it.
We we held out 2000 test data points so you can use a bunch of them, um, as part of your validation
step uh test test set.
So one certain improvement to this, that would be more of a best practice would be to use an eval strategy
and pass in validation data as well as training data.
Um, but but it wasn't a super necessary in this case, so I didn't do it.
Um, okay.
Well, you may notice if you are glancing over this way that the GPU memory has shot up and out of the
40GB of GPU memory that I've got, 38.2 is currently being used.
So I really did squeeze this so that, uh, with 16 batch steps, uh, a batch size of 16, I would
really use up almost the entire GPU, very close to running out there and you can see that stuff is
happening down there.
So let's just, uh, scroll down and see what we're seeing here.
Um, so it's off and running and we can see that that things are happening.
Uh, and now to run the entire three epochs is going to take, uh, according to this estimate here,
uh, just over 24 hours.
26 hours or so.
Uh, so, you know, it is something like eight hours per epoch, um, on this, uh, even on this beefy
machine, because the training data is so enormous, but it's not strictly necessary to go through all
400,000, uh, training data.
And as I say, you can run this on a much lower end box and let it run for for a while, and that is
all completely fine.
Um, but what you're seeing here is it's it's it's off every 50 steps, just as we configured.
We're getting an output that shows us our training loss so far.
Um, and this is now running and outputting to weights and biases.
And in the next video, I will give you a peek into how that's looking.