Okay, time to reveal the results.
It has run to completion.
And here it is.
So a moment to pause.
It turns out that it's actually a little bit worse than the previous results before fine tuning.
Um, I would expect that.
It's just that it's very similar.
It's not actually that the model has gotten any worse.
I would suspect that fine tuning in this case has not helped us, which is obviously disappointing.
I did warn you at the beginning that there would be a disappointment in this session.
Uh, now, having said that, there are some things that it's definitely improved upon.
Unfortunately, they're not reflected in this business metric.
Uh, but it has improved in terms of the, the, the biggest outliers.
I don't know if you remember, but when we ran it before, it was guessing some things that were way
outside the range of a thousand.
I think I showed you there were points like that were far, far too high.
And from seeing our data set of 500, it's it's appreciated that there aren't things that are priced
that much.
And so that has caused something of a, of a nuanced correction to what it's doing.
But other than that it hasn't particularly helped it.
And in fact, unfortunately with this set, this this test set, at least it's actually appears to have
hindered it according to this business metric, the one that we're really focused on, the total difference.
So that's a sobering moment for us.
Uh, and uh, in the next video, I'll explain why.
And the times when fine tuning frontier models can be very helpful and when they can't.
And fear not, there is, there is, there is good news ahead.
Even if this is a is a setback for us, we will we will see more.
See you next time.