And welcome back to our challenge again.
And this time we are working with our beautiful prototype.
Uh, with this time the default set to the Python Hard challenge rather than the simple pi code.
We bring this up.
Here it is.
We're going to kick off the Python run, which you may remember takes about 2720 eight seconds.
So we'll be sitting here for a little while.
While that runs, I am then going to do use GPT to convert the code to C plus plus and run that.
And then we'll do the same with Claude and see how Claude fares and see if there are any differences
from last time.
Uh, almost there.
You can watch Gradio gives us a little timer, which is very handy in these situations.
So we'll know that there we go.
We get the answer that is the right answer.
And it took about 28 seconds.
All right.
We asked GPT to convert this into C plus plus code.
Here is the C plus plus code.
There is the result.
And we will then run that C plus plus code.
And it has the same problem as before.
There's I believe it's a number overflow that's resulting in the answer being zero and not even that
quick.
Because in giving that answer zero, it also had some nested loops happening.
Let's see how Claude fares.
We switch to Claude, we convert the code.
Here we go.
Let's see what Claude has done.
Aha!
Interestingly, this time Claude has not seen that it can do the single loop.
So we've got a different answer from Claude this time.
There we go.
We'll see how Claude does if its code runs.
At least let's run that C plus.
Plus it got the right answer and it took 0.6 seconds.
So Claude at least gets the right answer.
But it is not this time.
This time that we ran with Claude.
It didn't crush it like last time, because it didn't actually spot that opportunity to to to collapse
the loop using Canon's algorithm.
And now remember it's called.
So I guess we can try one more time converting the code and see if it if it gets it on a second attempt.
Let's see.
Let's see, let's see.
That does look like it's one loop doesn't it.
All right.
Let's let's see if it's, uh, that's going to work for us.
It did work.
Second time lucky.
We get the right answer.
And again we have that breathtaking.
Oh my goodness.
It's a whole lot better than last time as well.
Obviously there's some, uh, dependency on what else is running on on my processor.
We're probably down to the sort of the noise levels, but that is 0.4 of a, uh, of a millisecond,
uh, compared to, uh, let's do the maths one more time.
It's embarrassing that I can't do these sort of orders of magnitude in my head, but I'm too afraid
that I'll get off by by too much.
28.3 divided by 0.000446.
Uh, it's more than 60,000 times faster.
Uh, which, of course, is not that surprising given that it's found an algorithm that involves a single
loop rather than a nested loop, but it is great to see that.
So in summary, Claude managed to to to work where GPT four failed.
And Claude sometimes not always, is breathtakingly faster.
Breathtaking, I will say, from doing my experiments that there were a couple of occasions when Claude
also made a mistake with the number rounding and and both Claude and GPT four zero got zero.
But GPT four zero seems to consistently make that mistake.
And more often than not, Claude not only gets it right, but also spots this opportunity to rewrite
the algorithm and be staggeringly faster.
So I think, again, I double down and say, this is a victory for Claude.
Um, and then next week we are going to switch to open source.
We're going to assess open source models.
We're going to see how open source models generate code and use a solution with open source Llms.
The question will be can open source compete with Claude?
3.5 sonnet.
Uh, with this astoundingly fast, uh, result.
See you then.