Welcome back to the the moment when we bring it all together into a beautiful user interface.
But first, just a quick look one more time at the inference endpoint screen in Huggingface where you
can see my running code.
Quine 1.57 billion chat inference.
We can.
I just wanted to show you that you can come into this and take a look at how your inference endpoint
is running, and you can do things like see analytics, see what's going on, see the number of requests,
which, even though I made some, is not enough to get on the radar.
Uh, latency, CPU usage.
Oh, there we go.
A little blip in CPU usage from what we just did and GPU usage.
Nice.
Uh, and you can also go over to cost and see that I've spent $3.64 so far on, uh, on this particular,
uh, model.
Um, okay.
Now, with that in mind, let's leave it, I think, on the analytics, let's go back to our Jupyter
lab.
Um, and let's wrap this code to call Code Kwan in a nice little stream method, just like the other
stream methods that we've already done before for GPT four and for Claude stream Quen.
Same kind of method, but of course it's the same function.
It's going to do it very differently.
It's going to create a tokenizer.
It's going to, of course, uh, turn Python into the usual messages list.
It's going to apply the chat template.
So we now have this in the text that is ready for tokenization.
And then we make the magical call to inference client using the URL for our endpoint and passing in
our Huggingface token.
And here we are doing client text generation.
Here's our text we want to stream.
And that's our max new tokens.
And then back comes the results.
As we stream back each token we yield the total of everything so far because hopefully you remember
that is what Gradio expects.
It expects to a sort of cumulative total of everything that's been received so far in all of its chunks.
So that function there stream collection is a companion function to the others we wrote before for stream
GPT for stream Claude.
So now we can have an optimized method that will replace the previous optimized method for optimizing
code, which can flip between three models GPT Claude or Code Kwan.
Um.
And here we have the total of our user interface code for Gradio.
Make sure I run this.
Uh, so you'll remember how simple this is.
It's crazy.
Uh, we have a nice little title, and then we have a row for our Python code and C plus plus code.
We have a row for selecting the model.
And now we've added code Kwon to the three to the previously two models that you could choose between.
And we've got a button to convert the code, a button to run Python, a button to run C plus plus,
and then some output boxes for the Python results and the cplusplus results.
And then these are the three actions.
The three places where if a button is clicked, we take some kind of action.
And I love the way that it just simply reads like English.
If someone wants to convert, if they press convert button, it calls the optimize function.
This is the inputs and that's the output.
If they press the Python run button, it executes Python.
The input is the Python code, the output is the python out, and the same for the C plus plus button
as well.
It should it look super simple.
And that's because it is super simple.
And with that we're going to launch it.
Fingers crossed this is going to work beautifully for us.
All right so here is our user interface.
Uh, and um, what you're seeing here of course, is the Python code for the simple, uh, pi calculation.
And why not?
Let's just try doing it for, uh, for GPT.
You'll remember that's the C plus plus equivalent.
Let's run the Python variation.
If I remember right this is about eight seconds.
So we have to wait for this to count to about eight.
And we should get the Python results.
There it is 8.6 seconds.
There is good old pi at least to some number of decimal places.
And now we'll run the C plus.
Plus that came back from GPT four.
And great in 0.06 of a second a nice greater than 100 x improvement.
Now one more time for Claude.
We convert the the code courtesy of Anthropic's Claude.
Um, there it is.
And now we will run Claude's C plus.
Plus.
And it narrowly beats, uh, GPT four again.
But I think it has this line in here, and maybe it has allowed it to be slightly faster.
Maybe Claude's code really is quicker.
Um, they're so similar that I am suspicious.
This is.
This all gets optimized anyway by the compiler.
But it's possible this is consistently slightly faster.
So you may be a C plus plus expert.
That can tell me.
And you may be able to try it yourself and satisfy yourself, whether on your architecture it is faster
or not.
But anyway, that is not the point.
What we're here to see is how does code quality measure up?
Can it convert?
Does it make sense and is it any different?
Let's press the convert code button and see what happens.
So first of all as we know it's more it's got some chattiness to it.
It hasn't correctly stripped out its explanation.
So we will need to delete that.
But we'll let it get away with that.
We won't ding the code model for adding that extra.
Remember, this is all streaming right now as as we watch it from the endpoint.
If I go over to here, I may need to refresh that.
We should be seeing that we do.
We do indeed see a blip of CPU and GPU as it streams back the results, I love it.
Uh, and so here by now, uh, go down to the, uh, sorry to our gradient screen.
Here we go.
Uh, we have the, the, uh, full solution.
So what we're going to do now is we're going to remove the stuff at the top, and we're going to remove
the explanation at the end that we don't need.
And we are going to run this C plus plus code to see how code Quinn has done.
Let's give it a try.
And it ran and it was fast.
It was about the same as GPT four.
Oh, I imagine it's about the same.
Uh, and I see it doesn't have that pragma thing in there, but it seems to have done a great job.
It's got the same answer.
And I think that is certainly a success for code.
Quinn.
Uh, and again, remember the difference in model parameters code Quinn running here with its 7 billion
parameters and compared with the, uh, the uh, hundreds of, of of, uh, sorry, there are more than
2 trillion parameters that you've got in GPT four and Claude.
Uh, so let's now go back here and let's, let's, uh, raise the bar.
Let's make the challenge harder.
Let's change this value to be the Python hard, the code which calculates the maximum subarray sum.
And we will see now how our open source model can handle this complicated case.
So what's it doing its thing.
So already there is a problem.
There is a problem.
And that problem is that, uh, it has decided to reimplement the approach for generating random numbers,
changing the approach that we had set with this, uh, LCG, um, technique for generating repeatable
and consistent random numbers between the implementations.
Now, that's despite the fact that I very clearly put in the system prompt that it should not change
the functionality around random number generation.
So again, I was not able to convince Coetquen to change that strategy.
Uh, you should experiment with this, see if you can do better.
But I was not able to do so myself with some experimenting.
Uh, it's almost finished.
There we go.
It's done.
So we will take out what comes at the end, and we will take out what comes at the beginning.
And now the moment of truth.
We will run the C plus plus code from Code Kwan and scroll down.
Uh, and what we find is, of course, that the number does not match.
If you remember the the result from before.
So unfortunately, Code Kwan has not been successful in replicating the number, and that's no surprise.
That is because of course, it has, uh, got a its own random number generator.
Um, it's, um, done some, uh, some interesting stuff here.
It, uh, does appear to have potentially recognized the more efficient methodology, but since the
numbers don't match, we can't validate that it has, in fact done everything correctly and got the
right number.
So unfortunately, unfortunately, I was so very hopeful.
Codeclan did Laudably Codeclan was able to pass the the pie test, the simple test, but Codeclan did
stumble with the harder test and wasn't able to reproduce the same exact answer as the Python code,
which was its mission.
So from that perspective, unfortunately, the frontier models come up on top.
Clawed again for the win.
Uh, and Codeclan didn't quite make it.
I will see you next time for a wrap up.