Hopefully you found this super satisfying to be able to have this nice business result and have it calling
an LLM twice, we can make this result a little bit more satisfying by adding in something called streaming,
which is so common that you see in these tools that we've experienced ourselves in the chat user interfaces
and streaming is when the information flows back from the LLM and appears in what they call the typewriter
style interface.
And the way you do it is remarkably simple.
Uh, when you are creating your, uh, call to OpenAI chat completions, create, uh, if you want to
not just receive it all back in one go, but you want it to flow back, you simply pass in another parameter
stream equals true.
We'll find when we go and use Claude that it has a slightly different API, but this is one of the rare
times when GPT and Claude are a bit different.
But that's how you do it with GPT stream equals true.
Now what does that mean?
What it means is that what comes back into stream is no longer the the single text response, but instead
you get back something that you can iterate over, and as you iterate over, each chunk of the response
will come through, um, and you can just sort of say for chunk in stream print chunk.
Now, in our case, we want to be a little bit smarter than that because we're we're showing this in
markdown.
Um, and actually it's a little tiny bit fiddly because markdown is something where the, the markdown
characters, you might it might start to stream some markdown characters, and you would then need it
to sort of incorporate that in what it's showing.
So there's a bit of hokey code here to handle the fact that we're going to want to rewrite the full
markdown version for each chunk so you can look through it.
But basically I keep a sort of running track of everything in response, a kind of cumulative track
of all of the chunks that have come back.
And so for each chunk that comes back, I basically, uh, include that in response.
Uh, I strip out if the word markdown is actually in there, I strip it out of the response.
And finally I update the full display to show that.
So this is all a little bit complex.
You wouldn't need this if you were just simply writing text.
Uh, the only reason that you need it this way is because we want to show it in a nice, fancy markdown
way.
And let me show you what I mean by that.
If we now repeat this for anthropic, we'll have to wait a minute while it, uh, while it does that
first finding the links and getting all these pages, and then here you go.
Look at that.
It's the familiar streaming interface that you can see there, and it's done.
And I love the fact that because it's markdown, we also get things like the links in here.
And just to show you what I was talking about.
If you didn't want to have it displaying in markdown, the simpler way that you could do this is that
you could have it say like, uh, something like, um, for chunk in stream.
Uh, print chunk.
And then we'd have to have something like end equals that stops it from printing a new line each time.
And I think this is now let's see if that works.
Uh, this this would now just print it as a series of, uh, little, uh, pieces of text so we won't
get the markdown formatting.
Oh, that's not going to work.
Uh, sorry.
This should of course, be chunk.
I might as well put in all of this like that.
Otherwise, we're getting the objects that are flowing back.
You probably saw me doing that and thought, idiot, there we go.
It shows you this is real time.
Uh, okay, let's try that again.
See if we get a better result now.
So if we simply put in here the the the the the print statement.
Like this.
Then what you'll see coming back.
Here you go.
You see it comes back.
It's super simple.
It just won't be nicely formatted.
But obviously the code, if you do it this way is really simple.
This is all the code you need to be able to stream back results.
So you set stream equals true and then you iterate back that way.
Uh, so now let me remove that and I will uncomment this if you don't know by the way then then a command
and the divide by sign or on the windows it's a windows key and divide by sign, uh is something which
will comment out or uncomment a block of code like that.
It's a useful shortcut to know, uh, all right, let's run this again and just see anthropic one more
time with the nice great formatting.
Finds the links.
And here it is again.
And of course it's going to be amazingly a different brochure every time.
Uh, and, uh, there it is this time.
And now let's try a different company.
Let's try hugging face the ubiquitous open source platform for, uh, AI engineers.
Let's see how we get.
We get some links.
And here comes the hugging face brochure with career opportunities.
Perks.
Get in touch.
And then.
And together, let's build the future of AI.
Very nice.
All right, let's do one more thing to make this fun.
Let's just go all the way back to where we created the system prompt.
Where was that?
All the way back here.
System prompt.
So one of the things about the system prompt is that this is the place where you not only describe the
task that is to be done, but also you talk about the tone and character that the LLM should adopt in
generating this content.
So let's comment this and uncomment a variation right here.
And this variation just includes that.
It should be a short, humorous, entertaining, jokey brochure and we will run that code.
Uh, and because I use very naughtily here, system prompt is like a global variable that I refer to
elsewhere.
So I do believe I don't need to rerun any of this because I have that like hardcoded, I should be able
to come all the way down here and just simply rerun this.
And I believe, if I'm not mistaken, that we're now going to get a jovial, jokey ha ha, here we go.
Welcome to anthropic, where I gets a safety net.
Uh, at anthropic, we're building AI systems you can actually trust, ensuring they're more reliable
than your morning coffee, and easier to understand than your cats mood swings.
Based on sunny land of San Francisco, we're on a mission to make AI a friend, not a foe.
So, I mean, it's just It's extraordinary that by making a small change to the system prompt like that,
we can have such a wonderfully different lens on our company brochure.
I find that just just really, truly remarkable.
I will uncomment I will comment that so that it doesn't confuse you when you see it.
But um, this of course lends me to the first thing I want to say, which is that as you experiment
with this, this is your opportunity to really understand deeply what it means to use prompting to affect
the character of what's generated.
So you can take it a step further.
Make a snarky brochure that's loaded with sarcasm.
Make a brochure in Spanish.
Uh, do add something that will that will translate to a different language.
Uh, try all of these different things.
Um, and this is, this will be very important part of learning how you use prompting to affect the
type of result that you get.
Okay, let me wrap this up before we start talking about too much about exercises.
So what we did today was we extended what we did in day one instead of just calling one LM call to summarize
a website, we ended up making two calls to LM one to collect relevant links and one to then from a
scrape of all of that data to then build a robust company brochure.
Uh, and, you know, as I say, this is like a toy starting version of a gigantic AI in a small way,
because we're dividing up a bigger problem into smaller steps.
But I did want to impress upon you that this this is very applicable to many different kinds of business
problem, this kind of synthesizing information and then generating as a result of it.
Uh, and so you can imagine that you could do this to write marketing content to generate a product.
Uh, tutorial like a guide from product spec, uh, to create some personalized email content.
By reading through a bunch of emails.
So there are so many ways that you can imagine this kind of two step synthesize some data and summarize
it in JSON, and then use that as a way to build some kind of output.
Many different applications of that.
And so what I'd love to see you do is figure out a way to apply this to what you do day to day.
And interesting commercial angle here.
Uh, something that would allow you to think about your area of expertise, where you have the greatest
domain knowledge.
And now given this new these, these skills you have, how could you put that to good use?
So have a think about that.
Feel free to bounce ideas off me at any point, and try and build some examples of that and put them
in your GitHub.
So you have some nice examples of using multiple calls to llms to synthesize information and generate
content.
I will see you for the next video.