You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 

430 lines
12 KiB

WEBVTT
00:00.080 --> 00:07.280
Hopefully you found this super satisfying to be able to have this nice business result and have it calling
00:07.370 --> 00:13.340
an LLM twice, we can make this result a little bit more satisfying by adding in something called streaming,
00:13.340 --> 00:19.310
which is so common that you see in these tools that we've experienced ourselves in the chat user interfaces
00:19.610 --> 00:26.420
and streaming is when the information flows back from the LLM and appears in what they call the typewriter
00:26.420 --> 00:27.980
style interface.
00:28.430 --> 00:31.670
And the way you do it is remarkably simple.
00:31.760 --> 00:42.350
Uh, when you are creating your, uh, call to OpenAI chat completions, create, uh, if you want to
00:42.380 --> 00:48.920
not just receive it all back in one go, but you want it to flow back, you simply pass in another parameter
00:48.920 --> 00:50.480
stream equals true.
00:50.510 --> 00:55.880
We'll find when we go and use Claude that it has a slightly different API, but this is one of the rare
00:55.880 --> 00:59.510
times when GPT and Claude are a bit different.
00:59.510 --> 01:02.200
But that's how you do it with GPT stream equals true.
01:02.230 --> 01:03.460
Now what does that mean?
01:03.460 --> 01:10.420
What it means is that what comes back into stream is no longer the the single text response, but instead
01:10.420 --> 01:16.450
you get back something that you can iterate over, and as you iterate over, each chunk of the response
01:16.450 --> 01:22.510
will come through, um, and you can just sort of say for chunk in stream print chunk.
01:22.540 --> 01:27.640
Now, in our case, we want to be a little bit smarter than that because we're we're showing this in
01:27.640 --> 01:28.750
markdown.
01:28.960 --> 01:37.090
Um, and actually it's a little tiny bit fiddly because markdown is something where the, the markdown
01:37.090 --> 01:41.920
characters, you might it might start to stream some markdown characters, and you would then need it
01:41.920 --> 01:43.990
to sort of incorporate that in what it's showing.
01:43.990 --> 01:49.600
So there's a bit of hokey code here to handle the fact that we're going to want to rewrite the full
01:49.600 --> 01:53.230
markdown version for each chunk so you can look through it.
01:53.230 --> 01:59.100
But basically I keep a sort of running track of everything in response, a kind of cumulative track
01:59.100 --> 02:01.980
of all of the chunks that have come back.
02:01.980 --> 02:09.120
And so for each chunk that comes back, I basically, uh, include that in response.
02:09.120 --> 02:14.070
Uh, I strip out if the word markdown is actually in there, I strip it out of the response.
02:14.070 --> 02:18.090
And finally I update the full display to show that.
02:18.090 --> 02:20.850
So this is all a little bit complex.
02:20.850 --> 02:24.270
You wouldn't need this if you were just simply writing text.
02:24.270 --> 02:30.450
Uh, the only reason that you need it this way is because we want to show it in a nice, fancy markdown
02:30.450 --> 02:30.990
way.
02:30.990 --> 02:32.880
And let me show you what I mean by that.
02:32.880 --> 02:38.880
If we now repeat this for anthropic, we'll have to wait a minute while it, uh, while it does that
02:38.880 --> 02:42.780
first finding the links and getting all these pages, and then here you go.
02:42.810 --> 02:43.560
Look at that.
02:43.560 --> 02:50.760
It's the familiar streaming interface that you can see there, and it's done.
02:50.760 --> 02:55.320
And I love the fact that because it's markdown, we also get things like the links in here.
02:55.320 --> 02:57.680
And just to show you what I was talking about.
02:57.680 --> 03:03.590
If you didn't want to have it displaying in markdown, the simpler way that you could do this is that
03:03.590 --> 03:10.550
you could have it say like, uh, something like, um, for chunk in stream.
03:12.560 --> 03:15.260
Uh, print chunk.
03:15.260 --> 03:20.060
And then we'd have to have something like end equals that stops it from printing a new line each time.
03:20.060 --> 03:22.640
And I think this is now let's see if that works.
03:23.090 --> 03:29.330
Uh, this this would now just print it as a series of, uh, little, uh, pieces of text so we won't
03:29.330 --> 03:30.890
get the markdown formatting.
03:31.280 --> 03:33.560
Oh, that's not going to work.
03:34.040 --> 03:34.940
Uh, sorry.
03:34.940 --> 03:37.850
This should of course, be chunk.
03:38.810 --> 03:41.720
I might as well put in all of this like that.
03:41.720 --> 03:43.940
Otherwise, we're getting the objects that are flowing back.
03:43.970 --> 03:47.510
You probably saw me doing that and thought, idiot, there we go.
03:47.600 --> 03:49.340
It shows you this is real time.
03:49.550 --> 03:51.560
Uh, okay, let's try that again.
03:51.590 --> 03:53.180
See if we get a better result now.
03:53.990 --> 03:58.990
So if we simply put in here the the the the the print statement.
03:58.990 --> 03:59.770
Like this.
03:59.770 --> 04:01.090
Then what you'll see coming back.
04:01.090 --> 04:01.930
Here you go.
04:01.960 --> 04:03.700
You see it comes back.
04:03.700 --> 04:05.080
It's super simple.
04:05.080 --> 04:07.450
It just won't be nicely formatted.
04:07.450 --> 04:10.510
But obviously the code, if you do it this way is really simple.
04:10.510 --> 04:13.570
This is all the code you need to be able to stream back results.
04:13.570 --> 04:18.250
So you set stream equals true and then you iterate back that way.
04:19.180 --> 04:29.170
Uh, so now let me remove that and I will uncomment this if you don't know by the way then then a command
04:29.170 --> 04:36.250
and the divide by sign or on the windows it's a windows key and divide by sign, uh is something which
04:36.250 --> 04:39.010
will comment out or uncomment a block of code like that.
04:39.010 --> 04:44.020
It's a useful shortcut to know, uh, all right, let's run this again and just see anthropic one more
04:44.020 --> 04:47.530
time with the nice great formatting.
04:47.800 --> 04:49.180
Finds the links.
04:49.180 --> 04:50.800
And here it is again.
04:50.800 --> 04:55.350
And of course it's going to be amazingly a different brochure every time.
04:55.590 --> 04:58.860
Uh, and, uh, there it is this time.
04:58.860 --> 05:00.600
And now let's try a different company.
05:00.600 --> 05:07.530
Let's try hugging face the ubiquitous open source platform for, uh, AI engineers.
05:07.560 --> 05:09.090
Let's see how we get.
05:09.120 --> 05:10.800
We get some links.
05:10.800 --> 05:16.380
And here comes the hugging face brochure with career opportunities.
05:16.380 --> 05:17.010
Perks.
05:17.040 --> 05:18.060
Get in touch.
05:18.060 --> 05:18.780
And then.
05:18.810 --> 05:21.300
And together, let's build the future of AI.
05:21.330 --> 05:22.530
Very nice.
05:22.560 --> 05:25.740
All right, let's do one more thing to make this fun.
05:25.740 --> 05:31.050
Let's just go all the way back to where we created the system prompt.
05:31.050 --> 05:32.910
Where was that?
05:32.940 --> 05:34.710
All the way back here.
05:34.710 --> 05:36.210
System prompt.
05:36.210 --> 05:41.220
So one of the things about the system prompt is that this is the place where you not only describe the
05:41.220 --> 05:49.200
task that is to be done, but also you talk about the tone and character that the LLM should adopt in
05:49.200 --> 05:50.760
generating this content.
05:50.760 --> 05:57.690
So let's comment this and uncomment a variation right here.
05:57.690 --> 05:59.730
And this variation just includes that.
05:59.730 --> 06:05.610
It should be a short, humorous, entertaining, jokey brochure and we will run that code.
06:05.820 --> 06:11.970
Uh, and because I use very naughtily here, system prompt is like a global variable that I refer to
06:12.000 --> 06:12.600
elsewhere.
06:12.600 --> 06:17.640
So I do believe I don't need to rerun any of this because I have that like hardcoded, I should be able
06:17.640 --> 06:21.600
to come all the way down here and just simply rerun this.
06:21.630 --> 06:29.760
And I believe, if I'm not mistaken, that we're now going to get a jovial, jokey ha ha, here we go.
06:29.790 --> 06:33.900
Welcome to anthropic, where I gets a safety net.
06:34.380 --> 06:39.240
Uh, at anthropic, we're building AI systems you can actually trust, ensuring they're more reliable
06:39.240 --> 06:43.920
than your morning coffee, and easier to understand than your cats mood swings.
06:44.670 --> 06:50.100
Based on sunny land of San Francisco, we're on a mission to make AI a friend, not a foe.
06:50.130 --> 06:57.620
So, I mean, it's just It's extraordinary that by making a small change to the system prompt like that,
06:57.650 --> 07:03.740
we can have such a wonderfully different lens on our company brochure.
07:03.860 --> 07:07.220
I find that just just really, truly remarkable.
07:07.430 --> 07:12.860
I will uncomment I will comment that so that it doesn't confuse you when you see it.
07:13.010 --> 07:20.990
But um, this of course lends me to the first thing I want to say, which is that as you experiment
07:20.990 --> 07:28.460
with this, this is your opportunity to really understand deeply what it means to use prompting to affect
07:28.460 --> 07:30.560
the character of what's generated.
07:30.560 --> 07:32.840
So you can take it a step further.
07:32.840 --> 07:36.740
Make a snarky brochure that's loaded with sarcasm.
07:36.740 --> 07:38.810
Make a brochure in Spanish.
07:38.930 --> 07:43.790
Uh, do add something that will that will translate to a different language.
07:43.790 --> 07:46.010
Uh, try all of these different things.
07:46.160 --> 07:53.230
Um, and this is, this will be very important part of learning how you use prompting to affect the
07:53.230 --> 07:54.910
type of result that you get.
07:55.930 --> 08:01.180
Okay, let me wrap this up before we start talking about too much about exercises.
08:01.180 --> 08:09.010
So what we did today was we extended what we did in day one instead of just calling one LM call to summarize
08:09.010 --> 08:15.700
a website, we ended up making two calls to LM one to collect relevant links and one to then from a
08:15.700 --> 08:21.700
scrape of all of that data to then build a robust company brochure.
08:22.000 --> 08:28.210
Uh, and, you know, as I say, this is like a toy starting version of a gigantic AI in a small way,
08:28.210 --> 08:31.600
because we're dividing up a bigger problem into smaller steps.
08:31.600 --> 08:38.410
But I did want to impress upon you that this this is very applicable to many different kinds of business
08:38.410 --> 08:43.420
problem, this kind of synthesizing information and then generating as a result of it.
08:43.480 --> 08:49.810
Uh, and so you can imagine that you could do this to write marketing content to generate a product.
08:49.930 --> 08:57.240
Uh, tutorial like a guide from product spec, uh, to create some personalized email content.
08:57.300 --> 08:59.670
By reading through a bunch of emails.
08:59.670 --> 09:05.310
So there are so many ways that you can imagine this kind of two step synthesize some data and summarize
09:05.310 --> 09:09.930
it in JSON, and then use that as a way to build some kind of output.
09:09.960 --> 09:12.120
Many different applications of that.
09:12.120 --> 09:17.730
And so what I'd love to see you do is figure out a way to apply this to what you do day to day.
09:17.730 --> 09:19.920
And interesting commercial angle here.
09:19.950 --> 09:24.990
Uh, something that would allow you to think about your area of expertise, where you have the greatest
09:24.990 --> 09:25.860
domain knowledge.
09:25.860 --> 09:31.500
And now given this new these, these skills you have, how could you put that to good use?
09:31.500 --> 09:33.390
So have a think about that.
09:33.390 --> 09:39.570
Feel free to bounce ideas off me at any point, and try and build some examples of that and put them
09:39.570 --> 09:40.380
in your GitHub.
09:40.380 --> 09:47.700
So you have some nice examples of using multiple calls to llms to synthesize information and generate
09:47.700 --> 09:48.600
content.
09:48.630 --> 09:50.400
I will see you for the next video.