You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 

463 lines
12 KiB

WEBVTT
00:00.260 --> 00:02.750
Welcome back to making chatbots.
00:02.780 --> 00:04.070
Let's keep going.
00:04.070 --> 00:09.650
So for the next part we're going to beef up the system message to something a bit more interesting.
00:09.650 --> 00:10.700
System message.
00:10.700 --> 00:12.710
You're a helpful assistant in a clothes store.
00:12.740 --> 00:15.920
You should try to gently encourage the customer to try items on sale.
00:15.950 --> 00:17.780
Hats off 60% off.
00:17.780 --> 00:19.880
Most other items are 50% off.
00:19.880 --> 00:24.530
For example, if the customer says I'm looking to buy a hat, you could reply, wonderful!
00:24.560 --> 00:28.100
We have lots of hats, including several part of our sales event.
00:28.100 --> 00:32.330
Encourage the customer to buy hats if they're unsure what to get.
00:32.330 --> 00:36.050
So what you're seeing here is a few things going on in this system prompt.
00:36.050 --> 00:42.230
We've got some facts that are being provided about the sales, the hats and other items.
00:42.350 --> 00:47.930
Um, you've got an example, an example of if the customer says this, you could say that.
00:47.930 --> 00:52.280
And that example both is a way to establish tone and style.
00:52.400 --> 00:58.850
Um, and it's also a way to introduce more facts about hats, uh, into the conversation.
00:58.850 --> 01:01.670
So this is all example of one shot prompting.
01:01.670 --> 01:03.150
And you could argue Multi-shot prompting.
01:03.150 --> 01:06.720
So we're giving it a few different sort of nuances of how to reply.
01:06.930 --> 01:09.810
Um, and building that into the system message.
01:09.810 --> 01:13.950
There are other ways of doing it that we'll talk about, or at least at least another way of doing it.
01:14.040 --> 01:16.980
But this is one very effective way.
01:16.980 --> 01:25.950
So we we add in that system message, and we're now going to have a chat with the with with the chat
01:25.980 --> 01:26.580
bot.
01:26.670 --> 01:34.170
So again we write our method of our generator chat um which takes a message and history because that's
01:34.170 --> 01:36.030
what Gradio wants to call us with.
01:36.210 --> 01:43.650
Um, and we first convert that into the format that OpenAI expects, um, by building the usual list
01:43.650 --> 01:44.730
that you're familiar with.
01:44.730 --> 01:49.200
I should also mention, I don't know if I mentioned this last time, that at the end here we have to
01:49.230 --> 01:56.550
add in, of course, into that list the latest message that the user is sending that gets added to the
01:56.550 --> 01:58.770
bottom as role user content.
01:58.770 --> 02:00.480
And that message.
02:00.480 --> 02:08.460
Then of course, we make a call that at this point, you have ingrained in your deepest memory the create
02:08.460 --> 02:12.630
call and we have stream is true and we stream back results.
02:12.630 --> 02:13.530
So here we go.
02:13.560 --> 02:14.760
We'll bring that up.
02:14.760 --> 02:16.680
We'll bring it up in a separate window again.
02:16.710 --> 02:17.340
Why not.
02:17.340 --> 02:20.910
And let's talk to our shopping assistant.
02:21.210 --> 02:22.050
Hi there.
02:23.520 --> 02:24.870
Welcome to our store.
02:24.900 --> 02:25.920
How can I assist you today?
02:25.920 --> 02:27.510
Are you looking for anything specific?
02:27.510 --> 02:33.090
Say, uh, I'd like to buy some shoes.
02:34.110 --> 02:35.280
Great.
02:36.480 --> 02:38.760
We have a lovely selection.
02:38.760 --> 02:41.580
While you're browsing, I want to mention we have a fantastic sale going on.
02:41.580 --> 02:43.140
Most items are 50% off.
02:43.140 --> 02:47.340
If you're open to it, we have some stylish hats that are 60% off.
02:47.370 --> 02:50.340
They might be the perfect complement to your new shoes.
02:50.340 --> 02:52.170
Would you like to take a look at both?
02:52.620 --> 02:56.490
So I want to point out that that it's obviously figured out.
02:56.490 --> 03:01.170
It's got the knowledge that we supplied to it in the system prompt.
03:01.170 --> 03:07.990
But you hopefully will also notice that the sort of enthusiastic, effusive style that I used in that
03:07.990 --> 03:16.030
system prompt has rubbed off in a way that it's communicating in this kind of, uh, very, uh, amiable
03:16.030 --> 03:16.960
fashion.
03:17.290 --> 03:22.720
Um, and that's a big part of this kind of one shot or multi-shot prompting when you set the tone,
03:22.720 --> 03:25.930
give examples of how it should reply.
03:26.860 --> 03:28.510
Uh, so let's keep going.
03:28.510 --> 03:31.180
Let's take that system message and and add in.
03:31.180 --> 03:35.950
If the customer asks for shoes, you could respond that shoes are not on sale.
03:35.980 --> 03:37.720
Let's say should respond.
03:37.720 --> 03:39.910
Should respond that shoes are not on sale today.
03:39.910 --> 03:42.760
But remind the customer to look at hats.
03:42.760 --> 03:44.170
So let's try this again.
03:44.170 --> 03:45.580
Let's see how this does.
03:45.640 --> 03:48.160
Uh, it's got another fact to add.
03:48.310 --> 03:49.630
Uh, you could argue that that's.
03:49.660 --> 03:54.220
Yeah, that's like another, uh, multi-shot prompt.
03:54.250 --> 03:55.210
Let's say.
03:55.210 --> 03:55.990
Hi there.
03:57.640 --> 03:58.420
Are you looking.
03:58.420 --> 04:04.540
I'd like to buy some shoes.
04:04.540 --> 04:05.900
That sounds great.
04:05.930 --> 04:08.630
I should mention shoes aren't on sale today.
04:08.630 --> 04:10.940
But while you're here, have you thought about checking out our hats?
04:13.640 --> 04:20.030
You've got to feel sorry for this poor customer who's going to get repeatedly pitched hats.
04:20.330 --> 04:27.860
Um, so you can see again how it's established, um, that shoes aren't on sale, but that hats very
04:27.860 --> 04:28.910
much are.
04:29.180 --> 04:34.820
And that is an example, then, of Multi-shot prompting, uh, in that we're giving it more examples
04:34.820 --> 04:35.930
to learn from.
04:36.260 --> 04:44.510
Um, so another thing that we can do that is interesting, um, is that whenever you've seen these constructions
04:44.510 --> 04:47.960
so far, you've always seen us beginning with the system message.
04:47.960 --> 04:49.640
The system messages come at the top.
04:49.640 --> 04:54.830
But in fact, with the OpenAI call, there's you're not constrained to have the system message at the
04:54.830 --> 04:55.070
top.
04:55.070 --> 04:58.460
You can add in more system messages as you go.
04:58.730 --> 05:05.930
Um, and so for example, one of the things we can do, um, uh, and let me apologize for some very
05:05.970 --> 05:09.120
hacky code here that I will not recommend, but it's here to show the example.
05:09.120 --> 05:15.390
It's not the way that you should do it in practice, but what we could do is when we're building this
05:15.390 --> 05:23.280
chat generator, we could put in here if this current message that the user is sending us contains the
05:23.280 --> 05:24.510
word belt.
05:24.510 --> 05:28.020
And you can see in a rather unawesome way, I've just looked for the string belt.
05:28.050 --> 05:33.150
Of course, I should be testing whether it's the full word, and I should be thinking about uppercase,
05:33.180 --> 05:34.770
lowercase and so on.
05:34.770 --> 05:38.220
I'm not doing any of that, which is very naughty of me, but it shows the point.
05:38.220 --> 05:43.560
So if belt is in the word message, it's going to add into this set of messages.
05:43.590 --> 05:50.490
Another system message saying for added context, the store does not sell belts, but be sure to point
05:50.490 --> 05:58.500
out items on sale so that will be then added in to the prompt if the user asks for a belt.
05:59.940 --> 06:06.030
So let's give that a try and bring this up here.
06:08.250 --> 06:09.180
Hi there.
06:11.130 --> 06:12.060
Welcome to the store.
06:12.090 --> 06:13.080
I can assist you.
06:13.530 --> 06:17.310
I'd like to buy a belt.
06:19.110 --> 06:19.710
I'm sorry.
06:19.710 --> 06:20.910
We don't carry belts.
06:20.910 --> 06:24.390
However, we have fantastic items, including hats.
06:24.420 --> 06:26.130
60% off.
06:26.310 --> 06:27.780
So there you go.
06:27.810 --> 06:28.620
There you go.
06:28.650 --> 06:29.880
It's, um.
06:30.210 --> 06:33.270
Uh, definitely pays attention.
06:33.270 --> 06:41.310
You can see that when the system message is added in as another row in this, uh, in this list of messages,
06:41.310 --> 06:42.900
it pays attention to it.
06:42.900 --> 06:49.140
And that gives us the opportunity to add context into the conversation.
06:49.140 --> 06:58.740
And this is a, uh, whilst it is, of course, uh, very kludgy code to be detecting a word, a substring
06:58.740 --> 07:02.640
like that, you can imagine that you could beef this up to be a bit more robust.
07:02.670 --> 07:07.830
You could properly you could have a little dictionary which looks for particular words.
07:07.830 --> 07:15.910
And when it finds them, it could use them to then enrich the the context in the right way.
07:15.970 --> 07:23.140
So it gives you a little, a little ability to be looking things up and adding them into the context.
07:23.170 --> 07:29.950
Now, you may be familiar with with some things about Rag, and you may be aware that that is a lot
07:29.950 --> 07:31.150
of what rag is about.
07:31.180 --> 07:38.650
Rag is about finding extra information that's relevant to the prompt, and adding it in to the context
07:38.680 --> 07:41.200
of the message that gets sent to the LM.
07:41.230 --> 07:47.260
Now, of course, Rag does that in a much more sophisticated and intelligent way than this hokey piece
07:47.260 --> 07:48.430
of code right here.
07:48.610 --> 07:55.540
But you can think of this as a as a light, a baby version of rag, and as an exercise for you.
07:55.540 --> 08:01.030
You can certainly beef this up a bit, and at the very least, use regex to make it look for a particular
08:01.030 --> 08:01.540
word.
08:01.540 --> 08:07.930
Maybe have a little dictionary that has the the words, the different items in the store together with
08:07.930 --> 08:13.730
their price so that you could add that in as a system message to give it more context.
08:13.730 --> 08:20.270
You could try that out and see that you could build a chatbot that actually knows about the prices of
08:20.270 --> 08:23.000
the goods in its store, and that would be pretty cool.
08:23.750 --> 08:27.200
So that wraps up this particular experiment.
08:27.200 --> 08:32.270
I do just want to mention, I alluded earlier to the fact that there are other ways of doing Multi-shot
08:32.270 --> 08:38.840
prompting other than shoving it in the system prompt, and the other way is that you can have a user
08:38.840 --> 08:43.430
assistant, user assistant set of messages that hasn't actually happened.
08:43.460 --> 08:49.580
You can have a fictitious exchange between the user and the assistant that you include in the conversation
08:49.580 --> 08:56.930
before the current conversation, and use that as a way to prime the LM with similar conversations,
08:56.930 --> 09:02.990
so that it gets a sense of how it's responded to other questions.
09:03.140 --> 09:09.200
You can use that again, both to train it on style and also to supply extra facts.
09:09.200 --> 09:14.640
So there could have been an earlier interaction when there had been a question about a belt, and the
09:14.640 --> 09:19.740
assistant had already replied that there are no belts in the store, and it would have learnt from that.
09:19.740 --> 09:25.380
So either technique, they have pros and cons, whether you supply it in system prompts or whether you
09:25.410 --> 09:33.270
give example user assistant interactions for it to have as part of the input context for it to to be
09:33.300 --> 09:34.410
able to absorb.
09:34.560 --> 09:38.640
Um, and my ask to you is to try them both out.
09:38.640 --> 09:43.680
So update this so that it uses user assistant interactions instead of a system prompt and see how that
09:43.680 --> 09:44.160
works.
09:44.190 --> 09:47.430
See if you think you get a better or a worse clothes store assistant.
09:47.430 --> 09:53.460
And then also make this change to make this a whole lot more robust, have a dictionary of different
09:53.460 --> 10:00.780
items in the store, look up their prices or their sale amounts, and then add that as context into
10:00.780 --> 10:09.060
the conversation so that the assistant responds with some expertise and have fun doing it, and I will
10:09.060 --> 10:12.300
see you for the next video to wrap up this day.