You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 

424 lines
10 KiB

WEBVTT
00:00.800 --> 00:05.960
Welcome back to Jupyter Lab and welcome to Day Five's Lab.
00:05.960 --> 00:12.020
And this is going to be lots of creativity and hopefully lots of entertainment.
00:12.020 --> 00:16.910
So to start with I have copied the day four Jupyter Lab.
00:17.030 --> 00:19.370
And I've duplicated that.
00:19.370 --> 00:20.420
And then I've extended it.
00:20.420 --> 00:23.570
So everything above where I am now is just a repeat of day four.
00:23.600 --> 00:31.670
Creates the AI assistant for our airline called flight flight AI, something like that, and arms it
00:31.670 --> 00:33.590
with a tool to be able to get ticket prices.
00:33.590 --> 00:39.140
All of that is already there and I've executed it ready for our showtime today.
00:39.140 --> 00:40.820
We're going to go multi-modal.
00:40.820 --> 00:46.250
We're going to use Dall-E three, which is the image generation model that sits behind GPT four.
00:46.760 --> 00:48.800
We're going to use it to make some images.
00:48.800 --> 00:52.790
And let's start by putting it into a function called artist.
00:52.790 --> 00:57.770
Before that, there are two, uh, service announcements I should make.
00:57.950 --> 01:03.830
Uh, first of all, I should point out that the price associated with generating an image is not tiny.
01:03.880 --> 01:10.150
Everything that we've done so far, I hope, has had a de minimis price in the fractions of a cent.
01:10.300 --> 01:16.090
Unless you've been generating tons of lengthy brochures, you have not racked up a significant bill
01:16.090 --> 01:17.830
from running this course so far.
01:17.950 --> 01:21.880
But now we are doing something that's slightly more on the radar.
01:21.910 --> 01:25.420
Each image that we generate will cost $0.04.
01:25.450 --> 01:30.700
Now, I put it to you that when you see these images, you will agree that they are well worth $0.04
01:30.730 --> 01:31.360
each.
01:31.570 --> 01:34.720
And they are super creative and high value.
01:34.720 --> 01:35.590
And I love them.
01:35.590 --> 01:37.630
So I think it is money well spent.
01:37.630 --> 01:41.650
But I do want to inform you of that so that you can decide whether you want to spend your $0.04 each
01:41.650 --> 01:42.310
time.
01:42.700 --> 01:49.120
Uh, the other thing to mention is that there is a little bit of, uh, um, uh, there's there's a
01:49.120 --> 01:56.770
there's a point about whether or not one should use the term LM when referring to image generation and
01:56.770 --> 01:58.270
audio generation and the like.
01:58.300 --> 01:59.290
Text to audio.
01:59.320 --> 02:03.400
Because of course, these are not large language models sitting behind the scenes.
02:03.430 --> 02:10.090
Now, what tends to happen these days is that people use LM as a bit of a general term for the models
02:10.090 --> 02:12.280
that sit behind gen AI systems.
02:12.280 --> 02:19.450
So actually, in practice, I think this very much is part of the skill set and toolkit of an LM engineer.
02:19.450 --> 02:23.800
But I should mention that, of course, strictly speaking, these aren't language models.
02:23.800 --> 02:30.730
These are image models and audio models that we'll be playing with right now as we add them to our agent
02:30.730 --> 02:31.630
framework.
02:31.750 --> 02:34.720
Anyways, with that preamble, let's get on with it.
02:34.720 --> 02:39.220
So we start by importing some useful image libraries.
02:39.220 --> 02:40.420
Well, the first one isn't.
02:40.570 --> 02:44.260
First two aren't image libraries, but some, some, uh, utilities.
02:44.260 --> 02:51.520
And then the Python image library is going to be very useful for us, a very handy common library.
02:51.760 --> 03:00.820
Uh, so the next thing we do is we're going to write a function called artist and artist calls OpenAI
03:00.850 --> 03:03.520
dot images dot generate.
03:03.520 --> 03:09.460
So it's a very consistent style that you're used to OpenAI images generate.
03:09.460 --> 03:11.520
We pass in the name of a model.
03:11.520 --> 03:13.860
In this case, the model is Dall-E three.
03:13.890 --> 03:16.650
You could also try Dall-E two, its predecessor.
03:16.680 --> 03:18.480
The images are less awesome.
03:18.480 --> 03:19.440
It's a bit cheaper.
03:19.440 --> 03:24.450
I seem to remember it's about $0.02 rather than $0.04, so it's not massively cheaper and in my opinion
03:24.450 --> 03:26.160
well worth the extra $0.02.
03:26.190 --> 03:27.750
Stick with Dall-E three.
03:27.780 --> 03:32.400
We give it a prompt and this isn't now a clever list of dictionaries.
03:32.400 --> 03:33.360
It's just text.
03:33.360 --> 03:39.240
And in this case, the prompt I'm suggesting here is, we say, an image representing a vacation in
03:39.240 --> 03:46.980
city, showing tourist spots and everything unique about city in a vibrant pop art style.
03:46.980 --> 03:50.250
We give it a size that is the smallest size.
03:50.250 --> 03:53.070
Dall-E three will do, Dall-E two will go much smaller.
03:53.250 --> 03:58.680
Um and Dall-E three also does two larger sizes in a portrait and landscape format.
03:58.740 --> 04:00.870
Just google it if you'd like to know those dimensions.
04:00.870 --> 04:02.400
If you'd like to try those images.
04:02.430 --> 04:04.260
We just want one image back.
04:04.260 --> 04:06.210
We say we want this format.
04:06.450 --> 04:12.840
Back comes something in the, uh, this um, uh, base64 encoded format.
04:12.840 --> 04:20.040
We then decode that into bytes, and then we then create a bytes IO object on those bytes, which we
04:20.040 --> 04:26.850
can then pass in to the image dot open function, and that will return an image for us.
04:26.850 --> 04:28.320
Let's execute that.
04:28.320 --> 04:30.300
And now let's give it a try.
04:30.330 --> 04:35.040
So I'm going to say image equals artist.
04:36.870 --> 04:38.940
And what shall we say New York City.
04:42.660 --> 04:50.520
And then display image is the Jupiter way of then getting that to show.
04:50.550 --> 04:53.400
Let's run that or you're seeing one I ran already there.
04:53.400 --> 04:56.790
Sorry it's not that quick, but look how amazing that is.
04:56.940 --> 04:58.380
Uh, you're already getting.
04:58.380 --> 05:00.750
I'm spoiling you by showing you one right away.
05:00.750 --> 05:01.950
This is what it looks like.
05:01.950 --> 05:07.710
It's generating a second one above you get to see the Statue of Liberty, a few different Empire State
05:07.710 --> 05:16.200
buildings, some planes in the sky, and then a sort of image to Times Square with lots of signs and
05:16.200 --> 05:18.510
with New York, spelled out their taxi.
05:18.540 --> 05:19.140
Look at that.
05:19.170 --> 05:20.610
A yellow New York taxi.
05:20.640 --> 05:21.690
And Coca-Cola.
05:21.690 --> 05:23.040
And a hot dog.
05:23.070 --> 05:25.050
A very New York iconic thing.
05:25.080 --> 05:26.550
Fantastic.
05:26.580 --> 05:29.190
Meanwhile, it's built another image for us here.
05:29.190 --> 05:29.670
And.
05:29.670 --> 05:31.020
Wow, look at this one.
05:31.020 --> 05:32.340
It's different.
05:32.340 --> 05:33.120
It's great.
05:33.120 --> 05:35.430
It's got a big jet over here.
05:35.430 --> 05:40.620
It's got the Empire State Building, of course, multiple Empire State buildings, Statue of Liberty's.
05:40.620 --> 05:47.280
And it's got again the sort of thriving shops and taxi in the foreground like that, an iconic New York
05:47.310 --> 05:48.870
taxi and a hot dog again.
05:49.080 --> 05:54.330
Uh, so the thing to mention is that these images, they're so creative and they're so different, we've
05:54.330 --> 05:59.790
got two now that we can see the one I did a moment ago and this one here, uh, and you can see how
05:59.790 --> 06:01.470
great they look.
06:02.430 --> 06:03.060
All right.
06:03.060 --> 06:05.220
Well, I hope that you were entertained by that.
06:05.220 --> 06:10.920
And by all means, can I suggest spend some $0.04, generate a few images for yourself.
06:10.920 --> 06:12.180
They're great.
06:12.690 --> 06:14.940
All right, let's add one more function.
06:14.940 --> 06:20.450
We're going to make a function that uses OpenAI's speech to generate some audio.
06:20.450 --> 06:25.670
So we're going to use a couple of utility stuff here with a library called Pi Dub.
06:25.670 --> 06:26.630
That's very useful.
06:26.840 --> 06:29.300
We're going to write a function called talker.
06:29.300 --> 06:33.860
And talker is going to call OpenAI dot audio dot speech dot create.
06:33.860 --> 06:39.470
So if we look back up the image generation was OpenAI images generate.
06:39.470 --> 06:46.760
And for audio it's a case of uh OpenAI audio dot speech dot create.
06:46.760 --> 06:48.500
We pass in a model.
06:48.740 --> 06:56.300
Um, and this is the model we're using, TTS one TTS stands for text to speech and is, uh, the, the
06:56.330 --> 07:00.080
this kind of model that we're going for, we supply a voice.
07:00.080 --> 07:01.880
In this case we're going to try the voice.
07:01.910 --> 07:02.750
Onyx.
07:02.750 --> 07:04.880
There's something like eight different voices to try again.
07:04.910 --> 07:06.800
You can you can Google to see what they are.
07:06.800 --> 07:10.310
And we pass in the thing that this function was called.
07:10.310 --> 07:16.520
With what comes back, we again create a bytes IO object to represent those bytes.
07:16.520 --> 07:25.330
And then we use this to this audio segment, uh creating it from a file and the audio stream and get
07:25.330 --> 07:27.250
it to play that audio.
07:27.250 --> 07:31.180
So let's create that function and then let's say talker.
07:33.070 --> 07:35.470
Well hi there.
07:40.150 --> 07:41.110
Well hi there.
07:42.430 --> 07:43.240
There we go.
07:43.270 --> 07:44.500
As simple as that.
07:44.830 --> 07:47.410
Uh, let's see how another voice sounds.
07:47.410 --> 07:50.320
Let's see how alloy sounds.
07:50.470 --> 07:52.270
Let's put alloy in there.
07:55.000 --> 07:55.930
Well hi there.
07:56.860 --> 07:58.630
And that was alloy.
07:58.660 --> 08:01.180
I think we'll stick with onyx.
08:01.180 --> 08:03.070
But you can try either.
08:03.070 --> 08:09.580
And you can also put in some more there that you can experiment with and pick your favorite.
08:09.910 --> 08:10.810
All right.
08:10.810 --> 08:13.510
Well that's what we'll go with.
08:14.710 --> 08:20.920
Uh and now let's talk about the agent framework.
08:20.950 --> 08:23.650
I think we will break for the next video.
08:23.650 --> 08:26.200
And that's where we'll take on our full agent framework.
08:26.230 --> 08:27.280
See you then.