And welcome back to Jupyter Lab, one of my very favorite places to be.
When Jupyter Lab sprung up on your screen, you will probably arrive at this place the root directory
with all eight weeks of our work.
You may also already be within week one, which you get to of course by double clicking here.
We are in week one, and I'd like to ask you to head over to day five, which is where we will be spending
the next few moments.
So as I say, the business challenge that I have in store for you is to build on what we already built
in day one, to create a brochure for a company by scraping the web, finding out more about the company,
and using that for our brochure.
Um, and if you do encounter any problems with this, please do reach out to me.
But if possible, you should be executing this while I speak.
Or maybe afterwards come back in and go through it.
And the trick is to come in and add in some print statements and really convince yourself that you're
confident with what's going on at every point.
I'm going to start with some imports.
Remember, you press shift and enter to run that, um, if you have any problems with these imports?
The most likely explanation is that somehow you're not running in an activated environment.
JupyterLab was brought up in an activated environment.
Check to see whether Lmms is in the prompt, in your in your terminal window or in your Anaconda prompt.
And if not, then start that part again and look at the readme if you need help on that.
Uh, it's possible you might also, in some situations, need to restart the Python process which sits
behind this, uh, which is known as the kernel.
And to do that you go to the kernel menu and you say restart kernel and clear outputs of all cells.
And you simply start this notebook again and here we go again.
We'll run the import a second time.
All right.
Now we're going to initialize and set things up.
We're going to load in our dot env file.
And we're just going to check that the key looks good.
Uh and it does for me.
And hopefully it's looked good for you as well.
Otherwise head over to the troubleshooting notebook to figure out what's going on.
And we're setting our model to be GPT four mini, the cheap version of GPT four, which is still going
to be phenomenally good.
Okay, so this should look familiar to you.
In the next cell, we're looking at the class website that we created in week one.
And maybe now take a little bit of a closer look at it.
It's your second time of playing with this.
You'll remember that this is a class that we create by passing in a URL.
It uses the requests package to retrieve that URL.
It then collects the content, and it uses Beautifulsoup, that wonderful parsing package to parse it.
There's something different here.
We not only parse in the title and the contents and strip out some of the junk that we don't need,
but we also gather any links that are referred to on this page, and we collect those links in something
called self dot links.
So we're going to store all of our links in there.
And this little line here hopefully because we just went through an AST zero one preview for for some,
uh, no.
We asked, uh, sorry, uh, GPT four with canvas to explain some of this.
So maybe this is now very familiar to you.
Uh, and then we're going to have a method, getcontents, which is going to describe what this web
page does.
So let's run that.
So let's now again do, uh, what we did before editor is website.
Edward Dunham.
Com my wonderful website.
That's very simplistic, but it's a good test for us now.
And let's print print ad dot get contents.
Remember last time we printed just the title and the body?
Let's see what we get.
So now we do that.
What we get is again the title and we get the body.
But hopefully we're going to get something else as well.
Uh, we're also Uh, can I get.
Well, we get the title and the contents all in one long string as part of Getcontents.
But the other thing that I want to look at then is I want to look at what is editor dot links.
Let's see what this has.
And now you'll see that in this links variable we now have all of the links that you'll find on my web
page.
Uh, it might be easier if I don't have the print.
If I just do it this way.
We'll get them listed out there.
That's easier, isn't it?
So there they are.
Here are all of the links that you'll find on my web page.
And they're now being stored in this variable links.
Hopefully that's clear to you.
All right.
Now if we're building a company brochure and we want to provide a web page and we want it to use that
to gather more information, we want it to follow some of these links to figure out how it can.
It can collect more information from them.
But not all of these links are going to be relevant.
Some of these links are going to be red herrings, like this thing here, which is probably from from
one of the, uh, The analytics tags that's included.
Or there are some other things here like that's going to be irrelevant.
Now it's going to be really hard for us to write code to figure out whether or not a link is relevant
for the purposes of generating a sales brochure.
That's actually really hard.
The other thing we might want to do is take a link to something like slash about and replace it with
a full URL as well.
And maybe that's easier to do with code, but it's still not a not a simple task by any means.
The combined code to try and figure out which of these links are relevant and what's the full, full
URL, including the the host.
That would be a lot of coding, but of course, it turns out that's the kind of thing that GPT four
mini would be very good at doing for us.
We could just ask it to do that as a task.
It's an example of taking a sort of nuanced, complicated task, and rather than trying to hand code
it, we can just ship it off to a frontier model and say, do this for us, and that's what we're going
to do, and we're going to do that in the next video.