I'm excited to introduce you to your first exercise, and I'm looking forward to seeing what you make
of this.
As a reminder, you should have gone to your Anaconda prompt if you're on a PC or your terminal window.
If you're on a mac, you should have gone to the project root directory.
LM engineering activated Anaconda by doing conda activate LMS or the virtualenv equivalent if you're
using that, and then typed JupyterLab to bring up your Jupyter lab.
And in the file browser on the left, you should see the weeks like this.
Or you might be already in the week one folder, in which case it will look something like this.
And I now want you to go to the day two exercise notebook, which will come up like this.
And here is the plan.
What we're going to do is we're going to see how to call a llama from code.
So we're going to use Python code to call the llama model that's running on your computer.
And what we're then going to do and once our first set that up.
So we'll get that to work and you'll be able to see the results.
And the exercise for you will be to then update the summarization project that we completed yesterday
and use Olama use your local model instead of the call to OpenAI.
And if you didn't sign up for the OpenAI API, then this is your chance to do it for the first time.
So, uh, first of all, I explain here that we will be, uh, using a llama.
The benefits of using a llama, of course, is that there's no API charges.
It's open source.
It's running on your box.
It's free.
Another benefit is that the data will never leave your box.
So if you're ever working on something, whether it's confidential data that absolutely must not go
to the cloud, then of course this gives you techniques to be working locally without data leaving the
internet.
Uh, the disadvantage is that, uh, obviously the frontier models, they are many, many times larger
and more powerful than the open source models.
And so we should expect that the results won't be as strong.
But, you know, that's what you pay for when you pay your, uh, your, your, uh, fraction of a cent
each call.
First of all, a recap that you hopefully already installed Olama by going to Olama.
Com and you remember, it's just a matter of pressing that download button and you're off to the races.
If you've done that, then if you visit this link here localhost 11434, then you should see this Olama
is running message which tells you that it's running.
If that doesn't show, then bring up a terminal or a PowerShell and just enter Olama serve and it should
then be running.
And if you go there, you should see again a llama is running.
So with that, if that doesn't happen then then try and do a little bit of debugging and research and
then contact me and I'll help all right.
So I'm going to do a few imports.
Now I'm going to set some constants.
This here is a URL on my local box on this port which is you see the port that runs on slash API slash
chat.
I'm going to have also a constant called model which will be llama 3.2.
Now this here this messages, uh, hopefully you will recognize this construct because this is the same
construct as the messages.
Let me lay it out a bit differently for you.
This is the same as the messages that we talked about before.
Uh, that we use with OpenAI.
Messages is a list of dictionaries, the dictionaries.
Each dictionary has a key of role, and the value is either user or system and a key of content, and
the value is the user message or the system message.
So this very simply is saying I want to have a user prompt that says, describe some of the business
applications of generative AI.
Let's run that.
I'm now going to put that into a JSON object called a payload, which specifies the model, the messages,
and I don't want it to stream results.
I just want to get back the results.
And I'm then going to use the Python package requests to post that request to this URL pass in the JSON.
And then what I get back, I'm going to take the JSON look in the message content fields, and we'll
see what happens when we make that call.
So right now of course it's making web requests locally from my box to my box.
And it's connecting to the llama 3.2 model that's being served by llama.
And this is the result.
And I will tell you that the answers that it gives are really good.
So that since we are trying to learn about commercial applications, it would do you no harm to read
through some of its responses and see if there's anything that interests you.
Now, I wanted to show you that because I wanted to explain exactly what's going on behind the covers
and that we're making these basically these URL, these web requests to our local box.
But in fact, the friendly people at Allama have built a Python package, which makes this even simpler.
So you can just do this in one line.
So I could have started with this, but I wanted to show you the steps to making the web request so
you have a good intuition for what's actually happening.
But there is this nice package, Allama, that you can just import, and then you can say Allama dot
chat, pass in the model, pass in the messages, and then just take back the response content.
And if I run that, we should hopefully see that we will get basically the same thing.
And here we go.
There it is.
Uh, and I imagine yeah, I can Uh, already see that there are differences between them.
Of course, it's somewhat unique each time.
Uh, this one looks like a longer response.
Okay, that's the end of my teeing up.
Now it's over to you.
So you'll remember in day one, we built this solution that where we built something that would summarize
a website, and we made a call to OpenAI to achieve that.
Here, in fact is our call to OpenAI right here.
The challenge for you is to keep going with this day two exercise lab and add in that same summarizer
code so that you can build a website, summarizer, that uses your local Ulama open source model, llama
3.2 or a different model if you wish to do your summarization.
That's the exercise.
The solution is in the solutions folder should you need it.
But I think you've got this one and I will see you for the next video when you have that done.