I'm now going to talk for a bit about models.
A term you often hear is the term frontier models, which refers to the llms that are pioneering what
is possible today, the largest possible models.
And often when people say frontier models, they're referring to the closed source models, the paid
models like GPT and Claude.
Actually, sometimes people also say frontier models when they're referring to the biggest, strongest
open source models as well.
So depending on the context, it could mean either thing.
But let's first talk about the closed source frontier models.
And these are also sometimes called the super scalers.
They are the largest, highest scale of the models.
So first of all, the one that needs no introduction I'm sure is GPT from OpenAI.
When ChatGPT came out in late 2022, it caught us all off guard with its power.
And I'm sure you're familiar with it.
Claude from anthropic.
Anthropic is the competitor to OpenAI.
That's that's very well known.
And Claude is the one that's that's usually favored by data scientists.
but Claude and GPT are often considered neck and neck right now in the leaderboards.
Claude has the slight edge.
Gemini is Google's entrant.
You probably remember from when we looked at llama that they also Google also has Gemma.
The the open source variant as well.
And command R is one that you may or may not have come across from cohere.
Canadian AI company.
And then perplexity is a search engine which actually can use one of the other models, but also has
a model itself.
So these are some of the big frontier models.
And let's talk about some of the open source models.
So llama uh, after which llama is named llama from meta is of course the most famous of the open source
models because meta paved the way in the field of open source LMS by open sourcing the original llama
one.
Uh, there's one called Mistral from, uh, French company Mistral, which is a model, which is what
they call a mixture of experts.
It contains multiple smaller models.
Quen is a model that I mentioned back when we were playing with Obama.
It is a powerhouse model.
It's really super impressive from Alibaba Cloud, and we will use Quan from time to time because it's
it's as I say, it's very powerful for its size.
Gemma I mentioned is Google's smaller model and Fi is Microsoft's smaller open source model.
So this is confusing and it's super important.
And it's something which some of you may already know, but but it might be something that's been nagging
as well at a few of you.
And I want to be really clear on this.
There are different ways that you can use models that are in completely different approaches, and it's
important to understand the differences between them and when you come across them, being used in those
different ways.
Have in your mind what's going on here.
So first of all, there are chat interfaces to using models, obviously like ChatGPT, which is a web
front end where you are chatting and you are calling something that's running in the cloud, and that
the whole process of interacting with the LLM LM is being handled by OpenAI on their cloud.
In the case of ChatGPT, there's also, of course, a Cloud and Gemini Advance and others.
There are cloud APIs.
And this is where again, you are calling something that's running on the cloud, but you're doing it
with code, not through a user interface.
And what we did in the summarization Jupyter notebook, Jupyter Lab was calling OpenAI's API.
We were connecting to OpenAI, calling their API, and with the chat interfaces you typically it's it's
either free for a free tier, or you're paying a monthly subscription fee to use the user interface
chat almost as much as you want.
There are some limits there.
It's different with the APIs.
With the APIs, there's no subscription, there's no monthly charge, but rather you pay for every API
request you make.
If it's a paid API.
There are also open source free APIs too, so you can call the APIs directly.
There are also libraries like Lang Chain, which give you a kind of abstraction layer, and you can
use Lang chain.
And then within it you can call the different APIs.
And it presents you with one API that is unified across them.
And so there are some of these frameworks like Lang chain.
And if you see someone using Lang chain, it's really just using the lm API under the covers.
It's just giving you a nicer user, nicer API interface, more consistent, uh, on top of it.
And then there's another type of API which is a bit of a different take, which is using something called
a managed AI cloud service, which is where you are connecting with a provider like Amazon, Google
or Microsoft Azure.
And they are running the models on their cloud, and they're presenting you with a common interface
so that you can run behind the scenes.
It could be open source, it could be closed source.
And you'll hear of Amazon Bedrock.
That's Amazon's offering.
Google vertex AI is Google's and Azure ML.
It goes by some other names too is Microsoft's offering.
So these are the managed AI cloud services.
But what all of these have in common is that you are writing code Locally.
That then makes a call to an LM running in the cloud, and that is the cloud API.
And then there's a third approach, and that is when you get the code and the weights for an LM yourself
and you run it yourself, uh, on your box or potentially by remoting into a remote box.
And here again, there's two different ways that we will be doing it on this course.
And it's important to understand the differences between them.
One of them is using hugging face, where we will be able to get access to like the Python code and
the PyTorch code, which has that model in it.
And we'll be able to then work in a fairly granular way with that model, will be able to use it to
do things like tokenized text, and then call the model with the tokens, and you'll be actually operating
the model.
And we'll typically do that using something like Google Colab, where we can be running it on a very
high powered box in the cloud, because typically one's local box isn't powerful enough to run that
kind of, uh, that level of processing.
And as an alternative to that, people have taken this code and they've optimized it into high performance
C plus plus code and compiled it so that you can run it locally on your box.
And that is what Olama is.
It uses something called llama CPW behind the scenes as the C plus plus code.
Now that means that you can run it locally.
You're running the models in inference and execution mode on your box, but you don't have as much ability
to control what's going on because it's just fully compiled code.
So that gives you hopefully some insight.
I'm glossing over some of the details, but hopefully shows you the landscape of the three different
ways that you can work with models, and then some of the sort of sub sub techniques under that.
With all of that, what are we going to do now?
We're going to do an exercise, and it's going to be a useful exercise, and one that you'll be able
to continue using throughout the course, because it's going to involve olama.
And without further ado, I'm going to flip over to JupyterLab to explain the exercise.