Well, I'm delighted to welcome you to day three of our eight week journey together.
And today we're going to be looking at Frontier Models.
The idea that the goal of today is to get deep into these different models so that you can get a true
intuition for where are they strong, where are they weak, what are the differences between them?
And so that's what I want you to keep in mind throughout today's material, learning about the differences
between them and thinking about how you would apply them commercially to your business or to future
projects, and understanding when you would pick which model.
Let's get to it.
So we're going to be talking about six different models today from six different companies, starting
of course, with OpenAI's models.
OpenAI needs no introduction really.
GPT is the most famous model.
And we'll also, of course look at 0101 preview.
The the newest of their models and ChatGPT is their user interface, the screens where you can interact
with it.
We'll also look at the models from anthropic.
Anthropic is OpenAI's top competitor, based in San Francisco as well, and founded by some people that
left OpenAI, and their model is called Claude and Claude.
In fact, you may know, comes in sort of three powers.
The smallest one is called haiku.
Claude haiku, and then the sonnet.
And then there's opus.
But actually, because sonnet has had much more recent versions, the latest version of sonnet is stronger
than the bigger, more expensive opus, as we'll see.
That will make more sense later.
But Claude Claude 3.5 sonnet is the strongest of Claude's models.
Google has Google Gemini, probably latest to the party, and most of us know Gemini most well because
nowadays when we do a Google search, very often we see Gemini's responses.
Gemini is, of course, the next generation of what was originally called Bard from Google.
Cohere is one that you may have heard less about.
It's a Canadian AI company, and their model is most well known for being using using a technique called
Rag to make sure that it has expertise.
So we will see that.
And then we know the llama model from meta.
We've used it ourselves through llama.
This is an open source model, and you may not know that the meta actually also has a website, meta
AI, that lets you interact with the llama model.
And we will have a look at that.
And then perplexity is a bit different, because perplexity is actually a search engine powered by AI,
powered by Llms, and it can use some of the other models that we'll talk about.
But they do also have their own model too, so it's a slightly different beast, but we'll be looking
at perplexity as well.
So overall, these llms are astonishing in what they are capable of.
They are really very effective indeed at taking a detailed question, a nuanced question, and providing
a structured summary that appears well researched.
It often has a sort of introduction and a summary, and this is one of the ways that I use it all the
time.
and I find that across the board, these llms are shocking in how good they are at this.
It's something that a couple of years ago, none of us would have imagined that we could get this far
this quickly.
There are also really good, and I imagine that many of you do this a lot yourselves, and I do it if
you put in a few bullets, just a few notes on something and say, hey, can you turn this into an email
or can you turn this into a slide?
They are really good at fleshing it out and building, say, a blog post, and they're very good at
iterating.
So they'll do something and are like some of it I won't like others.
And you can give feedback and keep going backwards and forwards.
And it's a really effective way of working.
It's the kind of copilot construct that is so, so, so effective.
And then coding, of course.
And perhaps this for many of us is is the thing that is most staggering is how very good the llms are
at writing code and debugging problems and solving them.
It's something which is really remarkable.
I've had experiences myself when I've been working on something that's very complex, and it's something
that I believe I have deep subject matter expertise in, and I've got a fairly intricate error, and
I put the details and the stack trace in into Claude, say, and I get back not only a very precise
explanation of what's going wrong, but also the code that will fix it appearing as an artifact on the
right in Claude.
And it's it's it's amazing.
It's absolutely amazing.
And in fact, these are often things which I if I try and paste them, if I look for it in Stack Overflow,
there's no answer there.
Somehow it's it's able to look beyond just, just a regurgitating Stack Overflow answers.
And it seems to have real insight into what's going on.
And I suppose that's why it's not surprising, really, that Stack Overflow has seen a big falloff in
its traffic.
You can see that something started to happen in a big way after Q4 2022, which is when ChatGPT was
released.
So, you know, it's obviously changed the paradigm of how uh, how we how technology people work with
with researching our problems.
It's very effective.
And I encourage you, if you get stuck with some of the things we work on to give Claude or OpenAI GPT
a shot.
So what about where are they weak?
What are the things that they struggle with?
Where does humanity still have a chance in all of this?
Well, so first of all, they tend to not be as strong with specialized subject matter if it's something
that requires detailed knowledge.
Most llms are not yet at PhD level.
Now, I had to put the word most in there because literally just just just a few weeks ago for me in
October, uh, Claude, the newest version of Claude came out, uh, the latest Claude 3.5 sonnet,
uh, and it has surpassed PhD level in maths, physics, chemistry.
Uh, and so this is something where very quickly we're seeing these models achieving PhD level.
So far just Claude.
But the others I'm sure are not far behind, but those are in those specific sciences and in a particular
domain, like a business domain, they still won't have the specialist knowledge of an expert in that
space.
And secondly, recent events.
So the models have been trained up until a knowledge cutoff, which is for GPT.
October of last year.
And so they won't be able to answer questions on information that has come since then.
And then they have some strange blind spots.
There are some questions which they will just get wrong.
And when they get them wrong, one of the things that's quite concerning is that they do tend to be
confident in their responses.
They often don't volunteer the fact that they're uncertain.
They just state an answer with the same level of conviction as with something where they do get the
answer right.
And that that is something which of course causes concern when you see models hallucinate or come up
with with new information which it doesn't know and do so with confidence.
And we'll see some examples of that and talk about what are the reasons behind those blind spots.