So we're going to start our exploration into the world of frontier models by playing with the famous
GPT from OpenAI, which most of you are probably quite familiar with.
I have a pro license, which means I get access to all of the models, and I imagine some of you do
as well.
We'll start with a softball question, the kind of question that they're so good at answering, which
is how do I decide if a business problem is suitable for an LLM solution?
And it's useful for us because it's the kind of question that one might ask on this course.
And what we'll get back, of course, is a very carefully structured and reasoned response with an introduction,
with summaries, the nature of the problem, the scalability needs.
No doubt there'll be stuff in here about nuance, about unstructured data, contextual understanding,
cost, maintenance, lots of great, well-reasoned points with a good summary to boot.
So this is the kind of thing that it's really, really good at.
Now I'll ask it a question, which it usually gets right, but sometimes amazingly gets wrong.
Let's see what happens this time.
How many times does the letter A appear in this sentence?
Uh, so let's see how it does.
Uh, it's got it wrong.
The letter A appears five times in your sentence.
Sometimes it gets this right, and sometimes it gets it wrong.
It's, uh, difficult to know, but.
But, uh, it might shock you that, uh, it gets that wrong.
It doesn't mean that we humans still have an advantage in some ways.
But the truth is, it's to do with the way that this information is sent into the LM.
It's to do with this, this tokenization strategy.
And we'll be talking more about that later.
But it is interesting that it gets it wrong.
Uh, I'm going to ask it one more question, which is a tricky question.
I'm going to ask it, uh, Choose the word that best completes the analogy.
Feather is to bird as scale is to.
And then there's a few different options there.
And the best answer is in fact, reptile fish is a bit of a trick answer because fish do have scales,
but it's not as distinguishing feature as it is for reptiles.
This question I got from a website called vellum, which is a very a company that does a lot of this
kind of analysis that we will talk about later.
All right.
Let's switch to a different model.
Let's switch to zero one preview.
This is the model that was originally codenamed strawberry and is the strongest of OpenAI's models,
only available to Pro subscribers, but it will ultimately be available to everyone, and it gives you
a sense of what's to come.
It uses a sort of chain of reasoning approach to think through questions.
Let's ask it the same question how many times does the letter A appear in this sentence.
See if it can do better.
It's thinking.
You can see how it takes longer for sure.
Counting letter frequencies.
That sounds promising.
Taking a closer look.
Good to know.
And it gets the right the answer.
Correct.
The letter A appears four times in the sentence.
Once in many, uh, once within the quotes and twice in the word appear.
So it is correct.
Very good.
Uh, and then let's also ask strawberry.
Oh, one preview this, uh, this puzzle and let's see how it can approach this.
It's considering choosing the right analogy that's also promising cultivating.
And it gives the correct answer reptile.
So this gives you a sense of the different models, some of the different strengths between them from
GPT four zero and zero one preview.