From the uDemy course on LLM engineering.
https://www.udemy.com/course/llm-engineering-master-ai-and-large-language-models
You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
441 lines
18 KiB
441 lines
18 KiB
{ |
|
"cells": [ |
|
{ |
|
"cell_type": "markdown", |
|
"id": "d15d8294-3328-4e07-ad16-8a03e9bbfdb9", |
|
"metadata": {}, |
|
"source": [ |
|
"# Instant Gratification!\n", |
|
"\n", |
|
"Let's build a useful LLM solution - in a matter of minutes.\n", |
|
"\n", |
|
"Our goal is to code a new kind of Web Browser. Give it a URL, and it will respond with a summary. The Reader's Digest of the internet!!\n", |
|
"\n", |
|
"Before starting, be sure to have followed the instructions in the \"README\" file, including creating your API key with OpenAI and adding it to the `.env` file.\n", |
|
"\n", |
|
"## If you're new to Jupyer Lab\n", |
|
"\n", |
|
"Welcome to the wonderful world of Data Science experimentation! Once you've used Jupyter Lab, you'll wonder how you ever lived without it. Simply click in each \"cell\" with code in it, like the cell immediately below this text, and hit Shift+Return to execute that cell. As you wish, you can add a cell with the + button in the toolbar, and print values of variables, or try out variations.\n", |
|
"\n", |
|
"If you need to start again, go to Kernel menu >> Restart kernel." |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 1, |
|
"id": "4e2a9393-7767-488e-a8bf-27c12dca35bd", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# imports\n", |
|
"\n", |
|
"import os\n", |
|
"import requests\n", |
|
"from dotenv import load_dotenv\n", |
|
"from bs4 import BeautifulSoup\n", |
|
"from IPython.display import Markdown, display\n", |
|
"from openai import OpenAI" |
|
] |
|
}, |
|
{ |
|
"cell_type": "markdown", |
|
"id": "6900b2a8-6384-4316-8aaa-5e519fca4254", |
|
"metadata": {}, |
|
"source": [ |
|
"# Connecting to OpenAI\n", |
|
"\n", |
|
"The next cell is where we load in the environment variables in your `.env` file and connect to OpenAI.\n", |
|
"\n", |
|
"## Troubleshooting if you have problems:\n", |
|
"\n", |
|
"1. OpenAI takes a few minutes to register after you set up an account. If you receive an error about being over quota, try waiting a few minutes and try again.\n", |
|
"2. Also, double check you have the right kind of API token with the right permissions. You should find it on [this webpage](https://platform.openai.com/api-keys) and it should show with Permissions of \"All\". If not, try creating another key by:\n", |
|
"- Pressing \"Create new secret key\" on the top right\n", |
|
"- Select **Owned by:** you, **Project:** Default project, **Permissions:** All\n", |
|
"- Click Create secret key, and use that new key in the code and the `.env` file (it might take a few minutes to activate)\n", |
|
"- Do a Kernel >> Restart kernel, and execute the cells in this Jupyter lab starting at the top\n", |
|
"4. As a fallback, replace the line `openai = OpenAI()` with `openai = OpenAI(api_key=\"your-key-here\")` - while it's not recommended to hard code tokens in Jupyter lab, because then you can't share your lab with others, it's a workaround for now\n", |
|
"5. Contact me! Message me or email ed@edwarddonner.com and we will get this to work.\n", |
|
"\n", |
|
"Any concerns about API costs? See my notes in the README - costs should be minimal, and you can control it at every point." |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 2, |
|
"id": "7b87cadb-d513-4303-baee-a37b6f938e4d", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# Load environment variables in a file called .env\n", |
|
"\n", |
|
"load_dotenv()\n", |
|
"os.environ['OPENAI_API_KEY'] = os.getenv('OPENAI_API_KEY', 'your-key-if-not-using-env')\n", |
|
"openai = OpenAI()" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 3, |
|
"id": "c5e793b2-6775-426a-a139-4848291d0463", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# A class to represent a Webpage\n", |
|
"\n", |
|
"class Website:\n", |
|
" url: str\n", |
|
" title: str\n", |
|
" text: str\n", |
|
"\n", |
|
" def __init__(self, url):\n", |
|
" self.url = url\n", |
|
" response = requests.get(url)\n", |
|
" soup = BeautifulSoup(response.content, 'html.parser')\n", |
|
" self.title = soup.title.string if soup.title else \"No title found\"\n", |
|
" for irrelevant in soup.body([\"script\", \"style\", \"img\", \"input\"]):\n", |
|
" irrelevant.decompose()\n", |
|
" self.text = soup.body.get_text(separator=\"\\n\", strip=True)" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 4, |
|
"id": "2ef960cf-6dc2-4cda-afb3-b38be12f4c97", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"name": "stdout", |
|
"output_type": "stream", |
|
"text": [ |
|
"Home - Edward Donner\n", |
|
"Home\n", |
|
"Outsmart\n", |
|
"An arena that pits LLMs against each other in a battle of diplomacy and deviousness\n", |
|
"About\n", |
|
"Posts\n", |
|
"Well, hi there.\n", |
|
"I’m Ed. I like writing code and experimenting with LLMs, and hopefully you’re here because you do too. I also enjoy DJing (but I’m badly out of practice), amateur electronic music production (\n", |
|
"very\n", |
|
"amateur) and losing myself in\n", |
|
"Hacker News\n", |
|
", nodding my head sagely to things I only half understand.\n", |
|
"I’m the co-founder and CTO of\n", |
|
"Nebula.io\n", |
|
". We’re applying AI to a field where it can make a massive, positive impact: helping people discover their potential and pursue their reason for being. Recruiters use our product today to source, understand, engage and manage talent. I’m previously the founder and CEO of AI startup untapt,\n", |
|
"acquired in 2021\n", |
|
".\n", |
|
"We work with groundbreaking, proprietary LLMs verticalized for talent, we’ve\n", |
|
"patented\n", |
|
"our matching model, and our award-winning platform has happy customers and tons of press coverage.\n", |
|
"Connect\n", |
|
"with me for more!\n", |
|
"August 6, 2024\n", |
|
"Outsmart LLM Arena – a battle of diplomacy and deviousness\n", |
|
"June 26, 2024\n", |
|
"Choosing the Right LLM: Toolkit and Resources\n", |
|
"February 7, 2024\n", |
|
"Fine-tuning an LLM on your texts: a simulation of you\n", |
|
"January 31, 2024\n", |
|
"Fine-tuning an LLM on your texts: part 4 – QLoRA\n", |
|
"Navigation\n", |
|
"Home\n", |
|
"Outsmart\n", |
|
"An arena that pits LLMs against each other in a battle of diplomacy and deviousness\n", |
|
"About\n", |
|
"Posts\n", |
|
"Get in touch\n", |
|
"ed [at] edwarddonner [dot] com\n", |
|
"www.edwarddonner.com\n", |
|
"Follow me\n", |
|
"LinkedIn\n", |
|
"Twitter\n", |
|
"Facebook\n", |
|
"Subscribe to newsletter\n", |
|
"Type your email…\n", |
|
"Subscribe\n" |
|
] |
|
} |
|
], |
|
"source": [ |
|
"# Let's try one out\n", |
|
"\n", |
|
"ed = Website(\"https://edwarddonner.com\")\n", |
|
"print(ed.title)\n", |
|
"print(ed.text)" |
|
] |
|
}, |
|
{ |
|
"cell_type": "markdown", |
|
"id": "6a478a0c-2c53-48ff-869c-4d08199931e1", |
|
"metadata": {}, |
|
"source": [ |
|
"## Types of prompts\n", |
|
"\n", |
|
"You may know this already - but if not, you will get very familiar with it!\n", |
|
"\n", |
|
"Models like GPT4o have been trained to receive instructions in a particular way.\n", |
|
"\n", |
|
"They expect to receive:\n", |
|
"\n", |
|
"**A system prompt** that tells them what task they are performing and what tone they should use\n", |
|
"\n", |
|
"**A user prompt** -- the conversation starter that they should reply to" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 5, |
|
"id": "abdb8417-c5dc-44bc-9bee-2e059d162699", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"system_prompt = \"You are an assistant that analyzes the contents of a website \\\n", |
|
"and provides a short summary, ignoring text that might be navigation related. \\\n", |
|
"Respond in markdown.\"" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 6, |
|
"id": "f0275b1b-7cfe-4f9d-abfa-7650d378da0c", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"def user_prompt_for(website):\n", |
|
" user_prompt = f\"You are looking at a website titled {website.title}\"\n", |
|
" user_prompt += \"The contents of this website is as follows; \\\n", |
|
"please provide a short summary of this website in markdown. \\\n", |
|
"If it includes news or announcements, then summarize these too.\\n\\n\"\n", |
|
" user_prompt += website.text\n", |
|
" return user_prompt" |
|
] |
|
}, |
|
{ |
|
"cell_type": "markdown", |
|
"id": "ea211b5f-28e1-4a86-8e52-c0b7677cadcc", |
|
"metadata": {}, |
|
"source": [ |
|
"## Messages\n", |
|
"\n", |
|
"The API from OpenAI expects to receive messages in a particular structure.\n", |
|
"Many of the other APIs share this structure:\n", |
|
"\n", |
|
"```\n", |
|
"[\n", |
|
" {\"role\": \"system\", \"content\": \"system message goes here\"},\n", |
|
" {\"role\": \"user\", \"content\": \"user message goes here\"}\n", |
|
"]" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 7, |
|
"id": "0134dfa4-8299-48b5-b444-f2a8c3403c88", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"def messages_for(website):\n", |
|
" return [\n", |
|
" {\"role\": \"system\", \"content\": system_prompt},\n", |
|
" {\"role\": \"user\", \"content\": user_prompt_for(website)}\n", |
|
" ]" |
|
] |
|
}, |
|
{ |
|
"cell_type": "markdown", |
|
"id": "16f49d46-bf55-4c3e-928f-68fc0bf715b0", |
|
"metadata": {}, |
|
"source": [ |
|
"## Time to bring it together - the API for OpenAI is very simple!" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 10, |
|
"id": "905b9919-aba7-45b5-ae65-81b3d1d78e34", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"def summarize(url):\n", |
|
" website = Website(url)\n", |
|
" response = openai.chat.completions.create(\n", |
|
" model = \"gpt-4o-mini\",\n", |
|
" messages = messages_for(website)\n", |
|
" )\n", |
|
" return response.choices[0].message.content" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 12, |
|
"id": "05e38d41-dfa4-4b20-9c96-c46ea75d9fb5", |
|
"metadata": { |
|
"scrolled": true |
|
}, |
|
"outputs": [ |
|
{ |
|
"data": { |
|
"text/plain": [ |
|
"\"# Summary of Edward Donner's Website\\n\\nEdward Donner's website serves as a platform for sharing insights on artificial intelligence, specifically focusing on large language models (LLMs). He expresses a passion for coding, DJing, and electronic music production. Edward is the co-founder and CTO of Nebula.io, which utilizes AI to enhance talent discovery and management processes. He previously founded the AI startup untapt, which was acquired in 2021.\\n\\n## Recent Posts\\n- **Outsmart LLM Arena** (August 6, 2024): An announcement about an interactive platform where LLMs compete in strategy and negotiation.\\n- **Choosing the Right LLM: Toolkit and Resources** (June 26, 2024): A guide for selecting suitable LLMs for various applications.\\n- **Fine-tuning an LLM on Your Texts: A Simulation of You** (February 7, 2024): Discussion on personalizing LLMs for individual use.\\n- **Fine-tuning an LLM on Your Texts: Part 4 – QLoRA** (January 31, 2024): Detailed exploration of a specific method for fine-tuning LLMs.\\n\\nEdward invites readers to connect with him through various platforms for further discussion and engagement.\"" |
|
] |
|
}, |
|
"execution_count": 12, |
|
"metadata": {}, |
|
"output_type": "execute_result" |
|
} |
|
], |
|
"source": [ |
|
"summarize(\"https://edwarddonner.com\")" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 13, |
|
"id": "3d926d59-450e-4609-92ba-2d6f244f1342", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"def display_summary(url):\n", |
|
" summary = summarize(url)\n", |
|
" display(Markdown(summary))" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 14, |
|
"id": "3018853a-445f-41ff-9560-d925d1774b2f", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"data": { |
|
"text/markdown": [ |
|
"# Summary of Edward Donner's Website\n", |
|
"\n", |
|
"The website serves as a personal and professional platform for Ed Donner, a programmer, LLM (Large Language Model) enthusiast, and the co-founder and CTO of Nebula.io. The site includes insights into Ed's interests in coding, electronic music production, and his background in AI startups.\n", |
|
"\n", |
|
"## Key Highlights:\n", |
|
"\n", |
|
"- **Outsmart**: An initiative that features LLMs competing in scenarios that require diplomacy and cunning.\n", |
|
"- **About Ed**: Ed shares his professional journey, including the founding of Nebula.io and his previous startup, untapt, which was acquired in 2021. He emphasizes using AI to help people fulfill their potential.\n", |
|
"- **Posts and Resources**: The site includes several posts discussing topics such as selecting the right LLM, fine-tuning LLMs, and practical tools for working with them.\n", |
|
"\n", |
|
"## Recent Announcements:\n", |
|
"1. **August 6, 2024**: Announcement of the \"Outsmart LLM Arena\".\n", |
|
"2. **June 26, 2024**: Introduction of resources for selecting the right LLM.\n", |
|
"3. **February 7, 2024**: Post on fine-tuning LLMs to simulate personal text styles.\n", |
|
"4. **January 31, 2024**: Continuation of the fine-tuning series focusing on QLoRA.\n", |
|
"\n", |
|
"Overall, the website is a blend of personal exploration in AI, professional endeavors, and educational content centered around large language models." |
|
], |
|
"text/plain": [ |
|
"<IPython.core.display.Markdown object>" |
|
] |
|
}, |
|
"metadata": {}, |
|
"output_type": "display_data" |
|
} |
|
], |
|
"source": [ |
|
"display_summary(\"https://edwarddonner.com\")" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 15, |
|
"id": "45d83403-a24c-44b5-84ac-961449b4008f", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"data": { |
|
"text/markdown": [ |
|
"# Website Summary: CNN\n", |
|
"\n", |
|
"CNN is a leading news platform providing breaking news, latest headlines, and video content across a range of topics including politics, business, health, and global events. The site covers significant global issues such as the ongoing Ukraine-Russia war and the Israel-Hamas conflict, with live updates and analyses. \n", |
|
"\n", |
|
"## Recent News Highlights:\n", |
|
"- **Israel-Hamas Conflict**: The Israel Defense Forces (IDF) chief has indicated a potential ground incursion in Lebanon amid rising tensions with Hezbollah, with calls from the US and allies for a 21-day ceasefire to avert regional war.\n", |
|
"- **Geopolitical Developments**: Israel intercepted a Hezbollah missile aimed at Tel Aviv, marking a significant escalation in hostilities.\n", |
|
"- **International Concerns**: There are fears regarding potential Russian attacks on Ukrainian nuclear facilities.\n", |
|
"- **Cultural Updates**: A panda pair was welcomed in Hong Kong as China commemorates 75 years of Communist rule.\n", |
|
"- **Scientific Discoveries**: Astronomers are reassessing their understanding of the universe following the discovery of massive black hole jets.\n", |
|
"\n", |
|
"CNN also covers various other topics including entertainment, health, science, and climate-related news, showcasing a comprehensive view of both domestic and international affairs." |
|
], |
|
"text/plain": [ |
|
"<IPython.core.display.Markdown object>" |
|
] |
|
}, |
|
"metadata": {}, |
|
"output_type": "display_data" |
|
} |
|
], |
|
"source": [ |
|
"display_summary(\"https://cnn.com\")" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 16, |
|
"id": "75e9fd40-b354-4341-991e-863ef2e59db7", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"data": { |
|
"text/markdown": [ |
|
"# Anthropic Website Summary\n", |
|
"\n", |
|
"Anthropic is an AI safety and research company based in San Francisco, dedicated to developing AI models that prioritize safety and reliability. Their key AI model currently available is **Claude 3.5 Sonnet**, which is highlighted as their most intelligent model yet, released on **June 21, 2024**. The company also offers an API that allows businesses to utilize Claude for improved efficiency and new revenue streams.\n", |
|
"\n", |
|
"## Notable Announcements:\n", |
|
"- **Claude 3.5 Sonnet Released**: June 21, 2024\n", |
|
"- **Research on Harmlessness from AI Feedback**: December 15, 2022\n", |
|
"- **Core Views on AI Safety**: March 8, 2023\n", |
|
"\n", |
|
"Anthropic emphasizes an interdisciplinary approach, bringing together expertise from machine learning, physics, policy, and product development to further their mission in safe AI research. They actively seek to expand their team through open positions." |
|
], |
|
"text/plain": [ |
|
"<IPython.core.display.Markdown object>" |
|
] |
|
}, |
|
"metadata": {}, |
|
"output_type": "display_data" |
|
} |
|
], |
|
"source": [ |
|
"display_summary(\"https://anthropic.com\")" |
|
] |
|
}, |
|
{ |
|
"cell_type": "markdown", |
|
"id": "36ed9f14-b349-40e9-a42c-b367e77f8bda", |
|
"metadata": {}, |
|
"source": [ |
|
"## An extra exercise for those who enjoy web scraping\n", |
|
"\n", |
|
"You may notice that if you try `display_summary(\"https://openai.com\")` - it doesn't work! That's because OpenAI has a fancy website that uses Javascript. There are many ways around this that some of you might be familiar with. For example, Selenium is a hugely popular framework that runs a browser behind the scenes, renders the page, and allows you to query it. If you have experience with Selenium, Playwright or similar, then feel free to improve the Website class to use them. Please push your code afterwards so I can share it with other students!" |
|
] |
|
} |
|
], |
|
"metadata": { |
|
"kernelspec": { |
|
"display_name": "Python 3 (ipykernel)", |
|
"language": "python", |
|
"name": "python3" |
|
}, |
|
"language_info": { |
|
"codemirror_mode": { |
|
"name": "ipython", |
|
"version": 3 |
|
}, |
|
"file_extension": ".py", |
|
"mimetype": "text/x-python", |
|
"name": "python", |
|
"nbconvert_exporter": "python", |
|
"pygments_lexer": "ipython3", |
|
"version": "3.11.10" |
|
} |
|
}, |
|
"nbformat": 4, |
|
"nbformat_minor": 5 |
|
}
|
|
|