You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 

568 lines
26 KiB

{
"cells": [
{
"cell_type": "markdown",
"id": "d15d8294-3328-4e07-ad16-8a03e9bbfdb9",
"metadata": {},
"source": [
"# Welcome to your first assignment!\n",
"\n",
"Instructions are below. Please give this a try, and look in the solutions folder if you get stuck (or feel free to ask me!)"
]
},
{
"cell_type": "markdown",
"id": "ada885d9-4d42-4d9b-97f0-74fbbbfe93a9",
"metadata": {},
"source": [
"<table style=\"margin: 0; text-align: left;\">\n",
" <tr>\n",
" <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
" <img src=\"../resources.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" </td>\n",
" <td>\n",
" <h2 style=\"color:#f71;\">Just before we get to the assignment --</h2>\n",
" <span style=\"color:#f71;\">I thought I'd take a second to point you at this page of useful resources for the course. This includes links to all the slides.<br/>\n",
" <a href=\"https://edwarddonner.com/2024/11/13/llm-engineering-resources/\">https://edwarddonner.com/2024/11/13/llm-engineering-resources/</a><br/>\n",
" Please keep this bookmarked, and I'll continue to add more useful links there over time.\n",
" </span>\n",
" </td>\n",
" </tr>\n",
"</table>"
]
},
{
"cell_type": "markdown",
"id": "6e9fa1fc-eac5-4d1d-9be4-541b3f2b3458",
"metadata": {},
"source": [
"# HOMEWORK EXERCISE ASSIGNMENT\n",
"\n",
"Upgrade the day 1 project to summarize a webpage to use an Open Source model running locally via Ollama rather than OpenAI\n",
"\n",
"You'll be able to use this technique for all subsequent projects if you'd prefer not to use paid APIs.\n",
"\n",
"**Benefits:**\n",
"1. No API charges - open-source\n",
"2. Data doesn't leave your box\n",
"\n",
"**Disadvantages:**\n",
"1. Significantly less power than Frontier Model\n",
"\n",
"## Recap on installation of Ollama\n",
"\n",
"Simply visit [ollama.com](https://ollama.com) and install!\n",
"\n",
"Once complete, the ollama server should already be running locally. \n",
"If you visit: \n",
"[http://localhost:11434/](http://localhost:11434/)\n",
"\n",
"You should see the message `Ollama is running`. \n",
"\n",
"If not, bring up a new Terminal (Mac) or Powershell (Windows) and enter `ollama serve` \n",
"And in another Terminal (Mac) or Powershell (Windows), enter `ollama pull llama3.2` \n",
"Then try [http://localhost:11434/](http://localhost:11434/) again.\n",
"\n",
"If Ollama is slow on your machine, try using `llama3.2:1b` as an alternative. Run `ollama pull llama3.2:1b` from a Terminal or Powershell, and change the code below from `MODEL = \"llama3.2\"` to `MODEL = \"llama3.2:1b\"`"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "4e2a9393-7767-488e-a8bf-27c12dca35bd",
"metadata": {},
"outputs": [],
"source": [
"# imports\n",
"\n",
"import requests\n",
"from bs4 import BeautifulSoup\n",
"from IPython.display import Markdown, display"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "29ddd15d-a3c5-4f4e-a678-873f56162724",
"metadata": {},
"outputs": [],
"source": [
"# Constants\n",
"\n",
"OLLAMA_API = \"http://localhost:11434/api/chat\"\n",
"HEADERS = {\"Content-Type\": \"application/json\"}\n",
"MODEL = \"llama3.2\""
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "dac0a679-599c-441f-9bf2-ddc73d35b940",
"metadata": {},
"outputs": [],
"source": [
"# Create a messages list using the same format that we used for OpenAI\n",
"\n",
"messages = [\n",
" {\"role\": \"user\", \"content\": \"Describe some of the business applications of Generative AI\"}\n",
"]"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "7bb9c624-14f0-4945-a719-8ddb64f66f47",
"metadata": {},
"outputs": [],
"source": [
"payload = {\n",
" \"model\": MODEL,\n",
" \"messages\": messages,\n",
" \"stream\": False\n",
" }"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "42b9f644-522d-4e05-a691-56e7658c0ea9",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Generative AI has numerous business applications across various industries. Here are some examples:\n",
"\n",
"1. **Content Creation**: Generative AI can create high-quality content, such as articles, social media posts, and product descriptions, in a matter of minutes. This can be particularly useful for businesses that need to generate large amounts of content quickly.\n",
"2. **Marketing Automation**: Generative AI can help automate marketing processes, such as personalized email campaigns, ad copywriting, and lead generation. By analyzing customer data and behavior, generative AI can create targeted and relevant content that resonates with customers.\n",
"3. **Product Design and Development**: Generative AI can assist in the design and development of new products by generating 2D and 3D designs, prototypes, and even entire product lines. This can help reduce costs and speed up the product development process.\n",
"4. **Image and Video Generation**: Generative AI can create realistic images and videos that can be used for various business purposes, such as advertising, e-commerce, and social media content creation.\n",
"5. **Chatbots and Virtual Assistants**: Generative AI can power chatbots and virtual assistants that can engage with customers, provide support, and answer frequently asked questions. This can help businesses improve customer service and reduce the workload of human customer support agents.\n",
"6. **Supply Chain Optimization**: Generative AI can analyze supply chain data and generate optimized routes, schedules, and inventory management plans to improve logistics efficiency and reduce costs.\n",
"7. **Predictive Maintenance**: Generative AI can analyze equipment sensor data and predict maintenance needs, allowing businesses to schedule maintenance activities before equipment failures occur, reducing downtime and increasing overall efficiency.\n",
"8. **Financial Analysis and Forecasting**: Generative AI can analyze financial data and generate forecasts, identifying trends and patterns that can help businesses make informed investment decisions.\n",
"9. **Customer Service Chatbots**: Generative AI can create personalized chatbots that can engage with customers, answer questions, and provide support in multiple languages.\n",
"10. **Education and Training**: Generative AI can create personalized learning plans, generate educational content, and even develop adaptive learning systems that adjust to individual student needs.\n",
"\n",
"Some specific business applications of generative AI include:\n",
"\n",
"* **Amazon's Product Recommendations**: Amazon uses generative AI to recommend products based on customer behavior and preferences.\n",
"* **Google's Content Generation**: Google uses generative AI to create high-quality content for its search engine, such as news summaries and product descriptions.\n",
"* **IBM's Watson**: IBM uses generative AI in its Watson platform to analyze large amounts of data and provide insights for various industries, including healthcare and finance.\n",
"\n",
"Overall, the business applications of generative AI are vast and continue to expand as the technology improves.\n"
]
}
],
"source": [
"response = requests.post(OLLAMA_API, json=payload, headers=HEADERS)\n",
"print(response.json()['message']['content'])"
]
},
{
"cell_type": "markdown",
"id": "6a021f13-d6a1-4b96-8e18-4eae49d876fe",
"metadata": {},
"source": [
"# Introducing the ollama package\n",
"\n",
"And now we'll do the same thing, but using the elegant ollama python package instead of a direct HTTP call.\n",
"\n",
"Under the hood, it's making the same call as above to the ollama server running at localhost:11434"
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "7745b9c4-57dc-4867-9180-61fa5db55eb8",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Generative AI has numerous business applications across various industries, including:\n",
"\n",
"1. **Content Generation**: Use AI to generate high-quality content such as blog posts, social media posts, product descriptions, and more.\n",
"2. **Marketing Automation**: Utilize generative AI to personalize marketing messages, create targeted advertising campaigns, and automate lead generation.\n",
"3. **Product Design and Development**: Leverage generative AI to design and develop new products, such as 3D models, prototypes, and designs for packaging and branding materials.\n",
"4. **Customer Service Chatbots**: Create chatbots that use generative AI to understand customer inquiries and provide personalized responses.\n",
"5. **Language Translation**: Apply generative AI to translate text, speech, and audio content in real-time.\n",
"6. **Image Generation**: Use generative AI to create high-quality images for marketing materials, product packaging, and more.\n",
"7. **Music Composition**: Utilize generative AI to compose original music tracks, sound effects, and audio loops for various industries.\n",
"8. **Predictive Analytics**: Leverage generative AI to analyze large datasets, identify patterns, and make predictions about customer behavior, market trends, and more.\n",
"9. **Financial Modeling**: Apply generative AI to create financial models, forecast revenue, and predict potential risks.\n",
"10. **Creative Writing**: Use generative AI to assist in creative writing tasks such as generating plot outlines, character development, and dialogue.\n",
"\n",
"Industry-specific applications:\n",
"\n",
"1. **Healthcare**: Generate medical imaging reports, create personalized patient profiles, and develop new treatment options using generative AI.\n",
"2. **Finance**: Analyze financial data, identify trends, and predict market movements using generative AI.\n",
"3. **Education**: Develop personalized learning plans, create adaptive assessments, and generate educational content using generative AI.\n",
"4. **Retail**: Generate product descriptions, optimize pricing strategies, and create targeted marketing campaigns using generative AI.\n",
"\n",
"Benefits of Generative AI:\n",
"\n",
"1. **Increased Efficiency**: Automate repetitive tasks, reduce manual labor, and increase productivity.\n",
"2. **Improved Accuracy**: Reduce human error, improve data accuracy, and enhance decision-making.\n",
"3. **Enhanced Creativity**: Unlock new creative possibilities, generate innovative ideas, and discover new opportunities.\n",
"\n",
"However, there are also challenges associated with Generative AI, such as:\n",
"\n",
"1. **Bias and Fairness**: Ensure that generative models do not perpetuate existing biases or discriminatory practices.\n",
"2. **Explainability and Transparency**: Develop techniques to understand how generative models make decisions and provide transparency into their decision-making processes.\n",
"3. **Job Displacement**: Prepare employees for the impact of automation on jobs and develop new skills to work alongside AI systems.\n",
"\n",
"Overall, Generative AI has the potential to transform various industries and bring about significant benefits, but it is crucial to address the associated challenges to maximize its value.\n"
]
}
],
"source": [
"import ollama\n",
"\n",
"response = ollama.chat(model=MODEL, messages=messages)\n",
"print(response['message']['content'])"
]
},
{
"cell_type": "markdown",
"id": "a4704e10-f5fb-4c15-a935-f046c06fb13d",
"metadata": {},
"source": [
"## Alternative approach - using OpenAI python library to connect to Ollama"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "23057e00-b6fc-4678-93a9-6b31cb704bff",
"metadata": {},
"outputs": [],
"source": [
"# There's actually an alternative approach that some people might prefer\n",
"# You can use the OpenAI client python library to call Ollama:\n",
"\n",
"from openai import OpenAI\n",
"ollama_via_openai = OpenAI(base_url='http://localhost:11434/v1', api_key='ollama')\n",
"\n",
"response = ollama_via_openai.chat.completions.create(\n",
" model=MODEL,\n",
" messages=messages\n",
")\n",
"\n",
"print(response.choices[0].message.content)"
]
},
{
"cell_type": "markdown",
"id": "1622d9bb-5c68-4d4e-9ca4-b492c751f898",
"metadata": {},
"source": [
"# NOW the exercise for you\n",
"\n",
"Take the code from day1 and incorporate it here, to build a website summarizer that uses Llama 3.2 running locally instead of OpenAI; use either of the above approaches."
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "2cedd52a-cb2e-4cfb-8e48-176954cd64e7",
"metadata": {},
"outputs": [],
"source": [
"# imports\n",
"\n",
"import os\n",
"import requests\n",
"from bs4 import BeautifulSoup\n",
"from IPython.display import Markdown, display\n",
"import ollama\n"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "402d5686-4e76-4110-b65a-b3906c35c0a4",
"metadata": {},
"outputs": [],
"source": [
"# Constants\n",
"\n",
"OLLAMA_API = \"http://localhost:11434/api/chat\"\n",
"HEADERS = {\"Content-Type\": \"application/json\"}\n",
"MODEL = \"llama3.2\""
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "be0c1b6e-8d7c-446f-8a9c-18b1c33078d8",
"metadata": {},
"outputs": [],
"source": [
"# A class to represent a Webpage\n",
"# If you're not familiar with Classes, check out the \"Intermediate Python\" notebook\n",
"\n",
"# Some websites need you to use proper headers when fetching them:\n",
"headers = {\n",
" \"User-Agent\": \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36\"\n",
"}\n",
"\n",
"class Website:\n",
"\n",
" def __init__(self, url):\n",
" \"\"\"\n",
" Create this Website object from the given url using the BeautifulSoup library\n",
" \"\"\"\n",
" self.url = url\n",
" response = requests.get(url, headers=headers)\n",
" soup = BeautifulSoup(response.content, 'html.parser')\n",
" self.title = soup.title.string if soup.title else \"No title found\"\n",
" for irrelevant in soup.body([\"script\", \"style\", \"img\", \"input\"]):\n",
" irrelevant.decompose()\n",
" self.text = soup.body.get_text(separator=\"\\n\", strip=True)"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "265f85c4-132d-4bb8-bda2-792e0b0da017",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Home - Edward Donner\n",
"Home\n",
"Outsmart\n",
"An arena that pits LLMs against each other in a battle of diplomacy and deviousness\n",
"About\n",
"Posts\n",
"Well, hi there.\n",
"I’m Ed. I like writing code and experimenting with LLMs, and hopefully you’re here because you do too. I also enjoy DJing (but I’m badly out of practice), amateur electronic music production (\n",
"very\n",
"amateur) and losing myself in\n",
"Hacker News\n",
", nodding my head sagely to things I only half understand.\n",
"I’m the co-founder and CTO of\n",
"Nebula.io\n",
". We’re applying AI to a field where it can make a massive, positive impact: helping people discover their potential and pursue their reason for being. Recruiters use our product today to source, understand, engage and manage talent. I’m previously the founder and CEO of AI startup untapt,\n",
"acquired in 2021\n",
".\n",
"We work with groundbreaking, proprietary LLMs verticalized for talent, we’ve\n",
"patented\n",
"our matching model, and our award-winning platform has happy customers and tons of press coverage.\n",
"Connect\n",
"with me for more!\n",
"November 13, 2024\n",
"Mastering AI and LLM Engineering – Resources\n",
"October 16, 2024\n",
"From Software Engineer to AI Data Scientist – resources\n",
"August 6, 2024\n",
"Outsmart LLM Arena – a battle of diplomacy and deviousness\n",
"June 26, 2024\n",
"Choosing the Right LLM: Toolkit and Resources\n",
"Navigation\n",
"Home\n",
"Outsmart\n",
"An arena that pits LLMs against each other in a battle of diplomacy and deviousness\n",
"About\n",
"Posts\n",
"Get in touch\n",
"ed [at] edwarddonner [dot] com\n",
"www.edwarddonner.com\n",
"Follow me\n",
"LinkedIn\n",
"Twitter\n",
"Facebook\n",
"Subscribe to newsletter\n",
"Type your email…\n",
"Subscribe\n"
]
}
],
"source": [
"# Let's try one out. Change the website and add print statements to follow along.\n",
"\n",
"ed = Website(\"https://edwarddonner.com\")\n",
"print(ed.title)\n",
"print(ed.text)"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "24ea875b-2ba0-41ad-b6be-4be8baeac16e",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"You are looking at a website titled Home - Edward Donner\n",
"The contents of this website is as follows; please provide a short summary of this website in markdown. If it includes news or announcements, then summarize these too.\n",
"\n",
"Home\n",
"Outsmart\n",
"An arena that pits LLMs against each other in a battle of diplomacy and deviousness\n",
"About\n",
"Posts\n",
"Well, hi there.\n",
"I’m Ed. I like writing code and experimenting with LLMs, and hopefully you’re here because you do too. I also enjoy DJing (but I’m badly out of practice), amateur electronic music production (\n",
"very\n",
"amateur) and losing myself in\n",
"Hacker News\n",
", nodding my head sagely to things I only half understand.\n",
"I’m the co-founder and CTO of\n",
"Nebula.io\n",
". We’re applying AI to a field where it can make a massive, positive impact: helping people discover their potential and pursue their reason for being. Recruiters use our product today to source, understand, engage and manage talent. I’m previously the founder and CEO of AI startup untapt,\n",
"acquired in 2021\n",
".\n",
"We work with groundbreaking, proprietary LLMs verticalized for talent, we’ve\n",
"patented\n",
"our matching model, and our award-winning platform has happy customers and tons of press coverage.\n",
"Connect\n",
"with me for more!\n",
"November 13, 2024\n",
"Mastering AI and LLM Engineering – Resources\n",
"October 16, 2024\n",
"From Software Engineer to AI Data Scientist – resources\n",
"August 6, 2024\n",
"Outsmart LLM Arena – a battle of diplomacy and deviousness\n",
"June 26, 2024\n",
"Choosing the Right LLM: Toolkit and Resources\n",
"Navigation\n",
"Home\n",
"Outsmart\n",
"An arena that pits LLMs against each other in a battle of diplomacy and deviousness\n",
"About\n",
"Posts\n",
"Get in touch\n",
"ed [at] edwarddonner [dot] com\n",
"www.edwarddonner.com\n",
"Follow me\n",
"LinkedIn\n",
"Twitter\n",
"Facebook\n",
"Subscribe to newsletter\n",
"Type your email…\n",
"Subscribe\n"
]
}
],
"source": [
"# Define our system/user prompt - you can experiment with this later, changing the last sentence to 'Respond in markdown in Spanish.\"\n",
"\n",
"system_prompt = \"You are an assistant that analyzes the contents of a website \\\n",
"and provides a short summary, ignoring text that might be navigation related. \\\n",
"Respond in markdown.\"\n",
"\n",
"# A function that writes a User Prompt that asks for summaries of websites:\n",
"\n",
"def user_prompt_for(website):\n",
" user_prompt = f\"You are looking at a website titled {website.title}\"\n",
" user_prompt += \"\\nThe contents of this website is as follows; \\\n",
"please provide a short summary of this website in markdown. \\\n",
"If it includes news or announcements, then summarize these too.\\n\\n\"\n",
" user_prompt += website.text\n",
" return user_prompt\n",
"\n",
"user_prompt = user_prompt_for(ed)\n",
"print(user_prompt)\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "6d5f0f47-58a2-43b7-87a9-181a87e08081",
"metadata": {},
"outputs": [],
"source": [
"# create the message \n",
"def messages_for(website):\n",
" return [\n",
" {\"role\": \"system\", \"content\": system_prompt},\n",
" {\"role\": \"user\", \"content\": user_prompt}\n",
" ]"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "55454071-faa1-4cb1-a8dc-1f93abad6718",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[{'role': 'system', 'content': 'You are an assistant that analyzes the contents of a website and provides a short summary, ignoring text that might be navigation related. Respond in markdown.'}, {'role': 'user', 'content': 'You are looking at a website titled Home - Edward Donner\\nThe contents of this website is as follows; please provide a short summary of this website in markdown. If it includes news or announcements, then summarize these too.\\n\\nHome\\nOutsmart\\nAn arena that pits LLMs against each other in a battle of diplomacy and deviousness\\nAbout\\nPosts\\nWell, hi there.\\nI’m Ed. I like writing code and experimenting with LLMs, and hopefully you’re here because you do too. I also enjoy DJing (but I’m badly out of practice), amateur electronic music production (\\nvery\\namateur) and losing myself in\\nHacker News\\n, nodding my head sagely to things I only half understand.\\nI’m the co-founder and CTO of\\nNebula.io\\n. We’re applying AI to a field where it can make a massive, positive impact: helping people discover their potential and pursue their reason for being. Recruiters use our product today to source, understand, engage and manage talent. I’m previously the founder and CEO of AI startup untapt,\\nacquired in 2021\\n.\\nWe work with groundbreaking, proprietary LLMs verticalized for talent, we’ve\\npatented\\nour matching model, and our award-winning platform has happy customers and tons of press coverage.\\nConnect\\nwith me for more!\\nNovember 13, 2024\\nMastering AI and LLM Engineering – Resources\\nOctober 16, 2024\\nFrom Software Engineer to AI Data Scientist – resources\\nAugust 6, 2024\\nOutsmart LLM Arena – a battle of diplomacy and deviousness\\nJune 26, 2024\\nChoosing the Right LLM: Toolkit and Resources\\nNavigation\\nHome\\nOutsmart\\nAn arena that pits LLMs against each other in a battle of diplomacy and deviousness\\nAbout\\nPosts\\nGet in touch\\ned [at] edwarddonner [dot] com\\nwww.edwarddonner.com\\nFollow me\\nLinkedIn\\nTwitter\\nFacebook\\nSubscribe to newsletter\\nType your email…\\nSubscribe'}]\n"
]
}
],
"source": [
"print(messages_for(ed))"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "42e0d73b-7db5-431f-bf97-4767be627056",
"metadata": {},
"outputs": [],
"source": [
"# And now: call the OpenAI API. You will get very familiar with this!\n",
"\n",
"def summarize(url):\n",
" website = Website(url)\n",
" response = ollama.chat(model=MODEL, messages=messages_for(website)) \n",
" return response['message']['content']"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "419d4c04-a3c4-43d2-ac1f-5f7f3b84c92a",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'**Website Summary**\\n======================\\n\\n* The website is owned by Edward Donner, a co-founder and CTO of Nebula.io.\\n* It appears to be primarily focused on his experiences with Large Language Models (LLMs) and AI engineering.\\n\\n**News and Announcements**\\n---------------------------\\n\\n* **Mastering AI and LLM Engineering - Resources**: A collection of resources for learning about mastering AI and LLM engineering, announced on November 13, 2024.\\n* **From Software Engineer to AI Data Scientist – resources**: A list of resources to help move from a software engineer role to an AI data scientist, shared on October 16, 2024.\\n* **Outsmart LLM Arena – a battle of diplomacy and deviousness**: An introduction to the Outsmart arena where LLMs compete in a battle of diplomacy and strategy, announced on June 26, 2024.\\n\\n**Additional Information**\\n-------------------------\\n\\n* Edward Donner is also involved with various projects and companies, including Nebula.io and untapt (acquired in 2021).\\n* He shares his interests in DJing, electronic music production, and amateur coding endeavors.\\n* The website contains links to his social media profiles and a newsletter sign-up.'"
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"summarize(\"https://edwarddonner.com\")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.11"
}
},
"nbformat": 4,
"nbformat_minor": 5
}