You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 

493 lines
25 KiB

{
"cells": [
{
"cell_type": "markdown",
"id": "d15d8294-3328-4e07-ad16-8a03e9bbfdb9",
"metadata": {},
"source": [
"# Welcome to your first assignment!\n",
"\n",
"Instructions are below. Please give this a try, and look in the solutions folder if you get stuck (or feel free to ask me!)"
]
},
{
"cell_type": "markdown",
"id": "ada885d9-4d42-4d9b-97f0-74fbbbfe93a9",
"metadata": {},
"source": [
"<table style=\"margin: 0; text-align: left;\">\n",
" <tr>\n",
" <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
" <img src=\"../resources.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
" </td>\n",
" <td>\n",
" <h2 style=\"color:#f71;\">Just before we get to the assignment --</h2>\n",
" <span style=\"color:#f71;\">I thought I'd take a second to point you at this page of useful resources for the course. This includes links to all the slides.<br/>\n",
" <a href=\"https://edwarddonner.com/2024/11/13/llm-engineering-resources/\">https://edwarddonner.com/2024/11/13/llm-engineering-resources/</a><br/>\n",
" Please keep this bookmarked, and I'll continue to add more useful links there over time.\n",
" </span>\n",
" </td>\n",
" </tr>\n",
"</table>"
]
},
{
"cell_type": "markdown",
"id": "6e9fa1fc-eac5-4d1d-9be4-541b3f2b3458",
"metadata": {},
"source": [
"# HOMEWORK EXERCISE ASSIGNMENT\n",
"\n",
"Upgrade the day 1 project to summarize a webpage to use an Open Source model running locally via Ollama rather than OpenAI\n",
"\n",
"You'll be able to use this technique for all subsequent projects if you'd prefer not to use paid APIs.\n",
"\n",
"**Benefits:**\n",
"1. No API charges - open-source\n",
"2. Data doesn't leave your box\n",
"\n",
"**Disadvantages:**\n",
"1. Significantly less power than Frontier Model\n",
"\n",
"## Recap on installation of Ollama\n",
"\n",
"Simply visit [ollama.com](https://ollama.com) and install!\n",
"\n",
"Once complete, the ollama server should already be running locally. \n",
"If you visit: \n",
"[http://localhost:11434/](http://localhost:11434/)\n",
"\n",
"You should see the message `Ollama is running`. \n",
"\n",
"If not, bring up a new Terminal (Mac) or Powershell (Windows) and enter `ollama serve` \n",
"And in another Terminal (Mac) or Powershell (Windows), enter `ollama pull llama3.2` \n",
"Then try [http://localhost:11434/](http://localhost:11434/) again.\n",
"\n",
"If Ollama is slow on your machine, try using `llama3.2:1b` as an alternative. Run `ollama pull llama3.2:1b` from a Terminal or Powershell, and change the code below from `MODEL = \"llama3.2\"` to `MODEL = \"llama3.2:1b\"`"
]
},
{
"cell_type": "code",
"execution_count": 27,
"id": "4e2a9393-7767-488e-a8bf-27c12dca35bd",
"metadata": {},
"outputs": [],
"source": [
"# imports\n",
"\n",
"import requests\n",
"from bs4 import BeautifulSoup\n",
"from IPython.display import Markdown, display"
]
},
{
"cell_type": "code",
"execution_count": 29,
"id": "29ddd15d-a3c5-4f4e-a678-873f56162724",
"metadata": {},
"outputs": [],
"source": [
"# Constants\n",
"\n",
"OLLAMA_API = \"http://localhost:11434/api/chat\"\n",
"HEADERS = {\"Content-Type\": \"application/json\"}\n",
"MODEL = \"llama3.2:1b\""
]
},
{
"cell_type": "code",
"execution_count": 31,
"id": "dac0a679-599c-441f-9bf2-ddc73d35b940",
"metadata": {},
"outputs": [],
"source": [
"# Create a messages list using the same format that we used for OpenAI\n",
"\n",
"messages = [\n",
" {\"role\": \"user\", \"content\": \"Describe some of the business applications of Generative AI\"}\n",
"]"
]
},
{
"cell_type": "code",
"execution_count": 32,
"id": "7bb9c624-14f0-4945-a719-8ddb64f66f47",
"metadata": {},
"outputs": [],
"source": [
"payload = {\n",
" \"model\": MODEL,\n",
" \"messages\": messages,\n",
" \"stream\": False\n",
" }"
]
},
{
"cell_type": "code",
"execution_count": 33,
"id": "479ff514-e8bd-4985-a572-2ea28bb4fa40",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[?25lpulling manifest â ‹ \u001b[?25h\u001b[?25l\u001b[2K\u001b[1Gpulling manifest â ™ \u001b[?25h\u001b[?25l\u001b[2K\u001b[1Gpulling manifest \n",
"pulling 74701a8c35f6... 100% ▕████████████████â–<EFBFBD> 1.3 GB \n",
"pulling 966de95ca8a6... 100% ▕████████████████â–<EFBFBD> 1.4 KB \n",
"pulling fcc5a6bec9da... 100% ▕████████████████â–<EFBFBD> 7.7 KB \n",
"pulling a70ff7e570d9... 100% ▕████████████████â–<EFBFBD> 6.0 KB \n",
"pulling 4f659a1e86d7... 100% ▕████████████████â–<EFBFBD> 485 B \n",
"verifying sha256 digest \n",
"writing manifest \n",
"success \u001b[?25h\n"
]
}
],
"source": [
"# Let's just make sure the model is loaded\n",
"\n",
"!ollama pull llama3.2:1b"
]
},
{
"cell_type": "code",
"execution_count": 34,
"id": "42b9f644-522d-4e05-a691-56e7658c0ea9",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Generative AI has a wide range of business applications across various industries. Here are some examples:\n",
"\n",
"1. **Content Creation**: Generative AI can create high-quality content, such as articles, social media posts, and even entire books, at an unprecedented scale and speed. This can help businesses save time and resources, while also improving the quality and relevance of their content.\n",
"2. **Product Design**: Generative AI can generate design concepts, prototypes, and even entire products (e.g., 3D models) based on specific requirements or inputs. This can help companies like Apple, Ford, and IKEA streamline their product development processes.\n",
"3. **Virtual Reality (VR) and Augmented Reality (AR)**: Generative AI-powered tools can create immersive VR and AR experiences, allowing businesses to deliver more engaging and interactive experiences for customers, such as virtual product demonstrations or immersive training sessions.\n",
"4. **Predictive Maintenance**: Generative AI can analyze data from sensors and machines to predict when maintenance is required, reducing downtime and increasing overall efficiency. This can be particularly useful in industries like manufacturing, logistics, and energy.\n",
"5. **Personalized Marketing**: Generative AI can create personalized marketing messages, ads, and content based on customer behavior, preferences, and demographics. This can help businesses like Amazon, Facebook, and Google personalize their advertising efforts and improve customer engagement.\n",
"6. **Cybersecurity**: Generative AI can analyze network traffic and identify potential security threats, helping companies like IBM, Cisco, and Palo Alto to detect and prevent cyber attacks more efficiently.\n",
"7. **Business Process Automation**: Generative AI can automate repetitive business processes, such as data entry, reporting, and document processing, freeing up employees to focus on higher-value tasks and improving overall efficiency.\n",
"8. **Research and Development**: Generative AI can aid in research and development by generating new ideas, exploring new technologies, and identifying potential market opportunities. This can be particularly useful for companies like Google, Microsoft, and IBM.\n",
"9. **Training Data Generation**: Generative AI can generate large amounts of high-quality training data, such as text, images, or audio, which can help businesses like Netflix, Hulu, and Amazon improve their content offerings and reduce the need for human-generated data.\n",
"10. **Compliance and Risk Management**: Generative AI can analyze complex data sets to identify potential compliance risks and help companies like Accenture, Deloitte, and Ernst & Young assess and mitigate these risks.\n",
"\n",
"These are just a few examples of the business applications of Generative AI. As the technology continues to evolve, we can expect to see even more innovative uses across various industries.\n"
]
}
],
"source": [
"# If this doesn't work for any reason, try the 2 versions in the following cells\n",
"# And double check the instructions in the 'Recap on installation of Ollama' at the top of this lab\n",
"# And if none of that works - contact me!\n",
"\n",
"response = requests.post(OLLAMA_API, json=payload, headers=HEADERS)\n",
"print(response.json()['message']['content'])"
]
},
{
"cell_type": "code",
"execution_count": 35,
"id": "27c4ecfc-e6e5-4bd1-b581-168aa4f648fc",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"hellp\n"
]
}
],
"source": [
"print('hellp')"
]
},
{
"cell_type": "markdown",
"id": "6a021f13-d6a1-4b96-8e18-4eae49d876fe",
"metadata": {},
"source": [
"# Introducing the ollama package\n",
"\n",
"And now we'll do the same thing, but using the elegant ollama python package instead of a direct HTTP call.\n",
"\n",
"Under the hood, it's making the same call as above to the ollama server running at localhost:11434"
]
},
{
"cell_type": "code",
"execution_count": 36,
"id": "7745b9c4-57dc-4867-9180-61fa5db55eb8",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Generative AI has a wide range of business applications across various industries. Here are some examples:\n",
"\n",
"1. **Content Generation**: Generative AI can be used to create high-quality content, such as articles, videos, and social media posts, at scale and with minimal human intervention. This helps businesses save time and resources on content creation.\n",
"2. **Marketing Automation**: Generative AI-powered tools can analyze customer data and behavior to personalize marketing campaigns, predict customer preferences, and optimize ad targeting. This leads to improved ROI and better customer engagement.\n",
"3. **Predictive Maintenance**: Generative AI algorithms can be used to analyze sensor data from industrial equipment and predict when maintenance is required. This helps businesses reduce downtime, increase efficiency, and lower energy costs.\n",
"4. **Supply Chain Optimization**: Generative AI can optimize supply chain operations by analyzing historical data, predicting demand, and identifying potential bottlenecks. This leads to improved inventory management, reduced lead times, and increased customer satisfaction.\n",
"5. **Automated Customer Service**: Generative AI-powered chatbots can analyze customer queries, provide personalized responses, and route complex issues to human representatives for resolution. This improves the overall customer experience and reduces support costs.\n",
"6. **Product Design and Development**: Generative AI algorithms can generate product designs, prototypes, and test cases based on customer feedback, market trends, and product requirements. This streamlines the design-to-production process and reduces design errors.\n",
"7. **Financial Modeling and Analysis**: Generative AI can analyze large datasets to identify patterns, predict market trends, and optimize investment strategies. This helps businesses make informed decisions and improve their bottom line.\n",
"8. **Healthcare Research and Development**: Generative AI can analyze medical data, identify patterns, and generate new treatment options or research hypotheses. This accelerates the development of innovative treatments and improves patient outcomes.\n",
"9. **Cybersecurity Threat Detection**: Generative AI algorithms can analyze network traffic, identify anomalies, and predict potential security threats. This helps businesses protect their assets and prevent cyber attacks.\n",
"10. **Business Intelligence and Analytics**: Generative AI can generate predictive models, identify trends, and provide actionable insights to help businesses make data-driven decisions.\n",
"\n",
"These are just a few examples of the business applications of generative AI. As this technology continues to evolve, we can expect to see even more innovative use cases across various industries.\n"
]
}
],
"source": [
"import ollama\n",
"\n",
"response = ollama.chat(model=MODEL, messages=messages)\n",
"print(response['message']['content'])"
]
},
{
"cell_type": "markdown",
"id": "a4704e10-f5fb-4c15-a935-f046c06fb13d",
"metadata": {},
"source": [
"## Alternative approach - using OpenAI python library to connect to Ollama"
]
},
{
"cell_type": "code",
"execution_count": 46,
"id": "23057e00-b6fc-4678-93a9-6b31cb704bff",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Generative AI, including its various subfields such as StyleGAN, DALL-E, and AlphaDream, has a wide range of business applications across industries. Here are some examples:\n",
"\n",
"1. **Digital Product Design**: Generative AI can create unique digital products like 3D models, textures, patterns, or even entire virtual worlds. Companies in the luxury goods, fashion, automotive, medical, and gaming industries can use this technology to design bespoke items, reducing production time and costs.\n",
"\n",
"2. **Content Creation for Advertising and Marketing**: Platforms such as AI-generated art, music for commercials, or even personalized messages through text generators could revolutionize advertising and marketing by enabling mass production of unique content.\n",
"\n",
"3. **Training and Tutoring Materials**: Educators can utilize generative models to create customized lesson plans, practice materials, and adaptive assessments tailored to individual students' needs. This enhances engagement, outcomes, and student satisfaction.\n",
"\n",
"4. **Virtual Event Hosting**: With the rise of virtual events due to COVID-19, companies are increasingly using Generative AI-powered platforms to design immersive experiences with interactive simulations, personalized messages (via chatbots), gamification elements, and customized branding – significantly outpacing traditional event formats in terms of quality, engagement, and cost.\n",
"\n",
"5. **Product Design Simulation**: Product designers can use generative models to simulate production costs, resource usage, and time required to build certain components or entire products, saving consumers a significant amount of money and energy.\n",
"\n",
"6. **Quality Control and Assurance**: Generative AI models can analyze product images in real-time, quickly detecting defects, anomalies, or improvements by suggesting cosmetic edits, adjusting material properties, or re-designing faulty structures – greatly enhancing product quality without human intervention.\n",
"\n",
"7. **Language Translation Services**: By generating translated content that closely matches the style of spoken language, companies like Google Translate, deep learning-based translation tools, AI-powered interpretation platforms can bridge linguistic gaps across borders and languages faster than traditional translation methods.\n",
"\n",
"8. **Scientific Research Illustration**: Scientific illustration is significantly enhanced with Generative AI's ability to create realistic 3D models of complex systems and phenomena at scale. This streamlines scientific communication by making abstract concepts more visually engaging and accessible.\n",
"\n",
"9. **Virtual Product Displays and Catalogs**: Shopping websites could be transformed into immersive retail experiences with 360-degree product views, interactive augmented reality, or entirely recreated digital product catalogues – enhancing the customer experience and reducing returns significantly.\n",
"\n",
"10. **Predictive Maintenance Analytics**: Data from machines can be analyzed by Generative AI, enabling businesses to detect anomalies in real-time, predict maintenance needs, and prevent equipment failures – resulting in increased efficiency, reduced downtime, and extended machinery lifespan.\n"
]
}
],
"source": [
"# There's actually an alternative approach that some people might prefer\n",
"# You can use the OpenAI client python library to call Ollama:\n",
"\n",
"from openai import OpenAI\n",
"ollama_via_openai = OpenAI(base_url='http://localhost:11434/v1', api_key='ollama')\n",
"\n",
"response = ollama_via_openai.chat.completions.create(\n",
" model=MODEL,\n",
" messages=messages\n",
")\n",
"\n",
"print(response.choices[0].message.content)"
]
},
{
"cell_type": "markdown",
"id": "1622d9bb-5c68-4d4e-9ca4-b492c751f898",
"metadata": {},
"source": [
"# NOW the exercise for you\n",
"\n",
"Take the code from day1 and incorporate it here, to build a website summarizer that uses Llama 3.2 running locally instead of OpenAI; use either of the above approaches."
]
},
{
"cell_type": "code",
"execution_count": 44,
"id": "680736ba-6c91-49f1-8048-bb2f1fed4ab0",
"metadata": {},
"outputs": [],
"source": [
"# Some websites need you to use proper headers when fetching them:\n",
"headers = {\n",
" \"User-Agent\": \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36\"\n",
"}\n",
"\n",
"class Website:\n",
"\n",
" def __init__(self, url):\n",
" \"\"\"\n",
" Create this Website object from the given url using the BeautifulSoup library\n",
" \"\"\"\n",
" self.url = url\n",
" response = requests.get(url, headers=headers)\n",
" soup = BeautifulSoup(response.content, 'html.parser')\n",
" self.title = soup.title.string if soup.title else \"No title found\"\n",
" for irrelevant in soup.body([\"script\", \"style\", \"img\", \"input\"]):\n",
" irrelevant.decompose()\n",
" self.text = soup.body.get_text(separator=\"\\n\", strip=True)"
]
},
{
"cell_type": "code",
"execution_count": 38,
"id": "36c1ad8f-f008-40e4-acc1-6a9079bae204",
"metadata": {},
"outputs": [],
"source": [
"system_prompt = \"You are an assistant that analyzes the contents of a website \\\n",
"and provides a short summary, ignoring text that might be navigation related. \\\n",
"Respond in markdown.\""
]
},
{
"cell_type": "code",
"execution_count": 39,
"id": "16ed7a64-2d9c-4a4f-a3ee-bf40182a9112",
"metadata": {},
"outputs": [],
"source": [
"def user_prompt_for(website):\n",
" user_prompt = f\"You are looking at a website titled {website.title}\"\n",
" user_prompt += \"\\nThe contents of this website is as follows; \\\n",
"please provide a short summary of this website in markdown. \\\n",
"If it includes news or announcements, then summarize these too.\\n\\n\"\n",
" user_prompt += website.text\n",
" return user_prompt"
]
},
{
"cell_type": "code",
"execution_count": 40,
"id": "1c65eb6d-0cbb-494b-96e4-c01670714c20",
"metadata": {},
"outputs": [],
"source": [
"def messages_for(website):\n",
" return [\n",
" {\"role\": \"system\", \"content\": system_prompt},\n",
" {\"role\": \"user\", \"content\": user_prompt_for(website)}\n",
" ]"
]
},
{
"cell_type": "code",
"execution_count": 53,
"id": "402d5686-4e76-4110-b65a-b3906c35c0a4",
"metadata": {},
"outputs": [],
"source": [
"def summarize(url):\n",
" website = Website(url)\n",
" ollama_via_openai = OpenAI(base_url='http://localhost:11434/v1', api_key='ollama')\n",
" response = ollama_via_openai.chat.completions.create(\n",
" model=MODEL,\n",
" messages=messages\n",
" )\n",
" return response.choices[0].message.content"
]
},
{
"cell_type": "code",
"execution_count": 51,
"id": "235c9416-7ba9-4546-bee1-ed42ee40e529",
"metadata": {},
"outputs": [],
"source": [
"def display_summary(url):\n",
" summary = summarize(url)\n",
" display(Markdown(summary))"
]
},
{
"cell_type": "code",
"execution_count": 52,
"id": "aadccd53-48a2-42aa-9ce7-559392d77afd",
"metadata": {},
"outputs": [
{
"data": {
"text/markdown": [
"Generative AI has a wide range of business applications across various industries, including:\n",
"\n",
"1. **Content creation**: Generative AI can be used to generate high-quality content such as articles, videos, and graphics, reducing manual labor costs and increasing productivity.\n",
"2. **Design automation**: Generative AI can automate design tasks such as logo creation, brochures, and other visual elements, allowing for faster and more efficient design processes.\n",
"3. **Virtual product demonstrations**: Generative AI can create interactive 3D models of products, allowing end-users to view and interact with them in a virtual environment, reducing the need for physical prototypes.\n",
"4. **Product testing and validation**: Generative AI can generate test cases and validate product designs, reducing the time and cost associated with traditional testing methods.\n",
"5. **Medical imaging analysis**: Generative AI can analyze medical images such as MRI scans and CT images to help doctors diagnose and treat patients more accurately and efficiently.\n",
"6. **Customer service chatbots**: Generative AI-powered chatbots can automate customer support tasks such as answering frequently asked questions, routing complex issues to human representatives, and providing personalized product recommendations.\n",
"7. **Supply chain optimization**: Generative AI can analyze large datasets to identify trends and patterns, allowing companies to optimize their supply chains and reduce costs.\n",
"8. **Research and development**: Generative AI can assist scientists in designing new experiments, analyzing data, and identifying potential research opportunities.\n",
"9. **Risk management**: Generative AI can help companies model complex risks and develop mitigation strategies, reducing the likelihood of financial losses or product failures.\n",
"10. **Quality control**: Generative AI can analyze production data to detect anomalies and identify areas for improvement, reducing waste and inventory costs.\n",
"11. **Tourism and travel planning**: Generative AI can create personalized itineraries and recommend destinations based on travelers' interests and preferences.\n",
"12. **Manufacturing optimization**: Generative AI can optimize manufacturing processes by predicting demand, identifying bottlenecks, and suggesting improvements to streamline production.\n",
"\n",
"Additionally, generative AI is also being applied in areas such as:\n",
"\n",
"* **Cybersecurity**: to detect and respond to cyber threats\n",
"* **Education**: to personalize learning experiences and grade assignments\n",
"* **Finance**: to analyze market trends, identify investment opportunities, and optimize investment portfolios\n",
"* **Marketing**: to create targeted marketing campaigns and personalize customer interactions\n",
"\n",
"These are just a few examples of the many business applications of generative AI. As the technology continues to evolve, we can expect to see even more innovative uses across various industries."
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"display_summary(\"https://cnn.com\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "72d2a108-aa30-4c69-bd30-b994ffd9461b",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.11"
}
},
"nbformat": 4,
"nbformat_minor": 5
}