From the uDemy course on LLM engineering.
https://www.udemy.com/course/llm-engineering-master-ai-and-large-language-models
You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
326 lines
9.8 KiB
326 lines
9.8 KiB
{ |
|
"cells": [ |
|
{ |
|
"cell_type": "markdown", |
|
"id": "d15d8294-3328-4e07-ad16-8a03e9bbfdb9", |
|
"metadata": {}, |
|
"source": [ |
|
"# EXERCISE SOLUTION\n", |
|
"\n", |
|
"Upgrade the day 1 project to summarize a webpage to use an Open Source model running locally via Ollama rather than OpenAI\n", |
|
"\n", |
|
"You'll be able to use this technique for all subsequent projects if you'd prefer not to use paid APIs.\n", |
|
"\n", |
|
"**Benefits:**\n", |
|
"1. No API charges - open-source\n", |
|
"2. Data doesn't leave your box\n", |
|
"\n", |
|
"**Disadvantages:**\n", |
|
"1. Significantly less power than Frontier Model\n", |
|
"\n", |
|
"## Recap on installation of Ollama\n", |
|
"\n", |
|
"Simply visit [ollama.com](https://ollama.com) and install!\n", |
|
"\n", |
|
"Once complete, the ollama server should already be running locally. \n", |
|
"If you visit: \n", |
|
"[http://localhost:11434/](http://localhost:11434/)\n", |
|
"\n", |
|
"You should see the message `Ollama is running`. \n", |
|
"\n", |
|
"If not, bring up a new Terminal (Mac) or Powershell (Windows) and enter `ollama serve` \n", |
|
"Then try [http://localhost:11434/](http://localhost:11434/) again." |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": null, |
|
"id": "4e2a9393-7767-488e-a8bf-27c12dca35bd", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# imports\n", |
|
"\n", |
|
"import requests\n", |
|
"from bs4 import BeautifulSoup\n", |
|
"from IPython.display import Markdown, display\n", |
|
"import ollama" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": null, |
|
"id": "29ddd15d-a3c5-4f4e-a678-873f56162724", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# Constants\n", |
|
"\n", |
|
"MODEL = \"llama3.2\"" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": null, |
|
"id": "c5e793b2-6775-426a-a139-4848291d0463", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# A class to represent a Webpage\n", |
|
"\n", |
|
"class Website:\n", |
|
" \"\"\"\n", |
|
" A utility class to represent a Website that we have scraped\n", |
|
" \"\"\"\n", |
|
" url: str\n", |
|
" title: str\n", |
|
" text: str\n", |
|
"\n", |
|
" def __init__(self, url):\n", |
|
" \"\"\"\n", |
|
" Create this Website object from the given url using the BeautifulSoup library\n", |
|
" \"\"\"\n", |
|
" self.url = url\n", |
|
" response = requests.get(url)\n", |
|
" soup = BeautifulSoup(response.content, 'html.parser')\n", |
|
" self.title = soup.title.string if soup.title else \"No title found\"\n", |
|
" for irrelevant in soup.body([\"script\", \"style\", \"img\", \"input\"]):\n", |
|
" irrelevant.decompose()\n", |
|
" self.text = soup.body.get_text(separator=\"\\n\", strip=True)" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": null, |
|
"id": "2ef960cf-6dc2-4cda-afb3-b38be12f4c97", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# Let's try one out\n", |
|
"\n", |
|
"ed = Website(\"https://edwarddonner.com\")\n", |
|
"print(ed.title)\n", |
|
"print(ed.text)" |
|
] |
|
}, |
|
{ |
|
"cell_type": "markdown", |
|
"id": "6a478a0c-2c53-48ff-869c-4d08199931e1", |
|
"metadata": {}, |
|
"source": [ |
|
"## Types of prompts\n", |
|
"\n", |
|
"You may know this already - but if not, you will get very familiar with it!\n", |
|
"\n", |
|
"Models like GPT4o have been trained to receive instructions in a particular way.\n", |
|
"\n", |
|
"They expect to receive:\n", |
|
"\n", |
|
"**A system prompt** that tells them what task they are performing and what tone they should use\n", |
|
"\n", |
|
"**A user prompt** -- the conversation starter that they should reply to" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": null, |
|
"id": "abdb8417-c5dc-44bc-9bee-2e059d162699", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# Define our system prompt - you can experiment with this later, changing the last sentence to 'Respond in markdown in Spanish.\"\n", |
|
"\n", |
|
"system_prompt = \"You are an assistant that analyzes the contents of a website \\\n", |
|
"and provides a short summary, ignoring text that might be navigation related. \\\n", |
|
"Respond in markdown.\"" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": null, |
|
"id": "f0275b1b-7cfe-4f9d-abfa-7650d378da0c", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# A function that writes a User Prompt that asks for summaries of websites:\n", |
|
"\n", |
|
"def user_prompt_for(website):\n", |
|
" user_prompt = f\"You are looking at a website titled {website.title}\"\n", |
|
" user_prompt += \"The contents of this website is as follows; \\\n", |
|
"please provide a short summary of this website in markdown. \\\n", |
|
"If it includes news or announcements, then summarize these too.\\n\\n\"\n", |
|
" user_prompt += website.text\n", |
|
" return user_prompt" |
|
] |
|
}, |
|
{ |
|
"cell_type": "markdown", |
|
"id": "ea211b5f-28e1-4a86-8e52-c0b7677cadcc", |
|
"metadata": {}, |
|
"source": [ |
|
"## Messages\n", |
|
"\n", |
|
"The API from Ollama expects the same message format as OpenAI:\n", |
|
"\n", |
|
"```\n", |
|
"[\n", |
|
" {\"role\": \"system\", \"content\": \"system message goes here\"},\n", |
|
" {\"role\": \"user\", \"content\": \"user message goes here\"}\n", |
|
"]" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": null, |
|
"id": "0134dfa4-8299-48b5-b444-f2a8c3403c88", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# See how this function creates exactly the format above\n", |
|
"\n", |
|
"def messages_for(website):\n", |
|
" return [\n", |
|
" {\"role\": \"system\", \"content\": system_prompt},\n", |
|
" {\"role\": \"user\", \"content\": user_prompt_for(website)}\n", |
|
" ]" |
|
] |
|
}, |
|
{ |
|
"cell_type": "markdown", |
|
"id": "16f49d46-bf55-4c3e-928f-68fc0bf715b0", |
|
"metadata": {}, |
|
"source": [ |
|
"## Time to bring it together - now with Ollama instead of OpenAI" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": null, |
|
"id": "905b9919-aba7-45b5-ae65-81b3d1d78e34", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# And now: call the Ollama function instead of OpenAI\n", |
|
"\n", |
|
"def summarize(url):\n", |
|
" website = Website(url)\n", |
|
" messages = messages_for(website)\n", |
|
" response = ollama.chat(model=MODEL, messages=messages)\n", |
|
" return response['message']['content']" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": null, |
|
"id": "05e38d41-dfa4-4b20-9c96-c46ea75d9fb5", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"summarize(\"https://edwarddonner.com\")" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": null, |
|
"id": "3d926d59-450e-4609-92ba-2d6f244f1342", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# A function to display this nicely in the Jupyter output, using markdown\n", |
|
"\n", |
|
"def display_summary(url):\n", |
|
" summary = summarize(url)\n", |
|
" display(Markdown(summary))" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": null, |
|
"id": "3018853a-445f-41ff-9560-d925d1774b2f", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"display_summary(\"https://edwarddonner.com\")" |
|
] |
|
}, |
|
{ |
|
"cell_type": "markdown", |
|
"id": "b3bcf6f4-adce-45e9-97ad-d9a5d7a3a624", |
|
"metadata": {}, |
|
"source": [ |
|
"# Let's try more websites\n", |
|
"\n", |
|
"Note that this will only work on websites that can be scraped using this simplistic approach.\n", |
|
"\n", |
|
"Websites that are rendered with Javascript, like React apps, won't show up. See the community-contributions folder for a Selenium implementation that gets around this. You'll need to read up on installing Selenium (ask ChatGPT!)\n", |
|
"\n", |
|
"Also Websites protected with CloudFront (and similar) may give 403 errors - many thanks Andy J for pointing this out.\n", |
|
"\n", |
|
"But many websites will work just fine!" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": null, |
|
"id": "45d83403-a24c-44b5-84ac-961449b4008f", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"display_summary(\"https://cnn.com\")" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": null, |
|
"id": "75e9fd40-b354-4341-991e-863ef2e59db7", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"display_summary(\"https://anthropic.com\")" |
|
] |
|
}, |
|
{ |
|
"cell_type": "markdown", |
|
"id": "eeab24dc-5f90-4570-b542-b0585aca3eb6", |
|
"metadata": {}, |
|
"source": [ |
|
"# Sharing your code\n", |
|
"\n", |
|
"I'd love it if you share your code afterwards so I can share it with others! You'll notice that some students have already made changes (including a Selenium implementation) which you will find in the community-contributions folder. If you'd like add your changes to that folder, submit a Pull Request with your new versions in that folder and I'll merge your changes.\n", |
|
"\n", |
|
"If you're not an expert with git (and I am not!) then GPT has given some nice instructions on how to submit a Pull Request. It's a bit of an involved process, but once you've done it once it's pretty clear. As a pro-tip: it's best if you clear the outputs of your Jupyter notebooks (Edit >> Clean outputs of all cells, and then Save) for clean notebooks.\n", |
|
"\n", |
|
"PR instructions courtesy of an AI friend: https://chatgpt.com/share/670145d5-e8a8-8012-8f93-39ee4e248b4c" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": null, |
|
"id": "682eff74-55c4-4d4b-b267-703edbc293c7", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [] |
|
} |
|
], |
|
"metadata": { |
|
"kernelspec": { |
|
"display_name": "Python 3 (ipykernel)", |
|
"language": "python", |
|
"name": "python3" |
|
}, |
|
"language_info": { |
|
"codemirror_mode": { |
|
"name": "ipython", |
|
"version": 3 |
|
}, |
|
"file_extension": ".py", |
|
"mimetype": "text/x-python", |
|
"name": "python", |
|
"nbconvert_exporter": "python", |
|
"pygments_lexer": "ipython3", |
|
"version": "3.11.11" |
|
} |
|
}, |
|
"nbformat": 4, |
|
"nbformat_minor": 5 |
|
}
|
|
|