{ "cells": [ { "cell_type": "markdown", "id": "d15d8294-3328-4e07-ad16-8a03e9bbfdb9", "metadata": {}, "source": [ "# EXERCISE SOLUTION\n", "\n", "Upgrade the day 1 project to summarize a webpage to use an Open Source model running locally via Ollama rather than OpenAI\n", "\n", "You'll be able to use this technique for all subsequent projects if you'd prefer not to use paid APIs.\n", "\n", "**Benefits:**\n", "1. No API charges - open-source\n", "2. Data doesn't leave your box\n", "\n", "**Disadvantages:**\n", "1. Significantly less power than Frontier Model\n", "\n", "## Recap on installation of Ollama\n", "\n", "Simply visit [ollama.com](https://ollama.com) and install!\n", "\n", "Once complete, the ollama server should already be running locally. \n", "If you visit: \n", "[http://localhost:11434/](http://localhost:11434/)\n", "\n", "You should see the message `Ollama is running`. \n", "\n", "If not, bring up a new Terminal (Mac) or Powershell (Windows) and enter `ollama serve` \n", "Then try [http://localhost:11434/](http://localhost:11434/) again." ] }, { "cell_type": "code", "execution_count": null, "id": "4e2a9393-7767-488e-a8bf-27c12dca35bd", "metadata": {}, "outputs": [], "source": [ "# imports\n", "\n", "import requests\n", "from bs4 import BeautifulSoup\n", "from IPython.display import Markdown, display\n", "import ollama" ] }, { "cell_type": "code", "execution_count": null, "id": "29ddd15d-a3c5-4f4e-a678-873f56162724", "metadata": {}, "outputs": [], "source": [ "# Constants\n", "\n", "MODEL = \"llama3.2\"" ] }, { "cell_type": "code", "execution_count": null, "id": "c5e793b2-6775-426a-a139-4848291d0463", "metadata": {}, "outputs": [], "source": [ "# A class to represent a Webpage\n", "\n", "class Website:\n", " \"\"\"\n", " A utility class to represent a Website that we have scraped\n", " \"\"\"\n", " url: str\n", " title: str\n", " text: str\n", "\n", " def __init__(self, url):\n", " \"\"\"\n", " Create this Website object from the given url using the BeautifulSoup library\n", " \"\"\"\n", " self.url = url\n", " response = requests.get(url)\n", " soup = BeautifulSoup(response.content, 'html.parser')\n", " self.title = soup.title.string if soup.title else \"No title found\"\n", " for irrelevant in soup.body([\"script\", \"style\", \"img\", \"input\"]):\n", " irrelevant.decompose()\n", " self.text = soup.body.get_text(separator=\"\\n\", strip=True)" ] }, { "cell_type": "code", "execution_count": null, "id": "2ef960cf-6dc2-4cda-afb3-b38be12f4c97", "metadata": {}, "outputs": [], "source": [ "# Let's try one out\n", "\n", "ed = Website(\"https://edwarddonner.com\")\n", "print(ed.title)\n", "print(ed.text)" ] }, { "cell_type": "markdown", "id": "6a478a0c-2c53-48ff-869c-4d08199931e1", "metadata": {}, "source": [ "## Types of prompts\n", "\n", "You may know this already - but if not, you will get very familiar with it!\n", "\n", "Models like GPT4o have been trained to receive instructions in a particular way.\n", "\n", "They expect to receive:\n", "\n", "**A system prompt** that tells them what task they are performing and what tone they should use\n", "\n", "**A user prompt** -- the conversation starter that they should reply to" ] }, { "cell_type": "code", "execution_count": null, "id": "abdb8417-c5dc-44bc-9bee-2e059d162699", "metadata": {}, "outputs": [], "source": [ "# Define our system prompt - you can experiment with this later, changing the last sentence to 'Respond in markdown in Spanish.\"\n", "\n", "system_prompt = \"You are an assistant that analyzes the contents of a website \\\n", "and provides a short summary, ignoring text that might be navigation related. \\\n", "Respond in markdown.\"" ] }, { "cell_type": "code", "execution_count": null, "id": "f0275b1b-7cfe-4f9d-abfa-7650d378da0c", "metadata": {}, "outputs": [], "source": [ "# A function that writes a User Prompt that asks for summaries of websites:\n", "\n", "def user_prompt_for(website):\n", " user_prompt = f\"You are looking at a website titled {website.title}\"\n", " user_prompt += \"The contents of this website is as follows; \\\n", "please provide a short summary of this website in markdown. \\\n", "If it includes news or announcements, then summarize these too.\\n\\n\"\n", " user_prompt += website.text\n", " return user_prompt" ] }, { "cell_type": "markdown", "id": "ea211b5f-28e1-4a86-8e52-c0b7677cadcc", "metadata": {}, "source": [ "## Messages\n", "\n", "The API from Ollama expects the same message format as OpenAI:\n", "\n", "```\n", "[\n", " {\"role\": \"system\", \"content\": \"system message goes here\"},\n", " {\"role\": \"user\", \"content\": \"user message goes here\"}\n", "]" ] }, { "cell_type": "code", "execution_count": null, "id": "0134dfa4-8299-48b5-b444-f2a8c3403c88", "metadata": {}, "outputs": [], "source": [ "# See how this function creates exactly the format above\n", "\n", "def messages_for(website):\n", " return [\n", " {\"role\": \"system\", \"content\": system_prompt},\n", " {\"role\": \"user\", \"content\": user_prompt_for(website)}\n", " ]" ] }, { "cell_type": "markdown", "id": "16f49d46-bf55-4c3e-928f-68fc0bf715b0", "metadata": {}, "source": [ "## Time to bring it together - now with Ollama instead of OpenAI" ] }, { "cell_type": "code", "execution_count": null, "id": "905b9919-aba7-45b5-ae65-81b3d1d78e34", "metadata": {}, "outputs": [], "source": [ "# And now: call the Ollama function instead of OpenAI\n", "\n", "def summarize(url):\n", " website = Website(url)\n", " messages = messages_for(website)\n", " response = ollama.chat(model=MODEL, messages=messages)\n", " return response['message']['content']" ] }, { "cell_type": "code", "execution_count": null, "id": "05e38d41-dfa4-4b20-9c96-c46ea75d9fb5", "metadata": {}, "outputs": [], "source": [ "summarize(\"https://edwarddonner.com\")" ] }, { "cell_type": "code", "execution_count": null, "id": "3d926d59-450e-4609-92ba-2d6f244f1342", "metadata": {}, "outputs": [], "source": [ "# A function to display this nicely in the Jupyter output, using markdown\n", "\n", "def display_summary(url):\n", " summary = summarize(url)\n", " display(Markdown(summary))" ] }, { "cell_type": "code", "execution_count": null, "id": "3018853a-445f-41ff-9560-d925d1774b2f", "metadata": {}, "outputs": [], "source": [ "display_summary(\"https://edwarddonner.com\")" ] }, { "cell_type": "markdown", "id": "b3bcf6f4-adce-45e9-97ad-d9a5d7a3a624", "metadata": {}, "source": [ "# Let's try more websites\n", "\n", "Note that this will only work on websites that can be scraped using this simplistic approach.\n", "\n", "Websites that are rendered with Javascript, like React apps, won't show up. See the community-contributions folder for a Selenium implementation that gets around this. You'll need to read up on installing Selenium (ask ChatGPT!)\n", "\n", "Also Websites protected with CloudFront (and similar) may give 403 errors - many thanks Andy J for pointing this out.\n", "\n", "But many websites will work just fine!" ] }, { "cell_type": "code", "execution_count": null, "id": "45d83403-a24c-44b5-84ac-961449b4008f", "metadata": {}, "outputs": [], "source": [ "display_summary(\"https://cnn.com\")" ] }, { "cell_type": "code", "execution_count": null, "id": "75e9fd40-b354-4341-991e-863ef2e59db7", "metadata": {}, "outputs": [], "source": [ "display_summary(\"https://anthropic.com\")" ] }, { "cell_type": "markdown", "id": "eeab24dc-5f90-4570-b542-b0585aca3eb6", "metadata": {}, "source": [ "# Sharing your code\n", "\n", "I'd love it if you share your code afterwards so I can share it with others! You'll notice that some students have already made changes (including a Selenium implementation) which you will find in the community-contributions folder. If you'd like add your changes to that folder, submit a Pull Request with your new versions in that folder and I'll merge your changes.\n", "\n", "If you're not an expert with git (and I am not!) then GPT has given some nice instructions on how to submit a Pull Request. It's a bit of an involved process, but once you've done it once it's pretty clear. As a pro-tip: it's best if you clear the outputs of your Jupyter notebooks (Edit >> Clean outputs of all cells, and then Save) for clean notebooks.\n", "\n", "PR instructions courtesy of an AI friend: https://chatgpt.com/share/670145d5-e8a8-8012-8f93-39ee4e248b4c" ] }, { "cell_type": "code", "execution_count": null, "id": "682eff74-55c4-4d4b-b267-703edbc293c7", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.2" } }, "nbformat": 4, "nbformat_minor": 5 }