You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 

152 lines
5.1 KiB

{
"cells": [
{
"cell_type": "markdown",
"id": "6e9fa1fc-eac5-4d1d-9be4-541b3f2b3458",
"metadata": {},
"source": [
"# Day 2 EXERCISE Solution:\n",
"\n",
"Upgraded day 1 project that scrapes and summarizes any webpage using an Open Source model running locally via Ollama instead of OpenAI\n",
"\n",
"## Note:-\n",
"If Ollama is slow on your machine, try using `llama3.2:1b` as an alternative: \n",
"1. Run `ollama pull llama3.2:1b` from a Terminal or Powershell\n",
"2. **Ctrl + /** to comment this code line below: `MODEL = \"llama3.2\"`\n",
"3. same **Ctrl + /** to uncomment: `MODEL = \"llama3.2:1b\"`"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "4e2a9393-7767-488e-a8bf-27c12dca35bd",
"metadata": {},
"outputs": [],
"source": [
"# imports:-\n",
"\n",
"import requests\n",
"from bs4 import BeautifulSoup\n",
"from IPython.display import Markdown, display\n",
"import ollama"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "29ddd15d-a3c5-4f4e-a678-873f56162724",
"metadata": {},
"outputs": [],
"source": [
"# Constants:-\n",
"\n",
"# MODEL = \"llama3.2\"\n",
"MODEL = \"llama3.2:1b\"\n",
"# MODEL = \"deepseek-r1:1.5b\""
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "6de38216-6d1c-48c4-877b-86d403f4e0f8",
"metadata": {},
"outputs": [
{
"data": {
"text/markdown": [
"# Home - Edward Donner\n",
"### Overview of Outsmart Arena\n",
"Outsmart Arena is a platform where Large Language Models (LLMs) compete against each other in a battle of diplomacy and deviousness.\n",
"\n",
"# About\n",
"This website features the story of Edward Donner, who co-founded Nebula.io, an AI startup that uses LLMs to help people discover their potential and pursue their purpose. Donner also shares his personal interests in coding, DJing, electronic music production, and more.\n",
"\n",
"# Posts\n",
"### Recent Updates\n",
"\n",
"* **January 23, 2025**: Announcement of a new LLM Workshop: Hands-on with Agents.\n",
"* **December 21, 2024**: Invitation to join SuperDataScientists for an upcoming event.\n",
"* **November 13, 2024**: Announcement of Mastering AI and LLM Engineering resources.\n",
"* **October 16, 2024**: Introduction to From Software Engineer to AI Data Scientist.\n",
"\n",
"### Subscriptions\n",
"To stay up-to-date with the latest news and announcements from Outsmart Arena, subscribe to their newsletter."
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"class Website:\n",
" def __init__(self, url):\n",
" self.url = url\n",
" response = requests.get(url)\n",
" soup = BeautifulSoup(response.content, 'html.parser')\n",
" self.title = soup.title.string if soup.title else \"No title found\"\n",
" for irrelevant in soup.body([\"script\", \"style\", \"img\", \"input\"]):\n",
" irrelevant.decompose()\n",
" self.text = soup.body.get_text(separator=\"\\n\", strip=True)\n",
"\n",
"\n",
"system_prompt = \"You are an assistant that analyzes the contents of a website \\\n",
" and provides a short summary, ignoring text that might be navigation related. \\\n",
" Respond in markdown.\"\n",
"\n",
"\n",
"def user_prompt_for(website):\n",
" user_prompt = f\"You are looking at a website titled {website.title}\"\n",
" user_prompt += \"\\nThe contents of this website is as follows; \\\n",
" please provide a short summary of this website in markdown. \\\n",
" If it includes news or announcements, then summarize these too.\\n\\n\"\n",
" user_prompt += website.text\n",
" return user_prompt\n",
"\n",
"\n",
"def messages_for(website):\n",
" return [\n",
" {\"role\": \"system\", \"content\": system_prompt},\n",
" {\"role\": \"user\", \"content\": user_prompt_for(website)}\n",
" ]\n",
"\n",
"\n",
"def summary(url):\n",
" website = Website(url)\n",
" response = ollama.chat(\n",
" model = MODEL,\n",
" messages = messages_for(website)\n",
" )\n",
" return display(Markdown(response['message']['content']))\n",
"\n",
"\n",
"summary(\"https://edwarddonner.com\")\n",
"# summary(\"https://cnn.com\")\n",
"# summary(\"https://anthropic.com\")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.7"
}
},
"nbformat": 4,
"nbformat_minor": 5
}