Browse Source

Completed Day2 exercise. Included selenium chrome webdriver support to read webpages.

pull/106/head
Rakesh H K 4 months ago
parent
commit
dddc58fa13
  1. 139
      week1/day2 EXERCISE.ipynb

139
week1/day2 EXERCISE.ipynb

@ -215,10 +215,147 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 6,
"id": "402d5686-4e76-4110-b65a-b3906c35c0a4",
"metadata": {},
"outputs": [],
"source": [
"# imports\n",
"\n",
"import ollama\n",
"import requests\n",
"from bs4 import BeautifulSoup\n",
"from IPython.display import Markdown, display\n",
"from selenium import webdriver\n",
"from selenium.webdriver.chrome.service import Service\n",
"from selenium.webdriver.common.by import By\n",
"from selenium.webdriver.chrome.options import Options\n",
"from openai import OpenAI\n",
"\n",
"#!ollama pull llama3.2\n",
"MODEL = \"llama3.2\"\n",
"openai = OpenAI(base_url=\"http://localhost:11434/v1\", api_key=\"ollama\")\n"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "cca8ae91-ad1e-4239-951f-e1376a5ec934",
"metadata": {},
"outputs": [
{
"name": "stdin",
"output_type": "stream",
"text": [
"Please complete the verification in the browser and press Enter to continue... \n"
]
},
{
"data": {
"text/markdown": [
"This article discussing Serverless architecture is a comprehensive overview of the concept, its benefits, and challenges. Here's a summary of the main points:\n",
"\n",
"**What is Serverless Architecture?**\n",
"\n",
"Serverless computing allows developers to run their applications without managing servers or provisioning resources upfront. Instead, they pay only for the compute time used.\n",
"\n",
"**Key Techniques:**\n",
"\n",
"1. **Function as a Service (FaaS)**: Ephemeral function instances that can be executed with specific inputs and outputs.\n",
"2. **Background Services**: Tightly integrate third-party remote application services directly into the frontend of an app.\n",
"\n",
"**Benefits:**\n",
"\n",
"1. Reduced operational and development costs\n",
"2. Easier management and scaling\n",
"3. Reduced environmental impact\n",
"4. Faster time-to-market\n",
"\n",
"**Challenges:**\n",
"\n",
"1. Debugging and monitoring complexity\n",
"2. Limited control over server-side code execution\n",
"3. High dependencies on cloud providers\n",
"\n",
"**Serverless Landscape:**\n",
"\n",
"The author expects the Serverless community to grow, with upcoming conferences, meetups, and online groups.\n",
"\n",
"**Conclusion:**\n",
"\n",
"Serverless architecture offers significant advantages but also presents challenges. It's essential to weigh the pros and cons carefully before adopting a Serverless approach. Despite its \"slightly awkward teenage years,\" Serverless is expected to continue evolving and maturing in the near future.\n",
"\n",
"Key Takeaways:\n",
"\n",
"1. Understand the basics of Serverless computing.\n",
"2. Be-aware of the trade-offs between scalability, control, and cost.\n",
"3. Consider your use case before adopting a Serverless architecture.\n",
"4. Stay updated with the latest developments and best practices in the Serverless community."
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"headers = {\n",
" \"User-Agent\": \"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36\"\n",
"}\n",
"\n",
"PATH_TO_CHROME_DRIVER = \"/Users/rakeshhk/Rakesh/softwares/chromedriver-mac-arm64/chromedriver\"\n",
"\n",
"class Website:\n",
"\n",
" def __init__(self, url):\n",
" self.url = url\n",
"\n",
" options = Options()\n",
"\n",
" options.add_argument(\"--no-sandbox\")\n",
" options.add_argument(\"--disable-dev-shm-usage\")\n",
"\n",
" service = Service(PATH_TO_CHROME_DRIVER)\n",
" driver = webdriver.Chrome(service=service, options=options)\n",
" driver.get(url)\n",
"\n",
" input(\"Please complete the verification in the browser and press Enter to continue...\")\n",
" page_source = driver.page_source\n",
" driver.quit()\n",
" \n",
" soup = BeautifulSoup(page_source, 'html.parser')\n",
" self.title = soup.title.string if soup.title else \"No title found\"\n",
" for irrelevant in soup.body([\"script\", \"style\", \"img\", \"input\"]):\n",
" irrelevant.decompose()\n",
" self.text = soup.get_text(separator=\"\\n\", strip=True)\n",
"\n",
"def messages_for(website):\n",
" return [{\"role\":\"system\", \"content\": \"You are a technology trainer, please read the content provided and highlight the key points in less than 200 words.\"},\n",
" {\"role\":\"user\", \"content\":website.text}]\n",
"\n",
"def summarize(url):\n",
" website = Website(url)\n",
" response = openai.chat.completions.create(\n",
" model = MODEL,\n",
" messages = messages_for(website)\n",
" )\n",
" return response.choices[0].message.content\n",
"\n",
"def display_summary(url):\n",
" summary = summarize(url)\n",
" display(Markdown(summary))\n",
" \n",
"display_summary(\"https://martinfowler.com/articles/serverless.html\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "09f59679-22ff-46c4-a736-7309a6ca4365",
"metadata": {},
"outputs": [],
"source": []
}
],

Loading…
Cancel
Save