From the uDemy course on LLM engineering.
https://www.udemy.com/course/llm-engineering-master-ai-and-large-language-models
You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
401 lines
20 KiB
401 lines
20 KiB
{ |
|
"cells": [ |
|
{ |
|
"cell_type": "markdown", |
|
"id": "d15d8294-3328-4e07-ad16-8a03e9bbfdb9", |
|
"metadata": {}, |
|
"source": [ |
|
"# Welcome to your first assignment!\n", |
|
"\n", |
|
"Instructions are below. Please give this a try, and look in the solutions folder if you get stuck (or feel free to ask me!)" |
|
] |
|
}, |
|
{ |
|
"cell_type": "markdown", |
|
"id": "ada885d9-4d42-4d9b-97f0-74fbbbfe93a9", |
|
"metadata": {}, |
|
"source": [ |
|
"<table style=\"margin: 0; text-align: left;\">\n", |
|
" <tr>\n", |
|
" <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n", |
|
" <img src=\"../resources.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n", |
|
" </td>\n", |
|
" <td>\n", |
|
" <h2 style=\"color:#f71;\">Just before we get to the assignment --</h2>\n", |
|
" <span style=\"color:#f71;\">I thought I'd take a second to point you at this page of useful resources for the course. This includes links to all the slides.<br/>\n", |
|
" <a href=\"https://edwarddonner.com/2024/11/13/llm-engineering-resources/\">https://edwarddonner.com/2024/11/13/llm-engineering-resources/</a><br/>\n", |
|
" Please keep this bookmarked, and I'll continue to add more useful links there over time.\n", |
|
" </span>\n", |
|
" </td>\n", |
|
" </tr>\n", |
|
"</table>" |
|
] |
|
}, |
|
{ |
|
"cell_type": "markdown", |
|
"id": "6e9fa1fc-eac5-4d1d-9be4-541b3f2b3458", |
|
"metadata": {}, |
|
"source": [ |
|
"# HOMEWORK EXERCISE ASSIGNMENT\n", |
|
"\n", |
|
"Upgrade the day 1 project to summarize a webpage to use an Open Source model running locally via Ollama rather than OpenAI\n", |
|
"\n", |
|
"You'll be able to use this technique for all subsequent projects if you'd prefer not to use paid APIs.\n", |
|
"\n", |
|
"**Benefits:**\n", |
|
"1. No API charges - open-source\n", |
|
"2. Data doesn't leave your box\n", |
|
"\n", |
|
"**Disadvantages:**\n", |
|
"1. Significantly less power than Frontier Model\n", |
|
"\n", |
|
"## Recap on installation of Ollama\n", |
|
"\n", |
|
"Simply visit [ollama.com](https://ollama.com) and install!\n", |
|
"\n", |
|
"Once complete, the ollama server should already be running locally. \n", |
|
"If you visit: \n", |
|
"[http://localhost:11434/](http://localhost:11434/)\n", |
|
"\n", |
|
"You should see the message `Ollama is running`. \n", |
|
"\n", |
|
"If not, bring up a new Terminal (Mac) or Powershell (Windows) and enter `ollama serve` \n", |
|
"And in another Terminal (Mac) or Powershell (Windows), enter `ollama pull llama3.2` \n", |
|
"Then try [http://localhost:11434/](http://localhost:11434/) again.\n", |
|
"\n", |
|
"If Ollama is slow on your machine, try using `llama3.2:1b` as an alternative. Run `ollama pull llama3.2:1b` from a Terminal or Powershell, and change the code below from `MODEL = \"llama3.2\"` to `MODEL = \"llama3.2:1b\"`" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 1, |
|
"id": "4e2a9393-7767-488e-a8bf-27c12dca35bd", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# imports\n", |
|
"\n", |
|
"import requests\n", |
|
"from bs4 import BeautifulSoup\n", |
|
"from IPython.display import Markdown, display" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 2, |
|
"id": "29ddd15d-a3c5-4f4e-a678-873f56162724", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# Constants\n", |
|
"\n", |
|
"OLLAMA_API = \"http://localhost:11434/api/chat\"\n", |
|
"HEADERS = {\"Content-Type\": \"application/json\"}\n", |
|
"MODEL = \"llama3.2\"" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 3, |
|
"id": "dac0a679-599c-441f-9bf2-ddc73d35b940", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# Create a messages list using the same format that we used for OpenAI\n", |
|
"\n", |
|
"messages = [\n", |
|
" {\"role\": \"user\", \"content\": \"Describe some of the business applications of Generative AI\"}\n", |
|
"]" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 4, |
|
"id": "7bb9c624-14f0-4945-a719-8ddb64f66f47", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"payload = {\n", |
|
" \"model\": MODEL,\n", |
|
" \"messages\": messages,\n", |
|
" \"stream\": False\n", |
|
" }" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 5, |
|
"id": "479ff514-e8bd-4985-a572-2ea28bb4fa40", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# Let's just make sure the model is loaded\n", |
|
"\n", |
|
"!ollama pull llama3.2" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": null, |
|
"id": "42b9f644-522d-4e05-a691-56e7658c0ea9", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"name": "stdout", |
|
"output_type": "stream", |
|
"text": [ |
|
"Generative AI has numerous business applications across various industries, including:\n", |
|
"\n", |
|
"1. **Content Creation**: AI-powered tools can generate high-quality content such as articles, blog posts, social media posts, and product descriptions, saving time and resources for writers and marketers.\n", |
|
"2. **Visual Content Generation**: Generative AI can create images, videos, and 3D models, revolutionizing the way businesses produce visual content for marketing, advertising, and branding purposes.\n", |
|
"3. **Chatbots and Virtual Assistants**: AI-powered chatbots can interact with customers, provide customer support, and answer frequently asked questions, improving customer experience and reducing support costs.\n", |
|
"4. **Product Design and Development**: Generative AI can help designers and engineers create new product designs, prototypes, and models, streamlining the design process and reducing costs.\n", |
|
"5. **Marketing Automation**: AI-powered tools can analyze customer data, behavior, and preferences to generate targeted marketing campaigns, personalized emails, and offers, increasing campaign effectiveness and ROI.\n", |
|
"6. **Predictive Analytics**: Generative AI can analyze historical data, identify patterns, and make predictions about future market trends, helping businesses anticipate and prepare for changes in the market.\n", |
|
"7. **Financial Modeling and Analysis**: AI-powered tools can generate financial models, forecasts, and scenarios, enabling businesses to optimize investment strategies, predict revenue growth, and mitigate risks.\n", |
|
"8. **Data Analysis and Insights**: Generative AI can analyze large datasets, identify insights, and provide recommendations, helping businesses make data-driven decisions and drive business outcomes.\n", |
|
"9. **Cybersecurity**: AI-powered tools can detect anomalies, predict threats, and generate security alerts, enhancing the effectiveness of cybersecurity measures and protecting against cyber-attacks.\n", |
|
"10. **Supply Chain Optimization**: Generative AI can analyze supply chain data, identify bottlenecks, and optimize logistics and inventory management, reducing costs, improving efficiency, and increasing customer satisfaction.\n", |
|
"\n", |
|
"Some specific industries where Generative AI has already shown promise include:\n", |
|
"\n", |
|
"* E-commerce (product recommendations, personalized content)\n", |
|
"* Advertising (ad creative generation, targeting optimization)\n", |
|
"* Healthcare (medical imaging analysis, disease diagnosis)\n", |
|
"* Education (personalized learning plans, adaptive assessments)\n", |
|
"* Finance (risk analysis, portfolio optimization)\n", |
|
"\n", |
|
"These are just a few examples of the many business applications of Generative AI. As the technology continues to evolve, we can expect to see even more innovative uses across various industries and domains.\n" |
|
] |
|
} |
|
], |
|
"source": [ |
|
"# If this doesn't work for any reason, try the 2 versions in the following cells\n", |
|
"# And double check the instructions in the 'Recap on installation of Ollama' at the top of this lab\n", |
|
"# And if none of that works - contact me!\n", |
|
"\n", |
|
"response = requests.post(OLLAMA_API, json=payload, headers=HEADERS)\n", |
|
"print(response.json()['message']['content'])" |
|
] |
|
}, |
|
{ |
|
"cell_type": "markdown", |
|
"id": "6a021f13-d6a1-4b96-8e18-4eae49d876fe", |
|
"metadata": {}, |
|
"source": [ |
|
"# Introducing the ollama package\n", |
|
"\n", |
|
"And now we'll do the same thing, but using the elegant ollama python package instead of a direct HTTP call.\n", |
|
"\n", |
|
"Under the hood, it's making the same call as above to the ollama server running at localhost:11434" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 6, |
|
"id": "7745b9c4-57dc-4867-9180-61fa5db55eb8", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"name": "stdout", |
|
"output_type": "stream", |
|
"text": [ |
|
"Generative AI has numerous business applications across various industries. Here are some examples:\n", |
|
"\n", |
|
"1. **Content Creation**: Generative AI can be used to create personalized content, such as articles, social media posts, and product descriptions. This can help businesses automate their content creation process and reduce the cost of creating high-quality content.\n", |
|
"2. **Product Design**: Generative AI can be used to design new products, such as furniture, clothing, and electronics. This can help businesses create innovative products with unique designs that appeal to customers.\n", |
|
"3. **Marketing Automation**: Generative AI can be used to automate marketing campaigns, such as email marketing and social media advertising. This can help businesses personalize their marketing messages and improve the effectiveness of their campaigns.\n", |
|
"4. **Customer Service**: Generative AI can be used to create chatbots that provide customer support and answer common questions. This can help businesses improve their customer service experience and reduce the cost of providing support.\n", |
|
"5. **Predictive Maintenance**: Generative AI can be used to analyze data from sensors and machines to predict when maintenance is needed. This can help businesses reduce downtime, improve equipment efficiency, and extend equipment lifespan.\n", |
|
"6. **Recommendation Systems**: Generative AI can be used to create personalized product recommendations based on customer behavior and preferences. This can help businesses increase sales and improve customer satisfaction.\n", |
|
"7. **Data Analysis**: Generative AI can be used to analyze large datasets and identify patterns and trends that may not be visible to human analysts. This can help businesses make data-driven decisions and improve their operations.\n", |
|
"8. **Financial Modeling**: Generative AI can be used to create financial models that simulate different scenarios and predict outcomes. This can help businesses make informed investment decisions and reduce the risk of losses.\n", |
|
"9. **Supply Chain Optimization**: Generative AI can be used to optimize supply chain logistics, such as predicting demand, managing inventory, and optimizing shipping routes. This can help businesses improve their supply chain efficiency and reduce costs.\n", |
|
"10. **Cybersecurity**: Generative AI can be used to detect and respond to cyber threats in real-time. This can help businesses protect their data and systems from attacks.\n", |
|
"\n", |
|
"Some examples of companies that are using generative AI include:\n", |
|
"\n", |
|
"* **Netflix**: Using generative AI to create personalized movie recommendations\n", |
|
"* **Amazon**: Using generative AI to personalize product recommendations and improve customer service\n", |
|
"* **Google**: Using generative AI to improve search results and provide more accurate information\n", |
|
"* **BMW**: Using generative AI to design new car models and optimize production processes\n", |
|
"* **Airbnb**: Using generative AI to create personalized travel experiences for customers\n", |
|
"\n", |
|
"These are just a few examples of the many business applications of generative AI. As the technology continues to evolve, we can expect to see even more innovative uses in various industries.\n" |
|
] |
|
} |
|
], |
|
"source": [ |
|
"import ollama\n", |
|
"\n", |
|
"response = ollama.chat(model=MODEL, messages=messages)\n", |
|
"print(response['message']['content'])" |
|
] |
|
}, |
|
{ |
|
"cell_type": "markdown", |
|
"id": "a4704e10-f5fb-4c15-a935-f046c06fb13d", |
|
"metadata": {}, |
|
"source": [ |
|
"## Alternative approach - using OpenAI python library to connect to Ollama" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 7, |
|
"id": "23057e00-b6fc-4678-93a9-6b31cb704bff", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"name": "stdout", |
|
"output_type": "stream", |
|
"text": [ |
|
"Generative AI has numerous business applications across various industries, including:\n", |
|
"\n", |
|
"1. **Content Creation**: Generative AI can be used to generate high-quality content such as images, videos, articles, and social media posts automatically.\n", |
|
"2. **Personalization**: Generative AI can be used to create personalized experiences for customers by generating customized products or services based on their preferences, behavior, and demographics.\n", |
|
"3. **Marketing Automation**: Generative AI can automate marketing tasks such as lead generation, email campaigns, and ad creative optimization.\n", |
|
"4. **Product Design**: Generative AI can be used to generate designs for new products, packaging, and branding materials in various industries like fashion, furniture, and consumer goods.\n", |
|
"5. **Sales Predictive Analytics**: Generative AI can analyze historical sales data and generate predictive models to forecast demand, identify potential customers, and optimize sales strategies.\n", |
|
"6. **Finance and Banking**: Generative AI can be used for portfolio optimization, risk analysis, and credit scoring.\n", |
|
"7. **Healthcare**: Generative AI can be used to identify patterns in patient data, diagnose diseases more accurately, and generate personalized treatment plans.\n", |
|
"8. **Education**: Generative AI can create personalized learning materials, adaptive curricula, and virtual teaching assistants.\n", |
|
"9. **Supply Chain Optimization**: Generative AI can optimize logistics and supply chain management by analyzing traffic patterns, predicting demand, and identifying opportunities for cost savings.\n", |
|
"10. **Customer Service Chatbots**: Generative AI can generate conversational flows, respond to customer inquiries, and provide basic support.\n", |
|
"\n", |
|
"Some specific examples of business applications include:\n", |
|
"\n", |
|
"* **Google's Imagining Assistant**: uses Generative AI to create customized search results and suggestions based on a user's context.\n", |
|
"* **Facebook's Custom Content Creation Tool**: uses Generative AI to create personalized news articles and social media posts based on a user's interests.\n", |
|
"* **IBM Watson**: uses Generative AI to analyze vast amounts of customer data, identify patterns, and generate predictive models.\n", |
|
"\n", |
|
"These are just a few examples of the many business applications of Generative AI. As the technology continues to evolve, we can expect to see even more innovative solutions across various industries.\n" |
|
] |
|
} |
|
], |
|
"source": [ |
|
"# There's actually an alternative approach that some people might prefer\n", |
|
"# You can use the OpenAI client python library to call Ollama:\n", |
|
"\n", |
|
"from openai import OpenAI\n", |
|
"ollama_via_openai = OpenAI(base_url='http://localhost:11434/v1', api_key='ollama')\n", |
|
"\n", |
|
"response = ollama_via_openai.chat.completions.create(\n", |
|
" model=MODEL,\n", |
|
" messages=messages\n", |
|
")\n", |
|
"\n", |
|
"print(response.choices[0].message.content)" |
|
] |
|
}, |
|
{ |
|
"cell_type": "markdown", |
|
"id": "9f9e22da-b891-41f6-9ac9-bd0c0a5f4f44", |
|
"metadata": {}, |
|
"source": [ |
|
"## Are you confused about why that works?\n", |
|
"\n", |
|
"It seems strange, right? We just used OpenAI code to call Ollama?? What's going on?!\n", |
|
"\n", |
|
"Here's the scoop:\n", |
|
"\n", |
|
"The python class `OpenAI` is simply code written by OpenAI engineers that makes calls over the internet to an endpoint. \n", |
|
"\n", |
|
"When you call `openai.chat.completions.create()`, this python code just makes a web request to the following url: \"https://api.openai.com/v1/chat/completions\"\n", |
|
"\n", |
|
"Code like this is known as a \"client library\" - it's just wrapper code that runs on your machine to make web requests. The actual power of GPT is running on OpenAI's cloud behind this API, not on your computer!\n", |
|
"\n", |
|
"OpenAI was so popular, that lots of other AI providers provided identical web endpoints, so you could use the same approach.\n", |
|
"\n", |
|
"So Ollama has an endpoint running on your local box at http://localhost:11434/v1/chat/completions \n", |
|
"And in week 2 we'll discover that lots of other providers do this too, including Gemini and DeepSeek.\n", |
|
"\n", |
|
"And then the team at OpenAI had a great idea: they can extend their client library so you can specify a different 'base url', and use their library to call any compatible API.\n", |
|
"\n", |
|
"That's it!\n", |
|
"\n", |
|
"So when you say: `ollama_via_openai = OpenAI(base_url='http://localhost:11434/v1', api_key='ollama')` \n", |
|
"Then this will make the same endpoint calls, but to Ollama instead of OpenAI." |
|
] |
|
}, |
|
{ |
|
"cell_type": "markdown", |
|
"id": "bc7d1de3-e2ac-46ff-a302-3b4ba38c4c90", |
|
"metadata": {}, |
|
"source": [ |
|
"## Also trying the amazing reasoning model DeepSeek\n", |
|
"\n", |
|
"Here we use the version of DeepSeek-reasoner that's been distilled to 1.5B. \n", |
|
"This is actually a 1.5B variant of Qwen that has been fine-tuned using synethic data generated by Deepseek R1.\n", |
|
"\n", |
|
"Other sizes of DeepSeek are [here](https://ollama.com/library/deepseek-r1) all the way up to the full 671B parameter version, which would use up 404GB of your drive and is far too large for most!" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": null, |
|
"id": "cf9eb44e-fe5b-47aa-b719-0bb63669ab3d", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"!ollama pull deepseek-r1:1.5b" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": null, |
|
"id": "1d3d554b-e00d-4c08-9300-45e073950a76", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# This may take a few minutes to run! You should then see a fascinating \"thinking\" trace inside <think> tags, followed by some decent definitions\n", |
|
"\n", |
|
"response = ollama_via_openai.chat.completions.create(\n", |
|
" model=\"deepseek-r1:1.5b\",\n", |
|
" messages=[{\"role\": \"user\", \"content\": \"Please give definitions of some core concepts behind LLMs: a neural network, attention and the transformer\"}]\n", |
|
")\n", |
|
"\n", |
|
"print(response.choices[0].message.content)" |
|
] |
|
}, |
|
{ |
|
"cell_type": "markdown", |
|
"id": "1622d9bb-5c68-4d4e-9ca4-b492c751f898", |
|
"metadata": {}, |
|
"source": [ |
|
"# NOW the exercise for you\n", |
|
"\n", |
|
"Take the code from day1 and incorporate it here, to build a website summarizer that uses Llama 3.2 running locally instead of OpenAI; use either of the above approaches." |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": null, |
|
"id": "6de38216-6d1c-48c4-877b-86d403f4e0f8", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [] |
|
} |
|
], |
|
"metadata": { |
|
"kernelspec": { |
|
"display_name": "Python 3 (ipykernel)", |
|
"language": "python", |
|
"name": "python3" |
|
}, |
|
"language_info": { |
|
"codemirror_mode": { |
|
"name": "ipython", |
|
"version": 3 |
|
}, |
|
"file_extension": ".py", |
|
"mimetype": "text/x-python", |
|
"name": "python", |
|
"nbconvert_exporter": "python", |
|
"pygments_lexer": "ipython3", |
|
"version": "3.11.11" |
|
} |
|
}, |
|
"nbformat": 4, |
|
"nbformat_minor": 5 |
|
}
|
|
|