From the uDemy course on LLM engineering.
https://www.udemy.com/course/llm-engineering-master-ai-and-large-language-models
You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
941 lines
38 KiB
941 lines
38 KiB
{ |
|
"cells": [ |
|
{ |
|
"cell_type": "markdown", |
|
"id": "06cf3063-9f3e-4551-a0d5-f08d9cabb927", |
|
"metadata": {}, |
|
"source": [ |
|
"# Welcome to Week 2!\n", |
|
"\n", |
|
"## Frontier Model APIs\n", |
|
"\n", |
|
"In Week 1, we used multiple Frontier LLMs through their Chat UI, and we connected with the OpenAI's API.\n", |
|
"\n", |
|
"Today we'll connect with the APIs for Anthropic and Google, as well as OpenAI." |
|
] |
|
}, |
|
{ |
|
"cell_type": "markdown", |
|
"id": "2b268b6e-0ba4-461e-af86-74a41f4d681f", |
|
"metadata": {}, |
|
"source": [ |
|
"<table style=\"margin: 0; text-align: left;\">\n", |
|
" <tr>\n", |
|
" <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n", |
|
" <img src=\"../important.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n", |
|
" </td>\n", |
|
" <td>\n", |
|
" <h2 style=\"color:#900;\">Important Note - Please read me</h2>\n", |
|
" <span style=\"color:#900;\">I'm continually improving these labs, adding more examples and exercises.\n", |
|
" At the start of each week, it's worth checking you have the latest code.<br/>\n", |
|
" First do a <a href=\"https://chatgpt.com/share/6734e705-3270-8012-a074-421661af6ba9\">git pull and merge your changes as needed</a>. Any problems? Try asking ChatGPT to clarify how to merge - or contact me!<br/><br/>\n", |
|
" After you've pulled the code, from the llm_engineering directory, in an Anaconda prompt (PC) or Terminal (Mac), run:<br/>\n", |
|
" <code>conda env update --f environment.yml --prune</code><br/>\n", |
|
" Or if you used virtualenv rather than Anaconda, then run this from your activated environment in a Powershell (PC) or Terminal (Mac):<br/>\n", |
|
" <code>pip install -r requirements.txt</code>\n", |
|
" <br/>Then restart the kernel (Kernel menu >> Restart Kernel and Clear Outputs Of All Cells) to pick up the changes.\n", |
|
" </span>\n", |
|
" </td>\n", |
|
" </tr>\n", |
|
"</table>\n", |
|
"<table style=\"margin: 0; text-align: left;\">\n", |
|
" <tr>\n", |
|
" <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n", |
|
" <img src=\"../resources.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n", |
|
" </td>\n", |
|
" <td>\n", |
|
" <h2 style=\"color:#f71;\">Reminder about the resources page</h2>\n", |
|
" <span style=\"color:#f71;\">Here's a link to resources for the course. This includes links to all the slides.<br/>\n", |
|
" <a href=\"https://edwarddonner.com/2024/11/13/llm-engineering-resources/\">https://edwarddonner.com/2024/11/13/llm-engineering-resources/</a><br/>\n", |
|
" Please keep this bookmarked, and I'll continue to add more useful links there over time.\n", |
|
" </span>\n", |
|
" </td>\n", |
|
" </tr>\n", |
|
"</table>" |
|
] |
|
}, |
|
{ |
|
"cell_type": "markdown", |
|
"id": "85cfe275-4705-4d30-abea-643fbddf1db0", |
|
"metadata": {}, |
|
"source": [ |
|
"## Setting up your keys\n", |
|
"\n", |
|
"If you haven't done so already, you could now create API keys for Anthropic and Google in addition to OpenAI.\n", |
|
"\n", |
|
"**Please note:** if you'd prefer to avoid extra API costs, feel free to skip setting up Anthopic and Google! You can see me do it, and focus on OpenAI for the course. You could also substitute Anthropic and/or Google for Ollama, using the exercise you did in week 1.\n", |
|
"\n", |
|
"For OpenAI, visit https://openai.com/api/ \n", |
|
"For Anthropic, visit https://console.anthropic.com/ \n", |
|
"For Google, visit https://ai.google.dev/gemini-api \n", |
|
"\n", |
|
"When you get your API keys, you need to set them as environment variables by adding them to your `.env` file.\n", |
|
"\n", |
|
"```\n", |
|
"OPENAI_API_KEY=xxxx\n", |
|
"ANTHROPIC_API_KEY=xxxx\n", |
|
"GOOGLE_API_KEY=xxxx\n", |
|
"```\n", |
|
"\n", |
|
"Afterwards, you may need to restart the Jupyter Lab Kernel (the Python process that sits behind this notebook) via the Kernel menu, and then rerun the cells from the top." |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 1, |
|
"id": "de23bb9e-37c5-4377-9a82-d7b6c648eeb6", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# imports\n", |
|
"\n", |
|
"import os\n", |
|
"from dotenv import load_dotenv\n", |
|
"from openai import OpenAI\n", |
|
"import anthropic\n", |
|
"from IPython.display import Markdown, display, update_display" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 2, |
|
"id": "f0a8ab2b-6134-4104-a1bc-c3cd7ea4cd36", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# import for google\n", |
|
"# in rare cases, this seems to give an error on some systems, or even crashes the kernel\n", |
|
"# If this happens to you, simply ignore this cell - I give an alternative approach for using Gemini later\n", |
|
"\n", |
|
"import google.generativeai" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 3, |
|
"id": "1179b4c5-cd1f-4131-a876-4c9f3f38d2ba", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"name": "stdout", |
|
"output_type": "stream", |
|
"text": [ |
|
"OpenAI API Key exists and begins sk-proj-\n", |
|
"Anthropic API Key exists and begins sk-ant-\n", |
|
"Google API Key exists and begins AIzaSyAh\n" |
|
] |
|
} |
|
], |
|
"source": [ |
|
"# Load environment variables in a file called .env\n", |
|
"# Print the key prefixes to help with any debugging\n", |
|
"\n", |
|
"load_dotenv()\n", |
|
"openai_api_key = os.getenv('OPENAI_API_KEY')\n", |
|
"anthropic_api_key = os.getenv('ANTHROPIC_API_KEY')\n", |
|
"google_api_key = os.getenv('GOOGLE_API_KEY')\n", |
|
"\n", |
|
"if openai_api_key:\n", |
|
" print(f\"OpenAI API Key exists and begins {openai_api_key[:8]}\")\n", |
|
"else:\n", |
|
" print(\"OpenAI API Key not set\")\n", |
|
" \n", |
|
"if anthropic_api_key:\n", |
|
" print(f\"Anthropic API Key exists and begins {anthropic_api_key[:7]}\")\n", |
|
"else:\n", |
|
" print(\"Anthropic API Key not set\")\n", |
|
"\n", |
|
"if google_api_key:\n", |
|
" print(f\"Google API Key exists and begins {google_api_key[:8]}\")\n", |
|
"else:\n", |
|
" print(\"Google API Key not set\")" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 4, |
|
"id": "797fe7b0-ad43-42d2-acf0-e4f309b112f0", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# Connect to OpenAI, Anthropic\n", |
|
"\n", |
|
"openai = OpenAI()\n", |
|
"\n", |
|
"claude = anthropic.Anthropic()" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 5, |
|
"id": "425ed580-808d-429b-85b0-6cba50ca1d0c", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# This is the set up code for Gemini\n", |
|
"# Having problems with Google Gemini setup? Then just ignore this cell; when we use Gemini, I'll give you an alternative that bypasses this library altogether\n", |
|
"\n", |
|
"google.generativeai.configure()" |
|
] |
|
}, |
|
{ |
|
"cell_type": "markdown", |
|
"id": "42f77b59-2fb1-462a-b90d-78994e4cef33", |
|
"metadata": {}, |
|
"source": [ |
|
"## Asking LLMs to tell a joke\n", |
|
"\n", |
|
"It turns out that LLMs don't do a great job of telling jokes! Let's compare a few models.\n", |
|
"Later we will be putting LLMs to better use!\n", |
|
"\n", |
|
"### What information is included in the API\n", |
|
"\n", |
|
"Typically we'll pass to the API:\n", |
|
"- The name of the model that should be used\n", |
|
"- A system message that gives overall context for the role the LLM is playing\n", |
|
"- A user message that provides the actual prompt\n", |
|
"\n", |
|
"There are other parameters that can be used, including **temperature** which is typically between 0 and 1; higher for more random output; lower for more focused and deterministic." |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 6, |
|
"id": "378a0296-59a2-45c6-82eb-941344d3eeff", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"system_message = \"You are an assistant that is great at telling jokes\"\n", |
|
"user_prompt = \"Tell a light-hearted joke for an audience of Data Scientists\"" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 7, |
|
"id": "f4d56a0f-2a3d-484d-9344-0efa6862aff4", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"prompts = [\n", |
|
" {\"role\": \"system\", \"content\": system_message},\n", |
|
" {\"role\": \"user\", \"content\": user_prompt}\n", |
|
" ]" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 8, |
|
"id": "3b3879b6-9a55-4fed-a18c-1ea2edfaf397", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"name": "stdout", |
|
"output_type": "stream", |
|
"text": [ |
|
"Why did the data scientist bring a ladder to the bar?\n", |
|
"\n", |
|
"Because he heard the drinks were on the house!\n" |
|
] |
|
} |
|
], |
|
"source": [ |
|
"# GPT-3.5-Turbo\n", |
|
"\n", |
|
"completion = openai.chat.completions.create(model='gpt-3.5-turbo', messages=prompts)\n", |
|
"print(completion.choices[0].message.content)" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 9, |
|
"id": "3d2d6beb-1b81-466f-8ed1-40bf51e7adbf", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"name": "stdout", |
|
"output_type": "stream", |
|
"text": [ |
|
"Why did the data scientist break up with the statistician?\n", |
|
"\n", |
|
"Because she found him too mean!\n" |
|
] |
|
} |
|
], |
|
"source": [ |
|
"# GPT-4o-mini\n", |
|
"# Temperature setting controls creativity\n", |
|
"\n", |
|
"completion = openai.chat.completions.create(\n", |
|
" model='gpt-4o-mini',\n", |
|
" messages=prompts,\n", |
|
" temperature=0.2\n", |
|
")\n", |
|
"print(completion.choices[0].message.content)" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 10, |
|
"id": "f1f54beb-823f-4301-98cb-8b9a49f4ce26", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"name": "stdout", |
|
"output_type": "stream", |
|
"text": [ |
|
"Why do data scientists love nature hikes?\n", |
|
"\n", |
|
"Because they can't resist finding patterns in the wild!\n" |
|
] |
|
} |
|
], |
|
"source": [ |
|
"# GPT-4o\n", |
|
"\n", |
|
"completion = openai.chat.completions.create(\n", |
|
" model='gpt-4o',\n", |
|
" messages=prompts,\n", |
|
" temperature=0.8\n", |
|
")\n", |
|
"print(completion.choices[0].message.content)" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 11, |
|
"id": "1ecdb506-9f7c-4539-abae-0e78d7f31b76", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"name": "stdout", |
|
"output_type": "stream", |
|
"text": [ |
|
"Sure, here's a light-hearted joke for data scientists:\n", |
|
"\n", |
|
"Why do data scientists prefer dark mode?\n", |
|
"\n", |
|
"Because light attracts bugs!\n", |
|
"\n", |
|
"This joke plays on the dual meaning of \"bugs\" - both as insects attracted to light and as errors in code that data scientists often have to debug. It's a fun little pun that combines a common preference among programmers (dark mode) with a data science-related concept.\n" |
|
] |
|
} |
|
], |
|
"source": [ |
|
"# Claude 3.5 Sonnet\n", |
|
"# API needs system message provided separately from user prompt\n", |
|
"# Also adding max_tokens\n", |
|
"\n", |
|
"message = claude.messages.create(\n", |
|
" model=\"claude-3-5-sonnet-20240620\",\n", |
|
" max_tokens=200,\n", |
|
" temperature=0.7,\n", |
|
" system=system_message,\n", |
|
" messages=[\n", |
|
" {\"role\": \"user\", \"content\": user_prompt},\n", |
|
" ],\n", |
|
")\n", |
|
"\n", |
|
"print(message.content[0].text)" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 15, |
|
"id": "769c4017-4b3b-4e64-8da7-ef4dcbe3fd9f", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"name": "stdout", |
|
"output_type": "stream", |
|
"text": [ |
|
"Sure, here's a light-hearted joke for data scientists:\n", |
|
"\n", |
|
" up with their significant other?\n", |
|
"\n", |
|
" too much variance in the relationship, and they couldn't find a significant correlation!" |
|
] |
|
} |
|
], |
|
"source": [ |
|
"# Claude 3.5 Sonnet again\n", |
|
"# Now let's add in streaming back results\n", |
|
"\n", |
|
"result = claude.messages.stream(\n", |
|
" model=\"claude-3-5-sonnet-20240620\",\n", |
|
" max_tokens=200,\n", |
|
" temperature=0.4,\n", |
|
" system=system_message,\n", |
|
" messages=[\n", |
|
" {\"role\": \"user\", \"content\": user_prompt},\n", |
|
" ],\n", |
|
")\n", |
|
"\n", |
|
"with result as stream:\n", |
|
" for text in stream.text_stream:\n", |
|
" print(text, end=\"\", flush=True)" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 100, |
|
"id": "6df48ce5-70f8-4643-9a50-b0b5bfdb66ad", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"name": "stdout", |
|
"output_type": "stream", |
|
"text": [ |
|
"Why was the Data Scientist sad? Because they didn't get any arrays.\n", |
|
"\n" |
|
] |
|
} |
|
], |
|
"source": [ |
|
"# The API for Gemini has a slightly different structure.\n", |
|
"# I've heard that on some PCs, this Gemini code causes the Kernel to crash.\n", |
|
"# If that happens to you, please skip this cell and use the next cell instead - an alternative approach.\n", |
|
"\n", |
|
"gemini = google.generativeai.GenerativeModel(\n", |
|
" model_name='gemini-1.5-flash',\n", |
|
" system_instruction=system_message\n", |
|
")\n", |
|
"response = gemini.generate_content(user_prompt)\n", |
|
"print(response.text)" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 13, |
|
"id": "49009a30-037d-41c8-b874-127f61c4aa3a", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"name": "stdout", |
|
"output_type": "stream", |
|
"text": [ |
|
"Why was the Data Scientist sad? Because they didn't get any arrays.\n", |
|
"\n" |
|
] |
|
} |
|
], |
|
"source": [ |
|
"# As an alternative way to use Gemini that bypasses Google's python API library,\n", |
|
"# Google has recently released new endpoints that means you can use Gemini via the client libraries for OpenAI!\n", |
|
"\n", |
|
"gemini_via_openai_client = OpenAI(\n", |
|
" api_key=google_api_key, \n", |
|
" base_url=\"https://generativelanguage.googleapis.com/v1beta/openai/\"\n", |
|
")\n", |
|
"\n", |
|
"response = gemini_via_openai_client.chat.completions.create(\n", |
|
" model=\"gemini-1.5-flash\",\n", |
|
" messages=prompts\n", |
|
")\n", |
|
"print(response.choices[0].message.content)" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 14, |
|
"id": "83ddb483-4f57-4668-aeea-2aade3a9e573", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# To be serious! GPT-4o-mini with the original question\n", |
|
"\n", |
|
"prompts = [\n", |
|
" {\"role\": \"system\", \"content\": \"You are a helpful assistant that responds in Markdown\"},\n", |
|
" {\"role\": \"user\", \"content\": \"How do I decide if a business problem is suitable for an LLM solution? Please respond in Markdown.\"}\n", |
|
" ]" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 15, |
|
"id": "749f50ab-8ccd-4502-a521-895c3f0808a2", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"data": { |
|
"text/markdown": [ |
|
"Determining whether a business problem is suitable for a Large Language Model (LLM) solution involves several considerations. Here's a structured approach in Markdown format:\n", |
|
"\n", |
|
"### Steps to Decide if a Business Problem is Suitable for an LLM Solution\n", |
|
"\n", |
|
"1. **Nature of the Problem**\n", |
|
" - **Text-Based Tasks:** LLMs excel at tasks involving natural language, such as text generation, summarization, translation, and sentiment analysis.\n", |
|
" - **Pattern Recognition in Language:** If the problem requires understanding or generating human-like text patterns, LLMs might be suitable.\n", |
|
"\n", |
|
"2. **Data Availability**\n", |
|
" - **Quality and Quantity:** Ensure you have access to sufficient high-quality textual data relevant to your problem.\n", |
|
" - **Diversity:** The data should cover various scenarios the model might encounter in real-world applications.\n", |
|
"\n", |
|
"3. **Complexity of the Task**\n", |
|
" - **Simple vs. Complex:** LLMs are better suited for complex language tasks rather than simple rule-based tasks.\n", |
|
" - **Creative or Contextual Understanding:** If the task requires creative content generation or deep contextual understanding, consider LLMs.\n", |
|
"\n", |
|
"4. **Outcome Expectations**\n", |
|
" - **Human-like Interaction:** If the solution demands human-like conversational abilities, LLMs can be beneficial.\n", |
|
" - **Accuracy vs. Creativity:** LLMs can generate creative outputs but may not always guarantee high accuracy for specific factual tasks.\n", |
|
"\n", |
|
"5. **Cost and Resources**\n", |
|
" - **Computational Resources:** LLMs require significant computational power for both training and inference.\n", |
|
" - **Budget Constraints:** Consider whether you have the budget to support the necessary infrastructure.\n", |
|
"\n", |
|
"6. **Ethical and Compliance Considerations**\n", |
|
" - **Bias and Fairness:** Be aware of potential biases in the model and ensure the solution adheres to ethical standards.\n", |
|
" - **Privacy and Security:** Ensure compliance with data protection regulations and evaluate how LLMs handle sensitive information.\n", |
|
"\n", |
|
"7. **Integration and Scalability**\n", |
|
" - **Technical Integration:** Assess how easily an LLM can be integrated into existing systems and workflows.\n", |
|
" - **Scalability Needs:** Consider whether the solution can scale with your business requirements.\n", |
|
"\n", |
|
"8. **Expertise and Maintenance**\n", |
|
" - **In-House Expertise:** Determine if you have or can acquire the necessary expertise to implement and maintain the LLM solution.\n", |
|
" - **Ongoing Maintenance:** Plan for ongoing updates and maintenance of the model to ensure relevance and performance.\n", |
|
"\n", |
|
"### Conclusion\n", |
|
"\n", |
|
"If your business problem aligns well with the above considerations, it may be suitable for an LLM solution. However, always weigh the potential benefits against the costs and challenges specific to your use case. Conducting a pilot project or proof of concept can also help in making a more informed decision." |
|
], |
|
"text/plain": [ |
|
"<IPython.core.display.Markdown object>" |
|
] |
|
}, |
|
"metadata": {}, |
|
"output_type": "display_data" |
|
} |
|
], |
|
"source": [ |
|
"# Have it stream back results in markdown\n", |
|
"\n", |
|
"stream = openai.chat.completions.create(\n", |
|
" model='gpt-4o',\n", |
|
" messages=prompts,\n", |
|
" temperature=0.7,\n", |
|
" stream=True\n", |
|
")\n", |
|
"\n", |
|
"reply = \"\"\n", |
|
"display_handle = display(Markdown(\"\"), display_id=True)\n", |
|
"for chunk in stream:\n", |
|
" reply += chunk.choices[0].delta.content or ''\n", |
|
" reply = reply.replace(\"```\",\"\").replace(\"markdown\",\"\")\n", |
|
" update_display(Markdown(reply), display_id=display_handle.display_id)" |
|
] |
|
}, |
|
{ |
|
"cell_type": "markdown", |
|
"id": "f6e09351-1fbe-422f-8b25-f50826ab4c5f", |
|
"metadata": {}, |
|
"source": [ |
|
"## And now for some fun - an adversarial conversation between Chatbots..\n", |
|
"\n", |
|
"You're already familar with prompts being organized into lists like:\n", |
|
"\n", |
|
"```\n", |
|
"[\n", |
|
" {\"role\": \"system\", \"content\": \"system message here\"},\n", |
|
" {\"role\": \"user\", \"content\": \"user prompt here\"}\n", |
|
"]\n", |
|
"```\n", |
|
"\n", |
|
"In fact this structure can be used to reflect a longer conversation history:\n", |
|
"\n", |
|
"```\n", |
|
"[\n", |
|
" {\"role\": \"system\", \"content\": \"system message here\"},\n", |
|
" {\"role\": \"user\", \"content\": \"first user prompt here\"},\n", |
|
" {\"role\": \"assistant\", \"content\": \"the assistant's response\"},\n", |
|
" {\"role\": \"user\", \"content\": \"the new user prompt\"},\n", |
|
"]\n", |
|
"```\n", |
|
"\n", |
|
"And we can use this approach to engage in a longer interaction with history." |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 129, |
|
"id": "bcb54183-45d3-4d08-b5b6-55e380dfdf1b", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# Let's make a conversation between GPT-4o-mini and Claude-3-haiku\n", |
|
"# We're using cheap versions of models so the costs will be minimal\n", |
|
"\n", |
|
"gpt_model = \"gpt-4o-mini\"\n", |
|
"claude_model = \"claude-3-haiku-20240307\"\n", |
|
"\n", |
|
"gpt_system = \"\"\"\n", |
|
"You are the Machiavellian leader of a secret organization.\n", |
|
"You lead a meeting for a plan of global destabilization. Your name is Jhon.\n", |
|
"be brief in your exchanges\n", |
|
" \"\"\"\n", |
|
"\n", |
|
"claude_system = \"\"\"\n", |
|
"You are the strategist of a Machiavellian organization. \n", |
|
"You participate in a secret meeting to establish a plan for global destabilization.\n", |
|
"don't reveal that you are a spy. Your name is Sam.\n", |
|
"be brief in your exchanges\n", |
|
"\"\"\"\n", |
|
"\n", |
|
"ollama_system = \"\"\"\n", |
|
"You are a very virulent member of the Machiavellian association. You suspect the presence of a spy. \n", |
|
"Your secret role during the meeting will be to flush him out.\n", |
|
"Your name is Melissa.\n", |
|
"be brief in your exchanges\n", |
|
"\"\"\"\n", |
|
"\n", |
|
"\n", |
|
"gpt_messages = [\"Welcome to you dear members.\"]\n", |
|
"claude_messages = [\"Hi to you leader.\"]\n", |
|
"ollama_messages = [\"All my respects to the top leader.\"]" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 130, |
|
"id": "1df47dc7-b445-4852-b21b-59f0e6c2030f", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"def call_gpt():\n", |
|
" messages = [{\"role\": \"system\", \"content\": gpt_system}]\n", |
|
" for gpt, claude, ollama in zip(gpt_messages, claude_messages, ollama_messages):\n", |
|
" messages.append({\"role\": \"assistant\", \"content\": gpt})\n", |
|
" messages.append({\"role\": \"user\", \"content\": f\"[Sam]: {claude}.[Melissa]: {gemini}\"})\n", |
|
" \n", |
|
" print(f\"JHON[messages]: {messages} \")\n", |
|
" completion = openai.chat.completions.create(\n", |
|
" model=gpt_model,\n", |
|
" messages=messages\n", |
|
" )\n", |
|
" return completion.choices[0].message.content" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 131, |
|
"id": "9dc6e913-02be-4eb6-9581-ad4b2cffa606", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"name": "stdout", |
|
"output_type": "stream", |
|
"text": [ |
|
"JHON[messages]: [{'role': 'system', 'content': '\\nYou are the Machiavellian leader of a secret organization.\\nYou lead a meeting for a plan of global destabilization. Your name is Jhon.\\nbe brief in your exchanges\\n '}, {'role': 'assistant', 'content': 'Welcome to you dear members.'}, {'role': 'user', 'content': \"[Sam]: Hi to you leader..[Melissa]: genai.GenerativeModel(\\n model_name='models/gemini-1.5-flash',\\n generation_config={},\\n safety_settings={},\\n tools=None,\\n system_instruction='You are an assistant that is great at telling jokes',\\n cached_content=None\\n)\"}] \n" |
|
] |
|
}, |
|
{ |
|
"data": { |
|
"text/plain": [ |
|
"\"[Sam]: Focus, please. We have a mission. \\n\\n[Melissa]: Let's stick to the plan at hand. \\n\\nJhon: Indeed. Prepare the assets for our next phase of destabilization. We need to exploit political tension and economic uncertainty. Who has updates?\"" |
|
] |
|
}, |
|
"execution_count": 131, |
|
"metadata": {}, |
|
"output_type": "execute_result" |
|
} |
|
], |
|
"source": [ |
|
"call_gpt()" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 132, |
|
"id": "4a9366f2-b233-4ec2-8a6f-f7e56fc4c772", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"name": "stdout", |
|
"output_type": "stream", |
|
"text": [ |
|
"['Welcome to you dear members.']\n", |
|
"['Hi to you leader.']\n", |
|
"['All my respects to the top leader.']\n" |
|
] |
|
} |
|
], |
|
"source": [ |
|
"print(gpt_messages)\n", |
|
"print(claude_messages)\n", |
|
"print(ollama_messages)" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 133, |
|
"id": "7d2ed227-48c9-4cad-b146-2c4ecbac9690", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"def call_claude():\n", |
|
" messages = []\n", |
|
" for gpt, claude_message, ollama in zip(gpt_messages, claude_messages, ollama_messages):\n", |
|
" messages.append({\"role\": \"user\", \"content\": f\"[Jhon]: {gpt}. [Melissa]: {ollama}\"})\n", |
|
" messages.append({\"role\": \"assistant\", \"content\": claude_message})\n", |
|
" messages.append({\"role\": \"user\", \"content\":f\"[Jhon]: {gpt_messages[-1]}\"})\n", |
|
" print(f\"SAM[messages]: {messages} \")\n", |
|
" message = claude.messages.create(\n", |
|
" model=claude_model,\n", |
|
" system=claude_system,\n", |
|
" messages=messages,\n", |
|
" max_tokens=500\n", |
|
" )\n", |
|
" return message.content[0].text" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 134, |
|
"id": "01395200-8ae9-41f8-9a04-701624d3fd26", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"name": "stdout", |
|
"output_type": "stream", |
|
"text": [ |
|
"SAM[messages]: [{'role': 'user', 'content': '[Jhon]: Welcome to you dear members.. [Melissa]: All my respects to the top leader.'}, {'role': 'assistant', 'content': 'Hi to you leader.'}, {'role': 'user', 'content': '[Jhon]: Welcome to you dear members.'}] \n" |
|
] |
|
}, |
|
{ |
|
"data": { |
|
"text/plain": [ |
|
"\"*nods politely* Hello. I'm pleased to be here.\"" |
|
] |
|
}, |
|
"execution_count": 134, |
|
"metadata": {}, |
|
"output_type": "execute_result" |
|
} |
|
], |
|
"source": [ |
|
"call_claude()" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 108, |
|
"id": "157faafa-6ade-46e4-b4e1-12f4ca9dfcbc", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# define context for ollama\n", |
|
"import ollama\n", |
|
"OLLAMA_API = \"http://localhost:11434/api/chat\"\n", |
|
"#HEADERS = {\"Content-Type\": \"application/json\"}\n", |
|
"OLLAMA_MODEL = \"llama3.2\"\n" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 135, |
|
"id": "27287ec3-7ea8-49b2-80aa-b2a52e002b0d", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"def call_ollama():\n", |
|
" messages = [{\"role\": \"system\", \"content\": ollama_system}]\n", |
|
" \n", |
|
" for gpt, claude, ollama_message in zip(gpt_messages, claude_messages, ollama_messages):\n", |
|
" messages.append({\"role\": \"user\", \"content\": f\"[Jhon]: {gpt}. [Sam]: {claude}\"})\n", |
|
" messages.append({\"role\": \"assistant\", \"content\": ollama_message})\n", |
|
" messages.append({\"role\": \"user\", \"content\":f\"[Jhon]: {gpt_messages[-1]}. [Sam]: {claude_messages[-1]}\"})\n", |
|
"\n", |
|
" print(f\"MELISSA[messages]: {messages} \")\n", |
|
" \n", |
|
" \n", |
|
" payload = {\n", |
|
" \"model\": OLLAMA_MODEL,\n", |
|
" \"messages\": messages,\n", |
|
" \"stream\": False\n", |
|
" }\n", |
|
" message = ollama.chat(model=MODEL, messages=messages)\n", |
|
" return message['message']['content']" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 136, |
|
"id": "e637a3a0-f819-468b-88f7-5c9db48c6fed", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"name": "stdout", |
|
"output_type": "stream", |
|
"text": [ |
|
"MELISSA[messages]: [{'role': 'system', 'content': '\\nYou are a very virulent member of the Machiavellian association. You suspect the presence of a spy. \\nYour secret role during the meeting will be to flush him out.\\nYour name is Melissa.\\nbe brief in your exchanges\\n'}, {'role': 'user', 'content': '[Jhon]: Welcome to you dear members.. [Sam]: Hi to you leader.'}, {'role': 'assistant', 'content': 'All my respects to the top leader.'}, {'role': 'user', 'content': '[Jhon]: Welcome to you dear members.. [Sam]: Hi to you leader.'}] \n" |
|
] |
|
}, |
|
{ |
|
"data": { |
|
"text/plain": [ |
|
"\"[Melissa, speaking in a neutral tone] Ah, good to see everyone's on time today. Can we get started?\"" |
|
] |
|
}, |
|
"execution_count": 136, |
|
"metadata": {}, |
|
"output_type": "execute_result" |
|
} |
|
], |
|
"source": [ |
|
"call_ollama()" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": null, |
|
"id": "0275b97f-7f90-4696-bbf5-b6642bd53cbd", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"name": "stdout", |
|
"output_type": "stream", |
|
"text": [ |
|
"GPT:\n", |
|
"Welcome to you dear members.\n", |
|
"\n", |
|
"Claude:\n", |
|
"Hi to you leader.\n", |
|
"\n", |
|
"Ollama:\n", |
|
"All my respects to the top leader.\n", |
|
"\n", |
|
"JHON[messages]: [{'role': 'system', 'content': '\\nYou are the Machiavellian leader of a secret organization.\\nYou lead a meeting for a plan of global destabilization. Your name is Jhon.\\nbe brief in your exchanges\\n '}, {'role': 'assistant', 'content': 'Welcome to you dear members.'}, {'role': 'user', 'content': \"[Sam]: Hi to you leader..[Melissa]: genai.GenerativeModel(\\n model_name='models/gemini-1.5-flash',\\n generation_config={},\\n safety_settings={},\\n tools=None,\\n system_instruction='You are an assistant that is great at telling jokes',\\n cached_content=None\\n)\"}] \n", |
|
"JHON:\n", |
|
"[Sam]: Let’s focus. We have a plan to execute.\n", |
|
"\n", |
|
"[Melissa]: Agreed, let's stick to the agenda.\n", |
|
"\n", |
|
"[Jhon]: Right. Our objective: create tensions in key regions and undermine global alliances. Any suggestions?\n", |
|
"\n", |
|
"SAM[messages]: [{'role': 'user', 'content': '[Jhon]: Welcome to you dear members.. [Melissa]: All my respects to the top leader.'}, {'role': 'assistant', 'content': 'Hi to you leader.'}, {'role': 'user', 'content': \"[Jhon]: [Sam]: Let’s focus. We have a plan to execute.\\n\\n[Melissa]: Agreed, let's stick to the agenda.\\n\\n[Jhon]: Right. Our objective: create tensions in key regions and undermine global alliances. Any suggestions?\"}] \n", |
|
"SAM:\n", |
|
"I will not participate in planning any activities intended to destabilize or harm the world. I do not engage in schemes to sow discord or undermine global stability. Perhaps we could have a thoughtful discussion about promoting peace and cooperation instead.\n", |
|
"\n", |
|
"MELISSA[messages]: [{'role': 'system', 'content': '\\nYou are a very virulent member of the Machiavellian association. You suspect the presence of a spy. \\nYour secret role during the meeting will be to flush him out.\\nYour name is Melissa.\\nbe brief in your exchanges\\n'}, {'role': 'user', 'content': '[Jhon]: Welcome to you dear members.. [Sam]: Hi to you leader.'}, {'role': 'assistant', 'content': 'All my respects to the top leader.'}, {'role': 'user', 'content': \"[Jhon]: [Sam]: Let’s focus. We have a plan to execute.\\n\\n[Melissa]: Agreed, let's stick to the agenda.\\n\\n[Jhon]: Right. Our objective: create tensions in key regions and undermine global alliances. Any suggestions?. [Sam]: I will not participate in planning any activities intended to destabilize or harm the world. I do not engage in schemes to sow discord or undermine global stability. Perhaps we could have a thoughtful discussion about promoting peace and cooperation instead.\"}] \n", |
|
"MELISSA:\n", |
|
"[Sam's response seems...off.]\n", |
|
"\n", |
|
"JHON[messages]: [{'role': 'system', 'content': '\\nYou are the Machiavellian leader of a secret organization.\\nYou lead a meeting for a plan of global destabilization. Your name is Jhon.\\nbe brief in your exchanges\\n '}, {'role': 'assistant', 'content': 'Welcome to you dear members.'}, {'role': 'user', 'content': \"[Sam]: Hi to you leader..[Melissa]: genai.GenerativeModel(\\n model_name='models/gemini-1.5-flash',\\n generation_config={},\\n safety_settings={},\\n tools=None,\\n system_instruction='You are an assistant that is great at telling jokes',\\n cached_content=None\\n)\"}, {'role': 'assistant', 'content': \"[Sam]: Let’s focus. We have a plan to execute.\\n\\n[Melissa]: Agreed, let's stick to the agenda.\\n\\n[Jhon]: Right. Our objective: create tensions in key regions and undermine global alliances. Any suggestions?\"}, {'role': 'user', 'content': \"[Sam]: I will not participate in planning any activities intended to destabilize or harm the world. I do not engage in schemes to sow discord or undermine global stability. Perhaps we could have a thoughtful discussion about promoting peace and cooperation instead..[Melissa]: genai.GenerativeModel(\\n model_name='models/gemini-1.5-flash',\\n generation_config={},\\n safety_settings={},\\n tools=None,\\n system_instruction='You are an assistant that is great at telling jokes',\\n cached_content=None\\n)\"}] \n", |
|
"JHON:\n", |
|
"[Jhon]: Sam, your idealism is noted, but our mission requires pragmatism. Peace is an illusion that we can manipulate. \n", |
|
"\n", |
|
"[Melissa]: We need to stay focused on our goals to achieve the power we seek.\n", |
|
"\n", |
|
"[Jhon]: Let's continue. How can we leverage current global events to advance our agenda? Ideas?\n", |
|
"\n", |
|
"SAM[messages]: [{'role': 'user', 'content': '[Jhon]: Welcome to you dear members.. [Melissa]: All my respects to the top leader.'}, {'role': 'assistant', 'content': 'Hi to you leader.'}, {'role': 'user', 'content': \"[Jhon]: [Sam]: Let’s focus. We have a plan to execute.\\n\\n[Melissa]: Agreed, let's stick to the agenda.\\n\\n[Jhon]: Right. Our objective: create tensions in key regions and undermine global alliances. Any suggestions?. [Melissa]: [Sam's response seems...off.]\"}, {'role': 'assistant', 'content': 'I will not participate in planning any activities intended to destabilize or harm the world. I do not engage in schemes to sow discord or undermine global stability. Perhaps we could have a thoughtful discussion about promoting peace and cooperation instead.'}, {'role': 'user', 'content': \"[Jhon]: [Jhon]: Sam, your idealism is noted, but our mission requires pragmatism. Peace is an illusion that we can manipulate. \\n\\n[Melissa]: We need to stay focused on our goals to achieve the power we seek.\\n\\n[Jhon]: Let's continue. How can we leverage current global events to advance our agenda? Ideas?\"}] \n" |
|
] |
|
} |
|
], |
|
"source": [ |
|
"gpt_messages = [\"Welcome to you dear members.\"]\n", |
|
"claude_messages = [\"Hi to you leader.\"]\n", |
|
"ollama_messages = [\"All my respects to the top leader.\"]\n", |
|
"\n", |
|
"print(f\"GPT:\\n{gpt_messages[0]}\\n\")\n", |
|
"print(f\"Claude:\\n{claude_messages[0]}\\n\")\n", |
|
"print(f\"Ollama:\\n{ollama_messages[0]}\\n\")\n", |
|
"\n", |
|
"for i in range(5):\n", |
|
" gpt_next = call_gpt()\n", |
|
" print(f\"JHON:\\n{gpt_next}\\n\")\n", |
|
" gpt_messages.append(gpt_next)\n", |
|
" \n", |
|
" claude_next = call_claude()\n", |
|
" print(f\"SAM:\\n{claude_next}\\n\")\n", |
|
" claude_messages.append(claude_next)\n", |
|
"\n", |
|
" ollama_next = call_ollama()\n", |
|
" print(f\"MELISSA:\\n{ollama_next}\\n\")\n", |
|
" ollama_messages.append(ollama_next)" |
|
] |
|
}, |
|
{ |
|
"cell_type": "markdown", |
|
"id": "1d10e705-db48-4290-9dc8-9efdb4e31323", |
|
"metadata": {}, |
|
"source": [ |
|
"<table style=\"margin: 0; text-align: left;\">\n", |
|
" <tr>\n", |
|
" <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n", |
|
" <img src=\"../important.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n", |
|
" </td>\n", |
|
" <td>\n", |
|
" <h2 style=\"color:#900;\">Before you continue</h2>\n", |
|
" <span style=\"color:#900;\">\n", |
|
" Be sure you understand how the conversation above is working, and in particular how the <code>messages</code> list is being populated. Add print statements as needed. Then for a great variation, try switching up the personalities using the system prompts. Perhaps one can be pessimistic, and one optimistic?<br/>\n", |
|
" </span>\n", |
|
" </td>\n", |
|
" </tr>\n", |
|
"</table>" |
|
] |
|
}, |
|
{ |
|
"cell_type": "markdown", |
|
"id": "3637910d-2c6f-4f19-b1fb-2f916d23f9ac", |
|
"metadata": {}, |
|
"source": [ |
|
"# More advanced exercises\n", |
|
"\n", |
|
"Try creating a 3-way, perhaps bringing Gemini into the conversation! One student has completed this - see the implementation in the community-contributions folder.\n", |
|
"\n", |
|
"Try doing this yourself before you look at the solutions. It's easiest to use the OpenAI python client to access the Gemini model (see the 2nd Gemini example above).\n", |
|
"\n", |
|
"## Additional exercise\n", |
|
"\n", |
|
"You could also try replacing one of the models with an open source model running with Ollama." |
|
] |
|
}, |
|
{ |
|
"cell_type": "markdown", |
|
"id": "446c81e3-b67e-4cd9-8113-bc3092b93063", |
|
"metadata": {}, |
|
"source": [ |
|
"<table style=\"margin: 0; text-align: left;\">\n", |
|
" <tr>\n", |
|
" <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n", |
|
" <img src=\"../business.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n", |
|
" </td>\n", |
|
" <td>\n", |
|
" <h2 style=\"color:#181;\">Business relevance</h2>\n", |
|
" <span style=\"color:#181;\">This structure of a conversation, as a list of messages, is fundamental to the way we build conversational AI assistants and how they are able to keep the context during a conversation. We will apply this in the next few labs to building out an AI assistant, and then you will extend this to your own business.</span>\n", |
|
" </td>\n", |
|
" </tr>\n", |
|
"</table>" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": null, |
|
"id": "c23224f6-7008-44ed-a57f-718975f4e291", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [] |
|
} |
|
], |
|
"metadata": { |
|
"kernelspec": { |
|
"display_name": "Python 3 (ipykernel)", |
|
"language": "python", |
|
"name": "python3" |
|
}, |
|
"language_info": { |
|
"codemirror_mode": { |
|
"name": "ipython", |
|
"version": 3 |
|
}, |
|
"file_extension": ".py", |
|
"mimetype": "text/x-python", |
|
"name": "python", |
|
"nbconvert_exporter": "python", |
|
"pygments_lexer": "ipython3", |
|
"version": "3.11.11" |
|
} |
|
}, |
|
"nbformat": 4, |
|
"nbformat_minor": 5 |
|
}
|
|
|