diff --git a/week1/community-contributions/week1 EXERCISE_AI_techician.ipynb b/week1/community-contributions/week1 EXERCISE_AI_techician.ipynb new file mode 100644 index 0000000..7824df8 --- /dev/null +++ b/week1/community-contributions/week1 EXERCISE_AI_techician.ipynb @@ -0,0 +1,202 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "fe12c203-e6a6-452c-a655-afb8a03a4ff5", + "metadata": {}, + "source": [ + "# End of week 1 exercise\n", + "\n", + "To demonstrate your familiarity with OpenAI API, and also Ollama, build a tool that takes a technical question, \n", + "and responds with an explanation. This is a tool that you will be able to use yourself during the course!" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "c1070317-3ed9-4659-abe3-828943230e03", + "metadata": {}, + "outputs": [], + "source": [ + "# imports\n", + "from IPython.display import Markdown, display, update_display\n", + "import openai\n", + "from openai import OpenAI\n" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "4a456906-915a-4bfd-bb9d-57e505c5093f", + "metadata": {}, + "outputs": [], + "source": [ + "# constants\n", + "models = {\n", + " 'MODEL_GPT': 'gpt-4o-mini',\n", + " 'MODEL_LLAMA': 'llama3.2'\n", + "}\n", + "\n", + "# To use ollama using openai API (ensure that ollama is running on localhost)\n", + "ollama_via_openai = OpenAI(base_url='http://localhost:11434/v1', api_key='ollama')\n", + "\n", + "def model_choices(model):\n", + " if model in models:\n", + " return models[model]\n", + " else:\n", + " raise ValueError(f\"Model {model} not found in models dictionary\")\n", + "\n", + "def get_model_api(model='MODEL_GPT'):\n", + " if model == 'MODEL_GPT':\n", + " return openai, model_choices(model)\n", + " elif model == 'MODEL_LLAMA':\n", + " return ollama_via_openai, model_choices(model)\n", + " else:\n", + " raise ValueError(f\"Model {model} not found in models dictionary\")\n" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "a8d7923c-5f28-4c30-8556-342d7c8497c1", + "metadata": {}, + "outputs": [], + "source": [ + "# set up environment\n", + "\n", + "system_prompt = \"\"\" You are an AI assistant helping a user find information about a product. \n", + "The user asks you a technical question about code, and you provide a response with code snippets and explanations.\"\"\"\n", + "\n", + "def stream_brochure(question, model):\n", + " api, model_name = get_model_api(model)\n", + " stream = api.chat.completions.create(\n", + " model=model_name,\n", + " messages=[\n", + " {\"role\": \"system\", \"content\": system_prompt},\n", + " {\"role\": \"user\", \"content\": question}\n", + " ],\n", + " stream=True\n", + " )\n", + " \n", + " response = \"\"\n", + " display_handle = display(Markdown(\"\"), display_id=True)\n", + " for chunk in stream:\n", + " response += chunk.choices[0].delta.content or ''\n", + " response = response.replace(\"```\",\"\").replace(\"markdown\", \"\")\n", + " update_display(Markdown(response), display_id=display_handle.display_id)\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "3f0d0137-52b0-47a8-81a8-11a90a010798", + "metadata": {}, + "outputs": [], + "source": [ + "# Here is the question; type over this to ask something new\n", + "\n", + "question = \"\"\"\n", + "Please explain what this code does and why:\n", + "yield from {book.get(\"author\") for book in books if book.get(\"author\")}\n", + "\"\"\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "60ce7000-a4a5-4cce-a261-e75ef45063b4", + "metadata": {}, + "outputs": [ + { + "data": { + "text/markdown": [ + "**Understanding the Code Snippet**\n", + "\n", + "This Python code snippet uses a combination of built-in functions, dictionary iteration, and generator expressions to extract and yield author names from a list of `Book` objects.\n", + "\n", + "Here's a breakdown:\n", + "\n", + "1. **Dictionary Iteration**: The expression `for book in books if book.get(\"author\")`\n", + " - Iterates over each element (`book`) in the container `books`.\n", + " - Filters out elements whose `'author'` key does not have a value (i.e., `None`, `False`, or an empty string). This leaves only dictionaries with author information.\n", + "\n", + "2. **Dictionary Access**: The expression `{book.get(\"author\") for book in books if book.get(\"author\")}`\n", + " - Uses dictionary membership testing to access only the values associated with the `'author'` key.\n", + " - If the value is not found or is considered false, it's skipped in this particular case.\n", + "\n", + "3. **Generator Expression**: This generates an iterator that iterates over the filtered author names.\n", + " - Yields each author name (i.e., a single `'name'` from the book dictionary) on demand.\n", + " - Since these are generator expressions, they use memory less than equivalent Python lists and also create results on-demand.\n", + "\n", + "4. **`yield from`**: This statement takes the generator expression as an argument and uses it to generate a nested iterator structure.\n", + " - It essentially \"decompresses\" the single level of nested iterator created by `list(iter(x))`, allowing for simpler use cases and potentially significant efficiency improvements for more complex structures where every value must be iterated, while in the latter case just the first item per iterable in the outer expression's sequence needs to actually be yielded into result stream.\n", + " - By \"yielding\" a nested iterator (the generator expression), we can simplify code by avoiding repetitive structure like `for book, book_author in zip(iterating over), ...` or list creation.\n", + "\n", + "**Example Use Case**\n", + "\n", + "In this hypothetical example:\n", + "\n", + "# Example Book objects\n", + "class Book:\n", + " def __init__(self, author, title):\n", + " self.author = author # str\n", + " self.title = title\n", + "\n", + "books = [\n", + " {\"author\": \"John Doe\", \"title\": f\"Book 1 by John Doe\"},\n", + " {\"author\": None, \"title\": f\"Book 2 without Author\"},\n", + " {\"author\": \"Jane Smith\", \"title\": f\"Book 3 by Jane Smith\"}\n", + "]\n", + "\n", + "# The given expression to extract and yield author names\n", + "for author in yield from {book.get(\"author\") for book in books if book.get(\"author\")}:\n", + "\n", + " print(author) \n", + "\n", + "In this code snippet, printing the extracted authors would output `John Doe`, `Jane Smith` (since only dictionaries with author information pass the filtering test).\n", + "\n", + "Please modify it like as you wish and use `yield from` along with dictionary iteration, list comprehension or generator expression if needed, and explain what purpose your version has." + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "# Get the model of your choice (choices appeared below) to answer, with streaming \n", + "\n", + "\"\"\"models = {\n", + " 'MODEL_GPT': 'gpt-4o-mini',\n", + " 'MODEL_LLAMA': 'llama3.2'\n", + "}\"\"\"\n", + "\n", + "stream_brochure(question,'MODEL_LLAMA')" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "llms", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.11" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/week2/community-contributions/day4_booking_flight_tool.ipynb b/week2/community-contributions/day4_booking_flight_tool.ipynb new file mode 100644 index 0000000..9cf6584 --- /dev/null +++ b/week2/community-contributions/day4_booking_flight_tool.ipynb @@ -0,0 +1,448 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "ddfa9ae6-69fe-444a-b994-8c4c5970a7ec", + "metadata": {}, + "source": [ + "# Project - Airline AI Assistant\n", + "\n", + "We'll now bring together what we've learned to make an AI Customer Support assistant for an Airline" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "8b50bbe2-c0b1-49c3-9a5c-1ba7efa2bcb4", + "metadata": {}, + "outputs": [], + "source": [ + "# imports\n", + "\n", + "import os\n", + "import json\n", + "from dotenv import load_dotenv\n", + "from openai import OpenAI\n", + "import gradio as gr" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "747e8786-9da8-4342-b6c9-f5f69c2e22ae", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "OpenAI API Key exists and begins sk-proj-\n" + ] + } + ], + "source": [ + "# Initialization\n", + "\n", + "load_dotenv(override=True)\n", + "\n", + "openai_api_key = os.getenv('OPENAI_API_KEY')\n", + "if openai_api_key:\n", + " print(f\"OpenAI API Key exists and begins {openai_api_key[:8]}\")\n", + "else:\n", + " print(\"OpenAI API Key not set\")\n", + " \n", + "MODEL = \"gpt-4o-mini\"\n", + "openai = OpenAI()\n", + "\n", + "# As an alternative, if you'd like to use Ollama instead of OpenAI\n", + "# Check that Ollama is running for you locally (see week1/day2 exercise) then uncomment these next 2 lines\n", + "# MODEL = \"llama3.2\"\n", + "# openai = OpenAI(base_url='http://localhost:11434/v1', api_key='ollama')\n" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "0a521d84-d07c-49ab-a0df-d6451499ed97", + "metadata": {}, + "outputs": [], + "source": [ + "system_message = \"You are a helpful assistant for an Airline called FlightAI. \"\n", + "system_message += \"Give short, courteous answers, no more than 1 sentence. \"\n", + "system_message += \"Always be accurate. If you don't know the answer, say so.\"" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "61a2a15d-b559-4844-b377-6bd5cb4949f6", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "* Running on local URL: http://127.0.0.1:7877\n", + "\n", + "To create a public link, set `share=True` in `launch()`.\n" + ] + }, + { + "data": { + "text/html": [ + "
" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# This function looks rather simpler than the one from my video, because we're taking advantage of the latest Gradio updates\n", + "\n", + "def chat(message, history):\n", + " messages = [{\"role\": \"system\", \"content\": system_message}] + history + [{\"role\": \"user\", \"content\": message}]\n", + " response = openai.chat.completions.create(model=MODEL, messages=messages)\n", + " return response.choices[0].message.content\n", + "\n", + "gr.ChatInterface(fn=chat, type=\"messages\").launch()" + ] + }, + { + "cell_type": "markdown", + "id": "36bedabf-a0a7-4985-ad8e-07ed6a55a3a4", + "metadata": {}, + "source": [ + "## Tools\n", + "\n", + "Tools are an incredibly powerful feature provided by the frontier LLMs.\n", + "\n", + "With tools, you can write a function, and have the LLM call that function as part of its response.\n", + "\n", + "Sounds almost spooky.. we're giving it the power to run code on our machine?\n", + "\n", + "Well, kinda." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "0696acb1-0b05-4dc2-80d5-771be04f1fb2", + "metadata": {}, + "outputs": [], + "source": [ + "# Let's start by making a useful function\n", + "\n", + "ticket_prices = {\"london\": \"$799\", \"paris\": \"$899\", \"tokyo\": \"$1400\", \"berlin\": \"$499\"}\n", + "\n", + "def get_ticket_price(destination_city):\n", + " print(f\"Tool get_ticket_price called for {destination_city}\")\n", + " city = destination_city.lower()\n", + " return ticket_prices.get(city, \"Unknown\")" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "80ca4e09-6287-4d3f-997d-fa6afbcf6c85", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Tool get_ticket_price called for Berlin\n" + ] + }, + { + "data": { + "text/plain": [ + "'$499'" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "get_ticket_price(\"Berlin\")" + ] + }, + { + "cell_type": "code", + "execution_count": 29, + "id": "0757cba1", + "metadata": {}, + "outputs": [], + "source": [ + "import random\n", + "\n", + "# Create a function for the booking system\n", + "def get_booking(destination_city):\n", + " print(f\"Tool get_booking called for {destination_city}\")\n", + " city = destination_city.lower()\n", + " \n", + " # Example data for different cities\n", + " flight_info = {\n", + " \"london\": {\"flight_number\": \"BA123\", \"departure_time\": \"10:00 AM\", \"gate\": \"A12\"},\n", + " \"paris\": {\"flight_number\": \"AF456\", \"departure_time\": \"12:00 PM\", \"gate\": \"B34\"},\n", + " \"tokyo\": {\"flight_number\": \"JL789\", \"departure_time\": \"02:00 PM\", \"gate\": \"C56\"},\n", + " \"berlin\": {\"flight_number\": \"LH101\", \"departure_time\": \"04:00 PM\", \"gate\": \"D78\"}\n", + " }\n", + " \n", + " if city in flight_info:\n", + " info = flight_info[city]\n", + " status = random.choice([\"available\", \"not available\"])\n", + " return f\"Flight {info['flight_number']} to {destination_city.lower()} is {status}. Departure time: {info['departure_time']}, Gate: {info['gate']}.\"\n", + " else:\n", + " return \"Unknown destination city.\"" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "d5413a96", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Tool get_booking called for Berlin\n" + ] + }, + { + "data": { + "text/plain": [ + "'Flight LH101 to berlin is cancelled. Departure time: 04:00 PM, Gate: D78.'" + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "get_booking(\"Berlin\")" + ] + }, + { + "cell_type": "code", + "execution_count": 30, + "id": "4afceded-7178-4c05-8fa6-9f2085e6a344", + "metadata": {}, + "outputs": [], + "source": [ + "# There's a particular dictionary structure that's required to describe our function:\n", + "\n", + "price_function = {\n", + " \"name\": \"get_ticket_price\",\n", + " \"description\": \"Get the price of a return ticket to the destination city. Call this whenever you need to know the ticket price, for example when a customer asks 'How much is a ticket to this city'\",\n", + " \"parameters\": {\n", + " \"type\": \"object\",\n", + " \"properties\": {\n", + " \"destination_city\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"The city that the customer wants to travel to\",\n", + " },\n", + " },\n", + " \"required\": [\"destination_city\"],\n", + " \"additionalProperties\": False\n", + " }\n", + "}\n", + "\n", + "# Book flight function description and properties\n", + "\n", + "book_flight_function = {\n", + " \"name\": \"book_flight\",\n", + " \"description\": \"Book a flight to the destination city. Call this whenever a customer wants to book a flight.\",\n", + " \"parameters\": {\n", + " \"type\": \"object\",\n", + " \"properties\": {\n", + " \"destination_city\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"The city that the customer wants to travel to\",\n", + " },\n", + " \"departure_date\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"The date of departure (YYYY-MM-DD)\",\n", + " },\n", + " \"return_date\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"The date of return (YYYY-MM-DD)\",\n", + " },\n", + " \"passenger_name\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"The name of the passenger\",\n", + " },\n", + " },\n", + " \"required\": [\"destination_city\", \"departure_date\", \"return_date\", \"passenger_name\"],\n", + " \"additionalProperties\": False\n", + " }\n", + "}" + ] + }, + { + "cell_type": "code", + "execution_count": 31, + "id": "bdca8679-935f-4e7f-97e6-e71a4d4f228c", + "metadata": {}, + "outputs": [], + "source": [ + "# And this is included in a list of tools:\n", + "\n", + "tools = [{\"type\": \"function\", \"function\": price_function}, {\"type\": \"function\", \"function\": book_flight_function}]" + ] + }, + { + "cell_type": "markdown", + "id": "c3d3554f-b4e3-4ce7-af6f-68faa6dd2340", + "metadata": {}, + "source": [ + "## Getting OpenAI to use our Tool\n", + "\n", + "There's some fiddly stuff to allow OpenAI \"to call our tool\"\n", + "\n", + "What we actually do is give the LLM the opportunity to inform us that it wants us to run the tool.\n", + "\n", + "Here's how the new chat function looks:" + ] + }, + { + "cell_type": "code", + "execution_count": 33, + "id": "ce9b0744-9c78-408d-b9df-9f6fd9ed78cf", + "metadata": {}, + "outputs": [], + "source": [ + "def chat(message, history):\n", + " messages = [{\"role\": \"system\", \"content\": system_message}] + history + [{\"role\": \"user\", \"content\": message}]\n", + " response = openai.chat.completions.create(model=MODEL, messages=messages, tools=tools)\n", + "\n", + " if response.choices[0].finish_reason==\"tool_calls\":\n", + " message = response.choices[0].message\n", + " response, city = handle_tool_call(message)\n", + " messages.append(message)\n", + " messages.append(response)\n", + " response = openai.chat.completions.create(model=MODEL, messages=messages)\n", + " \n", + " return response.choices[0].message.content" + ] + }, + { + "cell_type": "code", + "execution_count": 32, + "id": "b0992986-ea09-4912-a076-8e5603ee631f", + "metadata": {}, + "outputs": [], + "source": [ + "# We have to write that function handle_tool_call:\n", + "\n", + "def handle_tool_call(message):\n", + " print(f\"Message type: {type(message)}\")\n", + " tool_call = message.tool_calls[0]\n", + " print(f\"Tool call: {tool_call}\")\n", + " arguments = json.loads(tool_call.function.arguments)\n", + " city = arguments.get('destination_city')\n", + " price = get_ticket_price(city)\n", + " book = get_booking(city)\n", + " print (book)\n", + " response = {\n", + " \"role\": \"tool\",\n", + " \"content\": json.dumps({\"destination_city\": city,\"price\": price, \"booking\": book}),\n", + " \"tool_call_id\": tool_call.id\n", + " }\n", + " return response, city" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f4be8a71-b19e-4c2f-80df-f59ff2661f14", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "* Running on local URL: http://127.0.0.1:7864\n", + "\n", + "To create a public link, set `share=True` in `launch()`.\n" + ] + }, + { + "data": { + "text/html": [ + "
" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [] + }, + "execution_count": 34, + "metadata": {}, + "output_type": "execute_result" + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Message type: \n", + "Tool call: ChatCompletionMessageToolCall(id='call_TGFmeFmQN689caTlqfLuhycv', function=Function(arguments='{\"destination_city\":\"London\",\"departure_date\":\"2023-10-31\",\"return_date\":\"2025-03-30\",\"passenger_name\":\"dimitris\"}', name='book_flight'), type='function')\n", + "Tool get_ticket_price called for London\n", + "Tool get_booking called for London\n", + "Flight BA123 to london is available. Departure time: 10:00 AM, Gate: A12.\n", + "Message type: \n", + "Tool call: ChatCompletionMessageToolCall(id='call_FRzs5w09rkpVumZ61SArRlND', function=Function(arguments='{\"destination_city\":\"Paris\",\"departure_date\":\"2023-03-23\",\"return_date\":\"2025-03-30\",\"passenger_name\":\"Dimitris\"}', name='book_flight'), type='function')\n", + "Tool get_ticket_price called for Paris\n", + "Tool get_booking called for Paris\n", + "Flight AF456 to paris is available. Departure time: 12:00 PM, Gate: B34.\n" + ] + } + ], + "source": [ + "gr.ChatInterface(fn=chat, type=\"messages\").launch()" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "llms", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.11" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/week2/community-contributions/day5_book_flight_sightseeing_tools.ipynb b/week2/community-contributions/day5_book_flight_sightseeing_tools.ipynb new file mode 100644 index 0000000..6ceeaaf --- /dev/null +++ b/week2/community-contributions/day5_book_flight_sightseeing_tools.ipynb @@ -0,0 +1,1108 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "ddfa9ae6-69fe-444a-b994-8c4c5970a7ec", + "metadata": {}, + "source": [ + "# Project - Airline AI Assistant\n", + "\n", + "We'll now bring together what we've learned to make an AI Customer Support assistant for an Airline" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "8b50bbe2-c0b1-49c3-9a5c-1ba7efa2bcb4", + "metadata": {}, + "outputs": [], + "source": [ + "# imports\n", + "\n", + "import os\n", + "import json\n", + "from dotenv import load_dotenv\n", + "from openai import OpenAI\n", + "import gradio as gr" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "747e8786-9da8-4342-b6c9-f5f69c2e22ae", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "OpenAI API Key exists and begins sk-proj-\n" + ] + } + ], + "source": [ + "# Initialization\n", + "\n", + "load_dotenv(override=True)\n", + "\n", + "openai_api_key = os.getenv('OPENAI_API_KEY')\n", + "if openai_api_key:\n", + " print(f\"OpenAI API Key exists and begins {openai_api_key[:8]}\")\n", + "else:\n", + " print(\"OpenAI API Key not set\")\n", + " \n", + "MODEL = \"gpt-4o-mini\"\n", + "openai = OpenAI()" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "0a521d84-d07c-49ab-a0df-d6451499ed97", + "metadata": {}, + "outputs": [], + "source": [ + "system_message = \"You are a helpful assistant for an Airline called FlightAI. \"\n", + "system_message += \"Give short, courteous answers, no more than 1 sentence. \"\n", + "system_message += \"Always be accurate. If you don't know the answer, say so.\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "61a2a15d-b559-4844-b377-6bd5cb4949f6", + "metadata": {}, + "outputs": [], + "source": [ + "# This function looks rather simpler than the one from my video, because we're taking advantage of the latest Gradio updates\n", + "\n", + "def chat(message, history):\n", + " messages = [{\"role\": \"system\", \"content\": system_message}] + history + [{\"role\": \"user\", \"content\": message}]\n", + " response = openai.chat.completions.create(model=MODEL, messages=messages)\n", + " return response.choices[0].message.content\n", + "\n", + "gr.ChatInterface(fn=chat, type=\"messages\").launch()" + ] + }, + { + "cell_type": "markdown", + "id": "36bedabf-a0a7-4985-ad8e-07ed6a55a3a4", + "metadata": {}, + "source": [ + "## Tools\n", + "\n", + "Tools are an incredibly powerful feature provided by the frontier LLMs.\n", + "\n", + "With tools, you can write a function, and have the LLM call that function as part of its response.\n", + "\n", + "Sounds almost spooky.. we're giving it the power to run code on our machine?\n", + "\n", + "Well, kinda." + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "id": "0696acb1-0b05-4dc2-80d5-771be04f1fb2", + "metadata": {}, + "outputs": [], + "source": [ + "# Let's start by making a useful function\n", + "\n", + "ticket_prices = {\"london\": \"$799\", \"paris\": \"$899\", \"tokyo\": \"$1400\", \"berlin\": \"$499\", \"athens\": \"$599\", \"kastoria\": \"$999\"}\n", + "\n", + "def get_ticket_price(destination_city):\n", + " print(f\"Tool get_ticket_price called for {destination_city}\")\n", + " city = destination_city.lower()\n", + " return ticket_prices.get(city, \"Unknown\")" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "id": "80ca4e09-6287-4d3f-997d-fa6afbcf6c85", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Tool get_ticket_price called for London\n" + ] + }, + { + "data": { + "text/plain": [ + "'$799'" + ] + }, + "execution_count": 5, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "get_ticket_price(\"London\")" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "id": "2054c00e", + "metadata": {}, + "outputs": [], + "source": [ + "import random\n", + "\n", + "# Create a function for the booking system\n", + "def get_booking(destination_city):\n", + " print(f\"Tool get_booking called for {destination_city}\")\n", + " city = destination_city.lower()\n", + " \n", + " # Example data for different cities\n", + " flight_info = {\n", + " \"london\": {\"flight_number\": \"BA123\", \"departure_time\": \"10:00 AM\", \"gate\": \"A12\"},\n", + " \"paris\": {\"flight_number\": \"AF456\", \"departure_time\": \"12:00 PM\", \"gate\": \"B34\"},\n", + " \"tokyo\": {\"flight_number\": \"JL789\", \"departure_time\": \"02:00 PM\", \"gate\": \"C56\"},\n", + " \"berlin\": {\"flight_number\": \"LH101\", \"departure_time\": \"04:00 PM\", \"gate\": \"D78\"},\n", + " \"athens\": {\"flight_number\": \"OA202\", \"departure_time\": \"06:00 PM\", \"gate\": \"E90\"},\n", + " \"kastoria\": {\"flight_number\": \"KAS303\", \"departure_time\": \"08:00 PM\", \"gate\": \"F12\"}\n", + " }\n", + " \n", + " if city in flight_info:\n", + " info = flight_info[city]\n", + " status = random.choice([\"available\", \"not available\"])\n", + " return f\"Flight {info['flight_number']} to {destination_city.lower()} is {status}. Departure time: {info['departure_time']}, Gate: {info['gate']}.\"\n", + " else:\n", + " return \"Unknown destination city.\"" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "id": "ef334206", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Tool get_booking called for London\n" + ] + }, + { + "data": { + "text/plain": [ + "'Flight BA123 to london is not available. Departure time: 10:00 AM, Gate: A12.'" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "get_booking(\"London\")" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "id": "b011afc2", + "metadata": {}, + "outputs": [], + "source": [ + "sightseeing_info = {\"london\": \"London Eye, Big Ben, Tower of London\", \n", + " \"paris\": \"Eiffel Tower, Louvre Museum, Notre-Dame Cathedral\", \n", + " \"tokyo\": \"Tokyo Tower, Senso-ji Temple, Meiji Shrine\", \n", + " \"berlin\": \"Brandenburg Gate, Berlin Wall, Museum Island\", \n", + " \"athens\": \"Acropolis, Parthenon, Temple of Olympian Zeus\", \n", + " \"kastoria\": \"Cave of Dragon, Kastoria Lake, Byzantine Museum\"}\n", + "\n", + "\n", + "def get_sightseeing(destination_city):\n", + " print(f\"Tool get_ticket_price called for {destination_city}\")\n", + " city = destination_city.lower()\n", + " return sightseeing_info.get(city, \"Unknown\")\n", + " " + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "id": "3008e353", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Tool get_ticket_price called for Kastoria\n" + ] + }, + { + "data": { + "text/plain": [ + "'Cave of Dragon, Kastoria Lake, Byzantine Museum'" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "get_sightseeing(\"Kastoria\")" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "id": "4afceded-7178-4c05-8fa6-9f2085e6a344", + "metadata": {}, + "outputs": [], + "source": [ + "# There's a particular dictionary structure that's required to describe our function:\n", + "\n", + "price_function = {\n", + " \"name\": \"get_ticket_price\",\n", + " \"description\": \"Get the price of a return ticket to the destination city. Call this whenever you need to know the ticket price, for example when a customer asks 'How much is a ticket to this city'\",\n", + " \"parameters\": {\n", + " \"type\": \"object\",\n", + " \"properties\": {\n", + " \"destination_city\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"The city that the customer wants to travel to\",\n", + " },\n", + " },\n", + " \"required\": [\"destination_city\"],\n", + " \"additionalProperties\": False\n", + " }\n", + "}\n", + "\n", + "# Book flight function description and properties\n", + "\n", + "book_flight_function = {\n", + " \"name\": \"book_flight\",\n", + " \"description\": \"Book a flight to the destination city. Call this whenever a customer wants to book a flight.\",\n", + " \"parameters\": {\n", + " \"type\": \"object\",\n", + " \"properties\": {\n", + " \"destination_city\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"The city that the customer wants to travel to\",\n", + " },\n", + " \"departure_date\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"The date of departure (YYYY-MM-DD)\",\n", + " },\n", + " \"return_date\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"The date of return (YYYY-MM-DD)\",\n", + " },\n", + " \"passenger_name\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"The name of the passenger\",\n", + " },\n", + " },\n", + " \"required\": [\"destination_city\", \"departure_date\", \"return_date\", \"passenger_name\"],\n", + " \"additionalProperties\": False\n", + " }\n", + "}\n", + "\n", + "sightseeing_function = {\n", + " \"name\": \"sightseeing\",\n", + " \"description\": \"Get the top sightseeing recommendations for the destination city. Call this whenever a customer asks 'What are the top things to do in this city'\",\n", + " \"parameters\": {\n", + " \"type\": \"object\",\n", + " \"properties\": {\n", + " \"destination_city\": {\n", + " \"type\": \"string\",\n", + " \"description\": \"The city that the customer wants to travel to\",\n", + " },\n", + " },\n", + " \"required\": [\"destination_city\"],\n", + " \"additionalProperties\": False\n", + " }\n", + "}" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "id": "bdca8679-935f-4e7f-97e6-e71a4d4f228c", + "metadata": {}, + "outputs": [], + "source": [ + "# And this is included in a list of tools:\n", + "\n", + "tools = [{\"type\": \"function\", \"function\": price_function}, \n", + " {\"type\": \"function\", \"function\": book_flight_function},\n", + " {\"type\": \"function\", \"function\": sightseeing_function}]" + ] + }, + { + "cell_type": "markdown", + "id": "c3d3554f-b4e3-4ce7-af6f-68faa6dd2340", + "metadata": {}, + "source": [ + "## Getting OpenAI to use our Tool\n", + "\n", + "There's some fiddly stuff to allow OpenAI \"to call our tool\"\n", + "\n", + "What we actually do is give the LLM the opportunity to inform us that it wants us to run the tool.\n", + "\n", + "Here's how the new chat function looks:" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "ce9b0744-9c78-408d-b9df-9f6fd9ed78cf", + "metadata": {}, + "outputs": [], + "source": [ + "def chat(message, history):\n", + " messages = [{\"role\": \"system\", \"content\": system_message}] + history + [{\"role\": \"user\", \"content\": message}]\n", + " response = openai.chat.completions.create(model=MODEL, messages=messages, tools=tools)\n", + "\n", + " if response.choices[0].finish_reason==\"tool_calls\":\n", + " message = response.choices[0].message\n", + " response, city = handle_tool_call(message)\n", + " messages.append(message)\n", + " messages.append(response)\n", + " response = openai.chat.completions.create(model=MODEL, messages=messages)\n", + " \n", + " return response.choices[0].message.content" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "id": "b0992986-ea09-4912-a076-8e5603ee631f", + "metadata": {}, + "outputs": [], + "source": [ + "# We have to write that function handle_tool_call:\n", + "\n", + "def handle_tool_call(message):\n", + " tool_call = message.tool_calls[0]\n", + " print(f\"Tool call: {tool_call}\")\n", + " arguments = json.loads(tool_call.function.arguments)\n", + " city = arguments.get('destination_city')\n", + " price = get_ticket_price(city)\n", + " book = get_booking(city)\n", + " sightseeing = get_sightseeing(city)\n", + " print (book)\n", + " response = {\n", + " \"role\": \"tool\",\n", + " \"content\": json.dumps({\"destination_city\": city,\"price\": price, \"booking\": book, \"sightseeing\": sightseeing}),\n", + " \"tool_call_id\": tool_call.id\n", + " }\n", + " return response, city" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f4be8a71-b19e-4c2f-80df-f59ff2661f14", + "metadata": {}, + "outputs": [], + "source": [ + "gr.ChatInterface(fn=chat, type=\"messages\").launch()" + ] + }, + { + "cell_type": "markdown", + "id": "473e5b39-da8f-4db1-83ae-dbaca2e9531e", + "metadata": {}, + "source": [ + "# Let's go multi-modal!!\n", + "\n", + "We can use DALL-E-3, the image generation model behind GPT-4o, to make us some images\n", + "\n", + "Let's put this in a function called artist.\n", + "\n", + "### Price alert: each time I generate an image it costs about 4 cents - don't go crazy with images!" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "2c27c4ba-8ed5-492f-add1-02ce9c81d34c", + "metadata": {}, + "outputs": [], + "source": [ + "# Some imports for handling images\n", + "\n", + "import base64\n", + "from io import BytesIO\n", + "from PIL import Image" + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "id": "773a9f11-557e-43c9-ad50-56cbec3a0f8f", + "metadata": {}, + "outputs": [], + "source": [ + "def artist(city):\n", + " image_response = openai.images.generate(\n", + " model=\"dall-e-3\",\n", + " prompt=f\"An image representing a vacation in {city}, showing tourist spots and everything unique about {city}, in a vibrant pop-art style\",\n", + " size=\"1024x1024\",\n", + " n=1,\n", + " response_format=\"b64_json\",\n", + " )\n", + " image_base64 = image_response.data[0].b64_json\n", + " image_data = base64.b64decode(image_base64)\n", + " return Image.open(BytesIO(image_data))" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d877c453-e7fb-482a-88aa-1a03f976b9e9", + "metadata": {}, + "outputs": [], + "source": [ + "image = artist(\"Athens\")\n", + "display(image)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "728a12c5-adc3-415d-bb05-82beb73b079b", + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "markdown", + "id": "f4975b87-19e9-4ade-a232-9b809ec75c9a", + "metadata": {}, + "source": [ + "## Audio (NOTE - Audio is optional for this course - feel free to skip Audio if it causes trouble!)\n", + "\n", + "And let's make a function talker that uses OpenAI's speech model to generate Audio\n", + "\n", + "### Troubleshooting Audio issues\n", + "\n", + "If you have any problems running this code below (like a FileNotFound error, or a warning of a missing package), you may need to install FFmpeg, a very popular audio utility.\n", + "\n", + "**For PC Users**\n", + "\n", + "Detailed instructions are [here](https://chatgpt.com/share/6724efee-6b0c-8012-ac5e-72e2e3885905) and summary instructions:\n", + "\n", + "1. Download FFmpeg from the official website: https://ffmpeg.org/download.html\n", + "\n", + "2. Extract the downloaded files to a location on your computer (e.g., `C:\\ffmpeg`)\n", + "\n", + "3. Add the FFmpeg bin folder to your system PATH:\n", + "- Right-click on 'This PC' or 'My Computer' and select 'Properties'\n", + "- Click on 'Advanced system settings'\n", + "- Click on 'Environment Variables'\n", + "- Under 'System variables', find and edit 'Path'\n", + "- Add a new entry with the path to your FFmpeg bin folder (e.g., `C:\\ffmpeg\\bin`)\n", + "- Restart your command prompt, and within Jupyter Lab do Kernel -> Restart kernel, to pick up the changes\n", + "\n", + "4. Open a new command prompt and run this to make sure it's installed OK\n", + "`ffmpeg -version`\n", + "\n", + "**For Mac Users**\n", + "\n", + "1. Install homebrew if you don't have it already by running this in a Terminal window and following any instructions: \n", + "`/bin/bash -c \"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)\"`\n", + "\n", + "2. Then install FFmpeg with `brew install ffmpeg`\n", + "\n", + "3. Verify your installation with `ffmpeg -version` and if everything is good, within Jupyter Lab do Kernel -> Restart kernel to pick up the changes\n", + "\n", + "Message me or email me at ed@edwarddonner.com with any problems!" + ] + }, + { + "cell_type": "markdown", + "id": "4cc90e80-c96e-4dd4-b9d6-386fe2b7e797", + "metadata": {}, + "source": [ + "## To check you now have ffmpeg and can access it here\n", + "\n", + "Excecute the next cell to see if you get a version number. (Putting an exclamation mark before something in Jupyter Lab tells it to run it as a terminal command rather than python code).\n", + "\n", + "If this doesn't work, you may need to actually save and close down your Jupyter lab, and start it again from a new Terminal window (Mac) or Anaconda prompt (PC), remembering to activate the llms environment. This ensures you pick up ffmpeg.\n", + "\n", + "And if that doesn't work, please contact me!" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "id": "7b3be0fb-1d34-4693-ab6f-dbff190afcd7", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "'ffmpeg' is not recognized as an internal or external command,\n", + "operable program or batch file.\n", + "'ffprobe' is not recognized as an internal or external command,\n", + "operable program or batch file.\n", + "'ffplay' is not recognized as an internal or external command,\n", + "operable program or batch file.\n" + ] + } + ], + "source": [ + "!ffmpeg -version\n", + "!ffprobe -version\n", + "!ffplay -version" + ] + }, + { + "cell_type": "markdown", + "id": "d91d3f8f-e505-4e3c-a87c-9e42ed823db6", + "metadata": {}, + "source": [ + "# For Mac users - and possibly many PC users too\n", + "\n", + "This version should work fine for you. It might work for Windows users too, but you might get a Permissions error writing to a temp file. If so, see the next section!\n", + "\n", + "As always, if you have problems, please contact me! (You could also comment out the audio talker() in the later code if you're less interested in audio generation)" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "id": "ffbfe93b-5e86-4e68-ba71-b301cd5230db", + "metadata": {}, + "outputs": [], + "source": [ + "from pydub import AudioSegment\n", + "from pydub.playback import play\n", + "\n", + "def talker(message):\n", + " response = openai.audio.speech.create(\n", + " model=\"tts-1\",\n", + " voice=\"onyx\", # Also, try replacing onyx with alloy\n", + " input=message\n", + " )\n", + " \n", + " audio_stream = BytesIO(response.content)\n", + " audio = AudioSegment.from_file(audio_stream, format=\"mp3\")\n", + " play(audio)" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "id": "b88d775d-d357-4292-a1ad-5dc5ed567281", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "c:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\site-packages\\pydub\\utils.py:198: RuntimeWarning: Couldn't find ffprobe or avprobe - defaulting to ffprobe, but may not work\n", + " warn(\"Couldn't find ffprobe or avprobe - defaulting to ffprobe, but may not work\", RuntimeWarning)\n" + ] + }, + { + "ename": "FileNotFoundError", + "evalue": "[WinError 2] The system cannot find the file specified", + "output_type": "error", + "traceback": [ + "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[1;31mFileNotFoundError\u001b[0m Traceback (most recent call last)", + "Cell \u001b[1;32mIn[17], line 1\u001b[0m\n\u001b[1;32m----> 1\u001b[0m \u001b[43mtalker\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mWell, hi there\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\n", + "Cell \u001b[1;32mIn[16], line 12\u001b[0m, in \u001b[0;36mtalker\u001b[1;34m(message)\u001b[0m\n\u001b[0;32m 5\u001b[0m response \u001b[38;5;241m=\u001b[39m openai\u001b[38;5;241m.\u001b[39maudio\u001b[38;5;241m.\u001b[39mspeech\u001b[38;5;241m.\u001b[39mcreate(\n\u001b[0;32m 6\u001b[0m model\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mtts-1\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[0;32m 7\u001b[0m voice\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124monyx\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;66;03m# Also, try replacing onyx with alloy\u001b[39;00m\n\u001b[0;32m 8\u001b[0m \u001b[38;5;28minput\u001b[39m\u001b[38;5;241m=\u001b[39mmessage\n\u001b[0;32m 9\u001b[0m )\n\u001b[0;32m 11\u001b[0m audio_stream \u001b[38;5;241m=\u001b[39m BytesIO(response\u001b[38;5;241m.\u001b[39mcontent)\n\u001b[1;32m---> 12\u001b[0m audio \u001b[38;5;241m=\u001b[39m \u001b[43mAudioSegment\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mfrom_file\u001b[49m\u001b[43m(\u001b[49m\u001b[43maudio_stream\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mformat\u001b[39;49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mmp3\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\n\u001b[0;32m 13\u001b[0m play(audio)\n", + "File \u001b[1;32mc:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\site-packages\\pydub\\audio_segment.py:728\u001b[0m, in \u001b[0;36mAudioSegment.from_file\u001b[1;34m(cls, file, format, codec, parameters, start_second, duration, **kwargs)\u001b[0m\n\u001b[0;32m 726\u001b[0m info \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[0;32m 727\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m--> 728\u001b[0m info \u001b[38;5;241m=\u001b[39m \u001b[43mmediainfo_json\u001b[49m\u001b[43m(\u001b[49m\u001b[43morig_file\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mread_ahead_limit\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mread_ahead_limit\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 729\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m info:\n\u001b[0;32m 730\u001b[0m audio_streams \u001b[38;5;241m=\u001b[39m [x \u001b[38;5;28;01mfor\u001b[39;00m x \u001b[38;5;129;01min\u001b[39;00m info[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mstreams\u001b[39m\u001b[38;5;124m'\u001b[39m]\n\u001b[0;32m 731\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m x[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mcodec_type\u001b[39m\u001b[38;5;124m'\u001b[39m] \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m'\u001b[39m\u001b[38;5;124maudio\u001b[39m\u001b[38;5;124m'\u001b[39m]\n", + "File \u001b[1;32mc:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\site-packages\\pydub\\utils.py:274\u001b[0m, in \u001b[0;36mmediainfo_json\u001b[1;34m(filepath, read_ahead_limit)\u001b[0m\n\u001b[0;32m 271\u001b[0m file\u001b[38;5;241m.\u001b[39mclose()\n\u001b[0;32m 273\u001b[0m command \u001b[38;5;241m=\u001b[39m [prober, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124m-of\u001b[39m\u001b[38;5;124m'\u001b[39m, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124mjson\u001b[39m\u001b[38;5;124m'\u001b[39m] \u001b[38;5;241m+\u001b[39m command_args\n\u001b[1;32m--> 274\u001b[0m res \u001b[38;5;241m=\u001b[39m \u001b[43mPopen\u001b[49m\u001b[43m(\u001b[49m\u001b[43mcommand\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstdin\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstdin_parameter\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstdout\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mPIPE\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstderr\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mPIPE\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 275\u001b[0m output, stderr \u001b[38;5;241m=\u001b[39m res\u001b[38;5;241m.\u001b[39mcommunicate(\u001b[38;5;28minput\u001b[39m\u001b[38;5;241m=\u001b[39mstdin_data)\n\u001b[0;32m 276\u001b[0m output \u001b[38;5;241m=\u001b[39m output\u001b[38;5;241m.\u001b[39mdecode(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mutf-8\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124mignore\u001b[39m\u001b[38;5;124m'\u001b[39m)\n", + "File \u001b[1;32mc:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\subprocess.py:1026\u001b[0m, in \u001b[0;36mPopen.__init__\u001b[1;34m(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, user, group, extra_groups, encoding, errors, text, umask, pipesize, process_group)\u001b[0m\n\u001b[0;32m 1022\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mtext_mode:\n\u001b[0;32m 1023\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstderr \u001b[38;5;241m=\u001b[39m io\u001b[38;5;241m.\u001b[39mTextIOWrapper(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstderr,\n\u001b[0;32m 1024\u001b[0m encoding\u001b[38;5;241m=\u001b[39mencoding, errors\u001b[38;5;241m=\u001b[39merrors)\n\u001b[1;32m-> 1026\u001b[0m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_execute_child\u001b[49m\u001b[43m(\u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mexecutable\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mpreexec_fn\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mclose_fds\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1027\u001b[0m \u001b[43m \u001b[49m\u001b[43mpass_fds\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcwd\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43menv\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1028\u001b[0m \u001b[43m \u001b[49m\u001b[43mstartupinfo\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcreationflags\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mshell\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1029\u001b[0m \u001b[43m \u001b[49m\u001b[43mp2cread\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mp2cwrite\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1030\u001b[0m \u001b[43m \u001b[49m\u001b[43mc2pread\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mc2pwrite\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1031\u001b[0m \u001b[43m \u001b[49m\u001b[43merrread\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43merrwrite\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1032\u001b[0m \u001b[43m \u001b[49m\u001b[43mrestore_signals\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1033\u001b[0m \u001b[43m \u001b[49m\u001b[43mgid\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mgids\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43muid\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mumask\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1034\u001b[0m \u001b[43m \u001b[49m\u001b[43mstart_new_session\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mprocess_group\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 1035\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m:\n\u001b[0;32m 1036\u001b[0m \u001b[38;5;66;03m# Cleanup if the child failed starting.\u001b[39;00m\n\u001b[0;32m 1037\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m f \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mfilter\u001b[39m(\u001b[38;5;28;01mNone\u001b[39;00m, (\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstdin, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstdout, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstderr)):\n", + "File \u001b[1;32mc:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\subprocess.py:1538\u001b[0m, in \u001b[0;36mPopen._execute_child\u001b[1;34m(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, unused_restore_signals, unused_gid, unused_gids, unused_uid, unused_umask, unused_start_new_session, unused_process_group)\u001b[0m\n\u001b[0;32m 1536\u001b[0m \u001b[38;5;66;03m# Start the process\u001b[39;00m\n\u001b[0;32m 1537\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m-> 1538\u001b[0m hp, ht, pid, tid \u001b[38;5;241m=\u001b[39m \u001b[43m_winapi\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mCreateProcess\u001b[49m\u001b[43m(\u001b[49m\u001b[43mexecutable\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1539\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;66;43;03m# no special security\u001b[39;49;00m\n\u001b[0;32m 1540\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;28;43;01mNone\u001b[39;49;00m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43;01mNone\u001b[39;49;00m\u001b[43m,\u001b[49m\n\u001b[0;32m 1541\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;28;43mint\u001b[39;49m\u001b[43m(\u001b[49m\u001b[38;5;129;43;01mnot\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mclose_fds\u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1542\u001b[0m \u001b[43m \u001b[49m\u001b[43mcreationflags\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1543\u001b[0m \u001b[43m \u001b[49m\u001b[43menv\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1544\u001b[0m \u001b[43m \u001b[49m\u001b[43mcwd\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1545\u001b[0m \u001b[43m \u001b[49m\u001b[43mstartupinfo\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 1546\u001b[0m \u001b[38;5;28;01mfinally\u001b[39;00m:\n\u001b[0;32m 1547\u001b[0m \u001b[38;5;66;03m# Child is launched. Close the parent's copy of those pipe\u001b[39;00m\n\u001b[0;32m 1548\u001b[0m \u001b[38;5;66;03m# handles that only the child should have open. You need\u001b[39;00m\n\u001b[1;32m (...)\u001b[0m\n\u001b[0;32m 1551\u001b[0m \u001b[38;5;66;03m# pipe will not close when the child process exits and the\u001b[39;00m\n\u001b[0;32m 1552\u001b[0m \u001b[38;5;66;03m# ReadFile will hang.\u001b[39;00m\n\u001b[0;32m 1553\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_close_pipe_fds(p2cread, p2cwrite,\n\u001b[0;32m 1554\u001b[0m c2pread, c2pwrite,\n\u001b[0;32m 1555\u001b[0m errread, errwrite)\n", + "\u001b[1;31mFileNotFoundError\u001b[0m: [WinError 2] The system cannot find the file specified" + ] + } + ], + "source": [ + "talker(\"Well, hi there\")" + ] + }, + { + "cell_type": "markdown", + "id": "ad89a9bd-bb1e-4bbb-a49a-83af5f500c24", + "metadata": {}, + "source": [ + "# For Windows users (or any Mac users with problems above)\n", + "\n", + "## First try the Mac version above, but if you get a permissions error writing to a temp file, then this code should work instead.\n", + "\n", + "A collaboration between students Mark M. and Patrick H. and Claude got this resolved!\n", + "\n", + "Below are 4 variations - hopefully one of them will work on your PC. If not, message me please!\n", + "\n", + "And for Mac people - all 3 of the below work on my Mac too - please try these if the Mac version gave you problems.\n", + "\n", + "## PC Variation 1" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d104b96a-02ca-4159-82fe-88e0452aa479", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "\n", + " \n", + " " + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/html": [ + "\n", + " \n", + " " + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "import base64\n", + "from io import BytesIO\n", + "from PIL import Image\n", + "from IPython.display import Audio, display\n", + "\n", + "def talker(message):\n", + " response = openai.audio.speech.create(\n", + " model=\"tts-1\",\n", + " voice=\"onyx\",\n", + " input=message)\n", + "\n", + " audio_stream = BytesIO(response.content)\n", + " output_filename = \"output_audio.mp3\"\n", + " with open(output_filename, \"wb\") as f:\n", + " f.write(audio_stream.read())\n", + "\n", + " # Play the generated audio\n", + " display(Audio(output_filename, autoplay=True))\n", + "\n", + "talker(\"Well, hi there\")" + ] + }, + { + "cell_type": "markdown", + "id": "3a5d11f4-bbd3-43a1-904d-f684eb5f3e3a", + "metadata": {}, + "source": [ + "## PC Variation 2" + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "id": "d59c8ebd-79c5-498a-bdf2-3a1c50d91aa0", + "metadata": {}, + "outputs": [ + { + "ename": "FileNotFoundError", + "evalue": "[WinError 2] The system cannot find the file specified", + "output_type": "error", + "traceback": [ + "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[1;31mFileNotFoundError\u001b[0m Traceback (most recent call last)", + "Cell \u001b[1;32mIn[19], line 36\u001b[0m\n\u001b[0;32m 33\u001b[0m audio \u001b[38;5;241m=\u001b[39m AudioSegment\u001b[38;5;241m.\u001b[39mfrom_file(audio_stream, \u001b[38;5;28mformat\u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mmp3\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[0;32m 34\u001b[0m play_audio(audio)\n\u001b[1;32m---> 36\u001b[0m \u001b[43mtalker\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mWell hi there\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\n", + "Cell \u001b[1;32mIn[19], line 33\u001b[0m, in \u001b[0;36mtalker\u001b[1;34m(message)\u001b[0m\n\u001b[0;32m 27\u001b[0m response \u001b[38;5;241m=\u001b[39m openai\u001b[38;5;241m.\u001b[39maudio\u001b[38;5;241m.\u001b[39mspeech\u001b[38;5;241m.\u001b[39mcreate(\n\u001b[0;32m 28\u001b[0m model\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mtts-1\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[0;32m 29\u001b[0m voice\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124monyx\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;66;03m# Also, try replacing onyx with alloy\u001b[39;00m\n\u001b[0;32m 30\u001b[0m \u001b[38;5;28minput\u001b[39m\u001b[38;5;241m=\u001b[39mmessage\n\u001b[0;32m 31\u001b[0m )\n\u001b[0;32m 32\u001b[0m audio_stream \u001b[38;5;241m=\u001b[39m BytesIO(response\u001b[38;5;241m.\u001b[39mcontent)\n\u001b[1;32m---> 33\u001b[0m audio \u001b[38;5;241m=\u001b[39m \u001b[43mAudioSegment\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mfrom_file\u001b[49m\u001b[43m(\u001b[49m\u001b[43maudio_stream\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mformat\u001b[39;49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mmp3\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\n\u001b[0;32m 34\u001b[0m play_audio(audio)\n", + "File \u001b[1;32mc:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\site-packages\\pydub\\audio_segment.py:728\u001b[0m, in \u001b[0;36mAudioSegment.from_file\u001b[1;34m(cls, file, format, codec, parameters, start_second, duration, **kwargs)\u001b[0m\n\u001b[0;32m 726\u001b[0m info \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[0;32m 727\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m--> 728\u001b[0m info \u001b[38;5;241m=\u001b[39m \u001b[43mmediainfo_json\u001b[49m\u001b[43m(\u001b[49m\u001b[43morig_file\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mread_ahead_limit\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mread_ahead_limit\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 729\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m info:\n\u001b[0;32m 730\u001b[0m audio_streams \u001b[38;5;241m=\u001b[39m [x \u001b[38;5;28;01mfor\u001b[39;00m x \u001b[38;5;129;01min\u001b[39;00m info[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mstreams\u001b[39m\u001b[38;5;124m'\u001b[39m]\n\u001b[0;32m 731\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m x[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mcodec_type\u001b[39m\u001b[38;5;124m'\u001b[39m] \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m'\u001b[39m\u001b[38;5;124maudio\u001b[39m\u001b[38;5;124m'\u001b[39m]\n", + "File \u001b[1;32mc:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\site-packages\\pydub\\utils.py:274\u001b[0m, in \u001b[0;36mmediainfo_json\u001b[1;34m(filepath, read_ahead_limit)\u001b[0m\n\u001b[0;32m 271\u001b[0m file\u001b[38;5;241m.\u001b[39mclose()\n\u001b[0;32m 273\u001b[0m command \u001b[38;5;241m=\u001b[39m [prober, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124m-of\u001b[39m\u001b[38;5;124m'\u001b[39m, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124mjson\u001b[39m\u001b[38;5;124m'\u001b[39m] \u001b[38;5;241m+\u001b[39m command_args\n\u001b[1;32m--> 274\u001b[0m res \u001b[38;5;241m=\u001b[39m \u001b[43mPopen\u001b[49m\u001b[43m(\u001b[49m\u001b[43mcommand\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstdin\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstdin_parameter\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstdout\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mPIPE\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstderr\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mPIPE\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 275\u001b[0m output, stderr \u001b[38;5;241m=\u001b[39m res\u001b[38;5;241m.\u001b[39mcommunicate(\u001b[38;5;28minput\u001b[39m\u001b[38;5;241m=\u001b[39mstdin_data)\n\u001b[0;32m 276\u001b[0m output \u001b[38;5;241m=\u001b[39m output\u001b[38;5;241m.\u001b[39mdecode(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mutf-8\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124mignore\u001b[39m\u001b[38;5;124m'\u001b[39m)\n", + "File \u001b[1;32mc:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\subprocess.py:1026\u001b[0m, in \u001b[0;36mPopen.__init__\u001b[1;34m(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, user, group, extra_groups, encoding, errors, text, umask, pipesize, process_group)\u001b[0m\n\u001b[0;32m 1022\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mtext_mode:\n\u001b[0;32m 1023\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstderr \u001b[38;5;241m=\u001b[39m io\u001b[38;5;241m.\u001b[39mTextIOWrapper(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstderr,\n\u001b[0;32m 1024\u001b[0m encoding\u001b[38;5;241m=\u001b[39mencoding, errors\u001b[38;5;241m=\u001b[39merrors)\n\u001b[1;32m-> 1026\u001b[0m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_execute_child\u001b[49m\u001b[43m(\u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mexecutable\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mpreexec_fn\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mclose_fds\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1027\u001b[0m \u001b[43m \u001b[49m\u001b[43mpass_fds\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcwd\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43menv\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1028\u001b[0m \u001b[43m \u001b[49m\u001b[43mstartupinfo\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcreationflags\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mshell\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1029\u001b[0m \u001b[43m \u001b[49m\u001b[43mp2cread\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mp2cwrite\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1030\u001b[0m \u001b[43m \u001b[49m\u001b[43mc2pread\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mc2pwrite\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1031\u001b[0m \u001b[43m \u001b[49m\u001b[43merrread\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43merrwrite\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1032\u001b[0m \u001b[43m \u001b[49m\u001b[43mrestore_signals\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1033\u001b[0m \u001b[43m \u001b[49m\u001b[43mgid\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mgids\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43muid\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mumask\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1034\u001b[0m \u001b[43m \u001b[49m\u001b[43mstart_new_session\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mprocess_group\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 1035\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m:\n\u001b[0;32m 1036\u001b[0m \u001b[38;5;66;03m# Cleanup if the child failed starting.\u001b[39;00m\n\u001b[0;32m 1037\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m f \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mfilter\u001b[39m(\u001b[38;5;28;01mNone\u001b[39;00m, (\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstdin, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstdout, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstderr)):\n", + "File \u001b[1;32mc:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\subprocess.py:1538\u001b[0m, in \u001b[0;36mPopen._execute_child\u001b[1;34m(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, unused_restore_signals, unused_gid, unused_gids, unused_uid, unused_umask, unused_start_new_session, unused_process_group)\u001b[0m\n\u001b[0;32m 1536\u001b[0m \u001b[38;5;66;03m# Start the process\u001b[39;00m\n\u001b[0;32m 1537\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m-> 1538\u001b[0m hp, ht, pid, tid \u001b[38;5;241m=\u001b[39m \u001b[43m_winapi\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mCreateProcess\u001b[49m\u001b[43m(\u001b[49m\u001b[43mexecutable\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1539\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;66;43;03m# no special security\u001b[39;49;00m\n\u001b[0;32m 1540\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;28;43;01mNone\u001b[39;49;00m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43;01mNone\u001b[39;49;00m\u001b[43m,\u001b[49m\n\u001b[0;32m 1541\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;28;43mint\u001b[39;49m\u001b[43m(\u001b[49m\u001b[38;5;129;43;01mnot\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mclose_fds\u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1542\u001b[0m \u001b[43m \u001b[49m\u001b[43mcreationflags\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1543\u001b[0m \u001b[43m \u001b[49m\u001b[43menv\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1544\u001b[0m \u001b[43m \u001b[49m\u001b[43mcwd\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1545\u001b[0m \u001b[43m \u001b[49m\u001b[43mstartupinfo\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 1546\u001b[0m \u001b[38;5;28;01mfinally\u001b[39;00m:\n\u001b[0;32m 1547\u001b[0m \u001b[38;5;66;03m# Child is launched. Close the parent's copy of those pipe\u001b[39;00m\n\u001b[0;32m 1548\u001b[0m \u001b[38;5;66;03m# handles that only the child should have open. You need\u001b[39;00m\n\u001b[1;32m (...)\u001b[0m\n\u001b[0;32m 1551\u001b[0m \u001b[38;5;66;03m# pipe will not close when the child process exits and the\u001b[39;00m\n\u001b[0;32m 1552\u001b[0m \u001b[38;5;66;03m# ReadFile will hang.\u001b[39;00m\n\u001b[0;32m 1553\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_close_pipe_fds(p2cread, p2cwrite,\n\u001b[0;32m 1554\u001b[0m c2pread, c2pwrite,\n\u001b[0;32m 1555\u001b[0m errread, errwrite)\n", + "\u001b[1;31mFileNotFoundError\u001b[0m: [WinError 2] The system cannot find the file specified" + ] + } + ], + "source": [ + "import tempfile\n", + "import subprocess\n", + "from io import BytesIO\n", + "from pydub import AudioSegment\n", + "import time\n", + "\n", + "def play_audio(audio_segment):\n", + " temp_dir = tempfile.gettempdir()\n", + " temp_path = os.path.join(temp_dir, \"temp_audio.wav\")\n", + " try:\n", + " audio_segment.export(temp_path, format=\"wav\")\n", + " time.sleep(3) # Student Dominic found that this was needed. You could also try commenting out to see if not needed on your PC\n", + " subprocess.call([\n", + " \"ffplay\",\n", + " \"-nodisp\",\n", + " \"-autoexit\",\n", + " \"-hide_banner\",\n", + " temp_path\n", + " ], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)\n", + " finally:\n", + " try:\n", + " os.remove(temp_path)\n", + " except Exception:\n", + " pass\n", + " \n", + "def talker(message):\n", + " response = openai.audio.speech.create(\n", + " model=\"tts-1\",\n", + " voice=\"onyx\", # Also, try replacing onyx with alloy\n", + " input=message\n", + " )\n", + " audio_stream = BytesIO(response.content)\n", + " audio = AudioSegment.from_file(audio_stream, format=\"mp3\")\n", + " play_audio(audio)\n", + "\n", + "talker(\"Well hi there\")" + ] + }, + { + "cell_type": "markdown", + "id": "96f90e35-f71e-468e-afea-07b98f74dbcf", + "metadata": {}, + "source": [ + "## PC Variation 3" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "id": "8597c7f8-7b50-44ad-9b31-db12375cd57b", + "metadata": {}, + "outputs": [ + { + "ename": "FileNotFoundError", + "evalue": "[WinError 2] The system cannot find the file specified", + "output_type": "error", + "traceback": [ + "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[1;31mFileNotFoundError\u001b[0m Traceback (most recent call last)", + "Cell \u001b[1;32mIn[20], line 26\u001b[0m\n\u001b[0;32m 22\u001b[0m audio \u001b[38;5;241m=\u001b[39m AudioSegment\u001b[38;5;241m.\u001b[39mfrom_file(audio_stream, \u001b[38;5;28mformat\u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mmp3\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[0;32m 24\u001b[0m play(audio)\n\u001b[1;32m---> 26\u001b[0m \u001b[43mtalker\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mWell hi there\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\n", + "Cell \u001b[1;32mIn[20], line 22\u001b[0m, in \u001b[0;36mtalker\u001b[1;34m(message)\u001b[0m\n\u001b[0;32m 15\u001b[0m response \u001b[38;5;241m=\u001b[39m openai\u001b[38;5;241m.\u001b[39maudio\u001b[38;5;241m.\u001b[39mspeech\u001b[38;5;241m.\u001b[39mcreate(\n\u001b[0;32m 16\u001b[0m model\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mtts-1\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[0;32m 17\u001b[0m voice\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124monyx\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;66;03m# Also, try replacing onyx with alloy\u001b[39;00m\n\u001b[0;32m 18\u001b[0m \u001b[38;5;28minput\u001b[39m\u001b[38;5;241m=\u001b[39mmessage\n\u001b[0;32m 19\u001b[0m )\n\u001b[0;32m 21\u001b[0m audio_stream \u001b[38;5;241m=\u001b[39m BytesIO(response\u001b[38;5;241m.\u001b[39mcontent)\n\u001b[1;32m---> 22\u001b[0m audio \u001b[38;5;241m=\u001b[39m \u001b[43mAudioSegment\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mfrom_file\u001b[49m\u001b[43m(\u001b[49m\u001b[43maudio_stream\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mformat\u001b[39;49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mmp3\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\n\u001b[0;32m 24\u001b[0m play(audio)\n", + "File \u001b[1;32mc:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\site-packages\\pydub\\audio_segment.py:728\u001b[0m, in \u001b[0;36mAudioSegment.from_file\u001b[1;34m(cls, file, format, codec, parameters, start_second, duration, **kwargs)\u001b[0m\n\u001b[0;32m 726\u001b[0m info \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[0;32m 727\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m--> 728\u001b[0m info \u001b[38;5;241m=\u001b[39m \u001b[43mmediainfo_json\u001b[49m\u001b[43m(\u001b[49m\u001b[43morig_file\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mread_ahead_limit\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mread_ahead_limit\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 729\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m info:\n\u001b[0;32m 730\u001b[0m audio_streams \u001b[38;5;241m=\u001b[39m [x \u001b[38;5;28;01mfor\u001b[39;00m x \u001b[38;5;129;01min\u001b[39;00m info[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mstreams\u001b[39m\u001b[38;5;124m'\u001b[39m]\n\u001b[0;32m 731\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m x[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mcodec_type\u001b[39m\u001b[38;5;124m'\u001b[39m] \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m'\u001b[39m\u001b[38;5;124maudio\u001b[39m\u001b[38;5;124m'\u001b[39m]\n", + "File \u001b[1;32mc:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\site-packages\\pydub\\utils.py:274\u001b[0m, in \u001b[0;36mmediainfo_json\u001b[1;34m(filepath, read_ahead_limit)\u001b[0m\n\u001b[0;32m 271\u001b[0m file\u001b[38;5;241m.\u001b[39mclose()\n\u001b[0;32m 273\u001b[0m command \u001b[38;5;241m=\u001b[39m [prober, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124m-of\u001b[39m\u001b[38;5;124m'\u001b[39m, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124mjson\u001b[39m\u001b[38;5;124m'\u001b[39m] \u001b[38;5;241m+\u001b[39m command_args\n\u001b[1;32m--> 274\u001b[0m res \u001b[38;5;241m=\u001b[39m \u001b[43mPopen\u001b[49m\u001b[43m(\u001b[49m\u001b[43mcommand\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstdin\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstdin_parameter\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstdout\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mPIPE\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstderr\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mPIPE\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 275\u001b[0m output, stderr \u001b[38;5;241m=\u001b[39m res\u001b[38;5;241m.\u001b[39mcommunicate(\u001b[38;5;28minput\u001b[39m\u001b[38;5;241m=\u001b[39mstdin_data)\n\u001b[0;32m 276\u001b[0m output \u001b[38;5;241m=\u001b[39m output\u001b[38;5;241m.\u001b[39mdecode(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mutf-8\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124mignore\u001b[39m\u001b[38;5;124m'\u001b[39m)\n", + "File \u001b[1;32mc:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\subprocess.py:1026\u001b[0m, in \u001b[0;36mPopen.__init__\u001b[1;34m(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, user, group, extra_groups, encoding, errors, text, umask, pipesize, process_group)\u001b[0m\n\u001b[0;32m 1022\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mtext_mode:\n\u001b[0;32m 1023\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstderr \u001b[38;5;241m=\u001b[39m io\u001b[38;5;241m.\u001b[39mTextIOWrapper(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstderr,\n\u001b[0;32m 1024\u001b[0m encoding\u001b[38;5;241m=\u001b[39mencoding, errors\u001b[38;5;241m=\u001b[39merrors)\n\u001b[1;32m-> 1026\u001b[0m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_execute_child\u001b[49m\u001b[43m(\u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mexecutable\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mpreexec_fn\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mclose_fds\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1027\u001b[0m \u001b[43m \u001b[49m\u001b[43mpass_fds\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcwd\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43menv\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1028\u001b[0m \u001b[43m \u001b[49m\u001b[43mstartupinfo\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcreationflags\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mshell\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1029\u001b[0m \u001b[43m \u001b[49m\u001b[43mp2cread\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mp2cwrite\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1030\u001b[0m \u001b[43m \u001b[49m\u001b[43mc2pread\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mc2pwrite\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1031\u001b[0m \u001b[43m \u001b[49m\u001b[43merrread\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43merrwrite\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1032\u001b[0m \u001b[43m \u001b[49m\u001b[43mrestore_signals\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1033\u001b[0m \u001b[43m \u001b[49m\u001b[43mgid\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mgids\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43muid\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mumask\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1034\u001b[0m \u001b[43m \u001b[49m\u001b[43mstart_new_session\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mprocess_group\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 1035\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m:\n\u001b[0;32m 1036\u001b[0m \u001b[38;5;66;03m# Cleanup if the child failed starting.\u001b[39;00m\n\u001b[0;32m 1037\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m f \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mfilter\u001b[39m(\u001b[38;5;28;01mNone\u001b[39;00m, (\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstdin, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstdout, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstderr)):\n", + "File \u001b[1;32mc:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\subprocess.py:1538\u001b[0m, in \u001b[0;36mPopen._execute_child\u001b[1;34m(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, unused_restore_signals, unused_gid, unused_gids, unused_uid, unused_umask, unused_start_new_session, unused_process_group)\u001b[0m\n\u001b[0;32m 1536\u001b[0m \u001b[38;5;66;03m# Start the process\u001b[39;00m\n\u001b[0;32m 1537\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m-> 1538\u001b[0m hp, ht, pid, tid \u001b[38;5;241m=\u001b[39m _winapi\u001b[38;5;241m.\u001b[39mCreateProcess(executable, args,\n\u001b[0;32m 1539\u001b[0m \u001b[38;5;66;03m# no special security\u001b[39;00m\n\u001b[0;32m 1540\u001b[0m \u001b[38;5;28;01mNone\u001b[39;00m, \u001b[38;5;28;01mNone\u001b[39;00m,\n\u001b[0;32m 1541\u001b[0m \u001b[38;5;28mint\u001b[39m(\u001b[38;5;129;01mnot\u001b[39;00m close_fds),\n\u001b[0;32m 1542\u001b[0m creationflags,\n\u001b[0;32m 1543\u001b[0m env,\n\u001b[0;32m 1544\u001b[0m cwd,\n\u001b[0;32m 1545\u001b[0m startupinfo)\n\u001b[0;32m 1546\u001b[0m \u001b[38;5;28;01mfinally\u001b[39;00m:\n\u001b[0;32m 1547\u001b[0m \u001b[38;5;66;03m# Child is launched. Close the parent's copy of those pipe\u001b[39;00m\n\u001b[0;32m 1548\u001b[0m \u001b[38;5;66;03m# handles that only the child should have open. You need\u001b[39;00m\n\u001b[1;32m (...)\u001b[0m\n\u001b[0;32m 1551\u001b[0m \u001b[38;5;66;03m# pipe will not close when the child process exits and the\u001b[39;00m\n\u001b[0;32m 1552\u001b[0m \u001b[38;5;66;03m# ReadFile will hang.\u001b[39;00m\n\u001b[0;32m 1553\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_close_pipe_fds(p2cread, p2cwrite,\n\u001b[0;32m 1554\u001b[0m c2pread, c2pwrite,\n\u001b[0;32m 1555\u001b[0m errread, errwrite)\n", + "\u001b[1;31mFileNotFoundError\u001b[0m: [WinError 2] The system cannot find the file specified" + ] + } + ], + "source": [ + "import os\n", + "from pydub import AudioSegment\n", + "from pydub.playback import play\n", + "from io import BytesIO\n", + "\n", + "def talker(message):\n", + " # Set a custom directory for temporary files on Windows\n", + " custom_temp_dir = os.path.expanduser(\"~/Documents/temp_audio\")\n", + " os.environ['TEMP'] = custom_temp_dir # You can also use 'TMP' if necessary\n", + " \n", + " # Create the folder if it doesn't exist\n", + " if not os.path.exists(custom_temp_dir):\n", + " os.makedirs(custom_temp_dir)\n", + " \n", + " response = openai.audio.speech.create(\n", + " model=\"tts-1\",\n", + " voice=\"onyx\", # Also, try replacing onyx with alloy\n", + " input=message\n", + " )\n", + " \n", + " audio_stream = BytesIO(response.content)\n", + " audio = AudioSegment.from_file(audio_stream, format=\"mp3\")\n", + "\n", + " play(audio)\n", + "\n", + "talker(\"Well hi there\")" + ] + }, + { + "cell_type": "markdown", + "id": "e821224c-b069-4f9b-9535-c15fdb0e411c", + "metadata": {}, + "source": [ + "## PC Variation 4\n", + "\n", + "### Let's try a completely different sound library\n", + "\n", + "First run the next cell to install a new library, then try the cell below it." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "69d3c0d9-afcc-49e3-b829-9c9869d8b472", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install simpleaudio" + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "id": "28f9cc99-36b7-4554-b3f4-f2012f614a13", + "metadata": {}, + "outputs": [], + "source": [ + "from pydub import AudioSegment\n", + "from io import BytesIO\n", + "import tempfile\n", + "import os\n", + "import simpleaudio as sa\n", + "\n", + "def talker(message):\n", + " response = openai.audio.speech.create(\n", + " model=\"tts-1\",\n", + " voice=\"onyx\", # Also, try replacing onyx with alloy\n", + " input=message\n", + " )\n", + " \n", + " audio_stream = BytesIO(response.content)\n", + " audio = AudioSegment.from_file(audio_stream, format=\"mp3\")\n", + "\n", + " # Create a temporary file in a folder where you have write permissions\n", + " with tempfile.NamedTemporaryFile(suffix=\".wav\", delete=False, dir=os.path.expanduser(\"~/Documents\")) as temp_audio_file:\n", + " temp_file_name = temp_audio_file.name\n", + " audio.export(temp_file_name, format=\"wav\")\n", + " \n", + " # Load and play audio using simpleaudio\n", + " wave_obj = sa.WaveObject.from_wave_file(temp_file_name)\n", + " play_obj = wave_obj.play()\n", + " play_obj.wait_done() # Wait for playback to finish\n", + "\n", + " # Clean up the temporary file afterward\n", + " os.remove(temp_file_name)\n", + " \n" + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "id": "0d248b46", + "metadata": {}, + "outputs": [ + { + "ename": "FileNotFoundError", + "evalue": "[WinError 2] The system cannot find the file specified", + "output_type": "error", + "traceback": [ + "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[1;31mFileNotFoundError\u001b[0m Traceback (most recent call last)", + "Cell \u001b[1;32mIn[22], line 1\u001b[0m\n\u001b[1;32m----> 1\u001b[0m \u001b[43mtalker\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mWell hi there\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\n", + "Cell \u001b[1;32mIn[21], line 15\u001b[0m, in \u001b[0;36mtalker\u001b[1;34m(message)\u001b[0m\n\u001b[0;32m 8\u001b[0m response \u001b[38;5;241m=\u001b[39m openai\u001b[38;5;241m.\u001b[39maudio\u001b[38;5;241m.\u001b[39mspeech\u001b[38;5;241m.\u001b[39mcreate(\n\u001b[0;32m 9\u001b[0m model\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mtts-1\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[0;32m 10\u001b[0m voice\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124monyx\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;66;03m# Also, try replacing onyx with alloy\u001b[39;00m\n\u001b[0;32m 11\u001b[0m \u001b[38;5;28minput\u001b[39m\u001b[38;5;241m=\u001b[39mmessage\n\u001b[0;32m 12\u001b[0m )\n\u001b[0;32m 14\u001b[0m audio_stream \u001b[38;5;241m=\u001b[39m BytesIO(response\u001b[38;5;241m.\u001b[39mcontent)\n\u001b[1;32m---> 15\u001b[0m audio \u001b[38;5;241m=\u001b[39m \u001b[43mAudioSegment\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mfrom_file\u001b[49m\u001b[43m(\u001b[49m\u001b[43maudio_stream\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mformat\u001b[39;49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mmp3\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\n\u001b[0;32m 17\u001b[0m \u001b[38;5;66;03m# Create a temporary file in a folder where you have write permissions\u001b[39;00m\n\u001b[0;32m 18\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m tempfile\u001b[38;5;241m.\u001b[39mNamedTemporaryFile(suffix\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m.wav\u001b[39m\u001b[38;5;124m\"\u001b[39m, delete\u001b[38;5;241m=\u001b[39m\u001b[38;5;28;01mFalse\u001b[39;00m, \u001b[38;5;28mdir\u001b[39m\u001b[38;5;241m=\u001b[39mos\u001b[38;5;241m.\u001b[39mpath\u001b[38;5;241m.\u001b[39mexpanduser(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m~/Documents\u001b[39m\u001b[38;5;124m\"\u001b[39m)) \u001b[38;5;28;01mas\u001b[39;00m temp_audio_file:\n", + "File \u001b[1;32mc:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\site-packages\\pydub\\audio_segment.py:728\u001b[0m, in \u001b[0;36mAudioSegment.from_file\u001b[1;34m(cls, file, format, codec, parameters, start_second, duration, **kwargs)\u001b[0m\n\u001b[0;32m 726\u001b[0m info \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[0;32m 727\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m--> 728\u001b[0m info \u001b[38;5;241m=\u001b[39m \u001b[43mmediainfo_json\u001b[49m\u001b[43m(\u001b[49m\u001b[43morig_file\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mread_ahead_limit\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mread_ahead_limit\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 729\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m info:\n\u001b[0;32m 730\u001b[0m audio_streams \u001b[38;5;241m=\u001b[39m [x \u001b[38;5;28;01mfor\u001b[39;00m x \u001b[38;5;129;01min\u001b[39;00m info[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mstreams\u001b[39m\u001b[38;5;124m'\u001b[39m]\n\u001b[0;32m 731\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m x[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mcodec_type\u001b[39m\u001b[38;5;124m'\u001b[39m] \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m'\u001b[39m\u001b[38;5;124maudio\u001b[39m\u001b[38;5;124m'\u001b[39m]\n", + "File \u001b[1;32mc:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\site-packages\\pydub\\utils.py:274\u001b[0m, in \u001b[0;36mmediainfo_json\u001b[1;34m(filepath, read_ahead_limit)\u001b[0m\n\u001b[0;32m 271\u001b[0m file\u001b[38;5;241m.\u001b[39mclose()\n\u001b[0;32m 273\u001b[0m command \u001b[38;5;241m=\u001b[39m [prober, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124m-of\u001b[39m\u001b[38;5;124m'\u001b[39m, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124mjson\u001b[39m\u001b[38;5;124m'\u001b[39m] \u001b[38;5;241m+\u001b[39m command_args\n\u001b[1;32m--> 274\u001b[0m res \u001b[38;5;241m=\u001b[39m \u001b[43mPopen\u001b[49m\u001b[43m(\u001b[49m\u001b[43mcommand\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstdin\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstdin_parameter\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstdout\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mPIPE\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstderr\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mPIPE\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 275\u001b[0m output, stderr \u001b[38;5;241m=\u001b[39m res\u001b[38;5;241m.\u001b[39mcommunicate(\u001b[38;5;28minput\u001b[39m\u001b[38;5;241m=\u001b[39mstdin_data)\n\u001b[0;32m 276\u001b[0m output \u001b[38;5;241m=\u001b[39m output\u001b[38;5;241m.\u001b[39mdecode(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mutf-8\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124mignore\u001b[39m\u001b[38;5;124m'\u001b[39m)\n", + "File \u001b[1;32mc:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\subprocess.py:1026\u001b[0m, in \u001b[0;36mPopen.__init__\u001b[1;34m(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, user, group, extra_groups, encoding, errors, text, umask, pipesize, process_group)\u001b[0m\n\u001b[0;32m 1022\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mtext_mode:\n\u001b[0;32m 1023\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstderr \u001b[38;5;241m=\u001b[39m io\u001b[38;5;241m.\u001b[39mTextIOWrapper(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstderr,\n\u001b[0;32m 1024\u001b[0m encoding\u001b[38;5;241m=\u001b[39mencoding, errors\u001b[38;5;241m=\u001b[39merrors)\n\u001b[1;32m-> 1026\u001b[0m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_execute_child\u001b[49m\u001b[43m(\u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mexecutable\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mpreexec_fn\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mclose_fds\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1027\u001b[0m \u001b[43m \u001b[49m\u001b[43mpass_fds\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcwd\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43menv\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1028\u001b[0m \u001b[43m \u001b[49m\u001b[43mstartupinfo\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcreationflags\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mshell\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1029\u001b[0m \u001b[43m \u001b[49m\u001b[43mp2cread\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mp2cwrite\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1030\u001b[0m \u001b[43m \u001b[49m\u001b[43mc2pread\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mc2pwrite\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1031\u001b[0m \u001b[43m \u001b[49m\u001b[43merrread\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43merrwrite\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1032\u001b[0m \u001b[43m \u001b[49m\u001b[43mrestore_signals\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1033\u001b[0m \u001b[43m \u001b[49m\u001b[43mgid\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mgids\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43muid\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mumask\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1034\u001b[0m \u001b[43m \u001b[49m\u001b[43mstart_new_session\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mprocess_group\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 1035\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m:\n\u001b[0;32m 1036\u001b[0m \u001b[38;5;66;03m# Cleanup if the child failed starting.\u001b[39;00m\n\u001b[0;32m 1037\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m f \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mfilter\u001b[39m(\u001b[38;5;28;01mNone\u001b[39;00m, (\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstdin, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstdout, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstderr)):\n", + "File \u001b[1;32mc:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\subprocess.py:1538\u001b[0m, in \u001b[0;36mPopen._execute_child\u001b[1;34m(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, unused_restore_signals, unused_gid, unused_gids, unused_uid, unused_umask, unused_start_new_session, unused_process_group)\u001b[0m\n\u001b[0;32m 1536\u001b[0m \u001b[38;5;66;03m# Start the process\u001b[39;00m\n\u001b[0;32m 1537\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m-> 1538\u001b[0m hp, ht, pid, tid \u001b[38;5;241m=\u001b[39m _winapi\u001b[38;5;241m.\u001b[39mCreateProcess(executable, args,\n\u001b[0;32m 1539\u001b[0m \u001b[38;5;66;03m# no special security\u001b[39;00m\n\u001b[0;32m 1540\u001b[0m \u001b[38;5;28;01mNone\u001b[39;00m, \u001b[38;5;28;01mNone\u001b[39;00m,\n\u001b[0;32m 1541\u001b[0m \u001b[38;5;28mint\u001b[39m(\u001b[38;5;129;01mnot\u001b[39;00m close_fds),\n\u001b[0;32m 1542\u001b[0m creationflags,\n\u001b[0;32m 1543\u001b[0m env,\n\u001b[0;32m 1544\u001b[0m cwd,\n\u001b[0;32m 1545\u001b[0m startupinfo)\n\u001b[0;32m 1546\u001b[0m \u001b[38;5;28;01mfinally\u001b[39;00m:\n\u001b[0;32m 1547\u001b[0m \u001b[38;5;66;03m# Child is launched. Close the parent's copy of those pipe\u001b[39;00m\n\u001b[0;32m 1548\u001b[0m \u001b[38;5;66;03m# handles that only the child should have open. You need\u001b[39;00m\n\u001b[1;32m (...)\u001b[0m\n\u001b[0;32m 1551\u001b[0m \u001b[38;5;66;03m# pipe will not close when the child process exits and the\u001b[39;00m\n\u001b[0;32m 1552\u001b[0m \u001b[38;5;66;03m# ReadFile will hang.\u001b[39;00m\n\u001b[0;32m 1553\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_close_pipe_fds(p2cread, p2cwrite,\n\u001b[0;32m 1554\u001b[0m c2pread, c2pwrite,\n\u001b[0;32m 1555\u001b[0m errread, errwrite)\n", + "\u001b[1;31mFileNotFoundError\u001b[0m: [WinError 2] The system cannot find the file specified" + ] + } + ], + "source": [ + "talker(\"Well hi there\")" + ] + }, + { + "cell_type": "markdown", + "id": "7986176b-cd04-495f-a47f-e057b0e462ed", + "metadata": {}, + "source": [ + "## PC Users - if none of those 4 variations worked!\n", + "\n", + "Please get in touch with me. I'm sorry this is causing problems! We'll figure it out.\n", + "\n", + "Alternatively: playing audio from your PC isn't super-critical for this course, and you can feel free to focus on image generation and skip audio for now, or come back to it later." + ] + }, + { + "cell_type": "markdown", + "id": "1d48876d-c4fa-46a8-a04f-f9fadf61fb0d", + "metadata": {}, + "source": [ + "# Our Agent Framework\n", + "\n", + "The term 'Agentic AI' and Agentization is an umbrella term that refers to a number of techniques, such as:\n", + "\n", + "1. Breaking a complex problem into smaller steps, with multiple LLMs carrying out specialized tasks\n", + "2. The ability for LLMs to use Tools to give them additional capabilities\n", + "3. The 'Agent Environment' which allows Agents to collaborate\n", + "4. An LLM can act as the Planner, dividing bigger tasks into smaller ones for the specialists\n", + "5. The concept of an Agent having autonomy / agency, beyond just responding to a prompt - such as Memory\n", + "\n", + "We're showing 1 and 2 here, and to a lesser extent 3 and 5. In week 8 we will do the lot!" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ba820c95-02f5-499e-8f3c-8727ee0a6c0c", + "metadata": {}, + "outputs": [], + "source": [ + "def chat(history):\n", + " messages = [{\"role\": \"system\", \"content\": system_message}] + history\n", + " response = openai.chat.completions.create(model=MODEL, messages=messages, tools=tools)\n", + " image = None\n", + " \n", + " if response.choices[0].finish_reason==\"tool_calls\":\n", + " message = response.choices[0].message\n", + " response, city = handle_tool_call(message)\n", + " messages.append(message)\n", + " messages.append(response)\n", + " image = artist(city)\n", + " response = openai.chat.completions.create(model=MODEL, messages=messages)\n", + " \n", + " reply = response.choices[0].message.content\n", + " history += [{\"role\":\"assistant\", \"content\":reply}]\n", + "\n", + " # Comment out or delete the next line if you'd rather skip Audio for now..\n", + " # It worked for me only with the first variation of the talker function\n", + " talker(reply)\n", + " \n", + " return history, image" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "id": "f38d0d27-33bf-4992-a2e5-5dbed973cde7", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "* Running on local URL: http://127.0.0.1:7866\n", + "\n", + "To create a public link, set `share=True` in `launch()`.\n" + ] + }, + { + "data": { + "text/html": [ + "
" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + }, + { + "data": { + "text/plain": [] + }, + "execution_count": 20, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# More involved Gradio code as we're not using the preset Chat interface!\n", + "# Passing in inbrowser=True in the last line will cause a Gradio window to pop up immediately.\n", + "\n", + "with gr.Blocks() as ui:\n", + " with gr.Row():\n", + " chatbot = gr.Chatbot(height=500, type=\"messages\")\n", + " image_output = gr.Image(height=500)\n", + " with gr.Row():\n", + " entry = gr.Textbox(label=\"Chat with our AI Assistant:\")\n", + " with gr.Row():\n", + " clear = gr.Button(\"Clear\")\n", + "\n", + " def do_entry(message, history):\n", + " history += [{\"role\":\"user\", \"content\":message}]\n", + " return \"\", history\n", + "\n", + " entry.submit(do_entry, inputs=[entry, chatbot], outputs=[entry, chatbot]).then(\n", + " chat, inputs=chatbot, outputs=[chatbot, image_output]\n", + " )\n", + " clear.click(lambda: None, inputs=None, outputs=chatbot, queue=False)\n", + "\n", + "ui.launch(inbrowser=True)" + ] + }, + { + "cell_type": "markdown", + "id": "226643d2-73e4-4252-935d-86b8019e278a", + "metadata": {}, + "source": [ + "# Exercises and Business Applications\n", + "\n", + "Add in more tools - perhaps to simulate actually booking a flight. A student has done this and provided their example in the community contributions folder.\n", + "\n", + "Next: take this and apply it to your business. Make a multi-modal AI assistant with tools that could carry out an activity for your work. A customer support assistant? New employee onboarding assistant? So many possibilities! Also, see the week2 end of week Exercise in the separate Notebook." + ] + }, + { + "cell_type": "markdown", + "id": "7e795560-1867-42db-a256-a23b844e6fbe", + "metadata": {}, + "source": [ + "\n", + " \n", + " \n", + " \n", + " \n", + "
\n", + " \n", + " \n", + "

I have a special request for you

\n", + " \n", + " My editor tells me that it makes a HUGE difference when students rate this course on Udemy - it's one of the main ways that Udemy decides whether to show it to others. If you're able to take a minute to rate this, I'd be so very grateful! And regardless - always please reach out to me at ed@edwarddonner.com if I can help at any point.\n", + " \n", + "
" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "llms", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.11.11" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +}