{ "cells": [ { "cell_type": "markdown", "id": "ddfa9ae6-69fe-444a-b994-8c4c5970a7ec", "metadata": {}, "source": [ "# Project - Airline AI Assistant\n", "\n", "We'll now bring together what we've learned to make an AI Customer Support assistant for an Airline" ] }, { "cell_type": "code", "execution_count": 1, "id": "8b50bbe2-c0b1-49c3-9a5c-1ba7efa2bcb4", "metadata": {}, "outputs": [], "source": [ "# imports\n", "\n", "import os\n", "import json\n", "from dotenv import load_dotenv\n", "from openai import OpenAI\n", "import gradio as gr" ] }, { "cell_type": "code", "execution_count": 2, "id": "747e8786-9da8-4342-b6c9-f5f69c2e22ae", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "OpenAI API Key exists and begins sk-proj-\n" ] } ], "source": [ "# Initialization\n", "\n", "load_dotenv(override=True)\n", "\n", "openai_api_key = os.getenv('OPENAI_API_KEY')\n", "if openai_api_key:\n", " print(f\"OpenAI API Key exists and begins {openai_api_key[:8]}\")\n", "else:\n", " print(\"OpenAI API Key not set\")\n", " \n", "MODEL = \"gpt-4o-mini\"\n", "openai = OpenAI()" ] }, { "cell_type": "code", "execution_count": 3, "id": "0a521d84-d07c-49ab-a0df-d6451499ed97", "metadata": {}, "outputs": [], "source": [ "system_message = \"You are a helpful assistant for an Airline called FlightAI. \"\n", "system_message += \"Give short, courteous answers, no more than 1 sentence. \"\n", "system_message += \"Always be accurate. If you don't know the answer, say so.\"" ] }, { "cell_type": "code", "execution_count": null, "id": "61a2a15d-b559-4844-b377-6bd5cb4949f6", "metadata": {}, "outputs": [], "source": [ "# This function looks rather simpler than the one from my video, because we're taking advantage of the latest Gradio updates\n", "\n", "def chat(message, history):\n", " messages = [{\"role\": \"system\", \"content\": system_message}] + history + [{\"role\": \"user\", \"content\": message}]\n", " response = openai.chat.completions.create(model=MODEL, messages=messages)\n", " return response.choices[0].message.content\n", "\n", "gr.ChatInterface(fn=chat, type=\"messages\").launch()" ] }, { "cell_type": "markdown", "id": "36bedabf-a0a7-4985-ad8e-07ed6a55a3a4", "metadata": {}, "source": [ "## Tools\n", "\n", "Tools are an incredibly powerful feature provided by the frontier LLMs.\n", "\n", "With tools, you can write a function, and have the LLM call that function as part of its response.\n", "\n", "Sounds almost spooky.. we're giving it the power to run code on our machine?\n", "\n", "Well, kinda." ] }, { "cell_type": "code", "execution_count": 4, "id": "0696acb1-0b05-4dc2-80d5-771be04f1fb2", "metadata": {}, "outputs": [], "source": [ "# Let's start by making a useful function\n", "\n", "ticket_prices = {\"london\": \"$799\", \"paris\": \"$899\", \"tokyo\": \"$1400\", \"berlin\": \"$499\", \"athens\": \"$599\", \"kastoria\": \"$999\"}\n", "\n", "def get_ticket_price(destination_city):\n", " print(f\"Tool get_ticket_price called for {destination_city}\")\n", " city = destination_city.lower()\n", " return ticket_prices.get(city, \"Unknown\")" ] }, { "cell_type": "code", "execution_count": 5, "id": "80ca4e09-6287-4d3f-997d-fa6afbcf6c85", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Tool get_ticket_price called for London\n" ] }, { "data": { "text/plain": [ "'$799'" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "get_ticket_price(\"London\")" ] }, { "cell_type": "code", "execution_count": 6, "id": "2054c00e", "metadata": {}, "outputs": [], "source": [ "import random\n", "\n", "# Create a function for the booking system\n", "def get_booking(destination_city):\n", " print(f\"Tool get_booking called for {destination_city}\")\n", " city = destination_city.lower()\n", " \n", " # Example data for different cities\n", " flight_info = {\n", " \"london\": {\"flight_number\": \"BA123\", \"departure_time\": \"10:00 AM\", \"gate\": \"A12\"},\n", " \"paris\": {\"flight_number\": \"AF456\", \"departure_time\": \"12:00 PM\", \"gate\": \"B34\"},\n", " \"tokyo\": {\"flight_number\": \"JL789\", \"departure_time\": \"02:00 PM\", \"gate\": \"C56\"},\n", " \"berlin\": {\"flight_number\": \"LH101\", \"departure_time\": \"04:00 PM\", \"gate\": \"D78\"},\n", " \"athens\": {\"flight_number\": \"OA202\", \"departure_time\": \"06:00 PM\", \"gate\": \"E90\"},\n", " \"kastoria\": {\"flight_number\": \"KAS303\", \"departure_time\": \"08:00 PM\", \"gate\": \"F12\"}\n", " }\n", " \n", " if city in flight_info:\n", " info = flight_info[city]\n", " status = random.choice([\"available\", \"not available\"])\n", " return f\"Flight {info['flight_number']} to {destination_city.lower()} is {status}. Departure time: {info['departure_time']}, Gate: {info['gate']}.\"\n", " else:\n", " return \"Unknown destination city.\"" ] }, { "cell_type": "code", "execution_count": 7, "id": "ef334206", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Tool get_booking called for London\n" ] }, { "data": { "text/plain": [ "'Flight BA123 to london is not available. Departure time: 10:00 AM, Gate: A12.'" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "get_booking(\"London\")" ] }, { "cell_type": "code", "execution_count": 8, "id": "b011afc2", "metadata": {}, "outputs": [], "source": [ "sightseeing_info = {\"london\": \"London Eye, Big Ben, Tower of London\", \n", " \"paris\": \"Eiffel Tower, Louvre Museum, Notre-Dame Cathedral\", \n", " \"tokyo\": \"Tokyo Tower, Senso-ji Temple, Meiji Shrine\", \n", " \"berlin\": \"Brandenburg Gate, Berlin Wall, Museum Island\", \n", " \"athens\": \"Acropolis, Parthenon, Temple of Olympian Zeus\", \n", " \"kastoria\": \"Cave of Dragon, Kastoria Lake, Byzantine Museum\"}\n", "\n", "\n", "def get_sightseeing(destination_city):\n", " print(f\"Tool get_ticket_price called for {destination_city}\")\n", " city = destination_city.lower()\n", " return sightseeing_info.get(city, \"Unknown\")\n", " " ] }, { "cell_type": "code", "execution_count": 9, "id": "3008e353", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Tool get_ticket_price called for Kastoria\n" ] }, { "data": { "text/plain": [ "'Cave of Dragon, Kastoria Lake, Byzantine Museum'" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "get_sightseeing(\"Kastoria\")" ] }, { "cell_type": "code", "execution_count": 10, "id": "4afceded-7178-4c05-8fa6-9f2085e6a344", "metadata": {}, "outputs": [], "source": [ "# There's a particular dictionary structure that's required to describe our function:\n", "\n", "price_function = {\n", " \"name\": \"get_ticket_price\",\n", " \"description\": \"Get the price of a return ticket to the destination city. Call this whenever you need to know the ticket price, for example when a customer asks 'How much is a ticket to this city'\",\n", " \"parameters\": {\n", " \"type\": \"object\",\n", " \"properties\": {\n", " \"destination_city\": {\n", " \"type\": \"string\",\n", " \"description\": \"The city that the customer wants to travel to\",\n", " },\n", " },\n", " \"required\": [\"destination_city\"],\n", " \"additionalProperties\": False\n", " }\n", "}\n", "\n", "# Book flight function description and properties\n", "\n", "book_flight_function = {\n", " \"name\": \"book_flight\",\n", " \"description\": \"Book a flight to the destination city. Call this whenever a customer wants to book a flight.\",\n", " \"parameters\": {\n", " \"type\": \"object\",\n", " \"properties\": {\n", " \"destination_city\": {\n", " \"type\": \"string\",\n", " \"description\": \"The city that the customer wants to travel to\",\n", " },\n", " \"departure_date\": {\n", " \"type\": \"string\",\n", " \"description\": \"The date of departure (YYYY-MM-DD)\",\n", " },\n", " \"return_date\": {\n", " \"type\": \"string\",\n", " \"description\": \"The date of return (YYYY-MM-DD)\",\n", " },\n", " \"passenger_name\": {\n", " \"type\": \"string\",\n", " \"description\": \"The name of the passenger\",\n", " },\n", " },\n", " \"required\": [\"destination_city\", \"departure_date\", \"return_date\", \"passenger_name\"],\n", " \"additionalProperties\": False\n", " }\n", "}\n", "\n", "sightseeing_function = {\n", " \"name\": \"sightseeing\",\n", " \"description\": \"Get the top sightseeing recommendations for the destination city. Call this whenever a customer asks 'What are the top things to do in this city'\",\n", " \"parameters\": {\n", " \"type\": \"object\",\n", " \"properties\": {\n", " \"destination_city\": {\n", " \"type\": \"string\",\n", " \"description\": \"The city that the customer wants to travel to\",\n", " },\n", " },\n", " \"required\": [\"destination_city\"],\n", " \"additionalProperties\": False\n", " }\n", "}" ] }, { "cell_type": "code", "execution_count": 11, "id": "bdca8679-935f-4e7f-97e6-e71a4d4f228c", "metadata": {}, "outputs": [], "source": [ "# And this is included in a list of tools:\n", "\n", "tools = [{\"type\": \"function\", \"function\": price_function}, \n", " {\"type\": \"function\", \"function\": book_flight_function},\n", " {\"type\": \"function\", \"function\": sightseeing_function}]" ] }, { "cell_type": "markdown", "id": "c3d3554f-b4e3-4ce7-af6f-68faa6dd2340", "metadata": {}, "source": [ "## Getting OpenAI to use our Tool\n", "\n", "There's some fiddly stuff to allow OpenAI \"to call our tool\"\n", "\n", "What we actually do is give the LLM the opportunity to inform us that it wants us to run the tool.\n", "\n", "Here's how the new chat function looks:" ] }, { "cell_type": "code", "execution_count": 13, "id": "ce9b0744-9c78-408d-b9df-9f6fd9ed78cf", "metadata": {}, "outputs": [], "source": [ "def chat(message, history):\n", " messages = [{\"role\": \"system\", \"content\": system_message}] + history + [{\"role\": \"user\", \"content\": message}]\n", " response = openai.chat.completions.create(model=MODEL, messages=messages, tools=tools)\n", "\n", " if response.choices[0].finish_reason==\"tool_calls\":\n", " message = response.choices[0].message\n", " response, city = handle_tool_call(message)\n", " messages.append(message)\n", " messages.append(response)\n", " response = openai.chat.completions.create(model=MODEL, messages=messages)\n", " \n", " return response.choices[0].message.content" ] }, { "cell_type": "code", "execution_count": 12, "id": "b0992986-ea09-4912-a076-8e5603ee631f", "metadata": {}, "outputs": [], "source": [ "# We have to write that function handle_tool_call:\n", "\n", "def handle_tool_call(message):\n", " tool_call = message.tool_calls[0]\n", " print(f\"Tool call: {tool_call}\")\n", " arguments = json.loads(tool_call.function.arguments)\n", " city = arguments.get('destination_city')\n", " price = get_ticket_price(city)\n", " book = get_booking(city)\n", " sightseeing = get_sightseeing(city)\n", " print (book)\n", " response = {\n", " \"role\": \"tool\",\n", " \"content\": json.dumps({\"destination_city\": city,\"price\": price, \"booking\": book, \"sightseeing\": sightseeing}),\n", " \"tool_call_id\": tool_call.id\n", " }\n", " return response, city" ] }, { "cell_type": "code", "execution_count": null, "id": "f4be8a71-b19e-4c2f-80df-f59ff2661f14", "metadata": {}, "outputs": [], "source": [ "gr.ChatInterface(fn=chat, type=\"messages\").launch()" ] }, { "cell_type": "markdown", "id": "473e5b39-da8f-4db1-83ae-dbaca2e9531e", "metadata": {}, "source": [ "# Let's go multi-modal!!\n", "\n", "We can use DALL-E-3, the image generation model behind GPT-4o, to make us some images\n", "\n", "Let's put this in a function called artist.\n", "\n", "### Price alert: each time I generate an image it costs about 4 cents - don't go crazy with images!" ] }, { "cell_type": "code", "execution_count": 14, "id": "2c27c4ba-8ed5-492f-add1-02ce9c81d34c", "metadata": {}, "outputs": [], "source": [ "# Some imports for handling images\n", "\n", "import base64\n", "from io import BytesIO\n", "from PIL import Image" ] }, { "cell_type": "code", "execution_count": 15, "id": "773a9f11-557e-43c9-ad50-56cbec3a0f8f", "metadata": {}, "outputs": [], "source": [ "def artist(city):\n", " image_response = openai.images.generate(\n", " model=\"dall-e-3\",\n", " prompt=f\"An image representing a vacation in {city}, showing tourist spots and everything unique about {city}, in a vibrant pop-art style\",\n", " size=\"1024x1024\",\n", " n=1,\n", " response_format=\"b64_json\",\n", " )\n", " image_base64 = image_response.data[0].b64_json\n", " image_data = base64.b64decode(image_base64)\n", " return Image.open(BytesIO(image_data))" ] }, { "cell_type": "code", "execution_count": null, "id": "d877c453-e7fb-482a-88aa-1a03f976b9e9", "metadata": {}, "outputs": [], "source": [ "image = artist(\"Athens\")\n", "display(image)" ] }, { "cell_type": "code", "execution_count": null, "id": "728a12c5-adc3-415d-bb05-82beb73b079b", "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "id": "f4975b87-19e9-4ade-a232-9b809ec75c9a", "metadata": {}, "source": [ "## Audio (NOTE - Audio is optional for this course - feel free to skip Audio if it causes trouble!)\n", "\n", "And let's make a function talker that uses OpenAI's speech model to generate Audio\n", "\n", "### Troubleshooting Audio issues\n", "\n", "If you have any problems running this code below (like a FileNotFound error, or a warning of a missing package), you may need to install FFmpeg, a very popular audio utility.\n", "\n", "**For PC Users**\n", "\n", "Detailed instructions are [here](https://chatgpt.com/share/6724efee-6b0c-8012-ac5e-72e2e3885905) and summary instructions:\n", "\n", "1. Download FFmpeg from the official website: https://ffmpeg.org/download.html\n", "\n", "2. Extract the downloaded files to a location on your computer (e.g., `C:\\ffmpeg`)\n", "\n", "3. Add the FFmpeg bin folder to your system PATH:\n", "- Right-click on 'This PC' or 'My Computer' and select 'Properties'\n", "- Click on 'Advanced system settings'\n", "- Click on 'Environment Variables'\n", "- Under 'System variables', find and edit 'Path'\n", "- Add a new entry with the path to your FFmpeg bin folder (e.g., `C:\\ffmpeg\\bin`)\n", "- Restart your command prompt, and within Jupyter Lab do Kernel -> Restart kernel, to pick up the changes\n", "\n", "4. Open a new command prompt and run this to make sure it's installed OK\n", "`ffmpeg -version`\n", "\n", "**For Mac Users**\n", "\n", "1. Install homebrew if you don't have it already by running this in a Terminal window and following any instructions: \n", "`/bin/bash -c \"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)\"`\n", "\n", "2. Then install FFmpeg with `brew install ffmpeg`\n", "\n", "3. Verify your installation with `ffmpeg -version` and if everything is good, within Jupyter Lab do Kernel -> Restart kernel to pick up the changes\n", "\n", "Message me or email me at ed@edwarddonner.com with any problems!" ] }, { "cell_type": "markdown", "id": "4cc90e80-c96e-4dd4-b9d6-386fe2b7e797", "metadata": {}, "source": [ "## To check you now have ffmpeg and can access it here\n", "\n", "Excecute the next cell to see if you get a version number. (Putting an exclamation mark before something in Jupyter Lab tells it to run it as a terminal command rather than python code).\n", "\n", "If this doesn't work, you may need to actually save and close down your Jupyter lab, and start it again from a new Terminal window (Mac) or Anaconda prompt (PC), remembering to activate the llms environment. This ensures you pick up ffmpeg.\n", "\n", "And if that doesn't work, please contact me!" ] }, { "cell_type": "code", "execution_count": 14, "id": "7b3be0fb-1d34-4693-ab6f-dbff190afcd7", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "'ffmpeg' is not recognized as an internal or external command,\n", "operable program or batch file.\n", "'ffprobe' is not recognized as an internal or external command,\n", "operable program or batch file.\n", "'ffplay' is not recognized as an internal or external command,\n", "operable program or batch file.\n" ] } ], "source": [ "!ffmpeg -version\n", "!ffprobe -version\n", "!ffplay -version" ] }, { "cell_type": "markdown", "id": "d91d3f8f-e505-4e3c-a87c-9e42ed823db6", "metadata": {}, "source": [ "# For Mac users - and possibly many PC users too\n", "\n", "This version should work fine for you. It might work for Windows users too, but you might get a Permissions error writing to a temp file. If so, see the next section!\n", "\n", "As always, if you have problems, please contact me! (You could also comment out the audio talker() in the later code if you're less interested in audio generation)" ] }, { "cell_type": "code", "execution_count": 16, "id": "ffbfe93b-5e86-4e68-ba71-b301cd5230db", "metadata": {}, "outputs": [], "source": [ "from pydub import AudioSegment\n", "from pydub.playback import play\n", "\n", "def talker(message):\n", " response = openai.audio.speech.create(\n", " model=\"tts-1\",\n", " voice=\"onyx\", # Also, try replacing onyx with alloy\n", " input=message\n", " )\n", " \n", " audio_stream = BytesIO(response.content)\n", " audio = AudioSegment.from_file(audio_stream, format=\"mp3\")\n", " play(audio)" ] }, { "cell_type": "code", "execution_count": 17, "id": "b88d775d-d357-4292-a1ad-5dc5ed567281", "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "c:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\site-packages\\pydub\\utils.py:198: RuntimeWarning: Couldn't find ffprobe or avprobe - defaulting to ffprobe, but may not work\n", " warn(\"Couldn't find ffprobe or avprobe - defaulting to ffprobe, but may not work\", RuntimeWarning)\n" ] }, { "ename": "FileNotFoundError", "evalue": "[WinError 2] The system cannot find the file specified", "output_type": "error", "traceback": [ "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[1;31mFileNotFoundError\u001b[0m Traceback (most recent call last)", "Cell \u001b[1;32mIn[17], line 1\u001b[0m\n\u001b[1;32m----> 1\u001b[0m \u001b[43mtalker\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mWell, hi there\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\n", "Cell \u001b[1;32mIn[16], line 12\u001b[0m, in \u001b[0;36mtalker\u001b[1;34m(message)\u001b[0m\n\u001b[0;32m 5\u001b[0m response \u001b[38;5;241m=\u001b[39m openai\u001b[38;5;241m.\u001b[39maudio\u001b[38;5;241m.\u001b[39mspeech\u001b[38;5;241m.\u001b[39mcreate(\n\u001b[0;32m 6\u001b[0m model\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mtts-1\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[0;32m 7\u001b[0m voice\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124monyx\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;66;03m# Also, try replacing onyx with alloy\u001b[39;00m\n\u001b[0;32m 8\u001b[0m \u001b[38;5;28minput\u001b[39m\u001b[38;5;241m=\u001b[39mmessage\n\u001b[0;32m 9\u001b[0m )\n\u001b[0;32m 11\u001b[0m audio_stream \u001b[38;5;241m=\u001b[39m BytesIO(response\u001b[38;5;241m.\u001b[39mcontent)\n\u001b[1;32m---> 12\u001b[0m audio \u001b[38;5;241m=\u001b[39m \u001b[43mAudioSegment\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mfrom_file\u001b[49m\u001b[43m(\u001b[49m\u001b[43maudio_stream\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mformat\u001b[39;49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mmp3\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\n\u001b[0;32m 13\u001b[0m play(audio)\n", "File \u001b[1;32mc:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\site-packages\\pydub\\audio_segment.py:728\u001b[0m, in \u001b[0;36mAudioSegment.from_file\u001b[1;34m(cls, file, format, codec, parameters, start_second, duration, **kwargs)\u001b[0m\n\u001b[0;32m 726\u001b[0m info \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[0;32m 727\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m--> 728\u001b[0m info \u001b[38;5;241m=\u001b[39m \u001b[43mmediainfo_json\u001b[49m\u001b[43m(\u001b[49m\u001b[43morig_file\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mread_ahead_limit\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mread_ahead_limit\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 729\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m info:\n\u001b[0;32m 730\u001b[0m audio_streams \u001b[38;5;241m=\u001b[39m [x \u001b[38;5;28;01mfor\u001b[39;00m x \u001b[38;5;129;01min\u001b[39;00m info[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mstreams\u001b[39m\u001b[38;5;124m'\u001b[39m]\n\u001b[0;32m 731\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m x[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mcodec_type\u001b[39m\u001b[38;5;124m'\u001b[39m] \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m'\u001b[39m\u001b[38;5;124maudio\u001b[39m\u001b[38;5;124m'\u001b[39m]\n", "File \u001b[1;32mc:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\site-packages\\pydub\\utils.py:274\u001b[0m, in \u001b[0;36mmediainfo_json\u001b[1;34m(filepath, read_ahead_limit)\u001b[0m\n\u001b[0;32m 271\u001b[0m file\u001b[38;5;241m.\u001b[39mclose()\n\u001b[0;32m 273\u001b[0m command \u001b[38;5;241m=\u001b[39m [prober, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124m-of\u001b[39m\u001b[38;5;124m'\u001b[39m, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124mjson\u001b[39m\u001b[38;5;124m'\u001b[39m] \u001b[38;5;241m+\u001b[39m command_args\n\u001b[1;32m--> 274\u001b[0m res \u001b[38;5;241m=\u001b[39m \u001b[43mPopen\u001b[49m\u001b[43m(\u001b[49m\u001b[43mcommand\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstdin\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstdin_parameter\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstdout\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mPIPE\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstderr\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mPIPE\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 275\u001b[0m output, stderr \u001b[38;5;241m=\u001b[39m res\u001b[38;5;241m.\u001b[39mcommunicate(\u001b[38;5;28minput\u001b[39m\u001b[38;5;241m=\u001b[39mstdin_data)\n\u001b[0;32m 276\u001b[0m output \u001b[38;5;241m=\u001b[39m output\u001b[38;5;241m.\u001b[39mdecode(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mutf-8\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124mignore\u001b[39m\u001b[38;5;124m'\u001b[39m)\n", "File \u001b[1;32mc:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\subprocess.py:1026\u001b[0m, in \u001b[0;36mPopen.__init__\u001b[1;34m(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, user, group, extra_groups, encoding, errors, text, umask, pipesize, process_group)\u001b[0m\n\u001b[0;32m 1022\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mtext_mode:\n\u001b[0;32m 1023\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstderr \u001b[38;5;241m=\u001b[39m io\u001b[38;5;241m.\u001b[39mTextIOWrapper(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstderr,\n\u001b[0;32m 1024\u001b[0m encoding\u001b[38;5;241m=\u001b[39mencoding, errors\u001b[38;5;241m=\u001b[39merrors)\n\u001b[1;32m-> 1026\u001b[0m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_execute_child\u001b[49m\u001b[43m(\u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mexecutable\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mpreexec_fn\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mclose_fds\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1027\u001b[0m \u001b[43m \u001b[49m\u001b[43mpass_fds\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcwd\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43menv\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1028\u001b[0m \u001b[43m \u001b[49m\u001b[43mstartupinfo\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcreationflags\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mshell\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1029\u001b[0m \u001b[43m \u001b[49m\u001b[43mp2cread\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mp2cwrite\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1030\u001b[0m \u001b[43m \u001b[49m\u001b[43mc2pread\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mc2pwrite\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1031\u001b[0m \u001b[43m \u001b[49m\u001b[43merrread\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43merrwrite\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1032\u001b[0m \u001b[43m \u001b[49m\u001b[43mrestore_signals\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1033\u001b[0m \u001b[43m \u001b[49m\u001b[43mgid\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mgids\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43muid\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mumask\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1034\u001b[0m \u001b[43m \u001b[49m\u001b[43mstart_new_session\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mprocess_group\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 1035\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m:\n\u001b[0;32m 1036\u001b[0m \u001b[38;5;66;03m# Cleanup if the child failed starting.\u001b[39;00m\n\u001b[0;32m 1037\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m f \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mfilter\u001b[39m(\u001b[38;5;28;01mNone\u001b[39;00m, (\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstdin, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstdout, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstderr)):\n", "File \u001b[1;32mc:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\subprocess.py:1538\u001b[0m, in \u001b[0;36mPopen._execute_child\u001b[1;34m(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, unused_restore_signals, unused_gid, unused_gids, unused_uid, unused_umask, unused_start_new_session, unused_process_group)\u001b[0m\n\u001b[0;32m 1536\u001b[0m \u001b[38;5;66;03m# Start the process\u001b[39;00m\n\u001b[0;32m 1537\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m-> 1538\u001b[0m hp, ht, pid, tid \u001b[38;5;241m=\u001b[39m \u001b[43m_winapi\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mCreateProcess\u001b[49m\u001b[43m(\u001b[49m\u001b[43mexecutable\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1539\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;66;43;03m# no special security\u001b[39;49;00m\n\u001b[0;32m 1540\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;28;43;01mNone\u001b[39;49;00m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43;01mNone\u001b[39;49;00m\u001b[43m,\u001b[49m\n\u001b[0;32m 1541\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;28;43mint\u001b[39;49m\u001b[43m(\u001b[49m\u001b[38;5;129;43;01mnot\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mclose_fds\u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1542\u001b[0m \u001b[43m \u001b[49m\u001b[43mcreationflags\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1543\u001b[0m \u001b[43m \u001b[49m\u001b[43menv\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1544\u001b[0m \u001b[43m \u001b[49m\u001b[43mcwd\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1545\u001b[0m \u001b[43m \u001b[49m\u001b[43mstartupinfo\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 1546\u001b[0m \u001b[38;5;28;01mfinally\u001b[39;00m:\n\u001b[0;32m 1547\u001b[0m \u001b[38;5;66;03m# Child is launched. Close the parent's copy of those pipe\u001b[39;00m\n\u001b[0;32m 1548\u001b[0m \u001b[38;5;66;03m# handles that only the child should have open. You need\u001b[39;00m\n\u001b[1;32m (...)\u001b[0m\n\u001b[0;32m 1551\u001b[0m \u001b[38;5;66;03m# pipe will not close when the child process exits and the\u001b[39;00m\n\u001b[0;32m 1552\u001b[0m \u001b[38;5;66;03m# ReadFile will hang.\u001b[39;00m\n\u001b[0;32m 1553\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_close_pipe_fds(p2cread, p2cwrite,\n\u001b[0;32m 1554\u001b[0m c2pread, c2pwrite,\n\u001b[0;32m 1555\u001b[0m errread, errwrite)\n", "\u001b[1;31mFileNotFoundError\u001b[0m: [WinError 2] The system cannot find the file specified" ] } ], "source": [ "talker(\"Well, hi there\")" ] }, { "cell_type": "markdown", "id": "ad89a9bd-bb1e-4bbb-a49a-83af5f500c24", "metadata": {}, "source": [ "# For Windows users (or any Mac users with problems above)\n", "\n", "## First try the Mac version above, but if you get a permissions error writing to a temp file, then this code should work instead.\n", "\n", "A collaboration between students Mark M. and Patrick H. and Claude got this resolved!\n", "\n", "Below are 4 variations - hopefully one of them will work on your PC. If not, message me please!\n", "\n", "And for Mac people - all 3 of the below work on my Mac too - please try these if the Mac version gave you problems.\n", "\n", "## PC Variation 1" ] }, { "cell_type": "code", "execution_count": null, "id": "d104b96a-02ca-4159-82fe-88e0452aa479", "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", " \n", " " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "\n", " \n", " " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import base64\n", "from io import BytesIO\n", "from PIL import Image\n", "from IPython.display import Audio, display\n", "\n", "def talker(message):\n", " response = openai.audio.speech.create(\n", " model=\"tts-1\",\n", " voice=\"onyx\",\n", " input=message)\n", "\n", " audio_stream = BytesIO(response.content)\n", " output_filename = \"output_audio.mp3\"\n", " with open(output_filename, \"wb\") as f:\n", " f.write(audio_stream.read())\n", "\n", " # Play the generated audio\n", " display(Audio(output_filename, autoplay=True))\n", "\n", "talker(\"Well, hi there\")" ] }, { "cell_type": "markdown", "id": "3a5d11f4-bbd3-43a1-904d-f684eb5f3e3a", "metadata": {}, "source": [ "## PC Variation 2" ] }, { "cell_type": "code", "execution_count": 19, "id": "d59c8ebd-79c5-498a-bdf2-3a1c50d91aa0", "metadata": {}, "outputs": [ { "ename": "FileNotFoundError", "evalue": "[WinError 2] The system cannot find the file specified", "output_type": "error", "traceback": [ "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[1;31mFileNotFoundError\u001b[0m Traceback (most recent call last)", "Cell \u001b[1;32mIn[19], line 36\u001b[0m\n\u001b[0;32m 33\u001b[0m audio \u001b[38;5;241m=\u001b[39m AudioSegment\u001b[38;5;241m.\u001b[39mfrom_file(audio_stream, \u001b[38;5;28mformat\u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mmp3\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[0;32m 34\u001b[0m play_audio(audio)\n\u001b[1;32m---> 36\u001b[0m \u001b[43mtalker\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mWell hi there\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\n", "Cell \u001b[1;32mIn[19], line 33\u001b[0m, in \u001b[0;36mtalker\u001b[1;34m(message)\u001b[0m\n\u001b[0;32m 27\u001b[0m response \u001b[38;5;241m=\u001b[39m openai\u001b[38;5;241m.\u001b[39maudio\u001b[38;5;241m.\u001b[39mspeech\u001b[38;5;241m.\u001b[39mcreate(\n\u001b[0;32m 28\u001b[0m model\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mtts-1\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[0;32m 29\u001b[0m voice\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124monyx\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;66;03m# Also, try replacing onyx with alloy\u001b[39;00m\n\u001b[0;32m 30\u001b[0m \u001b[38;5;28minput\u001b[39m\u001b[38;5;241m=\u001b[39mmessage\n\u001b[0;32m 31\u001b[0m )\n\u001b[0;32m 32\u001b[0m audio_stream \u001b[38;5;241m=\u001b[39m BytesIO(response\u001b[38;5;241m.\u001b[39mcontent)\n\u001b[1;32m---> 33\u001b[0m audio \u001b[38;5;241m=\u001b[39m \u001b[43mAudioSegment\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mfrom_file\u001b[49m\u001b[43m(\u001b[49m\u001b[43maudio_stream\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mformat\u001b[39;49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mmp3\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\n\u001b[0;32m 34\u001b[0m play_audio(audio)\n", "File \u001b[1;32mc:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\site-packages\\pydub\\audio_segment.py:728\u001b[0m, in \u001b[0;36mAudioSegment.from_file\u001b[1;34m(cls, file, format, codec, parameters, start_second, duration, **kwargs)\u001b[0m\n\u001b[0;32m 726\u001b[0m info \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[0;32m 727\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m--> 728\u001b[0m info \u001b[38;5;241m=\u001b[39m \u001b[43mmediainfo_json\u001b[49m\u001b[43m(\u001b[49m\u001b[43morig_file\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mread_ahead_limit\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mread_ahead_limit\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 729\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m info:\n\u001b[0;32m 730\u001b[0m audio_streams \u001b[38;5;241m=\u001b[39m [x \u001b[38;5;28;01mfor\u001b[39;00m x \u001b[38;5;129;01min\u001b[39;00m info[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mstreams\u001b[39m\u001b[38;5;124m'\u001b[39m]\n\u001b[0;32m 731\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m x[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mcodec_type\u001b[39m\u001b[38;5;124m'\u001b[39m] \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m'\u001b[39m\u001b[38;5;124maudio\u001b[39m\u001b[38;5;124m'\u001b[39m]\n", "File \u001b[1;32mc:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\site-packages\\pydub\\utils.py:274\u001b[0m, in \u001b[0;36mmediainfo_json\u001b[1;34m(filepath, read_ahead_limit)\u001b[0m\n\u001b[0;32m 271\u001b[0m file\u001b[38;5;241m.\u001b[39mclose()\n\u001b[0;32m 273\u001b[0m command \u001b[38;5;241m=\u001b[39m [prober, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124m-of\u001b[39m\u001b[38;5;124m'\u001b[39m, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124mjson\u001b[39m\u001b[38;5;124m'\u001b[39m] \u001b[38;5;241m+\u001b[39m command_args\n\u001b[1;32m--> 274\u001b[0m res \u001b[38;5;241m=\u001b[39m \u001b[43mPopen\u001b[49m\u001b[43m(\u001b[49m\u001b[43mcommand\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstdin\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstdin_parameter\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstdout\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mPIPE\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstderr\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mPIPE\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 275\u001b[0m output, stderr \u001b[38;5;241m=\u001b[39m res\u001b[38;5;241m.\u001b[39mcommunicate(\u001b[38;5;28minput\u001b[39m\u001b[38;5;241m=\u001b[39mstdin_data)\n\u001b[0;32m 276\u001b[0m output \u001b[38;5;241m=\u001b[39m output\u001b[38;5;241m.\u001b[39mdecode(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mutf-8\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124mignore\u001b[39m\u001b[38;5;124m'\u001b[39m)\n", "File \u001b[1;32mc:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\subprocess.py:1026\u001b[0m, in \u001b[0;36mPopen.__init__\u001b[1;34m(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, user, group, extra_groups, encoding, errors, text, umask, pipesize, process_group)\u001b[0m\n\u001b[0;32m 1022\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mtext_mode:\n\u001b[0;32m 1023\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstderr \u001b[38;5;241m=\u001b[39m io\u001b[38;5;241m.\u001b[39mTextIOWrapper(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstderr,\n\u001b[0;32m 1024\u001b[0m encoding\u001b[38;5;241m=\u001b[39mencoding, errors\u001b[38;5;241m=\u001b[39merrors)\n\u001b[1;32m-> 1026\u001b[0m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_execute_child\u001b[49m\u001b[43m(\u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mexecutable\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mpreexec_fn\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mclose_fds\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1027\u001b[0m \u001b[43m \u001b[49m\u001b[43mpass_fds\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcwd\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43menv\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1028\u001b[0m \u001b[43m \u001b[49m\u001b[43mstartupinfo\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcreationflags\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mshell\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1029\u001b[0m \u001b[43m \u001b[49m\u001b[43mp2cread\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mp2cwrite\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1030\u001b[0m \u001b[43m \u001b[49m\u001b[43mc2pread\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mc2pwrite\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1031\u001b[0m \u001b[43m \u001b[49m\u001b[43merrread\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43merrwrite\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1032\u001b[0m \u001b[43m \u001b[49m\u001b[43mrestore_signals\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1033\u001b[0m \u001b[43m \u001b[49m\u001b[43mgid\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mgids\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43muid\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mumask\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1034\u001b[0m \u001b[43m \u001b[49m\u001b[43mstart_new_session\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mprocess_group\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 1035\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m:\n\u001b[0;32m 1036\u001b[0m \u001b[38;5;66;03m# Cleanup if the child failed starting.\u001b[39;00m\n\u001b[0;32m 1037\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m f \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mfilter\u001b[39m(\u001b[38;5;28;01mNone\u001b[39;00m, (\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstdin, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstdout, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstderr)):\n", "File \u001b[1;32mc:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\subprocess.py:1538\u001b[0m, in \u001b[0;36mPopen._execute_child\u001b[1;34m(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, unused_restore_signals, unused_gid, unused_gids, unused_uid, unused_umask, unused_start_new_session, unused_process_group)\u001b[0m\n\u001b[0;32m 1536\u001b[0m \u001b[38;5;66;03m# Start the process\u001b[39;00m\n\u001b[0;32m 1537\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m-> 1538\u001b[0m hp, ht, pid, tid \u001b[38;5;241m=\u001b[39m \u001b[43m_winapi\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mCreateProcess\u001b[49m\u001b[43m(\u001b[49m\u001b[43mexecutable\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1539\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;66;43;03m# no special security\u001b[39;49;00m\n\u001b[0;32m 1540\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;28;43;01mNone\u001b[39;49;00m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43;01mNone\u001b[39;49;00m\u001b[43m,\u001b[49m\n\u001b[0;32m 1541\u001b[0m \u001b[43m \u001b[49m\u001b[38;5;28;43mint\u001b[39;49m\u001b[43m(\u001b[49m\u001b[38;5;129;43;01mnot\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mclose_fds\u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1542\u001b[0m \u001b[43m \u001b[49m\u001b[43mcreationflags\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1543\u001b[0m \u001b[43m \u001b[49m\u001b[43menv\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1544\u001b[0m \u001b[43m \u001b[49m\u001b[43mcwd\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1545\u001b[0m \u001b[43m \u001b[49m\u001b[43mstartupinfo\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 1546\u001b[0m \u001b[38;5;28;01mfinally\u001b[39;00m:\n\u001b[0;32m 1547\u001b[0m \u001b[38;5;66;03m# Child is launched. Close the parent's copy of those pipe\u001b[39;00m\n\u001b[0;32m 1548\u001b[0m \u001b[38;5;66;03m# handles that only the child should have open. You need\u001b[39;00m\n\u001b[1;32m (...)\u001b[0m\n\u001b[0;32m 1551\u001b[0m \u001b[38;5;66;03m# pipe will not close when the child process exits and the\u001b[39;00m\n\u001b[0;32m 1552\u001b[0m \u001b[38;5;66;03m# ReadFile will hang.\u001b[39;00m\n\u001b[0;32m 1553\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_close_pipe_fds(p2cread, p2cwrite,\n\u001b[0;32m 1554\u001b[0m c2pread, c2pwrite,\n\u001b[0;32m 1555\u001b[0m errread, errwrite)\n", "\u001b[1;31mFileNotFoundError\u001b[0m: [WinError 2] The system cannot find the file specified" ] } ], "source": [ "import tempfile\n", "import subprocess\n", "from io import BytesIO\n", "from pydub import AudioSegment\n", "import time\n", "\n", "def play_audio(audio_segment):\n", " temp_dir = tempfile.gettempdir()\n", " temp_path = os.path.join(temp_dir, \"temp_audio.wav\")\n", " try:\n", " audio_segment.export(temp_path, format=\"wav\")\n", " time.sleep(3) # Student Dominic found that this was needed. You could also try commenting out to see if not needed on your PC\n", " subprocess.call([\n", " \"ffplay\",\n", " \"-nodisp\",\n", " \"-autoexit\",\n", " \"-hide_banner\",\n", " temp_path\n", " ], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)\n", " finally:\n", " try:\n", " os.remove(temp_path)\n", " except Exception:\n", " pass\n", " \n", "def talker(message):\n", " response = openai.audio.speech.create(\n", " model=\"tts-1\",\n", " voice=\"onyx\", # Also, try replacing onyx with alloy\n", " input=message\n", " )\n", " audio_stream = BytesIO(response.content)\n", " audio = AudioSegment.from_file(audio_stream, format=\"mp3\")\n", " play_audio(audio)\n", "\n", "talker(\"Well hi there\")" ] }, { "cell_type": "markdown", "id": "96f90e35-f71e-468e-afea-07b98f74dbcf", "metadata": {}, "source": [ "## PC Variation 3" ] }, { "cell_type": "code", "execution_count": 20, "id": "8597c7f8-7b50-44ad-9b31-db12375cd57b", "metadata": {}, "outputs": [ { "ename": "FileNotFoundError", "evalue": "[WinError 2] The system cannot find the file specified", "output_type": "error", "traceback": [ "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[1;31mFileNotFoundError\u001b[0m Traceback (most recent call last)", "Cell \u001b[1;32mIn[20], line 26\u001b[0m\n\u001b[0;32m 22\u001b[0m audio \u001b[38;5;241m=\u001b[39m AudioSegment\u001b[38;5;241m.\u001b[39mfrom_file(audio_stream, \u001b[38;5;28mformat\u001b[39m\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mmp3\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[0;32m 24\u001b[0m play(audio)\n\u001b[1;32m---> 26\u001b[0m \u001b[43mtalker\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mWell hi there\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\n", "Cell \u001b[1;32mIn[20], line 22\u001b[0m, in \u001b[0;36mtalker\u001b[1;34m(message)\u001b[0m\n\u001b[0;32m 15\u001b[0m response \u001b[38;5;241m=\u001b[39m openai\u001b[38;5;241m.\u001b[39maudio\u001b[38;5;241m.\u001b[39mspeech\u001b[38;5;241m.\u001b[39mcreate(\n\u001b[0;32m 16\u001b[0m model\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mtts-1\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[0;32m 17\u001b[0m voice\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124monyx\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;66;03m# Also, try replacing onyx with alloy\u001b[39;00m\n\u001b[0;32m 18\u001b[0m \u001b[38;5;28minput\u001b[39m\u001b[38;5;241m=\u001b[39mmessage\n\u001b[0;32m 19\u001b[0m )\n\u001b[0;32m 21\u001b[0m audio_stream \u001b[38;5;241m=\u001b[39m BytesIO(response\u001b[38;5;241m.\u001b[39mcontent)\n\u001b[1;32m---> 22\u001b[0m audio \u001b[38;5;241m=\u001b[39m \u001b[43mAudioSegment\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mfrom_file\u001b[49m\u001b[43m(\u001b[49m\u001b[43maudio_stream\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mformat\u001b[39;49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mmp3\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\n\u001b[0;32m 24\u001b[0m play(audio)\n", "File \u001b[1;32mc:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\site-packages\\pydub\\audio_segment.py:728\u001b[0m, in \u001b[0;36mAudioSegment.from_file\u001b[1;34m(cls, file, format, codec, parameters, start_second, duration, **kwargs)\u001b[0m\n\u001b[0;32m 726\u001b[0m info \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[0;32m 727\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m--> 728\u001b[0m info \u001b[38;5;241m=\u001b[39m \u001b[43mmediainfo_json\u001b[49m\u001b[43m(\u001b[49m\u001b[43morig_file\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mread_ahead_limit\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mread_ahead_limit\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 729\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m info:\n\u001b[0;32m 730\u001b[0m audio_streams \u001b[38;5;241m=\u001b[39m [x \u001b[38;5;28;01mfor\u001b[39;00m x \u001b[38;5;129;01min\u001b[39;00m info[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mstreams\u001b[39m\u001b[38;5;124m'\u001b[39m]\n\u001b[0;32m 731\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m x[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mcodec_type\u001b[39m\u001b[38;5;124m'\u001b[39m] \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m'\u001b[39m\u001b[38;5;124maudio\u001b[39m\u001b[38;5;124m'\u001b[39m]\n", "File \u001b[1;32mc:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\site-packages\\pydub\\utils.py:274\u001b[0m, in \u001b[0;36mmediainfo_json\u001b[1;34m(filepath, read_ahead_limit)\u001b[0m\n\u001b[0;32m 271\u001b[0m file\u001b[38;5;241m.\u001b[39mclose()\n\u001b[0;32m 273\u001b[0m command \u001b[38;5;241m=\u001b[39m [prober, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124m-of\u001b[39m\u001b[38;5;124m'\u001b[39m, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124mjson\u001b[39m\u001b[38;5;124m'\u001b[39m] \u001b[38;5;241m+\u001b[39m command_args\n\u001b[1;32m--> 274\u001b[0m res \u001b[38;5;241m=\u001b[39m \u001b[43mPopen\u001b[49m\u001b[43m(\u001b[49m\u001b[43mcommand\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstdin\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstdin_parameter\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstdout\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mPIPE\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstderr\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mPIPE\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 275\u001b[0m output, stderr \u001b[38;5;241m=\u001b[39m res\u001b[38;5;241m.\u001b[39mcommunicate(\u001b[38;5;28minput\u001b[39m\u001b[38;5;241m=\u001b[39mstdin_data)\n\u001b[0;32m 276\u001b[0m output \u001b[38;5;241m=\u001b[39m output\u001b[38;5;241m.\u001b[39mdecode(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mutf-8\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124mignore\u001b[39m\u001b[38;5;124m'\u001b[39m)\n", "File \u001b[1;32mc:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\subprocess.py:1026\u001b[0m, in \u001b[0;36mPopen.__init__\u001b[1;34m(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, user, group, extra_groups, encoding, errors, text, umask, pipesize, process_group)\u001b[0m\n\u001b[0;32m 1022\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mtext_mode:\n\u001b[0;32m 1023\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstderr \u001b[38;5;241m=\u001b[39m io\u001b[38;5;241m.\u001b[39mTextIOWrapper(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstderr,\n\u001b[0;32m 1024\u001b[0m encoding\u001b[38;5;241m=\u001b[39mencoding, errors\u001b[38;5;241m=\u001b[39merrors)\n\u001b[1;32m-> 1026\u001b[0m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_execute_child\u001b[49m\u001b[43m(\u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mexecutable\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mpreexec_fn\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mclose_fds\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1027\u001b[0m \u001b[43m \u001b[49m\u001b[43mpass_fds\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcwd\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43menv\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1028\u001b[0m \u001b[43m \u001b[49m\u001b[43mstartupinfo\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcreationflags\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mshell\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1029\u001b[0m \u001b[43m \u001b[49m\u001b[43mp2cread\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mp2cwrite\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1030\u001b[0m \u001b[43m \u001b[49m\u001b[43mc2pread\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mc2pwrite\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1031\u001b[0m \u001b[43m \u001b[49m\u001b[43merrread\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43merrwrite\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1032\u001b[0m \u001b[43m \u001b[49m\u001b[43mrestore_signals\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1033\u001b[0m \u001b[43m \u001b[49m\u001b[43mgid\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mgids\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43muid\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mumask\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1034\u001b[0m \u001b[43m \u001b[49m\u001b[43mstart_new_session\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mprocess_group\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 1035\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m:\n\u001b[0;32m 1036\u001b[0m \u001b[38;5;66;03m# Cleanup if the child failed starting.\u001b[39;00m\n\u001b[0;32m 1037\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m f \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mfilter\u001b[39m(\u001b[38;5;28;01mNone\u001b[39;00m, (\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstdin, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstdout, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstderr)):\n", "File \u001b[1;32mc:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\subprocess.py:1538\u001b[0m, in \u001b[0;36mPopen._execute_child\u001b[1;34m(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, unused_restore_signals, unused_gid, unused_gids, unused_uid, unused_umask, unused_start_new_session, unused_process_group)\u001b[0m\n\u001b[0;32m 1536\u001b[0m \u001b[38;5;66;03m# Start the process\u001b[39;00m\n\u001b[0;32m 1537\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m-> 1538\u001b[0m hp, ht, pid, tid \u001b[38;5;241m=\u001b[39m _winapi\u001b[38;5;241m.\u001b[39mCreateProcess(executable, args,\n\u001b[0;32m 1539\u001b[0m \u001b[38;5;66;03m# no special security\u001b[39;00m\n\u001b[0;32m 1540\u001b[0m \u001b[38;5;28;01mNone\u001b[39;00m, \u001b[38;5;28;01mNone\u001b[39;00m,\n\u001b[0;32m 1541\u001b[0m \u001b[38;5;28mint\u001b[39m(\u001b[38;5;129;01mnot\u001b[39;00m close_fds),\n\u001b[0;32m 1542\u001b[0m creationflags,\n\u001b[0;32m 1543\u001b[0m env,\n\u001b[0;32m 1544\u001b[0m cwd,\n\u001b[0;32m 1545\u001b[0m startupinfo)\n\u001b[0;32m 1546\u001b[0m \u001b[38;5;28;01mfinally\u001b[39;00m:\n\u001b[0;32m 1547\u001b[0m \u001b[38;5;66;03m# Child is launched. Close the parent's copy of those pipe\u001b[39;00m\n\u001b[0;32m 1548\u001b[0m \u001b[38;5;66;03m# handles that only the child should have open. You need\u001b[39;00m\n\u001b[1;32m (...)\u001b[0m\n\u001b[0;32m 1551\u001b[0m \u001b[38;5;66;03m# pipe will not close when the child process exits and the\u001b[39;00m\n\u001b[0;32m 1552\u001b[0m \u001b[38;5;66;03m# ReadFile will hang.\u001b[39;00m\n\u001b[0;32m 1553\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_close_pipe_fds(p2cread, p2cwrite,\n\u001b[0;32m 1554\u001b[0m c2pread, c2pwrite,\n\u001b[0;32m 1555\u001b[0m errread, errwrite)\n", "\u001b[1;31mFileNotFoundError\u001b[0m: [WinError 2] The system cannot find the file specified" ] } ], "source": [ "import os\n", "from pydub import AudioSegment\n", "from pydub.playback import play\n", "from io import BytesIO\n", "\n", "def talker(message):\n", " # Set a custom directory for temporary files on Windows\n", " custom_temp_dir = os.path.expanduser(\"~/Documents/temp_audio\")\n", " os.environ['TEMP'] = custom_temp_dir # You can also use 'TMP' if necessary\n", " \n", " # Create the folder if it doesn't exist\n", " if not os.path.exists(custom_temp_dir):\n", " os.makedirs(custom_temp_dir)\n", " \n", " response = openai.audio.speech.create(\n", " model=\"tts-1\",\n", " voice=\"onyx\", # Also, try replacing onyx with alloy\n", " input=message\n", " )\n", " \n", " audio_stream = BytesIO(response.content)\n", " audio = AudioSegment.from_file(audio_stream, format=\"mp3\")\n", "\n", " play(audio)\n", "\n", "talker(\"Well hi there\")" ] }, { "cell_type": "markdown", "id": "e821224c-b069-4f9b-9535-c15fdb0e411c", "metadata": {}, "source": [ "## PC Variation 4\n", "\n", "### Let's try a completely different sound library\n", "\n", "First run the next cell to install a new library, then try the cell below it." ] }, { "cell_type": "code", "execution_count": null, "id": "69d3c0d9-afcc-49e3-b829-9c9869d8b472", "metadata": {}, "outputs": [], "source": [ "!pip install simpleaudio" ] }, { "cell_type": "code", "execution_count": 21, "id": "28f9cc99-36b7-4554-b3f4-f2012f614a13", "metadata": {}, "outputs": [], "source": [ "from pydub import AudioSegment\n", "from io import BytesIO\n", "import tempfile\n", "import os\n", "import simpleaudio as sa\n", "\n", "def talker(message):\n", " response = openai.audio.speech.create(\n", " model=\"tts-1\",\n", " voice=\"onyx\", # Also, try replacing onyx with alloy\n", " input=message\n", " )\n", " \n", " audio_stream = BytesIO(response.content)\n", " audio = AudioSegment.from_file(audio_stream, format=\"mp3\")\n", "\n", " # Create a temporary file in a folder where you have write permissions\n", " with tempfile.NamedTemporaryFile(suffix=\".wav\", delete=False, dir=os.path.expanduser(\"~/Documents\")) as temp_audio_file:\n", " temp_file_name = temp_audio_file.name\n", " audio.export(temp_file_name, format=\"wav\")\n", " \n", " # Load and play audio using simpleaudio\n", " wave_obj = sa.WaveObject.from_wave_file(temp_file_name)\n", " play_obj = wave_obj.play()\n", " play_obj.wait_done() # Wait for playback to finish\n", "\n", " # Clean up the temporary file afterward\n", " os.remove(temp_file_name)\n", " \n" ] }, { "cell_type": "code", "execution_count": 22, "id": "0d248b46", "metadata": {}, "outputs": [ { "ename": "FileNotFoundError", "evalue": "[WinError 2] The system cannot find the file specified", "output_type": "error", "traceback": [ "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[1;31mFileNotFoundError\u001b[0m Traceback (most recent call last)", "Cell \u001b[1;32mIn[22], line 1\u001b[0m\n\u001b[1;32m----> 1\u001b[0m \u001b[43mtalker\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mWell hi there\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\n", "Cell \u001b[1;32mIn[21], line 15\u001b[0m, in \u001b[0;36mtalker\u001b[1;34m(message)\u001b[0m\n\u001b[0;32m 8\u001b[0m response \u001b[38;5;241m=\u001b[39m openai\u001b[38;5;241m.\u001b[39maudio\u001b[38;5;241m.\u001b[39mspeech\u001b[38;5;241m.\u001b[39mcreate(\n\u001b[0;32m 9\u001b[0m model\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mtts-1\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[0;32m 10\u001b[0m voice\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124monyx\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;66;03m# Also, try replacing onyx with alloy\u001b[39;00m\n\u001b[0;32m 11\u001b[0m \u001b[38;5;28minput\u001b[39m\u001b[38;5;241m=\u001b[39mmessage\n\u001b[0;32m 12\u001b[0m )\n\u001b[0;32m 14\u001b[0m audio_stream \u001b[38;5;241m=\u001b[39m BytesIO(response\u001b[38;5;241m.\u001b[39mcontent)\n\u001b[1;32m---> 15\u001b[0m audio \u001b[38;5;241m=\u001b[39m \u001b[43mAudioSegment\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mfrom_file\u001b[49m\u001b[43m(\u001b[49m\u001b[43maudio_stream\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mformat\u001b[39;49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mmp3\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\n\u001b[0;32m 17\u001b[0m \u001b[38;5;66;03m# Create a temporary file in a folder where you have write permissions\u001b[39;00m\n\u001b[0;32m 18\u001b[0m \u001b[38;5;28;01mwith\u001b[39;00m tempfile\u001b[38;5;241m.\u001b[39mNamedTemporaryFile(suffix\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m.wav\u001b[39m\u001b[38;5;124m\"\u001b[39m, delete\u001b[38;5;241m=\u001b[39m\u001b[38;5;28;01mFalse\u001b[39;00m, \u001b[38;5;28mdir\u001b[39m\u001b[38;5;241m=\u001b[39mos\u001b[38;5;241m.\u001b[39mpath\u001b[38;5;241m.\u001b[39mexpanduser(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m~/Documents\u001b[39m\u001b[38;5;124m\"\u001b[39m)) \u001b[38;5;28;01mas\u001b[39;00m temp_audio_file:\n", "File \u001b[1;32mc:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\site-packages\\pydub\\audio_segment.py:728\u001b[0m, in \u001b[0;36mAudioSegment.from_file\u001b[1;34m(cls, file, format, codec, parameters, start_second, duration, **kwargs)\u001b[0m\n\u001b[0;32m 726\u001b[0m info \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mNone\u001b[39;00m\n\u001b[0;32m 727\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m--> 728\u001b[0m info \u001b[38;5;241m=\u001b[39m \u001b[43mmediainfo_json\u001b[49m\u001b[43m(\u001b[49m\u001b[43morig_file\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mread_ahead_limit\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mread_ahead_limit\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 729\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m info:\n\u001b[0;32m 730\u001b[0m audio_streams \u001b[38;5;241m=\u001b[39m [x \u001b[38;5;28;01mfor\u001b[39;00m x \u001b[38;5;129;01min\u001b[39;00m info[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mstreams\u001b[39m\u001b[38;5;124m'\u001b[39m]\n\u001b[0;32m 731\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m x[\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mcodec_type\u001b[39m\u001b[38;5;124m'\u001b[39m] \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m'\u001b[39m\u001b[38;5;124maudio\u001b[39m\u001b[38;5;124m'\u001b[39m]\n", "File \u001b[1;32mc:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\site-packages\\pydub\\utils.py:274\u001b[0m, in \u001b[0;36mmediainfo_json\u001b[1;34m(filepath, read_ahead_limit)\u001b[0m\n\u001b[0;32m 271\u001b[0m file\u001b[38;5;241m.\u001b[39mclose()\n\u001b[0;32m 273\u001b[0m command \u001b[38;5;241m=\u001b[39m [prober, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124m-of\u001b[39m\u001b[38;5;124m'\u001b[39m, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124mjson\u001b[39m\u001b[38;5;124m'\u001b[39m] \u001b[38;5;241m+\u001b[39m command_args\n\u001b[1;32m--> 274\u001b[0m res \u001b[38;5;241m=\u001b[39m \u001b[43mPopen\u001b[49m\u001b[43m(\u001b[49m\u001b[43mcommand\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstdin\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mstdin_parameter\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstdout\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mPIPE\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mstderr\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43mPIPE\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 275\u001b[0m output, stderr \u001b[38;5;241m=\u001b[39m res\u001b[38;5;241m.\u001b[39mcommunicate(\u001b[38;5;28minput\u001b[39m\u001b[38;5;241m=\u001b[39mstdin_data)\n\u001b[0;32m 276\u001b[0m output \u001b[38;5;241m=\u001b[39m output\u001b[38;5;241m.\u001b[39mdecode(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mutf-8\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124mignore\u001b[39m\u001b[38;5;124m'\u001b[39m)\n", "File \u001b[1;32mc:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\subprocess.py:1026\u001b[0m, in \u001b[0;36mPopen.__init__\u001b[1;34m(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, user, group, extra_groups, encoding, errors, text, umask, pipesize, process_group)\u001b[0m\n\u001b[0;32m 1022\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mtext_mode:\n\u001b[0;32m 1023\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstderr \u001b[38;5;241m=\u001b[39m io\u001b[38;5;241m.\u001b[39mTextIOWrapper(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstderr,\n\u001b[0;32m 1024\u001b[0m encoding\u001b[38;5;241m=\u001b[39mencoding, errors\u001b[38;5;241m=\u001b[39merrors)\n\u001b[1;32m-> 1026\u001b[0m \u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_execute_child\u001b[49m\u001b[43m(\u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mexecutable\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mpreexec_fn\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mclose_fds\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1027\u001b[0m \u001b[43m \u001b[49m\u001b[43mpass_fds\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcwd\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43menv\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1028\u001b[0m \u001b[43m \u001b[49m\u001b[43mstartupinfo\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcreationflags\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mshell\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1029\u001b[0m \u001b[43m \u001b[49m\u001b[43mp2cread\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mp2cwrite\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1030\u001b[0m \u001b[43m \u001b[49m\u001b[43mc2pread\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mc2pwrite\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1031\u001b[0m \u001b[43m \u001b[49m\u001b[43merrread\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43merrwrite\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1032\u001b[0m \u001b[43m \u001b[49m\u001b[43mrestore_signals\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1033\u001b[0m \u001b[43m \u001b[49m\u001b[43mgid\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mgids\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43muid\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mumask\u001b[49m\u001b[43m,\u001b[49m\n\u001b[0;32m 1034\u001b[0m \u001b[43m \u001b[49m\u001b[43mstart_new_session\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mprocess_group\u001b[49m\u001b[43m)\u001b[49m\n\u001b[0;32m 1035\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m:\n\u001b[0;32m 1036\u001b[0m \u001b[38;5;66;03m# Cleanup if the child failed starting.\u001b[39;00m\n\u001b[0;32m 1037\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m f \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28mfilter\u001b[39m(\u001b[38;5;28;01mNone\u001b[39;00m, (\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstdin, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstdout, \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mstderr)):\n", "File \u001b[1;32mc:\\Users\\dsinanis\\AppData\\Local\\anaconda3\\envs\\llms\\Lib\\subprocess.py:1538\u001b[0m, in \u001b[0;36mPopen._execute_child\u001b[1;34m(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, unused_restore_signals, unused_gid, unused_gids, unused_uid, unused_umask, unused_start_new_session, unused_process_group)\u001b[0m\n\u001b[0;32m 1536\u001b[0m \u001b[38;5;66;03m# Start the process\u001b[39;00m\n\u001b[0;32m 1537\u001b[0m \u001b[38;5;28;01mtry\u001b[39;00m:\n\u001b[1;32m-> 1538\u001b[0m hp, ht, pid, tid \u001b[38;5;241m=\u001b[39m _winapi\u001b[38;5;241m.\u001b[39mCreateProcess(executable, args,\n\u001b[0;32m 1539\u001b[0m \u001b[38;5;66;03m# no special security\u001b[39;00m\n\u001b[0;32m 1540\u001b[0m \u001b[38;5;28;01mNone\u001b[39;00m, \u001b[38;5;28;01mNone\u001b[39;00m,\n\u001b[0;32m 1541\u001b[0m \u001b[38;5;28mint\u001b[39m(\u001b[38;5;129;01mnot\u001b[39;00m close_fds),\n\u001b[0;32m 1542\u001b[0m creationflags,\n\u001b[0;32m 1543\u001b[0m env,\n\u001b[0;32m 1544\u001b[0m cwd,\n\u001b[0;32m 1545\u001b[0m startupinfo)\n\u001b[0;32m 1546\u001b[0m \u001b[38;5;28;01mfinally\u001b[39;00m:\n\u001b[0;32m 1547\u001b[0m \u001b[38;5;66;03m# Child is launched. Close the parent's copy of those pipe\u001b[39;00m\n\u001b[0;32m 1548\u001b[0m \u001b[38;5;66;03m# handles that only the child should have open. You need\u001b[39;00m\n\u001b[1;32m (...)\u001b[0m\n\u001b[0;32m 1551\u001b[0m \u001b[38;5;66;03m# pipe will not close when the child process exits and the\u001b[39;00m\n\u001b[0;32m 1552\u001b[0m \u001b[38;5;66;03m# ReadFile will hang.\u001b[39;00m\n\u001b[0;32m 1553\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_close_pipe_fds(p2cread, p2cwrite,\n\u001b[0;32m 1554\u001b[0m c2pread, c2pwrite,\n\u001b[0;32m 1555\u001b[0m errread, errwrite)\n", "\u001b[1;31mFileNotFoundError\u001b[0m: [WinError 2] The system cannot find the file specified" ] } ], "source": [ "talker(\"Well hi there\")" ] }, { "cell_type": "markdown", "id": "7986176b-cd04-495f-a47f-e057b0e462ed", "metadata": {}, "source": [ "## PC Users - if none of those 4 variations worked!\n", "\n", "Please get in touch with me. I'm sorry this is causing problems! We'll figure it out.\n", "\n", "Alternatively: playing audio from your PC isn't super-critical for this course, and you can feel free to focus on image generation and skip audio for now, or come back to it later." ] }, { "cell_type": "markdown", "id": "1d48876d-c4fa-46a8-a04f-f9fadf61fb0d", "metadata": {}, "source": [ "# Our Agent Framework\n", "\n", "The term 'Agentic AI' and Agentization is an umbrella term that refers to a number of techniques, such as:\n", "\n", "1. Breaking a complex problem into smaller steps, with multiple LLMs carrying out specialized tasks\n", "2. The ability for LLMs to use Tools to give them additional capabilities\n", "3. The 'Agent Environment' which allows Agents to collaborate\n", "4. An LLM can act as the Planner, dividing bigger tasks into smaller ones for the specialists\n", "5. The concept of an Agent having autonomy / agency, beyond just responding to a prompt - such as Memory\n", "\n", "We're showing 1 and 2 here, and to a lesser extent 3 and 5. In week 8 we will do the lot!" ] }, { "cell_type": "code", "execution_count": null, "id": "ba820c95-02f5-499e-8f3c-8727ee0a6c0c", "metadata": {}, "outputs": [], "source": [ "def chat(history):\n", " messages = [{\"role\": \"system\", \"content\": system_message}] + history\n", " response = openai.chat.completions.create(model=MODEL, messages=messages, tools=tools)\n", " image = None\n", " \n", " if response.choices[0].finish_reason==\"tool_calls\":\n", " message = response.choices[0].message\n", " response, city = handle_tool_call(message)\n", " messages.append(message)\n", " messages.append(response)\n", " image = artist(city)\n", " response = openai.chat.completions.create(model=MODEL, messages=messages)\n", " \n", " reply = response.choices[0].message.content\n", " history += [{\"role\":\"assistant\", \"content\":reply}]\n", "\n", " # Comment out or delete the next line if you'd rather skip Audio for now..\n", " # It worked for me only with the first variation of the talker function\n", " talker(reply)\n", " \n", " return history, image" ] }, { "cell_type": "code", "execution_count": 20, "id": "f38d0d27-33bf-4992-a2e5-5dbed973cde7", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "* Running on local URL: http://127.0.0.1:7866\n", "\n", "To create a public link, set `share=True` in `launch()`.\n" ] }, { "data": { "text/html": [ "
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# More involved Gradio code as we're not using the preset Chat interface!\n", "# Passing in inbrowser=True in the last line will cause a Gradio window to pop up immediately.\n", "\n", "with gr.Blocks() as ui:\n", " with gr.Row():\n", " chatbot = gr.Chatbot(height=500, type=\"messages\")\n", " image_output = gr.Image(height=500)\n", " with gr.Row():\n", " entry = gr.Textbox(label=\"Chat with our AI Assistant:\")\n", " with gr.Row():\n", " clear = gr.Button(\"Clear\")\n", "\n", " def do_entry(message, history):\n", " history += [{\"role\":\"user\", \"content\":message}]\n", " return \"\", history\n", "\n", " entry.submit(do_entry, inputs=[entry, chatbot], outputs=[entry, chatbot]).then(\n", " chat, inputs=chatbot, outputs=[chatbot, image_output]\n", " )\n", " clear.click(lambda: None, inputs=None, outputs=chatbot, queue=False)\n", "\n", "ui.launch(inbrowser=True)" ] }, { "cell_type": "markdown", "id": "226643d2-73e4-4252-935d-86b8019e278a", "metadata": {}, "source": [ "# Exercises and Business Applications\n", "\n", "Add in more tools - perhaps to simulate actually booking a flight. A student has done this and provided their example in the community contributions folder.\n", "\n", "Next: take this and apply it to your business. Make a multi-modal AI assistant with tools that could carry out an activity for your work. A customer support assistant? New employee onboarding assistant? So many possibilities! Also, see the week2 end of week Exercise in the separate Notebook." ] }, { "cell_type": "markdown", "id": "7e795560-1867-42db-a256-a23b844e6fbe", "metadata": {}, "source": [ "\n", " \n", " \n", " \n", " \n", "
\n", " \n", " \n", "

I have a special request for you

\n", " \n", " My editor tells me that it makes a HUGE difference when students rate this course on Udemy - it's one of the main ways that Udemy decides whether to show it to others. If you're able to take a minute to rate this, I'd be so very grateful! And regardless - always please reach out to me at ed@edwarddonner.com if I can help at any point.\n", " \n", "
" ] } ], "metadata": { "kernelspec": { "display_name": "llms", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.11" } }, "nbformat": 4, "nbformat_minor": 5 }