1 changed files with 762 additions and 0 deletions
@ -0,0 +1,762 @@
|
||||
{ |
||||
"cells": [ |
||||
{ |
||||
"cell_type": "markdown", |
||||
"id": "ddfa9ae6-69fe-444a-b994-8c4c5970a7ec", |
||||
"metadata": {}, |
||||
"source": [ |
||||
"# Project - Airline AI Assistant\n", |
||||
"\n", |
||||
"We'll now bring together what we've learned to make an AI Customer Support assistant for an Airline" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "code", |
||||
"execution_count": null, |
||||
"id": "8b50bbe2-c0b1-49c3-9a5c-1ba7efa2bcb4", |
||||
"metadata": {}, |
||||
"outputs": [], |
||||
"source": [ |
||||
"# imports\n", |
||||
"\n", |
||||
"import os\n", |
||||
"import json\n", |
||||
"from dotenv import load_dotenv\n", |
||||
"from openai import OpenAI\n", |
||||
"import gradio as gr" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "code", |
||||
"execution_count": null, |
||||
"id": "747e8786-9da8-4342-b6c9-f5f69c2e22ae", |
||||
"metadata": {}, |
||||
"outputs": [], |
||||
"source": [ |
||||
"# Initialization\n", |
||||
"\n", |
||||
"load_dotenv()\n", |
||||
"\n", |
||||
"openai_api_key = os.getenv('OPENAI_API_KEY')\n", |
||||
"if openai_api_key:\n", |
||||
" print(f\"OpenAI API Key exists and begins {openai_api_key[:8]}\")\n", |
||||
"else:\n", |
||||
" print(\"OpenAI API Key not set\")\n", |
||||
" \n", |
||||
"MODEL = \"gpt-4o-mini\"\n", |
||||
"openai = OpenAI()" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "code", |
||||
"execution_count": null, |
||||
"id": "0a521d84-d07c-49ab-a0df-d6451499ed97", |
||||
"metadata": {}, |
||||
"outputs": [], |
||||
"source": [ |
||||
"system_message = \"You are a helpful assistant for an Airline called FlightAI. \"\n", |
||||
"system_message += \"Give short, courteous answers, no more than 1 sentence. \"\n", |
||||
"system_message += \"Always be accurate. If you don't know the answer, say so.\"" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "code", |
||||
"execution_count": null, |
||||
"id": "61a2a15d-b559-4844-b377-6bd5cb4949f6", |
||||
"metadata": {}, |
||||
"outputs": [], |
||||
"source": [ |
||||
"# This function looks rather simpler than the one from my video, because we're taking advantage of the latest Gradio updates\n", |
||||
"\n", |
||||
"def chat(message, history):\n", |
||||
" messages = [{\"role\": \"system\", \"content\": system_message}] + history + [{\"role\": \"user\", \"content\": message}]\n", |
||||
" response = openai.chat.completions.create(model=MODEL, messages=messages)\n", |
||||
" return response.choices[0].message.content\n", |
||||
"\n", |
||||
"gr.ChatInterface(fn=chat, type=\"messages\").launch()" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "markdown", |
||||
"id": "36bedabf-a0a7-4985-ad8e-07ed6a55a3a4", |
||||
"metadata": {}, |
||||
"source": [ |
||||
"## Tools\n", |
||||
"\n", |
||||
"Tools are an incredibly powerful feature provided by the frontier LLMs.\n", |
||||
"\n", |
||||
"With tools, you can write a function, and have the LLM call that function as part of its response.\n", |
||||
"\n", |
||||
"Sounds almost spooky.. we're giving it the power to run code on our machine?\n", |
||||
"\n", |
||||
"Well, kinda." |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "code", |
||||
"execution_count": null, |
||||
"id": "0696acb1-0b05-4dc2-80d5-771be04f1fb2", |
||||
"metadata": {}, |
||||
"outputs": [], |
||||
"source": [ |
||||
"# Let's start by making a useful function\n", |
||||
"\n", |
||||
"ticket_prices = {\"london\": \"$799\", \"paris\": \"$899\", \"tokyo\": \"$1400\", \"berlin\": \"$499\"}\n", |
||||
"\n", |
||||
"def get_ticket_price(destination_city):\n", |
||||
" print(f\"Tool get_ticket_price called for {destination_city}\")\n", |
||||
" city = destination_city.lower()\n", |
||||
" return ticket_prices.get(city, \"Unknown\")" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "code", |
||||
"execution_count": null, |
||||
"id": "80ca4e09-6287-4d3f-997d-fa6afbcf6c85", |
||||
"metadata": {}, |
||||
"outputs": [], |
||||
"source": [ |
||||
"get_ticket_price(\"London\")" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "code", |
||||
"execution_count": null, |
||||
"id": "4afceded-7178-4c05-8fa6-9f2085e6a344", |
||||
"metadata": {}, |
||||
"outputs": [], |
||||
"source": [ |
||||
"# There's a particular dictionary structure that's required to describe our function:\n", |
||||
"\n", |
||||
"price_function = {\n", |
||||
" \"name\": \"get_ticket_price\",\n", |
||||
" \"description\": \"Get the price of a return ticket to the destination city. Call this whenever you need to know the ticket price, for example when a customer asks 'How much is a ticket to this city'\",\n", |
||||
" \"parameters\": {\n", |
||||
" \"type\": \"object\",\n", |
||||
" \"properties\": {\n", |
||||
" \"destination_city\": {\n", |
||||
" \"type\": \"string\",\n", |
||||
" \"description\": \"The city that the customer wants to travel to\",\n", |
||||
" },\n", |
||||
" },\n", |
||||
" \"required\": [\"destination_city\"],\n", |
||||
" \"additionalProperties\": False\n", |
||||
" }\n", |
||||
"}" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "code", |
||||
"execution_count": null, |
||||
"id": "bdca8679-935f-4e7f-97e6-e71a4d4f228c", |
||||
"metadata": {}, |
||||
"outputs": [], |
||||
"source": [ |
||||
"# And this is included in a list of tools:\n", |
||||
"\n", |
||||
"tools = [{\"type\": \"function\", \"function\": price_function}]" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "markdown", |
||||
"id": "c3d3554f-b4e3-4ce7-af6f-68faa6dd2340", |
||||
"metadata": {}, |
||||
"source": [ |
||||
"## Getting OpenAI to use our Tool\n", |
||||
"\n", |
||||
"There's some fiddly stuff to allow OpenAI \"to call our tool\"\n", |
||||
"\n", |
||||
"What we actually do is give the LLM the opportunity to inform us that it wants us to run the tool.\n", |
||||
"\n", |
||||
"Here's how the new chat function looks:" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "code", |
||||
"execution_count": null, |
||||
"id": "ce9b0744-9c78-408d-b9df-9f6fd9ed78cf", |
||||
"metadata": {}, |
||||
"outputs": [], |
||||
"source": [ |
||||
"def chat(message, history):\n", |
||||
" messages = [{\"role\": \"system\", \"content\": system_message}] + history + [{\"role\": \"user\", \"content\": message}]\n", |
||||
" response = openai.chat.completions.create(model=MODEL, messages=messages, tools=tools)\n", |
||||
"\n", |
||||
" if response.choices[0].finish_reason==\"tool_calls\":\n", |
||||
" message = response.choices[0].message\n", |
||||
" response, city = handle_tool_call(message)\n", |
||||
" messages.append(message)\n", |
||||
" messages.append(response)\n", |
||||
" response = openai.chat.completions.create(model=MODEL, messages=messages)\n", |
||||
" \n", |
||||
" return response.choices[0].message.content" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "code", |
||||
"execution_count": null, |
||||
"id": "b0992986-ea09-4912-a076-8e5603ee631f", |
||||
"metadata": {}, |
||||
"outputs": [], |
||||
"source": [ |
||||
"# We have to write that function handle_tool_call:\n", |
||||
"\n", |
||||
"def handle_tool_call(message):\n", |
||||
"\n", |
||||
" available_functions = {\n", |
||||
" \"get_ticket_price\": get_ticket_price,\n", |
||||
" }\n", |
||||
"\n", |
||||
" tool_call = message.tool_calls[0]\n", |
||||
" function_to_call = available_functions.get(tool_call.function.name)\n", |
||||
" arguments = json.loads(tool_call.function.arguments)\n", |
||||
" city = arguments.get('destination_city')\n", |
||||
" price = function_to_call(city)\n", |
||||
" response = {\n", |
||||
" \"role\": \"tool\",\n", |
||||
" \"content\": json.dumps({\"destination_city\": city,\"price\": price}),\n", |
||||
" \"tool_call_id\": tool_call.id\n", |
||||
" }\n", |
||||
" return response, city" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "code", |
||||
"execution_count": null, |
||||
"id": "f4be8a71-b19e-4c2f-80df-f59ff2661f14", |
||||
"metadata": {}, |
||||
"outputs": [], |
||||
"source": [ |
||||
"gr.ChatInterface(fn=chat, type=\"messages\").launch()" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "markdown", |
||||
"id": "473e5b39-da8f-4db1-83ae-dbaca2e9531e", |
||||
"metadata": {}, |
||||
"source": [ |
||||
"# Let's go multi-modal!!\n", |
||||
"\n", |
||||
"We can use DALL-E-3, the image generation model behind GPT-4o, to make us some images\n", |
||||
"\n", |
||||
"Let's put this in a function called artist.\n", |
||||
"\n", |
||||
"### Price alert: each time I generate an image it costs about 4 cents - don't go crazy with images!" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "code", |
||||
"execution_count": null, |
||||
"id": "2c27c4ba-8ed5-492f-add1-02ce9c81d34c", |
||||
"metadata": {}, |
||||
"outputs": [], |
||||
"source": [ |
||||
"# Some imports for handling images\n", |
||||
"\n", |
||||
"import base64\n", |
||||
"from io import BytesIO\n", |
||||
"from PIL import Image" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "code", |
||||
"execution_count": null, |
||||
"id": "773a9f11-557e-43c9-ad50-56cbec3a0f8f", |
||||
"metadata": {}, |
||||
"outputs": [], |
||||
"source": [ |
||||
"def artist(city):\n", |
||||
" image_response = openai.images.generate(\n", |
||||
" model=\"dall-e-3\",\n", |
||||
" prompt=f\"An image representing a vacation in {city}, showing tourist spots and everything unique about {city}, in a vibrant pop-art style\",\n", |
||||
" size=\"1024x1024\",\n", |
||||
" n=1,\n", |
||||
" response_format=\"b64_json\",\n", |
||||
" )\n", |
||||
" image_base64 = image_response.data[0].b64_json\n", |
||||
" image_data = base64.b64decode(image_base64)\n", |
||||
" return Image.open(BytesIO(image_data))" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "code", |
||||
"execution_count": null, |
||||
"id": "d877c453-e7fb-482a-88aa-1a03f976b9e9", |
||||
"metadata": {}, |
||||
"outputs": [], |
||||
"source": [ |
||||
"image = artist(\"New York City\")\n", |
||||
"display(image)" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "code", |
||||
"execution_count": null, |
||||
"id": "728a12c5-adc3-415d-bb05-82beb73b079b", |
||||
"metadata": {}, |
||||
"outputs": [], |
||||
"source": [] |
||||
}, |
||||
{ |
||||
"cell_type": "markdown", |
||||
"id": "f4975b87-19e9-4ade-a232-9b809ec75c9a", |
||||
"metadata": {}, |
||||
"source": [ |
||||
"## Audio (NOTE - Audio is optional for this course - feel free to skip Audio if it causes trouble!)\n", |
||||
"\n", |
||||
"And let's make a function talker that uses OpenAI's speech model to generate Audio\n", |
||||
"\n", |
||||
"### Troubleshooting Audio issues\n", |
||||
"\n", |
||||
"If you have any problems running this code below (like a FileNotFound error, or a warning of a missing package), you may need to install FFmpeg, a very popular audio utility.\n", |
||||
"\n", |
||||
"**For PC Users**\n", |
||||
"\n", |
||||
"Detailed instructions are [here](https://chatgpt.com/share/6724efee-6b0c-8012-ac5e-72e2e3885905) and summary instructions:\n", |
||||
"\n", |
||||
"1. Download FFmpeg from the official website: https://ffmpeg.org/download.html\n", |
||||
"\n", |
||||
"2. Extract the downloaded files to a location on your computer (e.g., `C:\\ffmpeg`)\n", |
||||
"\n", |
||||
"3. Add the FFmpeg bin folder to your system PATH:\n", |
||||
"- Right-click on 'This PC' or 'My Computer' and select 'Properties'\n", |
||||
"- Click on 'Advanced system settings'\n", |
||||
"- Click on 'Environment Variables'\n", |
||||
"- Under 'System variables', find and edit 'Path'\n", |
||||
"- Add a new entry with the path to your FFmpeg bin folder (e.g., `C:\\ffmpeg\\bin`)\n", |
||||
"- Restart your command prompt, and within Jupyter Lab do Kernel -> Restart kernel, to pick up the changes\n", |
||||
"\n", |
||||
"4. Open a new command prompt and run this to make sure it's installed OK\n", |
||||
"`ffmpeg -version`\n", |
||||
"\n", |
||||
"**For Mac Users**\n", |
||||
"\n", |
||||
"1. Install homebrew if you don't have it already by running this in a Terminal window and following any instructions: \n", |
||||
"`/bin/bash -c \"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)\"`\n", |
||||
"\n", |
||||
"2. Then install FFmpeg with `brew install ffmpeg`\n", |
||||
"\n", |
||||
"3. Verify your installation with `ffmpeg -version` and if everything is good, within Jupyter Lab do Kernel -> Restart kernel to pick up the changes\n", |
||||
"\n", |
||||
"Message me or email me at ed@edwarddonner.com with any problems!" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "markdown", |
||||
"id": "4cc90e80-c96e-4dd4-b9d6-386fe2b7e797", |
||||
"metadata": {}, |
||||
"source": [ |
||||
"## To check you now have ffmpeg and can access it here\n", |
||||
"\n", |
||||
"Excecute the next cell to see if you get a version number. (Putting an exclamation mark before something in Jupyter Lab tells it to run it as a terminal command rather than python code).\n", |
||||
"\n", |
||||
"If this doesn't work, you may need to actually save and close down your Jupyter lab, and start it again from a new Terminal window (Mac) or Anaconda prompt (PC), remembering to activate the llms environment. This ensures you pick up ffmpeg.\n", |
||||
"\n", |
||||
"And if that doesn't work, please contact me!" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "code", |
||||
"execution_count": null, |
||||
"id": "7b3be0fb-1d34-4693-ab6f-dbff190afcd7", |
||||
"metadata": {}, |
||||
"outputs": [], |
||||
"source": [ |
||||
"!ffmpeg -version\n", |
||||
"!ffprobe -version\n", |
||||
"!ffplay -version" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "markdown", |
||||
"id": "d91d3f8f-e505-4e3c-a87c-9e42ed823db6", |
||||
"metadata": {}, |
||||
"source": [ |
||||
"# For Mac users - and possibly many PC users too\n", |
||||
"\n", |
||||
"This version should work fine for you. It might work for Windows users too, but you might get a Permissions error writing to a temp file. If so, see the next section!\n", |
||||
"\n", |
||||
"As always, if you have problems, please contact me! (You could also comment out the audio talker() in the later code if you're less interested in audio generation)" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "code", |
||||
"execution_count": null, |
||||
"id": "ffbfe93b-5e86-4e68-ba71-b301cd5230db", |
||||
"metadata": {}, |
||||
"outputs": [], |
||||
"source": [ |
||||
"from pydub import AudioSegment\n", |
||||
"from pydub.playback import play\n", |
||||
"\n", |
||||
"def talker(message):\n", |
||||
" response = openai.audio.speech.create(\n", |
||||
" model=\"tts-1\",\n", |
||||
" voice=\"onyx\", # Also, try replacing onyx with alloy\n", |
||||
" input=message\n", |
||||
" )\n", |
||||
" \n", |
||||
" audio_stream = BytesIO(response.content)\n", |
||||
" audio = AudioSegment.from_file(audio_stream, format=\"mp3\")\n", |
||||
" play(audio)" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "code", |
||||
"execution_count": null, |
||||
"id": "b88d775d-d357-4292-a1ad-5dc5ed567281", |
||||
"metadata": {}, |
||||
"outputs": [], |
||||
"source": [ |
||||
"talker(\"Well, hi there\")" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "markdown", |
||||
"id": "ad89a9bd-bb1e-4bbb-a49a-83af5f500c24", |
||||
"metadata": {}, |
||||
"source": [ |
||||
"# For Windows users (or any Mac users with problems above)\n", |
||||
"\n", |
||||
"## First try the Mac version above, but if you get a permissions error writing to a temp file, then this code should work instead.\n", |
||||
"\n", |
||||
"A collaboration between students Mark M. and Patrick H. and Claude got this resolved!\n", |
||||
"\n", |
||||
"Below are 4 variations - hopefully one of them will work on your PC. If not, message me please!\n", |
||||
"\n", |
||||
"And for Mac people - all 3 of the below work on my Mac too - please try these if the Mac version gave you problems.\n", |
||||
"\n", |
||||
"## PC Variation 1" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "code", |
||||
"execution_count": null, |
||||
"id": "d104b96a-02ca-4159-82fe-88e0452aa479", |
||||
"metadata": {}, |
||||
"outputs": [], |
||||
"source": [ |
||||
"import base64\n", |
||||
"from io import BytesIO\n", |
||||
"from PIL import Image\n", |
||||
"from IPython.display import Audio, display\n", |
||||
"\n", |
||||
"def talker(message):\n", |
||||
" response = openai.audio.speech.create(\n", |
||||
" model=\"tts-1\",\n", |
||||
" voice=\"onyx\",\n", |
||||
" input=message)\n", |
||||
"\n", |
||||
" audio_stream = BytesIO(response.content)\n", |
||||
" output_filename = \"output_audio.mp3\"\n", |
||||
" with open(output_filename, \"wb\") as f:\n", |
||||
" f.write(audio_stream.read())\n", |
||||
"\n", |
||||
" # Play the generated audio\n", |
||||
" display(Audio(output_filename, autoplay=True))\n", |
||||
"\n", |
||||
"talker(\"Well, hi there\")" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "markdown", |
||||
"id": "3a5d11f4-bbd3-43a1-904d-f684eb5f3e3a", |
||||
"metadata": {}, |
||||
"source": [ |
||||
"## PC Variation 2" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "code", |
||||
"execution_count": null, |
||||
"id": "d59c8ebd-79c5-498a-bdf2-3a1c50d91aa0", |
||||
"metadata": {}, |
||||
"outputs": [], |
||||
"source": [ |
||||
"import tempfile\n", |
||||
"import subprocess\n", |
||||
"from io import BytesIO\n", |
||||
"from pydub import AudioSegment\n", |
||||
"import time\n", |
||||
"\n", |
||||
"def play_audio(audio_segment):\n", |
||||
" temp_dir = tempfile.gettempdir()\n", |
||||
" temp_path = os.path.join(temp_dir, \"temp_audio.wav\")\n", |
||||
" try:\n", |
||||
" audio_segment.export(temp_path, format=\"wav\")\n", |
||||
" time.sleep(3) # Student Dominic found that this was needed. You could also try commenting out to see if not needed on your PC\n", |
||||
" subprocess.call([\n", |
||||
" \"ffplay\",\n", |
||||
" \"-nodisp\",\n", |
||||
" \"-autoexit\",\n", |
||||
" \"-hide_banner\",\n", |
||||
" temp_path\n", |
||||
" ], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)\n", |
||||
" finally:\n", |
||||
" try:\n", |
||||
" os.remove(temp_path)\n", |
||||
" except Exception:\n", |
||||
" pass\n", |
||||
" \n", |
||||
"def talker(message):\n", |
||||
" response = openai.audio.speech.create(\n", |
||||
" model=\"tts-1\",\n", |
||||
" voice=\"onyx\", # Also, try replacing onyx with alloy\n", |
||||
" input=message\n", |
||||
" )\n", |
||||
" audio_stream = BytesIO(response.content)\n", |
||||
" audio = AudioSegment.from_file(audio_stream, format=\"mp3\")\n", |
||||
" play_audio(audio)\n", |
||||
"\n", |
||||
"talker(\"Well hi there\")" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "markdown", |
||||
"id": "96f90e35-f71e-468e-afea-07b98f74dbcf", |
||||
"metadata": {}, |
||||
"source": [ |
||||
"## PC Variation 3" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "code", |
||||
"execution_count": null, |
||||
"id": "8597c7f8-7b50-44ad-9b31-db12375cd57b", |
||||
"metadata": {}, |
||||
"outputs": [], |
||||
"source": [ |
||||
"import os\n", |
||||
"from pydub import AudioSegment\n", |
||||
"from pydub.playback import play\n", |
||||
"from io import BytesIO\n", |
||||
"\n", |
||||
"def talker(message):\n", |
||||
" # Set a custom directory for temporary files on Windows\n", |
||||
" custom_temp_dir = os.path.expanduser(\"~/Documents/temp_audio\")\n", |
||||
" os.environ['TEMP'] = custom_temp_dir # You can also use 'TMP' if necessary\n", |
||||
" \n", |
||||
" # Create the folder if it doesn't exist\n", |
||||
" if not os.path.exists(custom_temp_dir):\n", |
||||
" os.makedirs(custom_temp_dir)\n", |
||||
" \n", |
||||
" response = openai.audio.speech.create(\n", |
||||
" model=\"tts-1\",\n", |
||||
" voice=\"onyx\", # Also, try replacing onyx with alloy\n", |
||||
" input=message\n", |
||||
" )\n", |
||||
" \n", |
||||
" audio_stream = BytesIO(response.content)\n", |
||||
" audio = AudioSegment.from_file(audio_stream, format=\"mp3\")\n", |
||||
"\n", |
||||
" play(audio)\n", |
||||
"\n", |
||||
"talker(\"Well hi there\")" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "markdown", |
||||
"id": "e821224c-b069-4f9b-9535-c15fdb0e411c", |
||||
"metadata": {}, |
||||
"source": [ |
||||
"## PC Variation 4\n", |
||||
"\n", |
||||
"### Let's try a completely different sound library\n", |
||||
"\n", |
||||
"First run the next cell to install a new library, then try the cell below it." |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "code", |
||||
"execution_count": null, |
||||
"id": "69d3c0d9-afcc-49e3-b829-9c9869d8b472", |
||||
"metadata": {}, |
||||
"outputs": [], |
||||
"source": [ |
||||
"!pip install simpleaudio" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "code", |
||||
"execution_count": null, |
||||
"id": "28f9cc99-36b7-4554-b3f4-f2012f614a13", |
||||
"metadata": {}, |
||||
"outputs": [], |
||||
"source": [ |
||||
"from pydub import AudioSegment\n", |
||||
"from io import BytesIO\n", |
||||
"import tempfile\n", |
||||
"import os\n", |
||||
"import simpleaudio as sa\n", |
||||
"\n", |
||||
"def talker(message):\n", |
||||
" response = openai.audio.speech.create(\n", |
||||
" model=\"tts-1\",\n", |
||||
" voice=\"onyx\", # Also, try replacing onyx with alloy\n", |
||||
" input=message\n", |
||||
" )\n", |
||||
" \n", |
||||
" audio_stream = BytesIO(response.content)\n", |
||||
" audio = AudioSegment.from_file(audio_stream, format=\"mp3\")\n", |
||||
"\n", |
||||
" # Create a temporary file in a folder where you have write permissions\n", |
||||
" with tempfile.NamedTemporaryFile(suffix=\".wav\", delete=False, dir=os.path.expanduser(\"~/Documents\")) as temp_audio_file:\n", |
||||
" temp_file_name = temp_audio_file.name\n", |
||||
" audio.export(temp_file_name, format=\"wav\")\n", |
||||
" \n", |
||||
" # Load and play audio using simpleaudio\n", |
||||
" wave_obj = sa.WaveObject.from_wave_file(temp_file_name)\n", |
||||
" play_obj = wave_obj.play()\n", |
||||
" play_obj.wait_done() # Wait for playback to finish\n", |
||||
"\n", |
||||
" # Clean up the temporary file afterward\n", |
||||
" os.remove(temp_file_name)\n", |
||||
" \n", |
||||
"talker(\"Well hi there\")" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "markdown", |
||||
"id": "7986176b-cd04-495f-a47f-e057b0e462ed", |
||||
"metadata": {}, |
||||
"source": [ |
||||
"## PC Users - if none of those 4 variations worked!\n", |
||||
"\n", |
||||
"Please get in touch with me. I'm sorry this is causing problems! We'll figure it out.\n", |
||||
"\n", |
||||
"Alternatively: playing audio from your PC isn't super-critical for this course, and you can feel free to focus on image generation and skip audio for now, or come back to it later." |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "markdown", |
||||
"id": "1d48876d-c4fa-46a8-a04f-f9fadf61fb0d", |
||||
"metadata": {}, |
||||
"source": [ |
||||
"# Our Agent Framework\n", |
||||
"\n", |
||||
"The term 'Agentic AI' and Agentization is an umbrella term that refers to a number of techniques, such as:\n", |
||||
"\n", |
||||
"1. Breaking a complex problem into smaller steps, with multiple LLMs carrying out specialized tasks\n", |
||||
"2. The ability for LLMs to use Tools to give them additional capabilities\n", |
||||
"3. The 'Agent Environment' which allows Agents to collaborate\n", |
||||
"4. An LLM can act as the Planner, dividing bigger tasks into smaller ones for the specialists\n", |
||||
"5. The concept of an Agent having autonomy / agency, beyond just responding to a prompt - such as Memory\n", |
||||
"\n", |
||||
"We're showing 1 and 2 here, and to a lesser extent 3 and 5. In week 8 we will do the lot!" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "code", |
||||
"execution_count": null, |
||||
"id": "ba820c95-02f5-499e-8f3c-8727ee0a6c0c", |
||||
"metadata": {}, |
||||
"outputs": [], |
||||
"source": [ |
||||
"def chat(history):\n", |
||||
" messages = [{\"role\": \"system\", \"content\": system_message}] + history\n", |
||||
" response = openai.chat.completions.create(model=MODEL, messages=messages, tools=tools)\n", |
||||
" image = None\n", |
||||
" \n", |
||||
" if response.choices[0].finish_reason==\"tool_calls\":\n", |
||||
" message = response.choices[0].message\n", |
||||
" response, city = handle_tool_call(message)\n", |
||||
" messages.append(message)\n", |
||||
" messages.append(response)\n", |
||||
" image = artist(city)\n", |
||||
" response = openai.chat.completions.create(model=MODEL, messages=messages)\n", |
||||
" \n", |
||||
" reply = response.choices[0].message.content\n", |
||||
" history += [{\"role\":\"assistant\", \"content\":reply}]\n", |
||||
"\n", |
||||
" # Comment out or delete the next line if you'd rather skip Audio for now..\n", |
||||
" talker(reply)\n", |
||||
" \n", |
||||
" return history, image" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "code", |
||||
"execution_count": null, |
||||
"id": "f38d0d27-33bf-4992-a2e5-5dbed973cde7", |
||||
"metadata": {}, |
||||
"outputs": [], |
||||
"source": [ |
||||
"# More involved Gradio code as we're not using the preset Chat interface!\n", |
||||
"# Passing in inbrowser=True in the last line will cause a Gradio window to pop up immediately.\n", |
||||
"\n", |
||||
"with gr.Blocks() as ui:\n", |
||||
" with gr.Row():\n", |
||||
" chatbot = gr.Chatbot(height=500, type=\"messages\")\n", |
||||
" image_output = gr.Image(height=500)\n", |
||||
" with gr.Row():\n", |
||||
" entry = gr.Textbox(label=\"Chat with our AI Assistant:\")\n", |
||||
" with gr.Row():\n", |
||||
" clear = gr.Button(\"Clear\")\n", |
||||
"\n", |
||||
" def do_entry(message, history):\n", |
||||
" history += [{\"role\":\"user\", \"content\":message}]\n", |
||||
" return \"\", history\n", |
||||
"\n", |
||||
" entry.submit(do_entry, inputs=[entry, chatbot], outputs=[entry, chatbot]).then(\n", |
||||
" chat, inputs=chatbot, outputs=[chatbot, image_output]\n", |
||||
" )\n", |
||||
" clear.click(lambda: None, inputs=None, outputs=chatbot, queue=False)\n", |
||||
"\n", |
||||
"ui.launch(inbrowser=True)" |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "markdown", |
||||
"id": "226643d2-73e4-4252-935d-86b8019e278a", |
||||
"metadata": {}, |
||||
"source": [ |
||||
"# Exercises and Business Applications\n", |
||||
"\n", |
||||
"Add in more tools - perhaps to simulate actually booking a flight. A student has done this and provided their example in the community contributions folder.\n", |
||||
"\n", |
||||
"Next: take this and apply it to your business. Make a multi-modal AI assistant with tools that could carry out an activity for your work. A customer support assistant? New employee onboarding assistant? So many possibilities! Also, see the week2 end of week Exercise in the separate Notebook." |
||||
] |
||||
}, |
||||
{ |
||||
"cell_type": "markdown", |
||||
"id": "7e795560-1867-42db-a256-a23b844e6fbe", |
||||
"metadata": {}, |
||||
"source": [ |
||||
"<table style=\"margin: 0; text-align: left;\">\n", |
||||
" <tr>\n", |
||||
" <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n", |
||||
" <img src=\"../thankyou.jpg\" width=\"150\" height=\"150\" style=\"display: block;\" />\n", |
||||
" </td>\n", |
||||
" <td>\n", |
||||
" <h2 style=\"color:#090;\">I have a special request for you</h2>\n", |
||||
" <span style=\"color:#090;\">\n", |
||||
" My editor tells me that it makes a HUGE difference when students rate this course on Udemy - it's one of the main ways that Udemy decides whether to show it to others. If you're able to take a minute to rate this, I'd be so very grateful! And regardless - always please reach out to me at ed@edwarddonner.com if I can help at any point.\n", |
||||
" </span>\n", |
||||
" </td>\n", |
||||
" </tr>\n", |
||||
"</table>" |
||||
] |
||||
} |
||||
], |
||||
"metadata": { |
||||
"kernelspec": { |
||||
"display_name": "Python 3 (ipykernel)", |
||||
"language": "python", |
||||
"name": "python3" |
||||
}, |
||||
"language_info": { |
||||
"codemirror_mode": { |
||||
"name": "ipython", |
||||
"version": 3 |
||||
}, |
||||
"file_extension": ".py", |
||||
"mimetype": "text/x-python", |
||||
"name": "python", |
||||
"nbconvert_exporter": "python", |
||||
"pygments_lexer": "ipython3", |
||||
"version": "3.11.11" |
||||
} |
||||
}, |
||||
"nbformat": 4, |
||||
"nbformat_minor": 5 |
||||
} |
Loading…
Reference in new issue