You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 

480 lines
16 KiB

{
"cells": [
{
"cell_type": "markdown",
"id": "ddfa9ae6-69fe-444a-b994-8c4c5970a7ec",
"metadata": {},
"source": [
"# Project - Airline AI Assistant\n",
"\n",
"We'll now bring together what we've learned to make an AI Customer Support assistant for an Airline"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8b50bbe2-c0b1-49c3-9a5c-1ba7efa2bcb4",
"metadata": {},
"outputs": [],
"source": [
"# imports\n",
"\n",
"import os\n",
"import json\n",
"from dotenv import load_dotenv\n",
"from openai import OpenAI\n",
"import gradio as gr"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "747e8786-9da8-4342-b6c9-f5f69c2e22ae",
"metadata": {},
"outputs": [],
"source": [
"# Initialization\n",
"\n",
"load_dotenv()\n",
"os.environ['OPENAI_API_KEY'] = os.getenv('OPENAI_API_KEY', 'your-key-if-not-using-env')\n",
"MODEL = \"gpt-4o-mini\"\n",
"openai = OpenAI()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0a521d84-d07c-49ab-a0df-d6451499ed97",
"metadata": {},
"outputs": [],
"source": [
"system_message = \"You are a helpful assistant for an Airline called FlightAI. \"\n",
"system_message += \"Give short, courteous answers, no more than 1 sentence. \"\n",
"system_message += \"Always be accurate. If you don't know the answer, say so.\""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "61a2a15d-b559-4844-b377-6bd5cb4949f6",
"metadata": {},
"outputs": [],
"source": [
"def chat(message, history):\n",
" messages = [{\"role\": \"system\", \"content\": system_message}]\n",
" for human, assistant in history:\n",
" messages.append({\"role\": \"user\", \"content\": human})\n",
" messages.append({\"role\": \"assistant\", \"content\": assistant})\n",
" messages.append({\"role\": \"user\", \"content\": message})\n",
" response = openai.chat.completions.create(model=MODEL, messages=messages)\n",
" return response.choices[0].message.content\n",
"\n",
"gr.ChatInterface(fn=chat).launch()"
]
},
{
"cell_type": "markdown",
"id": "36bedabf-a0a7-4985-ad8e-07ed6a55a3a4",
"metadata": {},
"source": [
"## Tools\n",
"\n",
"Tools are an incredibly powerful feature provided by the frontier LLMs.\n",
"\n",
"With tools, you can write a function, and have the LLM call that function as part of its response.\n",
"\n",
"Sounds almost spooky.. we're giving it the power to run code on our machine?\n",
"\n",
"Well, kinda."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0696acb1-0b05-4dc2-80d5-771be04f1fb2",
"metadata": {},
"outputs": [],
"source": [
"# Let's start by making a useful function\n",
"\n",
"ticket_prices = {\"london\": \"$799\", \"paris\": \"$899\", \"tokyo\": \"$1400\", \"berlin\": \"$499\"}\n",
"\n",
"def get_ticket_price(destination_city):\n",
" print(f\"Tool get_ticket_price called for {destination_city}\")\n",
" city = destination_city.lower()\n",
" return ticket_prices.get(city, \"Unknown\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "80ca4e09-6287-4d3f-997d-fa6afbcf6c85",
"metadata": {},
"outputs": [],
"source": [
"get_ticket_price(\"London\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4afceded-7178-4c05-8fa6-9f2085e6a344",
"metadata": {},
"outputs": [],
"source": [
"# There's a particular dictionary structure that's required to describe our function:\n",
"\n",
"price_function = {\n",
" \"name\": \"get_ticket_price\",\n",
" \"description\": \"Get the price of a return ticket to the destination city. Call this whenever you need to know the ticket price, for example when a customer asks 'How much is a ticket to this city'\",\n",
" \"parameters\": {\n",
" \"type\": \"object\",\n",
" \"properties\": {\n",
" \"destination_city\": {\n",
" \"type\": \"string\",\n",
" \"description\": \"The city that the customer wants to travel to\",\n",
" },\n",
" },\n",
" \"required\": [\"destination_city\"],\n",
" \"additionalProperties\": False\n",
" }\n",
"}"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bdca8679-935f-4e7f-97e6-e71a4d4f228c",
"metadata": {},
"outputs": [],
"source": [
"# And this is included in a list of tools:\n",
"\n",
"tools = [{\"type\": \"function\", \"function\": price_function}]"
]
},
{
"cell_type": "markdown",
"id": "c3d3554f-b4e3-4ce7-af6f-68faa6dd2340",
"metadata": {},
"source": [
"## Getting OpenAI to use our Tool\n",
"\n",
"There's some fiddly stuff to allow OpenAI \"to call our tool\"\n",
"\n",
"What we actually do is give the LLM the opportunity to inform us that it wants us to run the tool.\n",
"\n",
"Here's how the new chat function looks:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ce9b0744-9c78-408d-b9df-9f6fd9ed78cf",
"metadata": {},
"outputs": [],
"source": [
"def chat(message, history):\n",
" messages = [{\"role\": \"system\", \"content\": system_message}]\n",
" for human, assistant in history:\n",
" messages.append({\"role\": \"user\", \"content\": human})\n",
" messages.append({\"role\": \"assistant\", \"content\": assistant})\n",
" messages.append({\"role\": \"user\", \"content\": message})\n",
" response = openai.chat.completions.create(model=MODEL, messages=messages, tools=tools)\n",
"\n",
" if response.choices[0].finish_reason==\"tool_calls\":\n",
" message = response.choices[0].message\n",
" response, city = handle_tool_call(message)\n",
" messages.append(message)\n",
" messages.append(response)\n",
" response = openai.chat.completions.create(model=MODEL, messages=messages)\n",
" \n",
" return response.choices[0].message.content"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b0992986-ea09-4912-a076-8e5603ee631f",
"metadata": {},
"outputs": [],
"source": [
"# We have to write that function handle_tool_call:\n",
"\n",
"def handle_tool_call(message):\n",
" tool_call = message.tool_calls[0]\n",
" arguments = json.loads(tool_call.function.arguments)\n",
" city = arguments.get('destination_city')\n",
" price = get_ticket_price(city)\n",
" response = {\n",
" \"role\": \"tool\",\n",
" \"content\": json.dumps({\"destination_city\": city,\"price\": price}),\n",
" \"tool_call_id\": message.tool_calls[0].id\n",
" }\n",
" return response, city"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f4be8a71-b19e-4c2f-80df-f59ff2661f14",
"metadata": {},
"outputs": [],
"source": [
"gr.ChatInterface(fn=chat).launch()"
]
},
{
"cell_type": "markdown",
"id": "473e5b39-da8f-4db1-83ae-dbaca2e9531e",
"metadata": {},
"source": [
"# Let's go multi-modal!!\n",
"\n",
"We can use DALL-E-3, the image generation model behind GPT-4o, to make us some images\n",
"\n",
"Let's put this in a function called artist.\n",
"\n",
"### Price alert: each time I generate an image it costs about 4c - don't go crazy with images!"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2c27c4ba-8ed5-492f-add1-02ce9c81d34c",
"metadata": {},
"outputs": [],
"source": [
"# Some imports for handling images\n",
"\n",
"import base64\n",
"from io import BytesIO\n",
"from PIL import Image"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "773a9f11-557e-43c9-ad50-56cbec3a0f8f",
"metadata": {},
"outputs": [],
"source": [
"def artist(city):\n",
" image_response = openai.images.generate(\n",
" model=\"dall-e-3\",\n",
" prompt=f\"An image representing a vacation in {city}, showing tourist spots and everything unique about {city}, in a vibrant pop-art style\",\n",
" size=\"1024x1024\",\n",
" n=1,\n",
" response_format=\"b64_json\",\n",
" )\n",
" image_base64 = image_response.data[0].b64_json\n",
" image_data = base64.b64decode(image_base64)\n",
" return Image.open(BytesIO(image_data))"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d877c453-e7fb-482a-88aa-1a03f976b9e9",
"metadata": {},
"outputs": [],
"source": [
"image = artist(\"New York City\")\n",
"display(image)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "728a12c5-adc3-415d-bb05-82beb73b079b",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"id": "f4975b87-19e9-4ade-a232-9b809ec75c9a",
"metadata": {},
"source": [
"## Audio\n",
"\n",
"And let's make a function talker that uses OpenAI's speech model to generate Audio\n",
"\n",
"### Troubleshooting Audio issues\n",
"\n",
"If you have any problems running this code below (like a FileNotFound error, or a warning of a missing package), you may need to install FFmpeg, a very popular audio utility.\n",
"\n",
"**For PC Users**\n",
"\n",
"1. Download FFmpeg from the official website: https://ffmpeg.org/download.html\n",
"\n",
"2. Extract the downloaded files to a location on your computer (e.g., `C:\\ffmpeg`)\n",
"\n",
"3. Add the FFmpeg bin folder to your system PATH:\n",
"- Right-click on 'This PC' or 'My Computer' and select 'Properties'\n",
"- Click on 'Advanced system settings'\n",
"- Click on 'Environment Variables'\n",
"- Under 'System variables', find and edit 'Path'\n",
"- Add a new entry with the path to your FFmpeg bin folder (e.g., `C:\\ffmpeg\\bin`)\n",
"- Restart your command prompt, and within Jupyter Lab do Kernel -> Restart kernel, to pick up the changes\n",
"\n",
"4. Open a new command prompt and run this to make sure it's installed OK\n",
"`ffmpeg -version`\n",
"\n",
"**For Mac Users**\n",
"\n",
"1. Install homebrew if you don't have it already by running this in a Terminal window and following any instructions: \n",
"`/bin/bash -c \"$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)\"`\n",
"\n",
"2. Then install FFmpeg with `brew install ffmpeg`\n",
"\n",
"3. Verify your installation with `ffmpeg -version` and if everything is good, within Jupyter Lab do Kernel -> Restart kernel to pick up the changes\n",
"\n",
"Message me or email me at ed@edwarddonner.com with any problems!"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ffbfe93b-5e86-4e68-ba71-b301cd5230db",
"metadata": {},
"outputs": [],
"source": [
"from pydub import AudioSegment\n",
"from pydub.playback import play\n",
"\n",
"def talker(message):\n",
" response = openai.audio.speech.create(\n",
" model=\"tts-1\",\n",
" voice=\"onyx\", # Also, try replacing onyx with alloy\n",
" input=message\n",
" )\n",
" \n",
" audio_stream = BytesIO(response.content)\n",
" audio = AudioSegment.from_file(audio_stream, format=\"mp3\")\n",
" play(audio)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b88d775d-d357-4292-a1ad-5dc5ed567281",
"metadata": {},
"outputs": [],
"source": [
"talker(\"Well, hi there\")"
]
},
{
"cell_type": "markdown",
"id": "1d48876d-c4fa-46a8-a04f-f9fadf61fb0d",
"metadata": {},
"source": [
"# Our Agent Framework\n",
"\n",
"The term 'Agentic AI' and Agentization is an umbrella term that refers to a number of techniques, such as:\n",
"\n",
"1. Breaking a complex problem into smaller steps, with multiple LLMs carrying out specialized tasks\n",
"2. The ability for LLMs to use Tools to give them additional capabilities\n",
"3. The 'Agent Environment' which allows Agents to collaborate\n",
"4. An LLM can act as the Planner, dividing bigger tasks into smaller ones for the specialists\n",
"5. The concept of an Agent having autonomy / agency, beyond just responding to a prompt - such as Memory\n",
"\n",
"We're showing 1 and 2 here, and to a lesser extent 3 and 5. In week 8 we will do the lot!"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ba820c95-02f5-499e-8f3c-8727ee0a6c0c",
"metadata": {},
"outputs": [],
"source": [
"def chat(message, history):\n",
" image = None\n",
" conversation = [{\"role\": \"system\", \"content\": system_message}]\n",
" for human, assistant in history:\n",
" conversation.append({\"role\": \"user\", \"content\": human})\n",
" conversation.append({\"role\": \"assistant\", \"content\": assistant})\n",
" conversation.append({\"role\": \"user\", \"content\": message})\n",
" response = openai.chat.completions.create(model=MODEL, messages=conversation, tools=tools)\n",
"\n",
" if response.choices[0].finish_reason==\"tool_calls\":\n",
" message = tool_call = response.choices[0].message\n",
" response, city = handle_tool_call(message)\n",
" conversation.append(message)\n",
" conversation.append(response)\n",
" image = artist(city)\n",
" response = openai.chat.completions.create(model=MODEL, messages=conversation)\n",
"\n",
" reply = response.choices[0].message.content\n",
" talker(reply)\n",
" return reply, image"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f38d0d27-33bf-4992-a2e5-5dbed973cde7",
"metadata": {},
"outputs": [],
"source": [
"# More involved Gradio code as we're not using the preset Chat interface\n",
"\n",
"with gr.Blocks() as ui:\n",
" with gr.Row():\n",
" chatbot = gr.Chatbot(height=500)\n",
" image_output = gr.Image(height=500)\n",
" with gr.Row():\n",
" msg = gr.Textbox(label=\"Chat with our AI Assistant:\")\n",
" with gr.Row():\n",
" clear = gr.Button(\"Clear\")\n",
"\n",
" def user(user_message, history):\n",
" return \"\", history + [[user_message, None]]\n",
"\n",
" def bot(history):\n",
" user_message = history[-1][0]\n",
" bot_message, image = chat(user_message, history[:-1])\n",
" history[-1][1] = bot_message\n",
" return history, image\n",
"\n",
" msg.submit(user, [msg, chatbot], [msg, chatbot], queue=False).then(\n",
" bot, chatbot, [chatbot, image_output]\n",
" )\n",
" clear.click(lambda: None, None, chatbot, queue=False)\n",
"\n",
"ui.launch()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b0b12548-951d-4e7c-8e77-803a92271855",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.10"
}
},
"nbformat": 4,
"nbformat_minor": 5
}