You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 

268 lines
6.5 KiB

{
"cells": [
{
"cell_type": "markdown",
"id": "e426cd04-c053-43e8-b505-63cee7956a53",
"metadata": {},
"source": [
"# Welcome to a very busy Week 8 folder\n",
"\n",
"## We have lots to do this week!\n",
"\n",
"We'll move at a faster pace than usual, particularly as you're becoming proficient LLM engineers.\n",
"\n",
"One quick admin thing: I've added a number of packages to the environment.yml file during September. To make sure you have the latest repo with the latest code, it's worth doing this from the `llm_engineering` project folder:\n",
"\n",
"```\n",
"git pull\n",
"conda env update --f environment.yml --prune\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bc0e1c1c-be6a-4395-bbbd-eeafc9330d7e",
"metadata": {},
"outputs": [],
"source": [
"# Just one import to start with!!\n",
"\n",
"import modal"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0d240622-8422-4c99-8464-c04d063e4cb6",
"metadata": {},
"outputs": [],
"source": [
"# !modal setup"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3b133701-f550-44a1-a67f-eb7ccc4769a9",
"metadata": {},
"outputs": [],
"source": [
"from hello import app, hello"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0f3f73ae-1295-49f3-9099-b8b41fc3429b",
"metadata": {},
"outputs": [],
"source": [
"with app.run(show_progress=False):\n",
" reply=hello.local()\n",
"reply"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c1d8c6f9-edc7-4e52-9b3a-c07d7cff1ac7",
"metadata": {},
"outputs": [],
"source": [
"with app.run(show_progress=False):\n",
" reply=hello.remote()\n",
"reply"
]
},
{
"cell_type": "markdown",
"id": "22e8d804-c027-45fb-8fef-06e7bba6295a",
"metadata": {},
"source": [
"# Before we move on -\n",
"\n",
"## We need to set your HuggingFace Token as a secret in Modal\n",
"\n",
"1. Go to modal.com, sign in and go to your dashboard\n",
"2. Click on Secrets in the nav bar\n",
"3. Create new secret, click on Hugging Face\n",
"4. Fill in your HF_TOKEN where it prompts you\n",
"\n",
"### And now back to business: time to work with Llama"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "cb8b6c41-8259-4329-b1c4-a1f67d26d1be",
"metadata": {},
"outputs": [],
"source": [
"from llama import app, generate"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "db4a718a-d95d-4f61-9688-c9df21d88fe6",
"metadata": {},
"outputs": [],
"source": [
"with modal.enable_output():\n",
" with app.run():\n",
" result=generate.remote(\"Life is a mystery, everyone must stand alone, I hear\")\n",
"result"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9a9a6844-29ec-4264-8e72-362d976b3968",
"metadata": {},
"outputs": [],
"source": [
"import modal\n",
"from pricer_ephemeral import app, price"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "50e6cf99-8959-4ae3-ba02-e325cb7fff94",
"metadata": {},
"outputs": [],
"source": [
"with modal.enable_output():\n",
" with app.run():\n",
" result=price.remote(\"Quadcast HyperX condenser mic, connects via usb-c to your computer for crystal clear audio\")\n",
"result"
]
},
{
"cell_type": "markdown",
"id": "04d8747f-8452-4077-8af6-27e03888508a",
"metadata": {},
"source": [
"## Transitioning From Ephemeral Apps to Deployed Apps\n",
"\n",
"From a command line, `modal deploy xxx` will deploy your code as a Deployed App\n",
"\n",
"This is how you could package your AI service behind an API to be used in a Production System.\n",
"\n",
"You can also build REST endpoints easily, although we won't cover that as we'll be calling direct from Python."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7f90d857-2f12-4521-bb90-28efd917f7d1",
"metadata": {},
"outputs": [],
"source": [
"!modal deploy pricer_service"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1dec70ff-1986-4405-8624-9bbbe0ce1f4a",
"metadata": {},
"outputs": [],
"source": [
"pricer = modal.Function.lookup(\"pricer-service\", \"price\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "17776139-0d9e-4ad0-bcd0-82d3a92ca61f",
"metadata": {},
"outputs": [],
"source": [
"pricer.remote(\"Quadcast HyperX condenser mic, connects via usb-c to your computer for crystal clear audio\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f56d1e55-2a03-4ce2-bb47-2ab6b9175a02",
"metadata": {},
"outputs": [],
"source": [
"!modal deploy pricer_service2"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9e19daeb-1281-484b-9d2f-95cc6fed2622",
"metadata": {},
"outputs": [],
"source": [
"Pricer = modal.Cls.lookup(\"pricer-service\", \"Pricer\")\n",
"pricer = Pricer()\n",
"reply = pricer.price.remote(\"Quadcast HyperX condenser mic, connects via usb-c to your computer for crystal clear audio\")\n",
"print(reply)"
]
},
{
"cell_type": "markdown",
"id": "3754cfdd-ae28-47c8-91f2-6e060e2c91b3",
"metadata": {},
"source": [
"## And now introducing our Agent class"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ba9aedca-6a7b-4d30-9f64-59d76f76fb6d",
"metadata": {},
"outputs": [],
"source": [
"from agents.specialist_agent import SpecialistAgent"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fe5843e5-e958-4a65-8326-8f5b4686de7f",
"metadata": {},
"outputs": [],
"source": [
"agent = SpecialistAgent()\n",
"agent.price(\"iPad Pro 2nd generation\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f5a3181b-1310-4102-8d7d-52caf4c00538",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.10"
}
},
"nbformat": 4,
"nbformat_minor": 5
}