From the uDemy course on LLM engineering.
https://www.udemy.com/course/llm-engineering-master-ai-and-large-language-models
You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
624 lines
18 KiB
624 lines
18 KiB
{ |
|
"cells": [ |
|
{ |
|
"cell_type": "markdown", |
|
"id": "5c291475-8c7c-461c-9b12-545a887b2432", |
|
"metadata": {}, |
|
"source": [ |
|
"# Intermediate Level Python\n", |
|
"\n", |
|
"## Getting you up to speed\n", |
|
"\n", |
|
"This course assumes that you're at an intermediate level of python. For example, you should have a decent idea what something like this might do:\n", |
|
"\n", |
|
"`yield from {book.get(\"author\") for book in books if book.get(\"author\")}`\n", |
|
"\n", |
|
"If not - then you've come to the right place! Welcome to the crash course in intermediate level python. The best way to learn is by doing!\n" |
|
] |
|
}, |
|
{ |
|
"cell_type": "markdown", |
|
"id": "542f0577-a826-4613-a5d7-4170e9666d04", |
|
"metadata": {}, |
|
"source": [ |
|
"## First: if you need a refresher on the foundations\n", |
|
"\n", |
|
"I'm going to defer to an AI friend for this, because these explanations are so well written with great examples. Copy and paste the code examples into a new cell to give them a try. Pick whichever section(s) you'd like to brush up on.\n", |
|
"\n", |
|
"**Python imports:** \n", |
|
"https://chatgpt.com/share/672f9f31-8114-8012-be09-29ef0d0140fb\n", |
|
"\n", |
|
"**Python functions** including default arguments: \n", |
|
"https://chatgpt.com/share/672f9f99-7060-8012-bfec-46d4cf77d672\n", |
|
"\n", |
|
"**Python strings**, including slicing, split/join, replace and literals: \n", |
|
"https://chatgpt.com/share/672fb526-0aa0-8012-9e00-ad1687c04518\n", |
|
"\n", |
|
"**Python f-strings** including number and date formatting: \n", |
|
"https://chatgpt.com/share/672fa125-0de0-8012-8e35-27918cbb481c\n", |
|
"\n", |
|
"**Python lists, dicts and sets**, including the `get()` method: \n", |
|
"https://chatgpt.com/share/672fa225-3f04-8012-91af-f9c95287da8d\n", |
|
"\n", |
|
"**Python files** including modes, encoding, context managers, Path, glob.glob: \n", |
|
"https://chatgpt.com/share/673b53b2-6d5c-8012-a344-221056c2f960\n", |
|
"\n", |
|
"**Python classes:** \n", |
|
"https://chatgpt.com/share/672fa07a-1014-8012-b2ea-6dc679552715\n", |
|
"\n", |
|
"**Pickling Python objects and converting to JSON:** \n", |
|
"https://chatgpt.com/share/673b553e-9d0c-8012-9919-f3bb5aa23e31" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 1, |
|
"id": "5802e2f0-0ea0-4237-bbb7-f375a34260f0", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# Next let's create some things:\n", |
|
"\n", |
|
"fruits = [\"Apples\", \"Bananas\", \"Pears\"]\n", |
|
"\n", |
|
"book1 = {\"title\": \"Great Expectations\", \"author\": \"Charles Dickens\"}\n", |
|
"book2 = {\"title\": \"Bleak House\", \"author\": \"Charles Dickens\"}\n", |
|
"book3 = {\"title\": \"An Book By No Author\"}\n", |
|
"book4 = {\"title\": \"Moby Dick\", \"author\": \"Herman Melville\"}\n", |
|
"\n", |
|
"books = [book1, book2, book3, book4]" |
|
] |
|
}, |
|
{ |
|
"cell_type": "markdown", |
|
"id": "9b941e6a-3658-4144-a8d4-72f5e72f3707", |
|
"metadata": {}, |
|
"source": [ |
|
"# Part 1: List and dict comprehensions" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 2, |
|
"id": "61992bb8-735d-4dad-8747-8c10b63aec82", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"name": "stdout", |
|
"output_type": "stream", |
|
"text": [ |
|
"Apples\n", |
|
"Bananas\n", |
|
"Pears\n" |
|
] |
|
} |
|
], |
|
"source": [ |
|
"# Simple enough to start\n", |
|
"\n", |
|
"for fruit in fruits:\n", |
|
" print(fruit)" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 3, |
|
"id": "c89c3842-9b74-47fa-8424-0fcb08e4177c", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"data": { |
|
"text/plain": [ |
|
"['APPLES', 'BANANAS', 'PEARS']" |
|
] |
|
}, |
|
"execution_count": 3, |
|
"metadata": {}, |
|
"output_type": "execute_result" |
|
} |
|
], |
|
"source": [ |
|
"# Let's make a new version of fruits\n", |
|
"\n", |
|
"fruits_shouted = []\n", |
|
"for fruit in fruits:\n", |
|
" fruits_shouted.append(fruit.upper())\n", |
|
"\n", |
|
"fruits_shouted" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 4, |
|
"id": "4ec13b3a-9545-44f1-874a-2910a0663560", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"data": { |
|
"text/plain": [ |
|
"['APPLES', 'BANANAS', 'PEARS']" |
|
] |
|
}, |
|
"execution_count": 4, |
|
"metadata": {}, |
|
"output_type": "execute_result" |
|
} |
|
], |
|
"source": [ |
|
"# You probably already know this\n", |
|
"# There's a nice Python construct called \"list comprehension\" that does this:\n", |
|
"\n", |
|
"fruits_shouted2 = [fruit.upper() for fruit in fruits]\n", |
|
"fruits_shouted2" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 5, |
|
"id": "ecc08c3c-181d-4b64-a3e1-b0ccffc6c0cd", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"data": { |
|
"text/plain": [ |
|
"{'Apples': 'APPLES', 'Bananas': 'BANANAS', 'Pears': 'PEARS'}" |
|
] |
|
}, |
|
"execution_count": 5, |
|
"metadata": {}, |
|
"output_type": "execute_result" |
|
} |
|
], |
|
"source": [ |
|
"# But you may not know that you can do this to create dictionaries, too:\n", |
|
"\n", |
|
"fruit_mapping = {fruit: fruit.upper() for fruit in fruits}\n", |
|
"fruit_mapping" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 14, |
|
"id": "68ad7090-6a5c-486d-b3fe-1e9d0a7a55b8", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"data": { |
|
"text/plain": [ |
|
"{'Apples': 'APPLES', 'Bananas': 'BANANAS', 'Pears': 'PEARS'}" |
|
] |
|
}, |
|
"execution_count": 14, |
|
"metadata": {}, |
|
"output_type": "execute_result" |
|
} |
|
], |
|
"source": [ |
|
"{fruit: fruit.upper() for fruit in fruits}" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 6, |
|
"id": "500c2406-00d2-4793-b57b-f49b612760c8", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"data": { |
|
"text/plain": [ |
|
"['APPLES', 'BANANAS']" |
|
] |
|
}, |
|
"execution_count": 6, |
|
"metadata": {}, |
|
"output_type": "execute_result" |
|
} |
|
], |
|
"source": [ |
|
"# you can also use the if statement to filter the results\n", |
|
"\n", |
|
"fruits_with_longer_names_shouted = [fruit.upper() for fruit in fruits if len(fruit)>5]\n", |
|
"fruits_with_longer_names_shouted" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 7, |
|
"id": "38c11c34-d71e-45ba-945b-a3d37dc29793", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"data": { |
|
"text/plain": [ |
|
"{'Bananas': 'BANANAS', 'Pears': 'PEARS'}" |
|
] |
|
}, |
|
"execution_count": 7, |
|
"metadata": {}, |
|
"output_type": "execute_result" |
|
} |
|
], |
|
"source": [ |
|
"fruit_mapping_unless_starts_with_a = {fruit: fruit.upper() for fruit in fruits if not fruit.startswith('A')}\n", |
|
"fruit_mapping_unless_starts_with_a" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 8, |
|
"id": "5c97d8e8-31de-4afa-973e-28d8e5cab749", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"data": { |
|
"text/plain": [ |
|
"['Great Expectations', 'Bleak House', 'An Book By No Author', 'Moby Dick']" |
|
] |
|
}, |
|
"execution_count": 8, |
|
"metadata": {}, |
|
"output_type": "execute_result" |
|
} |
|
], |
|
"source": [ |
|
"# Another comprehension\n", |
|
"\n", |
|
"[book['title'] for book in books]" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 9, |
|
"id": "50be0edc-a4cd-493f-a680-06080bb497b4", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"ename": "KeyError", |
|
"evalue": "'author'", |
|
"output_type": "error", |
|
"traceback": [ |
|
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", |
|
"\u001b[0;31mKeyError\u001b[0m Traceback (most recent call last)", |
|
"Cell \u001b[0;32mIn[9], line 3\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[38;5;66;03m# This code will fail with an error because one of our books doesn't have an author\u001b[39;00m\n\u001b[0;32m----> 3\u001b[0m \u001b[43m[\u001b[49m\u001b[43mbook\u001b[49m\u001b[43m[\u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43mauthor\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m]\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43;01mfor\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mbook\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;129;43;01min\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mbooks\u001b[49m\u001b[43m]\u001b[49m\n", |
|
"Cell \u001b[0;32mIn[9], line 3\u001b[0m, in \u001b[0;36m<listcomp>\u001b[0;34m(.0)\u001b[0m\n\u001b[1;32m 1\u001b[0m \u001b[38;5;66;03m# This code will fail with an error because one of our books doesn't have an author\u001b[39;00m\n\u001b[0;32m----> 3\u001b[0m [\u001b[43mbook\u001b[49m\u001b[43m[\u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43mauthor\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m]\u001b[49m \u001b[38;5;28;01mfor\u001b[39;00m book \u001b[38;5;129;01min\u001b[39;00m books]\n", |
|
"\u001b[0;31mKeyError\u001b[0m: 'author'" |
|
] |
|
} |
|
], |
|
"source": [ |
|
"# This code will fail with an error because one of our books doesn't have an author\n", |
|
"\n", |
|
"[book['author'] for book in books]" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 10, |
|
"id": "53794083-cc09-4edb-b448-2ffb7e8495c2", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"data": { |
|
"text/plain": [ |
|
"['Charles Dickens', 'Charles Dickens', None, 'Herman Melville']" |
|
] |
|
}, |
|
"execution_count": 10, |
|
"metadata": {}, |
|
"output_type": "execute_result" |
|
} |
|
], |
|
"source": [ |
|
"# But this will work, because get() returns None\n", |
|
"\n", |
|
"[book.get('author') for book in books]" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 11, |
|
"id": "b8e4b859-24f8-4016-8d74-c2cef226d049", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"data": { |
|
"text/plain": [ |
|
"['Charles Dickens', 'Charles Dickens', 'Herman Melville']" |
|
] |
|
}, |
|
"execution_count": 11, |
|
"metadata": {}, |
|
"output_type": "execute_result" |
|
} |
|
], |
|
"source": [ |
|
"# And this variation will filter out the None\n", |
|
"\n", |
|
"[book.get('author') for book in books if book.get('author')]" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 12, |
|
"id": "c44bb999-52b4-4dee-810b-8a400db8f25f", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"data": { |
|
"text/plain": [ |
|
"{'Charles Dickens', 'Herman Melville'}" |
|
] |
|
}, |
|
"execution_count": 12, |
|
"metadata": {}, |
|
"output_type": "execute_result" |
|
} |
|
], |
|
"source": [ |
|
"# And this version will convert it into a set, removing duplicates\n", |
|
"\n", |
|
"set([book.get('author') for book in books if book.get('author')])" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": 13, |
|
"id": "80a65156-6192-4bb4-b4e6-df3fdc933891", |
|
"metadata": {}, |
|
"outputs": [ |
|
{ |
|
"data": { |
|
"text/plain": [ |
|
"{'Charles Dickens', 'Herman Melville'}" |
|
] |
|
}, |
|
"execution_count": 13, |
|
"metadata": {}, |
|
"output_type": "execute_result" |
|
} |
|
], |
|
"source": [ |
|
"# And finally, this version is even nicer\n", |
|
"# curly braces creates a set, so this is a set comprehension\n", |
|
"\n", |
|
"{book.get('author') for book in books if book.get('author')}" |
|
] |
|
}, |
|
{ |
|
"cell_type": "markdown", |
|
"id": "c100e5db-5438-4715-921c-3f7152f83f4a", |
|
"metadata": {}, |
|
"source": [ |
|
"# Part 2: Generators\n", |
|
"\n", |
|
"We use Generators in the course because AI models can stream back results.\n", |
|
"\n", |
|
"If you've not used Generators before, please start with this excellent intro from ChatGPT:\n", |
|
"\n", |
|
"https://chatgpt.com/share/672faa6e-7dd0-8012-aae5-44fc0d0ec218\n", |
|
"\n", |
|
"Try pasting some of its examples into a cell." |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": null, |
|
"id": "1efc26fa-9144-4352-9a17-dfec1d246aad", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# First define a generator; it looks like a function, but it has yield instead of return\n", |
|
"\n", |
|
"import time\n", |
|
"\n", |
|
"def come_up_with_fruit_names():\n", |
|
" for fruit in fruits:\n", |
|
" time.sleep(1) # thinking of a fruit\n", |
|
" yield fruit" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": null, |
|
"id": "eac338bb-285c-45c8-8a3e-dbfc41409ca3", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# Then use it\n", |
|
"\n", |
|
"for fruit in come_up_with_fruit_names():\n", |
|
" print(fruit)" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": null, |
|
"id": "f6880578-a3de-4502-952a-4572b95eb9ff", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# Here's another one\n", |
|
"\n", |
|
"def authors_generator():\n", |
|
" for book in books:\n", |
|
" if book.get(\"author\"):\n", |
|
" yield book.get(\"author\")" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": null, |
|
"id": "9e316f02-f87f-441d-a01f-024ade949607", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# Use it\n", |
|
"\n", |
|
"for author in authors_generator():\n", |
|
" print(author)" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": null, |
|
"id": "7535c9d0-410e-4e56-a86c-ae6c0e16053f", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# Here's the same thing written with list comprehension\n", |
|
"\n", |
|
"def authors_generator():\n", |
|
" for author in [book.get(\"author\") for book in books if book.get(\"author\")]:\n", |
|
" yield author" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": null, |
|
"id": "dad34494-0f6c-4edb-b03f-b8d49ee186f2", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# Use it\n", |
|
"\n", |
|
"for author in authors_generator():\n", |
|
" print(author)" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": null, |
|
"id": "abeb7e61-d8aa-4af0-b05a-ae17323e678c", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# Here's a nice shortcut\n", |
|
"# You can use \"yield from\" to yield each item of an iterable\n", |
|
"\n", |
|
"def authors_generator():\n", |
|
" yield from [book.get(\"author\") for book in books if book.get(\"author\")]" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": null, |
|
"id": "05b0cb43-aa83-4762-a797-d3beb0f22c44", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# Use it\n", |
|
"\n", |
|
"for author in authors_generator():\n", |
|
" print(author)" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": null, |
|
"id": "fdfea58e-d809-4dd4-b7b0-c26427f8be55", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# And finally - we can replace the list comprehension with a set comprehension\n", |
|
"\n", |
|
"def unique_authors_generator():\n", |
|
" yield from {book.get(\"author\") for book in books if book.get(\"author\")}" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": null, |
|
"id": "3e821d08-97be-4db9-9a5b-ce5dced3eff8", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# Use it\n", |
|
"\n", |
|
"for author in unique_authors_generator():\n", |
|
" print(author)" |
|
] |
|
}, |
|
{ |
|
"cell_type": "code", |
|
"execution_count": null, |
|
"id": "905ba603-15d8-4d01-9a79-60ec293d7ca1", |
|
"metadata": {}, |
|
"outputs": [], |
|
"source": [ |
|
"# And for some fun - press the stop button in the toolbar when bored!\n", |
|
"# It's like we've made our own Large Language Model... although not particularly large..\n", |
|
"# See if you understand why it prints a letter at a time, instead of a word at a time. If you're unsure, try removing the keyword \"from\" everywhere in the code.\n", |
|
"\n", |
|
"import random\n", |
|
"import time\n", |
|
"\n", |
|
"pronouns = [\"I\", \"You\", \"We\", \"They\"]\n", |
|
"verbs = [\"eat\", \"detest\", \"bathe in\", \"deny the existence of\", \"resent\", \"pontificate about\", \"juggle\", \"impersonate\", \"worship\", \"misplace\", \"conspire with\", \"philosophize about\", \"tap dance on\", \"dramatically renounce\", \"secretly collect\"]\n", |
|
"adjectives = [\"turqoise\", \"smelly\", \"arrogant\", \"festering\", \"pleasing\", \"whimsical\", \"disheveled\", \"pretentious\", \"wobbly\", \"melodramatic\", \"pompous\", \"fluorescent\", \"bewildered\", \"suspicious\", \"overripe\"]\n", |
|
"nouns = [\"turnips\", \"rodents\", \"eels\", \"walruses\", \"kumquats\", \"monocles\", \"spreadsheets\", \"bagpipes\", \"wombats\", \"accordions\", \"mustaches\", \"calculators\", \"jellyfish\", \"thermostats\"]\n", |
|
"\n", |
|
"def infinite_random_sentences():\n", |
|
" while True:\n", |
|
" yield from random.choice(pronouns)\n", |
|
" yield \" \"\n", |
|
" yield from random.choice(verbs)\n", |
|
" yield \" \"\n", |
|
" yield from random.choice(adjectives)\n", |
|
" yield \" \"\n", |
|
" yield from random.choice(nouns)\n", |
|
" yield \". \"\n", |
|
"\n", |
|
"for letter in infinite_random_sentences():\n", |
|
" print(letter, end=\"\", flush=True)\n", |
|
" time.sleep(0.02)" |
|
] |
|
}, |
|
{ |
|
"cell_type": "markdown", |
|
"id": "04832ea2-2447-4473-a449-104f80e24d85", |
|
"metadata": {}, |
|
"source": [ |
|
"# Exercise\n", |
|
"\n", |
|
"Write some python classes for the books example.\n", |
|
"\n", |
|
"Write a Book class with a title and author. Include a method has_author()\n", |
|
"\n", |
|
"Write a BookShelf class with a list of books. Include a generator method unique_authors()" |
|
] |
|
}, |
|
{ |
|
"cell_type": "markdown", |
|
"id": "35760406-fe6c-41f9-b0c0-3e8cf73aafd0", |
|
"metadata": {}, |
|
"source": [ |
|
"# Finally\n", |
|
"\n", |
|
"Here are some intermediate level details of Classes from our AI friend, including use of type hints, inheritance and class methods. This includes a Book example.\n", |
|
"\n", |
|
"https://chatgpt.com/share/67348aca-65fc-8012-a4a9-fd1b8f04ba59" |
|
] |
|
} |
|
], |
|
"metadata": { |
|
"kernelspec": { |
|
"display_name": "Python 3 (ipykernel)", |
|
"language": "python", |
|
"name": "python3" |
|
}, |
|
"language_info": { |
|
"codemirror_mode": { |
|
"name": "ipython", |
|
"version": 3 |
|
}, |
|
"file_extension": ".py", |
|
"mimetype": "text/x-python", |
|
"name": "python", |
|
"nbconvert_exporter": "python", |
|
"pygments_lexer": "ipython3", |
|
"version": "3.11.11" |
|
} |
|
}, |
|
"nbformat": 4, |
|
"nbformat_minor": 5 |
|
}
|
|
|