{ "cells": [ { "cell_type": "markdown", "id": "fe12c203-e6a6-452c-a655-afb8a03a4ff5", "metadata": {}, "source": [ "# End of week 1 exercise\n", "\n", "To demonstrate your familiarity with OpenAI API, and also Ollama, build a tool that takes a technical question, \n", "and responds with an explanation. This is a tool that you will be able to use yourself during the course!" ] }, { "cell_type": "code", "execution_count": 6, "id": "c1070317-3ed9-4659-abe3-828943230e03", "metadata": {}, "outputs": [], "source": [ "# imports\n", "from dotenv import load_dotenv\n", "from IPython.display import Markdown, display, update_display\n", "from openai import OpenAI\n", "import ollama" ] }, { "cell_type": "code", "execution_count": 7, "id": "4a456906-915a-4bfd-bb9d-57e505c5093f", "metadata": {}, "outputs": [], "source": [ "# constants\n", "MODEL_GPT = 'gpt-4o-mini'\n", "MODEL_LLAMA = 'llama3.2'" ] }, { "cell_type": "code", "execution_count": 8, "id": "a8d7923c-5f28-4c30-8556-342d7c8497c1", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "API key looks good so far\n" ] } ], "source": [ "# set up environment\n", "load_dotenv(override=True)\n", "api_key = os.getenv('OPENAI_API_KEY')\n", "\n", "if api_key and api_key.startswith('sk-proj-') and len(api_key) > 10:\n", " print(\"API key looks good so far\")\n", "else:\n", " print(\"There might be problem with your API key? Please visit the troubleshooting notebook!\")\n", "openai = OpenAI()" ] }, { "cell_type": "code", "execution_count": 11, "id": "3f0d0137-52b0-47a8-81a8-11a90a010798", "metadata": {}, "outputs": [ { "name": "stdin", "output_type": "stream", "text": [ "Please enter your question What is an LLM\n" ] } ], "source": [ "# here is the question; type over this to ask something new\n", "\n", "my_question = input(\"Please enter your question\")\n", "\n", "system_prompt = \"You are a helpful technical tutor who answers questions about python code, software engineering, data science and LLMs\"\n", "user_prompt = \"Please give a detailed explanation to the following question: \" + my_question\n", "\n", "messages = [\n", " {\"role\": \"system\", \"content\": system_prompt},\n", " {\"role\": \"user\", \"content\": user_prompt}\n", "]" ] }, { "cell_type": "code", "execution_count": 13, "id": "60ce7000-a4a5-4cce-a261-e75ef45063b4", "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "**LLM**, or **Large Language Model**, refers to a type of artificial intelligence model that is designed to understand, generate, and manipulate human language. These models are typically built using deep learning techniques, particularly architectures such as Transformers, which excel at handling sequential data like text. Here’s a detailed breakdown of what LLMs are and how they work:\n", "\n", "### 1. **Basics of Language Models**\n", "Language models are statistical models that are trained to predict the probability of a sequence of words. For instance, given the sentence fragment \"The cat is on the\", a language model would assign a high probability to the word \"mat\" and lower probabilities to less relevant words. Traditional language models were simpler and often relied on n-grams or Markov models.\n", "\n", "### 2. **Deep Learning and the Rise of LLMs**\n", "The introduction of deep learning has revolutionized the approach to language modeling. LLMs use neural networks, particularly architectures that can manage long-range dependencies in text, which is crucial for understanding context. The Transformer model, introduced in the paper \"Attention is All You Need\" by Vaswani et al. in 2017, is a key development in LLMs. \n", "\n", "### 3. **Architecture of LLMs**\n", "- **Transformers**: LLMs typically use the Transformer architecture, which relies on a mechanism known as **self-attention**. This allows the model to weigh the importance of different words in a sentence, regardless of their position.\n", "- **Layers**: LLMs are comprised of multiple layers of attention and feed-forward neural networks. The layers allow the model to learn complex patterns in the data.\n", "- **Tokens**: Text input is divided into smaller units called tokens (which can be words, subwords, or characters); the model processes these tokens to learn their relationships and usages in context.\n", "\n", "### 4. **Training Large Language Models**\n", "LLMs are trained on vast datasets containing diverse text from books, websites, articles, and other written forms. The training process involves:\n", "- **Pre-training**: The model learns to predict the next word in a sentence (language modeling) using unsupervised learning on a large corpus of text.\n", "- **Fine-tuning**: After pre-training, the model can be fine-tuned on specific tasks (like sentiment analysis or question answering) with smaller and more targeted datasets, often using supervised learning techniques.\n", "\n", "### 5. **Scale and Capacity**\n", "The term \"large\" in LLM typically refers to the number of parameters (the weights in the neural network). Modern LLMs can have billions (or even trillions) of parameters, which enhances their ability to capture nuanced language features and relationships.\n", "\n", "### 6. **Applications of LLMs**\n", "LLMs have a wide range of applications, such as:\n", "- **Text generation**: Creating human-like text responses.\n", "- **Translation**: Converting text from one language to another.\n", "- **Summarization**: Producing concise summaries of longer texts.\n", "- **Chatbots**: Engaging in conversations with users on various topics.\n", "- **Sentiment analysis**: Determining the sentiment behind a piece of text.\n", "- **Code generation**: Assisting in programming by generating code snippets.\n", "\n", "### 7. **Limitations and Challenges**\n", "Despite their capabilities, LLMs face several challenges:\n", "- **Data Bias**: They can inherit biases present in the training data, leading to biased outputs.\n", "- **Resource Intensity**: Training and deploying LLMs requires significant computational resources.\n", "- **Interpretability**: Understanding how LLMs arrive at specific outputs can be difficult, posing challenges in transparency and trust.\n", "\n", "### 8. **Future Directions**\n", "Research in LLMs continues to evolve, focusing on:\n", "- Reducing the environmental impact of training large models.\n", "- Enhancing the accuracy and utility of models for specific applications.\n", "- Developing methods for better interpretability and addressing bias in model training.\n", "\n", "### Conclusion\n", "Large Language Models represent a significant advancement in artificial intelligence, demonstrating remarkable abilities in processing and generating human language. As technology and methodologies improve, LLMs are likely to become even more integral to various applications across industries.\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Get gpt-4o-mini to answer, with streaming\n", "\n", "stream = openai.chat.completions.create(model = MODEL_GPT, messages=messages, stream=True)\n", "\n", "response = \"\"\n", "display_handle = display(Markdown(\"\"), display_id=True)\n", "for chunk in stream:\n", " response += chunk.choices[0].delta.content or ''\n", " response = response.replace(\"```\",\"\").replace(\"markdown\",\"\")\n", " update_display(Markdown(response), display_id=display_handle.display_id)" ] }, { "cell_type": "code", "execution_count": 14, "id": "8f7c8ea8-4082-4ad0-8751-3301adcf6538", "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "**What is an LLM (Large Language Model)?**\n", "\n", "A Large Language Model (LLM) is a type of artificial intelligence (AI) model that uses natural language processing (NLP) techniques to understand, generate, and process human-like language. These models are designed to learn patterns and relationships in large amounts of text data, enabling them to perform a wide range of tasks, such as:\n", "\n", "1. **Text Generation**: LLMs can produce coherent, context-dependent text based on input prompts or topics.\n", "2. **Language Translation**: They can translate text from one language to another with reasonable accuracy.\n", "3. **Question Answering**: LLMs can answer questions by retrieving relevant information from a vast knowledge base.\n", "4. **Text Summarization**: They can summarize long pieces of text into concise, meaningful summaries.\n", "\n", "**Key Characteristics of LLMs**\n", "\n", "1. **Large Scale Training Data**: LLMs are trained on massive amounts of text data, which enables them to learn complex patterns and relationships in language.\n", "2. **Deep Learning Architecture**: These models typically employ a deep learning architecture, such as transformer or recurrent neural networks (RNNs), to process input sequences.\n", "3. **Self-Attention Mechanism**: LLMs often use self-attention mechanisms to focus on specific parts of the input text when generating output.\n", "\n", "**Types of LLMs**\n", "\n", "1. **Transformers**: These models use self-attention mechanisms and are particularly effective for tasks like language translation, question answering, and text generation.\n", "2. **Recurrent Neural Networks (RNNs)**: RNNs process input sequences sequentially, one token at a time, and are often used for tasks like language modeling, sentiment analysis, and chatbots.\n", "3. **Language-Conditional Models**: These models learn to predict specific outputs based on conditional input information.\n", "\n", "**How LLMs Work**\n", "\n", "1. **Text Input**: The model receives an input text sequence (e.g., a sentence or paragraph).\n", "2. **Tokenization**: The input text is broken down into individual tokens, such as words or subwords.\n", "3. **Model Processing**: The model processes the tokenized input using its self-attention mechanism and other processing techniques.\n", "4. **Output Generation**: The model generates an output based on the processed input.\n", "\n", "**Real-World Applications of LLMs**\n", "\n", "1. **Virtual Assistants**: Virtual assistants, like Siri or Alexa, use LLMs to understand voice commands and generate responses.\n", "2. **Language Translation**: Google Translate uses LLMs to translate text between languages.\n", "3. **Chatbots**: Chatbots employ LLMs to understand user input and respond accordingly.\n", "4. **Content Generation**: LLMs are used in content generation applications, such as automated writing tools or music composition software.\n", "\n", "In summary, Large Language Models are powerful AI models that use natural language processing techniques to process human-like language. They have numerous real-world applications and continue to evolve as the field of NLP advances.\n", "\n", "Would you like me to clarify anything or provide more examples?" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Get Llama 3.2 to answer\n", "\n", "response = ollama.chat(model=MODEL_LLAMA, messages=messages)\n", "reply = response['message']['content']\n", "display(Markdown(reply))" ] }, { "cell_type": "code", "execution_count": null, "id": "8b4c9798-dad7-4708-ab1e-f91043b7216d", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.11" } }, "nbformat": 4, "nbformat_minor": 5 }