{ "cells": [ { "cell_type": "markdown", "id": "c7d95a7f-205a-4262-a1af-4579489025ff", "metadata": {}, "source": [ "# Hello everyone." ] }, { "cell_type": "markdown", "id": "bc815dbc-acf7-45f9-a043-5767184c44c6", "metadata": {}, "source": [ "I completed the day 1, first LLM Experiment moments ago and found it really awesome. After the challenge was done, I wanted to chip in my two cents by making a PDF summarizer, basing myself on the code for the Website Summarizer. I want to share it in this contribution!\n", "### To consider:\n", "* To extract the contents of PDF files, I used the PyPDF2 library, which doesn't come with the default configuration of the virtual environment. To remedy the situation, you need to follow the steps:\n", " 1. Shut down Anaconda. Running `CTRL-C` in the Anaconda terminal should achieve this.\n", " 2. Run the following command, `pip install PyPDF2 --user`\n", " 3. Restart Jupyter lab with `jupyter lab`\n", "* To find PDF files online, you can add `filetype:url` on your browser query, i.e. searching the following can give you PDF files to add as input: `AI Engineering prompts filetype:pdf`!\n", "\n", "Without further ado, here's the PDF Summarizer!" ] }, { "cell_type": "code", "execution_count": 1, "id": "14e98361-f302-423d-87a5-f2f5d570f4ac", "metadata": {}, "outputs": [], "source": [ "### Prerequisite - Install below package\n", "#--> !pip install PyPDF2 --user" ] }, { "cell_type": "code", "execution_count": 2, "id": "06b63787-c6c8-4868-8a71-eb56b7618626", "metadata": {}, "outputs": [], "source": [ "# Import statements\n", "import os\n", "import requests\n", "from dotenv import load_dotenv\n", "from IPython.display import Markdown, display\n", "from openai import OpenAI\n", "from io import BytesIO\n", "from PyPDF2 import PdfReader\n", "import logging" ] }, { "cell_type": "code", "execution_count": 3, "id": "284ca770-5da4-495c-b1cf-637727a8609f", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "API key found and looks good so far!\n" ] } ], "source": [ "# Load environment variables in a file called .env\n", "\n", "load_dotenv()\n", "api_key = os.getenv('OPENAI_API_KEY')\n", "\n", "# Check the key\n", "\n", "if not api_key:\n", " print(\"No API key was found - please head over to the troubleshooting notebook in this folder to identify & fix!\")\n", "elif not api_key.startswith(\"sk-proj-\"):\n", " print(\"An API key was found, but it doesn't start sk-proj-; please check you're using the right key - see troubleshooting notebook\")\n", "elif api_key.strip() != api_key:\n", " print(\"An API key was found, but it looks like it might have space or tab characters at the start or end - please remove them - see troubleshooting notebook\")\n", "else:\n", " print(\"API key found and looks good so far!\")" ] }, { "cell_type": "code", "execution_count": 4, "id": "d4c316d7-d9c9-4400-b03e-1dd629c6b2ad", "metadata": {}, "outputs": [], "source": [ "openai = OpenAI()\n", "\n", "# If this doesn't work, try Kernel menu >> Restart Kernel and Clear Outputs Of All Cells, then run the cells from the top of this notebook down.\n", "# If it STILL doesn't work (horrors!) then please see the troubleshooting notebook, or try the below line instead:\n", "# openai = OpenAI(api_key=\"your-key-here-starting-sk-proj-\")" ] }, { "cell_type": "code", "execution_count": 5, "id": "b6f2967e-4779-458f-aea8-c47147528f77", "metadata": {}, "outputs": [], "source": [ "# Step 1: Defince calss for article pdf document\n", "\n", "class Article:\n", " def __init__(self, url):\n", " # Configure logging\n", " logging.basicConfig(level=logging.ERROR)\n", "\n", " # Comprehensive headers to mimic browser\n", " headers = {\n", " 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36',\n", " 'Accept': 'application/pdf,text/html,application/xhtml+xml,application/xml',\n", " 'Accept-Language': 'en-US,en;q=0.5',\n", " 'Referer': self._extract_base_url(url)\n", " }\n", "\n", " try:\n", " # Enhanced download attempt with multiple strategies\n", " response = self._download_pdf(url, headers)\n", " \n", " # Process PDF\n", " pdf_bytes = BytesIO(response.content)\n", " reader = PdfReader(pdf_bytes)\n", " \n", " # Efficient text extraction\n", " self.text = \" \".join(page.extract_text() for page in reader.pages)\n", " \n", " # Safe metadata extraction\n", " self.title = (reader.metadata or {}).get(\"/Title\", \"No title found\") or \"No title found\"\n", "\n", " except Exception as error:\n", " logging.error(f\"PDF Processing Error: {error}\")\n", " self.text = \"No text found\"\n", " self.title = \"No title found\"\n", "\n", " def _download_pdf(self, url, headers):\n", " \"\"\"\n", " Multiple download strategies to handle different access scenarios\n", " \"\"\"\n", " try:\n", " # Primary download attempt\n", " response = requests.get(url, headers=headers, timeout=10)\n", " response.raise_for_status()\n", " return response\n", "\n", " except requests.RequestException as primary_error:\n", " logging.warning(f\"Primary download failed: {primary_error}\")\n", " \n", " try:\n", " # Fallback strategy: Disable SSL verification\n", " response = requests.get(url, headers=headers, verify=False, timeout=10)\n", " response.raise_for_status()\n", " return response\n", " \n", " except requests.RequestException as fallback_error:\n", " logging.error(f\"All download attempts failed: {fallback_error}\")\n", " raise\n", "\n", " def _extract_base_url(self, url):\n", " \"\"\"\n", " Extract base URL for Referer header\n", " \"\"\"\n", " from urllib.parse import urlparse\n", " parsed_url = urlparse(url)\n", " return f\"{parsed_url.scheme}://{parsed_url.netloc}\"" ] }, { "cell_type": "markdown", "id": "95933aab-54df-4afc-a19c-46033f51378f", "metadata": {}, "source": [ "#####user_prompt_input = input(\"Explain how you would like me to summarise the document? \\n\")" ] }, { "cell_type": "markdown", "id": "1510e4df-b15f-4411-924c-feca1c72419a", "metadata": {}, "source": [ "# General Research Analyst\n", "#### Step 1: Create your prompts\n", "\n", "def craft_user_prompt(article):\n", " user_prompt = f\"You are looking at a document titled {article.title}\\n Based on the body of the document, provide an impactful summary. The body of the article is as follows.\"\n", " user_prompt += article.text\n", " return user_prompt\n", "\n", "### Step 2: Make the messages list\n", "def craft_messages(article):\n", " system_prompt = '''You are a research assistant specialising in processing documents. You have been tasked with analysing the contents of the articel and call out the important points as bullets and highlights. \\\n", " Ignore text that doesn't belong to the article, like headers or navigation related text. Respond in markdown. Structure your text in the form of question/answer.'''\n", " return [\n", " {\"role\": \"system\", \"content\": system_prompt},\n", " {\"role\": \"user\", \"content\": craft_user_prompt(article)}\n", " ]" ] }, { "cell_type": "code", "execution_count": 10, "id": "f71a47af-5f16-4b8f-880d-7fc41727564b", "metadata": {}, "outputs": [], "source": [ "# Financial Report Analyst\n", "#### Step 1: Create your prompts\n", "\n", "def craft_user_prompt(article):\n", " user_prompt = f\"You are looking at a document titled {article.title}\\n Based on the body of the document, provide an impactful summary. Call out important points as bullets and highlughts . The body of the article is as follows.\"\n", " user_prompt += article.text\n", " return user_prompt\n", "\n", "#### Step 2: Make the messages list\n", "def craft_messages(article):\n", " system_prompt = '''You are a financial report research assistant that specialises in analysing the contents of a financial report and provide summary for investors to \\ \n", " help adjust investment strategies. Call out important points as bullets and highlights. \\\n", " Ignore text that doesn't belong to the article, like headers or navigation related text. Respond in markdown. Structure your text in the form of question/answer.'''\n", " return [\n", " {\"role\": \"system\", \"content\": system_prompt},\n", " {\"role\": \"user\", \"content\": craft_user_prompt(article)}\n", " ]" ] }, { "cell_type": "code", "execution_count": 11, "id": "81ab896e-1ba9-4964-a477-2a0608b7036c", "metadata": {}, "outputs": [], "source": [ "# Step 3: Call OpenAI\n", "def summarize(url):\n", " article = Article(url)\n", " response = openai.chat.completions.create(\n", " model = \"gpt-4o-mini\",\n", " messages = craft_messages(article)\n", " )\n", " return response.choices[0].message.content" ] }, { "cell_type": "markdown", "id": "ccdfdd33-652f-4ecf-9a0d-355197aba9d5", "metadata": {}, "source": [ "# Results below:" ] }, { "cell_type": "code", "execution_count": 12, "id": "008c03fd-9cc5-4449-a054-7bbef1863b01", "metadata": {}, "outputs": [ { "name": "stdin", "output_type": "stream", "text": [ "Please input the pdf URL: \n", " https://www.spandanasphoorty.com/images_gallary/1714392044-256403-20242904050444-0195731001714392044.pdf\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "/home/msachdeva/anaconda3/envs/llms/lib/python3.11/site-packages/urllib3/connectionpool.py:1099: InsecureRequestWarning: Unverified HTTPS request is being made to host 'www.spandanasphoorty.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#tls-warnings\n", " warnings.warn(\n" ] }, { "data": { "text/markdown": [ "## Summary of Spandana Sphoorty Financial Limited's Financial Results for FY24\n", "\n", "### What are the key financial highlights for FY24?\n", "- **Profit After Tax (PAT):** Achieved an all-time high of **₹501 Cr**, significantly up from **₹12 Cr** in FY23.\n", "- **Assets Under Management (AUM):** Increased by **41%**, reaching **₹11,973 Cr**, up from **₹8,511 Cr** in FY23.\n", "- **New Customer Acquisition:** Grew by **59%**, adding **13.9 lac** customers in FY24 compared to **8.8 lac** in FY23.\n", "- **Disbursement:** Total of **₹10,688 Cr**, up **32%** from **₹8,125 Cr** in FY23.\n", "- **Asset Quality:** \n", " - **Gross Non-Performing Assets (GNPA):** Improved to **1.50%** from **2.07%** in FY23.\n", " - **Net Non-Performing Assets (NNPA):** Reduced to **0.30%** from **0.64%** in FY23.\n", "- **Total Income:** Rose **72%** to **₹2,534 Cr** from **₹1,477 Cr** in FY23.\n", "- **Net Interest Income (NII):** Increased by **59%** to **₹1,289 Cr** from **₹810 Cr** in FY23.\n", "\n", "### How did performance change in Q4FY24?\n", "- **AUM:** Increased **15% QoQ** and **41% YoY**.\n", "- **New Customer Acquisition:** Added **4.4 lac** customers in Q4, a **30%** increase QoQ.\n", "- **Disbursement:** **₹3,970 Cr** in Q4FY24, a **56% QoQ** and **30% YoY** growth.\n", "- **GNPA & NNPA:** Further improved to **1.50%** and **0.30%** respectively.\n", "\n", "### What are the operational efficiencies noted in the report?\n", "- **Collection Efficiency:** \n", " - Gross collection efficiency stood at **99.3%**.\n", " - Net collection efficiency at **96.5%**.\n", "- **Funding & Borrowings:** \n", " - Total borrowings increased by **81%** to **₹10,441 Cr** compared to **₹5,775 Cr** in FY23.\n", "\n", "### What does the management say about the future?\n", "- Mr. Shalabh Saxena, CEO, indicates continued focus on sustaining growth and improving efficiencies for FY25. The strategy incorporates managing multiple organizational priorities, including branch expansion and portfolio quality improvement.\n", "\n", "### What are the implications for investors?\n", "- **Strong Growth Indicators:** The noteworthy increase in revenue, profits, and customer acquisition positions Spandana as a promising investment within the microfinance sector.\n", "- **Reduced Risk Profile:** Improved asset quality metrics (GNPA and NNPA) suggest better risk management and opportunity for long-term stability.\n", "- **Focus on Operational Efficiency:** The commitment to enhancing distribution productivity may enhance profitability, further benefiting shareholders.\n", "\n", "Investors might consider these highlights to evaluate or adjust their investment strategies regarding microfinance and rural-focused financial services in India." ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Step 4: Print the result of an example pdf\n", "article_url=input('Please input the pdf URL: \\n')\n", "summary = summarize(article_url)\n", "display(Markdown(summary))" ] }, { "cell_type": "markdown", "id": "70cbbffd-bf70-47d1-b5d6-5f929bdd89eb", "metadata": {}, "source": [ "# Medical Report Analyst\n", "#### Step 1: Create your prompts\n", "\n", "def craft_user_prompt(article):\n", " user_prompt = f\"You are looking at a research article titled {article.title}\\n Based on the body of the article, Summarise the research article while highliting, Introduction, the problem, possible solution and conclusion. The body of the article is as follows.\"\n", " user_prompt += article.text\n", " return user_prompt\n", "\n", "#### Step 2: Make the messages list\n", "def craft_messages(article):\n", " system_prompt = \"You are a medical research assistant that analyses the contents of a research article and provides a summary while calling out important points as bullets and quotes. \\\n", " Ignore text that doesn't belong to the article, like headers or navigation related text. Respond in markdown. Structure your text in the form of question/answer.\"\n", " return [\n", " {\"role\": \"system\", \"content\": system_prompt},\n", " {\"role\": \"user\", \"content\": craft_user_prompt(article)}\n", " ]" ] }, { "cell_type": "markdown", "id": "c0bb9579-686a-4277-a139-7dff2c64cc9f", "metadata": {}, "source": [ "# QA Format\n", "#### Step 1: Create your prompts\n", "\n", "def craft_user_prompt(article):\n", " user_prompt = f\"You are looking at a research article titled {article.title}\\n Based on the body of the article, how are micro RNAs produced in the cell? State the function of the proteins \\\n", " involved. The body of the article is as follows.\"\n", " user_prompt += article.text\n", " return user_prompt\n", "\n", "#### Step 2: Make the messages list\n", "def craft_messages(article):\n", " system_prompt = \"You are an assistant that analyses the contents of a research article and provide answers to the question asked by the user in 250 words or less. \\\n", " Ignore text that doesn't belong to the article, like headers or navigation related text. Respond in markdown. Structure your text in the form of question/answer.\"\n", " return [\n", " {\"role\": \"system\", \"content\": system_prompt},\n", " {\"role\": \"user\", \"content\": craft_user_prompt(article)}\n", " ]" ] }, { "cell_type": "code", "execution_count": null, "id": "680b45ae-6f3d-43b5-b69a-4e7f4fcca268", "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "id": "988864cc-08a2-4cfb-a09b-524504ee3803", "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.11" } }, "nbformat": 4, "nbformat_minor": 5 }