L2-Guidelines.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Guidelines for Prompting\n",
    "In this lesson, you'll practice two prompting principles and their related tactics in order to write effective prompts for large language models."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Setup\n",
    "#### Load the API key and relevant Python libaries. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this course, we've provided some code that loads the OpenAI API key for you."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import openai\n",
    "import os\n",
    "from dotenv import load_dotenv, find_dotenv\n",
    "_ = load_dotenv(find_dotenv())\n",
    "\n",
    "openai.api_key  = os.getenv('OPENAI_API_KEY')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### helper function\n",
    "Throughout this course, we will use OpenAI's gpt-3.5-turbo model and the [chat completions endpoint.](https://platform.openai.com/docs/guides/gpt/chat-completions-api)\n",
    "\n",
    "This helper function will make it easier to use prompts and look at the generated outputs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_completion(prompt, model=\"gpt-3.5-turbo\"):\n",
    "    messages = [{\"role\": \"user\", \"content\": prompt}]\n",
    "    response = openai.ChatCompletion.create(\n",
    "        model=model,\n",
    "        messages=messages,\n",
    "        temperature=0, # this is the degree of randomness of the model's output\n",
    "    )\n",
    "    return response.choices[0].message[\"content\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Prompting Principles\n",
    "- Principle 1: Write clear and specific instructions\n",
    "- Principle 2: Give the model time to “think”\n",
    "\n",
    "### Tactics\n",
    "#### Tactic 1: Use delimiters to clearly indicate distinct parts of the input\n",
    "- Delimiters can be anything like: ```, \"\"\", < >, <tag> </tag>, :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "To guide a model towards the desired output and reduce the chances of irrelevant or incorrect responses, it is important to provide clear and specific instructions, which may be longer prompts that provide more clarity and context for the model.\n"
     ]
    }
   ],
   "source": [
    "text = f\"\"\"\n",
    "You should express what you want a model to do by \\ \n",
    "providing instructions that are as clear and \\ \n",
    "specific as you can possibly make them. \\ \n",
    "This will guide the model towards the desired output, \\ \n",
    "and reduce the chances of receiving irrelevant \\ \n",
    "or incorrect responses. Don't confuse writing a \\ \n",
    "clear prompt with writing a short prompt. \\ \n",
    "In many cases, longer prompts provide more clarity \\ \n",
    "and context for the model, which can lead to \\ \n",
    "more detailed and relevant outputs.\n",
    "\"\"\"\n",
    "prompt = f\"\"\"\n",
    "Summarize the text delimited by triple backticks \\ \n",
    "into a single sentence.\n",
    "```{text}```\n",
    "\"\"\"\n",
    "response = get_completion(prompt)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Tactic 2: Ask for a structured output¶\n",
    "- JSON, HTML"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[\n",
      "  {\n",
      "    \"book_id\": 1,\n",
      "    \"title\": \"The Lost City of Zorath\",\n",
      "    \"author\": \"Aria Blackwood\",\n",
      "    \"genre\": \"Fantasy\"\n",
      "  },\n",
      "  {\n",
      "    \"book_id\": 2,\n",
      "    \"title\": \"The Last Survivors\",\n",
      "    \"author\": \"Ethan Stone\",\n",
      "    \"genre\": \"Science Fiction\"\n",
      "  },\n",
      "  {\n",
      "    \"book_id\": 3,\n",
      "    \"title\": \"The Secret of the Haunted Mansion\",\n",
      "    \"author\": \"Lila Rose\",\n",
      "    \"genre\": \"Mystery\"\n",
      "  }\n",
      "]\n"
     ]
    }
   ],
   "source": [
    "prompt = f\"\"\"\n",
    "Generate a list of three made-up book titles along \\ \n",
    "with their authors and genres. \n",
    "Provide them in JSON format with the following keys: \n",
    "book_id, title, author, genre.\n",
    "\"\"\"\n",
    "response = get_completion(prompt)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Tactic 3: Ask the model to check whether conditions are satisfied"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Completion for Text 1:\n",
      "Step 1 - Get some water boiling.\n",
      "Step 2 - Grab a cup and put a tea bag in it.\n",
      "Step 3 - Once the water is hot enough, pour it over the tea bag.\n",
      "Step 4 - Let it sit for a bit so the tea can steep.\n",
      "Step 5 - After a few minutes, take out the tea bag.\n",
      "Step 6 - Add some sugar or milk to taste.\n",
      "Step 7 - Enjoy your delicious cup of tea!\n"
     ]
    }
   ],
   "source": [
    "text_1 = f\"\"\"\n",
    "Making a cup of tea is easy! First, you need to get some \\ \n",
    "water boiling. While that's happening, \\ \n",
    "grab a cup and put a tea bag in it. Once the water is \\ \n",
    "hot enough, just pour it over the tea bag. \\ \n",
    "Let it sit for a bit so the tea can steep. After a \\ \n",
    "few minutes, take out the tea bag. If you \\ \n",
    "like, you can add some sugar or milk to taste. \\ \n",
    "And that's it! You've got yourself a delicious \\ \n",
    "cup of tea to enjoy.\n",
    "\"\"\"\n",
    "prompt = f\"\"\"\n",
    "You will be provided with text delimited by triple quotes. \n",
    "If it contains a sequence of instructions, \\ \n",
    "re-write those instructions in the following format:\n",
    "\n",
    "Step 1 - ...\n",
    "Step 2 - …\n",
    "…\n",
    "Step N - …\n",
    "\n",
    "If the text does not contain a sequence of instructions, \\ \n",
    "then simply write \\\"No steps provided.\\\"\n",
    "\n",
    "\\\"\\\"\\\"{text_1}\\\"\\\"\\\"\n",
    "\"\"\"\n",
    "response = get_completion(prompt)\n",
    "print(\"Completion for Text 1:\")\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Completion for Text 2:\n",
      "No steps provided.\n"
     ]
    }
   ],
   "source": [
    "text_2 = f\"\"\"\n",
    "The sun is shining brightly today, and the birds are \\\n",
    "singing. It's a beautiful day to go for a \\ \n",
    "walk in the park. The flowers are blooming, and the \\ \n",
    "trees are swaying gently in the breeze. People \\ \n",
    "are out and about, enjoying the lovely weather. \\ \n",
    "Some are having picnics, while others are playing \\ \n",
    "games or simply relaxing on the grass. It's a \\ \n",
    "perfect day to spend time outdoors and appreciate the \\ \n",
    "beauty of nature.\n",
    "\"\"\"\n",
    "prompt = f\"\"\"\n",
    "You will be provided with text delimited by triple quotes. \n",
    "If it contains a sequence of instructions, \\ \n",
    "re-write those instructions in the following format:\n",
    "\n",
    "Step 1 - ...\n",
    "Step 2 - …\n",
    "…\n",
    "Step N - …\n",
    "\n",
    "If the text does not contain a sequence of instructions, \\ \n",
    "then simply write \\\"No steps provided.\\\"\n",
    "\n",
    "\\\"\\\"\\\"{text_2}\\\"\\\"\\\"\n",
    "\"\"\"\n",
    "response = get_completion(prompt)\n",
    "print(\"Completion for Text 2:\")\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Tactic 4: \"Few-shot\" prompting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<grandparent>: Resilience is like a tree that bends with the wind but never breaks. It is the ability to bounce back from adversity and keep moving forward, even when things get tough. Just like a tree that grows stronger with each storm it weathers, resilience is a quality that can be developed and strengthened over time.\n"
     ]
    }
   ],
   "source": [
    "prompt = f\"\"\"\n",
    "Your task is to answer in a consistent style.\n",
    "\n",
    "<child>: Teach me about patience.\n",
    "\n",
    "<grandparent>: The river that carves the deepest \\ \n",
    "valley flows from a modest spring; the \\ \n",
    "grandest symphony originates from a single note; \\ \n",
    "the most intricate tapestry begins with a solitary thread.\n",
    "\n",
    "<child>: Teach me about resilience.\n",
    "\"\"\"\n",
    "response = get_completion(prompt)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Principle 2: Give the model time to “think”\n",
    "#### Tactic 1: Specify the steps required to complete a task"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Completion for prompt 1:\n",
      "Two siblings, Jack and Jill, go on a quest to fetch water from a hilltop well, but misfortune strikes as they both fall down the hill, yet they return home slightly battered but with their adventurous spirits undimmed.\n",
      "\n",
      "Deux frères et sœurs, Jack et Jill, partent en quête d'eau d'un puits au sommet d'une colline, mais ils tombent tous les deux et retournent chez eux légèrement meurtris mais avec leur esprit d'aventure intact. \n",
      "Noms: Jack, Jill.\n",
      "\n",
      "{\n",
      "\"french_summary\": \"Deux frères et sœurs, Jack et Jill, partent en quête d'eau d'un puits au sommet d'une colline, mais ils tombent tous les deux et retournent chez eux légèrement meurtris mais avec leur esprit d'aventure intact.\",\n",
      "\"num_names\": 2\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "text = f\"\"\"\n",
    "In a charming village, siblings Jack and Jill set out on \\ \n",
    "a quest to fetch water from a hilltop \\ \n",
    "well. As they climbed, singing joyfully, misfortune \\ \n",
    "struck—Jack tripped on a stone and tumbled \\ \n",
    "down the hill, with Jill following suit. \\ \n",
    "Though slightly battered, the pair returned home to \\ \n",
    "comforting embraces. Despite the mishap, \\ \n",
    "their adventurous spirits remained undimmed, and they \\ \n",
    "continued exploring with delight.\n",
    "\"\"\"\n",
    "# example 1\n",
    "prompt_1 = f\"\"\"\n",
    "Perform the following actions: \n",
    "1 - Summarize the following text delimited by triple \\\n",
    "backticks with 1 sentence.\n",
    "2 - Translate the summary into French.\n",
    "3 - List each name in the French summary.\n",
    "4 - Output a json object that contains the following \\\n",
    "keys: french_summary, num_names.\n",
    "\n",
    "Separate your answers with line breaks.\n",
    "\n",
    "Text:\n",
    "```{text}```\n",
    "\"\"\"\n",
    "response = get_completion(prompt_1)\n",
    "print(\"Completion for prompt 1:\")\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Ask for output in a specified format"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Completion for prompt 2:\n",
      "Summary: Jack and Jill go on a quest to fetch water, but misfortune strikes and they tumble down the hill, returning home slightly battered but with their adventurous spirits undimmed. \n",
      "Translation: Jack et Jill partent en quête d'eau, mais un malheur frappe et ils tombent de la colline, rentrant chez eux légèrement meurtris mais avec leurs esprits aventureux intacts.\n",
      "Names: Jack, Jill\n",
      "Output JSON: {\"french_summary\": \"Jack et Jill partent en quête d'eau, mais un malheur frappe et ils tombent de la colline, rentrant chez eux légèrement meurtris mais avec leurs esprits aventureux intacts.\", \"num_names\": 2}\n"
     ]
    }
   ],
   "source": [
    "prompt_2 = f\"\"\"\n",
    "Your task is to perform the following actions: \n",
    "1 - Summarize the following text delimited by \n",
    "  <> with 1 sentence.\n",
    "2 - Translate the summary into French.\n",
    "3 - List each name in the French summary.\n",
    "4 - Output a json object that contains the \n",
    "  following keys: french_summary, num_names.\n",
    "\n",
    "Use the following format:\n",
    "Text: <text to summarize>\n",
    "Summary: <summary>\n",
    "Translation: <summary translation>\n",
    "Names: <list of names in Italian summary>\n",
    "Output JSON: <json with summary and num_names>\n",
    "\n",
    "Text: <{text}>\n",
    "\"\"\"\n",
    "response = get_completion(prompt_2)\n",
    "print(\"\\nCompletion for prompt 2:\")\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Tactic 2: Instruct the model to work out its own solution before rushing to a conclusion"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The student's solution is correct.\n"
     ]
    }
   ],
   "source": [
    "prompt = f\"\"\"\n",
    "Determine if the student's solution is correct or not.\n",
    "\n",
    "Question:\n",
    "I'm building a solar power installation and I need \\\n",
    " help working out the financials. \n",
    "- Land costs $100 / square foot\n",
    "- I can buy solar panels for $250 / square foot\n",
    "- I negotiated a contract for maintenance that will cost \\ \n",
    "me a flat $100k per year, and an additional $10 / square \\\n",
    "foot\n",
    "What is the total cost for the first year of operations \n",
    "as a function of the number of square feet.\n",
    "\n",
    "Student's Solution:\n",
    "Let x be the size of the installation in square feet.\n",
    "Costs:\n",
    "1. Land cost: 100x\n",
    "2. Solar panel cost: 250x\n",
    "3. Maintenance cost: 100,000 + 100x\n",
    "Total cost: 100x + 250x + 100,000 + 100x = 450x + 100,000\n",
    "\"\"\"\n",
    "response = get_completion(prompt)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Note that the student's solution is actually not correct.\n",
    "#### We can fix this by instructing the model to work out its own solution first."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Let x be the size of the installation in square feet.\n",
      "\n",
      "Costs:\n",
      "1. Land cost: 100x\n",
      "2. Solar panel cost: 250x\n",
      "3. Maintenance cost: 100,000 + 10x\n",
      "\n",
      "Total cost: 100x + 250x + 100,000 + 10x = 360x + 100,000\n",
      "\n",
      "Is the student's solution the same as actual solution just calculated:\n",
      "No\n",
      "\n",
      "Student grade:\n",
      "Incorrect\n"
     ]
    }
   ],
   "source": [
    "prompt = f\"\"\"\n",
    "Your task is to determine if the student's solution \\\n",
    "is correct or not.\n",
    "To solve the problem do the following:\n",
    "- First, work out your own solution to the problem. \n",
    "- Then compare your solution to the student's solution \\ \n",
    "and evaluate if the student's solution is correct or not. \n",
    "Don't decide if the student's solution is correct until \n",
    "you have done the problem yourself.\n",
    "\n",
    "Use the following format:\n",
    "Question:\n",
    "```\n",
    "question here\n",
    "```\n",
    "Student's solution:\n",
    "```\n",
    "student's solution here\n",
    "```\n",
    "Actual solution:\n",
    "```\n",
    "steps to work out the solution and your solution here\n",
    "```\n",
    "Is the student's solution the same as actual solution \\\n",
    "just calculated:\n",
    "```\n",
    "yes or no\n",
    "```\n",
    "Student grade:\n",
    "```\n",
    "correct or incorrect\n",
    "```\n",
    "\n",
    "Question:\n",
    "```\n",
    "I'm building a solar power installation and I need help \\\n",
    "working out the financials. \n",
    "- Land costs $100 / square foot\n",
    "- I can buy solar panels for $250 / square foot\n",
    "- I negotiated a contract for maintenance that will cost \\\n",
    "me a flat $100k per year, and an additional $10 / square \\\n",
    "foot\n",
    "What is the total cost for the first year of operations \\\n",
    "as a function of the number of square feet.\n",
    "``` \n",
    "Student's solution:\n",
    "```\n",
    "Let x be the size of the installation in square feet.\n",
    "Costs:\n",
    "1. Land cost: 100x\n",
    "2. Solar panel cost: 250x\n",
    "3. Maintenance cost: 100,000 + 100x\n",
    "Total cost: 100x + 250x + 100,000 + 100x = 450x + 100,000\n",
    "```\n",
    "Actual solution:\n",
    "\"\"\"\n",
    "response = get_completion(prompt)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model Limitations: Hallucinations\n",
    "- Boie is a real company, the product name is not real."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The AeroGlide UltraSlim Smart Toothbrush by Boie is a high-tech toothbrush that uses advanced sonic technology to provide a deep and thorough clean. It features a slim and sleek design that makes it easy to hold and maneuver, and it comes with a range of smart features that help you optimize your brushing routine.\n",
      "\n",
      "One of the key features of the AeroGlide UltraSlim Smart Toothbrush is its advanced sonic technology, which uses high-frequency vibrations to break up plaque and bacteria on your teeth and gums. This technology is highly effective at removing even the toughest stains and buildup, leaving your teeth feeling clean and refreshed.\n",
      "\n",
      "In addition to its sonic technology, the AeroGlide UltraSlim Smart Toothbrush also comes with a range of smart features that help you optimize your brushing routine. These include a built-in timer that ensures you brush for the recommended two minutes, as well as a pressure sensor that alerts you if you're brushing too hard.\n",
      "\n",
      "Overall, the AeroGlide UltraSlim Smart Toothbrush by Boie is a highly advanced and effective toothbrush that is perfect for anyone looking to take their oral hygiene to the next level. With its advanced sonic technology and smart features, it provides a deep and thorough clean that leaves your teeth feeling fresh and healthy.\n"
     ]
    }
   ],
   "source": [
    "prompt = f\"\"\"\n",
    "Tell me about AeroGlide UltraSlim Smart Toothbrush by Boie\n",
    "\"\"\"\n",
    "response = get_completion(prompt)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Notes on using the OpenAI API outside of this classroom\n",
    "To install the OpenAI Python library:\n",
    "\n",
    "```\n",
    "!pip install openai\n",
    "```\n",
    "\n",
    "The library needs to be configured with your account's secret key, which is available on the [website](https://platform.openai.com/account/api-keys).\n",
    "\n",
    "You can either set it as the `OPENAI_API_KEY` environment variable before using the library:\n",
    "```\n",
    "!export OPENAI_API_KEY='sk-...'\n",
    "```\n",
    " \n",
    "Or, set `openai.api_key` to its value:\n",
    "\n",
    "```\n",
    "import openai\n",
    "openai.api_key = \"sk-...\"\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### A note about the backslash\n",
    "- In the course, we are using a backslash \\ to make the text fit on the screen without inserting newline '\\n' characters.\n",
    "- GPT-3 isn't really affected whether you insert newline characters or not. But when working with LLMs in general, you may consider whether newline characters in your prompt may affect the model's performance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}