6_RAG_langchain_231122.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "id": "86fc5bb2-017f-434e-8cd6-53ab214a5604",
      "metadata": {
        "id": "86fc5bb2-017f-434e-8cd6-53ab214a5604"
      },
      "source": [
        "# Retrieval-augmented generation (RAG)\n",
        "\n",
        "231122, version fixed\n",
        "\n",
        "-[ref](https://python.langchain.com/docs/use_cases/question_answering/)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "de913d6d-c57f-4927-82fe-18902a636861",
      "metadata": {
        "id": "de913d6d-c57f-4927-82fe-18902a636861"
      },
      "source": [
        "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1HkhOK7gec-M_QqREj_L_cC36Do3g0QpA?usp=sharing)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "5151afed",
      "metadata": {
        "id": "5151afed"
      },
      "source": [
        "## Overview\n",
        "\n",
        "### What is RAG?\n",
        "\n",
        "RAG is a technique for augmenting LLM knowledge with additional, often private or real-time, data.\n",
        "\n",
        "LLMs can reason about wide-ranging topics, but their knowledge is limited to the public data up to a specific point in time that they were trained on. If you want to build AI applications that can reason about private data or data introduced after a model's cutoff date, you need to augment the knowledge of the model with the specific information it needs. The process of bringing the appropriate information and inserting it into the model prompt is known as Retrieval Augmented Generation (RAG).\n",
        "\n",
        "### What's in this guide?\n",
        "\n",
        "LangChain has a number of components specifically designed to help build RAG applications. To familiarize ourselves with these, we'll build a simple question-answering application over a text data source. Specifically, we'll build a QA bot over the [LLM Powered Autonomous Agents](https://lilianweng.github.io/posts/2023-06-23-agent/) blog post by Lilian Weng. Along the way we'll go over a typical QA architecture, discuss the relevant LangChain components, and highlight additional resources for more advanced QA techniques. We'll also see how LangSmith can help us trace and understand our application. LangSmith will become increasingly helpful as our application grows in complexity.\n",
        "\n",
        "**Note**\n",
        "Here we focus on RAG for unstructured data. Two RAG use cases which we cover elsewhere are:\n",
        "- [QA over structured data](/docs/use_cases/qa_structured/sql) (e.g., SQL)\n",
        "- [QA over code](/docs/use_cases/question_answering/code_understanding) (e.g., Python)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "2f25cbbd-0938-4e3d-87e4-17a204a03ffb",
      "metadata": {
        "id": "2f25cbbd-0938-4e3d-87e4-17a204a03ffb"
      },
      "source": [
        "## Architecture\n",
        "A typical RAG application has two main components:\n",
        "\n",
        "**Indexing**: a pipeline for ingesting data from a source and indexing it. *This usually happen offline.*\n",
        "\n",
        "**Retrieval and generation**: the actual RAG chain, which takes the user query at run time and retrieves the relevant data from the index, then passes that to the model.\n",
        "\n",
        "The most common full sequence from raw data to answer looks like:\n",
        "\n",
        "#### Indexing\n",
        "1. **Load**: First we need to load our data. We'll use [DocumentLoaders](/docs/modules/data_connection/document_loaders/) for this.\n",
        "2. **Split**: [Text splitters](/docs/modules/data_connection/document_transformers/) break large `Documents` into smaller chunks. This is useful both for indexing data and for passing it in to a model, since large chunks are harder to search over and won't in a model's finite context window.\n",
        "3. **Store**: We need somewhere to store and index our splits, so that they can later be searched over. This is often done using a [VectorStore](/docs/modules/data_connection/vectorstores/) and [Embeddings](/docs/modules/data_connection/text_embedding/) model.\n",
        "\n",
        "![index_diagram](/img/rag_indexing.png)\n",
        "\n",
        "#### Retrieval and generation\n",
        "4. **Retrieve**: Given a user input, relevant splits are retrieved from storage using a [Retriever](/docs/modules/data_connection/retrievers/).\n",
        "5. **Generate**: A [ChatModel](/docs/modules/model_io/chat_models) / [LLM](/docs/modules/model_io/llms/) produces an answer using a prompt that includes the question and the retrieved data\n",
        "\n",
        "![retrieval_diagram](/img/rag_retrieval_generation.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "487d8d79-5ee9-4aa4-9fdf-cd5f4303e099",
      "metadata": {
        "id": "487d8d79-5ee9-4aa4-9fdf-cd5f4303e099"
      },
      "source": [
        "## Setup\n",
        "\n",
        "### Dependencies\n",
        "\n",
        "We'll use an OpenAI chat model and embeddings and a Chroma vector store in this walkthrough, but everything shown here works with any [ChatModel](/docs/integrations/chat/) or [LLM](/docs/integrations/llms/), [Embeddings](/docs/integrations/text_embedding/), and [VectorStore](/docs/integrations/vectorstores/) or [Retriever](/docs/integrations/retrievers).\n",
        "\n",
        "We'll use the following packages:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "28d272cd-4e31-40aa-bbb4-0be0a1f49a14",
      "metadata": {
        "id": "28d272cd-4e31-40aa-bbb4-0be0a1f49a14",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "260274ce-d756-4db5-964a-f40efd7eb7d1"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Collecting langchain==0.0.339\n",
            "  Downloading langchain-0.0.339-py3-none-any.whl (2.0 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.0/2.0 MB\u001b[0m \u001b[31m14.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting openai==1.3.4\n",
            "  Downloading openai-1.3.4-py3-none-any.whl (220 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m220.5/220.5 kB\u001b[0m \u001b[31m18.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting chromadb==0.4.17\n",
            "  Downloading chromadb-0.4.17-py3-none-any.whl (496 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m496.8/496.8 kB\u001b[0m \u001b[31m24.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting langchainhub==0.1.14\n",
            "  Downloading langchainhub-0.1.14-py3-none-any.whl (3.4 kB)\n",
            "Collecting bs4==0.0.1\n",
            "  Downloading bs4-0.0.1.tar.gz (1.1 kB)\n",
            "  Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "Collecting tiktoken==0.5.1\n",
            "  Downloading tiktoken-0.5.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.0 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.0/2.0 MB\u001b[0m \u001b[31m36.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: PyYAML>=5.3 in /usr/local/lib/python3.10/dist-packages (from langchain==0.0.339) (6.0.1)\n",
            "Requirement already satisfied: SQLAlchemy<3,>=1.4 in /usr/local/lib/python3.10/dist-packages (from langchain==0.0.339) (2.0.23)\n",
            "Requirement already satisfied: aiohttp<4.0.0,>=3.8.3 in /usr/local/lib/python3.10/dist-packages (from langchain==0.0.339) (3.8.6)\n",
            "Requirement already satisfied: anyio<4.0 in /usr/local/lib/python3.10/dist-packages (from langchain==0.0.339) (3.7.1)\n",
            "Requirement already satisfied: async-timeout<5.0.0,>=4.0.0 in /usr/local/lib/python3.10/dist-packages (from langchain==0.0.339) (4.0.3)\n",
            "Collecting dataclasses-json<0.7,>=0.5.7 (from langchain==0.0.339)\n",
            "  Downloading dataclasses_json-0.6.2-py3-none-any.whl (28 kB)\n",
            "Collecting jsonpatch<2.0,>=1.33 (from langchain==0.0.339)\n",
            "  Downloading jsonpatch-1.33-py2.py3-none-any.whl (12 kB)\n",
            "Collecting langsmith<0.1.0,>=0.0.63 (from langchain==0.0.339)\n",
            "  Downloading langsmith-0.0.66-py3-none-any.whl (46 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m46.8/46.8 kB\u001b[0m \u001b[31m6.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: numpy<2,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain==0.0.339) (1.23.5)\n",
            "Requirement already satisfied: pydantic<3,>=1 in /usr/local/lib/python3.10/dist-packages (from langchain==0.0.339) (1.10.13)\n",
            "Requirement already satisfied: requests<3,>=2 in /usr/local/lib/python3.10/dist-packages (from langchain==0.0.339) (2.31.0)\n",
            "Requirement already satisfied: tenacity<9.0.0,>=8.1.0 in /usr/local/lib/python3.10/dist-packages (from langchain==0.0.339) (8.2.3)\n",
            "Requirement already satisfied: distro<2,>=1.7.0 in /usr/lib/python3/dist-packages (from openai==1.3.4) (1.7.0)\n",
            "Collecting httpx<1,>=0.23.0 (from openai==1.3.4)\n",
            "  Downloading httpx-0.25.1-py3-none-any.whl (75 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m75.0/75.0 kB\u001b[0m \u001b[31m11.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: tqdm>4 in /usr/local/lib/python3.10/dist-packages (from openai==1.3.4) (4.66.1)\n",
            "Requirement already satisfied: typing-extensions<5,>=4.5 in /usr/local/lib/python3.10/dist-packages (from openai==1.3.4) (4.5.0)\n",
            "Collecting chroma-hnswlib==0.7.3 (from chromadb==0.4.17)\n",
            "  Downloading chroma_hnswlib-0.7.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.4 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.4/2.4 MB\u001b[0m \u001b[31m49.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting fastapi>=0.95.2 (from chromadb==0.4.17)\n",
            "  Downloading fastapi-0.104.1-py3-none-any.whl (92 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m92.9/92.9 kB\u001b[0m \u001b[31m11.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting uvicorn[standard]>=0.18.3 (from chromadb==0.4.17)\n",
            "  Downloading uvicorn-0.24.0.post1-py3-none-any.whl (59 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m59.7/59.7 kB\u001b[0m \u001b[31m9.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting posthog>=2.4.0 (from chromadb==0.4.17)\n",
            "  Downloading posthog-3.0.2-py2.py3-none-any.whl (37 kB)\n",
            "Collecting pulsar-client>=3.1.0 (from chromadb==0.4.17)\n",
            "  Downloading pulsar_client-3.3.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.4 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m5.4/5.4 MB\u001b[0m \u001b[31m74.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting onnxruntime>=1.14.1 (from chromadb==0.4.17)\n",
            "  Downloading onnxruntime-1.16.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (6.4 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.4/6.4 MB\u001b[0m \u001b[31m108.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting opentelemetry-api>=1.2.0 (from chromadb==0.4.17)\n",
            "  Downloading opentelemetry_api-1.21.0-py3-none-any.whl (57 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m57.9/57.9 kB\u001b[0m \u001b[31m8.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting opentelemetry-exporter-otlp-proto-grpc>=1.2.0 (from chromadb==0.4.17)\n",
            "  Downloading opentelemetry_exporter_otlp_proto_grpc-1.21.0-py3-none-any.whl (18 kB)\n",
            "Collecting opentelemetry-sdk>=1.2.0 (from chromadb==0.4.17)\n",
            "  Downloading opentelemetry_sdk-1.21.0-py3-none-any.whl (105 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m105.3/105.3 kB\u001b[0m \u001b[31m16.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: tokenizers>=0.13.2 in /usr/local/lib/python3.10/dist-packages (from chromadb==0.4.17) (0.15.0)\n",
            "Collecting pypika>=0.48.9 (from chromadb==0.4.17)\n",
            "  Downloading PyPika-0.48.9.tar.gz (67 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m67.3/67.3 kB\u001b[0m \u001b[31m9.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h  Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n",
            "  Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n",
            "  Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
            "Collecting overrides>=7.3.1 (from chromadb==0.4.17)\n",
            "  Downloading overrides-7.4.0-py3-none-any.whl (17 kB)\n",
            "Requirement already satisfied: importlib-resources in /usr/local/lib/python3.10/dist-packages (from chromadb==0.4.17) (6.1.1)\n",
            "Requirement already satisfied: grpcio>=1.58.0 in /usr/local/lib/python3.10/dist-packages (from chromadb==0.4.17) (1.59.2)\n",
            "Collecting bcrypt>=4.0.1 (from chromadb==0.4.17)\n",
            "  Downloading bcrypt-4.0.1-cp36-abi3-manylinux_2_28_x86_64.whl (593 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m593.7/593.7 kB\u001b[0m \u001b[31m54.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: typer>=0.9.0 in /usr/local/lib/python3.10/dist-packages (from chromadb==0.4.17) (0.9.0)\n",
            "Collecting kubernetes>=28.1.0 (from chromadb==0.4.17)\n",
            "  Downloading kubernetes-28.1.0-py2.py3-none-any.whl (1.6 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.6/1.6 MB\u001b[0m \u001b[31m85.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting types-requests<3.0.0.0,>=2.31.0.2 (from langchainhub==0.1.14)\n",
            "  Downloading types_requests-2.31.0.10-py3-none-any.whl (14 kB)\n",
            "Requirement already satisfied: beautifulsoup4 in /usr/local/lib/python3.10/dist-packages (from bs4==0.0.1) (4.11.2)\n",
            "Requirement already satisfied: regex>=2022.1.18 in /usr/local/lib/python3.10/dist-packages (from tiktoken==0.5.1) (2023.6.3)\n",
            "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.339) (23.1.0)\n",
            "Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.339) (3.3.2)\n",
            "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.339) (6.0.4)\n",
            "Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.339) (1.9.2)\n",
            "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.339) (1.4.0)\n",
            "Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.10/dist-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.339) (1.3.1)\n",
            "Requirement already satisfied: idna>=2.8 in /usr/local/lib/python3.10/dist-packages (from anyio<4.0->langchain==0.0.339) (3.4)\n",
            "Requirement already satisfied: sniffio>=1.1 in /usr/local/lib/python3.10/dist-packages (from anyio<4.0->langchain==0.0.339) (1.3.0)\n",
            "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<4.0->langchain==0.0.339) (1.1.3)\n",
            "Collecting marshmallow<4.0.0,>=3.18.0 (from dataclasses-json<0.7,>=0.5.7->langchain==0.0.339)\n",
            "  Downloading marshmallow-3.20.1-py3-none-any.whl (49 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m49.4/49.4 kB\u001b[0m \u001b[31m7.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting typing-inspect<1,>=0.4.0 (from dataclasses-json<0.7,>=0.5.7->langchain==0.0.339)\n",
            "  Downloading typing_inspect-0.9.0-py3-none-any.whl (8.8 kB)\n",
            "Collecting starlette<0.28.0,>=0.27.0 (from fastapi>=0.95.2->chromadb==0.4.17)\n",
            "  Downloading starlette-0.27.0-py3-none-any.whl (66 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m67.0/67.0 kB\u001b[0m \u001b[31m8.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting typing-extensions<5,>=4.5 (from openai==1.3.4)\n",
            "  Downloading typing_extensions-4.8.0-py3-none-any.whl (31 kB)\n",
            "Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.23.0->openai==1.3.4) (2023.7.22)\n",
            "Collecting httpcore (from httpx<1,>=0.23.0->openai==1.3.4)\n",
            "  Downloading httpcore-1.0.2-py3-none-any.whl (76 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m76.9/76.9 kB\u001b[0m \u001b[31m10.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting jsonpointer>=1.9 (from jsonpatch<2.0,>=1.33->langchain==0.0.339)\n",
            "  Downloading jsonpointer-2.4-py2.py3-none-any.whl (7.8 kB)\n",
            "Requirement already satisfied: six>=1.9.0 in /usr/local/lib/python3.10/dist-packages (from kubernetes>=28.1.0->chromadb==0.4.17) (1.16.0)\n",
            "Requirement already satisfied: python-dateutil>=2.5.3 in /usr/local/lib/python3.10/dist-packages (from kubernetes>=28.1.0->chromadb==0.4.17) (2.8.2)\n",
            "Requirement already satisfied: google-auth>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from kubernetes>=28.1.0->chromadb==0.4.17) (2.17.3)\n",
            "Requirement already satisfied: websocket-client!=0.40.0,!=0.41.*,!=0.42.*,>=0.32.0 in /usr/local/lib/python3.10/dist-packages (from kubernetes>=28.1.0->chromadb==0.4.17) (1.6.4)\n",
            "Requirement already satisfied: requests-oauthlib in /usr/local/lib/python3.10/dist-packages (from kubernetes>=28.1.0->chromadb==0.4.17) (1.3.1)\n",
            "Requirement already satisfied: oauthlib>=3.2.2 in /usr/local/lib/python3.10/dist-packages (from kubernetes>=28.1.0->chromadb==0.4.17) (3.2.2)\n",
            "Collecting urllib3<2.0,>=1.24.2 (from kubernetes>=28.1.0->chromadb==0.4.17)\n",
            "  Downloading urllib3-1.26.18-py2.py3-none-any.whl (143 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m143.8/143.8 kB\u001b[0m \u001b[31m21.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting coloredlogs (from onnxruntime>=1.14.1->chromadb==0.4.17)\n",
            "  Downloading coloredlogs-15.0.1-py2.py3-none-any.whl (46 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m46.0/46.0 kB\u001b[0m \u001b[31m6.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: flatbuffers in /usr/local/lib/python3.10/dist-packages (from onnxruntime>=1.14.1->chromadb==0.4.17) (23.5.26)\n",
            "Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from onnxruntime>=1.14.1->chromadb==0.4.17) (23.2)\n",
            "Requirement already satisfied: protobuf in /usr/local/lib/python3.10/dist-packages (from onnxruntime>=1.14.1->chromadb==0.4.17) (3.20.3)\n",
            "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from onnxruntime>=1.14.1->chromadb==0.4.17) (1.12)\n",
            "Collecting deprecated>=1.2.6 (from opentelemetry-api>=1.2.0->chromadb==0.4.17)\n",
            "  Downloading Deprecated-1.2.14-py2.py3-none-any.whl (9.6 kB)\n",
            "Requirement already satisfied: importlib-metadata<7.0,>=6.0 in /usr/local/lib/python3.10/dist-packages (from opentelemetry-api>=1.2.0->chromadb==0.4.17) (6.8.0)\n",
            "Collecting backoff<3.0.0,>=1.10.0 (from opentelemetry-exporter-otlp-proto-grpc>=1.2.0->chromadb==0.4.17)\n",
            "  Downloading backoff-2.2.1-py3-none-any.whl (15 kB)\n",
            "Requirement already satisfied: googleapis-common-protos~=1.52 in /usr/local/lib/python3.10/dist-packages (from opentelemetry-exporter-otlp-proto-grpc>=1.2.0->chromadb==0.4.17) (1.61.0)\n",
            "Collecting opentelemetry-exporter-otlp-proto-common==1.21.0 (from opentelemetry-exporter-otlp-proto-grpc>=1.2.0->chromadb==0.4.17)\n",
            "  Downloading opentelemetry_exporter_otlp_proto_common-1.21.0-py3-none-any.whl (17 kB)\n",
            "Collecting opentelemetry-proto==1.21.0 (from opentelemetry-exporter-otlp-proto-grpc>=1.2.0->chromadb==0.4.17)\n",
            "  Downloading opentelemetry_proto-1.21.0-py3-none-any.whl (50 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m50.8/50.8 kB\u001b[0m \u001b[31m7.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting opentelemetry-semantic-conventions==0.42b0 (from opentelemetry-sdk>=1.2.0->chromadb==0.4.17)\n",
            "  Downloading opentelemetry_semantic_conventions-0.42b0-py3-none-any.whl (36 kB)\n",
            "Collecting monotonic>=1.5 (from posthog>=2.4.0->chromadb==0.4.17)\n",
            "  Downloading monotonic-1.6-py2.py3-none-any.whl (8.2 kB)\n",
            "Requirement already satisfied: greenlet!=0.4.17 in /usr/local/lib/python3.10/dist-packages (from SQLAlchemy<3,>=1.4->langchain==0.0.339) (3.0.1)\n",
            "Requirement already satisfied: huggingface_hub<1.0,>=0.16.4 in /usr/local/lib/python3.10/dist-packages (from tokenizers>=0.13.2->chromadb==0.4.17) (0.19.3)\n",
            "Requirement already satisfied: click<9.0.0,>=7.1.1 in /usr/local/lib/python3.10/dist-packages (from typer>=0.9.0->chromadb==0.4.17) (8.1.7)\n",
            "INFO: pip is looking at multiple versions of types-requests to determine which version is compatible with other requirements. This could take a while.\n",
            "Collecting types-requests<3.0.0.0,>=2.31.0.2 (from langchainhub==0.1.14)\n",
            "  Downloading types_requests-2.31.0.9-py3-none-any.whl (14 kB)\n",
            "  Downloading types_requests-2.31.0.8-py3-none-any.whl (14 kB)\n",
            "  Downloading types_requests-2.31.0.7-py3-none-any.whl (14 kB)\n",
            "  Downloading types_requests-2.31.0.6-py3-none-any.whl (14 kB)\n",
            "Collecting types-urllib3 (from types-requests<3.0.0.0,>=2.31.0.2->langchainhub==0.1.14)\n",
            "  Downloading types_urllib3-1.26.25.14-py3-none-any.whl (15 kB)\n",
            "Collecting h11>=0.8 (from uvicorn[standard]>=0.18.3->chromadb==0.4.17)\n",
            "  Downloading h11-0.14.0-py3-none-any.whl (58 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m58.3/58.3 kB\u001b[0m \u001b[31m9.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting httptools>=0.5.0 (from uvicorn[standard]>=0.18.3->chromadb==0.4.17)\n",
            "  Downloading httptools-0.6.1-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (341 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m341.4/341.4 kB\u001b[0m \u001b[31m38.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting python-dotenv>=0.13 (from uvicorn[standard]>=0.18.3->chromadb==0.4.17)\n",
            "  Downloading python_dotenv-1.0.0-py3-none-any.whl (19 kB)\n",
            "Collecting uvloop!=0.15.0,!=0.15.1,>=0.14.0 (from uvicorn[standard]>=0.18.3->chromadb==0.4.17)\n",
            "  Downloading uvloop-0.19.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.4 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.4/3.4 MB\u001b[0m \u001b[31m76.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting watchfiles>=0.13 (from uvicorn[standard]>=0.18.3->chromadb==0.4.17)\n",
            "  Downloading watchfiles-0.21.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m91.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting websockets>=10.4 (from uvicorn[standard]>=0.18.3->chromadb==0.4.17)\n",
            "  Downloading websockets-12.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (130 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m130.2/130.2 kB\u001b[0m \u001b[31m17.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: soupsieve>1.2 in /usr/local/lib/python3.10/dist-packages (from beautifulsoup4->bs4==0.0.1) (2.5)\n",
            "Requirement already satisfied: wrapt<2,>=1.10 in /usr/local/lib/python3.10/dist-packages (from deprecated>=1.2.6->opentelemetry-api>=1.2.0->chromadb==0.4.17) (1.14.1)\n",
            "Requirement already satisfied: cachetools<6.0,>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from google-auth>=1.0.1->kubernetes>=28.1.0->chromadb==0.4.17) (5.3.2)\n",
            "Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.10/dist-packages (from google-auth>=1.0.1->kubernetes>=28.1.0->chromadb==0.4.17) (0.3.0)\n",
            "Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.10/dist-packages (from google-auth>=1.0.1->kubernetes>=28.1.0->chromadb==0.4.17) (4.9)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from huggingface_hub<1.0,>=0.16.4->tokenizers>=0.13.2->chromadb==0.4.17) (3.13.1)\n",
            "Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/dist-packages (from huggingface_hub<1.0,>=0.16.4->tokenizers>=0.13.2->chromadb==0.4.17) (2023.6.0)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.10/dist-packages (from importlib-metadata<7.0,>=6.0->opentelemetry-api>=1.2.0->chromadb==0.4.17) (3.17.0)\n",
            "Collecting mypy-extensions>=0.3.0 (from typing-inspect<1,>=0.4.0->dataclasses-json<0.7,>=0.5.7->langchain==0.0.339)\n",
            "  Downloading mypy_extensions-1.0.0-py3-none-any.whl (4.7 kB)\n",
            "Collecting humanfriendly>=9.1 (from coloredlogs->onnxruntime>=1.14.1->chromadb==0.4.17)\n",
            "  Downloading humanfriendly-10.0-py2.py3-none-any.whl (86 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m86.8/86.8 kB\u001b[0m \u001b[31m11.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: mpmath>=0.19 in /usr/local/lib/python3.10/dist-packages (from sympy->onnxruntime>=1.14.1->chromadb==0.4.17) (1.3.0)\n",
            "Requirement already satisfied: pyasn1<0.6.0,>=0.4.6 in /usr/local/lib/python3.10/dist-packages (from pyasn1-modules>=0.2.1->google-auth>=1.0.1->kubernetes>=28.1.0->chromadb==0.4.17) (0.5.0)\n",
            "Building wheels for collected packages: bs4, pypika\n",
            "  Building wheel for bs4 (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for bs4: filename=bs4-0.0.1-py3-none-any.whl size=1257 sha256=68562e36ca7cb892afd0c329eb7a37ef86f4da6e954b877c13de1b0a65ad5f99\n",
            "  Stored in directory: /root/.cache/pip/wheels/25/42/45/b773edc52acb16cd2db4cf1a0b47117e2f69bb4eb300ed0e70\n",
            "  Building wheel for pypika (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
            "  Created wheel for pypika: filename=PyPika-0.48.9-py2.py3-none-any.whl size=53723 sha256=8fb2e12834713053b9209a0799cb2a36c056c865a282388d8116e5222e6c3c40\n",
            "  Stored in directory: /root/.cache/pip/wheels/e1/26/51/d0bffb3d2fd82256676d7ad3003faea3bd6dddc9577af665f4\n",
            "Successfully built bs4 pypika\n",
            "Installing collected packages: types-urllib3, pypika, monotonic, websockets, uvloop, urllib3, typing-extensions, types-requests, python-dotenv, pulsar-client, overrides, opentelemetry-semantic-conventions, opentelemetry-proto, mypy-extensions, marshmallow, jsonpointer, humanfriendly, httptools, h11, deprecated, chroma-hnswlib, bcrypt, backoff, watchfiles, uvicorn, typing-inspect, starlette, opentelemetry-exporter-otlp-proto-common, opentelemetry-api, jsonpatch, httpcore, coloredlogs, bs4, tiktoken, posthog, opentelemetry-sdk, onnxruntime, langsmith, langchainhub, httpx, fastapi, dataclasses-json, opentelemetry-exporter-otlp-proto-grpc, openai, langchain, kubernetes, chromadb\n",
            "  Attempting uninstall: urllib3\n",
            "    Found existing installation: urllib3 2.0.7\n",
            "    Uninstalling urllib3-2.0.7:\n",
            "      Successfully uninstalled urllib3-2.0.7\n",
            "  Attempting uninstall: typing-extensions\n",
            "    Found existing installation: typing_extensions 4.5.0\n",
            "    Uninstalling typing_extensions-4.5.0:\n",
            "      Successfully uninstalled typing_extensions-4.5.0\n",
            "\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "lida 0.0.10 requires kaleido, which is not installed.\n",
            "lida 0.0.10 requires python-multipart, which is not installed.\n",
            "llmx 0.0.15a0 requires cohere, which is not installed.\n",
            "tensorflow-probability 0.22.0 requires typing-extensions<4.6.0, but you have typing-extensions 4.8.0 which is incompatible.\u001b[0m\u001b[31m\n",
            "\u001b[0mSuccessfully installed backoff-2.2.1 bcrypt-4.0.1 bs4-0.0.1 chroma-hnswlib-0.7.3 chromadb-0.4.17 coloredlogs-15.0.1 dataclasses-json-0.6.2 deprecated-1.2.14 fastapi-0.104.1 h11-0.14.0 httpcore-1.0.2 httptools-0.6.1 httpx-0.25.1 humanfriendly-10.0 jsonpatch-1.33 jsonpointer-2.4 kubernetes-28.1.0 langchain-0.0.339 langchainhub-0.1.14 langsmith-0.0.66 marshmallow-3.20.1 monotonic-1.6 mypy-extensions-1.0.0 onnxruntime-1.16.3 openai-1.3.4 opentelemetry-api-1.21.0 opentelemetry-exporter-otlp-proto-common-1.21.0 opentelemetry-exporter-otlp-proto-grpc-1.21.0 opentelemetry-proto-1.21.0 opentelemetry-sdk-1.21.0 opentelemetry-semantic-conventions-0.42b0 overrides-7.4.0 posthog-3.0.2 pulsar-client-3.3.0 pypika-0.48.9 python-dotenv-1.0.0 starlette-0.27.0 tiktoken-0.5.1 types-requests-2.31.0.6 types-urllib3-1.26.25.14 typing-extensions-4.8.0 typing-inspect-0.9.0 urllib3-1.26.18 uvicorn-0.24.0.post1 uvloop-0.19.0 watchfiles-0.21.0 websockets-12.0\n"
          ]
        }
      ],
      "source": [
        "!pip install langchain==0.0.339 openai==1.3.4 chromadb==0.4.17 langchainhub==0.1.14 bs4==0.0.1 tiktoken==0.5.1"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "51ef48de-70b6-4f43-8e0b-ab9b84c9c02a",
      "metadata": {
        "id": "51ef48de-70b6-4f43-8e0b-ab9b84c9c02a"
      },
      "source": [
        "We need to set environment variable `OPENAI_API_KEY`, which can be done directly or loaded from a `.env` file like so:"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 0) OpenAI API키 설정\n",
        "\n",
        "- [API key 확인](https://platform.openai.com/api-keys)\n",
        "- [API key 발급받기](https://m.blog.naver.com/mynameistk/223062993136)\n",
        "\n",
        "본인 OPENAI_API_KEY를 아래에 입력하세요(예시: OPENAI_API_KEY: 'sk-xxxxxxxx')"
      ],
      "metadata": {
        "id": "zbhgktjYstpL"
      },
      "id": "zbhgktjYstpL"
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "143787ca-d8e6-4dc9-8281-4374f4d71720",
      "metadata": {
        "id": "143787ca-d8e6-4dc9-8281-4374f4d71720"
      },
      "outputs": [],
      "source": [
        "import getpass\n",
        "import os\n",
        "\n",
        "OPENAI_API_KEY='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'\n",
        "os.environ['OPENAI_API_KEY'] = OPENAI_API_KEY"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "fa6ba684-26cf-4860-904e-a4d51380c134",
      "metadata": {
        "id": "fa6ba684-26cf-4860-904e-a4d51380c134"
      },
      "source": [
        "## Quickstart\n",
        "\n",
        "Suppose we want to build a QA app over the [LLM Powered Autonomous Agents](https://lilianweng.github.io/posts/2023-06-23-agent/) blog post by Lilian Weng. We can create a simple pipeline for this in ~20 lines of code:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "d8a913b1-0eea-442a-8a64-ec73333f104b",
      "metadata": {
        "id": "d8a913b1-0eea-442a-8a64-ec73333f104b"
      },
      "outputs": [],
      "source": [
        "import bs4\n",
        "from langchain import hub\n",
        "from langchain.chat_models import ChatOpenAI\n",
        "from langchain.document_loaders import WebBaseLoader\n",
        "from langchain.embeddings import OpenAIEmbeddings\n",
        "from langchain.schema import StrOutputParser\n",
        "from langchain.schema.runnable import RunnablePassthrough\n",
        "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
        "from langchain.vectorstores import Chroma"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "820244ae-74b4-4593-b392-822979dd91b8",
      "metadata": {
        "id": "820244ae-74b4-4593-b392-822979dd91b8"
      },
      "outputs": [],
      "source": [
        "loader = WebBaseLoader(\n",
        "    web_paths=(\"https://lilianweng.github.io/posts/2023-06-23-agent/\",),\n",
        "    bs_kwargs=dict(\n",
        "        parse_only=bs4.SoupStrainer(\n",
        "            class_=(\"post-content\", \"post-title\", \"post-header\")\n",
        "        )\n",
        "    ),\n",
        ")\n",
        "docs = loader.load()\n",
        "\n",
        "text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)\n",
        "splits = text_splitter.split_documents(docs)\n",
        "\n",
        "vectorstore = Chroma.from_documents(documents=splits, embedding=OpenAIEmbeddings())\n",
        "retriever = vectorstore.as_retriever()\n",
        "\n",
        "prompt = hub.pull(\"rlm/rag-prompt\")\n",
        "llm = ChatOpenAI(model_name=\"gpt-3.5-turbo\", temperature=0)\n",
        "\n",
        "\n",
        "def format_docs(docs):\n",
        "    return \"\\n\\n\".join(doc.page_content for doc in docs)\n",
        "\n",
        "\n",
        "rag_chain = (\n",
        "    {\"context\": retriever | format_docs, \"question\": RunnablePassthrough()}\n",
        "    | prompt\n",
        "    | llm\n",
        "    | StrOutputParser()\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "0d3b0f36-7b56-49c0-8e40-a1aa9ebcbf24",
      "metadata": {
        "id": "0d3b0f36-7b56-49c0-8e40-a1aa9ebcbf24",
        "outputId": "007d2b9b-af1b-4062-f8c6-e022f31b4fab",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 54
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'Task decomposition is a technique used to break down complex tasks into smaller and simpler steps. It can be done through prompting techniques like Chain of Thought (CoT) or Tree of Thoughts, which guide the model to think step by step and explore multiple reasoning possibilities. Task decomposition can also be achieved through task-specific instructions or human inputs.'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 6
        }
      ],
      "source": [
        "rag_chain.invoke(\"What is Task Decomposition?\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "7cb344e0-c423-400c-a079-964c08e07e32",
      "metadata": {
        "id": "7cb344e0-c423-400c-a079-964c08e07e32"
      },
      "outputs": [],
      "source": [
        "# cleanup\n",
        "vectorstore.delete_collection()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "639dc31a-7f16-40f6-ba2a-20e7c2ecfe60",
      "metadata": {
        "id": "639dc31a-7f16-40f6-ba2a-20e7c2ecfe60"
      },
      "source": [
        ":::tip Check out the [LangSmith trace](https://smith.langchain.com/public/1c6ca97e-445b-4d00-84b4-c7befcbc59fe/r)\n",
        ":::"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "842cf72d-abbc-468e-a2eb-022470347727",
      "metadata": {
        "id": "842cf72d-abbc-468e-a2eb-022470347727"
      },
      "source": [
        "## Detailed walkthrough\n",
        "\n",
        "Let's go through the above code step-by-step to really understand what's going on."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "ba5daed6",
      "metadata": {
        "id": "ba5daed6"
      },
      "source": [
        "## Step 1. Load\n",
        "\n",
        "We need to first load the blog post contents. We can use `DocumentLoader`s for this, which are objects that load in data from a source as `Documents`.  A `Document` is an object with `page_content` (str) and `metadata` (dict) attributes.\n",
        "\n",
        "In this case we'll use the `WebBaseLoader`, which uses `urllib` and `BeautifulSoup` to load and parse the passed in web urls, returning one `Document` per url. We can customize the html -> text parsing by passing in parameters to the `BeautifulSoup` parser via `bs_kwargs` (see [BeautifulSoup docs](https://beautiful-soup-4.readthedocs.io/en/latest/#beautifulsoup)). In this case only HTML tags with class \"post-content\", \"post-title\", or \"post-header\" are relevant, so we'll remove all others."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "cf4d5c72",
      "metadata": {
        "id": "cf4d5c72"
      },
      "outputs": [],
      "source": [
        "from langchain.document_loaders import WebBaseLoader\n",
        "\n",
        "loader = WebBaseLoader(\n",
        "    web_paths=(\"https://lilianweng.github.io/posts/2023-06-23-agent/\",),\n",
        "    bs_kwargs={\n",
        "        \"parse_only\": bs4.SoupStrainer(\n",
        "            class_=(\"post-content\", \"post-title\", \"post-header\")\n",
        "        )\n",
        "    },\n",
        ")\n",
        "docs = loader.load()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "207f87a3-effa-4457-b013-6d233bc7a088",
      "metadata": {
        "id": "207f87a3-effa-4457-b013-6d233bc7a088",
        "outputId": "b031032d-0820-488a-a687-def58f1372fc",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "42824"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ],
      "source": [
        "len(docs[0].page_content)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "52469796-5ce4-4c12-bd2a-a903872dac33",
      "metadata": {
        "id": "52469796-5ce4-4c12-bd2a-a903872dac33",
        "outputId": "0bfbaa4c-f4fd-4ea8-abc6-b76185167df9",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "\n",
            "      LLM Powered Autonomous Agents\n",
            "    \n",
            "Date: June 23, 2023  |  Estimated Reading Time: 31 min  |  Author: Lilian Weng\n",
            "\n",
            "\n",
            "Building agents with LLM (large language model) as its core controller is a cool concept. Several proof-of-concepts demos, such as AutoGPT, GPT-Engineer and BabyAGI, serve as inspiring examples. The potentiality of LLM extends beyond generating well-written copies, stories, essays and programs; it can be framed as a powerful general problem solver.\n",
            "Agent System Overview#\n",
            "In a LLM-powered autonomous agent system, LLM functions as the agent’s brain, complemented by several key components:\n",
            "\n",
            "Planning\n",
            "\n",
            "Subgoal and decomposition: The agent breaks down large tasks into smaller, manageable subgoals, enabling efficient handling of complex tasks.\n",
            "Reflection and refinement: The agent can do self-criticism and self-reflection over past actions, learn from mistakes and refine them for future steps, thereby improving the quality of final results.\n",
            "\n",
            "\n",
            "Memory\n",
            "\n",
            "Short-term memory: I would consider all the in-context learning (See Prompt Engineering) as utilizing short-term memory of the model to learn.\n",
            "Long-term memory: This provides the agent with the capability to retain and recall (infinite) information over extended periods, often by leveraging an external vector store and fast retrieval.\n",
            "\n",
            "\n",
            "Tool use\n",
            "\n",
            "The agent learns to call external APIs for extra information that is missing from the model weights (often hard to change after pre-training), including current information\n"
          ]
        }
      ],
      "source": [
        "print(docs[0].page_content[:1500])"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "ee5c6556-56be-4067-adbc-98b5aa19ef6e",
      "metadata": {
        "id": "ee5c6556-56be-4067-adbc-98b5aa19ef6e"
      },
      "source": [
        "### Go deeper\n",
        "`DocumentLoader`: Object that load data from a source as `Documents`.\n",
        "- [Docs](/docs/modules/data_connection/document_loaders/): Further documentation on how to use `DocumentLoader`s.\n",
        "- [Integrations](/docs/integrations/document_loaders/): Find the relevant `DocumentLoader` integration (of the > 160 of them) for your use case."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "fd2cc9a7",
      "metadata": {
        "id": "fd2cc9a7"
      },
      "source": [
        "## Step 2. Split\n",
        "\n",
        "Our loaded document is over 42k characters long. This is too long to fit in the context window of many models. And even for those models that could fit the full post in their context window, empirically models struggle to find the relevant context in very long prompts.\n",
        "\n",
        "So we'll split the `Document` into chunks for embedding and vector storage. This should help us retrieve only the most relevant bits of the blog post at run time.\n",
        "\n",
        "In this case we'll split our documents into chunks of 1000 characters with 200 characters of overlap between chunks. The overlap helps mitigate the possibility of separating a statement from important context related to it. We use the `RecursiveCharacterTextSplitter`, which will (recursively) split the document using common separators (like new lines) until each chunk is the appropriate size."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "4b11c01d",
      "metadata": {
        "id": "4b11c01d"
      },
      "outputs": [],
      "source": [
        "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
        "\n",
        "text_splitter = RecursiveCharacterTextSplitter(\n",
        "    chunk_size=1000, chunk_overlap=200, add_start_index=True\n",
        ")\n",
        "all_splits = text_splitter.split_documents(docs)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "3741eb67-9caf-40f2-a001-62f49349bff5",
      "metadata": {
        "id": "3741eb67-9caf-40f2-a001-62f49349bff5",
        "outputId": "efba0a3f-182e-41ec-dd9a-d7dc2393a3b1",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "66"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ],
      "source": [
        "len(all_splits)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "f868d0e5-5670-4d54-b562-f50265e907f4",
      "metadata": {
        "id": "f868d0e5-5670-4d54-b562-f50265e907f4",
        "outputId": "cc5780a7-a66d-46db-e2ff-c5f8df20fcae",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "969"
            ]
          },
          "metadata": {},
          "execution_count": 13
        }
      ],
      "source": [
        "len(all_splits[0].page_content)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "5c9e5f27-c8e3-4ca7-8a8e-45c5de2901cc",
      "metadata": {
        "id": "5c9e5f27-c8e3-4ca7-8a8e-45c5de2901cc",
        "outputId": "62b6be8f-4a10-4897-d7fc-88ffaf8598dd",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'source': 'https://lilianweng.github.io/posts/2023-06-23-agent/',\n",
              " 'start_index': 7056}"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ],
      "source": [
        "all_splits[10].metadata"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "0a33bd4d",
      "metadata": {
        "id": "0a33bd4d"
      },
      "source": [
        "### Go deeper\n",
        "\n",
        "`DocumentSplitter`: Object that splits a list of `Document`s into smaller chunks. Subclass of `DocumentTransformer`s.\n",
        "- Explore `Context-aware splitters`, which keep the location (\"context\") of each split in the original `Document`:\n",
        "    - [Markdown files](/docs/use_cases/question_answering/document-context-aware-QA)\n",
        "    - [Code (py or js)](docs/integrations/document_loaders/source_code)\n",
        "    - [Scientific papers](/docs/integrations/document_loaders/grobid)\n",
        "\n",
        "`DocumentTransformer`: Object that performs a transformation on a list of `Document`s.\n",
        "- [Docs](/docs/modules/data_connection/document_transformers/): Further documentation on how to use `DocumentTransformer`s\n",
        "- [Integrations](/docs/integrations/document_transformers/)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "46547031-2352-4321-9970-d6ea27285c2e",
      "metadata": {
        "id": "46547031-2352-4321-9970-d6ea27285c2e"
      },
      "source": [
        "## Step 3. Store\n",
        "\n",
        "Now that we've got 66 text chunks in memory, we need to store and index them so that we can search them later in our RAG app. The most common way to do this is to embed the contents of each document split and upload those embeddings to a vector store.\n",
        "\n",
        "Then, when we want to search over our splits, we take the search query, embed it as well, and perform some sort of \"similarity\" search to identify the stored splits with the most similar embeddings to our query embedding. The simplest similarity measure is cosine similarity — we measure the cosine of the angle between each pair of embeddings (which are just very high dimensional vectors).\n",
        "\n",
        "We can embed and store all of our document splits in a single command using the `Chroma` vector store and `OpenAIEmbeddings` model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "e9c302c8",
      "metadata": {
        "id": "e9c302c8"
      },
      "outputs": [],
      "source": [
        "from langchain.embeddings import OpenAIEmbeddings\n",
        "from langchain.vectorstores import Chroma\n",
        "\n",
        "vectorstore = Chroma.from_documents(documents=all_splits, embedding=OpenAIEmbeddings())"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "dc6f22b0",
      "metadata": {
        "id": "dc6f22b0"
      },
      "source": [
        "### Go deeper\n",
        "`Embeddings`: Wrapper around a text embedding model, used for converting text to embeddings.\n",
        "- [Docs](/docs/modules/data_connection/text_embedding): Further documentation on the interface.\n",
        "- [Integrations](/docs/integrations/text_embedding/): Browse the > 30 text embedding integrations\n",
        "\n",
        "`VectorStore`: Wrapper around a vector database, used for storing and querying embeddings.\n",
        "- [Docs](/docs/modules/data_connection/vectorstores/): Further documentation on the interface.\n",
        "- [Integrations](/docs/integrations/vectorstores/): Browse the > 40 `VectorStore` integrations.\n",
        "\n",
        "This completes the **Indexing** portion of the pipeline. At this point we have an query-able vector store containing the chunked contents of our blog post. Given a user question, we should ideally be able to return the snippets of the blog post that answer the question:"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "70d64d40-e475-43d9-b64c-925922bb5ef7",
      "metadata": {
        "id": "70d64d40-e475-43d9-b64c-925922bb5ef7"
      },
      "source": [
        "## Step 4. Retrieve\n",
        "\n",
        "Now let's write the actual application logic. We want to create a simple application that let's the user ask a question, searches for documents relevant to that question, passes the retrieved documents and initial question to a model, and finally returns an answer.\n",
        "\n",
        "LangChain defines a `Retriever` interface which wraps an index that can return relevant documents given a string query. All retrievers implement a common method `get_relevant_documents()` (and its asynchronous variant `aget_relevant_documents()`).\n",
        "\n",
        "The most common type of `Retriever` is the `VectorStoreRetriever`, which uses the similarity search capabilities of a vector store to facillitate retrieval. Any `VectorStore` can easily be turned into a `Retriever`:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "4414df0d-5d43-46d0-85a9-5f47be0dd099",
      "metadata": {
        "id": "4414df0d-5d43-46d0-85a9-5f47be0dd099"
      },
      "outputs": [],
      "source": [
        "retriever = vectorstore.as_retriever(search_type=\"similarity\", search_kwargs={\"k\": 6})"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "e2c26b7d",
      "metadata": {
        "id": "e2c26b7d"
      },
      "outputs": [],
      "source": [
        "retrieved_docs = retriever.get_relevant_documents(\n",
        "    \"What are the approaches to Task Decomposition?\"\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "8684291d-0f5e-453a-8d3e-ff9feea765d0",
      "metadata": {
        "id": "8684291d-0f5e-453a-8d3e-ff9feea765d0",
        "outputId": "bc29ebca-4146-422d-85b9-0a7e1f0d47f1",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "6"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ],
      "source": [
        "len(retrieved_docs)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "9a5dc074-816d-409a-b005-ab4eddfd76af",
      "metadata": {
        "id": "9a5dc074-816d-409a-b005-ab4eddfd76af",
        "outputId": "8ac0ecfa-1bda-4c50-aff9-05ccb97b080e",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Tree of Thoughts (Yao et al. 2023) extends CoT by exploring multiple reasoning possibilities at each step. It first decomposes the problem into multiple thought steps and generates multiple thoughts per step, creating a tree structure. The search process can be BFS (breadth-first search) or DFS (depth-first search) with each state evaluated by a classifier (via a prompt) or majority vote.\n",
            "Task decomposition can be done (1) by LLM with simple prompting like \"Steps for XYZ.\\n1.\", \"What are the subgoals for achieving XYZ?\", (2) by using task-specific instructions; e.g. \"Write a story outline.\" for writing a novel, or (3) with human inputs.\n"
          ]
        }
      ],
      "source": [
        "print(retrieved_docs[0].page_content)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "5d5a113b",
      "metadata": {
        "id": "5d5a113b"
      },
      "source": [
        "### Go deeper\n",
        "Vector stores are commonly used for retrieval, but there are plenty of other ways to do retrieval.\n",
        "\n",
        "`Retriever`: An object that returns `Document`s given a text query\n",
        "- [Docs](/docs/modules/data_connection/retrievers/): Further documentation on the interface and built-in retrieval techniques. Some of which include:\n",
        "    - `MultiQueryRetriever` [generates variants of the input question](/docs/modules/data_connection/retrievers/MultiQueryRetriever) to improve retrieval hit rate.\n",
        "    - `MultiVectorRetriever` (diagram below) instead generates [variants of the embeddings](/docs/modules/data_connection/retrievers/multi_vector), also in order to improve retrieval hit rate.\n",
        "    - `Max marginal relevance` selects for [relevance and diversity](https://www.cs.cmu.edu/~jgc/publication/The_Use_MMR_Diversity_Based_LTMIR_1998.pdf) among the retrieved documents to avoid passing in duplicate context.\n",
        "    - Documents can be filtered during vector store retrieval using [`metadata` filters](/docs/use_cases/question_answering/document-context-aware-QA).\n",
        "- [Integrations](/docs/integrations/retrievers/): Integrations with retrieval services."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "415d6824",
      "metadata": {
        "id": "415d6824"
      },
      "source": [
        "## Step 5. Generate\n",
        "\n",
        "Let's put it all together into a chain that takes a question, retrieves relevant documents, constructs a prompt, passes that to a model, and parses the output.\n",
        "\n",
        "We'll use the gpt-3.5-turbo OpenAI chat model, but any LangChain `LLM` or `ChatModel` could be substituted in."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "d34d998c-9abf-4e01-a4ad-06dadfcf131c",
      "metadata": {
        "id": "d34d998c-9abf-4e01-a4ad-06dadfcf131c"
      },
      "outputs": [],
      "source": [
        "from langchain.chat_models import ChatOpenAI\n",
        "\n",
        "llm = ChatOpenAI(model_name=\"gpt-3.5-turbo\", temperature=0)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "bc826723-36fc-45d1-a3ef-df8c2c8471a8",
      "metadata": {
        "id": "bc826723-36fc-45d1-a3ef-df8c2c8471a8"
      },
      "source": [
        "We'll use a prompt for RAG that is checked into the LangChain prompt hub ([here](https://smith.langchain.com/hub/rlm/rag-prompt))."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "bede955b-9aeb-4fd3-964d-8e43f214ce70",
      "metadata": {
        "id": "bede955b-9aeb-4fd3-964d-8e43f214ce70"
      },
      "outputs": [],
      "source": [
        "from langchain import hub\n",
        "\n",
        "prompt = hub.pull(\"rlm/rag-prompt\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "11c35354-f275-47ec-9f72-ebd5c23731eb",
      "metadata": {
        "id": "11c35354-f275-47ec-9f72-ebd5c23731eb",
        "outputId": "980ef8a5-4df2-4103-e11d-a1bc17df1889",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Human: You are an assistant for question-answering tasks. Use the following pieces of retrieved context to answer the question. If you don't know the answer, just say that you don't know. Use three sentences maximum and keep the answer concise.\n",
            "Question: filler question \n",
            "Context: filler context \n",
            "Answer:\n"
          ]
        }
      ],
      "source": [
        "print(\n",
        "    prompt.invoke(\n",
        "        {\"context\": \"filler context\", \"question\": \"filler question\"}\n",
        "    ).to_string()\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "51f9a210-1eee-4054-99d7-9d9ddf7e3593",
      "metadata": {
        "id": "51f9a210-1eee-4054-99d7-9d9ddf7e3593"
      },
      "source": [
        "We'll use the [LCEL Runnable](https://python.langchain.com/docs/expression_language/) protocol to define the chain, allowing us to\n",
        "- pipe together components and functions in a transparent way\n",
        "- automatically trace our chain in LangSmith\n",
        "- get streaming, async, and batched calling out of the box"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "99fa1aec",
      "metadata": {
        "id": "99fa1aec"
      },
      "outputs": [],
      "source": [
        "from langchain.schema import StrOutputParser\n",
        "from langchain.schema.runnable import RunnablePassthrough\n",
        "\n",
        "\n",
        "def format_docs(docs):\n",
        "    return \"\\n\\n\".join(doc.page_content for doc in docs)\n",
        "\n",
        "\n",
        "rag_chain = (\n",
        "    {\"context\": retriever | format_docs, \"question\": RunnablePassthrough()}\n",
        "    | prompt\n",
        "    | llm\n",
        "    | StrOutputParser()\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "8655a152-d7cf-466f-b1bc-fbff9ae2b889",
      "metadata": {
        "id": "8655a152-d7cf-466f-b1bc-fbff9ae2b889",
        "outputId": "dc4878fe-ce73-4386-e27a-a8f746f043cb",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Task decomposition is a technique used to break down complex tasks into smaller and simpler steps. It involves transforming big tasks into multiple manageable tasks, allowing for easier interpretation and execution by autonomous agents or models. Task decomposition can be done through various methods, such as using prompting techniques, task-specific instructions, or human inputs."
          ]
        }
      ],
      "source": [
        "for chunk in rag_chain.stream(\"What is Task Decomposition?\"):\n",
        "    print(chunk, end=\"\", flush=True)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "2c000e5f-2b7f-4eb9-8876-9f4b186b4a08",
      "metadata": {
        "id": "2c000e5f-2b7f-4eb9-8876-9f4b186b4a08"
      },
      "source": [
        ":::tip Check out the [LangSmith trace](https://smith.langchain.com/public/1799e8db-8a6d-4eb2-84d5-46e8d7d5a99b/r)\n",
        ":::"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "f7d52c84",
      "metadata": {
        "id": "f7d52c84"
      },
      "source": [
        "### Go deeper\n",
        "\n",
        "#### Choosing LLMs\n",
        "`ChatModel`: An LLM-backed chat model wrapper. Takes in a sequence of messages and returns a message.\n",
        "- [Docs](/docs/modules/model_io/chat/)\n",
        "- [Integrations](/docs/integrations/chat/): Explore over 25 `ChatModel` integrations.\n",
        "\n",
        "`LLM`: A text-in-text-out LLM. Takes in a string and returns a string.\n",
        "- [Docs](/docs/modules/model_io/llms)\n",
        "- [Integrations](/docs/integrations/llms): Explore over 75 `LLM` integrations.\n",
        "\n",
        "See a guide on RAG with locally-running models [here](/docs/modules/use_cases/question_answering/local_retrieval_qa)."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "fa82f437",
      "metadata": {
        "id": "fa82f437"
      },
      "source": [
        "#### Customizing the prompt\n",
        "\n",
        "As shown above, we can load prompts (e.g., [this RAG prompt](https://smith.langchain.com/hub/rlm/rag-prompt)) from the prompt hub. The prompt can also be easily customized:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "e4fee704",
      "metadata": {
        "id": "e4fee704",
        "outputId": "90dd4070-8e3f-4c35-df0d-02288195a041",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 54
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'Task decomposition is a technique used to break down complex tasks into smaller and simpler steps. It involves transforming big tasks into multiple manageable tasks, allowing for a more systematic and organized approach to problem-solving. Thanks for asking!'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 25
        }
      ],
      "source": [
        "from langchain.prompts import PromptTemplate\n",
        "\n",
        "template = \"\"\"Use the following pieces of context to answer the question at the end.\n",
        "If you don't know the answer, just say that you don't know, don't try to make up an answer.\n",
        "Use three sentences maximum and keep the answer as concise as possible.\n",
        "Always say \"thanks for asking!\" at the end of the answer.\n",
        "{context}\n",
        "Question: {question}\n",
        "Helpful Answer:\"\"\"\n",
        "rag_prompt_custom = PromptTemplate.from_template(template)\n",
        "\n",
        "rag_chain = (\n",
        "    {\"context\": retriever | format_docs, \"question\": RunnablePassthrough()}\n",
        "    | rag_prompt_custom\n",
        "    | llm\n",
        "    | StrOutputParser()\n",
        ")\n",
        "\n",
        "rag_chain.invoke(\"What is Task Decomposition?\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "94b952e6-dc4b-415b-9cf3-1ad333e48366",
      "metadata": {
        "id": "94b952e6-dc4b-415b-9cf3-1ad333e48366"
      },
      "source": [
        ":::tip Check out the [LangSmith trace](https://smith.langchain.com/public/da23c4d8-3b33-47fd-84df-a3a582eedf84/r)\n",
        ":::"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "1c2f99b5-80b4-4178-bf30-c1c0a152638f",
      "metadata": {
        "id": "1c2f99b5-80b4-4178-bf30-c1c0a152638f"
      },
      "source": [
        "### Adding sources\n",
        "\n",
        "With LCEL it's easy to return the retrieved documents or certain source metadata from the documents:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "ded41680-b749-4e2a-9daa-b1165d74783b",
      "metadata": {
        "id": "ded41680-b749-4e2a-9daa-b1165d74783b",
        "outputId": "d323b828-9e97-4d72-ed7d-379eeb5fffff",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'documents': [{'source': 'https://lilianweng.github.io/posts/2023-06-23-agent/',\n",
              "   'start_index': 1585},\n",
              "  {'source': 'https://lilianweng.github.io/posts/2023-06-23-agent/',\n",
              "   'start_index': 2192},\n",
              "  {'source': 'https://lilianweng.github.io/posts/2023-06-23-agent/',\n",
              "   'start_index': 17804},\n",
              "  {'source': 'https://lilianweng.github.io/posts/2023-06-23-agent/',\n",
              "   'start_index': 17414},\n",
              "  {'source': 'https://lilianweng.github.io/posts/2023-06-23-agent/',\n",
              "   'start_index': 29630},\n",
              "  {'source': 'https://lilianweng.github.io/posts/2023-06-23-agent/',\n",
              "   'start_index': 19373}],\n",
              " 'answer': 'Task decomposition is a technique used to break down complex tasks into smaller and simpler steps. It involves transforming big tasks into multiple manageable tasks, allowing for a more systematic and organized approach to problem-solving. Thanks for asking!'}"
            ]
          },
          "metadata": {},
          "execution_count": 26
        }
      ],
      "source": [
        "from operator import itemgetter\n",
        "\n",
        "from langchain.schema.runnable import RunnableMap\n",
        "\n",
        "rag_chain_from_docs = (\n",
        "    {\n",
        "        \"context\": lambda input: format_docs(input[\"documents\"]),\n",
        "        \"question\": itemgetter(\"question\"),\n",
        "    }\n",
        "    | rag_prompt_custom\n",
        "    | llm\n",
        "    | StrOutputParser()\n",
        ")\n",
        "rag_chain_with_source = RunnableMap(\n",
        "    {\"documents\": retriever, \"question\": RunnablePassthrough()}\n",
        ") | {\n",
        "    \"documents\": lambda input: [doc.metadata for doc in input[\"documents\"]],\n",
        "    \"answer\": rag_chain_from_docs,\n",
        "}\n",
        "\n",
        "rag_chain_with_source.invoke(\"What is Task Decomposition\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "b437da5d-ca09-4d15-9be2-c35e5a1ace77",
      "metadata": {
        "id": "b437da5d-ca09-4d15-9be2-c35e5a1ace77"
      },
      "source": [
        ":::tip Check out the [LangSmith trace](https://smith.langchain.com/public/007d7e01-cb62-4a84-8b71-b24767f953ee/r)\n",
        ":::"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "776ae958-cbdc-4471-8669-c6087436f0b5",
      "metadata": {
        "id": "776ae958-cbdc-4471-8669-c6087436f0b5"
      },
      "source": [
        "### Adding memory\n",
        "\n",
        "Suppose we want to create a stateful application that remembers past user inputs. There are two main things we need to do to support this.\n",
        "1. Add a messages placeholder to our chain which allows us to pass in historical messages\n",
        "2. Add a chain that takes the latest user query and reformulates it in the context of the chat history into a standalone question that can be passed to our retriever.\n",
        "\n",
        "Let's start with 2. We can build a \"condense question\" chain that looks something like this:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "2b685428-8b82-4af1-be4f-7232c5d55b73",
      "metadata": {
        "id": "2b685428-8b82-4af1-be4f-7232c5d55b73"
      },
      "outputs": [],
      "source": [
        "from langchain.prompts import ChatPromptTemplate, MessagesPlaceholder\n",
        "\n",
        "condense_q_system_prompt = \"\"\"Given a chat history and the latest user question \\\n",
        "which might reference the chat history, formulate a standalone question \\\n",
        "which can be understood without the chat history. Do NOT answer the question, \\\n",
        "just reformulate it if needed and otherwise return it as is.\"\"\"\n",
        "condense_q_prompt = ChatPromptTemplate.from_messages(\n",
        "    [\n",
        "        (\"system\", condense_q_system_prompt),\n",
        "        MessagesPlaceholder(variable_name=\"chat_history\"),\n",
        "        (\"human\", \"{question}\"),\n",
        "    ]\n",
        ")\n",
        "condense_q_chain = condense_q_prompt | llm | StrOutputParser()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "46ee9aa1-16f1-4509-8dae-f8c71f4ad47d",
      "metadata": {
        "id": "46ee9aa1-16f1-4509-8dae-f8c71f4ad47d",
        "outputId": "8d29f08d-1ff3-401d-d046-88640267bd4e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'What is the definition of \"large\" in the context of a language model?'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 28
        }
      ],
      "source": [
        "from langchain.schema.messages import AIMessage, HumanMessage\n",
        "\n",
        "condense_q_chain.invoke(\n",
        "    {\n",
        "        \"chat_history\": [\n",
        "            HumanMessage(content=\"What does LLM stand for?\"),\n",
        "            AIMessage(content=\"Large language model\"),\n",
        "        ],\n",
        "        \"question\": \"What is meant by large\",\n",
        "    }\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "31ee8481-ce37-41ae-8ca5-62196619d4b3",
      "metadata": {
        "id": "31ee8481-ce37-41ae-8ca5-62196619d4b3",
        "outputId": "e622fd3c-f1b8-4ab4-be26-88f2a7535524",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'How do transformer models function?'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 29
        }
      ],
      "source": [
        "condense_q_chain.invoke(\n",
        "    {\n",
        "        \"chat_history\": [\n",
        "            HumanMessage(content=\"What does LLM stand for?\"),\n",
        "            AIMessage(content=\"Large language model\"),\n",
        "        ],\n",
        "        \"question\": \"How do transformers work\",\n",
        "    }\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "42a47168-4a1f-4e39-bd2d-d5b03609a243",
      "metadata": {
        "id": "42a47168-4a1f-4e39-bd2d-d5b03609a243"
      },
      "source": [
        "And now we can build our full QA chain. Notice we add some routing functionality to only run the \"condense question chain\" when our chat history isn't empty."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "66f275f3-ddef-4678-b90d-ee64576878f9",
      "metadata": {
        "id": "66f275f3-ddef-4678-b90d-ee64576878f9"
      },
      "outputs": [],
      "source": [
        "qa_system_prompt = \"\"\"You are an assistant for question-answering tasks. \\\n",
        "Use the following pieces of retrieved context to answer the question. \\\n",
        "If you don't know the answer, just say that you don't know. \\\n",
        "Use three sentences maximum and keep the answer concise.\\\n",
        "\n",
        "{context}\"\"\"\n",
        "qa_prompt = ChatPromptTemplate.from_messages(\n",
        "    [\n",
        "        (\"system\", qa_system_prompt),\n",
        "        MessagesPlaceholder(variable_name=\"chat_history\"),\n",
        "        (\"human\", \"{question}\"),\n",
        "    ]\n",
        ")\n",
        "\n",
        "\n",
        "def condense_question(input: dict):\n",
        "    if input.get(\"chat_history\"):\n",
        "        return condense_q_chain\n",
        "    else:\n",
        "        return input[\"question\"]\n",
        "\n",
        "\n",
        "rag_chain = (\n",
        "    RunnablePassthrough.assign(context=condense_question | retriever | format_docs)\n",
        "    | qa_prompt\n",
        "    | llm\n",
        ")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "51fd0e54-5bb4-4a9a-b012-87a18ebe2bef",
      "metadata": {
        "id": "51fd0e54-5bb4-4a9a-b012-87a18ebe2bef",
        "outputId": "c2c75930-66ea-4d0d-fbba-bc18a14bcfeb",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "AIMessage(content='Common ways of task decomposition include:\\n\\n1. Using Chain of Thought (CoT): CoT is a prompting technique that instructs a model to \"think step by step\" and decompose complex tasks into smaller and simpler steps. This approach utilizes more computation at test time and sheds light on the model\\'s thinking process.\\n\\n2. Prompting with LLM: Language Model (LLM) can be used to prompt the model with simple instructions like \"Steps for XYZ\" or \"What are the subgoals for achieving XYZ?\" This helps in breaking down the task into manageable subtasks.\\n\\n3. Task-specific instructions: For certain tasks, specific instructions can be provided to guide the model in decomposing the task. For example, for writing a novel, an instruction like \"Write a story outline\" can be given to help break down the task into smaller components.\\n\\n4. Human inputs: In some cases, human inputs can be used to assist in task decomposition. Humans can provide insights, expertise, and domain knowledge to help break down complex tasks into more manageable subtasks.')"
            ]
          },
          "metadata": {},
          "execution_count": 31
        }
      ],
      "source": [
        "chat_history = []\n",
        "\n",
        "question = \"What is Task Decomposition?\"\n",
        "ai_msg = rag_chain.invoke({\"question\": question, \"chat_history\": chat_history})\n",
        "chat_history.extend([HumanMessage(content=question), ai_msg])\n",
        "\n",
        "second_question = \"What are common ways of doing it?\"\n",
        "rag_chain.invoke({\"question\": second_question, \"chat_history\": chat_history})"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "53263a65-4de2-4dd8-9291-6a8169ab6f1d",
      "metadata": {
        "id": "53263a65-4de2-4dd8-9291-6a8169ab6f1d"
      },
      "source": [
        ":::tip Check out the [LangSmith trace](https://smith.langchain.com/public/b3001782-bb30-476a-886b-12da17ec258f/r)\n",
        ":::"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "fdf6c7e0-84f8-4747-b2ae-e84315152bd9",
      "metadata": {
        "id": "fdf6c7e0-84f8-4747-b2ae-e84315152bd9"
      },
      "source": [
        "Here we've gone over how to add chain logic for incorporating historical outputs. But how do we actually store and retrieve historical outputs for different sessions? For that check out the LCEL [How to add message history (memory)](/docs/expression_language/how_to/message_history) page."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "580e18de-132d-4009-ba67-4aaf2c7717a2",
      "metadata": {
        "id": "580e18de-132d-4009-ba67-4aaf2c7717a2"
      },
      "source": [
        "## Next steps\n",
        "\n",
        "That's a lot of content we've covered in a short amount of time. There's plenty of nuances, features, integrations, etc to explore in each of the above sections. Aside from the sources mentioned above, good next steps include:\n",
        "\n",
        "- Reading up on more advanced retrieval techniques in the [Retrievers](/docs/modules/data_connection/retrievers/) section.\n",
        "- Learning about the LangChain [Indexing API](/docs/modules/data_connection/indexing), which helps repeatedly sync data sources and vector stores without redundant computation or storage.\n",
        "- Exploring RAG [LangChain Templates](/docs/templates/#-advanced-retrieval), which are reference applications that can easily be deployed with [LangServe](/docs/langserve).\n",
        "- Learning about [evaluating RAG applications with LangSmith](https://github.com/langchain-ai/langsmith-cookbook/blob/main/testing-examples/qa-correctness/qa-correctness.ipynb)."
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    },
    "colab": {
      "provenance": [],
      "machine_shape": "hm",
      "gpuType": "V100"
    },
    "accelerator": "GPU"
  },
  "nbformat": 4,
  "nbformat_minor": 5
}