rag-with-discovery.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Retrieval Augmented Generation w/ Watson Discovery & watsonx.ai\n",
    "\n",
    "This notebook introduces RAG (Retrieval Augmented Generation) by making use of several IBM tools including watsonx.ai to generate answers and garner insights from documents and Watson Discovery as an API to search and answer questions about business documents using custom natural language processing."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This lab should take 30-40 minutes.\n",
    "\n",
    "Before we begin lets start off by ensuring we have completed some pre-requisites; ensure you gave the following\n",
    "\n",
    "- IBM Cloud API key \n",
    "- Project ID associated with your watsonx instance\n",
    "- Project ID associated with your Watson Discovery instanace \n",
    "- Service URL for Watson Discovery instance\n",
    "\n",
    "You can use the following support links if you need any help with the pre-requisites above\n",
    "\n",
    "- [Creating IBM Cloud API Key](https://cloud.ibm.com/docs/account?topic=account-userapikey&interface=ui#create_user_key)\n",
    "- [Finding watsonx Project ID](https://www.ibm.com/docs/en/watsonx-as-a-service?topic=library-project-id)\n",
    "- [Finding Watson Discovery Project ID](https://cloud.ibm.com/docs/discovery-data?topic=discovery-data-getting-started)\n",
    "- [Determining Watson Discovery Service URL](https://cloud.ibm.com/apidocs/discovery-data?code=python#endpoint-cloud)\n",
    "    - Note this support link shows you the base url, you will need to find the instance url from resource list in the IBM Cloud dashboard."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import the necessary packages\n",
    "\n",
    "from ibm_watson import DiscoveryV2\n",
    "from ibm_watson.discovery_v2 import QueryLargePassages\n",
    "from ibm_cloud_sdk_core.authenticators import IAMAuthenticator\n",
    "from ibm_cloud_sdk_core import IAMTokenManager\n",
    "from ibm_watson_machine_learning.foundation_models import Model\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Setting up authentication and Watson Discovery\n",
    "\n",
    "Watson Discovery is a cognitive search and content analytics engine that you can add to applications to identify patterns, \n",
    "trends and actionable insights to drive better decision-making. For this lab, we will be making use of Watson Discovery \n",
    "to store, analyze, and search the documents which will act as a knowledge base for the question we will eventually be \n",
    "asking watsonx.ai.\n",
    "\n",
    "If you are completing this lab in a workshop setting, a Watson Discovery instance with a collection of documents may \n",
    "already be set-up for you; ask your lab instructor. If not, you can create your own Watson Discovery instance from \n",
    "IBM Cloud with your own collection of documents, for consistency of this lab you can find a link to the documents used \n",
    "for this collections linked [here](https://github.com/ibm-build-lab/VAD-VAR-Workshop/tree/87155f66db7248994ff17fc0dfe80a3b99b64fc9/content/Watsonx/WatsonxAI/docs)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your IBM Cloud API key\n",
    "\n",
    "# Project ID of your watson discovery collection instance \n",
    "discovery_project_id = \"\"\n",
    "\n",
    "# Service URL for your watson discovery instance check your DISCOVERY INSTANCE LOCATION and check link above\n",
    "service_url = 'https://api.us-south.discovery.watson.cloud.ibm.com'\n",
    "\n",
    "# Project ID of your watsonx project\n",
    "watsonx_project_id = \"\"\n",
    "\n",
    "# Use the api key from the Discovery page in IBM cloud services\n",
    "# This is NOT your cloud account api key\n",
    "discovery_api_key = \"\"\n",
    "\n",
    "ibm_cloud_url = \"https://us-south.ml.cloud.ibm.com\"\n",
    "\n",
    "# Create IBM IAM authenticator object using your Discovery API Key\n",
    "authenticator = IAMAuthenticator(discovery_api_key)\n",
    "\n",
    "# Create an Watson Discovery object using the authenticator object previously created\n",
    "discovery = DiscoveryV2(\n",
    "    version='2020-08-30',\n",
    "    authenticator=authenticator\n",
    ")\n",
    "\n",
    "# Set the Watson Discovery service url within our discovery object\n",
    "discovery.set_service_url(service_url)\n",
    "print(\"done\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Natural language query for Watson Discovery collection\n",
    "discovery_question=\"What is the outstanding long term debt at the end of 2023?\"\n",
    "\n",
    "def query_discovery(question):\n",
    "  passages = {\n",
    "    \"enabled\": True, \n",
    "    \"per_document\": True, \n",
    "    \"find_answers\": True,\n",
    "    \"max_per_document\": 1, \n",
    "    \"characters\": 500\n",
    "   }\n",
    "\n",
    "  query_large_passages_model = QueryLargePassages.from_dict(passages)\n",
    "\n",
    "  return discovery.query(\n",
    "          project_id=discovery_project_id,\n",
    "          natural_language_query=question,\n",
    "          passages=query_large_passages_model,\n",
    "          count=1\n",
    "      ).get_result()\n",
    "\n",
    "\n",
    "discovery_json = query_discovery(discovery_question)\n",
    "print(\"done\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Watson Discovery returns a list of documents broken into passages from a given collection\n",
    "# We want to combine this passages so it can serve as a knowledge base that we provide to watsonx.ai\n",
    "disc_results = []\n",
    "combined_disc_results = []\n",
    "for doc_index in range(len(discovery_json[\"results\"])):\n",
    "    for j in range(len(discovery_json[\"results\"][doc_index])):\n",
    "        passages = discovery_json[\"results\"][doc_index][\"document_passages\"]\n",
    "        disc_results = []\n",
    "        for item in passages:\n",
    "            item = item[\"passage_text\"].replace(\"<em>\",\"\")\n",
    "            item = item.replace(\"</em>\", \"\")\n",
    "            disc_results.append(item)\n",
    "    combined_disc_results.append(\"\\n\".join(disc_results))\n",
    "print(\"done\")\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# LLM that we want to use with watsonx.ai\n",
    "model_id= \"google/flan-ul2\"\n",
    "\n",
    "endpoint= \"https://us-south.ml.cloud.ibm.com\"\n",
    "\n",
    "access_token = ''\n",
    "\n",
    "# Here put your cloud account api key in, you can generate it from IAM \n",
    "cloud_account_api_key = ''\n",
    "\n",
    "try:\n",
    "  access_token = IAMTokenManager(\n",
    "    apikey = cloud_account_api_key,\n",
    "    url = \"https://iam.cloud.ibm.com/identity/token\"\n",
    "  ).get_token()\n",
    "except:\n",
    "  print('Issue obtaining access token. Check variables?')\n",
    "\n",
    "credentials = { \n",
    "    \"url\"    : endpoint, \n",
    "    \"token\" : access_token\n",
    "}\n",
    "\n",
    "# watsonx.ai tuning parameters\n",
    "gen_params = {\n",
    "    \"DECODING_METHOD\" : \"greedy\",\n",
    "    \"MAX_NEW_TOKENS\" : 500,\n",
    "    \"MIN_NEW_TOKENS\" : 2,\n",
    "    \"STREAM\" : False,\n",
    "    \"TEMPERATURE\" : 1.7,\n",
    "    \"TOP_K\" : 50,\n",
    "    \"TOP_P\" : 1,\n",
    "    \"RANDOM_SEED\" : 10\n",
    "}\n",
    "\n",
    "model = Model( model_id, credentials, gen_params, watsonx_project_id )\n",
    "print(\"done\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#######################################################################################\n",
    "prompt_template = \"\"\"\n",
    "Article:\n",
    "###\n",
    "%s\n",
    "###\n",
    "\n",
    "Answer the following question using only information from the article. \n",
    "Answer in a complete sentence, with proper capitalization and punctuation. \n",
    "If there is no good answer in the article, say \"I don't know\".\n",
    "\n",
    "Question: %s\n",
    "Answer: \n",
    "\"\"\"\n",
    "\n",
    "#######################################################################################\n",
    "def augment( template_in, context_in, query_in ):\n",
    "    return template_in % ( context_in,  query_in )\n",
    "\n",
    "\n",
    "#######################################################################################\n",
    "import json\n",
    "\n",
    "def generate( model_in, augmented_prompt_in ):\n",
    "    \n",
    "    generated_response = model_in.generate( augmented_prompt_in )\n",
    "\n",
    "    if ( \"results\" in generated_response ) \\\n",
    "       and ( len( generated_response[\"results\"] ) > 0 ) \\\n",
    "       and ( \"generated_text\" in generated_response[\"results\"][0] ):\n",
    "        return generated_response[\"results\"][0][\"generated_text\"]\n",
    "    else:\n",
    "        print( \"The model failed to generate an answer\" )\n",
    "        print( \"\\nDebug info:\\n\" + json.dumps( generated_response, indent=3 ) )\n",
    "        return \"\"\n",
    "\n",
    "\n",
    "########################################################################################\n",
    "import re\n",
    "\n",
    "def query_watsonx(question):\n",
    "    augmented_prompt = augment( prompt_template, combined_disc_results, question )\n",
    "    output = generate( model, augmented_prompt )\n",
    "    if not re.match( r\"\\S+\", output ):\n",
    "        print( \"The model failed to generate an answer\")\n",
    "    print( \"\\nAnswer:\\n\" + output )\n",
    "\n",
    "\n",
    "query_watsonx(discovery_question)\n",
    "\n",
    "print(\"\\ndone\")\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.10",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}