1 CDD ML Bioactivity Data.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "f69ebab5",
   "metadata": {},
   "source": [
    "## Computational Drug Discovery - Bioactivity Data\n",
    "\n",
    "In this project, I will be building a machine learning model using the ChEMBL bioactivity data.\n",
    "\n",
    "In **Part 1**, I will be performing data collection and pre-processing from the ChEMBL Database.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8375fed5",
   "metadata": {},
   "source": [
    "## **ChEMBL Database**\n",
    "\n",
    "The [*ChEMBL Database*](https://www.ebi.ac.uk/chembl/) is a database that contains curated bioactivity data of more than 2 million compounds. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "de9bed28",
   "metadata": {},
   "source": [
    "## Installing libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "ce4fa499",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: chembl_webresource_client in /Users/kiannahendricks/opt/anaconda3/lib/python3.9/site-packages (0.10.8)\n",
      "Requirement already satisfied: requests>=2.18.4 in /Users/kiannahendricks/opt/anaconda3/lib/python3.9/site-packages (from chembl_webresource_client) (2.27.1)\n",
      "Requirement already satisfied: requests-cache~=0.7.0 in /Users/kiannahendricks/opt/anaconda3/lib/python3.9/site-packages (from chembl_webresource_client) (0.7.5)\n",
      "Requirement already satisfied: easydict in /Users/kiannahendricks/opt/anaconda3/lib/python3.9/site-packages (from chembl_webresource_client) (1.10)\n",
      "Requirement already satisfied: urllib3 in /Users/kiannahendricks/opt/anaconda3/lib/python3.9/site-packages (from chembl_webresource_client) (1.26.9)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /Users/kiannahendricks/opt/anaconda3/lib/python3.9/site-packages (from requests>=2.18.4->chembl_webresource_client) (2021.10.8)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /Users/kiannahendricks/opt/anaconda3/lib/python3.9/site-packages (from requests>=2.18.4->chembl_webresource_client) (3.3)\n",
      "Requirement already satisfied: charset-normalizer~=2.0.0 in /Users/kiannahendricks/opt/anaconda3/lib/python3.9/site-packages (from requests>=2.18.4->chembl_webresource_client) (2.0.4)\n",
      "Requirement already satisfied: attrs<22.0,>=21.2 in /Users/kiannahendricks/opt/anaconda3/lib/python3.9/site-packages (from requests-cache~=0.7.0->chembl_webresource_client) (21.4.0)\n",
      "Requirement already satisfied: itsdangerous>=2.0.1 in /Users/kiannahendricks/opt/anaconda3/lib/python3.9/site-packages (from requests-cache~=0.7.0->chembl_webresource_client) (2.0.1)\n",
      "Requirement already satisfied: url-normalize<2.0,>=1.4 in /Users/kiannahendricks/opt/anaconda3/lib/python3.9/site-packages (from requests-cache~=0.7.0->chembl_webresource_client) (1.4.3)\n",
      "Requirement already satisfied: pyyaml>=5.4 in /Users/kiannahendricks/opt/anaconda3/lib/python3.9/site-packages (from requests-cache~=0.7.0->chembl_webresource_client) (6.0)\n",
      "Requirement already satisfied: six in /Users/kiannahendricks/opt/anaconda3/lib/python3.9/site-packages (from url-normalize<2.0,>=1.4->requests-cache~=0.7.0->chembl_webresource_client) (1.16.0)\n"
     ]
    }
   ],
   "source": [
    "# install the ChEMBL web service package so I can download the biological activity data directly from the ChEMBL Database\n",
    "! pip install chembl_webresource_client"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2696e10",
   "metadata": {},
   "source": [
    "## Importing libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "a2703b08",
   "metadata": {},
   "outputs": [],
   "source": [
    "# necessary libraries\n",
    "import pandas as pd\n",
    "from chembl_webresource_client.new_client import new_client"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3294b0b",
   "metadata": {},
   "source": [
    "## Search for Target protein"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "970902de",
   "metadata": {},
   "source": [
    "### Target search for Beta Secretase 1\n",
    "I will be choosing a single protein for further investigation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 273,
   "id": "7d68bc39",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cross_references</th>\n",
       "      <th>organism</th>\n",
       "      <th>pref_name</th>\n",
       "      <th>score</th>\n",
       "      <th>species_group_flag</th>\n",
       "      <th>target_chembl_id</th>\n",
       "      <th>target_components</th>\n",
       "      <th>target_type</th>\n",
       "      <th>tax_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>[]</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta-secretase (BACE)</td>\n",
       "      <td>20.0</td>\n",
       "      <td>False</td>\n",
       "      <td>CHEMBL2111390</td>\n",
       "      <td>[{'accession': 'Q9Y5Z0', 'component_descriptio...</td>\n",
       "      <td>PROTEIN FAMILY</td>\n",
       "      <td>9606.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>[]</td>\n",
       "      <td>Rattus norvegicus</td>\n",
       "      <td>Beta-secretase 2</td>\n",
       "      <td>19.0</td>\n",
       "      <td>False</td>\n",
       "      <td>CHEMBL2331066</td>\n",
       "      <td>[{'accession': 'Q6IE75', 'component_descriptio...</td>\n",
       "      <td>SINGLE PROTEIN</td>\n",
       "      <td>10116.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>[{'xref_id': 'Q9Y5Z0', 'xref_name': None, 'xre...</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta secretase 2</td>\n",
       "      <td>18.0</td>\n",
       "      <td>False</td>\n",
       "      <td>CHEMBL2525</td>\n",
       "      <td>[{'accession': 'Q9Y5Z0', 'component_descriptio...</td>\n",
       "      <td>SINGLE PROTEIN</td>\n",
       "      <td>9606.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>[{'xref_id': 'Beta-secretase_1', 'xref_name': ...</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta-secretase 1</td>\n",
       "      <td>18.0</td>\n",
       "      <td>False</td>\n",
       "      <td>CHEMBL4822</td>\n",
       "      <td>[{'accession': 'P56817', 'component_descriptio...</td>\n",
       "      <td>SINGLE PROTEIN</td>\n",
       "      <td>9606.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>[{'xref_id': 'P56818', 'xref_name': None, 'xre...</td>\n",
       "      <td>Mus musculus</td>\n",
       "      <td>Beta-secretase 1</td>\n",
       "      <td>18.0</td>\n",
       "      <td>False</td>\n",
       "      <td>CHEMBL4593</td>\n",
       "      <td>[{'accession': 'P56818', 'component_descriptio...</td>\n",
       "      <td>SINGLE PROTEIN</td>\n",
       "      <td>10090.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1160</th>\n",
       "      <td>[]</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Caspase</td>\n",
       "      <td>1.0</td>\n",
       "      <td>False</td>\n",
       "      <td>CHEMBL3831289</td>\n",
       "      <td>[{'accession': 'P49662', 'component_descriptio...</td>\n",
       "      <td>PROTEIN FAMILY</td>\n",
       "      <td>9606.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1161</th>\n",
       "      <td>[]</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>mTORC1</td>\n",
       "      <td>1.0</td>\n",
       "      <td>False</td>\n",
       "      <td>CHEMBL4296661</td>\n",
       "      <td>[{'accession': 'P42345', 'component_descriptio...</td>\n",
       "      <td>PROTEIN COMPLEX</td>\n",
       "      <td>9606.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1162</th>\n",
       "      <td>[]</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>mTORC2</td>\n",
       "      <td>1.0</td>\n",
       "      <td>False</td>\n",
       "      <td>CHEMBL4523999</td>\n",
       "      <td>[{'accession': 'P42345', 'component_descriptio...</td>\n",
       "      <td>PROTEIN COMPLEX</td>\n",
       "      <td>9606.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1163</th>\n",
       "      <td>[{'xref_id': 'C3TDZ2', 'xref_name': None, 'xre...</td>\n",
       "      <td>Escherichia coli</td>\n",
       "      <td>3-oxoacyl-[acyl-carrier-protein] synthase 3</td>\n",
       "      <td>0.0</td>\n",
       "      <td>False</td>\n",
       "      <td>CHEMBL1795135</td>\n",
       "      <td>[{'accession': 'C3TDZ2', 'component_descriptio...</td>\n",
       "      <td>SINGLE PROTEIN</td>\n",
       "      <td>562.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1164</th>\n",
       "      <td>[]</td>\n",
       "      <td>Bacteria</td>\n",
       "      <td>Bacterial 70S ribosome</td>\n",
       "      <td>0.0</td>\n",
       "      <td>True</td>\n",
       "      <td>CHEMBL2363135</td>\n",
       "      <td>[{'accession': 'P0A7V0', 'component_descriptio...</td>\n",
       "      <td>PROTEIN NUCLEIC-ACID COMPLEX</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1165 rows × 9 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                       cross_references           organism  \\\n",
       "0                                                    []       Homo sapiens   \n",
       "1                                                    []  Rattus norvegicus   \n",
       "2     [{'xref_id': 'Q9Y5Z0', 'xref_name': None, 'xre...       Homo sapiens   \n",
       "3     [{'xref_id': 'Beta-secretase_1', 'xref_name': ...       Homo sapiens   \n",
       "4     [{'xref_id': 'P56818', 'xref_name': None, 'xre...       Mus musculus   \n",
       "...                                                 ...                ...   \n",
       "1160                                                 []       Homo sapiens   \n",
       "1161                                                 []       Homo sapiens   \n",
       "1162                                                 []       Homo sapiens   \n",
       "1163  [{'xref_id': 'C3TDZ2', 'xref_name': None, 'xre...   Escherichia coli   \n",
       "1164                                                 []           Bacteria   \n",
       "\n",
       "                                        pref_name  score  species_group_flag  \\\n",
       "0                           Beta-secretase (BACE)   20.0               False   \n",
       "1                                Beta-secretase 2   19.0               False   \n",
       "2                                Beta secretase 2   18.0               False   \n",
       "3                                Beta-secretase 1   18.0               False   \n",
       "4                                Beta-secretase 1   18.0               False   \n",
       "...                                           ...    ...                 ...   \n",
       "1160                                      Caspase    1.0               False   \n",
       "1161                                       mTORC1    1.0               False   \n",
       "1162                                       mTORC2    1.0               False   \n",
       "1163  3-oxoacyl-[acyl-carrier-protein] synthase 3    0.0               False   \n",
       "1164                       Bacterial 70S ribosome    0.0                True   \n",
       "\n",
       "     target_chembl_id                                  target_components  \\\n",
       "0       CHEMBL2111390  [{'accession': 'Q9Y5Z0', 'component_descriptio...   \n",
       "1       CHEMBL2331066  [{'accession': 'Q6IE75', 'component_descriptio...   \n",
       "2          CHEMBL2525  [{'accession': 'Q9Y5Z0', 'component_descriptio...   \n",
       "3          CHEMBL4822  [{'accession': 'P56817', 'component_descriptio...   \n",
       "4          CHEMBL4593  [{'accession': 'P56818', 'component_descriptio...   \n",
       "...               ...                                                ...   \n",
       "1160    CHEMBL3831289  [{'accession': 'P49662', 'component_descriptio...   \n",
       "1161    CHEMBL4296661  [{'accession': 'P42345', 'component_descriptio...   \n",
       "1162    CHEMBL4523999  [{'accession': 'P42345', 'component_descriptio...   \n",
       "1163    CHEMBL1795135  [{'accession': 'C3TDZ2', 'component_descriptio...   \n",
       "1164    CHEMBL2363135  [{'accession': 'P0A7V0', 'component_descriptio...   \n",
       "\n",
       "                       target_type   tax_id  \n",
       "0                   PROTEIN FAMILY   9606.0  \n",
       "1                   SINGLE PROTEIN  10116.0  \n",
       "2                   SINGLE PROTEIN   9606.0  \n",
       "3                   SINGLE PROTEIN   9606.0  \n",
       "4                   SINGLE PROTEIN  10090.0  \n",
       "...                            ...      ...  \n",
       "1160                PROTEIN FAMILY   9606.0  \n",
       "1161               PROTEIN COMPLEX   9606.0  \n",
       "1162               PROTEIN COMPLEX   9606.0  \n",
       "1163                SINGLE PROTEIN    562.0  \n",
       "1164  PROTEIN NUCLEIC-ACID COMPLEX      2.0  \n",
       "\n",
       "[1165 rows x 9 columns]"
      ]
     },
     "execution_count": 273,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# target search for beta secretase\n",
    "target = new_client.target\n",
    "target_query = target.search('Beta-secretase')\n",
    "targets = pd.DataFrame.from_dict(target_query)\n",
    "targets"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e98becb5",
   "metadata": {},
   "source": [
    "## Select and retrieve bioactivity data for Beta-secretase 1 (fourth entry)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52e05d56",
   "metadata": {},
   "source": [
    "I will assign the fourth entry (which corresponds to the target protein, Beta-secretase 1) to the **selected_target**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 274,
   "id": "42109da1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'CHEMBL4822'"
      ]
     },
     "execution_count": 274,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "selected_target = targets.target_chembl_id[3]\n",
    "selected_target"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d0adf106",
   "metadata": {},
   "source": [
    "Here, I will retrieve only bioactivity data for Beta-secretase 1 (CHEMBL4822) that are reported as values in nM (nanomolar) unit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 269,
   "id": "7017f510",
   "metadata": {},
   "outputs": [],
   "source": [
    "activity = new_client.activity\n",
    "# define a specific standard type so the dataset is more uniform and\n",
    "# there will not be a mixture of different bioactivity units\n",
    "res = activity.filter(target_chembl_id=selected_target).filter(standard_type=\"IC50\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 271,
   "id": "4dd76186",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.DataFrame.from_dict(res)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 272,
   "id": "fd7cc236",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>activity_comment</th>\n",
       "      <th>activity_id</th>\n",
       "      <th>activity_properties</th>\n",
       "      <th>assay_chembl_id</th>\n",
       "      <th>assay_description</th>\n",
       "      <th>assay_type</th>\n",
       "      <th>assay_variant_accession</th>\n",
       "      <th>assay_variant_mutation</th>\n",
       "      <th>bao_endpoint</th>\n",
       "      <th>bao_format</th>\n",
       "      <th>...</th>\n",
       "      <th>target_organism</th>\n",
       "      <th>target_pref_name</th>\n",
       "      <th>target_tax_id</th>\n",
       "      <th>text_value</th>\n",
       "      <th>toid</th>\n",
       "      <th>type</th>\n",
       "      <th>units</th>\n",
       "      <th>uo_units</th>\n",
       "      <th>upper_value</th>\n",
       "      <th>value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>None</td>\n",
       "      <td>78857</td>\n",
       "      <td>[]</td>\n",
       "      <td>CHEMBL653511</td>\n",
       "      <td>Inhibitory activity against Beta-secretase 1 w...</td>\n",
       "      <td>B</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>BAO_0000190</td>\n",
       "      <td>BAO_0000357</td>\n",
       "      <td>...</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta-secretase 1</td>\n",
       "      <td>9606</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>IC50</td>\n",
       "      <td>nM</td>\n",
       "      <td>UO_0000065</td>\n",
       "      <td>None</td>\n",
       "      <td>413.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>None</td>\n",
       "      <td>391560</td>\n",
       "      <td>[]</td>\n",
       "      <td>CHEMBL653332</td>\n",
       "      <td>Compound was tested for its inhibitory activit...</td>\n",
       "      <td>B</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>BAO_0000190</td>\n",
       "      <td>BAO_0000357</td>\n",
       "      <td>...</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta-secretase 1</td>\n",
       "      <td>9606</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>IC50</td>\n",
       "      <td>uM</td>\n",
       "      <td>UO_0000065</td>\n",
       "      <td>None</td>\n",
       "      <td>0.002</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>None</td>\n",
       "      <td>391983</td>\n",
       "      <td>[]</td>\n",
       "      <td>CHEMBL653512</td>\n",
       "      <td>Inhibition of human Beta-secretase 1</td>\n",
       "      <td>B</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>BAO_0000190</td>\n",
       "      <td>BAO_0000357</td>\n",
       "      <td>...</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta-secretase 1</td>\n",
       "      <td>9606</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>IC50</td>\n",
       "      <td>uM</td>\n",
       "      <td>UO_0000065</td>\n",
       "      <td>None</td>\n",
       "      <td>0.46</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>None</td>\n",
       "      <td>395858</td>\n",
       "      <td>[]</td>\n",
       "      <td>CHEMBL653512</td>\n",
       "      <td>Inhibition of human Beta-secretase 1</td>\n",
       "      <td>B</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>BAO_0000190</td>\n",
       "      <td>BAO_0000357</td>\n",
       "      <td>...</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta-secretase 1</td>\n",
       "      <td>9606</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>IC50</td>\n",
       "      <td>uM</td>\n",
       "      <td>UO_0000065</td>\n",
       "      <td>None</td>\n",
       "      <td>9.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>None</td>\n",
       "      <td>395859</td>\n",
       "      <td>[]</td>\n",
       "      <td>CHEMBL653512</td>\n",
       "      <td>Inhibition of human Beta-secretase 1</td>\n",
       "      <td>B</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>BAO_0000190</td>\n",
       "      <td>BAO_0000357</td>\n",
       "      <td>...</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta-secretase 1</td>\n",
       "      <td>9606</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>IC50</td>\n",
       "      <td>uM</td>\n",
       "      <td>UO_0000065</td>\n",
       "      <td>None</td>\n",
       "      <td>5.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10426</th>\n",
       "      <td>None</td>\n",
       "      <td>23311823</td>\n",
       "      <td>[{'comments': None, 'relation': '=', 'result_f...</td>\n",
       "      <td>CHEMBL4844257</td>\n",
       "      <td>Inhibition of human recombinant BACE-1 express...</td>\n",
       "      <td>B</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>BAO_0000190</td>\n",
       "      <td>BAO_0000019</td>\n",
       "      <td>...</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta-secretase 1</td>\n",
       "      <td>9606</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>IC50</td>\n",
       "      <td>uM</td>\n",
       "      <td>UO_0000065</td>\n",
       "      <td>None</td>\n",
       "      <td>1.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10427</th>\n",
       "      <td>None</td>\n",
       "      <td>23311824</td>\n",
       "      <td>[{'comments': None, 'relation': '=', 'result_f...</td>\n",
       "      <td>CHEMBL4844257</td>\n",
       "      <td>Inhibition of human recombinant BACE-1 express...</td>\n",
       "      <td>B</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>BAO_0000190</td>\n",
       "      <td>BAO_0000019</td>\n",
       "      <td>...</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta-secretase 1</td>\n",
       "      <td>9606</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>IC50</td>\n",
       "      <td>uM</td>\n",
       "      <td>UO_0000065</td>\n",
       "      <td>None</td>\n",
       "      <td>4.18</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10428</th>\n",
       "      <td>Not Determined</td>\n",
       "      <td>23311825</td>\n",
       "      <td>[{'comments': None, 'relation': '=', 'result_f...</td>\n",
       "      <td>CHEMBL4844257</td>\n",
       "      <td>Inhibition of human recombinant BACE-1 express...</td>\n",
       "      <td>B</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>BAO_0000190</td>\n",
       "      <td>BAO_0000019</td>\n",
       "      <td>...</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta-secretase 1</td>\n",
       "      <td>9606</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>IC50</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10429</th>\n",
       "      <td>Not Determined</td>\n",
       "      <td>23311826</td>\n",
       "      <td>[{'comments': None, 'relation': '=', 'result_f...</td>\n",
       "      <td>CHEMBL4844257</td>\n",
       "      <td>Inhibition of human recombinant BACE-1 express...</td>\n",
       "      <td>B</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>BAO_0000190</td>\n",
       "      <td>BAO_0000019</td>\n",
       "      <td>...</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta-secretase 1</td>\n",
       "      <td>9606</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>IC50</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10430</th>\n",
       "      <td>Not Determined</td>\n",
       "      <td>23311827</td>\n",
       "      <td>[{'comments': None, 'relation': '=', 'result_f...</td>\n",
       "      <td>CHEMBL4844257</td>\n",
       "      <td>Inhibition of human recombinant BACE-1 express...</td>\n",
       "      <td>B</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>BAO_0000190</td>\n",
       "      <td>BAO_0000019</td>\n",
       "      <td>...</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta-secretase 1</td>\n",
       "      <td>9606</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>IC50</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10431 rows × 45 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      activity_comment  activity_id  \\\n",
       "0                 None        78857   \n",
       "1                 None       391560   \n",
       "2                 None       391983   \n",
       "3                 None       395858   \n",
       "4                 None       395859   \n",
       "...                ...          ...   \n",
       "10426             None     23311823   \n",
       "10427             None     23311824   \n",
       "10428   Not Determined     23311825   \n",
       "10429   Not Determined     23311826   \n",
       "10430   Not Determined     23311827   \n",
       "\n",
       "                                     activity_properties assay_chembl_id  \\\n",
       "0                                                     []    CHEMBL653511   \n",
       "1                                                     []    CHEMBL653332   \n",
       "2                                                     []    CHEMBL653512   \n",
       "3                                                     []    CHEMBL653512   \n",
       "4                                                     []    CHEMBL653512   \n",
       "...                                                  ...             ...   \n",
       "10426  [{'comments': None, 'relation': '=', 'result_f...   CHEMBL4844257   \n",
       "10427  [{'comments': None, 'relation': '=', 'result_f...   CHEMBL4844257   \n",
       "10428  [{'comments': None, 'relation': '=', 'result_f...   CHEMBL4844257   \n",
       "10429  [{'comments': None, 'relation': '=', 'result_f...   CHEMBL4844257   \n",
       "10430  [{'comments': None, 'relation': '=', 'result_f...   CHEMBL4844257   \n",
       "\n",
       "                                       assay_description assay_type  \\\n",
       "0      Inhibitory activity against Beta-secretase 1 w...          B   \n",
       "1      Compound was tested for its inhibitory activit...          B   \n",
       "2                   Inhibition of human Beta-secretase 1          B   \n",
       "3                   Inhibition of human Beta-secretase 1          B   \n",
       "4                   Inhibition of human Beta-secretase 1          B   \n",
       "...                                                  ...        ...   \n",
       "10426  Inhibition of human recombinant BACE-1 express...          B   \n",
       "10427  Inhibition of human recombinant BACE-1 express...          B   \n",
       "10428  Inhibition of human recombinant BACE-1 express...          B   \n",
       "10429  Inhibition of human recombinant BACE-1 express...          B   \n",
       "10430  Inhibition of human recombinant BACE-1 express...          B   \n",
       "\n",
       "      assay_variant_accession assay_variant_mutation bao_endpoint  \\\n",
       "0                        None                   None  BAO_0000190   \n",
       "1                        None                   None  BAO_0000190   \n",
       "2                        None                   None  BAO_0000190   \n",
       "3                        None                   None  BAO_0000190   \n",
       "4                        None                   None  BAO_0000190   \n",
       "...                       ...                    ...          ...   \n",
       "10426                    None                   None  BAO_0000190   \n",
       "10427                    None                   None  BAO_0000190   \n",
       "10428                    None                   None  BAO_0000190   \n",
       "10429                    None                   None  BAO_0000190   \n",
       "10430                    None                   None  BAO_0000190   \n",
       "\n",
       "        bao_format  ... target_organism  target_pref_name target_tax_id  \\\n",
       "0      BAO_0000357  ...    Homo sapiens  Beta-secretase 1          9606   \n",
       "1      BAO_0000357  ...    Homo sapiens  Beta-secretase 1          9606   \n",
       "2      BAO_0000357  ...    Homo sapiens  Beta-secretase 1          9606   \n",
       "3      BAO_0000357  ...    Homo sapiens  Beta-secretase 1          9606   \n",
       "4      BAO_0000357  ...    Homo sapiens  Beta-secretase 1          9606   \n",
       "...            ...  ...             ...               ...           ...   \n",
       "10426  BAO_0000019  ...    Homo sapiens  Beta-secretase 1          9606   \n",
       "10427  BAO_0000019  ...    Homo sapiens  Beta-secretase 1          9606   \n",
       "10428  BAO_0000019  ...    Homo sapiens  Beta-secretase 1          9606   \n",
       "10429  BAO_0000019  ...    Homo sapiens  Beta-secretase 1          9606   \n",
       "10430  BAO_0000019  ...    Homo sapiens  Beta-secretase 1          9606   \n",
       "\n",
       "      text_value  toid  type  units    uo_units upper_value  value  \n",
       "0           None  None  IC50     nM  UO_0000065        None  413.0  \n",
       "1           None  None  IC50     uM  UO_0000065        None  0.002  \n",
       "2           None  None  IC50     uM  UO_0000065        None   0.46  \n",
       "3           None  None  IC50     uM  UO_0000065        None    9.0  \n",
       "4           None  None  IC50     uM  UO_0000065        None    5.6  \n",
       "...          ...   ...   ...    ...         ...         ...    ...  \n",
       "10426       None  None  IC50     uM  UO_0000065        None    1.5  \n",
       "10427       None  None  IC50     uM  UO_0000065        None   4.18  \n",
       "10428       None  None  IC50   None        None        None   None  \n",
       "10429       None  None  IC50   None        None        None   None  \n",
       "10430       None  None  IC50   None        None        None   None  \n",
       "\n",
       "[10431 rows x 45 columns]"
      ]
     },
     "execution_count": 272,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 275,
   "id": "0f8ecd44",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['IC50'], dtype=object)"
      ]
     },
     "execution_count": 275,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.standard_type.unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 276,
   "id": "147a6bdb",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.to_csv('bioactivity_data.csv', index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7a6f0f5e",
   "metadata": {},
   "source": [
    "## Handling missing data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 277,
   "id": "846ee254",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/var/folders/c1/_3m7mbjn7pzcqcl5463xw4mr0000gn/T/ipykernel_27986/818330610.py:3: UserWarning: Boolean Series key will be reindexed to match DataFrame index.\n",
      "  df2 = df2[df.canonical_smiles.notna()]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>activity_comment</th>\n",
       "      <th>activity_id</th>\n",
       "      <th>activity_properties</th>\n",
       "      <th>assay_chembl_id</th>\n",
       "      <th>assay_description</th>\n",
       "      <th>assay_type</th>\n",
       "      <th>assay_variant_accession</th>\n",
       "      <th>assay_variant_mutation</th>\n",
       "      <th>bao_endpoint</th>\n",
       "      <th>bao_format</th>\n",
       "      <th>...</th>\n",
       "      <th>target_organism</th>\n",
       "      <th>target_pref_name</th>\n",
       "      <th>target_tax_id</th>\n",
       "      <th>text_value</th>\n",
       "      <th>toid</th>\n",
       "      <th>type</th>\n",
       "      <th>units</th>\n",
       "      <th>uo_units</th>\n",
       "      <th>upper_value</th>\n",
       "      <th>value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>None</td>\n",
       "      <td>78857</td>\n",
       "      <td>[]</td>\n",
       "      <td>CHEMBL653511</td>\n",
       "      <td>Inhibitory activity against Beta-secretase 1 w...</td>\n",
       "      <td>B</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>BAO_0000190</td>\n",
       "      <td>BAO_0000357</td>\n",
       "      <td>...</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta-secretase 1</td>\n",
       "      <td>9606</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>IC50</td>\n",
       "      <td>nM</td>\n",
       "      <td>UO_0000065</td>\n",
       "      <td>None</td>\n",
       "      <td>413.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>None</td>\n",
       "      <td>391560</td>\n",
       "      <td>[]</td>\n",
       "      <td>CHEMBL653332</td>\n",
       "      <td>Compound was tested for its inhibitory activit...</td>\n",
       "      <td>B</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>BAO_0000190</td>\n",
       "      <td>BAO_0000357</td>\n",
       "      <td>...</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta-secretase 1</td>\n",
       "      <td>9606</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>IC50</td>\n",
       "      <td>uM</td>\n",
       "      <td>UO_0000065</td>\n",
       "      <td>None</td>\n",
       "      <td>0.002</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>None</td>\n",
       "      <td>391983</td>\n",
       "      <td>[]</td>\n",
       "      <td>CHEMBL653512</td>\n",
       "      <td>Inhibition of human Beta-secretase 1</td>\n",
       "      <td>B</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>BAO_0000190</td>\n",
       "      <td>BAO_0000357</td>\n",
       "      <td>...</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta-secretase 1</td>\n",
       "      <td>9606</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>IC50</td>\n",
       "      <td>uM</td>\n",
       "      <td>UO_0000065</td>\n",
       "      <td>None</td>\n",
       "      <td>0.46</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>None</td>\n",
       "      <td>395858</td>\n",
       "      <td>[]</td>\n",
       "      <td>CHEMBL653512</td>\n",
       "      <td>Inhibition of human Beta-secretase 1</td>\n",
       "      <td>B</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>BAO_0000190</td>\n",
       "      <td>BAO_0000357</td>\n",
       "      <td>...</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta-secretase 1</td>\n",
       "      <td>9606</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>IC50</td>\n",
       "      <td>uM</td>\n",
       "      <td>UO_0000065</td>\n",
       "      <td>None</td>\n",
       "      <td>9.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>None</td>\n",
       "      <td>395859</td>\n",
       "      <td>[]</td>\n",
       "      <td>CHEMBL653512</td>\n",
       "      <td>Inhibition of human Beta-secretase 1</td>\n",
       "      <td>B</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>BAO_0000190</td>\n",
       "      <td>BAO_0000357</td>\n",
       "      <td>...</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta-secretase 1</td>\n",
       "      <td>9606</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>IC50</td>\n",
       "      <td>uM</td>\n",
       "      <td>UO_0000065</td>\n",
       "      <td>None</td>\n",
       "      <td>5.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10423</th>\n",
       "      <td>None</td>\n",
       "      <td>23300103</td>\n",
       "      <td>[{'comments': None, 'relation': '=', 'result_f...</td>\n",
       "      <td>CHEMBL4841886</td>\n",
       "      <td>Binding affinity to human recombinant BACE-1 (...</td>\n",
       "      <td>B</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>BAO_0000190</td>\n",
       "      <td>BAO_0000357</td>\n",
       "      <td>...</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta-secretase 1</td>\n",
       "      <td>9606</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>IC50</td>\n",
       "      <td>nM</td>\n",
       "      <td>UO_0000065</td>\n",
       "      <td>None</td>\n",
       "      <td>0.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10424</th>\n",
       "      <td>None</td>\n",
       "      <td>23311821</td>\n",
       "      <td>[{'comments': None, 'relation': '=', 'result_f...</td>\n",
       "      <td>CHEMBL4844257</td>\n",
       "      <td>Inhibition of human recombinant BACE-1 express...</td>\n",
       "      <td>B</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>BAO_0000190</td>\n",
       "      <td>BAO_0000019</td>\n",
       "      <td>...</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta-secretase 1</td>\n",
       "      <td>9606</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>IC50</td>\n",
       "      <td>uM</td>\n",
       "      <td>UO_0000065</td>\n",
       "      <td>None</td>\n",
       "      <td>10.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10425</th>\n",
       "      <td>None</td>\n",
       "      <td>23311822</td>\n",
       "      <td>[{'comments': None, 'relation': '=', 'result_f...</td>\n",
       "      <td>CHEMBL4844257</td>\n",
       "      <td>Inhibition of human recombinant BACE-1 express...</td>\n",
       "      <td>B</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>BAO_0000190</td>\n",
       "      <td>BAO_0000019</td>\n",
       "      <td>...</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta-secretase 1</td>\n",
       "      <td>9606</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>IC50</td>\n",
       "      <td>uM</td>\n",
       "      <td>UO_0000065</td>\n",
       "      <td>None</td>\n",
       "      <td>2.71</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10426</th>\n",
       "      <td>None</td>\n",
       "      <td>23311823</td>\n",
       "      <td>[{'comments': None, 'relation': '=', 'result_f...</td>\n",
       "      <td>CHEMBL4844257</td>\n",
       "      <td>Inhibition of human recombinant BACE-1 express...</td>\n",
       "      <td>B</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>BAO_0000190</td>\n",
       "      <td>BAO_0000019</td>\n",
       "      <td>...</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta-secretase 1</td>\n",
       "      <td>9606</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>IC50</td>\n",
       "      <td>uM</td>\n",
       "      <td>UO_0000065</td>\n",
       "      <td>None</td>\n",
       "      <td>1.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10427</th>\n",
       "      <td>None</td>\n",
       "      <td>23311824</td>\n",
       "      <td>[{'comments': None, 'relation': '=', 'result_f...</td>\n",
       "      <td>CHEMBL4844257</td>\n",
       "      <td>Inhibition of human recombinant BACE-1 express...</td>\n",
       "      <td>B</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>BAO_0000190</td>\n",
       "      <td>BAO_0000019</td>\n",
       "      <td>...</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta-secretase 1</td>\n",
       "      <td>9606</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>IC50</td>\n",
       "      <td>uM</td>\n",
       "      <td>UO_0000065</td>\n",
       "      <td>None</td>\n",
       "      <td>4.18</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10287 rows × 45 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      activity_comment  activity_id  \\\n",
       "0                 None        78857   \n",
       "1                 None       391560   \n",
       "2                 None       391983   \n",
       "3                 None       395858   \n",
       "4                 None       395859   \n",
       "...                ...          ...   \n",
       "10423             None     23300103   \n",
       "10424             None     23311821   \n",
       "10425             None     23311822   \n",
       "10426             None     23311823   \n",
       "10427             None     23311824   \n",
       "\n",
       "                                     activity_properties assay_chembl_id  \\\n",
       "0                                                     []    CHEMBL653511   \n",
       "1                                                     []    CHEMBL653332   \n",
       "2                                                     []    CHEMBL653512   \n",
       "3                                                     []    CHEMBL653512   \n",
       "4                                                     []    CHEMBL653512   \n",
       "...                                                  ...             ...   \n",
       "10423  [{'comments': None, 'relation': '=', 'result_f...   CHEMBL4841886   \n",
       "10424  [{'comments': None, 'relation': '=', 'result_f...   CHEMBL4844257   \n",
       "10425  [{'comments': None, 'relation': '=', 'result_f...   CHEMBL4844257   \n",
       "10426  [{'comments': None, 'relation': '=', 'result_f...   CHEMBL4844257   \n",
       "10427  [{'comments': None, 'relation': '=', 'result_f...   CHEMBL4844257   \n",
       "\n",
       "                                       assay_description assay_type  \\\n",
       "0      Inhibitory activity against Beta-secretase 1 w...          B   \n",
       "1      Compound was tested for its inhibitory activit...          B   \n",
       "2                   Inhibition of human Beta-secretase 1          B   \n",
       "3                   Inhibition of human Beta-secretase 1          B   \n",
       "4                   Inhibition of human Beta-secretase 1          B   \n",
       "...                                                  ...        ...   \n",
       "10423  Binding affinity to human recombinant BACE-1 (...          B   \n",
       "10424  Inhibition of human recombinant BACE-1 express...          B   \n",
       "10425  Inhibition of human recombinant BACE-1 express...          B   \n",
       "10426  Inhibition of human recombinant BACE-1 express...          B   \n",
       "10427  Inhibition of human recombinant BACE-1 express...          B   \n",
       "\n",
       "      assay_variant_accession assay_variant_mutation bao_endpoint  \\\n",
       "0                        None                   None  BAO_0000190   \n",
       "1                        None                   None  BAO_0000190   \n",
       "2                        None                   None  BAO_0000190   \n",
       "3                        None                   None  BAO_0000190   \n",
       "4                        None                   None  BAO_0000190   \n",
       "...                       ...                    ...          ...   \n",
       "10423                    None                   None  BAO_0000190   \n",
       "10424                    None                   None  BAO_0000190   \n",
       "10425                    None                   None  BAO_0000190   \n",
       "10426                    None                   None  BAO_0000190   \n",
       "10427                    None                   None  BAO_0000190   \n",
       "\n",
       "        bao_format  ... target_organism  target_pref_name target_tax_id  \\\n",
       "0      BAO_0000357  ...    Homo sapiens  Beta-secretase 1          9606   \n",
       "1      BAO_0000357  ...    Homo sapiens  Beta-secretase 1          9606   \n",
       "2      BAO_0000357  ...    Homo sapiens  Beta-secretase 1          9606   \n",
       "3      BAO_0000357  ...    Homo sapiens  Beta-secretase 1          9606   \n",
       "4      BAO_0000357  ...    Homo sapiens  Beta-secretase 1          9606   \n",
       "...            ...  ...             ...               ...           ...   \n",
       "10423  BAO_0000357  ...    Homo sapiens  Beta-secretase 1          9606   \n",
       "10424  BAO_0000019  ...    Homo sapiens  Beta-secretase 1          9606   \n",
       "10425  BAO_0000019  ...    Homo sapiens  Beta-secretase 1          9606   \n",
       "10426  BAO_0000019  ...    Homo sapiens  Beta-secretase 1          9606   \n",
       "10427  BAO_0000019  ...    Homo sapiens  Beta-secretase 1          9606   \n",
       "\n",
       "      text_value  toid  type  units    uo_units upper_value  value  \n",
       "0           None  None  IC50     nM  UO_0000065        None  413.0  \n",
       "1           None  None  IC50     uM  UO_0000065        None  0.002  \n",
       "2           None  None  IC50     uM  UO_0000065        None   0.46  \n",
       "3           None  None  IC50     uM  UO_0000065        None    9.0  \n",
       "4           None  None  IC50     uM  UO_0000065        None    5.6  \n",
       "...          ...   ...   ...    ...         ...         ...    ...  \n",
       "10423       None  None  IC50     nM  UO_0000065        None    0.6  \n",
       "10424       None  None  IC50     uM  UO_0000065        None   10.8  \n",
       "10425       None  None  IC50     uM  UO_0000065        None   2.71  \n",
       "10426       None  None  IC50     uM  UO_0000065        None    1.5  \n",
       "10427       None  None  IC50     uM  UO_0000065        None   4.18  \n",
       "\n",
       "[10287 rows x 45 columns]"
      ]
     },
     "execution_count": 277,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# dropping any missing values for the standard_value column\n",
    "df2 = df[df.standard_value.notna()]\n",
    "df2 = df2[df.canonical_smiles.notna()]\n",
    "df2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 278,
   "id": "9c2e3d7e",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "7234"
      ]
     },
     "execution_count": 278,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(df2.canonical_smiles.unique())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 279,
   "id": "62b63bd0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>activity_comment</th>\n",
       "      <th>activity_id</th>\n",
       "      <th>activity_properties</th>\n",
       "      <th>assay_chembl_id</th>\n",
       "      <th>assay_description</th>\n",
       "      <th>assay_type</th>\n",
       "      <th>assay_variant_accession</th>\n",
       "      <th>assay_variant_mutation</th>\n",
       "      <th>bao_endpoint</th>\n",
       "      <th>bao_format</th>\n",
       "      <th>...</th>\n",
       "      <th>target_organism</th>\n",
       "      <th>target_pref_name</th>\n",
       "      <th>target_tax_id</th>\n",
       "      <th>text_value</th>\n",
       "      <th>toid</th>\n",
       "      <th>type</th>\n",
       "      <th>units</th>\n",
       "      <th>uo_units</th>\n",
       "      <th>upper_value</th>\n",
       "      <th>value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>None</td>\n",
       "      <td>78857</td>\n",
       "      <td>[]</td>\n",
       "      <td>CHEMBL653511</td>\n",
       "      <td>Inhibitory activity against Beta-secretase 1 w...</td>\n",
       "      <td>B</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>BAO_0000190</td>\n",
       "      <td>BAO_0000357</td>\n",
       "      <td>...</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta-secretase 1</td>\n",
       "      <td>9606</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>IC50</td>\n",
       "      <td>nM</td>\n",
       "      <td>UO_0000065</td>\n",
       "      <td>None</td>\n",
       "      <td>413.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>None</td>\n",
       "      <td>391560</td>\n",
       "      <td>[]</td>\n",
       "      <td>CHEMBL653332</td>\n",
       "      <td>Compound was tested for its inhibitory activit...</td>\n",
       "      <td>B</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>BAO_0000190</td>\n",
       "      <td>BAO_0000357</td>\n",
       "      <td>...</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta-secretase 1</td>\n",
       "      <td>9606</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>IC50</td>\n",
       "      <td>uM</td>\n",
       "      <td>UO_0000065</td>\n",
       "      <td>None</td>\n",
       "      <td>0.002</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>None</td>\n",
       "      <td>391983</td>\n",
       "      <td>[]</td>\n",
       "      <td>CHEMBL653512</td>\n",
       "      <td>Inhibition of human Beta-secretase 1</td>\n",
       "      <td>B</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>BAO_0000190</td>\n",
       "      <td>BAO_0000357</td>\n",
       "      <td>...</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta-secretase 1</td>\n",
       "      <td>9606</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>IC50</td>\n",
       "      <td>uM</td>\n",
       "      <td>UO_0000065</td>\n",
       "      <td>None</td>\n",
       "      <td>0.46</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>None</td>\n",
       "      <td>395858</td>\n",
       "      <td>[]</td>\n",
       "      <td>CHEMBL653512</td>\n",
       "      <td>Inhibition of human Beta-secretase 1</td>\n",
       "      <td>B</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>BAO_0000190</td>\n",
       "      <td>BAO_0000357</td>\n",
       "      <td>...</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta-secretase 1</td>\n",
       "      <td>9606</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>IC50</td>\n",
       "      <td>uM</td>\n",
       "      <td>UO_0000065</td>\n",
       "      <td>None</td>\n",
       "      <td>9.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>None</td>\n",
       "      <td>395859</td>\n",
       "      <td>[]</td>\n",
       "      <td>CHEMBL653512</td>\n",
       "      <td>Inhibition of human Beta-secretase 1</td>\n",
       "      <td>B</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>BAO_0000190</td>\n",
       "      <td>BAO_0000357</td>\n",
       "      <td>...</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta-secretase 1</td>\n",
       "      <td>9606</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>IC50</td>\n",
       "      <td>uM</td>\n",
       "      <td>UO_0000065</td>\n",
       "      <td>None</td>\n",
       "      <td>5.6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10419</th>\n",
       "      <td>None</td>\n",
       "      <td>23300099</td>\n",
       "      <td>[{'comments': None, 'relation': '=', 'result_f...</td>\n",
       "      <td>CHEMBL4841885</td>\n",
       "      <td>Inhibition of recombinant BACE-1 (unknown orig...</td>\n",
       "      <td>B</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>BAO_0000190</td>\n",
       "      <td>BAO_0000357</td>\n",
       "      <td>...</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta-secretase 1</td>\n",
       "      <td>9606</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>IC50</td>\n",
       "      <td>uM</td>\n",
       "      <td>UO_0000065</td>\n",
       "      <td>None</td>\n",
       "      <td>4.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10420</th>\n",
       "      <td>None</td>\n",
       "      <td>23300100</td>\n",
       "      <td>[{'comments': None, 'relation': '=', 'result_f...</td>\n",
       "      <td>CHEMBL4841885</td>\n",
       "      <td>Inhibition of recombinant BACE-1 (unknown orig...</td>\n",
       "      <td>B</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>BAO_0000190</td>\n",
       "      <td>BAO_0000357</td>\n",
       "      <td>...</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta-secretase 1</td>\n",
       "      <td>9606</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>IC50</td>\n",
       "      <td>uM</td>\n",
       "      <td>UO_0000065</td>\n",
       "      <td>None</td>\n",
       "      <td>9.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10421</th>\n",
       "      <td>None</td>\n",
       "      <td>23300101</td>\n",
       "      <td>[{'comments': None, 'relation': '=', 'result_f...</td>\n",
       "      <td>CHEMBL4841885</td>\n",
       "      <td>Inhibition of recombinant BACE-1 (unknown orig...</td>\n",
       "      <td>B</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>BAO_0000190</td>\n",
       "      <td>BAO_0000357</td>\n",
       "      <td>...</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta-secretase 1</td>\n",
       "      <td>9606</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>IC50</td>\n",
       "      <td>uM</td>\n",
       "      <td>UO_0000065</td>\n",
       "      <td>None</td>\n",
       "      <td>10.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10424</th>\n",
       "      <td>None</td>\n",
       "      <td>23311821</td>\n",
       "      <td>[{'comments': None, 'relation': '=', 'result_f...</td>\n",
       "      <td>CHEMBL4844257</td>\n",
       "      <td>Inhibition of human recombinant BACE-1 express...</td>\n",
       "      <td>B</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>BAO_0000190</td>\n",
       "      <td>BAO_0000019</td>\n",
       "      <td>...</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta-secretase 1</td>\n",
       "      <td>9606</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>IC50</td>\n",
       "      <td>uM</td>\n",
       "      <td>UO_0000065</td>\n",
       "      <td>None</td>\n",
       "      <td>10.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10425</th>\n",
       "      <td>None</td>\n",
       "      <td>23311822</td>\n",
       "      <td>[{'comments': None, 'relation': '=', 'result_f...</td>\n",
       "      <td>CHEMBL4844257</td>\n",
       "      <td>Inhibition of human recombinant BACE-1 express...</td>\n",
       "      <td>B</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>BAO_0000190</td>\n",
       "      <td>BAO_0000019</td>\n",
       "      <td>...</td>\n",
       "      <td>Homo sapiens</td>\n",
       "      <td>Beta-secretase 1</td>\n",
       "      <td>9606</td>\n",
       "      <td>None</td>\n",
       "      <td>None</td>\n",
       "      <td>IC50</td>\n",
       "      <td>uM</td>\n",
       "      <td>UO_0000065</td>\n",
       "      <td>None</td>\n",
       "      <td>2.71</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>7234 rows × 45 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      activity_comment  activity_id  \\\n",
       "0                 None        78857   \n",
       "1                 None       391560   \n",
       "2                 None       391983   \n",
       "3                 None       395858   \n",
       "4                 None       395859   \n",
       "...                ...          ...   \n",
       "10419             None     23300099   \n",
       "10420             None     23300100   \n",
       "10421             None     23300101   \n",
       "10424             None     23311821   \n",
       "10425             None     23311822   \n",
       "\n",
       "                                     activity_properties assay_chembl_id  \\\n",
       "0                                                     []    CHEMBL653511   \n",
       "1                                                     []    CHEMBL653332   \n",
       "2                                                     []    CHEMBL653512   \n",
       "3                                                     []    CHEMBL653512   \n",
       "4                                                     []    CHEMBL653512   \n",
       "...                                                  ...             ...   \n",
       "10419  [{'comments': None, 'relation': '=', 'result_f...   CHEMBL4841885   \n",
       "10420  [{'comments': None, 'relation': '=', 'result_f...   CHEMBL4841885   \n",
       "10421  [{'comments': None, 'relation': '=', 'result_f...   CHEMBL4841885   \n",
       "10424  [{'comments': None, 'relation': '=', 'result_f...   CHEMBL4844257   \n",
       "10425  [{'comments': None, 'relation': '=', 'result_f...   CHEMBL4844257   \n",
       "\n",
       "                                       assay_description assay_type  \\\n",
       "0      Inhibitory activity against Beta-secretase 1 w...          B   \n",
       "1      Compound was tested for its inhibitory activit...          B   \n",
       "2                   Inhibition of human Beta-secretase 1          B   \n",
       "3                   Inhibition of human Beta-secretase 1          B   \n",
       "4                   Inhibition of human Beta-secretase 1          B   \n",
       "...                                                  ...        ...   \n",
       "10419  Inhibition of recombinant BACE-1 (unknown orig...          B   \n",
       "10420  Inhibition of recombinant BACE-1 (unknown orig...          B   \n",
       "10421  Inhibition of recombinant BACE-1 (unknown orig...          B   \n",
       "10424  Inhibition of human recombinant BACE-1 express...          B   \n",
       "10425  Inhibition of human recombinant BACE-1 express...          B   \n",
       "\n",
       "      assay_variant_accession assay_variant_mutation bao_endpoint  \\\n",
       "0                        None                   None  BAO_0000190   \n",
       "1                        None                   None  BAO_0000190   \n",
       "2                        None                   None  BAO_0000190   \n",
       "3                        None                   None  BAO_0000190   \n",
       "4                        None                   None  BAO_0000190   \n",
       "...                       ...                    ...          ...   \n",
       "10419                    None                   None  BAO_0000190   \n",
       "10420                    None                   None  BAO_0000190   \n",
       "10421                    None                   None  BAO_0000190   \n",
       "10424                    None                   None  BAO_0000190   \n",
       "10425                    None                   None  BAO_0000190   \n",
       "\n",
       "        bao_format  ... target_organism  target_pref_name target_tax_id  \\\n",
       "0      BAO_0000357  ...    Homo sapiens  Beta-secretase 1          9606   \n",
       "1      BAO_0000357  ...    Homo sapiens  Beta-secretase 1          9606   \n",
       "2      BAO_0000357  ...    Homo sapiens  Beta-secretase 1          9606   \n",
       "3      BAO_0000357  ...    Homo sapiens  Beta-secretase 1          9606   \n",
       "4      BAO_0000357  ...    Homo sapiens  Beta-secretase 1          9606   \n",
       "...            ...  ...             ...               ...           ...   \n",
       "10419  BAO_0000357  ...    Homo sapiens  Beta-secretase 1          9606   \n",
       "10420  BAO_0000357  ...    Homo sapiens  Beta-secretase 1          9606   \n",
       "10421  BAO_0000357  ...    Homo sapiens  Beta-secretase 1          9606   \n",
       "10424  BAO_0000019  ...    Homo sapiens  Beta-secretase 1          9606   \n",
       "10425  BAO_0000019  ...    Homo sapiens  Beta-secretase 1          9606   \n",
       "\n",
       "      text_value  toid  type  units    uo_units upper_value  value  \n",
       "0           None  None  IC50     nM  UO_0000065        None  413.0  \n",
       "1           None  None  IC50     uM  UO_0000065        None  0.002  \n",
       "2           None  None  IC50     uM  UO_0000065        None   0.46  \n",
       "3           None  None  IC50     uM  UO_0000065        None    9.0  \n",
       "4           None  None  IC50     uM  UO_0000065        None    5.6  \n",
       "...          ...   ...   ...    ...         ...         ...    ...  \n",
       "10419       None  None  IC50     uM  UO_0000065        None    4.4  \n",
       "10420       None  None  IC50     uM  UO_0000065        None    9.2  \n",
       "10421       None  None  IC50     uM  UO_0000065        None   10.0  \n",
       "10424       None  None  IC50     uM  UO_0000065        None   10.8  \n",
       "10425       None  None  IC50     uM  UO_0000065        None   2.71  \n",
       "\n",
       "[7234 rows x 45 columns]"
      ]
     },
     "execution_count": 279,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2_nr = df2.drop_duplicates(['canonical_smiles'])\n",
    "df2_nr"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "774c4faf",
   "metadata": {},
   "source": [
    "## Data pre-processing of the bioactivity data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa9b8ecd",
   "metadata": {},
   "source": [
    "### Combine molecule_chembl_id,canonical_smiles,standard_value and bioactivity_class into a DataFrame"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 280,
   "id": "8253d302",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>molecule_chembl_id</th>\n",
       "      <th>canonical_smiles</th>\n",
       "      <th>standard_value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>CHEMBL406146</td>\n",
       "      <td>CC(C)C[C@H](NC(=O)[C@@H](NC(=O)[C@@H](N)CCC(=O...</td>\n",
       "      <td>413.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>CHEMBL78946</td>\n",
       "      <td>CC(C)C[C@H](NC(=O)[C@H](CC(N)=O)NC(=O)[C@@H](N...</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>CHEMBL324109</td>\n",
       "      <td>CCC(C)C[C@H](NC(=O)[C@H](CC(C)C)NC(C)=O)[C@@H]...</td>\n",
       "      <td>460.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>CHEMBL114147</td>\n",
       "      <td>CC(=O)NCC(=O)N[C@@H](Cc1ccccc1)[C@@H](O)CC(=O)...</td>\n",
       "      <td>9000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>CHEMBL419949</td>\n",
       "      <td>CC(=O)N[C@@H](Cc1ccccc1)C(=O)N[C@@H](Cc1ccccc1...</td>\n",
       "      <td>5600.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10419</th>\n",
       "      <td>CHEMBL4849016</td>\n",
       "      <td>CC[C@H](C)[C@H](NC(=O)[C@@H](N)CCC(=O)O)C(=O)N...</td>\n",
       "      <td>4400.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10420</th>\n",
       "      <td>CHEMBL4872824</td>\n",
       "      <td>CC[C@H](C)[C@H](NC(=O)[C@@H](N)CCC(=O)O)C(=O)N...</td>\n",
       "      <td>9200.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10421</th>\n",
       "      <td>CHEMBL4853052</td>\n",
       "      <td>CC[C@H](C)[C@H](NC(=O)[C@@H](N)CCC(=O)O)C(=O)N...</td>\n",
       "      <td>10000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10424</th>\n",
       "      <td>CHEMBL4862716</td>\n",
       "      <td>CC1=CC2Cc3nc4cc(Cl)ccc4c(NCCCCCCCCCNS(=O)(=O)c...</td>\n",
       "      <td>10800.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10425</th>\n",
       "      <td>CHEMBL4859103</td>\n",
       "      <td>CC1=CC2Cc3nc4cc(Cl)ccc4c(NCCCCCCCCCNC(=O)c4cc(...</td>\n",
       "      <td>2710.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>7234 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      molecule_chembl_id                                   canonical_smiles  \\\n",
       "0           CHEMBL406146  CC(C)C[C@H](NC(=O)[C@@H](NC(=O)[C@@H](N)CCC(=O...   \n",
       "1            CHEMBL78946  CC(C)C[C@H](NC(=O)[C@H](CC(N)=O)NC(=O)[C@@H](N...   \n",
       "2           CHEMBL324109  CCC(C)C[C@H](NC(=O)[C@H](CC(C)C)NC(C)=O)[C@@H]...   \n",
       "3           CHEMBL114147  CC(=O)NCC(=O)N[C@@H](Cc1ccccc1)[C@@H](O)CC(=O)...   \n",
       "4           CHEMBL419949  CC(=O)N[C@@H](Cc1ccccc1)C(=O)N[C@@H](Cc1ccccc1...   \n",
       "...                  ...                                                ...   \n",
       "10419      CHEMBL4849016  CC[C@H](C)[C@H](NC(=O)[C@@H](N)CCC(=O)O)C(=O)N...   \n",
       "10420      CHEMBL4872824  CC[C@H](C)[C@H](NC(=O)[C@@H](N)CCC(=O)O)C(=O)N...   \n",
       "10421      CHEMBL4853052  CC[C@H](C)[C@H](NC(=O)[C@@H](N)CCC(=O)O)C(=O)N...   \n",
       "10424      CHEMBL4862716  CC1=CC2Cc3nc4cc(Cl)ccc4c(NCCCCCCCCCNS(=O)(=O)c...   \n",
       "10425      CHEMBL4859103  CC1=CC2Cc3nc4cc(Cl)ccc4c(NCCCCCCCCCNC(=O)c4cc(...   \n",
       "\n",
       "      standard_value  \n",
       "0              413.0  \n",
       "1                2.0  \n",
       "2              460.0  \n",
       "3             9000.0  \n",
       "4             5600.0  \n",
       "...              ...  \n",
       "10419         4400.0  \n",
       "10420         9200.0  \n",
       "10421        10000.0  \n",
       "10424        10800.0  \n",
       "10425         2710.0  \n",
       "\n",
       "[7234 rows x 3 columns]"
      ]
     },
     "execution_count": 280,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "selection = ['molecule_chembl_id','canonical_smiles','standard_value']\n",
    "df3 = df2_nr[selection]\n",
    "df3  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 281,
   "id": "69f6fb87",
   "metadata": {},
   "outputs": [],
   "source": [
    "df3.to_csv('beta_secretase_data_preprocessed.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2fa7fe7b",
   "metadata": {},
   "source": [
    "### Labeling compounds as either being active, inactive, or intermediate"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2323ee0",
   "metadata": {},
   "source": [
    "The bioactivity data is in the IC50 unit. Compounds having values of less than 1000 nM will be considered **active**. Values in between 1,000 and 10,000 nM will be refered to as **intermediate** and values greater than 10,000 nM will be considered **inactive**. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 282,
   "id": "9c7c8e4f",
   "metadata": {},
   "outputs": [],
   "source": [
    "df4 = pd.read_csv('data/beta_secretase_data_preprocessed.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 283,
   "id": "4628ac3a",
   "metadata": {},
   "outputs": [],
   "source": [
    "bioactivity_threshold = []\n",
    "for i in df4.standard_value:\n",
    "    if float(i) >= 10000:\n",
    "        bioactivity_threshold.append(\"inactive\")\n",
    "    elif float(i) <= 1000:\n",
    "        bioactivity_threshold.append(\"active\")\n",
    "    else:\n",
    "        bioactivity_threshold.append(\"intermediate\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 284,
   "id": "f1e03e31",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>molecule_chembl_id</th>\n",
       "      <th>canonical_smiles</th>\n",
       "      <th>standard_value</th>\n",
       "      <th>class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>CHEMBL406146</td>\n",
       "      <td>CC(C)C[C@H](NC(=O)[C@@H](NC(=O)[C@@H](N)CCC(=O...</td>\n",
       "      <td>413.0</td>\n",
       "      <td>active</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>CHEMBL78946</td>\n",
       "      <td>CC(C)C[C@H](NC(=O)[C@H](CC(N)=O)NC(=O)[C@@H](N...</td>\n",
       "      <td>2.0</td>\n",
       "      <td>active</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>CHEMBL324109</td>\n",
       "      <td>CCC(C)C[C@H](NC(=O)[C@H](CC(C)C)NC(C)=O)[C@@H]...</td>\n",
       "      <td>460.0</td>\n",
       "      <td>active</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>CHEMBL114147</td>\n",
       "      <td>CC(=O)NCC(=O)N[C@@H](Cc1ccccc1)[C@@H](O)CC(=O)...</td>\n",
       "      <td>9000.0</td>\n",
       "      <td>intermediate</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>CHEMBL419949</td>\n",
       "      <td>CC(=O)N[C@@H](Cc1ccccc1)C(=O)N[C@@H](Cc1ccccc1...</td>\n",
       "      <td>5600.0</td>\n",
       "      <td>intermediate</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7229</th>\n",
       "      <td>10419</td>\n",
       "      <td>CHEMBL4849016</td>\n",
       "      <td>CC[C@H](C)[C@H](NC(=O)[C@@H](N)CCC(=O)O)C(=O)N...</td>\n",
       "      <td>4400.0</td>\n",
       "      <td>intermediate</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7230</th>\n",
       "      <td>10420</td>\n",
       "      <td>CHEMBL4872824</td>\n",
       "      <td>CC[C@H](C)[C@H](NC(=O)[C@@H](N)CCC(=O)O)C(=O)N...</td>\n",
       "      <td>9200.0</td>\n",
       "      <td>intermediate</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7231</th>\n",
       "      <td>10421</td>\n",
       "      <td>CHEMBL4853052</td>\n",
       "      <td>CC[C@H](C)[C@H](NC(=O)[C@@H](N)CCC(=O)O)C(=O)N...</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>inactive</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7232</th>\n",
       "      <td>10424</td>\n",
       "      <td>CHEMBL4862716</td>\n",
       "      <td>CC1=CC2Cc3nc4cc(Cl)ccc4c(NCCCCCCCCCNS(=O)(=O)c...</td>\n",
       "      <td>10800.0</td>\n",
       "      <td>inactive</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7233</th>\n",
       "      <td>10425</td>\n",
       "      <td>CHEMBL4859103</td>\n",
       "      <td>CC1=CC2Cc3nc4cc(Cl)ccc4c(NCCCCCCCCCNC(=O)c4cc(...</td>\n",
       "      <td>2710.0</td>\n",
       "      <td>intermediate</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>7234 rows × 5 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      Unnamed: 0 molecule_chembl_id  \\\n",
       "0              0       CHEMBL406146   \n",
       "1              1        CHEMBL78946   \n",
       "2              2       CHEMBL324109   \n",
       "3              3       CHEMBL114147   \n",
       "4              4       CHEMBL419949   \n",
       "...          ...                ...   \n",
       "7229       10419      CHEMBL4849016   \n",
       "7230       10420      CHEMBL4872824   \n",
       "7231       10421      CHEMBL4853052   \n",
       "7232       10424      CHEMBL4862716   \n",
       "7233       10425      CHEMBL4859103   \n",
       "\n",
       "                                       canonical_smiles  standard_value  \\\n",
       "0     CC(C)C[C@H](NC(=O)[C@@H](NC(=O)[C@@H](N)CCC(=O...           413.0   \n",
       "1     CC(C)C[C@H](NC(=O)[C@H](CC(N)=O)NC(=O)[C@@H](N...             2.0   \n",
       "2     CCC(C)C[C@H](NC(=O)[C@H](CC(C)C)NC(C)=O)[C@@H]...           460.0   \n",
       "3     CC(=O)NCC(=O)N[C@@H](Cc1ccccc1)[C@@H](O)CC(=O)...          9000.0   \n",
       "4     CC(=O)N[C@@H](Cc1ccccc1)C(=O)N[C@@H](Cc1ccccc1...          5600.0   \n",
       "...                                                 ...             ...   \n",
       "7229  CC[C@H](C)[C@H](NC(=O)[C@@H](N)CCC(=O)O)C(=O)N...          4400.0   \n",
       "7230  CC[C@H](C)[C@H](NC(=O)[C@@H](N)CCC(=O)O)C(=O)N...          9200.0   \n",
       "7231  CC[C@H](C)[C@H](NC(=O)[C@@H](N)CCC(=O)O)C(=O)N...         10000.0   \n",
       "7232  CC1=CC2Cc3nc4cc(Cl)ccc4c(NCCCCCCCCCNS(=O)(=O)c...         10800.0   \n",
       "7233  CC1=CC2Cc3nc4cc(Cl)ccc4c(NCCCCCCCCCNC(=O)c4cc(...          2710.0   \n",
       "\n",
       "             class  \n",
       "0           active  \n",
       "1           active  \n",
       "2           active  \n",
       "3     intermediate  \n",
       "4     intermediate  \n",
       "...            ...  \n",
       "7229  intermediate  \n",
       "7230  intermediate  \n",
       "7231      inactive  \n",
       "7232      inactive  \n",
       "7233  intermediate  \n",
       "\n",
       "[7234 rows x 5 columns]"
      ]
     },
     "execution_count": 284,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bioactivity_class = pd.Series(bioactivity_threshold, name='class')\n",
    "df5 = pd.concat([df4, bioactivity_class], axis=1)\n",
    "df5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 285,
   "id": "c12d2d37",
   "metadata": {},
   "outputs": [],
   "source": [
    "df5.to_csv('beta_secretase_bioactivity_data_curated.csv', index=False)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}