LPA - Tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Latent Personal Analysis in Python\n",
    "by Uri Alon"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This short tutorial will demonstrate how to use the Python implementation of LPA, as described in the article - https://arxiv.org/pdf/2004.02346.pdf.\n",
    "The first implementation for LPA was written in SQL and can be found here - https://github.com/hagitbenshoshan/text_distance/\n",
    "For very large datasets, users of this package are encouraged to switch to the SQL implementation using cloud infrustructure, as the calculation of the results will be much faster."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Dependencies:\n",
    "- pandas\n",
    "- numpy\n",
    "- scipy "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Dependecies  for further analysis demonstrated in this notebook:\n",
    "- scikit-learn\n",
    "- plotly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "from scipy.spatial import distance\n",
    "import LPA\n",
    "import plotly.express as px\n",
    "from sklearn.decomposition import PCA"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The algorithm assumes correct input as in the following format (element, category_id, frequency_in_category). Failure to use data in the correct form will lead to erroneous results. In our case, elements are clone IDs, categories are samples and the frequency are instances."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv('clones_D145.csv').iloc[:,0:3]\n",
    "df.columns = ['element','category_id','frequency_in_category']\n",
    "df['category_id'] = df['category_id'].astype(str)\n",
    "df['element'] = df['element'].astype(str)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>element</th>\n",
       "      <th>category_id</th>\n",
       "      <th>frequency_in_category</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>917504</td>\n",
       "      <td>657</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>917505</td>\n",
       "      <td>158</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>917505</td>\n",
       "      <td>159</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>917505</td>\n",
       "      <td>647</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>917506</td>\n",
       "      <td>657</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  element category_id  frequency_in_category\n",
       "0  917504         657                      8\n",
       "1  917505         158                      1\n",
       "2  917505         159                      2\n",
       "3  917505         647                      1\n",
       "4  917506         657                      6"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once the data exists, one can deploy LPA to receive various results. The following examples will demonstrate this."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Distance from the world\n",
    "The following method returns the distance of every category from the domain, as a similarity measure between 0 and 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>distance_summary</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>category_id</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>148</th>\n",
       "      <td>0.908867</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>149</th>\n",
       "      <td>0.954583</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>150</th>\n",
       "      <td>0.872462</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>151</th>\n",
       "      <td>0.664769</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>152</th>\n",
       "      <td>0.651656</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>153</th>\n",
       "      <td>0.624774</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>154</th>\n",
       "      <td>0.671368</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>155</th>\n",
       "      <td>0.654299</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>156</th>\n",
       "      <td>0.688119</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>157</th>\n",
       "      <td>0.535183</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>158</th>\n",
       "      <td>0.475372</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>159</th>\n",
       "      <td>0.532676</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>160</th>\n",
       "      <td>0.636982</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>161</th>\n",
       "      <td>0.669561</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>162</th>\n",
       "      <td>0.710576</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>163</th>\n",
       "      <td>0.642145</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>164</th>\n",
       "      <td>0.642152</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>165</th>\n",
       "      <td>0.628498</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>166</th>\n",
       "      <td>0.802827</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>167</th>\n",
       "      <td>0.895838</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>168</th>\n",
       "      <td>0.690796</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>169</th>\n",
       "      <td>0.995890</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>170</th>\n",
       "      <td>0.972419</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>171</th>\n",
       "      <td>0.997712</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>172</th>\n",
       "      <td>0.579597</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>173</th>\n",
       "      <td>0.585825</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>174</th>\n",
       "      <td>0.576517</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>637</th>\n",
       "      <td>0.707133</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>638</th>\n",
       "      <td>0.720589</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>639</th>\n",
       "      <td>0.706946</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>640</th>\n",
       "      <td>0.505803</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>641</th>\n",
       "      <td>0.527935</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>642</th>\n",
       "      <td>0.531471</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>643</th>\n",
       "      <td>0.494476</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>644</th>\n",
       "      <td>0.488336</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>645</th>\n",
       "      <td>0.476230</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>646</th>\n",
       "      <td>0.345012</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>647</th>\n",
       "      <td>0.370471</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>648</th>\n",
       "      <td>0.375078</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>649</th>\n",
       "      <td>0.464065</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>650</th>\n",
       "      <td>0.491589</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>651</th>\n",
       "      <td>0.494187</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>652</th>\n",
       "      <td>0.462028</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>653</th>\n",
       "      <td>0.468774</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>654</th>\n",
       "      <td>0.517223</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>655</th>\n",
       "      <td>0.588917</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>656</th>\n",
       "      <td>0.614783</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>657</th>\n",
       "      <td>0.620170</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>658</th>\n",
       "      <td>0.820259</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>659</th>\n",
       "      <td>0.850650</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>660</th>\n",
       "      <td>0.811993</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>661</th>\n",
       "      <td>0.441040</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>662</th>\n",
       "      <td>0.449382</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>663</th>\n",
       "      <td>0.459783</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             distance_summary\n",
       "category_id                  \n",
       "148                  0.908867\n",
       "149                  0.954583\n",
       "150                  0.872462\n",
       "151                  0.664769\n",
       "152                  0.651656\n",
       "153                  0.624774\n",
       "154                  0.671368\n",
       "155                  0.654299\n",
       "156                  0.688119\n",
       "157                  0.535183\n",
       "158                  0.475372\n",
       "159                  0.532676\n",
       "160                  0.636982\n",
       "161                  0.669561\n",
       "162                  0.710576\n",
       "163                  0.642145\n",
       "164                  0.642152\n",
       "165                  0.628498\n",
       "166                  0.802827\n",
       "167                  0.895838\n",
       "168                  0.690796\n",
       "169                  0.995890\n",
       "170                  0.972419\n",
       "171                  0.997712\n",
       "172                  0.579597\n",
       "173                  0.585825\n",
       "174                  0.576517\n",
       "637                  0.707133\n",
       "638                  0.720589\n",
       "639                  0.706946\n",
       "640                  0.505803\n",
       "641                  0.527935\n",
       "642                  0.531471\n",
       "643                  0.494476\n",
       "644                  0.488336\n",
       "645                  0.476230\n",
       "646                  0.345012\n",
       "647                  0.370471\n",
       "648                  0.375078\n",
       "649                  0.464065\n",
       "650                  0.491589\n",
       "651                  0.494187\n",
       "652                  0.462028\n",
       "653                  0.468774\n",
       "654                  0.517223\n",
       "655                  0.588917\n",
       "656                  0.614783\n",
       "657                  0.620170\n",
       "658                  0.820259\n",
       "659                  0.850650\n",
       "660                  0.811993\n",
       "661                  0.441040\n",
       "662                  0.449382\n",
       "663                  0.459783"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "LPA.distance_from_world(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Creating Signatures\n",
    "Another prominent use of LPA is creating a signature for every category, which is made up of the most meaningful terms for every category, whether in their prominence or absence. By default the signature length is 500 and the epsilon is set to 1*(corpus size * 2), but both these parameters can be changed when calling the method.\n",
    "Epsilon frac - a number greater than 1 will decrease the weight of epsilon (the default weight given to missing terms) while a number between 0 and 1 will increase it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>category_id</th>\n",
       "      <th>element</th>\n",
       "      <th>KL</th>\n",
       "      <th>existing_element_flag</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>148</td>\n",
       "      <td>912073</td>\n",
       "      <td>0.092313</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>148</td>\n",
       "      <td>910708</td>\n",
       "      <td>0.057108</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>148</td>\n",
       "      <td>937487</td>\n",
       "      <td>0.041921</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>148</td>\n",
       "      <td>906708</td>\n",
       "      <td>0.035227</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>148</td>\n",
       "      <td>913922</td>\n",
       "      <td>0.029040</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>495</th>\n",
       "      <td>663</td>\n",
       "      <td>898718</td>\n",
       "      <td>0.000442</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>496</th>\n",
       "      <td>663</td>\n",
       "      <td>916272</td>\n",
       "      <td>0.000442</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>497</th>\n",
       "      <td>663</td>\n",
       "      <td>909442</td>\n",
       "      <td>0.000442</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>498</th>\n",
       "      <td>663</td>\n",
       "      <td>887844</td>\n",
       "      <td>0.000442</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>499</th>\n",
       "      <td>663</td>\n",
       "      <td>936318</td>\n",
       "      <td>0.000442</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>27000 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    category_id element        KL  existing_element_flag\n",
       "0           148  912073  0.092313                    0.0\n",
       "1           148  910708  0.057108                    1.0\n",
       "2           148  937487  0.041921                    1.0\n",
       "3           148  906708  0.035227                    1.0\n",
       "4           148  913922  0.029040                    0.0\n",
       "..          ...     ...       ...                    ...\n",
       "495         663  898718  0.000442                    1.0\n",
       "496         663  916272  0.000442                    1.0\n",
       "497         663  909442  0.000442                    1.0\n",
       "498         663  887844  0.000442                    1.0\n",
       "499         663  936318  0.000442                    1.0\n",
       "\n",
       "[27000 rows x 4 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "LPA.create_signatures(df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>category_id</th>\n",
       "      <th>element</th>\n",
       "      <th>KL</th>\n",
       "      <th>existing_element_flag</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>148</td>\n",
       "      <td>912073</td>\n",
       "      <td>0.102636</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>148</td>\n",
       "      <td>910708</td>\n",
       "      <td>0.057108</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>148</td>\n",
       "      <td>937487</td>\n",
       "      <td>0.041921</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>148</td>\n",
       "      <td>906708</td>\n",
       "      <td>0.035227</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>148</td>\n",
       "      <td>913922</td>\n",
       "      <td>0.032759</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>663</td>\n",
       "      <td>889373</td>\n",
       "      <td>0.006784</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>663</td>\n",
       "      <td>907782</td>\n",
       "      <td>0.006750</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>663</td>\n",
       "      <td>909584</td>\n",
       "      <td>0.006199</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>663</td>\n",
       "      <td>886894</td>\n",
       "      <td>0.006065</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>663</td>\n",
       "      <td>908003</td>\n",
       "      <td>0.005964</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1080 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   category_id element        KL  existing_element_flag\n",
       "0          148  912073  0.102636                    0.0\n",
       "1          148  910708  0.057108                    1.0\n",
       "2          148  937487  0.041921                    1.0\n",
       "3          148  906708  0.035227                    1.0\n",
       "4          148  913922  0.032759                    0.0\n",
       "..         ...     ...       ...                    ...\n",
       "15         663  889373  0.006784                    1.0\n",
       "16         663  907782  0.006750                    1.0\n",
       "17         663  909584  0.006199                    0.0\n",
       "18         663  886894  0.006065                    0.0\n",
       "19         663  908003  0.005964                    0.0\n",
       "\n",
       "[1080 rows x 4 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "LPA.create_signatures(df,epsilon_frac=5,sig_length=20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Distances between pairs of categories (Sockpuppet Distance)\n",
    "Finally, one can use the signatures created to calculate the L1 distance between every pair of categories. An elaboration on the method can be found here - https://github.com/hagitbenshoshan/text_distance/blob/master/Step3.md\n",
    "Different signature lengths and epsilons can have dramatic effects on the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user1</th>\n",
       "      <th>user2</th>\n",
       "      <th>distance_between_users</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>657</td>\n",
       "      <td>657</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>657</td>\n",
       "      <td>158</td>\n",
       "      <td>0.886054</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>657</td>\n",
       "      <td>159</td>\n",
       "      <td>0.909390</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>657</td>\n",
       "      <td>647</td>\n",
       "      <td>0.911908</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>657</td>\n",
       "      <td>157</td>\n",
       "      <td>0.902767</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2911</th>\n",
       "      <td>170</td>\n",
       "      <td>164</td>\n",
       "      <td>0.489834</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2912</th>\n",
       "      <td>170</td>\n",
       "      <td>166</td>\n",
       "      <td>0.315109</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2913</th>\n",
       "      <td>170</td>\n",
       "      <td>167</td>\n",
       "      <td>0.289273</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2914</th>\n",
       "      <td>170</td>\n",
       "      <td>168</td>\n",
       "      <td>0.461529</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2915</th>\n",
       "      <td>170</td>\n",
       "      <td>170</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2916 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     user1 user2  distance_between_users\n",
       "0      657   657                0.000000\n",
       "1      657   158                0.886054\n",
       "2      657   159                0.909390\n",
       "3      657   647                0.911908\n",
       "4      657   157                0.902767\n",
       "...    ...   ...                     ...\n",
       "2911   170   164                0.489834\n",
       "2912   170   166                0.315109\n",
       "2913   170   167                0.289273\n",
       "2914   170   168                0.461529\n",
       "2915   170   170                0.000000\n",
       "\n",
       "[2916 rows x 3 columns]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "LPA.SockPuppetDistance(LPA.create_signatures(df),df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user1</th>\n",
       "      <th>user2</th>\n",
       "      <th>distance_between_users</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>657</td>\n",
       "      <td>657</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>657</td>\n",
       "      <td>158</td>\n",
       "      <td>0.841226</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>657</td>\n",
       "      <td>159</td>\n",
       "      <td>0.840656</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>657</td>\n",
       "      <td>647</td>\n",
       "      <td>0.812306</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>657</td>\n",
       "      <td>157</td>\n",
       "      <td>0.826292</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2911</th>\n",
       "      <td>170</td>\n",
       "      <td>164</td>\n",
       "      <td>0.629986</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2912</th>\n",
       "      <td>170</td>\n",
       "      <td>166</td>\n",
       "      <td>0.442135</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2913</th>\n",
       "      <td>170</td>\n",
       "      <td>167</td>\n",
       "      <td>0.429387</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2914</th>\n",
       "      <td>170</td>\n",
       "      <td>168</td>\n",
       "      <td>0.492493</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2915</th>\n",
       "      <td>170</td>\n",
       "      <td>170</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2916 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     user1 user2  distance_between_users\n",
       "0      657   657                0.000000\n",
       "1      657   158                0.841226\n",
       "2      657   159                0.840656\n",
       "3      657   647                0.812306\n",
       "4      657   157                0.826292\n",
       "...    ...   ...                     ...\n",
       "2911   170   164                0.629986\n",
       "2912   170   166                0.442135\n",
       "2913   170   167                0.429387\n",
       "2914   170   168                0.492493\n",
       "2915   170   170                0.000000\n",
       "\n",
       "[2916 rows x 3 columns]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "LPA.SockPuppetDistance(LPA.create_signatures(df, epsilon_frac=10, sig_length=300),df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Examples of Further analysis - PCA\n",
    "Once we have calculated the distances between every category, we can perform further analysis on the results. The following example demonstrates this using scikit-learn's PCA method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv('D145_PCA.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "tissue_colors =    {\n",
    "                   'PBL' : '#67001f',\n",
    "                   'BM' : '#b2182b',\n",
    "                   'SPL' : '#d6604d',\n",
    "                   'Lung' : '#f4a582',\n",
    "                   'MLN' : '#515151',\n",
    "                   'Duodenum' : '#a1daf7',\n",
    "                   'Jejunum' : '#92c5de',\n",
    "                   'Ileum' : '#4393c3',                  \n",
    "                   'Colon' : '#2166ac',\n",
    "                    }\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = px.scatter_3d(df, x='x', y='y', z='z', color='tissue', color_discrete_map = tissue_colors)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "hovertemplate": "tissue=BM<br>x=%{x}<br>y=%{y}<br>z=%{z}<extra></extra>",
         "legendgroup": "BM",
         "marker": {
          "color": "#b2182b",
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "BM",
         "scene": "scene",
         "showlegend": true,
         "type": "scatter3d",
         "x": [
          -1.2097377894984631,
          -1.3937251923381355,
          -1.3744493597883556,
          1.78962959417916,
          1.7220935067146281,
          1.8696293946040896
         ],
         "y": [
          0.005830064098043207,
          0.040911310778791836,
          0.05249215142582351,
          -0.17109971451102912,
          -0.19154831154798851,
          -0.13465539803965984
         ],
         "z": [
          0.025700420128892072,
          0.022102659520183403,
          0.01747737999917922,
          0.3446845958767744,
          0.3560916217719509,
          0.3371332541493113
         ]
        },
        {
         "hovertemplate": "tissue=Colon<br>x=%{x}<br>y=%{y}<br>z=%{z}<extra></extra>",
         "legendgroup": "Colon",
         "marker": {
          "color": "#2166ac",
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Colon",
         "scene": "scene",
         "showlegend": true,
         "type": "scatter3d",
         "x": [
          -1.250485009131479,
          -1.4148174863891352,
          -1.3688949404596922,
          1.7823504479824537,
          0.7821889215959092,
          0.7648802899687155
         ],
         "y": [
          0.024353944978865224,
          0.07968452154185864,
          0.07161245351047242,
          -0.3271816979161226,
          -0.3523377500484797,
          -0.357881678495251
         ],
         "z": [
          -0.02439330380907184,
          -0.02472860588767813,
          -0.031598048499609926,
          0.012637003626191395,
          -0.08325146839633175,
          -0.09629709533082104
         ]
        },
        {
         "hovertemplate": "tissue=Ileum<br>x=%{x}<br>y=%{y}<br>z=%{z}<extra></extra>",
         "legendgroup": "Ileum",
         "marker": {
          "color": "#4393c3",
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Ileum",
         "scene": "scene",
         "showlegend": true,
         "type": "scatter3d",
         "x": [
          -1.1727109575406316,
          -1.3759498922586006,
          -1.4687613318876165,
          0.9727047581834544,
          0.8920571128892912,
          0.9111083860385188
         ],
         "y": [
          -0.014369727820599578,
          0.047057510846592,
          0.06449573873892624,
          -0.4727500883685964,
          -0.462408139000603,
          -0.4651823189675018
         ],
         "z": [
          -0.059878119774301165,
          -0.06318081892362187,
          -0.049217593125878784,
          -0.3021769884176447,
          -0.29196477602078863,
          -0.3086985282141059
         ]
        },
        {
         "hovertemplate": "tissue=Jejunum<br>x=%{x}<br>y=%{y}<br>z=%{z}<extra></extra>",
         "legendgroup": "Jejunum",
         "marker": {
          "color": "#92c5de",
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Jejunum",
         "scene": "scene",
         "showlegend": true,
         "type": "scatter3d",
         "x": [
          -0.799581100475114,
          -1.322432922040788,
          -1.4471037914568503,
          2.1504316855539485,
          1.912189633925008,
          2.0268994305881023
         ],
         "y": [
          -0.0809182274418526,
          0.03209610056546477,
          0.07063869500495175,
          -0.35015895552924275,
          -0.4238252420732617,
          -0.4029399852455632
         ],
         "z": [
          -0.08320272715012954,
          -0.05951517745740635,
          -0.04279439405443063,
          -0.22228934968313774,
          -0.22239248288210292,
          -0.20314878624123295
         ]
        },
        {
         "hovertemplate": "tissue=Lung<br>x=%{x}<br>y=%{y}<br>z=%{z}<extra></extra>",
         "legendgroup": "Lung",
         "marker": {
          "color": "#f4a582",
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "Lung",
         "scene": "scene",
         "showlegend": true,
         "type": "scatter3d",
         "x": [
          -1.5880971142256326,
          -1.5783453350225731,
          -1.5735555699163744,
          -1.1567008907884084,
          -1.1893756198724341,
          -1.3289820357756152
         ],
         "y": [
          0.0997030619612206,
          0.10243431740489373,
          0.10553476675005463,
          0.06038238006799195,
          0.06254589455750048,
          0.08111283598776249
         ],
         "z": [
          0.003695728870635265,
          0.0036870688444793406,
          0.0016304856094484287,
          0.02568721692148105,
          0.02440808838898921,
          0.022026010154458927
         ]
        },
        {
         "hovertemplate": "tissue=MLN<br>x=%{x}<br>y=%{y}<br>z=%{z}<extra></extra>",
         "legendgroup": "MLN",
         "marker": {
          "color": "#515151",
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "MLN",
         "scene": "scene",
         "showlegend": true,
         "type": "scatter3d",
         "x": [
          -1.5508649190480943,
          -1.6314014013345497,
          -1.590085111116731,
          2.878359340562925,
          3.0685440270823667,
          3.001623185571876
         ],
         "y": [
          0.07039728043313875,
          0.09108812063075432,
          0.09166579404094176,
          0.4368351871139271,
          0.517705428800557,
          0.4937730339585641
         ],
         "z": [
          -0.0024876777808731762,
          0.001008972416954312,
          -0.0020570568077521267,
          -0.13935971903797895,
          -0.2553442775566029,
          -0.2154106055430102
         ]
        },
        {
         "hovertemplate": "tissue=PBL<br>x=%{x}<br>y=%{y}<br>z=%{z}<extra></extra>",
         "legendgroup": "PBL",
         "marker": {
          "color": "#67001f",
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "PBL",
         "scene": "scene",
         "showlegend": true,
         "type": "scatter3d",
         "x": [
          -1.173722987207167,
          -1.6310561897475464,
          -1.619841148835984,
          -1.2669038599426476,
          -1.3747343220176884,
          -0.9188942233662092
         ],
         "y": [
          -0.01369834889497836,
          0.0914182720794543,
          0.08746797258441938,
          0.03635896017002546,
          0.05685612826086784,
          -0.04169941876391163
         ],
         "z": [
          0.011098905998144856,
          0.0015255389857131152,
          -0.0010695005420857818,
          0.015573435509820766,
          0.015619595779834116,
          0.045096509710364566
         ]
        },
        {
         "hovertemplate": "tissue=SPL<br>x=%{x}<br>y=%{y}<br>z=%{z}<extra></extra>",
         "legendgroup": "SPL",
         "marker": {
          "color": "#d6604d",
          "symbol": "circle"
         },
         "mode": "markers",
         "name": "SPL",
         "scene": "scene",
         "showlegend": true,
         "type": "scatter3d",
         "x": [
          -1.1999037055386566,
          -1.3337552647541016,
          -1.323680391640247,
          1.8882460679373527,
          1.8887837873938635,
          1.883686920019908
         ],
         "y": [
          0.040466750720094175,
          0.07523703572304095,
          0.07976897698912301,
          -0.13417724801996467,
          -0.13610482645496627,
          -0.1367686160714187
         ],
         "z": [
          0.04813736695792465,
          0.039832582685062005,
          0.03539540370415575,
          0.456352254722516,
          0.46803891188955893,
          0.462334081583473
         ]
        }
       ],
       "layout": {
        "legend": {
         "title": {
          "text": "tissue"
         },
         "tracegroupgap": 0
        },
        "margin": {
         "t": 60
        },
        "scene": {
         "domain": {
          "x": [
           0,
           1
          ],
          "y": [
           0,
           1
          ]
         },
         "xaxis": {
          "title": {
           "text": "x"
          }
         },
         "yaxis": {
          "title": {
           "text": "y"
          }
         },
         "zaxis": {
          "title": {
           "text": "z"
          }
         }
        },
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "#E5ECF6",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "#E5ECF6",
          "polar": {
           "angularaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "radialaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "yaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "zaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "caxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          }
         }
        }
       }
      },
      "text/html": [
       "<div>                            <div id=\"98fc0a18-1fa2-43de-a9b3-40e25957641a\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                require([\"plotly\"], function(Plotly) {                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"98fc0a18-1fa2-43de-a9b3-40e25957641a\")) {                    Plotly.newPlot(                        \"98fc0a18-1fa2-43de-a9b3-40e25957641a\",                        [{\"hovertemplate\": \"tissue=BM<br>x=%{x}<br>y=%{y}<br>z=%{z}<extra></extra>\", \"legendgroup\": \"BM\", \"marker\": {\"color\": \"#b2182b\", \"symbol\": \"circle\"}, \"mode\": \"markers\", \"name\": \"BM\", \"scene\": \"scene\", \"showlegend\": true, \"type\": \"scatter3d\", \"x\": [-1.2097377894984631, -1.3937251923381355, -1.3744493597883556, 1.78962959417916, 1.7220935067146281, 1.8696293946040896], \"y\": [0.005830064098043207, 0.040911310778791836, 0.05249215142582351, -0.17109971451102912, -0.19154831154798851, -0.13465539803965984], \"z\": [0.025700420128892072, 0.022102659520183403, 0.01747737999917922, 0.3446845958767744, 0.3560916217719509, 0.3371332541493113]}, {\"hovertemplate\": \"tissue=Colon<br>x=%{x}<br>y=%{y}<br>z=%{z}<extra></extra>\", \"legendgroup\": \"Colon\", \"marker\": {\"color\": \"#2166ac\", \"symbol\": \"circle\"}, \"mode\": \"markers\", \"name\": \"Colon\", \"scene\": \"scene\", \"showlegend\": true, \"type\": \"scatter3d\", \"x\": [-1.250485009131479, -1.4148174863891352, -1.3688949404596922, 1.7823504479824537, 0.7821889215959092, 0.7648802899687155], \"y\": [0.024353944978865224, 0.07968452154185864, 0.07161245351047242, -0.3271816979161226, -0.3523377500484797, -0.357881678495251], \"z\": [-0.02439330380907184, -0.02472860588767813, -0.031598048499609926, 0.012637003626191395, -0.08325146839633175, -0.09629709533082104]}, {\"hovertemplate\": \"tissue=Ileum<br>x=%{x}<br>y=%{y}<br>z=%{z}<extra></extra>\", \"legendgroup\": \"Ileum\", \"marker\": {\"color\": \"#4393c3\", \"symbol\": \"circle\"}, \"mode\": \"markers\", \"name\": \"Ileum\", \"scene\": \"scene\", \"showlegend\": true, \"type\": \"scatter3d\", \"x\": [-1.1727109575406316, -1.3759498922586006, -1.4687613318876165, 0.9727047581834544, 0.8920571128892912, 0.9111083860385188], \"y\": [-0.014369727820599578, 0.047057510846592, 0.06449573873892624, -0.4727500883685964, -0.462408139000603, -0.4651823189675018], \"z\": [-0.059878119774301165, -0.06318081892362187, -0.049217593125878784, -0.3021769884176447, -0.29196477602078863, -0.3086985282141059]}, {\"hovertemplate\": \"tissue=Jejunum<br>x=%{x}<br>y=%{y}<br>z=%{z}<extra></extra>\", \"legendgroup\": \"Jejunum\", \"marker\": {\"color\": \"#92c5de\", \"symbol\": \"circle\"}, \"mode\": \"markers\", \"name\": \"Jejunum\", \"scene\": \"scene\", \"showlegend\": true, \"type\": \"scatter3d\", \"x\": [-0.799581100475114, -1.322432922040788, -1.4471037914568503, 2.1504316855539485, 1.912189633925008, 2.0268994305881023], \"y\": [-0.0809182274418526, 0.03209610056546477, 0.07063869500495175, -0.35015895552924275, -0.4238252420732617, -0.4029399852455632], \"z\": [-0.08320272715012954, -0.05951517745740635, -0.04279439405443063, -0.22228934968313774, -0.22239248288210292, -0.20314878624123295]}, {\"hovertemplate\": \"tissue=Lung<br>x=%{x}<br>y=%{y}<br>z=%{z}<extra></extra>\", \"legendgroup\": \"Lung\", \"marker\": {\"color\": \"#f4a582\", \"symbol\": \"circle\"}, \"mode\": \"markers\", \"name\": \"Lung\", \"scene\": \"scene\", \"showlegend\": true, \"type\": \"scatter3d\", \"x\": [-1.5880971142256326, -1.5783453350225731, -1.5735555699163744, -1.1567008907884084, -1.1893756198724341, -1.3289820357756152], \"y\": [0.0997030619612206, 0.10243431740489373, 0.10553476675005463, 0.06038238006799195, 0.06254589455750048, 0.08111283598776249], \"z\": [0.003695728870635265, 0.0036870688444793406, 0.0016304856094484287, 0.02568721692148105, 0.02440808838898921, 0.022026010154458927]}, {\"hovertemplate\": \"tissue=MLN<br>x=%{x}<br>y=%{y}<br>z=%{z}<extra></extra>\", \"legendgroup\": \"MLN\", \"marker\": {\"color\": \"#515151\", \"symbol\": \"circle\"}, \"mode\": \"markers\", \"name\": \"MLN\", \"scene\": \"scene\", \"showlegend\": true, \"type\": \"scatter3d\", \"x\": [-1.5508649190480943, -1.6314014013345497, -1.590085111116731, 2.878359340562925, 3.0685440270823667, 3.001623185571876], \"y\": [0.07039728043313875, 0.09108812063075432, 0.09166579404094176, 0.4368351871139271, 0.517705428800557, 0.4937730339585641], \"z\": [-0.0024876777808731762, 0.001008972416954312, -0.0020570568077521267, -0.13935971903797895, -0.2553442775566029, -0.2154106055430102]}, {\"hovertemplate\": \"tissue=PBL<br>x=%{x}<br>y=%{y}<br>z=%{z}<extra></extra>\", \"legendgroup\": \"PBL\", \"marker\": {\"color\": \"#67001f\", \"symbol\": \"circle\"}, \"mode\": \"markers\", \"name\": \"PBL\", \"scene\": \"scene\", \"showlegend\": true, \"type\": \"scatter3d\", \"x\": [-1.173722987207167, -1.6310561897475464, -1.619841148835984, -1.2669038599426476, -1.3747343220176884, -0.9188942233662092], \"y\": [-0.01369834889497836, 0.0914182720794543, 0.08746797258441938, 0.03635896017002546, 0.05685612826086784, -0.04169941876391163], \"z\": [0.011098905998144856, 0.0015255389857131152, -0.0010695005420857818, 0.015573435509820766, 0.015619595779834116, 0.045096509710364566]}, {\"hovertemplate\": \"tissue=SPL<br>x=%{x}<br>y=%{y}<br>z=%{z}<extra></extra>\", \"legendgroup\": \"SPL\", \"marker\": {\"color\": \"#d6604d\", \"symbol\": \"circle\"}, \"mode\": \"markers\", \"name\": \"SPL\", \"scene\": \"scene\", \"showlegend\": true, \"type\": \"scatter3d\", \"x\": [-1.1999037055386566, -1.3337552647541016, -1.323680391640247, 1.8882460679373527, 1.8887837873938635, 1.883686920019908], \"y\": [0.040466750720094175, 0.07523703572304095, 0.07976897698912301, -0.13417724801996467, -0.13610482645496627, -0.1367686160714187], \"z\": [0.04813736695792465, 0.039832582685062005, 0.03539540370415575, 0.456352254722516, 0.46803891188955893, 0.462334081583473]}],                        {\"legend\": {\"title\": {\"text\": \"tissue\"}, \"tracegroupgap\": 0}, \"margin\": {\"t\": 60}, \"scene\": {\"domain\": {\"x\": [0.0, 1.0], \"y\": [0.0, 1.0]}, \"xaxis\": {\"title\": {\"text\": \"x\"}}, \"yaxis\": {\"title\": {\"text\": \"y\"}}, \"zaxis\": {\"title\": {\"text\": \"z\"}}}, \"template\": {\"data\": {\"bar\": [{\"error_x\": {\"color\": \"#2a3f5f\"}, \"error_y\": {\"color\": \"#2a3f5f\"}, \"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"bar\"}], \"barpolar\": [{\"marker\": {\"line\": {\"color\": \"#E5ECF6\", \"width\": 0.5}}, \"type\": \"barpolar\"}], \"carpet\": [{\"aaxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"baxis\": {\"endlinecolor\": \"#2a3f5f\", \"gridcolor\": \"white\", \"linecolor\": \"white\", \"minorgridcolor\": \"white\", \"startlinecolor\": \"#2a3f5f\"}, \"type\": \"carpet\"}], \"choropleth\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"choropleth\"}], \"contour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"contour\"}], \"contourcarpet\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"contourcarpet\"}], \"heatmap\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmap\"}], \"heatmapgl\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"heatmapgl\"}], \"histogram\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"histogram\"}], \"histogram2d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2d\"}], \"histogram2dcontour\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"histogram2dcontour\"}], \"mesh3d\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"type\": \"mesh3d\"}], \"parcoords\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"parcoords\"}], \"pie\": [{\"automargin\": true, \"type\": \"pie\"}], \"scatter\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter\"}], \"scatter3d\": [{\"line\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatter3d\"}], \"scattercarpet\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattercarpet\"}], \"scattergeo\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergeo\"}], \"scattergl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattergl\"}], \"scattermapbox\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scattermapbox\"}], \"scatterpolar\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolar\"}], \"scatterpolargl\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterpolargl\"}], \"scatterternary\": [{\"marker\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"type\": \"scatterternary\"}], \"surface\": [{\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}, \"colorscale\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"type\": \"surface\"}], \"table\": [{\"cells\": {\"fill\": {\"color\": \"#EBF0F8\"}, \"line\": {\"color\": \"white\"}}, \"header\": {\"fill\": {\"color\": \"#C8D4E3\"}, \"line\": {\"color\": \"white\"}}, \"type\": \"table\"}]}, \"layout\": {\"annotationdefaults\": {\"arrowcolor\": \"#2a3f5f\", \"arrowhead\": 0, \"arrowwidth\": 1}, \"autotypenumbers\": \"strict\", \"coloraxis\": {\"colorbar\": {\"outlinewidth\": 0, \"ticks\": \"\"}}, \"colorscale\": {\"diverging\": [[0, \"#8e0152\"], [0.1, \"#c51b7d\"], [0.2, \"#de77ae\"], [0.3, \"#f1b6da\"], [0.4, \"#fde0ef\"], [0.5, \"#f7f7f7\"], [0.6, \"#e6f5d0\"], [0.7, \"#b8e186\"], [0.8, \"#7fbc41\"], [0.9, \"#4d9221\"], [1, \"#276419\"]], \"sequential\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]], \"sequentialminus\": [[0.0, \"#0d0887\"], [0.1111111111111111, \"#46039f\"], [0.2222222222222222, \"#7201a8\"], [0.3333333333333333, \"#9c179e\"], [0.4444444444444444, \"#bd3786\"], [0.5555555555555556, \"#d8576b\"], [0.6666666666666666, \"#ed7953\"], [0.7777777777777778, \"#fb9f3a\"], [0.8888888888888888, \"#fdca26\"], [1.0, \"#f0f921\"]]}, \"colorway\": [\"#636efa\", \"#EF553B\", \"#00cc96\", \"#ab63fa\", \"#FFA15A\", \"#19d3f3\", \"#FF6692\", \"#B6E880\", \"#FF97FF\", \"#FECB52\"], \"font\": {\"color\": \"#2a3f5f\"}, \"geo\": {\"bgcolor\": \"white\", \"lakecolor\": \"white\", \"landcolor\": \"#E5ECF6\", \"showlakes\": true, \"showland\": true, \"subunitcolor\": \"white\"}, \"hoverlabel\": {\"align\": \"left\"}, \"hovermode\": \"closest\", \"mapbox\": {\"style\": \"light\"}, \"paper_bgcolor\": \"white\", \"plot_bgcolor\": \"#E5ECF6\", \"polar\": {\"angularaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"radialaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"scene\": {\"xaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"yaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}, \"zaxis\": {\"backgroundcolor\": \"#E5ECF6\", \"gridcolor\": \"white\", \"gridwidth\": 2, \"linecolor\": \"white\", \"showbackground\": true, \"ticks\": \"\", \"zerolinecolor\": \"white\"}}, \"shapedefaults\": {\"line\": {\"color\": \"#2a3f5f\"}}, \"ternary\": {\"aaxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"baxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}, \"bgcolor\": \"#E5ECF6\", \"caxis\": {\"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\"}}, \"title\": {\"x\": 0.05}, \"xaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}, \"yaxis\": {\"automargin\": true, \"gridcolor\": \"white\", \"linecolor\": \"white\", \"ticks\": \"\", \"title\": {\"standoff\": 15}, \"zerolinecolor\": \"white\", \"zerolinewidth\": 2}}}},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('98fc0a18-1fa2-43de-a9b3-40e25957641a');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                });            </script>        </div>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Good Luck!"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "### code to turn sock puppet distance results to PCA\n",
    "X = df.iloc[:,:3].copy()\n",
    "X = pd.pivot_table(X, values='distance_between_users', index='user1',columns='user2')\n",
    "X.fillna(0, inplace= True)\n",
    "\n",
    "pca = PCA(n_components=3, svd_solver='full')\n",
    "clusters = pca.fit_transform(X)\n",
    "\n",
    "sample_clusters = pd.DataFrame(index=X.index, data = clusters, columns=['x','y', 'z'])\n",
    "\n",
    "sample_clusters.reset_index(inplace = True)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}