w207_openbarbell_part1_exercise_classification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "view-in-github"
   },
   "source": [
    "[View in Colaboratory](https://colab.research.google.com/github/YuliaZamriy/W207_Group_Project/blob/master/w207_openbarbell_baseline_submission_v2.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Dgn9Us1z_9W4"
   },
   "source": [
    "# W207 Final Project\n",
    "# Part 1: Exercise Classification\n",
    "\n",
    "## 7 August 2018\n",
    "\n",
    "### Project Description\n",
    "\n",
    "OpenBarbell is a device that is used to record and measure key lifting metrics during a lifter's workout. Examples of user-entered metrics are weight, exercise label, and exercise tags. Examples of metrics captured through the device are range of motion (ROM), peak velocity and rep duration. A better understanding of these metrics can help lifters improve performance, prevent injury, build training programs, and streamline their workouts.  Additionally, ML can add convenience for the user, allowing them to focus on their workouts, and less time navigating an app.  The goal of this project is to use the data produced by OpenBarbell to add greater intelligence to the user experience.\n",
    "\n",
    "### Project Goals\n",
    "\n",
    "1. **Exercise classification**  \n",
    "Build an algorithm to determine the exercise being performed. This serves two purposes and can be broken down into two subgoals:\n",
    "   \n",
    "    1. **Label existing data**  \n",
    "        *Description:* Exercise names are acquired through user input. This field is freeform and completely optional. Hence, we would like to build an algorithm to determine the exercise for each rep based on its characteristics (weight, range of motion, velocity, etc.). This will allows us to reliably prepare and clean existing, mislabeled data.\n",
    "   \n",
    "        *Goal Benchmark:* Classify exercises with >70% accuracy\n",
    "   \n",
    "    2. **Real-time exercise logging**  \n",
    "       *Description:* Workouts can require precise timing between exercises and repetitions. We also understand that workouts can be both physically and mentally exhausting. We would like to build a model that is able to take in device-captured data and classify the exercise in real-time. This would take that burden off of the user and allow them to better focus on their workout. \n",
    "   \n",
    "       *Goal Benchmark:* Classify exercises in <1000ms\n",
    "     \n",
    "2. **Lifter segmentation**  \n",
    "    *Description:* Every indivdual has their own unique combination of lifestyle habits, health goals, and athletic proficiency. Ideally, every person would have a customized training program to fit their needs. The precursor to a customized training program is getting a better understanding of the individual. We would like to create distinct groups of lifters based on their lifting characteristics for each exercise (weight, range of motion, velocity, etc.) as a way to segment populations of lifters.\n",
    "\n",
    "    *Goal Benchmark:* Identify a users best lift, as determined by volume and weight\n",
    "\n",
    "\n",
    "### Data and Source\n",
    "\n",
    " - [Project GitHub Repository](https://github.com/YuliaZamriy/W207_Group_Project)\n",
    "\n",
    "### Team\n",
    " - Renzee Reyes\n",
    " - Tim Witthoefft\n",
    " - Jack Workman\n",
    " - Yulia Zamriy\n",
    " \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 820
    },
    "colab_type": "code",
    "id": "3QptuiWS_9W5",
    "outputId": "0e2b9f0c-12f4-4704-f9b4-b8901af3aa6f"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'en_US.UTF-8'"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import os\n",
    "import zipfile\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "import locale\n",
    "import sys\n",
    "import itertools\n",
    "import json\n",
    "import random\n",
    "import time\n",
    "from locale import atof\n",
    "from sklearn import preprocessing\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.naive_bayes import GaussianNB\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.model_selection import cross_val_score\n",
    "from sklearn.model_selection import KFold\n",
    "from sklearn.metrics import confusion_matrix\n",
    "from sklearn.metrics import classification_report\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.decomposition import PCA\n",
    "from sklearn import metrics\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from IPython.display import display\n",
    "from scipy.stats import mode\n",
    "import statistics\n",
    "#from google.colab import files\n",
    "\n",
    "#!pip install ipywidgets\n",
    "from ipywidgets import IntProgress, HTML, VBox\n",
    "\n",
    "%matplotlib inline\n",
    "\n",
    "csv_data_file = 'ob_data_w207_filtered.csv'\n",
    "zip_data_file = 'ob_data_w207_filtered.zip'\n",
    "\n",
    "locale.setlocale(locale.LC_NUMERIC, '')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 17
    },
    "colab_type": "code",
    "id": "fY49NYF__9W9",
    "outputId": "f2f81566-c0bb-4cd3-844f-c15aef7ccde6"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>.container { width:90% !important; }</style>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Jupyter viewing options\n",
    "from IPython.core.display import display, HTML\n",
    "display(HTML(\"<style>.container { width:90% !important; }</style>\"))\n",
    "pd.set_option('display.max_columns', 500)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "F1eME6J6PxV2"
   },
   "source": [
    "# 1. Import Processed Data\n",
    "\n",
    "At this stage, we are following up on the work performed in the data processing notebook, which produces a csv file of clean and transformed data. Please refer to the data processing notebook for more detail on how the csv file is produced. Otherwise, we can simply import the csv into our main dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 198
    },
    "colab_type": "code",
    "id": "u_Eq-Cn-_9XB",
    "outputId": "72429fbb-5335-4034-b7a2-bd96c82502f7"
   },
   "outputs": [],
   "source": [
    "# Declare our main dataframe. This will be the primary dataframe on which\n",
    "# we perform our work and learning\n",
    "df_main = None\n",
    "\n",
    "# The project only comes with a zip file containing our data. This is to\n",
    "# minimize project size. If the zip file is found but not the csv file,\n",
    "# unzip the archive to extract the data file.\n",
    "if not os.path.exists(csv_data_file) and os.path.exists(zip_data_file):\n",
    "    zip_ref = zipfile.ZipFile(zip_data_file, 'r')\n",
    "    zip_ref.extractall()\n",
    "    zip_ref.close()\n",
    "    \n",
    "# If the zip extraction was successful or if the file already exists,\n",
    "# load it into our main df. If the file doesn't exist at this point,\n",
    "# raise an exception\n",
    "if os.path.exists(csv_data_file):\n",
    "    df_main = pd.read_csv(csv_data_file)\n",
    "else:\n",
    "    raise Exception('Data file not found! {0}'.format(csv_data_file))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ohhh21Ie_9XD"
   },
   "source": [
    "# 2. Exploratory Data Analysis\n",
    "\n",
    "To ensure that the data contains no hidden surprises, we do a quick EDA."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "CEpUdS3d_9XE"
   },
   "source": [
    "## 2.1 Shape and Head"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "colab_type": "code",
    "id": "mxbEwLQa_9XE",
    "outputId": "ba3ef5c7-9948-4435-de71-4e319e378b4e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Data shape: (181224, 45)\n"
     ]
    }
   ],
   "source": [
    "print(\"Data shape: {0}\".format(df_main.shape))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 479
    },
    "colab_type": "code",
    "id": "GK_fNNMY_9XG",
    "outputId": "8e41382e-3c36-4474-bcb0-0f8cfcbe7ded"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>_id</th>\n",
       "      <th>deleted</th>\n",
       "      <th>endTime</th>\n",
       "      <th>exercise</th>\n",
       "      <th>initialStartTime</th>\n",
       "      <th>metric</th>\n",
       "      <th>removed_set</th>\n",
       "      <th>reps</th>\n",
       "      <th>rpe</th>\n",
       "      <th>setID</th>\n",
       "      <th>setNumber</th>\n",
       "      <th>startTime</th>\n",
       "      <th>tags</th>\n",
       "      <th>userID</th>\n",
       "      <th>videoFileURL</th>\n",
       "      <th>videoType</th>\n",
       "      <th>weight</th>\n",
       "      <th>workoutID</th>\n",
       "      <th>exercise_clean</th>\n",
       "      <th>rpe_num</th>\n",
       "      <th>weight_lbs</th>\n",
       "      <th>appVersion</th>\n",
       "      <th>deviceIdentifier</th>\n",
       "      <th>deviceName</th>\n",
       "      <th>hardware</th>\n",
       "      <th>isValid</th>\n",
       "      <th>removed_rep</th>\n",
       "      <th>time</th>\n",
       "      <th>StartMessg</th>\n",
       "      <th>RepN</th>\n",
       "      <th>AvgVel</th>\n",
       "      <th>ROM</th>\n",
       "      <th>PeakVel</th>\n",
       "      <th>PeakVelLoc</th>\n",
       "      <th>PeakAccel</th>\n",
       "      <th>RepDur</th>\n",
       "      <th>TimeBWReps</th>\n",
       "      <th>TimeRepComp</th>\n",
       "      <th>TimeRepWait</th>\n",
       "      <th>SlowAllow</th>\n",
       "      <th>Backlight</th>\n",
       "      <th>MinAllow</th>\n",
       "      <th>setID_ix</th>\n",
       "      <th>RepCount</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>000051e0-54b2-48d8-98d2-79f65505c3e6</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Bench</td>\n",
       "      <td>2018-01-03 01:29:46.904</td>\n",
       "      <td>kgs</td>\n",
       "      <td>0.0</td>\n",
       "      <td>[{'isValid': True, 'removed': False, 'hardware...</td>\n",
       "      <td>7.5</td>\n",
       "      <td>000051e0-54b2-48d8-98d2-79f65505c3e6</td>\n",
       "      <td>3</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[]</td>\n",
       "      <td>5a2e998b05cbc9b8ef26ac7e</td>\n",
       "      <td>assets-library://asset/asset.mov?id=CFFFFAC8-0...</td>\n",
       "      <td>lift</td>\n",
       "      <td>87.5</td>\n",
       "      <td>5bac5444-7c37-49d0-be8e-32be1a09fbb1</td>\n",
       "      <td>bench</td>\n",
       "      <td>7.5</td>\n",
       "      <td>192.90425</td>\n",
       "      <td>3.2.3</td>\n",
       "      <td>709A15B7-B43C-AF0E-953A-6D4CEFEB4C83</td>\n",
       "      <td>OB 6666</td>\n",
       "      <td>ios</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>2018-01-03T01:29:46.902Z</td>\n",
       "      <td>-3456.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>0.258460</td>\n",
       "      <td>252.0</td>\n",
       "      <td>0.308408</td>\n",
       "      <td>37.0</td>\n",
       "      <td>11.073610</td>\n",
       "      <td>978699.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.752134e+09</td>\n",
       "      <td>0.0</td>\n",
       "      <td>130000.0</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>150000.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>000051e0-54b2-48d8-98d2-79f65505c3e6</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Bench</td>\n",
       "      <td>2018-01-03 01:29:46.904</td>\n",
       "      <td>kgs</td>\n",
       "      <td>0.0</td>\n",
       "      <td>[{'isValid': True, 'removed': False, 'hardware...</td>\n",
       "      <td>7.5</td>\n",
       "      <td>000051e0-54b2-48d8-98d2-79f65505c3e6</td>\n",
       "      <td>3</td>\n",
       "      <td>NaN</td>\n",
       "      <td>[]</td>\n",
       "      <td>5a2e998b05cbc9b8ef26ac7e</td>\n",
       "      <td>assets-library://asset/asset.mov?id=CFFFFAC8-0...</td>\n",
       "      <td>lift</td>\n",
       "      <td>87.5</td>\n",
       "      <td>5bac5444-7c37-49d0-be8e-32be1a09fbb1</td>\n",
       "      <td>bench</td>\n",
       "      <td>7.5</td>\n",
       "      <td>192.90425</td>\n",
       "      <td>3.2.3</td>\n",
       "      <td>709A15B7-B43C-AF0E-953A-6D4CEFEB4C83</td>\n",
       "      <td>OB 6666</td>\n",
       "      <td>ios</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>2018-01-03T01:29:49.526Z</td>\n",
       "      <td>-3456.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>0.273285</td>\n",
       "      <td>250.0</td>\n",
       "      <td>0.368293</td>\n",
       "      <td>72.0</td>\n",
       "      <td>0.714399</td>\n",
       "      <td>915894.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.754801e+09</td>\n",
       "      <td>0.0</td>\n",
       "      <td>130000.0</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>150000.0</td>\n",
       "      <td>1</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0                                   _id  deleted  endTime  \\\n",
       "0           0  000051e0-54b2-48d8-98d2-79f65505c3e6      NaN      NaN   \n",
       "1           1  000051e0-54b2-48d8-98d2-79f65505c3e6      NaN      NaN   \n",
       "\n",
       "  exercise         initialStartTime metric  removed_set  \\\n",
       "0    Bench  2018-01-03 01:29:46.904    kgs          0.0   \n",
       "1    Bench  2018-01-03 01:29:46.904    kgs          0.0   \n",
       "\n",
       "                                                reps  rpe  \\\n",
       "0  [{'isValid': True, 'removed': False, 'hardware...  7.5   \n",
       "1  [{'isValid': True, 'removed': False, 'hardware...  7.5   \n",
       "\n",
       "                                  setID  setNumber  startTime tags  \\\n",
       "0  000051e0-54b2-48d8-98d2-79f65505c3e6          3        NaN   []   \n",
       "1  000051e0-54b2-48d8-98d2-79f65505c3e6          3        NaN   []   \n",
       "\n",
       "                     userID  \\\n",
       "0  5a2e998b05cbc9b8ef26ac7e   \n",
       "1  5a2e998b05cbc9b8ef26ac7e   \n",
       "\n",
       "                                        videoFileURL videoType weight  \\\n",
       "0  assets-library://asset/asset.mov?id=CFFFFAC8-0...      lift   87.5   \n",
       "1  assets-library://asset/asset.mov?id=CFFFFAC8-0...      lift   87.5   \n",
       "\n",
       "                              workoutID exercise_clean  rpe_num  weight_lbs  \\\n",
       "0  5bac5444-7c37-49d0-be8e-32be1a09fbb1          bench      7.5   192.90425   \n",
       "1  5bac5444-7c37-49d0-be8e-32be1a09fbb1          bench      7.5   192.90425   \n",
       "\n",
       "  appVersion                      deviceIdentifier deviceName hardware  \\\n",
       "0      3.2.3  709A15B7-B43C-AF0E-953A-6D4CEFEB4C83    OB 6666      ios   \n",
       "1      3.2.3  709A15B7-B43C-AF0E-953A-6D4CEFEB4C83    OB 6666      ios   \n",
       "\n",
       "   isValid  removed_rep                      time  StartMessg  RepN    AvgVel  \\\n",
       "0     True        False  2018-01-03T01:29:46.902Z     -3456.0  11.0  0.258460   \n",
       "1     True        False  2018-01-03T01:29:49.526Z     -3456.0  12.0  0.273285   \n",
       "\n",
       "     ROM   PeakVel  PeakVelLoc  PeakAccel    RepDur  TimeBWReps   TimeRepComp  \\\n",
       "0  252.0  0.308408        37.0  11.073610  978699.0         0.0  1.752134e+09   \n",
       "1  250.0  0.368293        72.0   0.714399  915894.0         0.0  1.754801e+09   \n",
       "\n",
       "   TimeRepWait  SlowAllow  Backlight  MinAllow  setID_ix  RepCount  \n",
       "0          0.0   130000.0    10000.0  150000.0         1       1.0  \n",
       "1          0.0   130000.0    10000.0  150000.0         1       2.0  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_main.head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "LL-8hEX5_9XJ"
   },
   "source": [
    "## 2.2 Check for Null Values\n",
    "\n",
    "Here we observe that there are, in fact, quite a few null values in the dataset. Prior domain knowledge tells us that these are a result of older versions that do not support some of the newer fields and user input fields:\n",
    "\n",
    " - `endTime` and `startTime` replaced by `initialStartTime` and `time` in newer versions of the app\n",
    " - `rpe_num` (cleaned version of `rpe`) is entered through user input\n",
    " - `videoFileURL` and `videoType` are place holder for user's videos\n",
    "\n",
    "These null values seen here can be safely ignored."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 587
    },
    "colab_type": "code",
    "id": "NeBd2aFB_9XK",
    "outputId": "d50caf1d-885b-43d9-cf13-2085722b50f5"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABEcAAAJBCAYAAACkr9upAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzs3X24bXVZL/zvHYj5hqhs3wCDDE20IiW0F0uzFKwETRM0JTNR097tUesUZnnKXvQ8Hs3CJMGTgmkml+ExjonaSZSNEmjqwxZRthBsAZXUVOh+/phj6WS79tp7rzm3e8Pv87muea057jHGPX5zrbnWmvM7x0t1dwAAAABG9S27ewAAAAAAu5NwBAAAABiacAQAAAAYmnAEAAAAGJpwBAAAABiacAQAAAAYmnAEAPYAVfVzVfXPc9NdVd+xi7a1y3pvY3s3emy7S1UdPD32vfeAsXy4qh6yu8cBAMwIRwBgCarq0qq6sqpuM1f7hao6Z8nb+cuqOm2V+ndX1Zer6o7L3N6eqKoeUVXvrqrrqmpLVb2rqh61u8e1M7r7vt19znrWnQKeL1TVf1TV1VX1jqp6/E6s/5Cq2ryebe+Mb9Z2AGAZhCMAsDx7J/mVXbyN1yR5zHwIM3lykrd29zW7ePu7VVU9NsnfJjktyYFJ7pLkd5P81O4c127wPd192yT3zuw58fKqOmn3DgkAbrqEIwCwPH+S5DlVtd/WM1Y7pKOqzqmqX9iZDXT3e5N8OslPz/XZK8kTkpw6TR9ZVe+tqs9W1RVV9fKq2me1fluPYZXDe76zqs6uqmuq6mNV9TNz8x5ZVf827cHx6ap6zhpDr6r6n1X1uar6aFU9bCo+rqrO32rB36iqv1+tQZKXJPn97v6r7v5cd/9Xd7+ru582LfMtVfXfquqTVXVVVZ1WVbffxoAuraofm5t+QVX9r+n+ys/rKVV1WVVdW1XPqKrvq6oLp+/ty7f+vlXVn07LfqKqjl7jm/G1bU/bfcM01uumQ26OWON7+TXd/Znufm2SZyZ5flXdaer5lKr6yNTvkqp6+lS/TZK3Jbn7tOfJf1TV3dd6ztTMS6fv5+emx3+/ad4tp8f8qZrtOfUXVXWrbW1nRx4TAOwOwhEAWJ6NSc5JslZIsAynZbanyIofS3KLzN6MJskNSX4tyf5Jvj/Jw5L84s5uZHqDe3aS1yW5c5Ljk/x5Vd13WuTVSZ7e3bdLcr8k/7RGuwcmuWQa00lJ/q5mhwCdmeSQqrrP3LI/m+S1q/S4d5KDkrxxje383HR7aJJvT3LbJC9fY/nteWCSQ5M8Psn/SPLbmX2/75vkZ6rqR7Za9mOZPcY/TvLqKdDZEY9KcnqS/TL7nuzsmN+S2Z5LR07TVyX5yST7JnlKkpdW1f27+wtJjk5yeXffdrpdnrWfMw9P8sNJ7jWN7/FJrp7mvXiqH57kO5IckOR319gOAOyRhCMAsFy/m+SXqmrDLtzGa5P8SFUdOE0/OcnruvurSdLd53f3ud19fXdfmuQvk/zI6q3W9JNJLu3uv556fSDJm5I8dpr/1SSHVdW+3X3tNH9brkryP7r7q919RmYhwk9095eTnJFZIJIpeDk4yVtX6XGn6esVa2zniUle0t2XdPd/JHl+kuNq/Sdh/f3u/s/u/sckX0jy+u6+qrs/neQ9Sb53btlPdveruvuGzPbiuVtmh/3siH/u7rOmdV+b5Ht2ZpDTz/4zSe44Tf9Dd3+8Z96V5B+TPHiN9dd6znw1ye2SfGeS6u6PdPcVU/DztCS/1t3XdPd1Sf57kuN2ZuwAsCcQjgDAEnX3hzJ7Y/+8XbiNTyV5d5KfrarbJjk20yE1SVJV96qqt1bVv1fV5zN7w7r/Ojb1bUkeOB1q8dmq+mxm4cNdp/k/neSRST5Zs5Oifv8avT7d3T03/ckkK4dZnJrkCdOb7SclecMUmmxtZW+Fu62xnbtPvee3s3d2PKTY2pVz97+0yvRt56b/feVOd39xujs/fy3/Pnf/i0m+dWcCnaq6RZINSa6Zpo+uqnOnw6E+m9nPaZvPgbWeM939T5ntyfKKJFdW1clVte+0vVsnOX/u+fG/pzoA3KQIRwBg+U7K7BP1A+ZqX5i+3nqudtes36mZ7THy00k+sdVeG69M8tEkh3b3vkl+K8m2Du/4whpjuizJu7p7v7nbbbv7mUnS3ed19zGZHXLz90nesMZ4D9jqEJN7JLl86nNukq9ktmfDE7L6ITXJbG+TyzJ3vpVVXJ5ZqDO/netz41BjxVqP/abmmMwe5/ur6paZ7eHzp0nu0t37JTkrX38O9Crrr/mc6e6XdfcDMjuc6F5JfjOzPVW+lOS+c8+P208nit3WdgBgjyQcAYAl6+5NmR0q8stztS2ZnUj1Z6tqr6r6+ST3XGAzb8rs/Bu/l7m9Ria3S/L5JP9RVd+Z2ck6t+WCzK5+c+uq+o4kT52b99Yk96qqJ1XVLabb91XVfapqn6p6YlXdfjqk4/OZnbdiW+6c5JenHo9Lcp/M3rCvOC2zvROu7+5/Xq3BtOfJryf5nemEo/vW7ASsP1RVJ0+LvT7Jr1XVIdNeNf89yRndff02Hvtx05iOyNcPF7rJqKo7VtUTM9ur48XdfXWSfZLcMsmWJNdPJ4Z9+NxqVya5U934RLXbfM5MP/MHTnunfCHJfya5obv/K8mrMjufyZ2nZQ+oqkessR0A2CMJRwBg13hhkq0vt/u0zD5xvzqzT+D/Zb3NpxNergQkf7PV7OdktgfGdZm9eT1jjVYvzWyvjSszC1m+1ms6h8TDMzuHxOWZHfrx4szeeCezQ2AunQ7DeEam84Zsw/syO7HpZ5K8KMljpzfyK16b2Uldt7XXyMqY3pjZCUF/fhrTlUn+ILMTkibJKVOPdyf5RGZv5H9pG+1+J7OA6trMQqbXrbXtPcy/VtV/JNmU5BcyO+/H7yZf+7n9cmZ78lyb2XPhzJUVu/ujmYVIl0yHw9w9az9n9p1q12Z2mNLVme2VkiTPncZw7vQ8+D+ZnTh3W9sBgD1S3fjwXwCAb76qulVmJ229f3dfvLvHAwCMxZ4jAMCe4JlJzhOMAAC7w3ovawcAsBRVdWlmJ/88djcPBQAYlMNqAAAAgKE5rAYAAAAYmnAEAAAAGNrN7pwj+++/fx988MG7exgAAADAbnT++ed/prs37MiyN7tw5OCDD87GjRt39zAAAACA3aiqPrmjyzqsBgAAABiacAQAAAAYmnAEAAAAGJpwBAAAABiacAQAAAAYmnAEAAAAGJpwBAAAABiacAQAAAAYmnAEAAAAGJpwBAAAABiacAQAAAAYmnAEAAAAGJpwBAAAABiacAQAAAAYmnAEAAAAGJpwBAAAABiacAQAAAAYmnAEAAAAGJpwBAAAABiacAQAAAAY2t67ewCQJB/5zvss3OM+H/3IEkYCu953nfpdC/e46ISLljASAAAgsecIAAAAMDjhCAAAADA04QgAAAAwNOEIAAAAMDThCAAAADA04QgAAAAwNOEIAAAAMDThCAAAADA04QgAAAAwNOEIAAAAMLTthiNVdUpVXVVVH5qrnVFVF0y3S6vqgql+cFV9aW7eX8yt84CquqiqNlXVy6qqpvodq+rsqrp4+nqHqV7Tcpuq6sKquv/yHz4AAAAwuh3Zc+Q1SY6aL3T347v78O4+PMmbkvzd3OyPr8zr7mfM1V+Z5MQkh063lZ7PS/KO7j40yTum6SQ5em7ZE6f1AQAAAJZqu+FId787yTWrzZv2/viZJK9fq0dV3S3Jvt393u7uJKclOXaafUySU6f7p25VP61nzk2y39QHAAAAYGkWPefIg5Nc2d0Xz9UOqaoPVtW7qurBU+2AJJvnltk81ZLkLt19RZJMX+88t85l21gHAAAAYCn2XnD943PjvUauSHKP7r66qh6Q5O+r6r5JapV1ezu9d3idqjoxs0Nvco973GO7gwYAAABYse49R6pq7ySPSXLGSq27v9zdV0/3z0/y8ST3ymyvjwPnVj8wyeXT/StXDpeZvl411TcnOWgb69xId5/c3Ud09xEbNmxY70MCAAAABrTIYTU/luSj3f21w2WqakNV7TXd//bMTqZ6yXS4zHVV9aDpPCVPTvKWabUzk5ww3T9hq/qTp6vWPCjJ51YOvwEAAABYlh25lO/rk7w3yb2ranNVPXWadVy+8USsP5zkwqr61yRvTPKM7l45meszk/xVkk2Z7VHytqn+R0l+vKouTvLj03SSnJXkkmn5VyX5xZ1/eAAAAABr2+45R7r7+G3Uf26V2psyu7TvastvTHK/VepXJ3nYKvVO8qztjQ8AAABgEYterQYAAADgJk04AgAAAAxNOAIAAAAMbbvnHAGAZfmzx//kwj1+44y33mh68/Pes3DPA//owQv3AADgpsueIwAAAMDQhCMAAADA0IQjAAAAwNCEIwAAAMDQhCMAAADA0IQjAAAAwNCEIwAAAMDQhCMAAADA0IQjAAAAwNCEIwAAAMDQhCMAAADA0IQjAAAAwNCEIwAAAMDQhCMAAADA0IQjAAAAwNCEIwAAAMDQhCMAAADA0IQjAAAAwNCEIwAAAMDQhCMAAADA0IQjAAAAwNCEIwAAAMDQhCMAAADA0IQjAAAAwNCEIwAAAMDQhCMAAADA0IQjAAAAwNCEIwAAAMDQhCMAAADA0IQjAAAAwNCEIwAAAMDQhCMAAADA0IQjAAAAwNCEIwAAAMDQhCMAAADA0IQjAAAAwNCEIwAAAMDQhCMAAADA0IQjAAAAwNCEIwAAAMDQhCMAAADA0IQjAAAAwNCEIwAAAMDQhCMAAADA0IQjAAAAwNCEIwAAAMDQhCMAAADA0IQjAAAAwNC2G45U1SlVdVVVfWiu9oKq+nRVXTDdHjk37/lVtamqPlZVj5irHzXVNlXV8+bqh1TV+6rq4qo6o6r2meq3nKY3TfMPXtaDBgAAAFixI3uOvCbJUavUX9rdh0+3s5Kkqg5LclyS+07r/HlV7VVVeyV5RZKjkxyW5Php2SR58dTr0CTXJnnqVH9qkmu7+zuSvHRaDgAAAGCpthuOdPe7k1yzg/2OSXJ6d3+5uz+RZFOSI6fbpu6+pLu/kuT0JMdUVSX50SRvnNY/Ncmxc71One6/McnDpuUBAAAAlmaRc448u6ounA67ucNUOyDJZXPLbJ5q26rfKclnu/v6reo36jXN/9y0/DeoqhOramNVbdyyZcsCDwkAAAAYzXrDkVcmuWeSw5NckeTPpvpqe3b0Oupr9frGYvfJ3X1Edx+xYcOGtcYNAAAAcCPrCke6+8ruvqG7/yvJqzI7bCaZ7flx0NyiBya5fI36Z5LsV1V7b1W/Ua9p/u2z44f3AAAAAOyQdYUjVXW3uclHJ1m5ks2ZSY6brjRzSJJDk7w/yXlJDp2uTLNPZidtPbO7O8k7kzx2Wv+EJG+Z63XCdP+xSf5pWh4AAABgafbe3gJV9fokD0myf1VtTnJSkodU1eGZHeZyaZKnJ0l3f7iq3pDk35Jcn+RZ3X3D1OfZSd6eZK8kp3T3h6dNPDfJ6VX1B0k+mOTVU/3VSV5bVZsy22PkuIUfLQAAAMBWthuOdPfxq5RfvUptZfkXJXnRKvWzkpy1Sv2SfP2wnPn6fyZ53PbGBwAAALCIRa5WAwAAAHCTJxwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGtt1wpKpOqaqrqupDc7U/qaqPVtWFVfXmqtpvqh9cVV+qqgum21/MrfOAqrqoqjZV1cuqqqb6Havq7Kq6ePp6h6le03Kbpu3cf/kPHwAAABjdjuw58pokR21VOzvJ/br7u5P8f0mePzfv4919+HR7xlz9lUlOTHLodFvp+bwk7+juQ5O8Y5pOkqPnlj1xWh8AAABgqbYbjnT3u5Ncs1XtH7v7+mny3CQHrtWjqu6WZN/ufm93d5LTkhw7zT4myanT/VO3qp/WM+cm2W/qAwAAALA0yzjnyM8nedvc9CFV9cGqeldVPXiqHZBk89wym6daktylu69IkunrnefWuWwb69xIVZ1YVRurauOWLVsWezQAAADAUBYKR6rqt5Ncn+RvptIVSe7R3d+b5NeTvK6q9k1Sq6ze22u/o+t098ndfUR3H7Fhw4YdGzwAAABAkr3Xu2JVnZDkJ5M8bDpUJt395SRfnu6fX1UfT3KvzPb6mD/05sAkl0/3r6yqu3X3FdNhM1dN9c1JDtrGOgAAAABLsa49R6rqqCTPTfKo7v7iXH1DVe013f/2zE6mesl0uMx1VfWg6So1T07ylmm1M5OcMN0/Yav6k6er1jwoyedWDr8BAAAAWJbt7jlSVa9P8pAk+1fV5iQnZXZ1mlsmOXu6Iu+505VpfjjJC6vq+iQ3JHlGd6+czPWZmV355laZnaNk5Twlf5TkDVX11CSfSvK4qX5Wkkcm2ZTki0messgDBQAAAFjNdsOR7j5+lfKrt7Hsm5K8aRvzNia53yr1q5M8bJV6J3nW9sYHAAAAsIhlXK0GAAAA4CZLOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADG27l/IFAMZx8PP+YeEel/7RTyxhJAAA3zz2HAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIa29+4eADCuu77zgoXW//eHHr6kkQAAACOz5wgAAAAwNOEIAAAAMDThCAAAADA04QgAAAAwNOEIAAAAMDRXq2Gnfdep37Vwj4tOuGgJIwEAAIDF2XMEAAAAGJpwBAAAABiacAQAAAAYmnAEAAAAGJpwBAAAABiacAQAAAAYmnAEAAAAGJpwBAAAABiacAQAAAAYmnAEAAAAGJpwBAAAABiacAQAAAAYmnAEAAAAGJpwBAAAABiacAQAAAAYmnAEAAAAGJpwBAAAABiacAQAAAAYmnAEAAAAGNoOhSNVdUpVXVVVH5qr3bGqzq6qi6evd5jqVVUvq6pNVXVhVd1/bp0TpuUvrqoT5uoPqKqLpnVeVlW11jYAAAAAlmVH9xx5TZKjtqo9L8k7uvvQJO+YppPk6CSHTrcTk7wymQUdSU5K8sAkRyY5aS7seOW07Mp6R21nGwAAAABLsUPhSHe/O8k1W5WPSXLqdP/UJMfO1U/rmXOT7FdVd0vyiCRnd/c13X1tkrOTHDXN27e739vdneS0rXqttg0AAACApVjknCN36e4rkmT6euepfkCSy+aW2zzV1qpvXqW+1jYAAAAAlmJXnJC1Vqn1Ouo7vsGqE6tqY1Vt3LJly86sCgAAAAxukXDkyumQmExfr5rqm5McNLfcgUku3079wFXqa23jRrr75O4+oruP2LBhwwIPCQAAABjNIuHImUlWrjhzQpK3zNWfPF215kFJPjcdEvP2JA+vqjtMJ2J9eJK3T/Ouq6oHTVepefJWvVbbBgAAAMBS7L0jC1XV65M8JMn+VbU5s6vO/FGSN1TVU5N8KsnjpsXPSvLIJJuSfDHJU5Kku6+pqt9Pct603Au7e+Ukr8/M7Io4t0rytumWNbYBAAAAsBQ7FI509/HbmPWwVZbtJM/aRp9TkpyySn1jkvutUr96tW0AAAAALMuuOCErAAAAwE2GcAQAAAAYmnAEAAAAGJpwBAAAABiacAQAAAAYmnAEAAAAGJpwBAAAABiacAQAAAAYmnAEAAAAGJpwBAAAABiacAQAAAAYmnAEAAAAGJpwBAAAABiacAQAAAAYmnAEAAAAGJpwBAAAABiacAQAAAAYmnAEAAAAGJpwBAAAABiacAQAAAAYmnAEAAAAGJpwBAAAABiacAQAAAAYmnAEAAAAGJpwBAAAABiacAQAAAAYmnAEAAAAGJpwBAAAABiacAQAAAAYmnAEAAAAGJpwBAAAABiacAQAAAAYmnAEAAAAGJpwBAAAABiacAQAAAAYmnAEAAAAGJpwBAAAABiacAQAAAAYmnAEAAAAGJpwBAAAABiacAQAAAAYmnAEAAAAGJpwBAAAABiacAQAAAAYmnAEAAAAGJpwBAAAABiacAQAAAAYmnAEAAAAGJpwBAAAABiacAQAAAAYmnAEAAAAGJpwBAAAABjausORqrp3VV0wd/t8Vf1qVb2gqj49V3/k3DrPr6pNVfWxqnrEXP2oqbapqp43Vz+kqt5XVRdX1RlVtc/6HyoAAADAN1p3ONLdH+vuw7v78CQPSPLFJG+eZr90ZV53n5UkVXVYkuOS3DfJUUn+vKr2qqq9krwiydFJDkty/LRskrx46nVokmuTPHW94wUAAABYzbIOq3lYko939yfXWOaYJKd395e7+xNJNiU5crpt6u5LuvsrSU5PckxVVZIfTfLGaf1Tkxy7pPECAAAAJFleOHJcktfPTT+7qi6sqlOq6g5T7YAkl80ts3mqbat+pySf7e7rt6oDAAAALM3C4ch0HpBHJfnbqfTKJPdMcniSK5L82cqiq6ze66ivNoYTq2pjVW3csmXLToweAAAAGN0y9hw5OskHuvvKJOnuK7v7hu7+rySvyuywmWS258dBc+sdmOTyNeqfSbJfVe29Vf0bdPfJ3X1Edx+xYcOGJTwkAAAAYBTLCEeOz9whNVV1t7l5j07yoen+mUmOq6pbVtUhSQ5N8v4k5yU5dLoyzT6ZHaJzZnd3kncmeey0/glJ3rKE8QIAAAB8zd7bX2TbqurWSX48ydPnyn9cVYdndgjMpSvzuvvDVfWGJP+W5Pokz+ruG6Y+z07y9iR7JTmluz889XpuktOr6g+SfDDJqxcZLwAAAMDWFgpHuvuLmZ04db72pDWWf1GSF61SPyvJWavUL8nXD8sBAAAAWLplXa0GAAAA4CZJOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADE04AgAAAAxt4XCkqi6tqouq6oKq2jjV7lhVZ1fVxdPXO0z1qqqXVdWmqrqwqu4/1+eEafmLq+qEufoDpv6bpnVr0TEDAAAArFjWniMP7e7Du/uIafp5Sd7R3Ycmecc0nSRHJzl0up2Y5JXJLExJclKSByY5MslJK4HKtMyJc+sdtaQxAwAAAOyyw2qOSXLqdP/UJMfO1U/rmXOT7FdVd0vyiCRnd/c13X1tkrOTHDXN27e739vdneS0uV4AAAAAC1tGONJJ/rGqzq+qE6faXbr7iiSZvt55qh+Q5LK5dTdPtbXqm1ep30hVnVhVG6tq45YtW5bwkAAAAIBR7L2EHj/Y3ZdX1Z2TnF1VH11j2dXOF9LrqN+40H1ykpOT5IgjjviG+QAAAADbsvCeI919+fT1qiRvzuycIVdOh8Rk+nrVtPjmJAfNrX5gksu3Uz9wlToAAADAUiwUjlTVbarqdiv3kzw8yYeSnJlk5YozJyR5y3T/zCRPnq5a86Akn5sOu3l7kodX1R2mE7E+PMnbp3nXVdWDpqvUPHmuFwAAAMDCFj2s5i5J3jxdXXfvJK/r7v9dVecleUNVPTXJp5I8blr+rCSPTLIpyReTPCVJuvuaqvr9JOdNy72wu6+Z7j8zyWuS3CrJ26YbAAAAwFIsFI509yVJvmeV+tVJHrZKvZM8axu9Tklyyir1jUnut8g4AQAAALZlV13KFwAAAOAmQTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMTTgCAAAADE04AgAAAAxNOAIAAAAMbd3hSFUdVFXvrKqPVNWHq+pXpvoLqurTVXXBdHvk3DrPr6pNVfWENcFLAAAgAElEQVSxqnrEXP2oqbapqp43Vz+kqt5XVRdX1RlVtc96xwsAAACwmkX2HLk+yW90932SPCjJs6rqsGneS7v78Ol2VpJM845Lct8kRyX586raq6r2SvKKJEcnOSzJ8XN9Xjz1OjTJtUmeusB4AQAAAL7BusOR7r6iuz8w3b8uyUeSHLDGKsckOb27v9zdn0iyKcmR021Td1/S3V9JcnqSY6qqkvxokjdO65+a5Nj1jhcAAABgNUs550hVHZzke5O8byo9u6ourKpTquoOU+2AJJfNrbZ5qm2rfqckn+3u67eqr7b9E6tqY1Vt3LJlyxIeEQAAADCKhcORqrptkjcl+dXu/nySVya5Z5LDk1yR5M9WFl1l9V5H/RuL3Sd39xHdfcSGDRt28hEAAAAAI9t7kZWr6haZBSN/091/lyTdfeXc/Fclees0uTnJQXOrH5jk8un+avXPJNmvqvae9h6ZXx4AAABgKRa5Wk0leXWSj3T3S+bqd5tb7NFJPjTdPzPJcVV1y6o6JMmhSd6f5Lwkh05Xptkns5O2ntndneSdSR47rX9Ckresd7wAAAAAq1lkz5EfTPKkJBdV1QVT7bcyu9rM4ZkdAnNpkqcnSXd/uKrekOTfMrvSzbO6+4YkqapnJ3l7kr2SnNLdH576PTfJ6VX1B0k+mFkYAwAAALA06w5Huvufs/p5Qc5aY50XJXnRKvWzVluvuy/J7Go2AAAAALvEUq5WAwAAAHBTJRwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIYmHAEAAACGJhwBAAAAhiYcAQAAAIa2x4cjVXVUVX2sqjZV1fN293gAAACAm5c9Ohypqr2SvCLJ0UkOS3J8VR22e0cFAAAA3Jzs0eFIkiOTbOruS7r7K0lOT3LMbh4TAAAAcDOyp4cjByS5bG5681QDAAAAWIrq7t09hm2qqscleUR3/8I0/aQkR3b3L2213IlJTpwm753kY9/UgX7z7J/kM3rqqaeeeu72njeFMeqpp5566rnn9LwpjFFPPW+Ovq27N+zIgnvv6pEsaHOSg+amD0xy+dYLdffJSU7+Zg1qd6mqjd19hJ566qmnnru3501hjHrqqaeeeu45PW8KY9RTz2X2vCna0w+rOS/JoVV1SFXtk+S4JGfu5jEBAAAANyN79J4j3X19VT07yduT7JXklO7+8G4eFgAAAHAzskeHI0nS3WclOWt3j2MPsSsOHdJTTz311HP399NTTz311PPm3fOmMEY99RzaHn1CVgAAAIBdbU8/5wgAAADALiUcAQAAAIYmHNlD1cwDq+oxVfXo6X7t7nEBwCim/8UH7e5xACRJVd2mqr5lun+vqnpUVd1iSb33rarbLanX43akBnsa4cgeqKoenuTiJC9I8sgkP5Hk95JcPM1bb9/HrHVbyuBvAqrqh6rqKdP9DVV1yBJ63qqq7r346L7W79FVdfu56f2q6tgFe/7gjtQW6H+HqvruZfVblul7933T7fbbX2O7/b6lqn5gGWNbpfe3VdWPTfdvtawXKVO/b6mqfZfVb1erqj/d3WPYWlX9yo7UdrLnUp+fc333qqq7V9U9Vm5L6LlvVd1x5baEfj9YVbeZ7v9sVb2kqr5t0b7L1LMTs/397h7HzU1V7VNV311V31VV++zu8dzUzP8ernbb3eP7ZqmqQ6rqW+emb1VVB+++Ed3Y9Hf415bc9t1JvrWqDkjyjiRPSfKaRRpW1RFVdVGSC5N8qKr+taoesOA4n7+DtZ1SVS+sqh9f+d+xDLsiyKmqb19k/W30/NZVavsv2HPpr2tu6pyQdQ9UVR9JcnR3X7pV/ZAkZ3X3fdbZ96+nu3dO8gNJ/mmafmiSc7p7pwOSqrouyTafRN297jdjVXWvJK9Mcpfuvt/0xvtR3f0HC/Q8KckRSe7d3feqqrsn+dvuXndIUFU/leRPk+zT3YdU1eFJXtjdj1qg5wXdffhWtQ929/cu0PMD3X3/7dV2suc5SR6V2ZWvLkiyJcm7uvvXF+i5lJ/79IL75CTHJvlEkkrybUnenOQZ3f2VBcb43u7+/vWuv42eT0tyYpI7dvc9q+rQJH/R3Q9boOfrkjwjyQ1Jzk9y+yQv6e4/WaDnQ5P8UpKVMPAjSV7e3eest+c2tvOp7l73G/qq+o4kr0hy1+7+nul59BPd/YcL9Fztd2hdv5e7+Pn5S0lOSnJlkv+ayt3d6wovq+rpSV6Y5Ev5+t/77u6FXvxV1YVJvifJdyd5bZJXJ3lMd//Ign1/IMnBmbsiX3eftkC/VyR5TXeft8i4tup56yS/keQe3f206ff93t391gV6Lu1/5vY+MOnuv1vnMFNVP5HkL5J8PLPn/SFJnt7db1tnv13xWmG11zafS7IxyW909yU72e+dq/Rb0Tv7d76qPjH1W22P4nX/bu6Kn/tW38uV8a6MvRd8nbgxyQ+s/L2c/q7+3+7+vj1lnFV1Tnc/ZD3rbqPfB7r7/tPf+Vt19x8v4fXhhUme1d3vmaZ/KMmfr+d/RlUdndkHuz+T5Iy5WfsmOay7j1zvOKf+P5/kh5J8f5Lrkrwnybu7+y0L9NwVr4/fneSAJOdlFmi9p7svWm+/qedFSZ7W3edO0z+d5A+7+14L9Fza65qbiz3+Ur6D2jvJ5lXqn06y7l3nuntlb4m3ZvYH6opp+m6ZvYlYT8/bTT1emOTfM3uBW0memGTRT71fleQ3k/zltK0Lpzd7637Bk+TRSb43yQemnpcv4dP5FyQ5Msk5U88LlvDJxWp7da3r97Wqvj+zMGxDVc2HFvsm2Ws9Pefcvrs/X1W/kOSvu/uk6Z/sIpb1c/9vmf2+HNTd1yXJ9LN+RZLfmW7r9Y/TP6W/6+UlzM/K7Hn0viTp7our6s4L9jxs+vk8MbNLoj83s5BkXeHI9Kbm5Zm9Uf69zH7X75/klKp69nTp9WVZ9DDCv0ryW/n637aLkrw+yU6HI1V1fJInJDmkqs6cm3W7JFevc3y78vn5K5m90V7v2Lb2nCT37e7PLKnfiuu7u6vqmCT/b3e/uqpOWKRhVb02yT0zC2tvmMqdZN3hSGYfIDyjqi5N8oV8/c3SInvK/XVmv4srIevmJH+bZN3hSJb7P/Onpq+rfpiSZN3hSJI/S/LQ7t6UJFV1zyT/kGRd4Uh2zWuFlyS5PMnrMvt5H5fkrkk+luSUJA/ZyX7PWaX2oCT/T5KrdnZw3b3wHq/b8MbMfncumKbn/w531vFzX3mduIvsPR8kd/dX1rsn0i4c5/+tqpdnFhR8YW57H1hnv5pe1z0xyVOn2qLv565bCUamsf3zFBatx+WZhYiPyuxv3Ne2kWThvWi6+5TMXnPcNbMA5jmZfbC00z+/uSDngKp62dysfZNcv+A4f3h6Ln5fZn8v/qGqbtvdi+zZ9YTMHvs5Se6e5E5JfnQ9jXbR65qbBeHInumUJOdV1elJLptqB2X2z/nVS+h/8EowMrkyybpTx8kjuvuBc9OvrKr3JfnjBXreurvfXzc+1cpCf6ySfGV6Md7J7NjNBfslsxf4n6vlnhJmY1W9JLM3Sp3ZJ/Xnr73KNu2T5LaZ/b7P//P4fJLHLjLIJHtP4drPJPntBXutWNbP/TFJjuzuL64Uuvu6qvrFJOdmsTefv57kNkluqKovZQmfgCX58vTCLklSVXtnjb2ydtAtanYs8rGZ7d3x1ZXn/jr9ZpJju/tf52oXTJ/e/c/MApgdVtve/buyeDhym+7+l5Xv5/R7/9V19vqXJFck2T+zN3YrrstsN+T12JXPz8sy+5R7WT6e5IvbXWrnXVdVz0/ypCQPrqq9ssAHAJMjMgsFl7lb7NFL7LXint39+OkFarr7S7X4P5Gl/c/cFR+mzLlqJRiZXJJ1BARzdsVrhaO2ek1zclWd290vrKrf2tlm3f21/99V9SOZ/X7fMrO9xNYbCmV6zjwxySHd/fs1O3zurt39/nW2/Okkj89sb663JHn9Vj+rhUx7JBza3X9ds8MBbtfdn1ig5ZaqelR3nzn1PybJwiHukse5chjuC+dqnXW+qU3yq5kdnvLm7v5wzQ7feOc6e614f1X9ZWYfIHRmz4Fzqur+yc4FOdPrg3+tqr/p7kV/D79BVf1VksMye+/ynsxex643aNplQc70HHrwdNsvs+D7PWuutB3dfVFVvSizD6KvS/LD3b3ah+k7Yle8rrlZEI7sgbr7D6vq75Mck9mnSpXZp0pP7O5/W8Imzqmqt+frfwSPy+J/WG+YPp0+fep5fL7+qd16fWb6RGklyHhsZr/Ii3jD9A9gv5odxvDzmX3qtIgPVdUTkuxVs12jf/n/b++8wySrqvX9fjMgGcQLEq5kuSAiKkEyCGLgXkHCCAwoBhRRVLiIXjEhIAL+REVQBAVEJAgiikrOmYGRIQlGUDAhCAJK5vv9sfaZrq6p7pk6+5zpanq/zzNPT53uWrW769Q5e6+91vcRF50cPkJMnn5AvP8XEZUFfWP7SuBKSd+1/QdJC9n+12yfOGccDFxIlLHelG7Sv8mM2dT7/nznwrPC9uOZCYK2dpiuTJPuBSS9EfgQ8NPMmMcB9wK3Alcp9BwezYi3dFdiBJi5U7tUjXjTGbk0vG4io+IhRStidR5tR1S39Y3tPwB/YGiXvwlaOz+JxeYVkn4OPNUR+ys14x0AXJcS3p3xPpo1ypiA7wq81/Zf08KudstX4g5ihz/3XjGTdN3sXCwtSSScc3ha0gIMnZ+r0PG3rUkb98w2NlPulHQecCYx1rcTG0I7QK3WjTZ+7+cl7URUUsDwjYRan09Jbybu608Ch9rOnXcBfJNondsSOIRY2JxN7Fb3je1zgHPSxtHbgCMl/Qfw6TSXqI062pqJyqkXAd8HcrTP9gJOVbS+QSSG3zlI47S9Rc54esSr5nSLSlrE0eKVey2u2rgP7Dq+EX0mciSdaXsn4JZe97LMijuIaonJwCPAP4AH6yZhOhI5p9nOnXN0cyWReDmMkESo3SpbIekEojJyLeI6/FNJx9juO2Hd0rzmBUHRHJmgpEnIpunhVemGmBNvReAo4uZh4FpgX3fppvQZc2WiJ38j4GGiL/8dOTFT3DcCbyIWZBfavjgz3oJE1cTMmMAhtp/Mids0qQzzBGBh28tLejXR5/2hMR7aMEZ433dLF/J+4txKlDL2WnhfbvvVGWPs3q1bDlgmY7cOhfr8Hgw/j77T8A44kuapO5GQNN12T5G20b43Fig0R44nStf/TiyWdsm8Ju0AHEG0GlTVLbUqhlo+P7snuADYPqhmvGnANURrUqVhgu2Taw1weOyliXYyAzfZrpXA6oh3OTHRn8bwRE6OBlQbWlVvJFqr1iCS3xsD73aGdk9T186umMcAqzJ8M+W3tj+SEfOkUb5t2+/tM17jc4UU8yiGFg7XEzvJfwLWsX1Nn/FuApYkkn/Xd3+/bouFhvQnZmoESLo15/qRYkwG3kK832sCn7R9YWbMGaS25o6x3tbAYhlJCxNrmrqtIJ2xGh2npM/1Om774F7H5yDeukTSZhHi/vEIkWCuW13cKJKWsf0XjSCunXM96nqdVwBvJj6Xk22/LCPWxkSL/ApE4UB1b6+tqyXpxcR1fTMiWfk8cL3t2lWhCnHfr1XzQoWI+1ds7zH6M0eN2di85oVCSY4UBp60gzGpoZveQsCTtp9TuMusBpzfQsa4FpK+ZntfST+lx+5U5gT/RmL369yOG/4dttfMiFlNIDcgxns9kRTLKZOtYme97wp9gOfpvfjMvekdm2JvafsVkhYHLnINEbg20XCNmYp/AtNtz+jxvdnFe4QQFpvlW8AmthfvN+YIr7MasL/t9zcQazHiXvdIA7F+C2xj+64GYt1LS+dn00i6znbjDk0KraLPEXoWAjYnxKxPzIjZU8w1Z9e7rUVd2pHfgPjdb3CGpktKrk6xfWaT98wUu9HNlLZo+vduEoVGwGiCrHV1A24kkkI3pSTJksS9qJaYokJweyqRsLwEOMP2zXVi9Yg9zfbrOhI6CxGLxdqfo3R9P5BYgELs1h9su3ZbYdPjlPSxjofzA28F7uo3CdgRrzHx1I6YLwZ2Z1Yh69yKlMaR9FbierQZsDgx77w6875xN5FkmU5H1bszdbtSAmfzNN6NgD86U3C8aZqc17xQKG01EwhJ19jeRLOqsA/LEkpa3PbDfcZuQy1+2MVaQ9oBORfrq4je9sWJG//NRGn3bhnjXJcQflyR4TeVOjeqU9LXVmxMbd+n4X3Zua1PpxH959unx7sQrVXrj/iM2ZAWDAcSauSWdA0x2enrJmV7xbpjmAPWr3br0ms9rEw7ynTDP4RZdy5ysvfrpn9Ve87/EMrpe0k6y3a/mkBvG+V7fZ+z6TrxZUJY7MeEbsk3ifPnyFGeOiexFydK2DvPoy/0e23r4m9NTSDaPD/TAukTwCuJyXj1mnV73C+XtCdxHnVWY/wjZ5yEhs1rq892+uxfR+hu1cL2lYoWrypROc12jp4FtKNVBeFkMJn4vG8mqbYLjO3nJX0YONPNtU1WsX9EngDrMJqeL7QxV5D0JULQ9QngAsJVaV/b368Tzw26lXTxdcLh6qUKLYIpREVSXS4l9AauITRRdpe0e/XNzPlXG23NJxKtdDulx+8kqir6dl/soNFx2h52L1PY1J87wo/PCU2Kp1acR2hdDasOzEHSBsQ9/RVEa9Jk4F8NVCRsTczlj7L958xYFf90hvZPLyT9jhBwvoZw53qPa7bWKLUqKdxqeq3fchL1jc1rXiiUypHCLKiGfZWkK0lq8Q1WJVxHj4u1M0q51Y4F2q+I3717nLVKB1M568m231F3TCPE/SGhwH8MsVv5UWBd27tkxLzRw0XrUIjWbZAR82LixldNQncDXm97qz7jjHoOu75SfOO7dSnmb4kJ3e1u6MKs0Bba0fbj6fHCRA/99kT1yBpNvE7G+G4kFknXEyXcnyASbp91Zlta+t1vYOg82hXY2PabasSqJtqbE3oWP2Z4kqCOvWWb5+dFhF7R/kRP/ruAv9v+v5rxelWCZVe3SLqUsK3vtOE8r9/PelfMnYjWhSuIieOmwMdt/3C0580m5v5Ea8kbif7x9wKn2T46I+aJRN/4nQy3W661m5xifpZYzHe7YvSdxJrTzZSa42x0vtDSXGGG7ddI2p4QtP5fMtvdUtz5CT2pTYi/69WEZXvt652k1YE3EO/NpTkLHc3GLSrnb5riN93WPMP2a2Z3rEbcRsfZFXtxImm7as3nfxVYkOHiqQ8TWjO17h115v1zEPNmYsPsLGKTZnfg5bazxfubSoB33Id3IpI3P2L4vT3nPjzJdlOJptZalSQdRUPzmhcKJTky4Eja0/bxIz1u6TX7ThZIusn2ehre95p1g2rpYn0LMTH5KrCHQ+n7dtuvyoh5je1NGhskMxd229TNMo8QcwmiBWYrmCnyuk9O2aCkw4l+10qId2dit+kbUHtSPot2haSbba/bZ5zRxO6csYuOQnx4Z2Ad4Luk3TrbZ2XEvBx4Q1M30xTzLuDVHYvP+YAZjlagOp/z7l2LYfS7e9F9jZB0HyEAmVvRNNJ5VEsXRQ1rJKSYbZ6f022vo47WD0lXekDKeTXU7vUa4FWEK4aJyqRptvfKiH0r8MZqspwSl5c0sKhtelH3y6aTk20lsZqm6flCS3OFO22/UtK3gbNtX6BmtDzOJERTq6TtVGBx22/vM87PiUTyj91wpVDHayxMnD9Z8RW6NafZzhWq7xX7eiL5eU16vDHwZdu1RSYVQt5/qRJWCuHkpVxTw6brvjmZ0J452PYxNeM1fu9QaFk8TjiqNFIdWM3buu5D2S2akt5OVJxeQWYCvOX78MuIyplKi/EaYs5d111mTl7z+n7P/RHmN1mJ+vFOaasZfLr70Rv1ix2BOhmzNtTiT0kljY1drGnHAu1AhbXYpTSXdb0XuFbhPd65A1jXbQJHP3vt9qER2Dl9/UDX8fcS50KdSfnlknYhnAwgEg8/7zeIk0K8pPm7d+XS7l1tbJ8qaTqxWwdhb5tblvgJ4Ly0q9qEwwjE5PkGST9Jj7cBTle0BdRxvnprxlh6Mb+k1zJ0XXscWEuKuvicXRtCzX9KNWFK1R+1ymadbE2bpM3zkyGnn79I+h/CrjBHrG73Xsdtf69myMrt6XfpX8VPevxsv0zq2kV8CJiUGzQlQxrbPQaul7SGm3GgA8D2Sk3F0sg229Vr5dyHm54vtDFX+KlCh+AJ4EMpydaEyPpqXQmWy1NCr1+OJ3blvybpMqKKoClHjA8S86SF0uPHgSNsf7NmyN8QrjfLEFVNp7uG5tUI7AV8T6E9AlFBMWr1yxxwFkP2uxDtx2dR0wGI4ffNZ4k2htoWt27Y/SbxNFFx92mG1gB153AV/07VgDMUbWp/IZ1TmXwGWK87Ac6Qs9Qc09LfsuIkYg5WJT7fkY69scXX7Hvu0Mb8ZrxTKkcKs1BnF0btqOTvDRxKVCbMvFgP4C7Y94HVabY8upfbhF1T3TzFbLSHug3SoriqGqi+TmYoQWT3Wc7d63xuYqcxlWNWpdHXZi7kq1aIx5m1NLyWw0hH3HWJnQsB1zhTXE/R9nWhM1ofOmJdQQsihSn2w8BiRKLARM9zJdJn26Mu/kaI+fUeh/8J3Gy71sK+jfNToV9zNbAcsXO1KHCQ7Vp97pI620fmJ5KCv7A9ZYSnjBmS/h/RrnJ6OrQzcJtrtBSp3daSzQgNl78SC/omeseRtCbhgNOpNdN3EitVoYxks511H256vtDWXEHR/vCoQ8B9QWBR57spfZdoo7khPV4feJdrusalqoZtiUTJhoR2xOl1K5skfYZ4Xz7ssIit3q+jgBudpyO3QhrnLsT5eToh+PrrujE7Yi8KYPtRSTvaPjsjVq9WndpVQ5JeRcwRIYRY76g7thRvKeCLwLK2t5a0BrCh7RMyYv6O0FKrLQrdI+YKhPX3i4i2tMUI4djfZsYdVu2tEKO+1XkV4I2K16eYrbR8zeY166zfTqK3AcSErRwpyZEBZIQP6Uwyd5Ln5PXrlNuvZPsedajFV8cyxtHYxVrtusBkteWMEPPt7mrR6HWsz5iN91CnCeN+wPK295S0KrEz9rOMmI2USCssQv+TKF/elaFJ/qLE5HT1kZ47B7E/R+wGnJ3ibkdYe+ZMHPtuHeoj9ksZvlj6Y2a8c4F3OsMRoG1SEmdEXKN1R9LxxCS3+hzuSCRFlwN+b3vfPmK1dn62TdqlPaXudXOk63BFzvU4xd+RoYRgbXcVSStXC8SmUWgM7UdDWlUp5oGEPfQaxCJ5ayIhOjBJLLXgqtPSwu7twAVpLvMZYG1C0Dk3CX4X4ZJXXYOXB+4izoGs5JhC2PZkYC3bo17/RonxK6IVs7uabQFiAfpfdcfXFe+1hJhq7bGOEvuPtpfPeP7FwNFVMlnS24CP2n7D6M+cJc5iRDXccoTIrYg2wj8Cb7P9aM3xnU9UIHza9qslzQPckpkcOJewuv933RgdsbYDrnO+EPZI8RtLgHfEPI3e4vWrE3O7fsXrkXQJ0XZdjXMqIcra13nU52vWSY7s2PFwfkKT7s8eQKeiuUVpqxlMqpLj1Ygyvmq3bxt622g2TZ0P7tnA2h7em/pDQpOhLncC2RfqRJsuMDc0XR5NlLR2J0J6HeuHedPX/yZ2lv4hZXdpnURYn1UlqPcTY6ydHCHKzdezfVPm2N4MvJtoJ+hMKD5KuAvlMJVw2ah6kg8HfkFU5tTlEklvsn1R5thmImlbwvVlWeABYiJ+N+FiksOTwO1pEtnZ9tXXzVRDQqczQwAPEroouer7pxKT74vd3C7Aywn75mcBFJbOFxFlsrf3GWuk8/MxMs9PteAe1sW/CYHSurTixlWRdo1r7xx3cBawjqRLW5jQ/rFuJc8oTCEqAm+x/Z60w/yd3KCpgmJVhidYa81F3I6rTpNzhYrP2j5LYZP6ZuKcPZYMJ7bEW7JH1kF6j3ciqjGWIc7ZrDL57sRIOvaEpCw9LEnzEr//LsQ880ogqzJypJfKfP5ewKkKrRSIeU3P1sLZcAjhiLilk5ZYSg4eTlQ6faTm+JZIycUDAGw/KylXp+s5ov3lcoa3ptVZIL8D+IakfwPXEg5k19q+M3OM1Zg+3pUAP75uAryD/yDWMJV4/YHEGmYzYo7bd3KEaC8/htA4NPF3aLsao+9zv7vKStLpRJvShKUkRwYQpzL6VGa/drVIkPR58hbHc/r6c9ynq1BJfyWwWNdCZ1Fq9L510djF2vb09N+bgSc6blSTCQHRHDYB3pXKkLPKoyVtTSQv/rOrhH9Rolc1hzZ6qFexvbOkqTBzApU7MdmCsJq9l1h41/p7OlT1T84tsR2Be4nzu/r7zcdw7YQ67A18QtJTRCtIE1a+hxDORJfYfq2kLYjETi4/p4YOTA+26XHsJYTuyB62L8uI/V1gD2KS9gPgu7nlvESlx0IMtecsRJQ2P5fetzmm5fPz2yQ3kPRat6WdsbpWqZ2VHpOIyoQzR37G6Ni+cg5f92zbO87+J1trgZmUJsn/1R+WTOAAACAASURBVKuiM7OK8+70nnTbI+doVT2Rkg/PKtoMHiBPMwBJ7wP2IZJ4M4jryfVA7ZY34GKFA1C2q06iyYVdZ0yIHeRjbf8kzcGysP2HlHBZ1fZJCqH0RfqtslVorEwlNtF+BHzC9rW54wPul/QG25d2vd6W1NSFUYgZTyW0N24kBNz3bDA51k1WMtz274ANFIK0ykjUb0VUxnRWhj0v6VP0n0zv5F8K2/NKs2cDhu5Jdflx+pdNVakmaUVi42wj4AOSlicc/v67gddoKgFesTyhu1LxDLBCmtP2dW+vSBW6WVWQ3Shsy6uNiV/3qN59ZwMvsyrx95iwlOTIYNP9YX0aWHFshjIiqxE3vBczfKHzGPD+zNiNXaw7uJS4YT2eHi9A7PzmqGc3uRP0ZyKBsy2Rra54jGiDqY3tT0o6gqEe6n8T7hBATGDcf5/y06nctrpJr0LH5LQmW2c+v5trJZ1AA/25Cu0FE7/jnalywkTlwDU5g7S9yOx/qm+esf2QpEkKW7nL0zmQhe2T0/u+vO1fZcTpucOp6FU+k4xdWtsXABekXe/dCOHDe4jEwemuJ4j3JWIRdgWx6N4M+GJqD6i707KmpFkqeZyhLwQsaHtaV54yJ7naWenxLPAHt6i438EcL+ydHMMa/hztQrTMzcNQRWdTLEBcRzqtpU0sdOtyc5o8f5u4fzwOTMuIB5EYWQ+4wfYWaUMkd7e/2j3du+NYjvhjG3OFP0k6jpgvHKFw+soW9k3JtnWJudNJhB7D94ld8H7YiKhAuMQNOpwBHwV+Iuka4hwy8f5vTMd8oU8+RQhT7p+RABuGRnZOE7BUZuwvAl+y/Uh6vDjwMduf6TPU073uM6nSI2eetB9RUb6KpGsJ95u+3I56jOlkhXhq1Tb1K9vPjPacOYh5r0JcfIH0r/p/LXokvmd+i/yNpMbE6zvmiT2pk7RN783xxP3oHuJ3XkHSOcBeTkLMrqFn0/F3Vfr6V6B2i9ILgaI5MsBI+jRRLnkOccJuT5SifnFMB9YDSRvavn6sxzE71KBAkqRFHeJfPYUdcyYBqfx0HjIXn32+Zl+9iqlC5J3E7vwaRJJpY+Ddtq9oZZA1UIP9uZJGVcFP1QD9xlzd9t0KgddeMWv3uCt6XrcDDgOWIHaS17Xd70S8O+42xIL5RbZXkvQawpqwsV2Sfs/HEWIsTuh57E6065xGVHqt6pqCsgrHhdcRE4lptv+cOcaPdTycn0g23+U8QefzgQ8TvdJrK9xA9rDddOKxVeqcAylBe7/tpyS9nuhN/1610Kk5jq1t13I6GivSru2itm/LjFPZ7s4gdD2eqnvP7OM16yTqmx7DgsTGx+22f5M+969yZttj+ju+lhA0rmyMZ1qd1ojX6/PxTyKBWSshmha0uxJVwSLalk51j3abPuPu0b0pIelw25+sEWuF0b7vPO2eWXT3al6L7iYqZnq5Tn7f9isyxjgPkWATDSQy0rXyZKIqVoROyrtco30uVcZsSCRtfgXckP7d5hpaX3MLSesQ84Ms8fqW5okHA6sQiZCqm2AR4BvEZ/2zdcZa6E2pHBlgbB+aJrmbpkPvsX3LWI5pFB6SdCkN9LhLOtP2TiPtDNSdRCT+JWntasGZLoZP1Ix1GrGQqXZXOm+AuRZobyEtPoFWFp896KsdxrYl7UPsfG6Qnr+PGxTFa4jG+nPr3NTmgP2APQltkFlekrzy9VuJXvz/JaonFgMWzohX8XkiQXAFgO0Zkpq0EV2NzAokSWcS4nenATt2VDqcKqmv62iPBNZ96evSkpbOSWDZHva+S/oyQzpTddmb2GVaXdKfiJ2md9QNpmiZPAJ4KfE5b2Knri3OBtaV9HLgBOJveRrRrtgXkt7hcPNaQ9IsCxlntNWkBegexAK0U8sjJyn2PcKl6Grbd9eN08X9qRrlx0Q7zMNEhWObHMEc2CbPZq5gZ4iN2/63pAeIxdJviIqp39SN18HT6d5ZVVvmWpt+kxCLrQQ/10z//w9Je9VJ5qQkyImdxyRNlrSb7VMzxjpF0pNVDEnfpGZbc07yYw6YLGk+208BlRhtnXH+heF6Up1kuR6lxNedEMlESZ+wnWMReyTwpmozTqFbdTr1dAN3J6rWfkbobNzYo/2jb0baiKyosyHZtcl5T/o38/XqxGxpnrgD8Dp3COY6xKI/RCSespIjCn26zdLDK5xhqvBCoCRHBp8FiTaIkyQtqUwHmBZpssd9n/T1raP+VD32Ac6SVE3uliGUruuwN4DtxhaFHXyeWRefK7bwOp3UKSO7AVjZdhP6E23ReH+uwir1EGAF4jqas1g8kXjyFjljGoEtUsn188SuEJKydpITz9r+p4a3bfR9/qi3a8lLiM9lrcW8pA0cNpnfYQQx1u5dwTmgzQRWNwuSqRPhcFjZSg25gRDtRNvYviszTr/U0S96PiVAtwe+ZvvofpNhHVQL114Jxdyy21MIceQ3AwcTycvcv+93icX80Qr71RmEW89RdQPa3j799/MKTY/FCCv4NpnT972aK9xFzD86n19HQHEowKztL/NSr/2lmzMV7TovVuiGvJc80dx7iaqwaqG8BvG3OIRo0eorOaLQqtmb0Ff6CdEuuHeKOYMQuq7LDsC5CmHXrYF/uL6FcWeLRXW+zGwNyEzcfh+4VEMWp+8F+rbDbvqertB9+RYhsP5jws73e8TvfGhm+Hk7q5Rt/zpVMPeN7dVTsmEjwj3rkwr9llsJF5uTao6x10bkzJel3n2ze5Ozomox6TvmCPOamdTc5HzePZyEbD9eJVrrojAUWI+hz/Y+kja2fUBO3PFMaasZYDpvzrb/S9KyRJl07s25cTpKb2eWI9YtvZV0ke03zf4n+447iahwuImhcsS765YjNlH2P0rsG22v3/X3rF16O4evWads9JdEj+ofyBBPbZO02380saN2B1HqOSWn3Fxhw7kDUXKde2Nq/DyS9EHgQ0QZZqcI6SKEanztKoIU/wRCv+eThJ3tR4nJ1V59xtm865CBh4DfOPXQ1hhba5/LNuja9Z5MnJ8H2z5m5GeNGKsVG3hJ17Zx35G0T/fCvfOYarg3SboR+BrwaSKhc4+kO2yvmTHOjd0ldtnrWJ8xb3GIJN9me620ELnQdlaiTSEyvh5J2JoQae3bFlrSz4mFw4/dnnDmSK/db4vnLD+fe79Uw+0vXbHfSFRcinjPa7cQ9ZpnVcfqzMEUmgsPE6K7bwAWJypY97E9o+YYO3f8FyEW9tcCn4O8FuS2kPQWQm9GwEW2L8yINT9xP96EuNZfTdi199WmlJK8/0u8N1sTiZHP5iQ/O2KfmMZWOTu+A5jsEXTB+og7D1F9shnwAWAlN2zdPGj0mNcMw3MoSN4V81Yi0dQrMXR5TpVc2jB7jYcbVdwySPP4uU2pHBlstifdnAFs/1nRYzaIPKjo9a5256dQU9mcWBw0jkMl/EjbGxKL5FyyfXBH4Q5JuxLlnasSi8/rWnw9iB2ofhkPGgarEONcjljIr0/+te8+4I7cxEiLnAacT2iNdPZzP9bQRPQjxOLzKaL09kJip7IvOicJih7yVW3fKWkBSYs0UPHQKAodgv0ILaA902dztcwS1M4KuWeBv7mmVgDNi4ZW3Kxw/PkxzTmrALwL6J7Yv7s6VqclgLAx3Qs4NCVGViJ2gnM4mmhdmN2xfqiS8o9IWpMos18xIx6K1taFiMXT1cB6th+oGe54QpD2a5IuIz7n59VNWrZBRxJ45a6KuEWIxXcOTbe/zCQlQy5OcXPbVX6lsBQ/Iz3eGfi1QkC2zsbPyk56XJK+Q2g1LZ95Le7c8a++/k/6l9uCjBpw/+nGSdA7xd9Y0jds7z2bp43E9whR/aPT46lEEqJfEVV7SM/tx5L+3kRiJPFBokLoo8T7cyVhXd03qUVjI6LK6pVE+8/1wMdoeB6b1h27AFPrJMA1gtZbhWu0zI6U/JC0HDHWvpMjRMXedEaumsnlxUA1N1ysgXjjmlI5MsBImmb7ddWuSLo5Xz+I2bxUwns8cUF8mNTjbvveGrF+D+w/0vdzJuSSDiL6cX/UwI7/AwxNSGbBGTaCaRH2aYacDC4EDnHqge0z1g6jfb+BBc5A07EzuwlRhnok8CnbtZ1QJK1HJAOuZPhise+deUmPACOKntUswZxrpF2GhWw/mhHj/UTbyktsr5KSDt+y/YYasVr7e6YEwXRgd4e20gLENTlLnFLSqxnSlroqp6qpDVJ5eTd2TX0MhfX3rsRO6tUd31oEeM71xXInAyfnVkZ1xNuQuKftC3y141uLAttn7ta9j9BHWYto21gY+Jztb2XE/CqxS/sUkRy4ijg/6+pqVXoL2xKT+g2B8wi3p9YEUyX9yPao9630c4sRlQ2NJ4EVVsOrEk5khxGtFafZPnrUJ44cr7Nd5VwiOTKzXcV2LSeY9P5UVQkiXNO+SdjML2j78VGe3ivesCqcQa/Ea6vCWqHzNpVINt1DzBnrvve3dl8reh2bgzjdc+Mvdz6uM5eTtCSwpO1fdh1fk0jU/71GzB8RSZBrgelNJ1QV4sg7E/eQtYjP549s922PrGgVHAk3UMm3BJEEm0p89s+xPeL6ZixI9+PDgcthpgvfAbZHXN+80CnJkQGm6Zvz3EAN9LhLeojode2ZIa07IU+xHyN21p4jhFhr96dK+gOpLLQXzhBlkrQukRxZkaEqB9dJjHUsbF5KTPQvS4+3IISXZjsJHc90lK8fRrTBnKYeavR9xryIEBy7ndDzAMB23xaXkn4DvG+k79cpwWwbhZ7QXsTnaDqx0/AV2/+vZrwZhMbOjR0l7Le7nqNQa39PSTfbXlfD2936nuR2xdyHsD2vJrbbA8fnXOclnUyUwXdaUR6Zc+2czesdYPuwPn5+BWAleixqCUeD2rbDki4k2mmyJ+SpPPr1xLnembR4DPip7SYEOhtH0d//HmLhtLTtWqKXPeKuRWgXrVWnNH68Jeobbn9pvF0lxd2eqOjJErDuiPcc0SIL8XsvQIh6NyLCnBbdazBchLhvPY+OeI21PylESHchFrIPAT8g7IdHdcaZg7jfJZL9N6TH6xNOMH3praQ4Iy3aas2NJZ0BHNt9X5T05jTGXfuN2RVnaeLebuAm27WFaNMmylTgZcCZ6d9P3IDunyR1b5hKmt81HJpShf/2RPLmvwjH0Z1tvyxjfI1XuHTFX4ZoxxQxD8sSDB7vlOTIgNPkzbkN1EKP+6DvVFS0OU5JvyImtncwfPGdY0/3M+D9tv+SHi8DfGMCJEd+BvyJ6B+u3ImmZS5ob7a9bkPjGxfneyca6mnfjfib/h+xQ1TXjnKYxo6iT/kXNSe4bX4uryMWNtemar5ViF3012XEvA3Y0EnXoYkKwV7Jv9yE4Gxeb2DOYYXY5drE7vxMrYw696KOmCvkXHu7Yu0+yrdt+5RRvj+72B8mKpDWIXSgriKcay4b9Ymjx1wK2IlYNC4DnEWc830v6Cdyor4z2ZsqnJpoV6n+plsS7/UZxDyxdnKxTVKVx+uJ5Mh5RLvrNbanZMRsrMJaIRR7NSFw+9t07Pe2c9t+7iJ07v6YDi1PiAg/zxhrtEm60/YrR/herlbT+4gNxMuINczmhJ7WiaM+ceR4T5Pac5xsdpt4f1KcEzuTS+k8Otf1qlefAKYBnyHOb+eOs40Kl5QAW8T2D7uO7wY8MGjrzblJ0RwZcGxfrBCYmwdANa2lWqSNHvfWtDwkiXAFWMn2IYoewGVsT6sRrs3e67/b/mnDMVesEiOJvxFZ7Rc6O5GskW0/kpJCH5/Nc2bHJaohFjkC9wKowzqwotexAWFehYDkdsAxtp+Rsj62V0r6FLBASgh/CKh7/t+bM5DZcCDRh76cpFOJnup3Z8YUUYFT8Rz518BJkha3/TDEfYN27/e1xqt2LIL/nP5Norn703ySjmd4JR91JqTE7lw3ArYhyq5rJ0eIXf6vEInKrAVyxy7takRV0yecIUAL4CTumBLWa3Qn6nNiN4WGu6AM+xZ55+ZM/Q/bz0m6JzcxkmK9J12LtyZ2qr8p6WLbI1bPjSFTgFcTYo/vSYm3HKce6O3+8+2asXYkkoCXS7qASDY1MR99SwMxqqrqGxhqWZnmHg4mfTKaI00tt5oOPg681vZDAArXwOvosonug2WJFpWvpHPnzAbGWPEnScfa/mCqtPw59c+jTxHn0bHAaYp23CycnI96VbMoBH/rcBBx3+nmUqLaZcImR0rlyAAj6QOExd8TRIa5ujlnZ0kHGUmvdLKlayH2scTfckvbr0gXwYts95qwzmnMS7uzy72O9RnzDcTE9FIaEkCUdAzRpnU6MfnbBfit7Y/UjTlR0VB71lPEpDd7Uddr932QduQ7kfQRolrkNkJUb3ng+7Y3HfWJI8ebBOxBR5Uc8J3uMtc+Y85LiMxtlg5dSZQ213KnSjFPIVqpngB+T5SfPlg3Xoq5HyFMek46tB2hm/HVkZ8125i7AwcAPyQ+6zsRAqU5C+/RXq/WeapwfWrFIljSQm7IZUXhFPAtooVsZiLL9vTMuFWy/v+AXxLvUd96MxruBjILdTZUUkXC6cAlTi4GTdG9I50+/7fl7FI3haQfA0sTCaEzbP9xNk+Z07htt6vMSyzC3wtsanuJnHht0FHlMZ2oFnqMEDbvWbnQR9xGK6xT1cB2xBxsS6KV7JyczRA1IBqr0K3ZgKi62oioEPs9KVli+8wa4/o5UUF8XtfxrYGP2q4tuq8QiN7aqcVR0ouIFrBamlJdsZcjdEemAgsS78+nMmMeQbQIrwMcbvvszHgrp/HtQsy9D0zj/HVGzMbmiRql/Wy0700ESnJkgFH0zm+YO/meGyh6NY8FlnIIFa4FbGv7CzVijbRzA0ATC1A1oBmQsrULEiJGr2doh2FR4Hzbr8gY5/eB1QmV72piamdqBij6k6vF4lW2zxnt5wvto+jJ/U/CUWNXhp9H33ING862SeXRFSZ26Sfb/uwYDWkWFE4L8xITW4B3EmKftXdUJW1JCB9uSjgszCA+R1mOAYp+4kpQ8Srbt+TESzHXICb2Ai51h+BeZ1VJE6hmy45asAhWiKieACxse3mF2O0H3Gd/f1fM6bbXaXCM8xAVRx8DbgQOs/2rjHj3MOQCsjyhbSHCgeCPzujJl7QxIRj6L0nvIFqWjnJei+dAJ+oVQq87EOOan9CeOGPAqnYBUFjO7kJ81i8nqh0uzq0cagNJ32RoV/1jhG7XDNewi03n0Gm2W3XxS4nHtxOaEbXEOdWeaOxChLbQvlDPIjfN3X9GJFiqZO+6hPjyWzMX8t8DXkVoCBp4G9Fu8mvIa3Xsep3VgF1cT/Ots5VPwGfTGC9IY2xEB0nSq4j53U62V6nx/MbniZJ+TVTwPdt1fF7gl7ZX7TfmC4WSHBlgUlnfDg2UzbWOpCuJErrjOpIOuf2KBxP2hqcQF4LdiGz7lzJi3khk3G9KSZIlicqROhP7fYib0rKEpkV1sXoU+LbtYzLGWUuMcg7iVnaplygccSY3Udo70ZC0Wa/jtkd0SRkl1ruIhdK6wE0MP49Oburm3CSSPtbxcH7CjvaufpN3km5n9ERoju5GIw4BPeJOJlojtiCEOp/ISWBJ2sP2CV3HDrf9yZGek0vTFUmSPmX7i338fDUh3ZzYpW/MIjhd46cQ/eJN3Ys+DzxAVPd0jrNORcbewD5EVeDhOUmGHrG/Rfze56XHWwNb2f7Y6M8cNeZtRCvEWsS9+ARiXrJ55lgHPlGfKlp2JixYv9jUYq5JFIKapwMX2H4qVShMdX3b2bmCpBWBRetUSqXn78OQDs4PqKmDM0LsXtfkI2z/X814jYjGpqRKVTVSVTtPJ1ptrq97LVHYPu8KVNfIO4nEU99ipF1xDxzt+3WSGSnu1SQ9JaJiJscAopcTW0X2hmRTdM0Tb+74Vu15oqTDgaWAD3u45tnXgQfrnu8vBEpyZICR9FrC4u9Ghk/IalvEtoWkm2yv11WRMcMZFpdKIo2zO9ZnzN2Iyc7axI7yFOAzts+qGW8yYQt7SN0xjRD328BX3WWvlhmzMbvUiY6kTj2M+Qk19ukZO0uTiAntqU2Mb26TJlfn2n5zn88b1QUgc3f6F8Dbbf8uPV4Z+GFOUiCVCS9EiMJdTYitPVA3Xop5PtGSdGp6/E1gPtt75MSdzWv2VenRZGVgitfahFRd4r7pWK6jUK/yd7tGi6tC9PEB4O8MTwxW7RU5CcFZKlyUKR7dUW35OeBPtk9oIrk2yIl6SRsR5fCbEta4P7B99ejPGjvUoO1sm0gz28hWtn2wpOUJN6U6mm9VzBWIJElV5XM6UeWTU/HQfU3+BjB/3WuyGhKNTdeOXxC24me5eYvcpYH1iWrlLGeZrriNtTimeCszVMG5AbE+utr2/zb1Gk2gFjS1JO3ozHafjljzAF8g3P3+kMa3HJEA/6wzWpDHO0WQdbA5jlB5HmYXOqA8qHBuMICkKcBfRn/KbHkuJTPOSHGnMly4sG9sn6rod30DcSHYzhn97g5htf8GGk2OEBf+d6VJ+VM0MHEG9ibZpRLBfiPppdkjnYDYHiZipeh/rV3RZPt5hcbQuEyOEO1lfS8Um9w178HHCWG93xOfnxWIEuQcbiP6kdcE/gk8Iul6209kxNwBODdNfLcG/uGMFpA5pN9dkW+TKgMBbN+msHOulRxxjTL6PrgvLW6t6HH/KOEMURs3YBXZQZOxunlQ0meI0msD7yAsSXN4TNIBKdZmaUMgSwSxM1EPrEKUi3+LuC+PKZLuBR4h5h17As+m42tDvmVmU6i37aychBsHlG+SNN8IPb3HgLPpLVI8R6R7yBHAEWlD8URC26HvFpMOel2TcypxeonG1hGi3Zhod9ke2C+dq9enfzc7Q7xd0h6Es8zlxP3yaEm1nWVSzJktjkAjLY4Atn+vcIR5Ov3bAqjdxp7GejJhq/1Ierw4cGRm5ciXaF5T61pJJwDL2t5a0T67YXel05zgaKf5pKSDgJenw7/NnM+8ICiVIwOMpOtsbzTW45gTUib3eKLc72Fi92K3zJ3fFYGjiBuCCXXufW3fmxFzK9uXdB17l+2TR3rOHMQ8iFg0/cgNfaBG2lHP/Hs2ZpdaGE7aEbvNGa1Qkj5LCH3+gOEWpIPY597ZDjMZWJKw6OurlUzSNbY30aw6Q02JFM5HuG0IuDtn8tgVd2Ei0bI/sfM5X40YnSKaixCtJdcSE9RW3/d+d/7bqAxMMb7e4/A/iYn+T2rGXIK4b2xFvO8XEZPe2kmCVNmwH2G9umequlvN9s/qxkxxO6snFgDmySwRfwmxMNyM+DxdRXwua59LaTd5V2In+eq02/9629/LiDmDlKjvOJ9aaSXtF0lXMHQtqnRcKly3OrBp1JLtbJuoQc23jpiVEO0uRHLtSqLF5sc1YrV2TVbDorEp5oqE28g+wMts13UtQdKvgI3c5Sxje7WMmI23OKYYvyOssE8jPgMznCka3auast8Kyx4x29DUOp/oKPi07VenefwtmXPP+QmHwE2Ia941wLHObKsaz5TKkcHmckl7EpaWWX3ObaFwWqg4j8g6TyIWdzsStoK1SEmQt+WMrwefk7QjsahZmMjeP8WQaGMd9iNK7Z+V9CQNLOxa2lG/Us3ZpU5oJB3N0AR6EvAa4NbMsNUORecOlalRkTEXeGvH/58F/uYaAoC2N0lfG7cE71jQrmD7/ZJWlZS1oJX0YaKUdx2iDPVEYnJWh+nMmhD6n/Sv7fe9X3vKNioDIcrgVweqtsYdiX73PSRtYXvfOQ0kaV3bNzsEzHdrYGydnES8X9Vmxf3EmHPOpe7qiZeRUT2RKjoOsL1P3TH1IpXWf6Xj8R+B2omRxFO2n1ay/04T/IHYqbP9eogFQ/fiQPUtM9ugLdvZNnkmnafVdWRJalZFpznMVOJedCOp0iezfaO6JqvjayPX5JQMuTiNfbKk3VyjjVbS6gzpjmwMLE5Ujnyr7tgS9xOVPBWPAfdlxsT2fdXnPJFV/Z34OrGQn0pouVwp6SqnFtqaTFKHUHlKlOWukW9W2Pg2pqkFLGH7zFTNh+1nFU5YOXyPeL+rVryphL7U2zPjjltKcmSw2TV9PaDj2KAtlqpFzWpEaeRPiBvKO4ldq9qkG+f7gRXpOFczy9w2J1TSK+Guz9k+PSNeKwu7lvgkYZd6O/ABIplVp7SzMFwQ61lip+ranIANl+63SlPJO0k7VBMFNeygwtCCdsP0OHtBS9hvfoXQl8lyg7C9kkJrZsPcc6cXGm4duSTh3lJpZ/S7AN+bqAxcXdKfSJWBDQzz5YStetW6cCxR6fFG4jrVD99OFT2nE5/HJkuZV7G9s6SpALafUNeMvwaNtjk6WjybdNRps6prPCTqryO0yWZ3bExwCNieoyHb2f8FlkqfoSzb2Rb5OiFq/FJJh5I032rG+hRRObB/UxuGTd+DFda7exNtY+cSyZG9iRbFGfTZRivpQSIpfR2RlD+8qhpqgD8BN0oa5ixTbYC6nhhx4y2OaSxHAUd1VHB+nkgu57RSHQlcJ+mH6fHbgUNzxkk4yfybqBiqMGETXpd/paqeKsG4AVFtmcNqXdVblyvs6ycspa2m0AiSLgJ2rEqCJS1CCEa9JSNmdQOYTke22RliRCkbfByR1HkZ0Zt9RG47jKI/cVViJ7QaZ1ZyqGkU7gDnNdVaUGgeSWsCazD8PMrdpR1YOts7+m31mIPYN9tet8kS7jZQaJZsOPuf7Ctmm9aRk3JaP7ri/Qp4ne1/pseLEa0Wq9cpaVaydCSEKZ9mSJwxK5mX7kVvIJwR1k5VNKfbfl1GzMbbHCUdSdyHzmJ4a95AOV6lpOAedLQZAN/JvQ83gcahtXqFGrCdbZtU+VBpvl2am8RUS25fTdyLU6LhYaKy4w1ElceLiDa/vp11JC1WXSubRi04y6iFFscU90iilCbQlQAAHhZJREFUcmRhwqnnKkKQ9feZcV9J6JdU52ZjhghNodA+OprQPbuDaGme4pquTynmd4lr2w3p8frAu9y+9tnAUipHBpxxtFhanpiMVjxNVHzksKCbt5K6gci2n6jo8T6C6Cmtre0i6X2knk9iN2AD4mY4aJOTbYGvSbqKKEG9MHf3e6Ih6XJGLv+2M5x/0uTk9cTn/TxCCO4a8kvYBxmN8P8meDp9xqsdllXoKG0dIC5KrX6NaRYRgn2vJZwNsP3nlLCuRdqpOpDUkyzpGkLLIlfs80vADIXOgwitjC+mJMwloz2xF7Z/BRwEHKQQ/9sFuEzSXzMTQwcCFwDLSTqVKGd/d0Y8aKd64iWEOGfnvSd3p7IN3gZ8z/a3x3ogPXgz8d6+jNhN7rRW/9QYjWmOSBUUx6V/A0NqR9qLqBS7HTiuwbnHFElPusvtKydgg/filSstCEnfIXQyls9ILj+lsHR9mLhWfIJo8/wdcEhqKaxFlfxQg84yvVoc07U9lxuAL9n+WwOxZmL7Tkl/J623JC2f2gj7QtInbH9Jw9uvO18nx3F0FeJ8XI5orVuf/LX8+sDukqrfdXngLiVtuZyE/XilVI4MMCNdoG1PGctx9ULSp4GdiJJJE5PzH9g+LCPmFwhBqPOaGWVc7IjWmpU8ZCW3Yk6VR7qArAfcYPs1aWfkINs7NzPq5lCIl21N7KxuAlxs+31jO6rxwwhl6xsQk5QHbNdW3U/n0asJca1XS1qK2E3dZjZPHbdIupvob53ErDu1Wc4QacH5GeL6eRFpQWv7iowhN05qW1iIqI57ggbaFtSQdWRHvIuJ3bnvp0O7EaKcW9UdY0fsZYj2EgHTbP+5gZiTiJ3aqcB/E9fm7TJj/gfxWVeKV3sh0jHGgayeaBuFlfOWxDk1cIl6jXNr9UFDobvwDFEJvDVwr/vQE5pN7AWIlpUTGXKWyYrd1L24uxoytzpS0pnE33EhogrlDiJJsgnwGttvHeXps4s901nGdrazjKT/BJYhhOqfVrQM7kvcg5etO86O+NsSyXSAK21nJZZTvCOBZQmr9RWAu2y/skasbWz/NCWyZsF5BhC32V5L0Tb7xTTmT9lePyNmTwOIitzKy/FISY4MMONtsZTKvTZND6+yfUtmvMcIi9CniRtCE4uGY0lWcrZfkdphLspc1FZODjOA9W0/pQacHNpCQ+ru7wE2tb3kGA9pXCJpc+CzxC7VF22fnxmvWtBOJ0o7HwPuqHNzHi+kSpyRcG5peNML2vGCpP2J9oo3AocRYr+n2T561CeOHG+67XW6jt1se92a8Va3fXe6Z8xC3aSYpE2JhMh2xMLhDODsuqXoI42vIjN5txDwpO3n0uPJwHy2/50Rc34i4fJKhleb5uh0tcKgJ+oVAo+bzf4nC7NDHU5EqX1sWk6SIMVp01mmkXuxQiizqsIQoVn1b2rOZZWcXtLf8H7bS3d8L9f1pzFnGUn7Ap8GfkvMj44itLq+R1R8ZIl5SzqMSKhXycuphMPZASM/a7YxbyUStpc42hy3IBKke+aMtWk01IZ5GHC77dOU6aqT4nZqlC0BLOIhjbIJR2mrGWyesP28pGcVwk4PMFhirMNIE8Xak8UeLEbsUHZWeSyTGXP9tJN6C4DthxVCUTncL+nFxM35YkkPA9m7n00jqbK72wK4ghBj3WksxzQekfRmIinyJHCo7dEW+P1wczqPvk3o7DwOTGso9kBie4umY/ZY0FYTseVTmWyT16hG6NoFu8KZFrG2v5wqZx4lxLI/5zzryMsl7QKcmR5PAX6eEW8/wqnlyB7fMzVaEiXdB/yRSIgc1FDJdTW++QkNl1uJhc1ahJDqJhmxLyV68R9PjxcgKpxqt3gSDgN3E60hBxP3zyaFaRvD9jMKW0oTv/vbgIFJjhD38v0ZB9bq44Bnqv843DWaiNmaswwN3Ytt5wiE9uLpFPdZSd1zzGwXGDfnLLMnoXf1jzRv/y2wmZOmRQP8D1Ep8zyApJOBWxhuXtEvz9h+SNIkSZNsXy7piDqBJJ072vdtb1tviAD8SdJxxL3jCEnzEZW3tVGHRhkhZP8iokq0URvi8USpHBlgFL2TnyIWtB8jLtAzbL9nTAc2l2ipyuNGYvJ5U0qSLJliZmVdO+JvTiR1zrf9zOx+fm4i6Qxi4XC+iyhrLSTdRAhg/T9CV2YYTS28Ja0ILOoMka3xRNpF/iAdCQKiL73vz1BHNUrPBa2TffCgIOlwoi2vcxdsujMEBTurEhQipatR85qkmC1Xk+Tq62SGFoxZ1XxNIWmFzvLftKFgNyAem66dh9q+PT1ek3DJeHdGzFmqC3MrDjt2FavS63mJlpWB0r/qkaj/AXEfHqTWml67prY9sBtUg0rTFRRzk0G6F0t6gCHL5p3T/0mPd7K9VEbsHxLVHccQ1ZYfBda1vUuNWN3tRLUqUEaJfxvR1vmP9PglxKZCjpj1JUTF4WHAEsRm9Hq2+05WK3RL7iMEwW+E4Xpqtq/MGOeCROX37Q6Hs2WAVznDnSpVvb+WEASvqoZuy/l7jndKcmScMEgX6LmFhvrlG3ObkLQbcVNZGziZZCVn+6yMmKfYfufsjg0CqTWrSi5Ns/3AWI5nvKEQjqwumtVuVUVWG4ikS90l6Nrr2AsRhVjdvMRnEsIK/LmcMvs2FrRtkCZ6nbtgk4lWypyJ3nSixXFxQrzuZuDftmvZ73ZPdpsiTfT2I0QK95S0KrHjWLtyRtK6xO7XIsTn8xHgvbanZ8RsI5FxLfCRKqGq0DM6xhnORR3tAFcRAq9/Ja7zA7WgL4n6QpOoYeOClBDejRBUraqWl7Y9ppWcGkHDosJ5WhaNOct0JHEqdul87DxBUhSW6ocDl6exbgYcYPuMUZ84esyFCM2vScR7vxhwas3ffzLR0jqV2JT5OeFudmfd8bWJGtYoeyFQ2moGHIWo0Qqk90rSZh4wi9gWeSZdZCq3iSWJSpLa2D41LRwqK7ntnGklR/R3zySNuZdw55gi6e3Al4mdOgFHS/q47R+O+sTCTGy/HqK33/aTnd9T9Pv3TXregsASqTqq0zoyW7hsnLBeV9LzMkUPcA6rV4kRANt3SBpIHSDgxUBVrr9YA/Fk+9+S9gCOdijn52hAXS9pPds3NTC2Tk4iytar3bn7CRvanLaiE4EP2b4aZvZSn0RMUutyV0rgfZ+4H72D/HaVfYGzOsrjlyES9zkcn64hnyFEKhcmWgAHCtu7pET9G1MZ/0Am6ptedBeaR+24vH2TVLVMtKc9BpzN0MbSmNCd/NDgOst8vOtx7cR0L2yfnjaq1iPmS/9n+6+ZMau/4/OSfg485JrVAw4dqQuAC1Lby1TgCkkHu6buV8ucmVp1Xizp/YRG2XfGeExjSkmODDCp321n4JcMlTObUHifCHydcL95qaRDSVUeuUFt3030ZWch6QCi7WkBSY9Wh4m+0ONz47fAZ4hF6AMwM9l0CVCSI/1zHVF9NLtjc8IHiIXSssQkouqhfowocZ0IPCdpFdu/A5C0Mvk91G0saNvgi8AvNNzONqd3GmLzc0NisrtHOpZzv98C2EvSvUR5fFUOn7uztIrtndNOILafSLu2OTxWJUZSzGsU4t45vIdo+9onPb4KODYnoO2bFM5mqxF/z7sbaMU8hbB3XJGhKqzapfZtMR4S9S0tugvNM4Uh44L3pKRb7sKuDW26xlCHswyhpdWasww1NmjaSuJoVj2x+9PXZSUtW6etWdIGRBXKP4BDiGvoEsAkSbvbvqDmWOcjtFGmEtfjrzN4lupAKxpl456SHBlstiNKjCdk2WlLVR6N4bApPkzSYc5QyZ6LTOranXuITCGniYakpYH/JBJir2V4lceCdWLaPgo4StLngK/ZflTSZ4lEyyy6Ji9QPk6Ifv6e+JuuQCxIc2h8QdsS/0NUOzxMCIpm74IRE9sDgHNs35mSTTnCwVtnjmcknlZYcVbVgasAufe7aWkX7PQUd2di125tqKcLlKrEvpr+ZSFpS9uXSdqh61urSsJ2zgT6J8A/iSTrIM8bxkOivo1Fd6F52jAuaLxquWG+Rogunwtg+1ZJtZyV1OUsI6nTWSarArrpJA4hkN3Zztxd2VGnrfkYYpNzMeAyYGvbN6TE9elEBUhfKARi1wTOJ8TB76gxrrlKSoZcDFH9Lmk3T2Ar85IcGWx+T/ThD/Ikp1WaqvJoA4U3+CNVYkRh/bUdcC/wDdtPj+HwenGBpAuJCz7EoiHLfnYC8mZiN+VlxASi4lHiBpvDlNTfvAnRr3oksZiv7V8/XrB9aaU3wdAuetZ1r8kFbcucRLiebEtM6mcobESPqhswCb5d2fH494TAXt14f5j9T9XiQGLyuZykUwl1/Hdnxqxapw7sOr4RfTrhSLqdWSfgM6lZObM5MQnfpldI8nYXX2b7LRnPn1uMh0T9uHILnMC04fLWStVyk3h8OMs0lsRJ/B9wn5MVcNJg2ZGYc3++Zsx5nMRMU9vLDWmsd2cUMb6TqLD8L+CjHXEGSoA4Xdf2Jjb8ziWSI3sTm1UzGBKJn3AUQdYBRtLZxM7FpXQkSHLFjArNoHC+2d72n5OWwSWE0vVahC3YINkSAiBpR2IBIuAq2+eM8ZDGJZJ2tH12wzFb8a8fD2hImHMF2+9XhjCnpDNt7zTSwnYQRcbSLuV6pPYVYmG2eo04X7O9r6Sf0vt3z7EQbBxJpwC3E0J4vyfchB4c21ENkRLgI5KTNJI0OfWmN4ak4wmNmdtn+8NjiKT/R9wnOxP1t9v+xNiNajia4G6B4xE1aFyQKgeqquVLB6lqWePEWUbSjbbXV0OmCpJ+AWyVEjmbESKvHyES4q+wPaVOzOr37/G3aEWIfFCQ9BOiYvV64lxfnLDx3cf2jLEc21hTkiMDjEZQpu7u5yuMDeqwupL0ZeB525+QNImYRA3cIgxmZotnVo052aEV5pzUXnMosKztrSWtAWxo+4SMmD8D/kSoxa9DLBin1Z1IjCck/YDY9dvd9pqp1eJ613ADkbSM7b9I2o/YQbyv8/stVkHUQtKlwELEBOVq4BrXFKeUtI7t6QpL8VlwhoVgG0jakqia2ZRUNUMkbWtXzUj6D6JqZBMiQXQNcLBruA50xV0BWNX2Jen8nMcZNsGS/khUzfwAuMwZk7GOROA8wKpEoukpmtOGaZzxlKhvctFdaBapOWcZhTj6XsDLiaTtCR4ge+kKjRNnmSaTOCnezMSKpG8Af7f9+fS4lnuYhmymOy2mSY/ntz1vnbGOByTdbvtV6f+TgQcJ57hcja5xT0mOFAo16bqw/IKwErswPR44j3BJHyCU158g+meriXMpFe4TSecT7RCftv1qSfMQvemvyojZuH/9eEHSzbbXbWqHKT3/QGAnQmjtDOCHtv/WzIibQ9JXiWTYU8C1hDbK9bafyIi5EKktID2eDMxn+9+jP3Pu01TVTEe8i4m/4ffTod2A19veKiPm+4ny85fYXiVVNn3LGTbbKcGyDbEYWZtw6DnD9jU1YrVW4dImg5yo1wS2Vh9PSDqW5Cxj+xUKt6aLbPftLJOS9M8QSeqtgXtt79vogAeMkTZhK3I2Y5tM4qR4dxC2989KuhvY08m9s8mKl4nCRKuU6YeSHBlgJG1M9NFVVr5lMTtAKISrlgH+QugF/JftZ9Ki9qe21x3TAXYh6TdEdcPAlK2PVyTdZHu9rsV8rZ2LAki6jijrvNbhErAKcLrt1zUQey2ibH9H4P6cRXKbSFqYEJHdn9j5nC8j1g1E+fHjHbEvsr3R6M+cuzRZNdMRc7rtdbqO3ZxzPZY0A3gd0fZTfd5nJsdzSQu6o4DdbE9uIuYgM8iJeg1Zq19OuNV0im6fb/sVYzS0Qg+qBV0TifWuDa95iMrNgVssSlqJaCdZkeHJxey2STVoD9w0kj4N/DepwgFY27YlvRw42fbGYzrAcUZH1QwMr5wZKG2UsaAIsg42JwD/S5SbN9qbXGiEfYlF1zLAJh6yYVyaUP8eNH7HUMlgIY9/pfL9Ss1+A8IlolCPNoQ5Kx4A/kqIPr60oZiNIenDRFvJOsAfCOeaq0d90uyZv0qMANh+PFUmDRq3Eb/3msTn5xFJWVUzhOvRLsCZ6fEU4Od5w+Qph8UlMHPhlL2zlNqfdiZ2qW8iKp0mAvsDrxzQRH23tXrFY8A3xmREhdFo0llmppV2qk5oYHit8GNiffBTGnLRUfPOMo0ncWwfmhLqyxDJ/uoaPCm9TqEPJkIivi6lcmSAqcSMxnochRcGCuvZk4AbKQK/WShsQY8mFnV3AEsSbjOlJ70mKdm0AbFrcUPuwknSB4mF55KERegPbP8ye6ANI+njRBvI9Kb62yVdC3zEybZW0jrAMbY3bCJ+0zRcNfMYUY1SbShMZmh3rNZumKQvAY8AuxOT8A8Bv7RdOwku6R5CY+VM4NxB3a1tA0kXADsMaJvXesD9xPX8aHU5YgxS608BJO1GXOfXBk4mOcvYPqtGrHGxk97G2kBhMDCFuBZVFThZrSqSbiUSLrfTkcTxgGlfFQrdlOTIACPpcGJi9yOGL2Z/MWaDKswkTcJ7fYAG6kZaIWkaIU7YfaMqAr99IuntwIXAcsTEeX3gs+Wz2R8pyTQiOX/PdP08wxNQdV3SuoTQ55/ToWWAnW1PH/lZc58eVTNXAVfbviwz7ksIYdL5q2M5E3KFyPYewJuI6/uFwHecMYGStKjtR+s+fzwzyIl6teCIUWgXDbCzTBtI2pW4vl1EQ2sDNews0xmz7vMLhbGitNUMNtVFpbNX2sCWYzCWQhe2FxnrMfTJs7b3G+tBvED4rO2zklbAVsCRwLEMfWYLc8aR6ev8xHXuVmKCuxaxcNqkbmDbn8we3TgkLeRfBKwOrEb8Pe/uaPsbJBYg3AyarJp5H7AP8DKiMmMDoNK0qcsCwIm2v51eYzLDnQ3qsLSkc4ClHA5NawHb2v5CRszxwnHAZXQl6geEyR3VITsDxzts289O2jOFAUCzOssc19Q1ZBzwKuCdxFqg+vzkrg3uk7QRYEkvIpxlcpNMRyVh9MaSOIXC3KBUjhQKDSHppQzfqfzjGA5nFiQdSuzO/pThN6pSJtwn1e6KpMMId5nTOndcCv0h6QzgUNu3p8drAvvbfveYDmycknQ7BrKFpm0UtrbrEa1Zr0m7ygfZ3jkjZuMCt5KuBD5OLOoaKWMfL0i6btDEgSuKI8b4YCI6y1Sk83It2083GLNRZ5kU8zAiifM7OpI4tssGb2GgKZUjA4ik7t19E+rM19i+ZwyGVBgFSdsSO+DLEuKPKxAZ91eO5bh6sGv6ekDHMQNj7hAwDvmTpOOIicQRkuYjRMEK9Vi9SowA2L5DUnH+qc9FknYEfpTT+jFOedL2k5KQNJ/tuyWtlhmzDYHbBW1P6xJ9nCg735dL2pPBTNSfDlwp6UHCTedqgOSIUUS3B4c1OpxlTgCmjfF45ia3Ai8m5puNkDS+dmsqXmJ7YOUmkziFwtygJEcGk17tGisCn5b0edtnzOXxFEbnEKJ0+5JUTbAFMHWMxzQLtlca6zG8gNgJeAvwZduPKOybPz7GYxrP3CXpO8D3iYTdO8gv6Z3I7EcSJZX0BAOqg9QS90t6MeHocLGkhxnSXqnLvySt3SVwm+OoA/CgwrK6ctmYQtjCTwQGNlFfHDHGDePFWaYNlgLulnQTw5OLta18W7IHbjyJUyjMDUpbzTgiicxd4gH0XZ/ISLrZ9rpJmfu1tp+XNM3268Z6bJ2knc79gOVt7ylpVWA12z8b46EVJjipf/yDwGbp0FXAsbafHLtRFcY7ySp3MeCCnN3L5GByBg0K3EpaGTge2Ah4GLgH2M32H+rGLBQmCuPFWaYN0nVtFjJFpxt3lpF0BaEf1lgSp1CYG5TkyDij6BoMHpIuAbYDDgOWILLk6w1aT3Xq0Z0O7J4EABcArrdd2hcKhRcQim3U3YCVbB8iaTlgGdsTqfS8USTNSwMCtz3aZhcgqhL+BWD7KznjHA+URH2hMFi0ZA/ceBKnUJgblOTIOELSloR/exEzGiAkLQQ8SUyadyN2Kk/NEbJqg44Kl8as2gqFHCSdaXunJKI5y83I9lpjMKxxj6Rjid2/LW2/IrkqXWR7vTEe2rhC0pa2L5O0Q6/v2/5RjZgHpv+uRgjH/oS4d2wDXGX7fXXHO14oifpCoX8kXWN7E0mPMfx+mV0t04Y9cKEwXimaIwPICAuFlxAlvbvP/REVRsP2vzoenjxmA5k9T6dJaNXjvgodN8FCYQzYJ309iRDUu28Mx/JCYn3ba0u6BcD2w8mesdAfmxOWs9v0+J6BvpMjtg8CkHQRsLbtx9LjzwNn1R7p+GIV2ztLmgpg+wlNMNGIQqEGCwHY7qVLmEtj9sBtJnEKhblBSY4MJm/temzgoa5FeGGMGU83gDTx/BZwAbCcpFOBjYF3j+W4ChMb25UA5SLAccA/CG2HH9r+25gNbPzzjKTJDCVCl6Sjj7wwZ9iuqjzeZ/u5hsMvD3TqoDxNiCFOBEqivlDonzZL/Zt0lmkziVMotE5JjgwgRZBtfGB7k/R14G8Ati1pH+BNhLOOCA/7B8d2ZIXCzN30gyStBexMWGneb3urMR7aeOXrwDnASyUdCkwBPjO2QxrX3CPpAuAHwGUN2SOfAkyTdA6x6Nmewa48bISSqC8UavPSHppFM8nUK2rSWaboNRTGNSU5UihkIukU2++c3bEB4AZiZ+DnYz2QQmEEHgD+CjwEvHSMxzJusX2qpOnAG4hE6Ha2izVyfVYjWmv2Bk6Q9DPgDNvX1A2YLGPPBzZNh95j+5b8oQ42JVFfKNRmMrAw8ZlpmibtgdtM4hQKrVOSI4VCPq/sfCBpHmCdMRrLaGwBfEDSHwhnhKr9p4heFsYUSR8kKkaWBH4IvN/2L8d2VOMXSVvZvgS4u+PYu2y/4CsT2sD2E8CZwJlJ3PYo4EpisZIT9xfARBQ8LIn6QqF//mL74JZiHzj7H5lj2kziFAqtU5IjhUJNJB0AfApYQNKj1WGid/z4MRvYyGw91gMoFEZgBWBf2zPGeiAvED4naUdgf2KS+h1iN7AkR2qSbCl3Jq6jNwE7je2IxjUlUV8o9E9ryYaG7XXbTOIUCq1TrHwLhUwkHWb7gLEeR6FQKMBMXYePAR9Ihz5n+/QxHNK4RtI9wAyieuTcIo6eh6QVeh0vemuFwshIeontfzQcs3FjAUm32H5tY4MsFOYypXKkUKiJpNVt3w2cJWnt7u8Xf/hCoTBGLA6sD/wOeBmwgiQ1JCQ6EXm17Udn/2OFOaEkQQqF/mk6MZJow1nmDQ3GKhTmOiU5UijUZz9gT+DIHt+r5Q9fKBQKDXADcLjtE5Nl6hHAtcBGYzusccvSyVVmKdtrJlelbW1/YawHVigUChk0njBvKYlTKMw1SltNoVAoFAovICQtD2wOrGT74PR4RdtXjfHQxiWSrgQ+DhxXlYtLusP2mmM7skKhUKiPpPuBEd1jirNMYSJSKkcKhQaQtBGwIh2fKdvfG7MBFQqFicwBwPNE9drBwGNEhdt6YzmoccyCtqeFlMtMnh2rwRQKhUJDFGeZQqGLkhwpFDKRdAqwCiHY91w6bKAkRwqFwliwvu21Jd0CYPthSS8a60GNYx6UtAqpBF3SFOAvYzukQqFQyKY4yxQKXZTkSKGQz7rAGkXssFAoDAjPSJrM0GJ+SaKSpFCPvQl79tUl/Qm4B9htbIdUKBQK2ZSKkUKhi6I5UihkIuks4KO2y05ioVAYcyTtBuwMrA2cDEwBPmP7rDEd2DhD0n5dhxYAJgH/gtKPXygUxjdt2AMXCuOdUjlSKOSzBPBLSdOAp6qDtrcduyEVCoWJiu1TJU0nLBUFbGf7rjEe1niksrdcjdBr+Qnx93wnUMRtC4XCuKYkRgqFWSmVI4VCJpI273Xc9pVzeyyFQqFQaBZJFwE72n4sPV4EOMv2W8Z2ZIVCoVAoFJqkVI4UCpmUJEihUCi8oFkeeLrj8dOEO1mhUCgUCoUXECU5UijURNI1tjeR9BhJ+LD6FmDbi47R0AqFQqHQHKcA0ySdQ1zrtye0XAqFQqFQKLyAKG01hUKhUCgUCqMgaW1g0/TwKtu3jOV4CoVCoVAoNE9JjhQKhUKhUCgUCoVCoVCY0Ewa6wEUCoVCoVAoFAqFQqFQKIwlJTlSKBQKhUKhUCgUCoVCYUJTkiOFQqFQKBQKhUKhUCgUJjQlOVIoFAqFQqFQKBQKhUJhQlOSI4VCoVAoFAqFQqFQKBQmNP8ftwCOwNiBH/AAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a2aedec50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot a bar graph displaying the amonut of null values across\n",
    "# the columns within our dataset.\n",
    "null_sums = df_main.isnull().sum()\n",
    "fig, ax = plt.subplots()\n",
    "fig.set_size_inches(18.5, 8.5)\n",
    "for i, null_col in enumerate(null_sums):\n",
    "    plt.bar(i, null_col)\n",
    "\n",
    "plt.title('Null Values by Column in Dataset')\n",
    "plt.xticks(list(range(len(null_sums))), null_sums.index, rotation='vertical')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>deleted</th>\n",
       "      <th>endTime</th>\n",
       "      <th>removed_set</th>\n",
       "      <th>setNumber</th>\n",
       "      <th>startTime</th>\n",
       "      <th>rpe_num</th>\n",
       "      <th>weight_lbs</th>\n",
       "      <th>StartMessg</th>\n",
       "      <th>RepN</th>\n",
       "      <th>AvgVel</th>\n",
       "      <th>ROM</th>\n",
       "      <th>PeakVel</th>\n",
       "      <th>PeakVelLoc</th>\n",
       "      <th>PeakAccel</th>\n",
       "      <th>RepDur</th>\n",
       "      <th>TimeBWReps</th>\n",
       "      <th>TimeRepComp</th>\n",
       "      <th>TimeRepWait</th>\n",
       "      <th>SlowAllow</th>\n",
       "      <th>Backlight</th>\n",
       "      <th>MinAllow</th>\n",
       "      <th>setID_ix</th>\n",
       "      <th>RepCount</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>71232.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>181224.0</td>\n",
       "      <td>181224.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>61034.000000</td>\n",
       "      <td>181224.000000</td>\n",
       "      <td>181224.000000</td>\n",
       "      <td>181224.000000</td>\n",
       "      <td>181224.000000</td>\n",
       "      <td>181224.000000</td>\n",
       "      <td>181224.000000</td>\n",
       "      <td>181224.000000</td>\n",
       "      <td>181224.000000</td>\n",
       "      <td>1.812240e+05</td>\n",
       "      <td>181224.000000</td>\n",
       "      <td>1.812240e+05</td>\n",
       "      <td>1.812240e+05</td>\n",
       "      <td>181224.000000</td>\n",
       "      <td>181224.000000</td>\n",
       "      <td>181224.000000</td>\n",
       "      <td>181224.000000</td>\n",
       "      <td>181224.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>6.493941</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.266047</td>\n",
       "      <td>237.509353</td>\n",
       "      <td>-3219.539150</td>\n",
       "      <td>24.543455</td>\n",
       "      <td>0.468691</td>\n",
       "      <td>462.169144</td>\n",
       "      <td>0.759044</td>\n",
       "      <td>72.447121</td>\n",
       "      <td>-467.837734</td>\n",
       "      <td>1.129646e+06</td>\n",
       "      <td>0.351256</td>\n",
       "      <td>1.706150e+09</td>\n",
       "      <td>2.795605e+05</td>\n",
       "      <td>129349.114167</td>\n",
       "      <td>9952.900095</td>\n",
       "      <td>149247.324686</td>\n",
       "      <td>24055.649169</td>\n",
       "      <td>3.375546</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5.946106</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.541093</td>\n",
       "      <td>106.705020</td>\n",
       "      <td>468.222607</td>\n",
       "      <td>24.490327</td>\n",
       "      <td>0.223723</td>\n",
       "      <td>155.470025</td>\n",
       "      <td>0.368769</td>\n",
       "      <td>19.757754</td>\n",
       "      <td>1097.396166</td>\n",
       "      <td>4.931590e+05</td>\n",
       "      <td>5.669843</td>\n",
       "      <td>1.020477e+09</td>\n",
       "      <td>6.925327e+06</td>\n",
       "      <td>9147.219381</td>\n",
       "      <td>662.377762</td>\n",
       "      <td>10577.566980</td>\n",
       "      <td>13876.404474</td>\n",
       "      <td>3.284430</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-3456.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.039064</td>\n",
       "      <td>150.000000</td>\n",
       "      <td>0.034893</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>-9999.000000</td>\n",
       "      <td>5.000000e+00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-9.876000e+03</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>76.039300</td>\n",
       "      <td>76.039900</td>\n",
       "      <td>76.029900</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.500000</td>\n",
       "      <td>165.000000</td>\n",
       "      <td>-3456.000000</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>0.309058</td>\n",
       "      <td>328.000000</td>\n",
       "      <td>0.473064</td>\n",
       "      <td>65.000000</td>\n",
       "      <td>1.182262</td>\n",
       "      <td>7.780158e+05</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>9.008389e+08</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>130000.000000</td>\n",
       "      <td>10000.000000</td>\n",
       "      <td>150000.000000</td>\n",
       "      <td>12065.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7.500000</td>\n",
       "      <td>225.000000</td>\n",
       "      <td>-3456.000000</td>\n",
       "      <td>18.000000</td>\n",
       "      <td>0.421928</td>\n",
       "      <td>443.000000</td>\n",
       "      <td>0.706708</td>\n",
       "      <td>79.000000</td>\n",
       "      <td>4.288830</td>\n",
       "      <td>1.065460e+06</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.486152e+09</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>130000.000000</td>\n",
       "      <td>10000.000000</td>\n",
       "      <td>150000.000000</td>\n",
       "      <td>24033.000000</td>\n",
       "      <td>3.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>8.000000</td>\n",
       "      <td>308.646800</td>\n",
       "      <td>-3456.000000</td>\n",
       "      <td>33.000000</td>\n",
       "      <td>0.580345</td>\n",
       "      <td>595.000000</td>\n",
       "      <td>0.966042</td>\n",
       "      <td>87.000000</td>\n",
       "      <td>9.837069</td>\n",
       "      <td>1.407044e+06</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.375322e+09</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>130000.000000</td>\n",
       "      <td>10000.000000</td>\n",
       "      <td>150000.000000</td>\n",
       "      <td>36057.000000</td>\n",
       "      <td>4.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>169.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10.000000</td>\n",
       "      <td>1000.000000</td>\n",
       "      <td>-1234.000000</td>\n",
       "      <td>403.000000</td>\n",
       "      <td>2.960175</td>\n",
       "      <td>1761.000000</td>\n",
       "      <td>5.329412</td>\n",
       "      <td>99.000000</td>\n",
       "      <td>24150.160000</td>\n",
       "      <td>6.072204e+06</td>\n",
       "      <td>2295.000000</td>\n",
       "      <td>4.294947e+09</td>\n",
       "      <td>8.366950e+08</td>\n",
       "      <td>130000.000000</td>\n",
       "      <td>10000.000000</td>\n",
       "      <td>150000.000000</td>\n",
       "      <td>48128.000000</td>\n",
       "      <td>87.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>pct_missing</th>\n",
       "      <td>60.690000</td>\n",
       "      <td>100.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>100.0</td>\n",
       "      <td>66.320000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>pct_zeros</th>\n",
       "      <td>39.306052</td>\n",
       "      <td>0.0</td>\n",
       "      <td>100.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.027590</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.004414</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>94.022867</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>8.639695e+01</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                  deleted  endTime  removed_set      setNumber  startTime  \\\n",
       "count        71232.000000      0.0     181224.0  181224.000000        0.0   \n",
       "mean             0.000000      NaN          0.0       6.493941        NaN   \n",
       "std              0.000000      NaN          0.0       5.946106        NaN   \n",
       "min              0.000000      NaN          0.0       1.000000        NaN   \n",
       "25%              0.000000      NaN          0.0       2.000000        NaN   \n",
       "50%              0.000000      NaN          0.0       5.000000        NaN   \n",
       "75%              0.000000      NaN          0.0       9.000000        NaN   \n",
       "max              0.000000      NaN          0.0     169.000000        NaN   \n",
       "pct_missing     60.690000    100.0          0.0       0.000000      100.0   \n",
       "pct_zeros       39.306052      0.0        100.0       0.000000        0.0   \n",
       "\n",
       "                  rpe_num     weight_lbs     StartMessg           RepN  \\\n",
       "count        61034.000000  181224.000000  181224.000000  181224.000000   \n",
       "mean             7.266047     237.509353   -3219.539150      24.543455   \n",
       "std              1.541093     106.705020     468.222607      24.490327   \n",
       "min              0.000000       1.000000   -3456.000000       1.000000   \n",
       "25%              6.500000     165.000000   -3456.000000       8.000000   \n",
       "50%              7.500000     225.000000   -3456.000000      18.000000   \n",
       "75%              8.000000     308.646800   -3456.000000      33.000000   \n",
       "max             10.000000    1000.000000   -1234.000000     403.000000   \n",
       "pct_missing     66.320000       0.000000       0.000000       0.000000   \n",
       "pct_zeros        0.027590       0.000000       0.000000       0.000000   \n",
       "\n",
       "                    AvgVel            ROM        PeakVel     PeakVelLoc  \\\n",
       "count        181224.000000  181224.000000  181224.000000  181224.000000   \n",
       "mean              0.468691     462.169144       0.759044      72.447121   \n",
       "std               0.223723     155.470025       0.368769      19.757754   \n",
       "min               0.039064     150.000000       0.034893       6.000000   \n",
       "25%               0.309058     328.000000       0.473064      65.000000   \n",
       "50%               0.421928     443.000000       0.706708      79.000000   \n",
       "75%               0.580345     595.000000       0.966042      87.000000   \n",
       "max               2.960175    1761.000000       5.329412      99.000000   \n",
       "pct_missing       0.000000       0.000000       0.000000       0.000000   \n",
       "pct_zeros         0.000000       0.000000       0.000000       0.000000   \n",
       "\n",
       "                 PeakAccel        RepDur     TimeBWReps   TimeRepComp  \\\n",
       "count        181224.000000  1.812240e+05  181224.000000  1.812240e+05   \n",
       "mean           -467.837734  1.129646e+06       0.351256  1.706150e+09   \n",
       "std            1097.396166  4.931590e+05       5.669843  1.020477e+09   \n",
       "min           -9999.000000  5.000000e+00       0.000000 -9.876000e+03   \n",
       "25%               1.182262  7.780158e+05       0.000000  9.008389e+08   \n",
       "50%               4.288830  1.065460e+06       0.000000  1.486152e+09   \n",
       "75%               9.837069  1.407044e+06       0.000000  2.375322e+09   \n",
       "max           24150.160000  6.072204e+06    2295.000000  4.294947e+09   \n",
       "pct_missing       0.000000  0.000000e+00       0.000000  0.000000e+00   \n",
       "pct_zeros         0.004414  0.000000e+00      94.022867  0.000000e+00   \n",
       "\n",
       "              TimeRepWait      SlowAllow      Backlight       MinAllow  \\\n",
       "count        1.812240e+05  181224.000000  181224.000000  181224.000000   \n",
       "mean         2.795605e+05  129349.114167    9952.900095  149247.324686   \n",
       "std          6.925327e+06    9147.219381     662.377762   10577.566980   \n",
       "min          0.000000e+00      76.039300      76.039900      76.029900   \n",
       "25%          0.000000e+00  130000.000000   10000.000000  150000.000000   \n",
       "50%          0.000000e+00  130000.000000   10000.000000  150000.000000   \n",
       "75%          0.000000e+00  130000.000000   10000.000000  150000.000000   \n",
       "max          8.366950e+08  130000.000000   10000.000000  150000.000000   \n",
       "pct_missing  0.000000e+00       0.000000       0.000000       0.000000   \n",
       "pct_zeros    8.639695e+01       0.000000       0.000000       0.000000   \n",
       "\n",
       "                  setID_ix       RepCount  \n",
       "count        181224.000000  181224.000000  \n",
       "mean          24055.649169       3.375546  \n",
       "std           13876.404474       3.284430  \n",
       "min               1.000000       1.000000  \n",
       "25%           12065.000000       1.000000  \n",
       "50%           24033.000000       3.000000  \n",
       "75%           36057.000000       4.000000  \n",
       "max           48128.000000      87.000000  \n",
       "pct_missing       0.000000       0.000000  \n",
       "pct_zeros         0.000000       0.000000  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def get_descr(data, features=None):\n",
    "    \"\"\" \n",
    "    Prints out descriptive information for an input feature:\n",
    "    - descriptive statistics\n",
    "    - number of lowest occuring values\n",
    "    - number of highest occuring values\n",
    "    - historgram\n",
    "    \n",
    "    Args:\n",
    "    feature: series or array\n",
    "  \n",
    "    Kwargs:\n",
    "    ends: int, number of lowest/highest values to analyze\n",
    "    \"\"\"\n",
    "    \n",
    "    \n",
    "    if features is None:\n",
    "        features = data.columns\n",
    "    data_descr = data[features].describe()\n",
    "    pct_missing = pd.Series(100-np.around(data_descr.loc['count',]/data.shape[0]*100,2))\n",
    "    pct_missing.name = 'pct_missing'\n",
    "    pct_zero = pd.Series([100*sum(data[v] == 0)/data.shape[0] for v in features])\n",
    "    pct_zero.name = 'pct_zeros'\n",
    "    pct_zero.index = pct_missing.index\n",
    "    data_descr = data_descr.append(pct_missing)\n",
    "    data_descr = data_descr.append(pct_zero)\n",
    "    \n",
    "    return data_descr\n",
    "\n",
    "\n",
    "get_descr(df_main, features=['deleted', 'endTime', 'removed_set', 'setNumber', 'startTime', 'rpe_num'\n",
    "                             , 'weight_lbs', 'StartMessg', 'RepN', 'AvgVel', 'ROM', 'PeakVel', 'PeakVelLoc'\n",
    "                             , 'PeakAccel', 'RepDur', 'TimeBWReps', 'TimeRepComp', 'TimeRepWait', 'SlowAllow'\n",
    "                             , 'Backlight', 'MinAllow', 'setID_ix', 'RepCount'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "9jiO2BfX_9XM"
   },
   "source": [
    "## 2.3 Target label variable\n",
    "\n",
    "Our primary variable of interest is `exercise_clean`. As we can see based on table below, there are three possible values:\n",
    "\n",
    " - 3 *clean* values: **bench**, **deadlift**, **squat** where user entries for `exercise` were clean and specific "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "fUVaWgVs_9XN",
    "outputId": "34d3d53a-6257-4106-be2a-c6715454b584"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Exercise</th>\n",
       "      <th>Row count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>81938</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>32540</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>66746</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Exercise  Row count\n",
       "0         0      81938\n",
       "1         1      32540\n",
       "2         2      66746"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def get_label_distribution(label):\n",
    "    \"\"\" \n",
    "    Produces the distribution of labels within our main dataset\n",
    "    \n",
    "    Args:\n",
    "    label: the labels  within our dataset\n",
    "  \n",
    "    Kwargs:\n",
    "    None\n",
    "    \n",
    "    Returns:\n",
    "    Dataframe containing the exercise label and its respective count\n",
    "    \"\"\"\n",
    "    i, j = np.unique(label, return_counts = True)\n",
    "    return pd.DataFrame({\"Exercise\": i, \"Row count\": j})\n",
    "        \n",
    "get_label_distribution(df_main['exercise_clean'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "tm3DTFEc_9XQ"
   },
   "source": [
    "## 2.4 Analyze Key Features\n",
    "\n",
    "We are going to focus our analysis on the following set of features:\n",
    "\n",
    "**Rep specific:**\n",
    "\n",
    " - `AvgVel`: Average velocity (m/s)\n",
    " - `ROM`: Range of motion (mm)\n",
    " - `PeakVel`: Peak Velocity (m/s) \n",
    " - `PeakVelLoc`: Peak Velocity Location (%)\n",
    " - `PeakAccel`: Peak Acceleration (potentially drop as it has tracking issues)\n",
    " - `RepDur`: Duration of Rep (microseconds)\n",
    "\n",
    "**Set specific:**\n",
    "\n",
    " - `weight_lbs`: weight (in lbs) for each set (user entry).\n",
    "\n",
    "The following *rep* specific metrics have inconsistent data and are be dropped for now:\n",
    " - `TimeBWReps`: Time between Reps (minutes)\n",
    " - `TimeRepComp`: Timestamp of Rep Completion (microseconds)\n",
    " - `TimeRepWait`: Timestamp of time \"waiting\" in rep (microseconds)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "y9Nb0xCS_9XQ",
    "outputId": "82596097-f7fb-4ef2-bdd1-41ad48943222"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AvgVel</th>\n",
       "      <th>ROM</th>\n",
       "      <th>PeakVel</th>\n",
       "      <th>PeakVelLoc</th>\n",
       "      <th>RepDur</th>\n",
       "      <th>weight_lbs</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>181224.000000</td>\n",
       "      <td>181224.000000</td>\n",
       "      <td>181224.000000</td>\n",
       "      <td>181224.000000</td>\n",
       "      <td>1.812240e+05</td>\n",
       "      <td>181224.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.468691</td>\n",
       "      <td>462.169144</td>\n",
       "      <td>0.759044</td>\n",
       "      <td>72.447121</td>\n",
       "      <td>1.129646e+06</td>\n",
       "      <td>237.509353</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.223723</td>\n",
       "      <td>155.470025</td>\n",
       "      <td>0.368769</td>\n",
       "      <td>19.757754</td>\n",
       "      <td>4.931590e+05</td>\n",
       "      <td>106.705020</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.039064</td>\n",
       "      <td>150.000000</td>\n",
       "      <td>0.034893</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>5.000000e+00</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.309058</td>\n",
       "      <td>328.000000</td>\n",
       "      <td>0.473064</td>\n",
       "      <td>65.000000</td>\n",
       "      <td>7.780158e+05</td>\n",
       "      <td>165.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.421928</td>\n",
       "      <td>443.000000</td>\n",
       "      <td>0.706708</td>\n",
       "      <td>79.000000</td>\n",
       "      <td>1.065460e+06</td>\n",
       "      <td>225.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.580345</td>\n",
       "      <td>595.000000</td>\n",
       "      <td>0.966042</td>\n",
       "      <td>87.000000</td>\n",
       "      <td>1.407044e+06</td>\n",
       "      <td>308.646800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>2.960175</td>\n",
       "      <td>1761.000000</td>\n",
       "      <td>5.329412</td>\n",
       "      <td>99.000000</td>\n",
       "      <td>6.072204e+06</td>\n",
       "      <td>1000.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              AvgVel            ROM        PeakVel     PeakVelLoc  \\\n",
       "count  181224.000000  181224.000000  181224.000000  181224.000000   \n",
       "mean        0.468691     462.169144       0.759044      72.447121   \n",
       "std         0.223723     155.470025       0.368769      19.757754   \n",
       "min         0.039064     150.000000       0.034893       6.000000   \n",
       "25%         0.309058     328.000000       0.473064      65.000000   \n",
       "50%         0.421928     443.000000       0.706708      79.000000   \n",
       "75%         0.580345     595.000000       0.966042      87.000000   \n",
       "max         2.960175    1761.000000       5.329412      99.000000   \n",
       "\n",
       "             RepDur     weight_lbs  \n",
       "count  1.812240e+05  181224.000000  \n",
       "mean   1.129646e+06     237.509353  \n",
       "std    4.931590e+05     106.705020  \n",
       "min    5.000000e+00       1.000000  \n",
       "25%    7.780158e+05     165.000000  \n",
       "50%    1.065460e+06     225.000000  \n",
       "75%    1.407044e+06     308.646800  \n",
       "max    6.072204e+06    1000.000000  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_main[['AvgVel', 'ROM', 'PeakVel', 'PeakVelLoc', 'RepDur', 'weight_lbs']].describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "8b6euZ8p_9XT"
   },
   "source": [
    "# 3. Feature Engineer the Data\n",
    "\n",
    "## 3.1 Trim Data to Key Features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "RZE1X1UP_9XY",
    "outputId": "712d5748-420a-48e8-95eb-8f678e340ec4"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Data shape after feature selection: (181224, 9)\n"
     ]
    }
   ],
   "source": [
    "# Declare lists containing columns of importance. This includes our features\n",
    "# for our models, the primary label, and supplementary ID columns needed for\n",
    "# post-model tasks.\n",
    "key_features = ['AvgVel', 'ROM', 'PeakVel', 'PeakVelLoc', 'RepDur', 'weight_lbs']\n",
    "primary_label = 'exercise_clean'\n",
    "id_columns = ['setID_ix', 'RepCount']\n",
    "\n",
    "# Keep only the columns we care about\n",
    "df_main = df_main[[primary_label] + key_features + id_columns]\n",
    "print(\"Data shape after feature selection: {0}\".format(df_main.shape))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "S2Tdrf2O_9Xh"
   },
   "source": [
    "## 3.2 Factorize Target Variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "ZR6RqZuR_9Xh",
    "outputId": "732d33f1-668b-4132-971a-b382ced36d93"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['0', '1', '2']\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Exercise</th>\n",
       "      <th>Row count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>81938</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>32540</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>66746</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Exercise  Row count\n",
       "0         0      81938\n",
       "1         1      32540\n",
       "2         2      66746"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Factorize the primary label column\n",
    "df_main[primary_label], labels_names = pd.factorize(df_main[primary_label], sort=True)\n",
    "labels_names = list(map(str, labels_names))\n",
    "\n",
    "# Ensure that the new labels and counts align accurately\n",
    "# with what was produced earlier\n",
    "print(labels_names)\n",
    "get_label_distribution(df_main[primary_label])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.3 Describe Final Dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "iqfiXATr_9Xb",
    "outputId": "7605fccc-9564-4921-dd14-f3cc23c39315"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>exercise_clean</th>\n",
       "      <th>AvgVel</th>\n",
       "      <th>ROM</th>\n",
       "      <th>PeakVel</th>\n",
       "      <th>PeakVelLoc</th>\n",
       "      <th>RepDur</th>\n",
       "      <th>weight_lbs</th>\n",
       "      <th>setID_ix</th>\n",
       "      <th>RepCount</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>181224.000000</td>\n",
       "      <td>181224.000000</td>\n",
       "      <td>181224.000000</td>\n",
       "      <td>181224.000000</td>\n",
       "      <td>181224.000000</td>\n",
       "      <td>1.812240e+05</td>\n",
       "      <td>181224.000000</td>\n",
       "      <td>181224.000000</td>\n",
       "      <td>181224.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.916170</td>\n",
       "      <td>0.468691</td>\n",
       "      <td>462.169144</td>\n",
       "      <td>0.759044</td>\n",
       "      <td>72.447121</td>\n",
       "      <td>1.129646e+06</td>\n",
       "      <td>237.509353</td>\n",
       "      <td>24055.649169</td>\n",
       "      <td>3.375546</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.901898</td>\n",
       "      <td>0.223723</td>\n",
       "      <td>155.470025</td>\n",
       "      <td>0.368769</td>\n",
       "      <td>19.757754</td>\n",
       "      <td>4.931590e+05</td>\n",
       "      <td>106.705020</td>\n",
       "      <td>13876.404474</td>\n",
       "      <td>3.284430</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.039064</td>\n",
       "      <td>150.000000</td>\n",
       "      <td>0.034893</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>5.000000e+00</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.309058</td>\n",
       "      <td>328.000000</td>\n",
       "      <td>0.473064</td>\n",
       "      <td>65.000000</td>\n",
       "      <td>7.780158e+05</td>\n",
       "      <td>165.000000</td>\n",
       "      <td>12065.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.421928</td>\n",
       "      <td>443.000000</td>\n",
       "      <td>0.706708</td>\n",
       "      <td>79.000000</td>\n",
       "      <td>1.065460e+06</td>\n",
       "      <td>225.000000</td>\n",
       "      <td>24033.000000</td>\n",
       "      <td>3.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>2.000000</td>\n",
       "      <td>0.580345</td>\n",
       "      <td>595.000000</td>\n",
       "      <td>0.966042</td>\n",
       "      <td>87.000000</td>\n",
       "      <td>1.407044e+06</td>\n",
       "      <td>308.646800</td>\n",
       "      <td>36057.000000</td>\n",
       "      <td>4.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>2.000000</td>\n",
       "      <td>2.960175</td>\n",
       "      <td>1761.000000</td>\n",
       "      <td>5.329412</td>\n",
       "      <td>99.000000</td>\n",
       "      <td>6.072204e+06</td>\n",
       "      <td>1000.000000</td>\n",
       "      <td>48128.000000</td>\n",
       "      <td>87.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       exercise_clean         AvgVel            ROM        PeakVel  \\\n",
       "count   181224.000000  181224.000000  181224.000000  181224.000000   \n",
       "mean         0.916170       0.468691     462.169144       0.759044   \n",
       "std          0.901898       0.223723     155.470025       0.368769   \n",
       "min          0.000000       0.039064     150.000000       0.034893   \n",
       "25%          0.000000       0.309058     328.000000       0.473064   \n",
       "50%          1.000000       0.421928     443.000000       0.706708   \n",
       "75%          2.000000       0.580345     595.000000       0.966042   \n",
       "max          2.000000       2.960175    1761.000000       5.329412   \n",
       "\n",
       "          PeakVelLoc        RepDur     weight_lbs       setID_ix  \\\n",
       "count  181224.000000  1.812240e+05  181224.000000  181224.000000   \n",
       "mean       72.447121  1.129646e+06     237.509353   24055.649169   \n",
       "std        19.757754  4.931590e+05     106.705020   13876.404474   \n",
       "min         6.000000  5.000000e+00       1.000000       1.000000   \n",
       "25%        65.000000  7.780158e+05     165.000000   12065.000000   \n",
       "50%        79.000000  1.065460e+06     225.000000   24033.000000   \n",
       "75%        87.000000  1.407044e+06     308.646800   36057.000000   \n",
       "max        99.000000  6.072204e+06    1000.000000   48128.000000   \n",
       "\n",
       "            RepCount  \n",
       "count  181224.000000  \n",
       "mean        3.375546  \n",
       "std         3.284430  \n",
       "min         1.000000  \n",
       "25%         1.000000  \n",
       "50%         3.000000  \n",
       "75%         4.000000  \n",
       "max        87.000000  "
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_main.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "I0RJyQEv_9Xv"
   },
   "source": [
    "# 4. Separate Data Into Train, Dev, & Test Sets\n",
    "\n",
    "For this project, we are splitting the dataset into the following three categories:\n",
    "1. Train - 80%\n",
    "2. Dev - 10%\n",
    "3. Test - 10%\n",
    "\n",
    "Train will be used to train the model. We will use Dev to test our pipeline and model modifications. We will use Test as a final checkpoint of model performance.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "QDtlP9dy_9Xx"
   },
   "source": [
    "## 4.1 Shuffle and Partition Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "sfAVULDS_9X0",
    "outputId": "b8759afd-81d1-4c57-bc39-a21d098ab3ed"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dev data & labels shapes: (18124, 9), 18124\n",
      "Test data & labels shapes: (18123, 9), 18123\n",
      "Train data & labels shapes: (144977, 9), 144977\n"
     ]
    }
   ],
   "source": [
    "def partition_data(data, labels, dev_pct, test_pct, seed):\n",
    "    \"\"\"\n",
    "    Partitions input dataset into training, development and testing. Ensure\n",
    "    that reps within the same set are partitioned into the same dataset.\n",
    "    \n",
    "    Args:\n",
    "        data: feature dataset\n",
    "        labels: array of target labels\n",
    "        dev_pct: float, percent of data to allocate to development\n",
    "        test_pct: float, percent of data to allocate to testing\n",
    "        seed: int, random seed\n",
    "        \n",
    "    Returns:\n",
    "        6-tuple with 3 feature datasets and 3 target label arrays\n",
    "    \"\"\"\n",
    "    \n",
    "    # Group the data by their sets. Shuffle them by this grouping\n",
    "    # and concatenate them back together\n",
    "    data_shuffled = [data for _, data in data.groupby('setID_ix')]\n",
    "    random.seed(seed)\n",
    "    random.shuffle(data_shuffled)\n",
    "    data_shuffled = pd.concat(data_shuffled)\n",
    "\n",
    "    # Iterate through the shuffled data and grab the associated\n",
    "    # label for each row. Append that label into a new list. This\n",
    "    # will produce a list of labels in the same order as the data\n",
    "    labels_shuffled = []\n",
    "    for row in data_shuffled.itertuples():\n",
    "        labels_shuffled.append(labels[row.Index])\n",
    "    \n",
    "    data_shuffled = data_shuffled.reset_index(drop=True)\n",
    "\n",
    "    # Calculate the indicies at which we partition the data\n",
    "    # for the dev and test datasets. If it's next neighbor belongs\n",
    "    # to the same set, move the partition index to include that rep.\n",
    "    # Continue to do this until we find a clean partition cutoff.\n",
    "    dev_cnt = int(np.round(len(data_shuffled) * dev_pct))\n",
    "    while (data_shuffled.iloc[dev_cnt]['setID_ix'] == data_shuffled.iloc[dev_cnt + 1]['setID_ix']):\n",
    "        dev_cnt += 1\n",
    "        \n",
    "    test_cnt = int(np.round(len(data_shuffled) * test_pct))\n",
    "    while (data_shuffled.iloc[dev_cnt + test_cnt]['setID_ix'] == data_shuffled.iloc[dev_cnt + test_cnt + 1]['setID_ix']):\n",
    "        test_cnt += 1\n",
    "    \n",
    "    # Slice the data using the partition cutoffs and return them\n",
    "    dev_data, dev_labels = data_shuffled[:dev_cnt], labels_shuffled[:dev_cnt]\n",
    "    test_data, test_labels = data_shuffled[dev_cnt:dev_cnt + test_cnt], labels_shuffled[dev_cnt:dev_cnt + test_cnt]\n",
    "    train_data, train_labels = data_shuffled[dev_cnt + test_cnt:], labels_shuffled[dev_cnt + test_cnt:]\n",
    "    return train_data, test_data, dev_data, train_labels, test_labels, dev_labels\n",
    "\n",
    "\n",
    "train_data, test_data, dev_data, train_labels, test_labels, dev_labels = partition_data(df_main, df_main[primary_label], .1, .1, 5)\n",
    "\n",
    "print(f'Dev data & labels shapes: {dev_data.shape}, {len(dev_labels)}')\n",
    "print(f'Test data & labels shapes: {test_data.shape}, {len(test_labels)}')\n",
    "print(f'Train data & labels shapes: {train_data.shape}, {len(train_labels)}')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "bNH3YKHx_9X4"
   },
   "source": [
    "## 4.2 Standardize Training Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "oIq-u_-A_9X4",
    "outputId": "2c384974-632f-4658-9bd8-fb7b71a36f36"
   },
   "outputs": [],
   "source": [
    "def standardize_data(train_data, test_data, dev_data, data_fit):\n",
    "    \"\"\"\n",
    "    Standardize dataset and apply it to the input datasets\n",
    "    \n",
    "    Args:\n",
    "        train_data: train dataset to apply standardization to\n",
    "        test_data: test dataset to apply standardization to\n",
    "        dev_data: dev dataset to apply standardization to\n",
    "        data_fit: dataset used to fit standardization\n",
    "    \n",
    "    Returns:\n",
    "        None\n",
    "    \"\"\"\n",
    "    \n",
    "    scaler = preprocessing.StandardScaler().fit(data_fit[key_features])\n",
    "    train_data[key_features] = scaler.transform(train_data[key_features])\n",
    "    test_data[key_features] = scaler.transform(test_data[key_features])\n",
    "    dev_data[key_features] = scaler.transform(dev_data[key_features])\n",
    "\n",
    "\n",
    "standardize_data(train_data, test_data, dev_data, train_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "lV4SOo-U_9X6"
   },
   "source": [
    "# 5. Train and Test the Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "tXkIlqs8_9X6"
   },
   "source": [
    "## 5.1 Define general function for classfication"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "8yy1QXdE_9X9"
   },
   "outputs": [],
   "source": [
    "# borrowed from scikit learn tutorial\n",
    "# http://scikit-learn.org/stable/auto_examples/model_selection/plot_confusion_matrix.html#sphx-glr-auto-examples-model-selection-plot-confusion-matrix-py\n",
    "\n",
    "np.set_printoptions(precision=2)\n",
    "\n",
    "def plot_confusion_matrix(true_labs, pred_labs,\n",
    "                          lab_names,\n",
    "                          normalize=False,\n",
    "                          title='Confusion matrix',\n",
    "                          cmap=plt.cm.Blues):\n",
    "    \"\"\"\n",
    "    This function prints and plots the confusion matrix.\n",
    "    Normalization can be applied by setting `normalize=True`.\n",
    "    \"\"\"\n",
    "    \n",
    "    cm = confusion_matrix(true_labs, pred_labs)\n",
    "    if normalize:\n",
    "        cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\n",
    "    \n",
    "    plt.figure()\n",
    "    plt.imshow(cm, interpolation='nearest', cmap=cmap)\n",
    "    plt.title(title)\n",
    "    plt.colorbar()\n",
    "    tick_marks = np.arange(len(lab_names))\n",
    "    plt.xticks(tick_marks, lab_names, rotation=45)\n",
    "    plt.yticks(tick_marks, lab_names)\n",
    "\n",
    "    fmt = '.2f' if normalize else 'd'\n",
    "    thresh = cm.max() / 2.\n",
    "    for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):\n",
    "        plt.text(j, i, format(cm[i, j], fmt),\n",
    "                 horizontalalignment=\"center\",\n",
    "                 color=\"white\" if cm[i, j] > thresh else \"black\")\n",
    "\n",
    "    plt.tight_layout()\n",
    "    plt.ylabel('True label')\n",
    "    plt.xlabel('Predicted label')\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "QuNXlNMk_9X-"
   },
   "outputs": [],
   "source": [
    "def get_performance(true_labs, pred_labs, lab_names):\n",
    "    \n",
    "    \"\"\"Reports out several model evaluation tools: classification report,\n",
    "    confusion matrix, and confusion matrix plot.\n",
    "    \n",
    "    Args:\n",
    "    true labels (array)\n",
    "    predicted labels (array)\n",
    "    label names (array)\n",
    "    \n",
    "    Returns:  none, prints output\n",
    "    \"\"\"\n",
    "\n",
    "    print(\"=\"*50)\n",
    "    print(\"Classification report by exercise:\")\n",
    "    print(classification_report(true_labs, pred_labs, target_names=lab_names))\n",
    "    print(\"=\"*50)\n",
    "    print(\"Confusion matrix:\")\n",
    "    print(pd.DataFrame(confusion_matrix(true_labs, pred_labs), columns=lab_names, index=lab_names))\n",
    "    plot_confusion_matrix(true_labs, pred_labs, lab_names, normalize=True, title='Normalized Confusion Matrix')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "z6t8UffL_9YA"
   },
   "source": [
    "## 5.2 KNN\n",
    "\n",
    "**Key findings**:\n",
    "\n",
    " - It appears that k value doesn't really matter\n",
    " - Validates well on development dataset with clean labels"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "xJPUeyvu_9YB"
   },
   "source": [
    "### 5.2.1 What k to use?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "j_JUc3Hi_9YC",
    "outputId": "08746c39-ccc8-4022-ab89-182bcfb9eda5"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy on dev data, k=1 93.90%\n",
      "Accuracy on dev data, k=6 94.76%\n",
      "Accuracy on dev data, k=11 94.83%\n",
      "Accuracy on dev data, k=16 94.78%\n",
      "Accuracy on dev data, k=21 94.73%\n",
      "Accuracy on dev data, k=26 94.69%\n"
     ]
    }
   ],
   "source": [
    "# check accuracy of dev data with a range of k values\n",
    "for k in range(1, 30, 5):\n",
    "    \n",
    "    knn = KNeighborsClassifier(k)\n",
    "    knn.fit(train_data[key_features], train_labels)\n",
    "    print(f\"Accuracy on dev data, k={k} {knn.score(dev_data[key_features], dev_labels)*100:.2f}%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "oeSjKDtW_9YE"
   },
   "source": [
    "### 5.2.2 Train the model with k = 10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total prediction runtime:  0.8019020557403564\n",
      "==================================================\n",
      "Classification report by exercise:\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "          0       0.97      0.99      0.98      8270\n",
      "          1       0.90      0.88      0.89      3303\n",
      "          2       0.95      0.94      0.94      6551\n",
      "\n",
      "avg / total       0.95      0.95      0.95     18124\n",
      "\n",
      "==================================================\n",
      "Confusion matrix:\n",
      "      0     1     2\n",
      "0  8171    41    58\n",
      "1   135  2893   275\n",
      "2   130   285  6136\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUgAAAEmCAYAAAAA6gkZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xd8FHX+x/HXJwlFQbooJBQpUk8BAU/FjoqKYBcLil1Psfs7T89ynme9s9zpnWdFsSMiVbHiqSfSLCcgCgKSgAqIIChgwuf3x0xwk+wmG9jsLMn76WMf7ux89zufmSyf/X6/M/Ndc3dERKSsrKgDEBHJVEqQIiIJKEGKiCSgBCkikoASpIhIAkqQIiIJKEFuA8zsJjN7Knze2szWmll2irexyMz6p7LOSmz7FjNbYWbfbEUdVXJc0s3MrjWzR6KOQwJKkGxODt+aWb2Y184xsykRhhWXu3/t7vXdvSid2zWzvmY2ycx+MLPvzWyamZ2ZgnpbAVcCXd195y2tpyqPi5l5+PnIiXktx8y+M7OkLiQ2swPMLL+icu5+q7ufszXxSuooQf4qB7h0ayuxQLU6rma2F/AW8A7QAWgKXAgcnoLq2wAr3f27FNRVlX6g5P4eAaxK5QZiE7Bkhmr1D3kr3QVcZWaN4q00s73NbLqZrQ7/v3fMuilm9hczex/4CWgXvnaLmf037PqNN7OmZva0ma0J62gbU8d9ZrYkXDfTzPZNEEfbsEWTY2Z7hXUXP9ab2aKwXJaZXWNmC8xspZm9YGZNYuoZamaLw3XXJXFsnnD3O9x9hQdmuvuJMfWda2bzw9blODNrGbPOzewCM/vSzFaZ2QPhF0l/4HWgZRj/iHgtrdjuf9iSnREep2/N7O7SxyVcbhnG8X0Y17kx9d0UHo8nzexHM5ttZr0rOAYjgdNjlk8HniwV55lmNjes8yszOz98vR7wSsx+rg3ju8nMXjSzp8xsDTDMSg6nnBTW0yBcPtzMvjGzHSuIVVLF3Wv8A1gE9AdeAm4JXzsHmBI+b0LQWhhK0NI8OVxuGq6fAnwNdAvX1wpfmw+0BxoCc4Avwu3kEPzjejwmhtMIWmY5BF3Ob4C64bqbgKfC520BB3JK7UPxNm8Lly8DpgJ5QB3g38Cz4bquwFpgv3Dd3UAh0D/OsdkeKAIOLOf4HQSsAHqF9f0D+E/MegcmAI2A1sByYEC47gAgP6ZsieXYv0/4/ANgaPi8PvDbeMeFoLX7T6Au0CPc5sExx3M9QSswG7gNmFrO/jnQHfg23IdG4fPugMeUOzL8exuwP8GXZa9y9usm4BfgaILGynaxf+uwzNPAiPCzsRQYGPW/l5r0UAuypBuA4XG+oY8EvnT3ke5e6O7PAp8DR8WUGeHus8P1v4SvPe7uC9x9NUELYoG7v+HuhcAooGfxm939KXdfGb7/bwSJplMlYv87sA4obg2eD1zn7vnuvoHgH97xYQvreGCCu/8nXHc9sClBvY0J/vEuK2fbpwKPufussL4/AHvFtpCB2939B3f/GnibIGltiV+ADmbWzN3XuvvU0gXCcc1+wO/dfb27fww8QvAFV+w9d5/kwZjlSGD3Cra7HhgPnAQMAcaFr23m7hPDv7e7+zvAa0DcnkCMD9z9ZXff5O4/x1l/EcEX0BRgvLtPqKA+SSElyBju/hlBS+eaUqtaAotLvbYYyI1ZXhKnym9jnv8cZ7l+8YKZXRl2z1ab2Q8Erc5mycQdduUOAE5x9+JE1wYYY8FJlR+AuQQtwZ3C/dkcr7uvA1YmqH4VQfJsUU4IJY6Pu68N64s9PrFnqH8iZt8r6WxgV+DzcJhiYIJ4vnf3H2NeK/33Kh1P3STGAJ8k6FqX6V7D5i7w1LBb/wNBC7Wiv2G8z81m7v4DwZdpd+BvFdQlKaYEWdaNwLmU/Me0lCDhxGoNFMQsb/G0SOF44++BE4HG7t4IWE3QVUvmvX8GBoct1WJLgMPdvVHMo667FxC0BlvF1LE9QReuDHf/iaBbe1w5YZQ4PuGYW1NKHp9krSPo1hfXlQ1sbtG7+5fufjLQHLgDeNFirj6IiaeJme0Q81rpv9eWeJfgi2In4L3YFWZWBxgN/BXYKfwbTuLXv2Giz0e5nxsz6wGcBTxL0EuQNFKCLMXd5wPPA5fEvDwJ2NXMTglPjpxEMI6Xqu7ODgRjgMuBHDO7AWhQ0ZvCruTzwOnu/kWp1Q8CfzGzNmHZHc1scLjuRWCgmfUzs9rAzZT/Wfg/ghMIV5tZ07C+3c3suXD9M8CZZtYjTBS3Ah+6+6IK97ysLwhac0eaWS3gjwTDDcX7fJqZ7Ri2lH8IXy5xaY+7LwH+C9xmZnXNbDeClufTWxBPbL1OMKwyKHweq3YY53Kg0MwOBw6NWf8t0NTMGia7PTOrCzwFXAucCeSa2e+2YhekkpQg47sZ2NwqcfeVwECCkycrCRLGQHdfkaLtTSYYo/yCoCu4ngq6XqGDgZ0JWlHFZ0dnh+vuIxgne83MfiQ4YbNnuD+zCca2niFoTa4CEl6j5+7/JRgHOwj4ysy+Bx4i+OLA3d8kGMccHdbXnmCcrtLCVvDvCMYMCwhalLGxDQBmm9nacB+HuPv6MhUFJ9LaErQmxwA3uvvrWxJTqfhmh8ev9Os/EnypvkBwPE8hOP7F6z8naAV+FQ57tCxdRxy3EZzY+Vc4tnsacIuZddza/ZDkWNkvQhERAbUgRUQSUoIUEUlACVJEJAElSBGRBDLq5njL2c6t9g4VF5TNenZpHXUIUgMsXryIFStWVHhdbmVkN2jjXhjv5qH4/Oflk919QCpjqEhmJcjaO1Cn04kVF5TN3v/w/qhDkBpgnz0rmsuj8rzw50r9e1//8QNJ3VmWShmVIEWkJjHI8JkBlSBFJBoGWEp77SmnBCki0VELUkQkHoOszP4JISVIEYmOutgiInEY6mKLiMRnakGKiCSkFqSISAJqQYqIxKMLxUVE4tOF4iIi5VALUkQkHoNsXSguIlKWroMUESmHxiBFROLRWWwRkcTUghQRSUAtSBGROEz3YouIJKYWpIhIAmpBiojEo7PYIiLxGfrJBRGR+NSCFBFJTGOQIiIJqAUpIpKAWpAiInGYxiBFRBJTC1JEJD7L8ASZ2e3bNDhk7y58MuZ6Pht7I1edeUiZ9a1bNGbSg8OZ9vwfmPzwpeQ2b7R53S2XDGbGqGuZMepajj+0VzrDjtRrk19lt26d6Na5A3fdeXuZ9Rs2bOC0U06iW+cO7Lv3nixetGjzurvuuI1unTuwW7dOvP7a5DRGHa0tPWYrV67ksP4H0qxRfS675OI0R121gp+ksaQfUajRCTIry7j3mhMZfPE/6XncLZwwYA86t9u5RJnbLj+GpydOo+9Jt3HrQ69w8/BBAAzo140eXVqx55Db2W/oX7nsjP7sUK9uFLuRVkVFRVx2yUWMHf8KH306h1HPPcvcOXNKlBnx2KM0btSY2Z/PZ/ill3Pdtb8HYO6cOYx6/jlmfTKbcRNe5dLhv6OoqCiK3UirrTlmdevW5Yab/sxtd/w1itCrlhmWlfwjCjU6Qfbp3pYFS1awqGAlvxQWMWryLAYesFuJMp3btWDKh/MAeGf6Fww84DcAdGm3M+/O/JKiok38tH4j//sin0P37pL2fUi36dOm0b59B3Zp147atWtzwklDmDB+bIkyE8aP5dShZwBw7HHHM+WtN3F3JowfywknDaFOnTq03WUX2rfvwPRp06LYjbTammNWr1499unXj7p1q+eXr1qQGaxl84bkf7tq83LBt6vI3bFhiTL/+6KAow/uAcDgg3anQf3taNKwHp9+UcBh+3Rlu7q1aNqoHvv33pW8nRunNf4oLF1aQF5eq83Lubl5FBQUlC3TKiiTk5NDg4YNWblyJQUFZd+7dGnJ91ZHW3PMqrtMT5BVepLGzAYA9wHZwCPuXnbwJUJG2YPupZb/cM8Y7vn9CZw2aE/enzWfgm9XUVhUxJtTP2ePbm14e8SVrFi1lg8/XUhh4ab0BB4h99JHqOxAe8IySby3OtqqY1bNZfo+VlmCNLNs4AHgECAfmG5m49x9TvnvTJ+C734gb6dfW325OzVm6fLVJcosW76aIVc9AkC97Wpz9ME9WLN2PQB3PjqZOx8NTjSMuHUY85d8l6bIo5Obm0d+/pLNywUF+bRs2bJsmSVLyMvLo7CwkDWrV9OkSRNy88q+t0WLku+tjrbmmFVrFj4yWFV2sfsC8939K3ffCDwHDK7C7VXajNmL6dB6R9q0bEqtnGxOOKwXE6d8WqJM00b1Nn/LXX3WYTwxdioQnOBp0rAeAN07tqR7x5a88cHn6d2BCPTu04f5879k0cKFbNy4kVHPP8eRAweVKHPkwEE8PfIJAF4a/SL7H3gQZsaRAwcx6vnn2LBhA4sWLmT+/C/p07dvFLuRVltzzKozI/nudXXsYucCS2KW84E9Sxcys/OA8wCoVb8KwymrqGgTl9/xAuP/eRHZWcYTY6cy96tvuP7CI5k152smvvM/9uvdkZuHD8Id3ps1n8tueyEINSebNx67DIAf167nrOueoKio+nexc3JyuOe++znqyMMoKirijGFn0bVbN26+6QZ67dGbgUcNYthZZ3PWsKF069yBxo2bMPLp5wDo2q0bx51wIj1360pOTg73/v0BsjP8h+NTYWuOGUCnDm35cc0aNm7cyPhxLzNh0mt06do1wj1KnUz/ErB4Yx8pqdjsBOAwdz8nXB4K9HX34Ynek7V9c6/T6cQqiae6WjX9/qhDkBpgnz17M3PmjJRms5ym7bzBEbckXX7VU6fOdPfeqYyhIlXZgswHWsUs5wFLq3B7IrKNyfQWZFWOQU4HOprZLmZWGxgCjKvC7YnItsQq+aioOrMBZjbPzOab2TVx1rc2s7fN7CMz+9TMjqioziprQbp7oZldDEwmuMznMXefXVXbE5Fti2FkZaWmjZbkVTN/BF5w93+ZWVdgEtC2vHqr9DpId58UBiEiUkYKu9ibr5oJ6y2+aiY2QTrQIHzekCSG/DSbj4hEp3L5sZmZzYhZfsjdHwqfJ3PVzE3Aa2Y2HKgH9K9og0qQIhINq3QLckU5Z7HjVVT6Ep2TgRHu/jcz2wsYaWbd3T3h9XlKkCISmRR2sZO5auZsYACAu39gZnWBZkDCW+Bq9GQVIhKtFN5Jk8xVM18DB4fb7QLUBZaXV6lakCISieJbDVMh0VUzZnYzMMPdxwFXAg+b2eUE3e9hXsGdMkqQIhKdFF4nHu+qGXe/Ieb5HGCfytSpBCki0aj8SZq0U4IUkcgoQYqIJBDVb80kSwlSRCKjFqSISBxRToSbLCVIEYmMEqSISAJKkCIiiWR2flSCFJHoqAUpIhKPLhQXEYnPgAzPj0qQIhIVI0sXiouIxKcutohIPKYutohIXAbqYouIJKIWpIhIAhqDFBGJR2OQIiLxBddBZnaGVIIUkYhoujMRkYQyPD8qQYpIREyX+YiIxKUxSBGRcmR4flSCFJHoqAUpIpJAhufHzEqQu3duzdvv3xd1GNuUxsc9GHUI25yFT54VdQjbnMJNnvpKNWGuiEh8mjBXRCQhXSguIpJQhudHJUgRiYguFBcRiU8XiouIlEMJUkQkgQzPj0qQIhIdtSBFROLRjOIiIvGZroMUEUksw/MjWVEHICI1V5ZZ0o+KmNkAM5tnZvPN7JoEZU40szlmNtvMnqmoTrUgRSQyqWpBmlk28ABwCJAPTDezce4+J6ZMR+APwD7uvsrMmldUrxKkiETCDLJTdydNX2C+u38V1G3PAYOBOTFlzgUecPdVAO7+XUWVqostIpExs6QfQDMzmxHzOC+mqlxgScxyfvharF2BXc3sfTObamYDKoovYQvSzBqU90Z3X1NR5SIi5alkF3uFu/dOVFWc10pPYpkDdAQOAPKAd82su7v/kGiD5XWxZ4cbiN1w8bIDrct5r4hIuYzgUp8UyQdaxSznAUvjlJnq7r8AC81sHkHCnJ6o0oQJ0t1bJVonIpIKKZzMZzrQ0cx2AQqAIcAppcq8DJwMjDCzZgRd7q/KjS+ZLZvZEDO7NnyeZ2Z7VDJ4EZGSKjH+WNEF5e5eCFwMTAbmAi+4+2wzu9nMBoXFJgMrzWwO8DZwtbuvLK/eCs9im9n9QC1gP+BW4CfgQaBPRe8VESlPKi8Ud/dJwKRSr90Q89yBK8JHUpK5zGdvd+9lZh+FG/nezGonuwERkXgMkroAPErJJMhfzCyL8IyQmTUFNlVpVCJSI2R4fkxqDPIBYDSwo5n9CXgPuKNKoxKRGiFVY5BVpcIWpLs/aWYzgf7hSye4+2dVG5aIVHcpvpOmSiR7q2E28AtBN1t334hISmR2ekwi2ZnZdcCzQEuCiy+fMbM/VHVgIlL9bfNdbOA0YA93/wnAzP4CzARuq8rARKR6C85iRx1F+ZJJkItLlcuhgqvPRUQqFGHLMFnlTVZxD8GY40/AbDObHC4fSnAmW0Rkq2R4fiy3BVl8pno2MDHm9alVF46I1CTbbAvS3R9NZyAiUrNsC2OQyZzFbm9mz5nZp2b2RfEjHcGlwxuvvUqf3bvSq3sn7vlr2evfN2zYwFlDT6ZX9070328vvl68qMT6JUu+Jm/Hhvzj3r+lKeLoHdKzFZ/8cwifPXgyVx3Xo8z6Vs3q8+otR/HBPccz7b4TOGyPYGa8nOwsHr70QKbfdwIf3X8SVx3XM92hR+atNybTr3d39urZhX/cc1eZ9Rs2bOD8M09lr55dOOLgfiwJP2e//PILl1xwNgfu3Yt9++7G3+++M82RV61MP4udzDWNI4DHCRL+4cALwHNVGFPaFBUVcfXllzDq5QlMnfU/Ro96ns/nzilRZuSIx2jYqDGzPpvHhcMv46Y/lrzC6br/u5L+h1Y4MXG1kZVl3Ht+Pwb/aSI9L36eE/btQOdWjUuU+f2JvRj93gL2uvxFTv/rG9x3/r4AHLdPO+rUyqbPpaPY+4rRnHNYV1o33yGK3UiroqIirr3qUp5+cRzvfPgJL7/4PPM+n1uizLMjH6dho0Z88NFczvvdJdxy03UAjH95NBs3buDt/85i8pSpjHz8kc3Jc1tnBtlmST+ikEyC3N7dJwO4+wJ3/yNwYNWGlR4zZ0yjXfv2tN2lHbVr1+bY409k0oRxJcq8MnEcJ582FIDBxxzHO1PeIpgUBCaOG0ubXXahc5euaY89Kn06NmfBN2tY9O2P/FK4iVHvLmBg37YlyrhDg+2D+Uwabl+bZavWbX59+zo5ZGcZ29XJZmNhET/+tDHdu5B2H82cTtt27WnTNvicDT7uRCZPGl+izKuTxnPiycHnbODgY3n3nbdxd8yMn9ato7CwkPXrf6Z27VrUb1DuZP/bFLPkH1FIJkFusKB9u8DMLjCzo4AKfw1sW7Bs6VJyc3+dF7hlbh7LlpachHhpTJmcnBwaNGjI9ytXsm7dOu67+05+f+0N1CQtm9Yjf8XazcsFK9eS27ReiTJ/eW4GQ/bvyPxHT2PMDUdwxUPBRQ8v/fcrftpQyMIRp/PFI6dx78ufsGrthrTGH4VvlpX8nLVomcs3ywrKlGmZmwcUf84a8P33Kxk4+Fi2r1eP3Tu1oXf3Dlww/HIaN26S1virUnXoYl8O1AcuAfYh+GWwsyp6k5k9ZmbfmVnG3rdd3BKMVeYPkaDM7bfcxIXDL6N+/fpVFF1mivvDH6WO0Yn7duCpt+bR4eynOObmSTx6+UGYBa3Pok1OuzNH0uW8p7n06N1pu1P172LH/ZyVOpKJPosfzZxOVnY2H3++iGmfzOPf99/L4kXV5zLkTG9BJjNZxYfh0x+BoZWoewRwP/Bk5cNKj5a5uRQU/PpDaEsL8tm5RYu4ZXLz8igsLGTNmtU0btKEGdOnMXbMS9x43TWsXv0DWVlZ1KlTl/MuvCjdu5FWBSvXkdfs1y+F3Kb1Wfr9TyXKnHFIZwb/Kbgy7MN531K3Vg7NGtTlxP078Nqsryks2sTy1ev5YO437NGhOYu+/TGt+5BuLVqW/JwtW1rATi1alimztCCflrnFn7M1NG7chDEvPseBBx9KrVq1aLZjc/rsuTeffDSLNm3bpXs3Us6wjJ8PMmEL0szGmNlLiR4VVezu/wG+T2m0KdZrjz4smD+fxYsWsnHjRl568QUOP/KoEmUGHHEUzz41EoCxY0az3/4HYma88sY7fPr5Aj79fAEXXnQJV1x9TbVPjgAzvvyODi0a0qb5DtTKyeKEfdszcdqiEmWWLF/LAbsF3cVOeY2oWzub5avXk798LQfsFvwS5/Z1cujbqTnz8lelexfSrkev3ixcMJ+vw8/Z2NEvcNjhA0uUOezwgbzwbPA5mzD2JfrtdwBmRm5ea97/zxTcnZ/WrWPmjA/p0LFTFLuRepVoPWZiC/L+dAQQ/rbteQB5rdL7Q4k5OTncefd9HDfoCIqKijj19GF06dqNW2++kR69enPEwKMYOuwsLjj7DHp170Tjxo159Mln0hpjpina5Fz+0HuMv+lIsrOMJ96cx9wlq7j+lN7Mmr+cidMWc83jH/DPi/Zn+KDf4A7n3vc2AA9O+oyHLjmQmf84ETMY+eY8Pluc0d+hKZGTk8Otd93LyccNpKioiCGnDaNTl67c+Zc/sXvPXhx2xFGcPPRMhp9/Jnv17EKjxk148LEgWZ55zgVcdtG5HLBXT9ydIaeeTtfuv4l4j1In0y8Ut3hjHymr3KwtMMHduydTvmev3v72+x9WXFA2azHk4ahD2OYsfLLCIXQp5bAD9uKTj2amNJs179DdT7prVNLl7z+268xyfhe7SiQ7H6SISEoZmd+CVIIUkchs87caFjOzOpWp2MyeBT4AOplZvpmdXdngRKT6Kv7JhWQfUUjmd7H7Ao8CDYHWZrY7cI67Dy/vfe5+cmpCFJHqqjq0IP8ODARWArj7J1STWw1FJFrb8mU+xbLcfXGpwdSiKopHRGqIYLqzzG5CJpMgl4TdbDezbGA4UG2mOxOR6GT6T6QmkyAvJOhmtwa+Bd4IXxMR2SoZ3oBM6l7s74AhaYhFRGoQs8y/FzuZs9gPE/xYVwnufl6VRCQiNUaG58ekuthvxDyvCxwDLElQVkQkaZl+mU8yXeznY5fNbCTwepVFJCI1gkFkF4Ana0tuNdwFaJPqQESkhrFq0II0s1X8OgaZRTDH4zVVGZSI1AylZ1bPNOUmyPC3aHYHin9AY5NX5fxoIlJjbPO/ix0mwzHuXhQ+lBxFJGWyLPlHJPElUWaamfWq8khEpMbJ9F81TNjFNrMcdy8E+gHnmtkCYB1By9jdXUlTRLbYttDFLm8MchrQCzg6TbGISE2S4ll6zGwAcB+QDTzi7rcnKHc8MAro4+4zyquzvARpAO6+YMvCFREpX6puNQwn0nkAOATIB6ab2Th3n1Oq3A7AJUBSP35VXoLc0cyuSLTS3e9OZgMiIvGkuIvdF5jv7l8BmNlzwGBgTqlyfwbuBK5KptLyTtJkA/WBHRI8RES2gpFtyT+AZmY2I+YROx9ELiVvgc4PX/t1a2Y9gVbuPiHZCMtrQS5z95uTrUhEpDKCXzWs1FtWlPOzr/Fq2nxZopllAfcAwyqzwQrHIEVEqkRqr2/MB1rFLOcBS2OWdwC6A1PCS4Z2BsaZ2aDyTtSUlyAP3vJYRUQqlsL5IKcDHc1sF4I7/4YApxSvdPfVQLPiZTObAlxV0VnshGOQ7v79VgYsIpJQcRc7FT/aFV6zfTEwGZgLvODus83sZjMbtKUxbslsPiIiKZHKGcXdfRIwqdRrNyQoe0AydSpBikhkqsOM4iIiKWdUj181FBFJPSOySSiSpQQpIpHJ7PSoBCkiETEovkMmYylBikhkMjw/KkGKSFSimwg3WUqQIhIJncUWESmHWpAiIglkdnrMsARpBrWyM73RnVmWPXtu1CFsc1oM0Cx+lbVh/rLUV6rrIEVE4tMYpIhIOdSCFBFJYFv+2VcRkSoTdLEzO0MqQYpIZDK8h60EKSJRMUwtSBGR+NSCFBGJQ2OQIiKJJPFjXFFTghSRyChBiogkoJM0IiJxGLpQXEQkoVT+LnZVUIIUkcioiy0iEoe62CIiCelOGhGR+HQdpIhIYhmeH5UgRSQawRhkZqdIJUgRiUxmp0clSBGJUoZnSCVIEYmMutgiIglkdnpUghSRKGV4hlSCFJFIGLrVUEQkPl0oLiKSWIbnR7KiDkBEajCrxKOiqswGmNk8M5tvZtfEWX+Fmc0xs0/N7E0za1NRnUqQIhIRq9R/5dZklg08ABwOdAVONrOupYp9BPR2992AF4E7K4pQCVJEImOW/KMCfYH57v6Vu28EngMGxxZw97fd/adwcSqQV1GlNT5Bvj75VXp278xuXTryt7tuL7N+w4YNnH7qEHbr0pED+v2WxYsWAfDWG6/T77e96dtrN/r9tjdT3n4rzZFH543XXqVPj670+k0n7vnrHWXWb9iwgbNOP5lev+lE//334uvFi0qsX7Lka/KaN+Qf9/4tTRFH75C+HfjkqeF89swlXHVqvzLrW+/UkEn3nMG0xy9k8n3DyN2xQYn1O2xfhwWjr+Sey45IV8hVrjK96zA/NjOzGTGP82KqywWWxCznh68lcjbwSkUx1ugEWVRUxBWXXsxL4yYx45PZjHr+OebOnVOizBOPP0qjRo34dO6XXHTJZVx/XTC00bRZM0a9NI5psz7l34+O4NyzTo9iF9KuqKiIq6+4hFFjJjB15v8YPep5Pi91zEY+8RgNGzVm1v/mceHFl3HT9X8osf66319J/0MHpDPsSGVlGfdefiSDr36Knqc/wAkH/4bObXYsUea23x3G05M/pu+Z/+LWJ97h5vP6l1h/4zkH8e7Hi9IYdXqYWdIPYIW79455PBRbVZzqPcE2TwN6A3dVFF+NTpAzpk+jXfsO7NKuHbVr1+b4E09i4vixJcpMHD+OU4eeAcAxxx7PlLffxN3ZvUdPWrRsCUDXrt3YsH49GzZsSPs+pNvMGdNo1649bXcJjtmxx5/IpAnjSpR5ZcI4Tj51KACDjzmOd6a8hXvwWZ04fixt2u5C5y6lh4eqrz5dcllQ8D2Llq3il8IiRr35GQP7dS4umP5uAAALZUlEQVRRpnPbHZkycyEA78xayMB+nTav67lrC5o3rscb0xekNe50SGEXOx9oFbOcBywtuz3rD1wHDHL3Cv/B1ugEuXRpAXmtfh2GyM3NY2lBQdkyecFxz8nJoWGDhqxcubJEmZfHjGa33XtSp06dqg86YsuWLiU379fPYcvcPJYtK/k5XBpTJicnhwYNGvL9ypWsW7eO++6+k99fe0NaY45ay2YNyP9u9eblguWryd1xhxJl/jf/G47eP/jSGLxfFxrUq0uTBtthZtx+0WFc+6/X0hpzuqTwJPZ0oKOZ7WJmtYEhQIlvbjPrCfybIDl+l0x8VZYgzayVmb1tZnPNbLaZXVpV29pSxa2aWFbqq6qiMnPmzOaGa6/h7w88mPoAM1Ayxyxez8bMuP2Wm7jw4suoX79+1QSXoeK1fkofxj/88zX27dGGDx65gH17tKXgu9UUFm3i/GP6MHnql+R/tyY9wabTFgxCJuLuhcDFwGRgLvCCu882s5vNbFBY7C6gPjDKzD42s3EJqtusKi8ULwSudPdZZrYDMNPMXnf3ORW9MV1yc/PIX5K/ebmgIH9zt7lEmfwl5OblUVhYyOo1q2nSpElQPj+fU044locee4J27dunNfaotMzNpSD/17HwpQX57Lxzi5JlWgZlcnODY7ZmzWoaN2nCjBnTGPvyS9z4x2tYvfoHsrKyqFO3LuddcFG6dyOtCpavIa95w83LuTs2ZOmKH0uUWbbyR4b88XkA6m1Xm6P368KadRvYs1sr9tmtNecd3Yd629Wmdq1s1v68kev//UZa96GqpPJWQ3efBEwq9doNMc/7l3lTBaosQbr7MmBZ+PxHM5tLcFYpYxLkHr37sGD+lyxauJCWubm8+MLzPPbk0yXKHDHwKJ4e+QR7/nYvxrz0IvsfcBBmxg8//MBxRw/kpltuZa+994loD9Kv1x59WLBgPosXLaRFy1xeevEFHn58ZIkyA448imefHknfPfdi7JjR7Lf/gZgZr7z+zuYyt//lT9SrV7/aJ0eAGZ8vpUNeE9q0aMTS5T9ywsHdGXbziyXKNG24Pd+v+Rl35+pT9+WJSR8BcOafR28uc9qAHuzRuWU1So661RAAM2sL9AQ+TMf2kpWTk8Pf7v0HRw8cQFFREUOHnUnXrt34859uoFev3hx51CDOOPNszjnzdHbr0pHGTZowYuSzAPz7X/fz1YL53HHrLdxx6y0AjJ04mebNm0e5S1UuJyeHO/92H8cNPoKioiJOPX0YXbp249Y/30iPXr054sijGHrGWVxwzhn0+k0nGjduzKNPPBN12JEqKtrE5fdOYvxfh5KdlcUTkz5i7qLlXH/Wgcyat5SJ789jvx5tufn8/rg7732ymMvumRh12GmR4fkRizemlNINmNUH3gH+4u4vxVl/HnAeQKvWrfeY++WiKo2nuvmlcFPUIWxzWgy4OeoQtjkbPnqYTT8uTWk+6757Lx/16rtJl+/asv5Md++dyhgqUqVnsc2sFjAaeDpecgRw94eKr2tq1mzHeEVEpJpK1a2GVaXKutgWnNp8FJjr7ndX1XZEZNuVleF97KpsQe4DDAUOCk+pf2xm1ec+KRHZeim8ELIqVOVZ7PfI/DFYEYmIZhQXEUlEM4qLiCSW4flRCVJEIpThGVIJUkQiEt3lO8lSghSRyGgMUkQkjgiv3kmaEqSIRCfDM6QSpIhEJivD+9hKkCISmcxOj0qQIhIVXSguIlKezM6QSpAiEgnNKC4iUo4Mz49KkCISHbUgRUQS0K2GIiKJZHZ+VIIUkehkeH5UghSRaJjpThoRkcQyOz8qQYpIdDI8PypBikh0MryHrQQpIlHRjOIiInFtC7caZkUdgIhIplILUkQik+ktSCVIEYmMxiBFROIILhSPOoryKUGKSHSUIEVE4lMXW0QkAZ2kERFJIMPzoxKkiEQowzOkEqSIRCbTxyDN3aOOYTMzWw4sjjqOOJoBK6IOYhujY7ZlMvW4tXH3HVNZoZm9SrC/yVrh7gNSGUNFMipBZiozm+HuvaOOY1uiY7ZldNwyi+7FFhFJQAlSRCQBJcjkPBR1ANsgHbMto+OWQTQGKSKSgFqQIiIJKEGKiCSgBCkikoASZAJm1snM9jKzWmaWHXU82wodq8oxsw5m1tvM6kQdi5SlkzRxmNmxwK1AQfiYAYxw9zWRBpbBzGxXd/8ifJ7t7kVRx5TpzGwgwedsJfANcGPxMZTMoBZkKWZWCzgJONvdDwbGAq2A/zOzBpEGl6HCf+gfm9kzAO5epJZk+cxsb+CvwBnufiCwCrgm2qikNCXI+BoAHcPnY4AJQG3gFLNMn8EuvcysHnAxcBmw0cyeAiXJJN3u7h+Fz28EmqirnVmUIEtx91+Au4FjzWxfd98EvAd8DPSLNLgM5O7rgLOAZ4CrgLqxSTLK2DLch8BLsHnctg7QhuDLGTNrGl1oUkwJMr53gdeAoWa2n7sXufszQEtg92hDyzzuvtTd17r7CuB8YLviJGlmvcysc7QRZp7wM1U8pm3AD8D37r7czE4FbjGz7aKLUEDzQcbl7uvN7GnAgT+E/8A3ADsByyINLsO5+0ozOx+4y8w+B7KBAyMOK6O5eyGw1syWmNltwKHAMHf/OeLQajwlyATcfZWZPQzMIWgVrQdOc/dvo40s87n7CjP7FDgcOMTd86OOKZOF49q1gH3D/x/s7l9GG5WALvNJSjhG5OF4pFTAzBoDLwBXuvunUcezrTCzYcB0d58ddSwSUIKUKmFmdd19fdRxbEvMzFz/IDOKEqSISAI6iy0ikoASpIhIAkqQIiIJKEGKiCSgBFlNmFmRmX1sZp+Z2Sgz234r6jrAzCaEzweZWcJJFMyskZn9bgu2cZOZXZXs66XKjDCz4yuxrbZm9lllYxRRgqw+fnb3Hu7eHdgIXBC70gKV/nu7+zh3v72cIo2ASidIkW2BEmT19C7QIWw5zTWzfwKzgFZmdqiZfWBms8KWZn0AMxtgZp+b2XvAscUVmdkwM7s/fL6TmY0xs0/Cx97A7UD7sPV6V1juajObbmafmtmfYuq6zszmmdkbQKeKdsLMzg3r+cTMRpdqFfc3s3fN7ItwujXMLNvM7orZ9vlbeyClZlOCrGbMLIfgFr//hS91Ap50957AOuCPQH9370UwEfAVZlYXeBg4iuB2t50TVP934B133x3oBcwmmMNwQdh6vdrMDiWYKq4v0APYw8z2M7M9gCFAT4IE3CeJ3XnJ3fuE25sLnB2zri2wP3Ak8GC4D2cDq929T1j/uWa2SxLbEYlL92JXH9uZ2cfh83eBRwlmH1rs7lPD138LdAXeD6e1rA18AHQGFhbf/xvOxHNenG0cBJwOm6cyWx3eVhjr0PBRPM9hfYKEuQMwxt1/CrcxLol96m5mtxB04+sDk2PWvRDe+vmlmX0V7sOhwG4x45MNw21rlm7ZIkqQ1cfP7t4j9oUwCa6LfQl43d1PLlWuB8HMRalgwG3u/u9S27hsC7YxAjja3T8J71M+IGZd6bo83PZwd49NpJhZ20puVwRQF7ummQrsY2YdAMxsezPbFfgc2MXM2oflTk7w/jeBC8P3Zoc/QfEjQeuw2GTgrJixzVwzaw78BzjGzLYzsx0IuvMV2QFYZsHPYJxaat0JZpYVxtwOmBdu+8KwPGa2qwUznotsEbUga5BwMtZhwLP269T+f3T3L8zsPGCima0gmEG9e5wqLgUeMrOzgSLgQnf/wMzeDy+jeSUch+wCfBC2YNcSTBM3y8yeJ5iZfTHBMEBFrieYeXsxwZhqbCKeB7xDMEfnBeEcno8QjE3OCqcQWw4cndzRESlLk1WIiCSgLraISAJKkCIiCShBiogkoAQpIpKAEqSISAJKkCIiCShBiogk8P9DVJvDIq2i9QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a2f6ae160>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Model K nearest neighbors\n",
    "\n",
    "knn_model = KNeighborsClassifier(n_neighbors=10)\n",
    "knn_model.fit(train_data[key_features], train_labels)\n",
    "\n",
    "start_time = time.time()\n",
    "knn_predicts = knn_model.predict(dev_data[key_features])\n",
    "runtime = time.time() - start_time\n",
    "\n",
    "# Score and Evaluate KNN\n",
    "print(\"Total prediction runtime: \", runtime)\n",
    "get_performance(dev_labels, knn_predicts, labels_names)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "iIbJisoX_9YL"
   },
   "source": [
    "## 5.3 Gaussian NB\n",
    "\n",
    "**Key findings**:\n",
    "\n",
    " - Lower accuracy of development data compared to KNN\n",
    " - Poor performance for deadlift classification\n",
    " - Dealift accuracy is still lagging"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "DmQ682LK_9YM"
   },
   "source": [
    "### 5.3.1 Model without any customization and validate on development data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "WLjXDB-6_9YM",
    "outputId": "a8e73c0a-0cc4-4ab5-8619-d53f2c1bc985"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total prediction runtime:  0.005456209182739258\n",
      "==================================================\n",
      "Classification report by exercise:\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "          0       0.91      0.95      0.93      8270\n",
      "          1       0.85      0.51      0.64      3303\n",
      "          2       0.81      0.92      0.86      6551\n",
      "\n",
      "avg / total       0.86      0.86      0.85     18124\n",
      "\n",
      "==================================================\n",
      "Confusion matrix:\n",
      "      0     1     2\n",
      "0  7896    53   321\n",
      "1   492  1685  1126\n",
      "2   257   238  6056\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUgAAAEmCAYAAAAA6gkZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xd8FHX+x/HXJwlVOkpLQJqAyEkRe1dERATPCneg2E9PPetZD5FTsXueesU7K/Z6IqJ4esKd/kSaFbEAgiQgvShSw+f3x0ziJuwmG9zNbJL308c+3Jn55jufmZDPfr/fmfmuuTsiIrK9rKgDEBHJVEqQIiIJKEGKiCSgBCkikoASpIhIAkqQIiIJKEFWAWY22syeCN+3M7MfzCw7xftYYGb9UllnBfZ9k5mtMLPvfkYdaTkvlc3MrjWzf0YdhwSUIClODkvNbKeYdWeb2eQIw4rL3b919wbuXliZ+zWzfcxsopmtMbNVZjbNzM5IQb1tgcuB7u7eakfrSed5MTMP/33kxKzLMbNlZpbUjcRmdpiZ5ZdXzt1vcfezf068kjpKkD/JAX73cyuxQLU6r2a2P/AfYArQGWgOnA8ck4LqdwVWuvuyFNSVTmsoebwDgdWp3EFsApbMUK3+kH+mO4ArzKxJvI1mdoCZTTezteH/D4jZNtnMbjaz94AfgY7hupvM7P/Crt+rZtbczJ40s3VhHe1j6rjXzBaF22aa2cEJ4mgftmhyzGz/sO6i10YzWxCWyzKzq81snpmtNLPnzKxZTD0jzGxhuO26JM7NY+5+m7uv8MBMdz8lpr5zzGxu2Locb2ZtYra5mf3GzL42s9Vm9kD4QdIP+DfQJoz/0Xgtrdjuf9iSnRGep6Vmdnfp8xIutwnjWBXGdU5MfaPD8/G4mX1vZrPNrG8552AccFrM8mnA46XiPMPM5oR1zjez88L1OwGvxxznD2F8o83sBTN7wszWASOt5HDKqWE9jcLlY8zsOzPbpZxYJVXcvca/gAVAP+Al4KZw3dnA5PB9M4LWwgiCluawcLl5uH0y8C2wR7i9VrhuLtAJaAx8DnwV7ieH4I/rkZgYhhO0zHIIupzfAXXDbaOBJ8L37QEHckodQ9E+x4bLlwBTgTygDvB34OlwW3fgB+CQcNvdwFagX5xzUx8oBA4v4/wdAawA+oT13Qf8N2a7AxOAJkA7YDkwINx2GJAfU7bEcuzvJ3z/PjAifN8A2C/eeSFo7f4FqAv0Cvd5ZMz53EjQCswGxgJTyzg+B3oAS8NjaBK+7wF4TLljw9+3AYcSfFj2KeO4RgNbgOMJGiv1Yn/XYZkngUfDfxuLgUFR/73UpJdakCWNAi6K8wl9LPC1u49z963u/jTwBXBcTJlH3X12uH1LuO4Rd5/n7msJWhDz3P0td98KPA/0Lvphd3/C3VeGP38XQaLpWoHY/wysB4pag+cB17l7vrtvIvjDOylsYZ0ETHD3/4bb/gBsS1BvU4I/3iVl7PvXwMPuPius7xpg/9gWMnCru69x92+BdwiS1o7YAnQ2s53d/Qd3n1q6QDiueRBwlbtvdPePgH8SfMAVedfdJ3owZjkO6FnOfjcCrwKnAkOB8eG6Yu7+Wvj7dnefArwJxO0JxHjf3f/l7tvcfUOc7b8l+ACaDLzq7hPKqU9SSAkyhrt/RtDSubrUpjbAwlLrFgK5McuL4lS5NOb9hjjLDYoWzOzysHu21szWELQ6d04m7rArdxjwK3cvSnS7Ai9bcFFlDTCHoCXYMjye4njdfT2wMkH1qwmSZ+syQihxftz9h7C+2PMTe4X6R2KOvYLOAroAX4TDFIMSxLPK3b+PWVf691U6nrpJjAE+TtC13q57DcVd4Klht34NQQu1vN9hvH83xdx9DcGHaQ/grnLqkhRTgtzeDcA5lPxjWkyQcGK1Awpilnd4WqRwvPEq4BSgqbs3AdYSdNWS+dk/AkPClmqRRcAx7t4k5lXX3QsIWoNtY+qoT9CF2467/0jQrT2xjDBKnJ9wzK05Jc9PstYTdOuL6soGilv07v61uw8DWgC3AS9YzN0HMfE0M7OGMetK/752xP8IPihaAu/GbjCzOsCLwJ1Ay/B3OJGffoeJ/n2U+e/GzHoBZwJPE/QSpBIpQZbi7nOBZ4GLY1ZPBLqY2a/CiyOnEozjpaq705BgDHA5kGNmo4BG5f1Q2JV8FjjN3b8qtflvwM1mtmtYdhczGxJuewEYZGYHmVltYAxl/1v4PcEFhCvNrHlYX08zeybc/hRwhpn1ChPFLcAH7r6g3CPf3lcErbljzawWcD3BcEPRMQ83s13ClvKacHWJW3vcfRHwf8BYM6trZnsStDyf3IF4Yut1gmGVweH7WLXDOJcDW83sGKB/zPalQHMza5zs/sysLvAEcC1wBpBrZhf8jEOQClKCjG8MUNwqcfeVwCCCiycrCRLGIHdfkaL9TSIYo/yKoCu4kXK6XqEjgVYEraiiq6Ozw233EoyTvWlm3xNcsNk3PJ7ZBGNbTxG0JlcDCe/Rc/f/IxgHOwKYb2argAcJPjhw97cJxjFfDOvrRDBOV2FhK/gCgjHDAoIWZWxsA4DZZvZDeIxD3X3jdhUFF9LaE7QmXwZucPd/70hMpeKbHZ6/0uu/J/hQfY7gfP6K4PwXbf+CoBU4Pxz2aFO6jjjGElzY+Ws4tjscuMnMdvu5xyHJse0/CEVEBNSCFBFJSAlSRCQBJUgRkQSUIEVEEsioh+Mtp55b7YblF5RivXdvF3UIVc42XZessG+/XcDKFSvKvS+3IrIb7eq+Nd7DQ/H5huWT3H1AKmMoT2YlyNoNqdP1lPILSrH3Prg/6hCqnE1bKnWmuGrhsAP3TXmdvnVDhf7eN370QFJPlqVSRiVIEalJDDJ8ZkAlSBGJhgGW0l57yilBikh01IIUEYnHICuzv0JICVJEoqMutohIHIa62CIi8ZlakCIiCakFKSKSgFqQIiLx6EZxEZH4dKO4iEgZ1IIUEYnHIFs3iouIbE/3QYqIlEFjkCIi8egqtohIYmpBiogkoBakiEgcpmexRUQSUwtSRCQBtSBFROLRVWwRkfgMfeWCiEh8akGKiCSmMUgRkQTUghQRSUAtSBGROExjkCIiiakFKSISn2V4gszs9m0lOOqA3fn45T/w2Ss3cMUZR223vV3rpkz820VMe/YaJv3jd+S2aFK87YcZf2bqM1cz9Zmref5P51Vm2JF6c9Ib7LlHV/bo1pk7br91u+2bNm1i+K9OZY9unTn4gH1ZuGABACtXruTofoezc5MGXHLxhZUcdbTeevMN+vbsTu8eXbnnztu2275p0ybOGDGM3j26cuQh+7Nw4YIS2xct+pbcXRpz35/uqqSI0y/4ShpL+hWFGp0gs7KMP119CkMu/Au9T7yJkwfsRbeOrUqUGXvpL3nytWnsc+pYbnnwdcZcNLh424ZNW9hv6K3sN/RWTr7k75UdfiQKCwu55OLf8sqrr/PhJ5/z/DNPM+fzz0uUefThh2japCmzv5jLRb+7lOuuvQqAunXrMmr0Hxl7251RhB6ZwsJCrrj0Yl741wQ+mPUpLzz/LF/MKXnOxj36ME2aNOXDz77kgosuYfT115TYfu3vL6df/wGVGXb6mWFZyb+iUKMT5N492jNv0QoWFKxky9ZCnp80i0GH7VmiTLeOrZn8wZcATJn+FYMO+0UUoWaM6dOm0alTZzp07Ejt2rU5+dShTHj1lRJlJrz6Cr8ecToAJ5x4EpP/8zbuzk477cSBBx1E3bp1owg9MjNnTKNjp0607xCcsxNPOoWJE8aXKDPxtfEMGz4CgCG/PJEpk/+DuwMwYfwrtO/QgW67d6/02NNNLcgM1qZFY/KXri5eLli6mtxdGpco8+lXBRx/ZC8AhhzRk0YN6tGs8U4A1K2dw7tP/p4pj13OcaUSa3W1eHEBeXlti5dzc/MoKCjYvkzboExOTg6NGjdm5cqVlRpnJlmyeDG5uT+dsza5eSxZvDhhmZycHBo1asyqlStZv3499959O1ddO6pSY64sqUyQZjbAzL40s7lmdnWc7e3M7B0z+9DMPjGzgeXVmdaLNGY2ALgXyAb+6e7bD1hFyNj+pHup5WvueZl7rjqZ4YP35b1ZcylYupqthYUAdBk4iiXL19I+tzlvPHgxn81dzDf5Kyoh8ugUtWpilf7Hm0yZmiTe+Sh99TbRORt702guuOgSGjRokKboopWqfxdmlg08ABwF5APTzWy8u8eOZVwPPOfufzWz7sBEoH1Z9aYtQSYZcKQKlq0hr2XT4uXclk1ZvHxtiTJLlq9l6BX/BGCnerU5/sherPthY/E2gAUFK/nvjK/p1S2v2ifI3Nw88vMXFS8XFOTTpk2b7cssWkReXh5bt25l3dq1NGvWrLJDzRhtcnMpKPjpnC0uyKd169Zxy+QWnbN1a2narBkzp0/jlZdfYtR1V7N27RqysrKoU6cu557/28o+jNSz8JUa+wBz3X0+gJk9AwwBYvONA43C942Bks34ONLZxS4O2N03A0UBZ4wZsxfSud0u7NqmObVysjn56D68NvmTEmWaN9mp+FPuyjOP5rFXpgLQpGE9atfKKS6zf6+OzJn/XeUeQAT67r03c+d+zYJvvmHz5s08/+wzHDtocIkyxw4azJPjHgPgpRdf4NDDj6jRLcg+e+3NvLlzWbAgOGcvvvAcxxx7XIkyxww8jqefGAfAKy+/yCGHHo6Z8fpbU/j0i3l8+sU8zv/txVx+5dXVIzkS9OAq2MXe2cxmxLzOjakuF1gUs5wfros1GhhuZvkErceLyosxnV3seAHvW7pQeJDBgdaq3G5EYeE2Lr3tOV79y2/JzjIee2Uqc+Z/xx/OP5ZZn3/La1M+5ZC+uzHmosG4w7uz5nLJ2OcA6NaxFfddN4xtvo0sy+LOR/7NFzUgQebk5HDPvfdz3LFHU1hYyOkjz6T7HnswZvQo+uzVl0HHDWbkmWdx5sgR7NGtM02bNmPck88U/3zXzu35ft06Nm/ezKvj/8WEiW+ye/fqd/EhVk5ODnfcfS8nDh5IYWEhw08bye7d9+DmMTfQu09fBg46jhEjz+S8s06nd4+uNG3alIcffyrqsCtFBT84V7h730RVxVlXetxiGPCou99lZvsD48ysh7tvSxhf3PGRFDCzk4Gj3f3scHkEsI+7J8zaWfVbeJ2up6Qlnupq9fT7ow6hytm0pTDqEKqcww7clw9nzUhpNyCneUdvNPCmpMuvfuLXMxMlyDDhjXb3o8PlawDcfWxMmdnAAHdfFC7PB/Zz92WJ9pnOLnY+0DZmOY8k+vwiUnOk8Cr2dGA3M+tgZrWBocD4UmW+BY4M97s7UBdYXlal6UyQyQQsIjWVVfBVBnffClwITALmEFytnm1mY8ysaJD8cuAcM/sYeBoY6eV0odM2BunuW82sKOBs4GF3n52u/YlI1WIYWVmpa6O5+0SCiy+x60bFvP8cOLAidab1Psh4AYuIFMn0uxs0m4+IRCez86MSpIhExNSCFBFJSAlSRCQBJUgRkTiKHjXMZEqQIhKdzM6PSpAiEhFdpBERSUwJUkQkgai+ayZZSpAiEhm1IEVE4ojyy7iSpQQpIpFRghQRSUAJUkQkkczOj0qQIhIdtSBFROLRjeIiIvEZkOH5UQlSRKJiZOlGcRGR+NTFFhGJx9TFFhGJy0BdbBGRRNSCFBFJQGOQIiLxaAxSRCS+4D7IzM6QSpAiEhFNdyYiklCG50clSBGJiOk2HxGRuDQGKSJShgzPj0qQIhIdtSBFRBLI8PyYWQmy+255vPT67VGHUaVc8MKnUYdQ5fTr0jTqEKqctRs3p75STZgrIhKfJswVEUlIN4qLiCSU4flRCVJEIqIbxUVE4tON4iIiZcj0BJkVdQAiUnOZJf8qvy4bYGZfmtlcM7s6QZlTzOxzM5ttZk+VV6dakCISmVS1IM0sG3gAOArIB6ab2Xh3/zymzG7ANcCB7r7azFqUV69akCISjQq0HpPIo/sAc919vrtvBp4BhpQqcw7wgLuvBnD3ZeVVqgQpIpGw8D7IZF/AzmY2I+Z1bkx1ucCimOX8cF2sLkAXM3vPzKaa2YDyYlQXW0QiU8Ee9gp375uoqjjrvNRyDrAbcBiQB/zPzHq4+5pEO1SCFJHIZKXuKnY+0DZmOQ9YHKfMVHffAnxjZl8SJMzpCeNLVXQiIhWVwjHI6cBuZtbBzGoDQ4Hxpcr8Czg82K/tTNDlnl9WpWpBikgkzCA7RU/SuPtWM7sQmARkAw+7+2wzGwPMcPfx4bb+ZvY5UAhc6e4ry6pXCVJEIpPKG8XdfSIwsdS6UTHvHbgsfCUlYYI0s0blBLMu2Z2IiMST4Q/SlNmCnE1wFSj2EIqWHWiXxrhEpJozglt9MlnCBOnubRNtExFJhQyfzCe5q9hmNtTMrg3f55nZXukNS0SqvQrcJB7VpBblJkgzu5/g0viIcNWPwN/SGZSI1AypnKwiHZK5in2Au/cxsw8B3H1VeJ+RiMgOM1J6o3haJJMgt5hZFuFjO2bWHNiW1qhEpEbI8PyY1BjkA8CLwC5mdiPwLnBbWqMSkRoh08cgy21BuvvjZjYT6BeuOtndP0tvWCJS3aXySZp0SfZJmmxgC0E3W89vi0hKZHZ6TO4q9nXA00AbghkynjKza9IdmIhUf1W+iw0MB/Zy9x8BzOxmYCYwNp2BiUj1FlzFjjqKsiWTIBeWKpdDOVMEiYiUK8KWYbLKmqziHoIxxx+B2WY2KVzuT3AlW0TkZ8nw/FhmC7LoSvVs4LWY9VPTF46I1CRVtgXp7g9VZiAiUrNUhTHIZK5idzKzZ8zsEzP7quhVGcFVhv/+502OPqgXR+3/Cx68787ttk9//11+edQBdM9rxBsTXi6xbffchgzptx9D+u3Hb04/ubJCjlyPVg24ZWAXxh7bhYG777Ld9gM7NOHe43dn9NGdGX10Zw7u2LR426WHtuf+E7rzu4N3rcyQI/fx/73DFSccymVDDmL8Iw9st/2tF8Zx1Sn9uGbY0dx45gnkzy/5J7ZiSQFnHtSV1x6vXtMgVIer2I8CNwF3AscAZ1BNHjUsLCxkzLWX8cizr9KydS4nHXMwR/Q/ls5ddy8u0zqvLWPv/TsP//Xe7X6+bt16vPJWzRpxMIPhfdtw1zvfsGrDVkYd1YmPCtaxeN2mEuWmfbuWJ2eV/s4keOOL5dTOzuKwTs0qK+TIbSss5NFbr+eavzxFs5at+cOIQfQ59CjyOnYpLnPAgOPpd1IwH8zMKW/y5N1juOr+J4q3P3H3jfQ84PBKjz2dzCA7w7vYydz0Xd/dJwG4+zx3v57wi2+quk8+nMGu7TvSdtcO1K5dm2OHnMTbkyaUKJPXdle6df8FWVm6Px6gY7P6LPt+M8vXb6Fwm/PBt2vplVvm5PMlzFm6no1bq8Xna9Lmzf6Ilm3b0yJvV3Jq1Wa//oOZOfnNEmXqN2hY/H7Thh9LXL2Y8c4btMhtR16nLlQ3mT6bTzJ/9ZssaN/OM7PfmNlxQIs0x1Upln63mFa5ecXLLVvnsvS7JUn//KZNGznh6IM45djDeOv1V9MRYsZpUi+HVT9uKV5evWELTevV2q7cXm0bceOAzlxwYDua1t9+e02yatl3NG/Zpni5WcvWrF7+3Xbl3nzuUS4dfCBP//kWTr9yDAAbN/zIq4/9lRPOvbTS4q1M1aGLfSnQALgYuBloDJxZ3g+Z2cPAIGCZu/f4OUGmS/AdPiVV5BfxzowvadmqNYsWfsPpJw2ky+570K59x1SGmHHinR4v9f3sHxV8zwcL17J1m3NYp2acvW8ed7zzTSVFmIGS/HfW/5SR9D9lJO+9/jL/+uef+c2Ye3jxb3dxzK/Opm79nSoj0kqX4T3spCar+CB8+z0/TZqbjEeB+4HHKx5W5WjVOpfvCvKLl5cuKaBFy1ZJ/3zLVq0BaLtrB/Y54GA+/+zjap8gV/+4lWYxLcKm9WqxZsPWEmXWby4sfj9l/ipO6pn8Oa2OmrVszcqlP43Hrlq6hCY7t0xYfv+jh/DI2OsAmPfZh0x7eyJP//kWfvx+HZZl1KpTl/6njkx32GlnWNWdD9LMXga2/+gLufsJZVXs7v81s/Y7HFkl+EWvvVjwzTwWfbuAlq3a8NorL3DXXx5J6mfXrllNvXr1qV2nDqtWrmDW9KmcfUH17AbF+mbVj7RsWIedd6rF6g1b2bddY/7+/qISZRrXzWHtxiBp9m7TiCWlLuDUNB279+S7RQtYVvAtzVq0Yuqb4/ntzfeVKPPdt9/Qql0HAD56921atWsPwKiHXiou8+Lf76ZuvfrVIjkCEOHYYrLKakHeXxkBmNm5wLkAbXIr93vCcnJyGHXLXZw9bAiFhYWcOPQ0duvanXtv/yM9evbhyKOP5ZOPZnLhmUNZt2YN7/z7de6742ZemzKDeV9/yQ2/vwjLysK3beOcCy8vcfW7utrm8MTMxVx2aAeysuDd+atZvG4Tx/dowYJVG/ho8ff069KcXrmN2LbN+WFzIQ998FMr/eojO9K6YR3q5GRx5+BuPDItn9nf/RDhEaVfdk4OI3//R267cDjbCgs5dMip5HXqygt/vZMO3fdkr0P78+azj/LZtHfJzslhp4aN+c2N90QddqXI9BvFLd44XMoqD1qQE5Idg+zRs4+/NElPMVbErZPnRR1CldOvS9PyC0kJ1w8fyPzPP0lpNmvRuYefesfzSZe//4TuM929bypjKE+y80GKiKSUkfktSCVIEYlMlX/UsIiZ1alIxWb2NPA+0NXM8s3srIoGJyLVV9FXLiT7ikK5LUgz2wd4iOD+x3Zm1hM4290vKuvn3H1YakIUkeqqOrQg/0xww/dKAHf/mGryqKGIRCvTHzVMZgwyy90XlhpMLUxUWEQkGcF0Z5ndhEwmQS4Ku9luZtnARUC1me5MRKKT6VPAJJMgzyfoZrcDlgJvhetERH6WDG9AJvUs9jJgaCXEIiI1iFkVfha7iJn9gzjPZLv7uWmJSERqjAzPj0l1sd+KeV8X+CWwKEFZEZGkZfptPsl0sZ+NXTazccC/0xaRiNQIBpHdAJ6sHXnUsANQs75xSURSz6pBC9LMVvPTGGQWsAq4Op1BiUjNYGR2hiwzQYbfRdMTKAhXbfN0zo8mIjVGlf9e7DAZvuzuheFLyVFEUibLkn+Vx8wGmNmXZjbXzBL2cs3sJDNzMyt3bslkbmSfZmZ9kignIlIhqfpWw/ApvweAY4DuwDAz6x6nXEOCLyD8oPS2eBImSDMr6n4fRJAkvzSzWWb2oZnNSqZyEZFEirrYKWpB7gPMdff57r4ZeAYYEqfcH4HbgY3JxFjWGOQ0oA9wfDIViYhUSMVn6dnZzGbELD/o7g+G73MpeX92PrBvid2Z9QbauvsEM7simR2WlSANwN31pScikhYVfNRwRRnfSROvouJrJmaWBdwDjKzIDstKkLuY2WWJNrr73RXZkYhIrBRfxc4HYr8WNQ9YHLPcEOgBTA7HM1sB481ssLvHtkpLKCtBZgMNiJ+ZRUR+JiM7dQ9jTwd2M7MOBLclDgV+VbTR3dcCOxfv2WwycEVZyRHKTpBL3H3Mz4lYRCSR4FsNU1OXu281swuBSQSNu4fdfbaZjQFmuPv4Ham33DFIEZG0SPGjhu4+EZhYat2oBGUPS6bOshLkkUlHJiKyA6rsfJDuvqoyAxGRmiWVXex02ZHZfEREUqLKtiBFRNItw/OjEqSIRMOoHt9qKCKSeka5k1BETQlSRCKT2elRCVJEImKQyidp0kIJUkQik+H5UQlSRKJS/kS4UVOCFJFI6Cq2iEgZ1IIUEUkgs9NjhiXIWtlZtGxcJ+owqpR7hmz3vURSjlZHXBd1CFXOpm+Xp75S3QcpIhKfxiBFRMqgFqSISAKpnDA3HZQgRSQSQRc7szOkEqSIRCbDe9hKkCISFcPUghQRiU8tSBGRODQGKSKSiKkFKSKSkBKkiEgCukgjIhKHoRvFRUQS0vdii4gkoC62iEgc6mKLiCSkJ2lEROLTfZAiIolleH5UghSRaARjkJmdIpUgRSQymZ0elSBFJEoZniGVIEUkMupii4gkkNnpUQlSRKKU4RlSCVJEImHoUUMRkfiqwI3iWVEHICI1l1XgVW5dZgPM7Eszm2tmV8fZfpmZfW5mn5jZ22a2a3l1KkGKSHRSlCHNLBt4ADgG6A4MM7PupYp9CPR19z2BF4DbywtPCVJEImIV+q8c+wBz3X2+u28GngGGxBZw93fc/cdwcSqQV16lSpAiEhmz5F/AzmY2I+Z1bkxVucCimOX8cF0iZwGvlxdfjU+Qb735Bn17dqd3j67cc+dt223ftGkTZ4wYRu8eXTnykP1ZuHBBie2LFn1L7i6Nue9Pd1VSxNHTOau4o/brwsfPXM5nz1/BFSMO3W57u1ZNmHjf2Uwb9zsmPXAuubs0AmDP3Voz+cHzmfnkpUwb9ztOOnLPyg49bSrSuw7bjyvcvW/M68FS1ZXmcfdrNhzoC9xRXow1OkEWFhZyxaUX88K/JvDBrE954fln+WLO5yXKjHv0YZo0acqHn33JBRddwujrrymx/drfX06//gMqM+xI6ZxVXFaW8afLhzDkskfoPeweTj6qF93atyhRZuxFA3ny9VnsM+Jebnn4bcacH5yfHzdu4awxz7HXr+9hyKUPc/slg2jcoG4Uh5EWZpb0qxz5QNuY5TxgcZz99QOuAwa7+6byKq3RCXLmjGl07NSJ9h06Urt2bU486RQmThhfoszE18YzbPgIAIb88kSmTP4P7sEH04Txr9C+Qwe67V56LLj60jmruL27t2Ve/koWLF7Flq2FPP/Wxww6pOTxd2vfksnT5wIwZea84u1zF61gXv5KAJas+J7lq9ezc5OdKvcA0qiCXeyyTAd2M7MOZlYbGAqU+IdpZr2BvxMkx2XJxFejE+SSxYvJzf3pQ6dNbh5LFi9OWCYnJ4dGjRqzauVK1q9fz713385V146q1JijpnNWcW12aUT+srXFywXL1hZ3oYt8OncJxx/+CwCGHLoHjXaqS7NG9UuU6ds9j9q1splfsCr9QVeSVN3m4+5bgQuBScBlWMkZAAAJpUlEQVQc4Dl3n21mY8xscFjsDqAB8LyZfWRm4xNUVyxtN4qbWVvgcaAVsA140N3vTdf+dkRRq6aEUh9V8cqYGWNvGs0FF11CgwYN0hRdZtI5q7h43cPSp+ia+17jnsuHMPzYvXjvw28oWLaWrYWFxdtbNW/IQ6NO5Zw/Phf/d1AVJXuDY5LcfSIwsdS6UTHv+1W0znQ+SbMVuNzdZ5lZQ2Cmmf3b3T8v7wcrS5vcXAoKfrrwtbggn9atW8ctk5uXx9atW1m3bi1NmzVj5vRpvPLyS4y67mrWrl1DVlYWderU5dzzf1vZh1GpdM4qrmDZWvJaNC5ezm3RmMUr1pUos2TF9wy95gkAdqpXm+MP78G69cEQWcP6dXjprpHc+OCbTJu9iOqkxj5q6O5LgCXh++/NbA7BZfeMSZB99tqbeXPnsmDBN7Rpk8uLLzzHPx8ZV6LMMQOP4+knxrHPvvvzyssvcsihh2NmvP7WlOIyY2+6kQYNGlT7P3TQOdsRM+bk07ltc3Zt3ZTFy9dxcr+ejLzh6RJlmjeuz6p1G3B3rjztMB6bMAOAWjnZPHvbCJ56fRYv/efTKMJPGyPzHzWslGexzaw90Bv4oDL2l6ycnBzuuPteThw8kMLCQoafNpLdu+/BzWNuoHefvgwcdBwjRp7JeWedTu8eXWnatCkPP/5U1GFHSues4goLt3HpXeN59U9nkp2VxWMTZjDnm2X84ZyjmDUnn9fencMhfToy5vwBuDvvfrSAS+78FwAnHvkLDurVgWaN6jN84F4AnHvT83zy9ZIoDyllMjw/YukezzCzBsAU4GZ3fynO9nOBcwHatm2316dfzk9rPCKtjrgu6hCqnE2fPsa2H5akNJ/16NnHn3/jf0mX796mwUx375vKGMqT1qvYZlYLeBF4Ml5yBHD3B4tu/Gy+8y7pDEdEMkwKHzVMi3RexTbgIWCOu9+drv2ISNWVleF97HS2IA8ERgBHhPccfWRmA9O4PxGpalI531kapPMq9rtk/hisiEREM4qLiCRSBWYUV4IUkchkeH5UghSRCGV4hlSCFJGIRHf7TrKUIEUkMhqDFBGJI8K7d5KmBCki0cnwDKkEKSKRycrwPrYSpIhEJrPToxKkiERFN4qLiJQlszOkEqSIREIziouIlCHD86MSpIhERy1IEZEE9KihiEgimZ0flSBFJDoZnh+VIEUkGmZ6kkZEJLHMzo9KkCISnQzPj0qQIhKdDO9hK0GKSFQ0o7iISFxV4VHDrKgDEBHJVGpBikhkMr0FqQQpIpHRGKSISBzBjeJRR1E2JUgRiY4SpIhIfOpii4gkoIs0IiIJZHh+VIIUkQhleIZUghSRyGT6GKS5e9QxFDOz5cDCqOOIY2dgRdRBVDE6ZzsmU8/bru6+SyorNLM3CI43WSvcfUAqYyhPRiXITGVmM9y9b9RxVCU6ZztG5y2z6FlsEZEElCBFRBJQgkzOg1EHUAXpnO0YnbcMojFIEZEE1IIUEUlACVJEJAElSBGRBJQgEzCzrma2v5nVMrPsqOOpKnSuKsbMOptZXzOrE3Ussj1dpInDzE4AbgEKwtcM4FF3XxdpYBnMzLq4+1fh+2x3L4w6pkxnZoMI/p2tBL4Dbig6h5IZ1IIsxcxqAacCZ7n7kcArQFvg92bWKNLgMlT4h/6RmT0F4O6FakmWzcwOAO4ETnf3w4HVwNXRRiWlKUHG1wjYLXz/MjABqA38yizTZ7CrXGa2E3AhcAmw2cyeACXJJN3q7h+G728AmqmrnVmUIEtx9y3A3cAJZnawu28D3gU+Ag6KNLgM5O7rgTOBp4ArgLqxSTLK2DLcB8BLUDxuWwfYleDDGTNrHl1oUkQJMr7/AW8CI8zsEHcvdPengDZAz2hDyzzuvtjdf3D3FcB5QL2iJGlmfcysW7QRZp7w31TRmLYBa4BV7r7czH4N3GRm9aKLUEDzQcbl7hvN7EnAgWvCP/BNQEtgSaTBZTh3X2lm5wF3mNkXQDZweMRhZTR33wr8YGaLzGws0B8Y6e4bIg6txlOCTMDdV5vZP4DPCVpFG4Hh7r402sgyn7uvMLNPgGOAo9w9P+qYMlk4rl0LODj8/5Hu/nW0UQnoNp+khGNEHo5HSjnMrCnwHHC5u38SdTxVhZmNBKa7++yoY5GAEqSkhZnVdfeNUcdRlZiZuf4gM4oSpIhIArqKLSKSgBKkiEgCSpAiIgkoQYqIJKAEWU2YWaGZfWRmn5nZ82ZW/2fUdZiZTQjfDzazhJMomFkTM7tgB/Yx2syuSHZ9qTKPmtlJFdhXezP7rKIxiihBVh8b3L2Xu/cANgO/id1ogQr/vt19vLvfWkaRJkCFE6RIVaAEWT39D+gctpzmmNlfgFlAWzPrb2bvm9mssKXZAMDMBpjZF2b2LnBCUUVmNtLM7g/ftzSzl83s4/B1AHAr0Clsvd4RlrvSzKab2SdmdmNMXdeZ2Zdm9hbQtbyDMLNzwno+NrMXS7WK+5nZ/8zsq3C6Ncws28zuiNn3eT/3RErNpgRZzZhZDsEjfp+Gq7oCj7t7b2A9cD3Qz937EEwEfJmZ1QX+ARxH8LhbqwTV/xmY4u49gT7AbII5DOeFrdcrzaw/wVRx+wC9gL3M7BAz2wsYCvQmSMB7J3E4L7n73uH+5gBnxWxrDxwKHAv8LTyGs4C17r53WP85ZtYhif2IxKVnsauPemb2Ufj+f8BDBLMPLXT3qeH6/YDuwHvhtJa1gfeBbsA3Rc//hjPxnBtnH0cAp0HxVGZrw8cKY/UPX0XzHDYgSJgNgZfd/cdwH+OTOKYeZnYTQTe+ATApZttz4aOfX5vZ/PAY+gN7xoxPNg73rVm6ZYcoQVYfG9y9V+yKMAmuj10F/Nvdh5Uq14tg5qJUMGCsu/+91D4u2YF9PAoc7+4fh88pHxazrXRdHu77InePTaSYWfsK7lcEUBe7ppkKHGhmnQHMrL6ZdQG+ADqYWaew3LAEP/82cH74s9nhV1B8T9A6LDIJODNmbDPXzFoA/wV+aWb1zKwhQXe+PA2BJRZ8DcavS2072cyywpg7Al+G+z4/LI+ZdbFgxnORHaIWZA0STsY6Enjafpra/3p3/8rMzgVeM7MVBDOo94hTxe+AB83sLKAQON/d3zez98LbaF4PxyF3B94PW7A/EEwTN8vMniWYmX0hwTBAef5AMPP2QoIx1dhE/CUwhWCOzt+Ec3j+k2BsclY4hdhy4Pjkzo7I9jRZhYhIAupii4gkoAQpIpKAEqSISAJKkCIiCShBiogkoAQpIpKAEqSISAL/D2TUEIrChtZsAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a2409ff60>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "GNB_model = GaussianNB()\n",
    "GNB_model.fit(train_data[key_features], train_labels)\n",
    "\n",
    "start_time = time.time()\n",
    "GNB_predicts = GNB_model.predict(dev_data[key_features])\n",
    "runtime = time.time() - start_time\n",
    "\n",
    "print(\"Total prediction runtime: \", runtime)\n",
    "get_performance(dev_labels, GNB_predicts, labels_names)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "2Yj3CInK_9YQ"
   },
   "source": [
    "## 5.4 Decision Trees"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "OUHogmch_9YR"
   },
   "source": [
    "### 5.4.1 Model formation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "VHsWz7yH_9YR"
   },
   "source": [
    "**Key findings**:\n",
    "\n",
    " - Good model performance on development data\n",
    " - Deadlift classification again is lagging in accuracy\n",
    " - Most of the confusion is between squat and deadlift"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Nh9-tDEy_9YV"
   },
   "outputs": [],
   "source": [
    "# Random Forest model\n",
    "\n",
    "rf_model = RandomForestClassifier(n_estimators=5)\n",
    "rf_model.fit(train_data[key_features], train_labels)\n",
    "\n",
    "start_time = time.time()\n",
    "rf_dev_predictions = rf_model.predict(dev_data[key_features])\n",
    "runtime = time.time() - start_time"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ufoJ04fv_9YX"
   },
   "source": [
    "### 5.4.2 Model evaluation on development data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "A51U58lR_9YX",
    "outputId": "684798a3-7a98-4514-eebf-17a61cb4946d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total prediction runtime:  0.02347087860107422\n",
      "Accuracy on development data: 94.23%\n",
      "==================================================\n",
      "Classification report by exercise:\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "          0       0.96      0.99      0.98      8270\n",
      "          1       0.89      0.85      0.87      3303\n",
      "          2       0.94      0.93      0.94      6551\n",
      "\n",
      "avg / total       0.94      0.94      0.94     18124\n",
      "\n",
      "==================================================\n",
      "Confusion matrix:\n",
      "      0     1     2\n",
      "0  8159    53    58\n",
      "1   151  2795   357\n",
      "2   147   279  6125\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUgAAAEmCAYAAAAA6gkZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xd8FHX+x/HXJwlF6U0lCVKVIh5dz4IVFaTYCyqKetazlzvbqeep2NtPr1kOe0HlRECxnHo2lGI5AQtVElAhIk0BCZ/fHzOJm7CbbHA3s0neTx/7cGfnu9/5zLD57Pf7nZnvmrsjIiKby4o6ABGRTKUEKSKSgBKkiEgCSpAiIgkoQYqIJKAEKSKSgBJkDWBm15rZY+Hz7c1sjZllp3gbC81sUCrrrMK2rzez5Wb2za+oIy3HpbqZ2RVm9kDUcUhACZLS5PCtmTWKee13ZvZmhGHF5e5fu3tjdy+uzu2a2S5mNtnMfjCz783sQzM7OQX1tgMuBnq4+3ZbWk86j4uZefj5yIl5LcfMvjOzpC4kNrN9zKygsnLufqO7/+7XxCupowT5ixzg/F9biQVq1XE1s92A/wBvAV2AVsBZwJAUVN8eKHL371JQVzr9QNn9PRhYkcoNxCZgyQy16g/5V7oVuMTMmsdbaWa7m9k0M1sZ/n/3mHVvmtkNZvYu8CPQKXztejN7L+z6vWhmrczscTNbFdbRIaaOu81scbhuhpkNTBBHh7BFk2Nmu4V1lzzWmdnCsFyWmV1mZvPMrMjMnjGzljH1jDKzReG6K5M4Ng+7+83uvtwDM9z96Jj6TjOzuWHrcoKZ5casczM708y+MrMVZnZf+EUyCHgVyA3jHxuvpRXb/Q9bstPD4/Stmd1R/riEy7lhHN+HcZ0WU9+14fF4xMxWm9ksM+tfyTF4FDgxZvlE4JFycZ5sZnPCOueb2Rnh642Al2L2c00Y37Vm9qyZPWZmq4DRVnY45Ziwnqbh8hAz+8bM2lQSq6SKu9f5B7AQGAQ8D1wfvvY74M3weUuC1sIogpbmyHC5Vbj+TeBrYKdwfb3wtblAZ6AZMBv4MtxODsEf179iYjiBoGWWQ9Dl/AZoGK67FngsfN4BcCCn3D6UbHNMuHwBMBXIBxoA/wCeDNf1ANYAe4Xr7gA2AoPiHJutgWJg3wqO337AcqBvWN//Af+NWe/ARKA5sD2wDBgcrtsHKIgpW2Y59t8nfP4+MCp83hj4bbzjQtDa/SvQEOgdbnP/mOO5jqAVmA2MAaZWsH8O9AS+Dfehefi8J+Ax5YaG/94G7E3wZdm3gv26FvgZOJSgsbJV7L91WOZxYGz42VgCDIv676UuPdSCLOtq4Nw439BDga/c/VF33+juTwKfA8Njyox191nh+p/D1/7l7vPcfSVBC2Keu7/m7huBcUCfkje7+2PuXhS+/3aCRNO1CrHfA6wFSlqDZwBXunuBu68n+MM7MmxhHQlMdPf/huv+BGxKUG8Lgj/epRVs+3jgIXefGdZ3ObBbbAsZuMndf3D3r4E3CJLWlvgZ6GJmrd19jbtPLV8gHNfcE/iju69z94+BBwi+4Eq84+6TPRizfBToVcl21wEvAscAxwITwtdKufuk8N/b3f0t4BUgbk8gxvvu/m933+TuP8VZ/3uCL6A3gRfdfWIl9UkKKUHGcPfPCFo6l5VblQssKvfaIiAvZnlxnCq/jXn+U5zlxiULZnZx2D1baWY/ELQ6WycTd9iV2wc4zt1LEl17YLwFJ1V+AOYQtAS3DfenNF53XwsUJah+BUHybFtBCGWOj7uvCeuLPT6xZ6h/JGbfq+hUYEfg83CYYliCeL5399Uxr5X/9yofT8MkxgAfIehab9a9htIu8NSwW/8DQQu1sn/DeJ+bUu7+A8GXaU/g9krqkhRTgtzcNcBplP1jWkKQcGJtDxTGLG/xtEjheOMfgaOBFu7eHFhJ0FVL5r1/AQ4JW6olFgND3L15zKOhuxcStAbbxdSxNUEXbjPu/iNBt/aICsIoc3zCMbdWlD0+yVpL0K0vqSsbKG3Ru/tX7j4S2Aa4GXjWYq4+iImnpZk1iXmt/L/Xlnib4ItiW+Cd2BVm1gB4DrgN2Db8N5zML/+GiT4fFX5uzKw3cArwJEEvQaqREmQ57j4XeBo4L+blycCOZnZceHLkGIJxvFR1d5oQjAEuA3LM7GqgaWVvCruSTwMnuvuX5Vb/HbjBzNqHZduY2SHhumeBYWa2p5nVB66j4s/CHwhOIFxqZq3C+nqZ2VPh+ieAk82sd5gobgQ+cPeFle755r4kaM0NNbN6wFUEww0l+3yCmbUJW8o/hC+XubTH3RcD7wFjzKyhmf2GoOX5+BbEE1uvEwyrjAifx6ofxrkM2GhmQ4ADY9Z/C7Qys2bJbs/MGgKPAVcAJwN5Znb2r9gFqSIlyPiuA0pbJe5eBAwjOHlSRJAwhrn78hRtbwrBGOWXBF3BdVTS9QrtD2xH0IoqOTs6K1x3N8E42StmtprghM2u4f7MIhjbeoKgNbkCSHiNnru/RzAOth8w38y+B/5J8MWBu79OMI75XFhfZ4JxuioLW8FnE4wZFhK0KGNjGwzMMrM14T4e6+7rNqsoOJHWgaA1OR64xt1f3ZKYysU3Kzx+5V9fTfCl+gzB8TyO4PiXrP+coBU4Pxz2yC1fRxxjCE7s/C0c2z0BuN7Mdvi1+yHJsc2/CEVEBNSCFBFJSAlSRCQBJUgRkQSUIEVEEsiom+MtZyu3+k0qLyil+nTfPuoQpA5YtGghy5cvr/S63KrIbtrefWO8m4fi85+WTXH3wamMoTKZlSDrN6FB16MrLyil3v3g3qhDkDpgj10rm8uj6nzjT1X6e1/38X1J3VmWShmVIEWkLjHI8JkBlSBFJBoGWEp77SmnBCki0VELUkQkHoOszP4JISVIEYmOutgiInEY6mKLiMRnakGKiCSkFqSISAJqQYqIxKMLxUVE4tOF4iIiFVALUkQkHoNsXSguIrI5XQcpIlIBjUGKiMSjs9giIompBSkikoBakCIicZjuxRYRSUwtSBGRBNSCFBGJR2exRUTiM/STCyIi8akFKSKSmMYgRUQSUAtSRCQBtSBFROIwjUGKiCSmFqSISHyW4Qkys9u31eCA3bvzyfg/8dkL13DJyQdstn77ti2Y/Pdz+fDpy5ly//nkbdO8dN315x3C9HFXMH3cFRx5YN/qDDtSr0x5md/s1JWdunXh1ltu2mz9+vXrOeG4Y9ipWxcG7r4rixYuBKCoqIiDBu1L6+aNueC8c6o56mjpmG0u+EkaS/oRhTqdILOyjLsuO5pDzvkrfY64nqMG96Nbp+3KlBlz4WE8PulDdjlmDDf+8yWuO3cEAIP33Ine3dux67E3sdeo27jgpEE0adQwit2oVsXFxVxw3u954cWX+OjT2Yx76knmzJ5dpszYhx6kRfMWzPp8LueefyFXXvFHABo2bMjV1/6FMTffFkXokdExS8AMy0r+EYU6nSAH9OzAvMXLWVhYxM8bixk3ZSbD9vlNmTLdOrXlzQ++AOCtaV8ybJ+dAejeaTvenvEVxcWb+HHdBv73ZQEH7t692vehuk378EM6d+5Cx06dqF+/PkcdcywTX3yhTJmJL77A8aNOAuDwI47kzf+8jrvTqFEj9thzTxo2rP1fJLF0zBJTCzKD5W7TjIJvV5QuF367grw2zcqU+d+XhRy6f28ADtmvF00bb0XLZo349MtCDtqjB1s1rEer5o3Yu/+O5G/Xolrjj8KSJYXk57crXc7Ly6ewsHDzMu2CMjk5OTRt1oyioqJqjTOT6JgllukJMq0nacxsMHA3kA084O6bD75EyNj8oHu55cvvHM+dfzyKE0bsyrsz51L47Qo2Fhfz+tTP6bdTe94YezHLV6zhg08XsHHjpuoJPELu5Y/Q5gPtyZSpS3TMEsv0fUxbgjSzbOA+4ACgAJhmZhPcfXbF76w+hd/9QP62v7T68rZtwZJlK8uUWbpsJcde8gAAjbaqz6H792bVmnUA3PLgFG55cAoAY28czdzF31VT5NHJy8unoGBx6XJhYQG5ubmbl1m8mPz8fDZu3MiqlStp2bJldYeaMXTMErDwkcHS2cXeBZjr7vPdfQPwFHBIGrdXZdNnLaLL9m1on9uKejnZHHVQXya9+WmZMq2aNyr9lrv0lIN4+IWpQHCCp2WzRgD03CGXnjvk8tr7n1fvDkSg/4ABzJ37FQsXLGDDhg2Me/ophg4bUabM0GEjePzRhwF4/rln2Xvf/TK+pZBOOmbxGcl3r2tjFzsPWByzXADsWr6QmZ0OnA5AvcZpDGdzxcWbuPDmZ3jxr78nO8t4+IWpzJn/DX86aygzZ3/NpLf+x179d+C6c0fgDu/MnMsFY54JQs3J5rWHLgBg9Zp1nHLlwxQX1/4udk5ODnfefS/Dhx5EcXExJ40+hR477cR1115N3379GTZ8BKNPOZVTRo9ip25daNGiJY8+/lTp+7t26cDqVavYsGEDL074NxMnv0L3Hj0i3KP00zFLLNO/BCze2EdKKjY7CjjI3X8XLo8CdnH3cxO9J2vrbbxB16PTEk9ttWLavVGHIHXAHrv2Z8aM6SnNZjmtOnnTg69PuvyKx46f4e79UxlDZdLZxS4A2sUs5wNL0rg9EalhUtnFNrPBZvaFmc01s8virN/ezN4ws4/M7FMzO7iyOtOZIKcBO5hZRzOrDxwLTEjj9kSkJrEqPiqq6peTwkOAHsBIMys/DnEV8Iy79yHIR3+tLMS0jUG6+0YzOweYQnCZz0PuPitd2xORmsUwsrJS1kYrPSkMYGYlJ4Vjr5pxoGn4vBlJ9GjTeh2ku08GJqdzGyJSc6XwJE0yJ4WvBV4xs3OBRsCgyiqt03fSiEjEqtbFbm1m02Mep5erqbzyZ6BHAmPdPR84GHjUrOIJKTXdmYhEw6rcglxewVnsZE4KnwoMBnD3982sIdAaSHiHh1qQIhKZFJ7FTuak8NfA/uF2uwMNgWUVVaoWpIhEJlVjkIlOCpvZdcB0d58AXAzcb2YXEnS/R3slF4IrQYpIJEpuNUyVeCeF3f3qmOezgT2qUqcSpIhEJ7PvNFSCFJGIVP0kTbVTghSRyChBiogkENVvzSRLCVJEIqMWpIhIHFFOhJssJUgRiYwSpIhIAkqQIiKJZHZ+VIIUkeioBSkiEo8uFBcRic+ADM+PSpAiEhUjSxeKi4jEpy62iEg8pi62iEhcBupii4gkohakiEgCGoMUEYlHY5AiIvEF10FmdoZUghSRiGi6MxGRhDI8PypBikhETJf5iIjEpTFIEZEKZHh+VIIUkeioBSkikkCG58fMSpC9um3PG+/cHXUYNUrr48ZGHUKNM+fvI6MOocb5udhTX6kmzBURiU8T5oqIJKQLxUVEEsrw/KgEKSIR0YXiIiLx6UJxEZEKKEGKiCSQ4flRCVJEoqMWpIhIPJpRXEQkPtN1kCIiiWV4fiQr6gBEpO7KMkv6URkzG2xmX5jZXDO7LEGZo81stpnNMrMnKqtTLUgRiUyqWpBmlg3cBxwAFADTzGyCu8+OKbMDcDmwh7uvMLNtKqtXCVJEImEG2am7k2YXYK67zw/qtqeAQ4DZMWVOA+5z9xUA7v5dZZWqiy0ikTGzpB9AazObHvM4PaaqPGBxzHJB+FqsHYEdzexdM5tqZoMriy9hC9LMmlb0RndfVVnlIiIVqWIXe7m7909UVZzXyk9imQPsAOwD5ANvm1lPd/8h0QYr6mLPCjcQu+GSZQe2r+C9IiIVMoJLfVKkAGgXs5wPLIlTZqq7/wwsMLMvCBLmtESVJkyQ7t4u0ToRkVRI4WQ+04AdzKwjUAgcCxxXrsy/gZHAWDNrTdDlnl9hfMls2cyONbMrwuf5ZtavisGLiJRVhfHHyi4od/eNwDnAFGAO8Iy7zzKz68xsRFhsClBkZrOBN4BL3b2oonorPYttZvcC9YC9gBuBH4G/AwMqe6+ISEVSeaG4u08GJpd77eqY5w5cFD6SksxlPru7e18z+yjcyPdmVj/ZDYiIxGOQ1AXgUUomQf5sZlmEZ4TMrBWwKa1RiUidkOH5MakxyPuA54A2ZvZn4B3g5rRGJSJ1QqrGINOl0hakuz9iZjOAQeFLR7n7Z+kNS0RquxTfSZMWyd5qmA38TNDN1t03IpISmZ0ek0h2ZnYl8CSQS3Dx5RNmdnm6AxOR2q/Gd7GBE4B+7v4jgJndAMwAxqQzMBGp3YKz2FFHUbFkEuSicuVyqOTqcxGRSkXYMkxWRZNV3Ekw5vgjMMvMpoTLBxKcyRYR+VUyPD9W2IIsOVM9C5gU8/rU9IUjInVJjW1BuvuD1RmIiNQtNWEMMpmz2J3N7Ckz+9TMvix5VEdw1eG1V15mQO8e9N25K3fetvn17+vXr+eUE0fSd+euDNp7N75etBCArxctpG2rxgz8bT8G/rYfF553djVHHp1BvfKYeddhfHLP4Vx0yM6brc9v1YjJVx/EuzcPZ+qtIziwTzBv6fZtGrPssRN475YRvHfLCO4+bbfqDj0yb77+CvvssjMD+/fgvrtu3Wz9B++9zcH7/paO2zRi0oTny6wbddRwenbcltEjD6uucKtNbTiLPRa4HrgNGAKcTC251bC4uJhLLzqP8S++TG5ePvsN/C1Dhg6nW/cepWUeffghmjVvwcz/fcFz457m2j9dzkOPPAlAh46deXvqjKjCj0SWGXecuisjrn+FwqIf+e+YYUye/jWfF64sLfPHI37D8+8v5IFXv6BbXjOeu/wAdjrnWQAWfLOa3f8wIaLoo1FcXMxVfzifx5+bRNvcfIYP2oMDBg9jx27dS8vk5rfj9nvv5x/33rnZ+88450J++uknHn/4geoMO+3MIDvDu9jJXPS9tbtPAXD3ee5+FbBvesOqHjOmf0inTp3p0LET9evX5/Ajj2byxLJ/vC9NnMDI40cBcMhhR/DWm/8hmBSkburfpTXzv1nNwu/W8HPxJp59bwFDB5SdO9kdmmxdD4CmW9dn6Yofowg1Y3w8cxodOnamfYfgczb8sKN45aUXy5Rpt30Huu+0M1lZm/9J7rn3fjRu3Li6wq1WZsk/opBMglxvQft2npmdaWbDgUp/DawmWLpkCXn5v8wLnJuXz9KlZSchXhJTJicnh6ZNm/F9UTCF3NeLFrDXbv0ZetC+vPfu29UXeIRyW25NQdHa0uXCorXktty6TJkbxn3MsQM788XfjuK5ywdxyUMflK5rv01j3r15OC9fO5jdu9WKj1Glvlm6hNy8/NLltrl5fLu0/GTXdVNt6GJfCDQGzgNuAJoBp1T2JjN7CBgGfOfuPX9NkOkSryW4+T9E/DLbbteW/32+gJatWvHxRzM4/pgjeH/6pzRtWuFP+dR48T6n5Q/jUXt05LE35/J/E2exyw5teODcgQy4+N98s+JHup/9LN+vWU/vjq146tL9GHDxv1n908/VE3xEkvuc1U2ZfhgqbUG6+wfuvtrdv3b3Ue4+wt3fTaLusUClvxoWpdy8PAoLfvkhtCWFBWy3XduyZXJ/KbNx40ZWrVpJi5YtadCgAS1btQKgd59+dOzUiXlza825q4QKi34kv1Wj0uW8Vo0260KftN8OPP/+AgA+/GoZDepl07pJQzZs3MT3a9YD8PGCIhZ8u5oubWv3FwoELcYlhQWly0uXFLJNuc9ZXWQYWZb8IwoJE6SZjTez5xM9KqvY3f8LfJ/SaFOsb78BzJs3l0ULF7Bhwwaef/YZhgwdXqbM4KHDefLxRwF4Yfxz7LX3vpgZy5cto7i4GICFC+Yzf+5cOnToVO37UN1mzFtO57ZNad+mMfWyszhy945Mnr64TJnFy9eyT89cALrmNaNhvWyWrVpH6yYNSj/oHbZpTOe2TVj47epq34fq1qtPfxbMn8vXi4LP2Yvjx3HAkGFRhxW9Kow/RtXSrKiLfW91BBD+tu3pAPntqveHEnNycrjl9rs54pCDKS4u5vgTR9O9x07c+Jdr6N23PwcPHc6ok07hzN+dRN+du9KiRQsefPgJAN57923GXH8t2dk5ZGdnc/s999GiZctqjT8KxZucix+ayr+vPIDsLOPRN+Yyp+AHrjq6NzPnFTF5xmKueGQa/3fG7pwztAcOnPHX4MarPXpsx1VH92ZjsVO8yTn//vdZsXZDtDtUDXJycvjLzXcx6qjhFBcXc8xxJ9G1Ww9uH/Nndu7djwOHDOOTmdM57cRjWLlyBa9NmcwdN/2F19/7CIAjhu7HvK++ZO3aNezSszO33vN39t7vgIj3KjUyfajB0nlG1sw6ABOTHYPs07e/v/HOB5UXlFL5ox+NOoQaZ87fR0YdQo0zdL/d+fTjGSnNZtt06enH3Dou6fL3Ht5jRgW/i50Wyc4HKSKSUkbmtyCVIEUkMjX+VsMSZtagKhWb2ZPA+0BXMysws1OrGpyI1F4lP7mQ7CMKyfwu9i7AgwTXP25vZr2A37n7uRW9z9010CMiFaoNLch7CC74LgJw90+oJbcaiki0avJlPiWy3H1RucHU4jTFIyJ1RDDdWWY3IZNJkIvDbrabWTZwLlD7bxkRkbTL9J9ITSZBnkXQzd4e+BZ4LXxNRORXyfAGZOUJ0t2/A46thlhEpA6xCO+xTlYyZ7HvJ86UNu5+eloiEpE6I8PzY1Jd7NdinjcEDgMWJygrIpK0TL/MJ5ku9tOxy2b2KPBq2iISkTrBILILwJO1JbcadgTapzoQEaljrBa0IM1sBb+MQWYRzPF4WTqDEpG6wcjsDFlhggx/i6YXUBi+tMnr8i9WiUjK1PjfxQ6T4Xh3Lw4fSo4ikjJZlvwjkviSKPOhmfVNeyQiUufU2F81NLMcd98I7AmcZmbzgLUELWN3dyVNEdliNaGLXdEY5IdAX+DQaopFROqSFM/SY2aDgbuBbOABd78pQbkjgXHAAHefXlGdFSVIA3D3eVsWrohIxVJ1q2E4kc59wAFAATDNzCa4++xy5ZoA5wFJ/fhVRQmyjZldlGilu9+RzAZEROJJcRd7F2Cuu88HMLOngEOA2eXK/QW4BbgkmUorOkmTDTQGmiR4iIj8Cka2Jf8AWpvZ9JhH7HwQeZS9BbogfO2XrZn1Adq5+8RkI6yoBbnU3a9LtiIRkaoIftWwSm9ZXsHPvsarqfSyRDPLAu4ERldlg5WOQYqIpEVqr28sANrFLOcDS2KWmwA9gTfDS4a2AyaY2YiKTtRUlCD33/JYRUQql8L5IKcBO5hZR4I7/44FjitZ6e4rgdYly2b2JnBJZWexE45Buvv3vzJgEZGESrrYqfjRrvCa7XOAKcAc4Bl3n2Vm15nZiC2NcUtm8xERSYlUziju7pOByeVeuzpB2X2SqVMJUkQiUxtmFBcRSTmjdvyqoYhI6hmRTUKRLCVIEYlMZqdHJUgRiYhByR0yGUsJUkQik+H5UQlSRKIS3US4yVKCFJFI6Cy2iEgF1IIUEUkgs9NjhiVIM8jJzvRDllkKHh4VdQg1TtuDNItfVa2ftzT1leo6SBGR+DQGKSJSAbUgRUQSqMk/+yoikjZBFzuzM6QSpIhEJsN72EqQIhIVw9SCFBGJTy1IEZE4NAYpIpJIEj/GFTUlSBGJjBKkiEgCOkkjIhKHoQvFRUQSSuXvYqeDEqSIREZdbBGRONTFFhFJSHfSiIjEp+sgRUQSy/D8qAQpItEIxiAzO0UqQYpIZDI7PSpBikiUMjxDKkGKSGTUxRYRSSCz06MSpIhEKcMzpBKkiETC0K2GIiLx6UJxEZHEMjw/khV1ACJSh1kVHpVVZTbYzL4ws7lmdlmc9ReZ2Wwz+9TMXjez9pXVqQQpIhGxKv1XYU1m2cB9wBCgBzDSzHqUK/YR0N/dfwM8C9xSWYRKkCISGbPkH5XYBZjr7vPdfQPwFHBIbAF3f8PdfwwXpwL5lVVa5xPkq6+8TJ+du9Orx47cfuvNm61fv349J51wLL167Mi+A3dj0cKFAPzntVcZuNsAdu3Xi4G7DeCtN/5TzZFH57VXXmZArx707dmVO2+Lf8xOGTWSvj27Mmiv3fh60cIy6xcv/pr8Ns34v7tur6aIo3fALl345LFz+eyJ87jk+D03W7/9ts2YfOdJfPivs5hy92jy2jQtff3d+89g6oNnMuPh3/O7Ef2rO/S0qUrvOsyPrc1seszj9Jjq8oDFMcsF4WuJnAq8VFmMdfokTXFxMReffy4vTJpCXn4+e++xK0OHDadb919a5o+MfYjmzVvwyewvefaZp7j6qst4+LGnaNW6Nc889wJtc3OZPeszDh0+hC/nL65ga7VDcXExl154HuMnvkxuXj77DfwtQ4aWPWaPjn2IZs1bMPOzL3hu3NNce9XlPPTok6Xrr/zDxQw6cHAU4UciK8u468KhDL3oEQqXreKdf57OxHe+4PNFy0rLjDn7IB6f8jGPv/wJe/ftyHWnD+LUG55nadEa9j37ATb8XEyjreozY+zZTHr3C5YWrY5wj1LHqnYae7m7J/qGiFeRJ9jmCUB/YO/KNlinW5DTp31Ip86d6dipE/Xr1+eIo45h4osTypSZ9OILHHfCiQAceviRvPnGf3B3evXuQ9vcXAC699iJdevWsX79+mrfh+o2Y3pwzDp0DI7Z4UcezeSJZY/ZS5MmMPKEUQAcctgRvPVmcMwAJk14gfYdO5ZJqLXdgO55zCv8noVLV/DzxmLGvf4Zw/bsVqZMtw5teHPGAgDemrmAYXt2BeDnjcVs+LkYgAb1ssnK9Cm4qyiFXewCoF3Mcj6wZPPt2SDgSmCEu1f6B1unE+TSJYXk5f9yTPPy8li6pLBMmSVLlpAflsnJyaFZ02YUFRWVKfPC+Ofo1asPDRo0SH/QEVu6ZAl5eb8cs9y8fJYuKfs5XBJTJicnh6ZNm/F9URFr167l7jtu4Y9XXF2tMUctt3VTCr5bWbpcuGwleW2alCnzv7nfcOjewZfGIXt1p2mjhrRsuhUA+ds05cN/ncVXz17E7U+8W2taj5DSk9jTgB3MrKOZ1QeOBcp8c5tZH+AfBMnxu2TiS1uCNLN2ZvaGmc0xs1lmdn66trWlSlo1sco3+SsrM2f2LK6+8nLuvvdvqQ8wAyUqDxXuAAAKUElEQVRzzEhQ5qbrr+Wscy+gcePGaYouM8Vr/ZQ/RJf/9RUG9m7P+w+cycDeHSj8biUbizcBUPDdKnY5+W/0HHkPJwzuxTYtGlVD1NVgCwYhE3H3jcA5wBRgDvCMu88ys+vMbERY7FagMTDOzD42swkJqiuVzjHIjcDF7j7TzJoAM8zsVXefncZtVkluXj6FBb+MGxYWFrJd29wyZfLy8igoWExefj4bN25k5aqVtGzZMihfUMDIo4/gHw+OpVPnztUae1Ry8/IoLPzlmC0pLGC7tm3jlik5ZqtWraRFy5ZMn/YhL4x/nmuuvIyVK38gKyuLBg0acvpZv6/u3ahWhctWkb9Ns9LlvDbNWLK8bCtwadFqjr3qaQAabVWfQ/fqzqq16zcrM3vBMvb4TXvGv5Uxf0a/SipvNXT3ycDkcq9dHfN8UFXrTFsL0t2XuvvM8Plqgqxe0Vmlatev/wDmzZ3LwgUL2LBhA8+Ne5qhw4aXKXPwsBE88dgjAPz7+WfZe599MTN++OEHjjxsOH/+yw3stvseUYQfib79gmO2aGFwzJ5/9hmGDC17zAYfPJwnH3sUCIYf9to7OGYvvfYWn34+j08/n8dZvz+Piy69rNYnR4Dpny+hS35L2rdtTr2cbI7avyeT3v28TJlWzbYubYlfevxAHp78EQB5bZrSsH7QjmneuCG77dyOLxcvr94dSBMjpWOQaVEtZ7HNrAPQB/igOraXrJycHG676x4OHT6ETcXFjDrpZLr32Inr/3wNffr1Y+iwEZw4+hROO+VEevXYkRYtW/KvR54A4J9/u4/58+Zy85gbuHnMDQC8MPFl2myzTZS7lHY5OTnccsfdHDHiYIqLizn+xNF077ETN153Db379ufgYcMZNfoUzjz1JPr27EqLFi14MDxmdVVx8SYuvGsyL942iuysLB6e/BFzFi7jT6fsy8wvljDp3S/Yq3cHrjtjEO7OO58s4oI7JwHQtX1rbvr9QbgHSeKup95j1vykhs9qhEw/5WTxxpRSugGzxsBbwA3u/nyc9acDpwO0a7d9v9lfLUhrPLXNxk3p/ferjdoedF3UIdQ46z+6n02rl6Q0n/Xs1dfHvfx20uV75DaeUcFlPmmR1rPYZlYPeA54PF5yBHD3f7p7f3fv37pNm3SGIyIZJlW3GqZL2rrYFgyoPAjMcfc70rUdEam5Mv2yznS2IPcARgH7hafUPzazg9O4PRGpaVJ4IWQ6pK0F6e7vkPljsCISEc0oLiKSiGYUFxFJLMPzoxKkiEQowzOkEqSIRCS6y3eSpQQpIpHRGKSISBwRXr2TNCVIEYlOhmdIJUgRiUxWhvexlSBFJDKZnR6VIEUkKrpQXESkIpmdIZUgRSQSJTOKZzIlSBGJTIbnRyVIEYmOWpAiIgnoVkMRkUQyOz8qQYpIdDI8PypBikg0zHQnjYhIYpmdH5UgRSQ6GZ4flSBFJDoZ3sNWghSRqGhGcRGRuGrCrYZZUQcgIpKp1IIUkchkegtSCVJEIqMxSBGROIILxaOOomJKkCISHSVIEZH41MUWEUlAJ2lERBLI8PyoBCkiEcrwDKkEKSKRyfQxSHP3qGMoZWbLgEVRxxFHa2B51EHUMDpmWyZTj1t7d2+TygrN7GWC/U3WcncfnMoYKpNRCTJTmdl0d+8fdRw1iY7ZltFxyyy6F1tEJAElSBGRBJQgk/PPqAOogXTMtoyOWwbRGKSISAJqQYqIJKAEKSKSgBKkiEgCSpAJmFlXM9vNzOqZWXbU8dQUOlZVY2ZdzKy/mTWIOhbZnE7SxGFmhwM3AoXhYzow1t1XRRpYBjOzHd39y/B5trsXRx1TpjOzYQSfsyLgG+CakmMomUEtyHLMrB5wDHCqu+8PvAC0A/5gZk0jDS5DhX/oH5vZEwDuXqyWZMXMbHfgNuAkd98XWAFcFm1UUp4SZHxNgR3C5+OBiUB94DizTJ/BrnqZWSPgHOACYIOZPQZKkkm6yd0/Cp9fA7RUVzuzKEGW4+4/A3cAh5vZQHffBLwDfAzsGWlwGcjd1wKnAE8AlwANY5NklLFluA+A56F03LYB0J7gyxkzaxVdaFJCCTK+t4FXgFFmtpe7F7v7E0Au0Cva0DKPuy9x9zXuvhw4A9iqJEmaWV8z6xZthJkn/EyVjGkb8APwvbsvM7PjgevNbKvoIhTQfJBxufs6M3sccODy8A98PbAtsDTS4DKcuxeZ2RnArWb2OZAN7BtxWBnN3TcCa8xssZmNAQ4ERrv7TxGHVucpQSbg7ivM7H5gNkGraB1wgrt/G21kmc/dl5vZp8AQ4AB3L4g6pkwWjmvXAwaG/9/f3b+KNioBXeaTlHCMyMPxSKmEmbUAngEudvdPo46npjCz0cA0d58VdSwSUIKUtDCzhu6+Luo4ahIzM9cfZEZRghQRSUBnsUVEElCCFBFJQAlSRCQBJUgRkQSUIGsJMys2s4/N7DMzG2dmW/+KuvYxs4nh8xFmlnASBTNrbmZnb8E2rjWzS5J9vVyZsWZ2ZBW21cHMPqtqjCJKkLXHT+7e2917AhuAM2NXWqDK/97uPsHdb6qgSHOgyglSpCZQgqyd3ga6hC2nOWb2V2Am0M7MDjSz981sZtjSbAxgZoPN7HMzewc4vKQiMxttZveGz7c1s/Fm9kn42B24Cegctl5vDctdambTzOxTM/tzTF1XmtkXZvYa0LWynTCz08J6PjGz58q1igeZ2dtm9mU43Rpmlm1mt8Zs+4xfeyClblOCrGXMLIfgFr//hS91BR5x9z7AWuAqYJC79yWYCPgiM2sI3A8MJ7jdbbsE1d8DvOXuvYC+wCyCOQznha3XS83sQIKp4nYBegP9zGwvM+sHHAv0IUjAA5LYnefdfUC4vTnAqTHrOgB7A0OBv4f7cCqw0t0HhPWfZmYdk9iOSFy6F7v22MrMPg6fvw08SDD70CJ3nxq+/lugB/BuOK1lfeB9oBuwoOT+33AmntPjbGM/4EQoncpsZXhbYawDw0fJPIeNCRJmE2C8u/8YbmNCEvvU08yuJ+jGNwamxKx7Jrz18yszmx/uw4HAb2LGJ5uF29Ys3bJFlCBrj5/cvXfsC2ESXBv7EvCqu48sV643wcxFqWDAGHf/R7ltXLAF2xgLHOrun4T3Ke8Ts658XR5u+1x3j02kmFmHKm5XBFAXu66ZCuxhZl0AzGxrM9sR+BzoaGadw3IjE7z/deCs8L3Z4U9QrCZoHZaYApwSM7aZZ2bbAP8FDjOzrcysCUF3vjJNgKUW/AzG8eXWHWVmWWHMnYAvwm2fFZbHzHa0YMZzkS2iFmQdEk7GOhp40n6Z2v8qd//SzE4HJpnZcoIZ1HvGqeJ84J9mdipQDJzl7u+b2bvhZTQvheOQ3YH3wxbsGoJp4maa2dMEM7MvIhgGqMyfCGbeXkQwphqbiL8A3iKYo/PMcA7PBwjGJmeGU4gtAw5N7uiIbE6TVYiIJKAutohIAkqQIiIJKEGKiCSgBCkikoASpIhIAkqQIiIJKEGKiCTw/9mQP5LsnCMjAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a1881fcf8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(\"Total prediction runtime: \", runtime)\n",
    "print(f\"Accuracy on development data: {rf_model.score(dev_data[key_features], dev_labels)*100:.2f}%\")\n",
    "get_performance(dev_labels, rf_dev_predictions, labels_names)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.4.3  Feature Importance"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* The most important feature was ROM - range of motion.  This makes sense, considering the different exercises target different muscle groups and involve different motions\n",
    "* PeakVel, PeakVelLoc, weight_lbs, and RepDur all had very similar contribution to the decision tree\n",
    "* The least-important feature was AvgVel - average velocity, which is surprising but may indicate that individuals have very different capabilities"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     features  importance\n",
      "1         ROM    0.519120\n",
      "3  PeakVelLoc    0.144928\n",
      "5  weight_lbs    0.122062\n",
      "2     PeakVel    0.105957\n",
      "4      RepDur    0.059454\n",
      "0      AvgVel    0.048479\n"
     ]
    }
   ],
   "source": [
    "# What were the most useful features?\n",
    "\n",
    "importance = rf_model.feature_importances_   # extract feature importance attribute\n",
    "features = dev_data[key_features].columns\n",
    "importance_table = pd.DataFrame({'importance': importance, 'features': features})  # make table\n",
    "importance_table.sort_values('importance', ascending=False, inplace=True)\n",
    "\n",
    "print(importance_table)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.5 Logistic Regression\n",
    "\n",
    "**Key findings**:\n",
    "\n",
    " - Bench press classificaiton was on par with KNN and Random Forest - 98%\n",
    " - Deadlift classification was the worst of the 4 models, and was only 37% accurate, basically random guessing.\n",
    " - Squat classification was also quite good at 93%, but not significantly better than other models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5.5.1  Model Formation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "LR_model = LogisticRegression()\n",
    "LR_model.fit(train_data[key_features], train_labels)\n",
    "\n",
    "start_time = time.time()\n",
    "LR_predicts = LR_model.predict(dev_data[key_features])\n",
    "runtime = time.time() - start_time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total prediction runtime:  0.02123427391052246\n",
      "Accuracy on development data: 84.78%\n",
      "==================================================\n",
      "Classification report by exercise:\n",
      "             precision    recall  f1-score   support\n",
      "\n",
      "          0       0.90      0.98      0.94      8270\n",
      "          1       0.81      0.37      0.51      3303\n",
      "          2       0.79      0.93      0.85      6551\n",
      "\n",
      "avg / total       0.85      0.85      0.83     18124\n",
      "\n",
      "==================================================\n",
      "Confusion matrix:\n",
      "      0     1     2\n",
      "0  8091    93    86\n",
      "1   585  1210  1508\n",
      "2   301   186  6064\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAUgAAAEmCAYAAAAA6gkZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3Xd8FVX6x/HPkwRE6b0kSK/i0sGGiiKCIPYu9rLuWlfd1bW7tl3Lrq7627WtvYBlRUSxrQWUKjZQkA6hCIhgoyQ8vz9mEm/CvckN5mZuku/b1315Z+bcM88MyZNzzsyca+6OiIhsLyPqAERE0pUSpIhIAkqQIiIJKEGKiCSgBCkikoASpIhIAkqQlYCZXW9mT4bvdzWzH8wss5z3sdjMhpRnnWXY901mttbMVv2KOlJyXiqamf3ZzB6KOg4JKEFSmBxWm1ntmHVnmdm7EYYVl7svdfc67p5fkfs1swFmNsHMvjOzb81smpmdXg71tgYuBbq7e4sdrSeV58XMPPz5yIpZl2Vm35hZUjcSm9n+Zra8tHLufou7n/Vr4pXyowT5iyzgol9biQWq1Hk1sz2Bd4D3gI5AY+A8YHg5VN8GWOfu35RDXan0HUWP9xBgfXnuIDYBS3qoUr/Iv9LtwGVm1iDeRjPby8ymm9mG8P97xWx718xuNrPJwE9A+3DdTWb2Ydj1e8XMGpvZU2a2MayjbUwdd5vZsnDbTDMblCCOtmGLJsvM9gzrLnhtMrPFYbkMM7vCzBaY2TozG2NmjWLqGW1mS8JtVyVxbh5z97+6+1oPzHT3Y2PqO9vM5oety3Fm1ipmm5vZb83sazNbb2b3hX9IhgBvAq3C+B+N19KK7f6HLdkZ4XlabWZ3FT8v4XKrMI5vw7jOjqnv+vB8PG5m35vZbDPrV8o5eAI4JWb5FODxYnGebmZfhnUuNLNzw/W1gddijvOHML7rzex5M3vSzDYCp1nR4ZTjwnrqhcvDzWyVmTUtJVYpL+5e7V/AYmAI8CJwU7juLODd8H0jgtbCaIKW5gnhcuNw+7vAUmC3cHuNcN18oANQH5gDzAv3k0Xwy/WfmBhOJmiZZRF0OVcBtcJt1wNPhu/bAg5kFTuGgn3eGi5fDEwBcoCdgH8Dz4TbugM/APuG2+4C8oAhcc7NLkA+MLiE83cAsBboE9b3T+D9mO0OjAcaALsCa4Bh4bb9geUxZYssx/77hO8/AkaH7+sAe8Q7LwSt3fuBWkCvcJ8HxpzPTQStwEzgVmBKCcfnQA9gdXgMDcL3PQCPKTci/Pc2YD+CP5Z9Sjiu64GtwOEEjZWdY/+twzJPAY+GPxsrgJFR/75Up5dakEVdC1wQ5y/0COBrd3/C3fPc/RngK+DQmDKPuvvscPvWcN1/3H2Bu28gaEEscPe33D0PGAv0Lviwuz/p7uvCz99JkGi6lCH2e4AfgYLW4LnAVe6+3N03E/ziHR22sI4Gxrv7++G2a4BtCeptSPDLu7KEfZ8EPOLuH4f1XQnsGdtCBm5z9+/cfSnwP4KktSO2Ah3NrIm7/+DuU4oXCMc19wH+5O6b3P0T4CGCP3AFJrn7BA/GLJ8Aepay303AK8BxwPHAuHBdIXd/Nfz3dnd/D3gDiNsTiPGRu//X3be5+89xtv+e4A/Qu8Ar7j6+lPqkHClBxnD3LwhaOlcU29QKWFJs3RIgO2Z5WZwqV8e8/znOcp2CBTO7NOyebTCz7whanU2SiTvsyu0PnOjuBYmuDfCSBRdVvgO+JGgJNg+PpzBed/8RWJeg+vUEybNlCSEUOT/u/kNYX+z5ib1C/RMxx15GZwKdga/CYYqRCeL51t2/j1lX/N+reDy1khgDfJyga71d9xoKu8BTwm79dwQt1NL+DeP93BRy9+8I/pj2AO4spS4pZ0qQ27sOOJuiv0wrCBJOrF2B3JjlHZ4WKRxv/BNwLNDQ3RsAGwi6asl89i/AYWFLtcAyYLi7N4h51XL3XILWYOuYOnYh6MJtx91/IujWHlVCGEXOTzjm1pii5ydZPxJ06wvqygQKW/Tu/rW7nwA0A/4KPG8xdx/ExNPIzOrGrCv+77UjPiD4Q9EcmBS7wcx2Al4A7gCah/+GE/jl3zDRz0eJPzdm1gs4A3iGoJcgFUgJshh3nw88B1wYs3oC0NnMTgwvjhxHMI5XXt2dugRjgGuALDO7FqhX2ofCruRzwCnuPq/Y5n8BN5tZm7BsUzM7LNz2PDDSzPYxs5rAjZT8s/BHggsIl5tZ47C+nmb2bLj9aeB0M+sVJopbgKnuvrjUI9/ePILW3AgzqwFcTTDcUHDMJ5tZ07Cl/F24usitPe6+DPgQuNXMapnZbwhank/tQDyx9TrBsMqo8H2smmGca4A8MxsODI3ZvhpobGb1k92fmdUCngT+DJwOZJvZ737FIUgZKUHGdyNQ2Cpx93XASIKLJ+sIEsZId19bTvubSDBGOY+gK7iJUrpeoQOBFgStqIKro7PDbXcTjJO9YWbfE1ywGRgez2yCsa2nCVqT64GE9+i5+4cE42AHAAvN7FvgAYI/HLj72wTjmC+E9XUgGKcrs7AV/DuCMcNcghZlbGzDgNlm9kN4jMe7+6btKgoupLUlaE2+BFzn7m/uSEzF4psdnr/i678n+KM6huB8nkhw/gu2f0XQClwYDnu0Kl5HHLcSXNj5v3Bs92TgJjPr9GuPQ5Jj2/8hFBERUAtSRCQhJUgRkQSUIEVEElCCFBFJIK0ejresnd1q1i29oBTq3W3XqEOQamDJksWsXbu21PtyyyKzXhv3vHgPD8XnP6+Z6O7DyjOG0qRXgqxZl526HFt6QSk0eeq9UYcg1cDeA0uby6PsPO/nMv2+b/rkvqSeLCtPaZUgRaQ6MUjzmQGVIEUkGgZYufbay50SpIhERy1IEZF4DDLS+yuElCBFJDrqYouIxGGoiy0iEp+pBSkikpBakCIiCagFKSISj24UFxGJTzeKi4iUQC1IEZF4DDJ1o7iIyPZ0H6SISAk0BikiEo+uYouIJKYWpIhIAmpBiojEYXoWW0QkMbUgRUQSUAtSRCQeXcUWEYnP0FcuiIjEpxakiEhiGoMUEUlALUgRkQTUghQRicM0BikikphakCIi8VmaJ8j0bt9WgIP26sanL13DFy9fx2WnH7Td9l1bNmTCvy5g2nNXMvHBi8hu1qBw280XHcbM569i1gtXc+cfj67IsCP1xsTX+c1uXdita0du/9tt223fvHkzJ594HLt17cigvQayZPFiANatW8fBQwbTpEEdLr7w/AqOOlo6Z9sLvpLGkn5FoVonyIwM4x9XHMth599P76Nu4phhfenavkWRMrdecgRPvTqNAcfdyi0PvMaNF4wCYI+e7dizV3v6H3sLfY+5mb67tWFQ305RHEaFys/P5+ILf8/Lr7zGrM/mMPbZZ/hyzpwiZR595GEaNmjI7K/mc8FFl3DVn/8EQK1atbj2+r9w61/viCL0yOicJWCGZST/ikK1TpD9e7RlwbK1LM5dx9a8fMZO/JiR+/+mSJmu7Vvy7tS5ALw3fR4j998dAHfYqWYNatbIYqeaWWRlZfLNtxsr/Bgq2vRp0+jQoSPt2renZs2aHHPc8Yx/5eUiZca/8jInjT4VgCOPOpp333kbd6d27drsvc8+1KpVK4rQI6NzlphakGmsVbP6LF+9vnA5d/V6spvWL1Lm83m5HH5gLwAOO6An9ersTKP6tZn62SLen/E1i968mUVv3MJbH37J3EWrKzT+KKxYkUtOTuvC5ezsHHJzc7cv0zook5WVRb369Vm3bl2FxplOdM4Sq9YJ0syGmdlcM5tvZlekcl87wtj+pHux5Sv//hKD+nbko2f+xKC+HcldvZ68/Hzat25Cl3bN6Xjw1XQ4+Cr2H9CZvft0qJjAI+Re/AxtP9CeTJnqROcssXRPkCm7im1mmcB9wEHAcmC6mY1z9zklf7Li5H7zHTnNGxYuZzdvyIo1G4qUWblmA8df9hAAtXeuyeEH9mLjD5s488i9mfb5Yn78eQsAEyfPZuDu7Zj88YKKO4AIZGfnsHz5ssLl3NzltGrVavsyy5aRk5NDXl4eGzdsoFGjRhUdatrQOUvAwlcaS2ULcgAw390XuvsW4FngsBTur8xmzF5Cx12b0qZVY2pkZXLMwX149d3PipRp3KB24V+vy884mMdengLAslXrGdS3I5mZGWRlZTCoTye+WrSqwo+hovXr35/5879m8aJFbNmyhbHPPcuIkaOKlBkxchRPPfEYAC++8Dz7DT6gWrSGEtE5i89IvvVY5VqQQDawLGZ5OTCweCEzOwc4B4AadVIYzvby87dxyV/H8Mr9vyczw3js5Sl8uXAV15w3go/nLOXV9z5n336duPGCUbjDpI/nc/GtYwB48a1Z7Ne/MzPG/BnHefPDL5nw/hcVGn8UsrKy+Pvd93LoiIPJz8/n1NPOoPtuu3Hj9dfSp28/Rh46itPOOJMzThvNbl070rBhI5546tnCz3fp2JbvN25ky5YtvDLuv4yf8AbduneP8IhST+cssXT/I2Dxxj7KpWKzY4CD3f2scHk0MMDdL0j0mYxdmvlOXY5NSTxV1frp90YdglQDew/sx8yZM8o1m2U1bu/1Drkp6fLrnzxpprv3S7TdzIYBdwOZwEPuflux7bsCjwENwjJXuPuEkvaZyi72cqB1zHIOsCKF+xORSqa8utgx1zyGA92BE8yseDP7amCMu/cGjgfuLy2+VCbI6UAnM2tnZjXDgMalcH8iUplYGV8lS+aahwP1wvf1SaLBlrIxSHfPM7PzgYkEzdlH3H12qvYnIpWLYWRklKmN1sTMZsQsP+DuD4Tvk7nmcT3whpldANQGhpS2w5ROVhH270vs44tI9VXGizRrSxiDjFdR8QssJwCPuvudZrYn8ISZ9XD3bYl2WK2fpBGRiJVfFzuZax5nAmMA3P0joBbQpKRKlSBFJBpWrk/SJHPNYylwIICZdSNIkGtKqlTzQYpIZMrrPshE1zzM7EZghruPAy4FHjSzSwi636d5Kfc5KkGKSGTK80bxeNc83P3amPdzgL3LUqcSpIhEouBRw3SmBCki0Unv/KgEKSIRsfR/FlsJUkQiowQpIpJAVN81kywlSBGJjFqQIiJxRDkRbrKUIEUkMkqQIiIJKEGKiCSS3vlRCVJEoqMWpIhIPLpRXEQkPgPSPD8qQYpIVIwM3SguIhKfutgiIvGYutgiInEZqIstIpKIWpAiIgloDFJEJB6NQYqIxBfcB5neGVIJUkQiounOREQSSvP8qAQpIhEx3eYjIhKXxiBFREqQ5vlRCVJEoqMWpIhIAmmeH9MrQXbrlMOY8bdFHUal8tyspVGHUOl8vW5T1CFUOiu+T8E504S5IiLxacJcEZGEdKO4iEhCaZ4flSBFJCK6UVxEJD7dKC4iUgIlSBGRBNI8PypBikh01IIUEYlHM4qLiMRnleA+yIyoAxCR6sss+VfpddkwM5trZvPN7IoEZY41szlmNtvMni6tTrUgRSQyGeXUgjSzTOA+4CBgOTDdzMa5+5yYMp2AK4G93X29mTUrNb5yiU5EZAeUYwtyADDf3Re6+xbgWeCwYmXOBu5z9/UA7v5NaZWqBSkikTCDzLI9SdPEzGbELD/g7g+E77OBZTHblgMDi32+c7BfmwxkAte7++sl7VAJUkQiU8aLNGvdvV+iquKs82LLWUAnYH8gB/jAzHq4+3eJdpgwQZpZvZIidfeNJW0XESlNOV7EXg60jlnOAVbEKTPF3bcCi8xsLkHCnJ6o0pJakLMJMnDsIRQsO7Br0qGLiBRjBLf6lJPpQCczawfkAscDJxYr81/gBOBRM2tC0OVeWFKlCROku7dOtE1EpDyU12Q+7p5nZucDEwnGFx9x99lmdiMww93HhduGmtkcIB+43N3XlVRvUmOQZnY80N7dbzGzHKC5u8/8NQckItWcle+N4u4+AZhQbN21Me8d+EP4Skqpt/mY2b3AYGB0uOon4F/J7kBEJJHyvFE8FZJpQe7l7n3MbBaAu39rZjVTHJeIVHFG+d0onirJJMitZpZBeMnczBoD21IalYhUC2meH5N6kuY+4AWgqZndAEwC/prSqESkWrBwHDKZVxRKbUG6++NmNhMYEq46xt2/SG1YIlLV7cCTNBUu2SdpMoGtBN1sPb8tIuUivdNjclexrwKeAVoR3J3+tJldmerARKTqq/RdbOBkoK+7/wRgZjcDM4FbUxmYiFRtwVXsqKMoWTIJckmxclmU8niOiEipImwZJqukySr+TjDm+BMw28wmhstDCa5ki4j8KmmeH0tsQRZcqZ4NvBqzfkrqwhGR6qTStiDd/eGKDEREqpfKMAaZzFXsDmb2rJl9ZmbzCl4VEVxFmPS/Nxm5b2+G792Th+69c7vtM6ZM4phh+9CzTQPeGP/fItvuvOlqDjugP4fu35dbrrmc4Fn4qu+zj97liqMH88cj92X8Y/dvt/2dF57k6hOGcs1Jw7n57KPIXRj8uHz4+ktcc9LwwtfpA9uyZN7sig4/EgtmvM//nXUw959xEB+OeSBhuS8/eJ2bh3dhxbzPC9etXvQVj15yHP8+dwQPnHcoeVs2V0TIFaIqXMV+FLgJuAMYDpxOFXnUMD8/n5uuvpQHn36ZFi2zOW7EfgweOoIOnbsWlmmZ3Zqb7voXj/77niKfnTVjCrNmTOHFN4MRh1OOOIjpH01iwF6DKvQYKtq2/Hye+Ns1XH7vUzRq1oIbTh1F70FDyG7fubDMngcfxgFHnQzArPff5Jl/3MRl9zzOXsOOYK9hRwCwbP5X3HPZWbTpvFskx1GRtuXn8/p9N3LiLf+hXpPmPHLR0XQaeABN23QsUm7zTz8wfdwTtOrSM+azeYz72+WMuvx2mrfvyk8b15ORWTW+CMAMMtO8i53MTd+7uPtEAHdf4O5XE8zuU+l9/skMdm3bntZt2lGjZk2GH3YU77wxvkiZ7NZt6NK9BxnF+gJmxpbNm9m6ZQtbtmxma14ejZs2rcjwI7Fw9ic0z2lLs+xdyapRk4FDD2XW+28WKbNznbqF7zf//FPcgfipb4xj4NBRqQ43LayY9xmNWrWhYcvWZNaoSff9RjBvytvblXvv8bvZ8+izyKq5U+G6hTMn06xdF5q3D/5o71KvIRmZmRUWe6ql+2w+ySTIzRa0bxeY2W/N7FCg1K9LrAy+WbmSFi2zC5ebt8jmm5Urk/psr74D6b/XIAb37cTgPp3Ye78D6dCpa+kfrOTWr1lFo+YtC5cbNmvJ+jWrtiv31tjHuPyIQYz5562cdOkN222f+uYr7HFw8S+dq5q+X7uauk1bFC7Xa9Kc79etLlJm1fw5bFy7ik4Di7Y9vs1dBGY8c9WZPHT+EXw09sEKibmipHsXO5kEeQlQB7gQ2JvgqxPPKO1DZvaImX1jZmn73LZv950+yV9VW7poAQu/nsvb07/inRlzmTb5PWZMqfp3P8UfZt3+nA055lRuf+kDjjn/Cl555J9Fti34YhY71dqZnA5dUhNk2onzcxZzznzbNt584FaGnP2n7cpty89n2eyZHPbH2zn1jqeZ++FbLJr1UUqjrUiVvgXp7lPd/Xt3X+ruo919lLtPTqLuR4FhvzrCFGreshWrVuYWLq9elUvTFi1K+MQv3nr9FXr2GcAuteuwS+067DN4KJ99nPC7f6qMRs1a8O3qX1rZ679ZScOmzROWHzh0FB+/90aRdVPfeKXadK8B6jZpwfcxreyNa1dTp/EvnbDNP//ImiXzePKPp3DvqQeQ+9UnjL3hPFbM+5y6TVrQZvcB7FK/ETVq7UyH/vuyakHVuLBlGBmW/CsKCROkmb1kZi8mepVWsbu/D3xbrtGWsx49+7J00QKWL13M1i1beO3lFxh80IikPtsyuzUzpkwiLy+PrVu3MmPKJNp3qvotonbde7J62SLW5C4lb+sWpr7xCr0HHVSkzKqliwrffzr5HZq3blu4vG3bNqa/82q1SpCtOu/OtysW892qZeRv3cKc916l8x4HFG6vVbsuf3huKuc/9g7nP/YO2V17ccx1/0erzrvTvu8+rF40l62bfmZbfh5LP59Ok107lrC3SqQMrcd0nFH83ooIwMzOAc6BIOlUpKysLP78lzs496TDyd+2jSOOG03HLt249/ab2K1nbwYPHcHnn8zk4rNOZOOG73j3zde4766befmd6QwdcTjTJr/HEUMGYmbss/8Q9j/okAqNPwqZWVmcfPmN3HHhKWzbls+gQ48lu0NnXvz3nbTr9ht673sQb499jNnTJpGZVYPa9epx9nV3FX5+7qypNGzWkmbZ1edLMTMyszj4vGt55uqz2JafT8+hR9G0TSfee/xuWnbuQec9Dkz42Z3r1mfgkafxyEVHY2Z06L8vnQbsX3HBp1i63yhuqbx3z8zaAuPdvUcy5Xfr2cfHTHg/ZfFURTNWpHUjPS19vW5T1CFUOg9feCQr531RrtmsWcceftztY5Muf++R3We6e7/yjKE0VeOGKhGpdIz0b0EqQYpIZCr9o4YFzGyn0ksVKf8M8BHQxcyWm9mZZQ1ORKqugq9cSPYVhVJbkGY2AHgYqA/samY9gbPc/YKSPufuJ5RPiCJSVVWFFuQ9wEhgHYC7f0oVedRQRKJVmW/zKZDh7kuKDabmpygeEakmgunO0rsJmUyCXBZ2s93MMoELgCoz3ZmIRCfdvyI1mQR5HkE3e1dgNfBWuE5E5FdJ8wZk6QnS3b8Bjq+AWESkGrEIn7FOVjJXsR8kznQk7n5OSiISkWojzfNjUl3st2Le1wKOAJalJhwRqU7S/TafZLrYz8Uum9kTwJsJiouIJMUgshvAk7Ujjxq2A9qUdyAiUs1YFWhBmtl6fhmDzCCY4/GKVAYlItWDxZmNPp2UmCDD76LpCRRMu73Nq8t3m4pISlX678UOk+FL7p4fvpQcRaTcZFjyr0jiS6LMNDPrk/JIRKTaSfdvNUzYxTazLHfPA/YBzjazBcCPBC1jd3clTRHZYZWhi13SGOQ0oA9weAXFIiLVSTnP0mNmw4C7gUzgIXe/LUG5o4GxQH93n1FSnSUlSANw9wU7Fq6ISMnK61HDcCKd+4CDgOXAdDMb5+5zipWrC1wITE2m3pISZFMz+0Oije5+V6JtIiKlKecu9gBgvrsvBDCzZ4HDgDnFyv0F+BtwWTKVlnSRJhOoA9RN8BIR+RWMTEv+BTQxsxkxr9j5ILIp+gj08nDdL3sz6w20dvfxyUZYUgtypbvfmGxFIiJlEXyrYZk+sraEr32NV1PhbYlmlgH8HTitLDssdQxSRCQlyvf+xuVA65jlHGBFzHJdoAfwbnjLUAtgnJmNKulCTUkJ8sAdj1VEpHTlOB/kdKCTmbUjePLveODEgo3uvgFoUrBsZu8Cl5V2FTvhGKS7f/srAxYRSaigi10eX9oV3rN9PjAR+BIY4+6zzexGMxu1ozHuyGw+IiLlojxnFHf3CcCEYuuuTVB2/2TqVIIUkchUhRnFRUTKnVE1vtVQRKT8GZFNQpEsJUgRiUx6p0clSBGJiEHBEzJpSwlSRCKT5vlRCVJEohLdRLjJUoIUkUjoKraISAnUghQRSSC902OaJciamRlkN9o56jAqlTZNcqIOodJpetANUYdQ6WxemYKpGXQfpIhIfBqDFBEpgVqQIiIJVOavfRURSZmgi53eGVIJUkQik+Y9bCVIEYmKYWpBiojEpxakiEgcGoMUEUkkiS/jipoSpIhERglSRCQBXaQREYnD0I3iIiIJlef3YqeCEqSIREZdbBGRONTFFhFJSE/SiIjEp/sgRUQSS/P8qAQpItEIxiDTO0UqQYpIZNI7PSpBikiU0jxDKkGKSGTUxRYRSSC906MSpIhEKc0zpBKkiETC0KOGIiLx6UZxEZHE0jw/KkGKSITSPENmRB2AiFRXVqb/Sq3NbJiZzTWz+WZ2RZztfzCzOWb2mZm9bWZtSqtTCVJEImOW/KvkeiwTuA8YDnQHTjCz7sWKzQL6uftvgOeBv5UWX7VPkG+98Tr9e3anT48u/P2Ov263ffPmzZwx+gT69OjCkH33ZOmSxQAsXbKYlo3qMGhgXwYN7MslF/yugiOPzptvvE7v3bvRs3tn7rw9/jk79eTj6dm9M4MH7cmSxYsBmDF9GnsN6MNeA/qwZ//ejHv5pQqOPDoHDejIp09ewBdPX8hlJ+2z3fZdm9dnwt9PZdp/zmPi3aeR3bRe4frJD57LlId/y8zHfs9Zo/pVdOgpY2V8lWIAMN/dF7r7FuBZ4LDYAu7+P3f/KVycAuSUVmm1HoPMz8/n8ksu5KXxr9MqO4cDBu3B8BGH0rXbL394nnj0Eeo3aMjHX8zlhbHPcf3VV/LIE88A0LZ9Bz6YOjOq8CORn5/PpRddwMuvTiQ7J4f99h7IiJFFz9njjz5CgwYN+XTOPJ4f8yzXXn0Fjz35LN1368H7H04jKyuLVStXsueA3hwy4lCysqr2j2FGhvGPS0Yw4g+Pk7tmI5MeOIfxk+by1ZI1hWVu/d3BPDXxE556/VP269OOG88Zwpk3v8jKdT8w+HcPsWVrPrV3rsnMR3/Hq5PnsnLd9xEeUfmxsl3GbmJmM2KWH3D3B8L32cCymG3LgYEl1HUm8FppO6zWLciZM6bRvkMH2rZrT82aNTny6GOZMH5ckTKvvTqOE04eDcBhRxzFe+++g7tHEW5amDE9OGft2gfn7KhjjmP8K0XP2auvvMyJJ58CwOFHHs27/wvO2S677FKYDDdt2lTWX45Kq3+3bBbkfsvilevZmpfP2Le/YOQ+XYuU6dq2Ke/OXATAex8vYuQ+XQDYmpfPlq35AOxUI5OMdJ+Cu4zK2MVe6+79Yl4PxFYVp/q4v6hmdjLQD7i9tPiqdYJcuWIF2dmtC5dbZeewcsWKImVWxJTJysqiXr36fLtuHQBLFy9i3z36MWLoYD6c/EHFBR6hlStyyc755ZxlZ2ezckVukTIrVqwgJ+eXc1a/Xn3Wheds+rSp9O+9O3v068k//nl/lW89ArRqUo/l32woXM5ds4HspnWLlPl8/ioO3y9ohR+2bzfq1a5Fo3o7A5DTrB77vi3CAAAK4ElEQVTT/nMeXz//B+58enKVaT1CuXaxlwOtY5ZzgBXFC5nZEOAqYJS7by6t0pQlSDNrbWb/M7MvzWy2mV2Uqn3tqHgtwe1aNQnKNG/Rks/nLuL9KTO4+bY7OPu00WzcuDFVoaaNZM5ZSWX6DxjI9Fmf8+7kqdx1+1/ZtGlTagJNI/EaysVP0ZX3v8GgXm346KHfMqhXW3K/2UBe/jYAln+zkQGn/x89TriHk4f1pFnD2hUQdQUo30HI6UAnM2tnZjWB44EiXRsz6w38myA5fpNMiKlsQeYBl7p7N2AP4PdxripFqlV2Nrm5vwxbrMhdTouWLROWycvLY+PGDTRs1IiddtqJRo0bA9CrT1/atW/Pgq/nVVzwEWmVnUPu8l/OWW5uLi1atipSJjs7m+XLfzlnGzZuoFGjRkXKdO3ajV12qc2c2V+kPuiI5a7ZSE6z+oXL2U3rs2Jt0VbgynXfc/zVz7HnWf/iugffBmDjj5u3KzNn0Rr2/k2pd6dUGuV1m4+75wHnAxOBL4Ex7j7bzG40s1FhsduBOsBYM/vEzMYlqK5QyhKku69094/D99+HQWenan87ok/f/iyYP58lixexZcsWXnx+DMNHHFqkzLBDDuWZJ58A4OWXXmDf/QZjZqxds4b8/GBsaPGihSycP5+27dpX+DFUtL79gnO2eFFwzl4Y+xwjRhY9Z4eMHMXTTz4OwH9ffJ799g/O2eJFi8jLywNg6ZIlfP31XHZt07aiD6HCzfhqBR1zGtGmZQNqZGVyzIE9eHXyV0XKNK6/S2Er+/KTBvHYhFkAZDetR62awTBEgzq12HP31sxbtrZiDyBFjPK7zQfA3Se4e2d37+DuN4frrnX3ceH7Ie7e3N17ha9RJddYQVexzawt0BuYWhH7S1ZWVhZ/u+tujhp1CPn5+Zx0yml0674bt9x4Hb369OOQkYcy+rQz+O2Zp9KnRxcaNmzIw48/DcCHkz/g1r9cT2ZWFpkZmdx5z300LNZKqoqysrK44x/3cPihw9mWn8/oU0+nW/fduOmG6+jdty8jRo7ilNPO4OwzTqFn9840bNSI/4Tn7KMPJ3HXHX+jRo0aZGRkcNfd99KkSZOIjyj18vO3cck/JvDKHaPJzMjgsQmz+HLxGq45YzAfz13Bq5Pnsm+vttx47hDcnUmfLuHiv78KQJc2Tbjt9wfjHiSJfzz7IbMXJtU7rBTS/ZKTpfqKrJnVAd4Dbnb3F+NsPwc4ByCn9a59P5+7MKXxVDVZVeyqZkVoetANUYdQ6Wye9SDbvl9Rrj9sPXr28bGvJ39xs3urOjPdvUJvBE3pVWwzqwG8ADwVLzkCuPsDBZftmzRpmspwRCTNlOejhqmQsi62BQMqDwNfuvtdqdqPiFRe6d4BSmULcm9gNHBAeMXoEzM7JIX7E5HKphxvhEyFlLUg3X0S6T8GKyIR0YziIiKJaEZxEZHE0jw/KkGKSITSPEMqQYpIRKK7fSdZSpAiEhmNQYqIxBHh3TtJU4IUkeikeYZUghSRyGSkeR9bCVJEIpPe6VEJUkSiohvFRURKkt4ZUglSRCJRMKN4OlOCFJHIpHl+VIIUkeioBSkikoAeNRQRSSS986MSpIhEJ83zoxKkiETDTE/SiIgklt75UQlSRKKT5vlRCVJEopPmPWwlSBGJimYUFxGJqzI8apgRdQAiIulKLUgRiUy6tyCVIEUkMhqDFBGJI7hRPOooSqYEKSLRUYIUEYlPXWwRkQR0kUZEJIE0z49KkCISoTTPkEqQIhKZdB+DNHePOoZCZrYGWBJ1HHE0AdZGHUQlo3O2Y9L1vLVx96blWaGZvU5wvMla6+7DyjOG0qRVgkxXZjbD3ftFHUdlonO2Y3Te0ouexRYRSUAJUkQkASXI5DwQdQCVkM7ZjtF5SyMagxQRSUAtSBGRBJQgRUQSUIIUEUlACTIBM+tiZnuaWQ0zy4w6nspC56pszKyjmfUzs52ijkW2p4s0cZjZkcAtQG74mgE86u4bIw0sjZlZZ3efF77PdPf8qGNKd2Y2kuDnbB2wCriu4BxKelALshgzqwEcB5zp7gcCLwOtgT+aWb1Ig0tT4S/6J2b2NIC756slWTIz2wu4AzjV3QcD64Eroo1KilOCjK8e0Cl8/xIwHqgJnGiW7jPYVSwzqw2cD1wMbDGzJ0FJMkm3ufus8P11QCN1tdOLEmQx7r4VuAs40swGufs2YBLwCbBPpMGlIXf/ETgDeBq4DKgVmySjjC3NTQVehMJx252ANgR/nDGzxtGFJgWUIOP7AHgDGG1m+7p7vrs/DbQCekYbWvpx9xXu/oO7rwXOBXYuSJJm1sfMukYbYfoJf6YKxrQN+A741t3XmNlJwE1mtnN0EQpoPsi43H2TmT0FOHBl+Au+GWgOrIw0uDTn7uvM7FzgdjP7CsgEBkccVlpz9zzgBzNbZma3AkOB09z954hDq/aUIBNw9/Vm9iAwh6BVtAk42d1XRxtZ+nP3tWb2GTAcOMjdl0cdUzoLx7VrAIPC/x/o7l9HG5WAbvNJSjhG5OF4pJTCzBoCY4BL3f2zqOOpLMzsNGC6u8+OOhYJKEFKSphZLXffFHUclYmZmesXMq0oQYqIJKCr2CIiCShBiogkoAQpIpKAEqSISAJKkFWEmeWb2Sdm9oWZjTWzXX5FXfub2fjw/SgzSziJgpk1MLPf7cA+rjezy5JdX6zMo2Z2dBn21dbMvihrjCJKkFXHz+7ey917AFuA38ZutECZ/73dfZy731ZCkQZAmROkSGWgBFk1fQB0DFtOX5rZ/cDHQGszG2pmH5nZx2FLsw6AmQ0zs6/MbBJwZEFFZnaamd0bvm9uZi+Z2afhay/gNqBD2Hq9PSx3uZlNN7PPzOyGmLquMrO5ZvYW0KW0gzCzs8N6PjWzF4q1ioeY2QdmNi+cbg0zyzSz22P2fe6vPZFSvSlBVjFmlkXwiN/n4aouwOPu3hv4EbgaGOLufQgmAv6DmdUCHgQOJXjcrUWC6u8B3nP3nkAfYDbBHIYLwtbr5WY2lGCquAFAL6Cvme1rZn2B44HeBAm4fxKH86K79w/39yVwZsy2tsB+wAjgX+ExnAlscPf+Yf1nm1m7JPYjEpeexa46djazT8L3HwAPE8w+tMTdp4Tr9wC6A5PDaS1rAh8BXYFFBc//hjPxnBNnHwcAp0DhVGYbwscKYw0NXwXzHNYhSJh1gZfc/adwH+OSOKYeZnYTQTe+DjAxZtuY8NHPr81sYXgMQ4HfxIxP1g/3rVm6ZYcoQVYdP7t7r9gVYRL8MXYV8Ka7n1CsXC+CmYvKgwG3uvu/i+3j4h3Yx6PA4e7+afic8v4x24rX5eG+L3D32ESKmbUt435FAHWxq5spwN5m1hHAzHYxs87AV0A7M+sQljshweffBs4LP5sZfgXF9wStwwITgTNixjazzawZ8D5whJntbGZ1CbrzpakLrLTgazBOKrbtGDPLCGNuD8wN931eWB4z62zBjOciO0QtyGoknIz1NOAZ+2Vq/6vdfZ6ZnQO8amZrCWZQ7xGniouAB8zsTCAfOM/dPzKzyeFtNK+F45DdgI/CFuwPBNPEfWxmzxHMzL6EYBigNNcQzLy9hGBMNTYRzwXeI5ij87fhHJ4PEYxNfhxOIbYGODy5syOyPU1WISKSgLrYIiIJKEGKiCSgBCkikoASpIhIAkqQIiIJKEGKiCSgBCkiksD/A5kxn7QRrnhjAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a23a17fd0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print(\"Total prediction runtime: \", runtime)\n",
    "print(f\"Accuracy on development data: {LR_model.score(dev_data[key_features], dev_labels)*100:.2f}%\")\n",
    "get_performance(dev_labels, LR_predicts, labels_names)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.6 Comparing Models\n",
    "\n",
    "To show more clearly the performance and robustness of each model, a 20-fold cross-validation is run on the training set.  For each fold, the accuracy is recorded.  This analysis provides an indication of how sensitive the model is to the training set.\n",
    "\n",
    "* KNN and Random Forests perform very similarly, with accuracies of 94+% and low standard deviations for the 20-fold CV test.\n",
    "* Gaussian Naive Bayes and Logistic Regression perform noticably worse, with accuracies of 87% and 85%, respectively.\n",
    "* The variation of the GNB and Logistic models was also higher than KNN and RF, implying that these models may be more sensitive to the specific training data values and outliers.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RandomForest: 0.944012 (0.003536)\n",
      "KNN: 0.948406 (0.003758)\n",
      "GNB: 0.869766 (0.007186)\n",
      "LogisticRegression: 0.849052 (0.009306)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAEVCAYAAAAIK+VbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzt3XucVXW9//HXG1DxhjIyx1JU9KTFHDXNyS4HU7LjQTs/zeynIOaNk5WJVnpEw18ihZaXrBR//SyRvICh5+SPzA56aNRDWTGIoIQYYeqI5Sh4wRsXP+eP9R1cbjcze5gNe2bW+/l4zIN1+e61Pmsx+72/67v23qOIwMzMiqVPrQswM7PNz+FvZlZADn8zswJy+JuZFZDD38ysgBz+ZmYF5PC3TpE0VdK3N9G2R0u6p531h0lq2RT77ukkfUPST2pdh/UcDn8rS9J9klZK2mpz7TMibo2II3I1hKT3ba79K3O2pEclvSqpRdLtkvbbXDVsrIi4NCL+tdZ1WM/h8Ld3kTQEOAQI4OjNtM9+m2M/HfgBcA5wNlAH7APcCXy6lkV1pJucO+thHP5WzsnA74CpwCntNZR0vqRnJS2X9K/53rqkHSTdJKlV0pOSLpLUJ607VdJvJF0taQUwIS2bk9Y/kHaxQNIqSSfk9nmupOfSfk/LLZ8q6TpJv0qP+Y2k90j6frqKeUzSgRs4jr2BrwCjIuLXEfFmRLyWrka+08njeVHSMkkfT8ufTvWeUlLrjyTdK+kVSfdL2iO3/gfpcS9LmifpkNy6CZLukHSLpJeBU9OyW9L6/mndC6mWuZJ2Tut2kTRT0gpJSyV9oWS7M9IxviJpkaTG9v7/redy+Fs5JwO3pp9/bguOUpJGAF8HPgW8Dzi0pMk1wA7AXmndycBpufUfAZYBfwdMyj8wIj6RJj8YEdtFxM/S/HvSNncFxgCTJQ3MPfR44CJgEPAm8CDwUJq/A/jeBo75cKAlIv6wgfWVHs9CYCdgGnAb8GGyc3MScK2k7XLtRwPfSrU9THa+28wFDiC7ApkG3C6pf279Mel4dix5HGQv2DsAu6VavgS8ntZNB1qAXYDPAZdKOjz32KNT3TsCM4Fr2zkf1oM5/O0dJA0D9gBmRMQ84M/AiRtofjxwY0QsiojXgEty2+kLnABcGBGvRMRfgKuAz+cevzwiromItRHxOpVZA0yMiDURcTewCnh/bv3PI2JeRLwB/Bx4IyJuioh1wM+Asj1/spB8dkM7rfB4noiIG3P72i3V+mZE3AOsJnshaPPLiHggIt4ExgMfk7QbQETcEhEvpHNzFbBVyXE+GBF3RsRbZc7dmnQ874uIdel8vJy2PQwYFxFvRMTDwE9KjmFORNydjuFm4IMbOifWszn8rdQpwD0R8Xyan8aGh352AZ7OzeenBwFbAk/mlj1J1mMv175SL0TE2tz8a0C+N/233PTrZebzbd+xXeC97ey3kuMp3RcR0d7+1x9/RKwCVpCd07ahrcWSXpL0IllPflC5x5ZxMzALuC0Nx10uaYu07RUR8Uo7x/DX3PRrQH/fU+idHP62nqStyXrzh0r6q6S/Al8DPiipXA/wWWBwbn633PTzZD3QPXLLdgeeyc13p6+UnQ0MbmeMu5Lj6az15ysNB9UBy9P4/jiy/4uBEbEj8BKg3GM3eO7SVdElEdEAfBz4F7IhquVAnaTtq3gM1kM5/C3vM8A6oIFsvPkAYCjw32ThUWoGcJqkoZK2Ab7ZtiING8wAJknaPt3M/DpwSyfq+RvZ+PomFxF/Aq4Dpiv7PMGW6cbpSEkXVOl4Sh0laZikLcnG/n8fEU8D2wNrgVagn6RvAgMq3aik4ZL2S0NVL5O9aK1L2/4tcFk6tv3J7puU3jOwAnD4W94pZGP4T0XEX9t+yG76jS69/I+IXwE/BJqApWQ3VyG70QowFniV7KbuHLIhpCmdqGcC8NP0jpXjN/KYOuNssmOdDLxIdr/jWOAXaX1Xj6fUNOBisuGeg8huAEM2ZPMr4HGyYZk36NwQ2XvIbga/DCwG7uftF6lRwBCyq4CfAxdHxL1dOAbroeQ/5mLVImko8CiwVcm4vJWQNJXs3UUX1boWKyb3/K1LJB2bhkgGAt8FfuHgN+v+HP7WVV8kG5v+M9n9gi/Xthwzq4SHfczMCsg9fzOzAnL4m5kVkMPfzKyAHP5mZgXk8DczKyCHv5lZATn8zcwKyOFvZlZADn8zswJy+JuZFZDD38ysgBz+ZmYF5PA3Mysgh7+ZWQH167jJ5jVo0KAYMmRIrcswM+tR5s2b93xE1FfavtuF/5AhQ2hubq51GWZmPYqkJzvTvqJhH0kjJC2RtFTSBWXW7yFptqSFku6TNDi3bndJ90haLOmPkoZ0pkAzM6u+DsNfUl9gMnAk0ACMktRQ0uxK4KaI2B+YCFyWW3cTcEVEDAUOBp6rRuFmZrbxKun5HwwsjYhlEbEauA04pqRNAzA7TTe1rU8vEv0i4l6AiFgVEa9VpXIzM9tolYT/rsDTufmWtCxvAXBcmj4W2F7STsA+wIuS/kPSfElXpCuJd5B0hqRmSc2tra2dPwozM+uUSsJfZZaV/tX384BDJc0HDgWeAdaS3VA+JK3/MLAXcOq7NhZxfUQ0RkRjfX3FN6vNzGwjVRL+LcBuufnBwPJ8g4hYHhGfjYgDgfFp2UvpsfPTkNFa4E7gQ1Wp3HqF6dOns++++9K3b1/23Xdfpk+fXuuSzAqhkrd6zgX2lrQnWY9+JHBivoGkQcCKiHgLuBCYknvsQEn1EdEKfBLw+zgNyIJ//Pjx3HDDDQwbNow5c+YwZswYAEaNGlXj6sx6tw57/qnHfhYwC1gMzIiIRZImSjo6NTsMWCLpcWBnYFJ67DqyIZ/Zkh4hG0L6cdWPwnqkSZMmccMNNzB8+HC22GILhg8fzg033MCkSZNqXZpZr1fRh7wi4m7g7pJl38xN3wHcsYHH3gvs34UarZdavHgxLS0t7LvvvixevJihQ4cybtw4Fi9eXOvSzHq9bvcJXyuOXXbZhXHjxnHrrbeuH/YZPXo0u+yyS61LM+v1/MVuVlMR0e68mW0aDn+rmeXLl3P55ZczduxY+vfvz9ixY7n88stZvnx5xw82sy7xsI/VzNChQxk8eDCPPvro+mVNTU0MHTq0hlWZFYN7/lYz48ePZ8yYMTQ1NbFmzRqampoYM2YM48ePr3VpZr2ee/5WM23v5R87duz6d/tMmjTJ7/E32wzU3W6wNTY2hr/P38yscyTNi4jGStt72MfMrIAc/mZmBeQxf9vkpHJfDLtxutswpVlP5fAvo5phBQ6sSo5fUuHPk9nm5PAvw2FVmbq6OlauXFm17VXrRXfgwIGsWLGiKtsy660c/rbRVpy9DhhQ6zLKWFfrAsy6PYe/bTRd8nK3vPqRREyodRVm3ZvD37qk2vdHqmHgwIG1LsGs23P420brjr1+M6uM3+dvZlZAhQr/uro6JFXlB6jaturq6mp8ZmrHf8DdrDYKNeyzcuXKbjlU0R3HzTeH6dOnc84557DtttsSEbz66qucc845gP+Au9mmVqiev3Uv559/Pn379mXKlCm8+eabTJkyhb59+3L++efXujSzXq9QPf+4eABM2KHWZbxLXNwd3yu/6bW0tHDhhRe+4yudTzvtNC677LJal2bW6xUq/P2+9O7nxhtvZNq0aev/gPuJJ55Y65LMCsHDPlYz/fr1Y/Xq1e9Ytnr1avr1K1SfxKwmKgp/SSMkLZG0VNIFZdbvIWm2pIWS7pM0uGT9AEnPSLq2WoVvrGq9Q6eaP0X9UNK6devo06cPp59+Ov379+f000+nT58+rFvnr2cw29Q6DH9JfYHJwJFAAzBKUkNJsyuBmyJif2AiUDpo+y3g/q6X2zURUbWfam6vqF9C1tDQwLBhw3j22Wd56623ePbZZxk2bBgNDaW/XmZWbZX0/A8GlkbEsohYDdwGHFPSpgGYnaab8uslHQTsDNzT9XKtNxk+fDh33XUXl156Ka+++iqXXnopd911F8OHD691aWa9XiXhvyvwdG6+JS3LWwAcl6aPBbaXtJOkPsBVwL91tVDrfZqamhg3bhxTpkxh++23Z8qUKYwbN46mpqZal2bW61VyZ63cJ5BK3zJzHnCtpFOBB4BngLXAmcDdEfF0ex9kknQGcAbA7rvvXkFJ1hssXryY+fPn8+1vf3v9sjVr1vitnmabQSU9/xZgt9z8YGB5vkFELI+Iz0bEgcD4tOwl4GPAWZL+QnZf4GRJ3yndQURcHxGNEdFYX1+/cUdiPc7QoUOZM2fOO5bNmTOHoUOH1qgis+KoJPznAntL2lPSlsBIYGa+gaRBaYgH4EJgCkBEjI6I3SNiCNnVwU0R8a53C1kxjR8/njFjxtDU1MSaNWtoampizJgxjB8/vtalmfV6HQ77RMRaSWcBs4C+wJSIWCRpItAcETOBw4DLJAXZsM9XNmHN1ku0fX9P/hO+kyZN8vf6mG0G6m6feG1sbIzm5uaa1uA/4G5mPY2keRHRWGl7f5SyDIe1mfV2/noHM7MCcvibmRWQw9/MrIAc/mZmBeTwNzMrIIe/mVkBOfzNzArI4W9mVkAOfzOzAnL4m5kVkMPfzKyAHP5mZgXk8DczKyCHv5lZATn8zcwKyOFvZlZADn8zswJy+JuZFZDD38ysgBz+ZmYF5PA3Mysgh7+ZWQE5/M3MCqii8Jc0QtISSUslXVBm/R6SZktaKOk+SYPT8gMkPShpUVp3QrUPwMzMOq/D8JfUF5gMHAk0AKMkNZQ0uxK4KSL2ByYCl6XlrwEnR8Q/ACOA70vasVrFm5nZxqmk538wsDQilkXEauA24JiSNg3A7DTd1LY+Ih6PiD+l6eXAc0B9NQo3M7ONV0n47wo8nZtvScvyFgDHpeljge0l7ZRvIOlgYEvgz6U7kHSGpGZJza2trZXWbmZmG6mS8FeZZVEyfx5wqKT5wKHAM8Da9RuQ3gvcDJwWEW+9a2MR10dEY0Q01tf7wsDMbFPrV0GbFmC33PxgYHm+QRrS+SyApO2A4yLipTQ/APglcFFE/K4aRZuZWddU0vOfC+wtaU9JWwIjgZn5BpIGSWrb1oXAlLR8S+DnZDeDb69e2WZm1hUdhn9ErAXOAmYBi4EZEbFI0kRJR6dmhwFLJD0O7AxMSsuPBz4BnCrp4fRzQLUPwszMOkcRpcP3tdXY2BjNzc21LsPMrEeRNC8iGitt70/4mpkVkMPfzKyAHP5mZgXk8DczKyCHv5lZATn8zcwKyOFvZlZADn8zswJy+JuZFZDD38ysgBz+ZmYF5PA3Mysgh7+ZWQE5/M3MCsjhb2ZWQA5/M7MCcvibmRWQw9/MrIAc/mZmBeTwNzMrIIe/mVkBOfzNzArI4W9mVkAOfzOzAqoo/CWNkLRE0lJJF5RZv4ek2ZIWSrpP0uDculMk/Sn9nFLN4s3MbON0GP6S+gKTgSOBBmCUpIaSZlcCN0XE/sBE4LL02DrgYuAjwMHAxZIGVq98MzPbGJX0/A8GlkbEsohYDdwGHFPSpgGYnaabcuv/Gbg3IlZExErgXmBE18s2M7OuqCT8dwWezs23pGV5C4Dj0vSxwPaSdqrwsUg6Q1KzpObW1tZKazczs41USfirzLIomT8POFTSfOBQ4BlgbYWPJSKuj4jGiGisr6+voCQzM+uKfhW0aQF2y80PBpbnG0TEcuCzAJK2A46LiJcktQCHlTz2vi7Ua2ZmVVBJz38usLekPSVtCYwEZuYbSBokqW1bFwJT0vQs4AhJA9ON3iPSMjMzq6EOwz8i1gJnkYX2YmBGRCySNFHS0anZYcASSY8DOwOT0mNXAN8iewGZC0xMy8zMrIYU8a4h+JpqbGyM5ubmWpdhZtajSJoXEY2VtvcnfM3MCsjhb2ZWQA5/M7MCcvibmRWQw9/MrIAc/mZmBeTwNzMrIIe/mVkBOfzNzArI4W9mVkAOfzOzAnL4m5kVkMPfzKyAHP5mZgXk8DczKyCHv5lZATn8zcwKyOFvZlZADn8zswJy+JuZFVC/WhdgZpWTVNXtRURVt2c9h8PfrAepNKwlOditXR72MTMroIrCX9IISUskLZV0QZn1u0tqkjRf0kJJR6XlW0j6qaRHJC2WdGG1D8DMzDqvw/CX1BeYDBwJNACjJDWUNLsImBERBwIjgevS8v8NbBUR+wEHAV+UNKQ6pZuZ2caqpOd/MLA0IpZFxGrgNuCYkjYBDEjTOwDLc8u3ldQP2BpYDbzc5arNzKxLKgn/XYGnc/MtaVneBOAkSS3A3cDYtPwO4FXgWeAp4MqIWNGVgs3MrOsqCf9y7y0rfRvBKGBqRAwGjgJultSH7KphHbALsCdwrqS93rUD6QxJzZKaW1tbO3UAZmbWeZWEfwuwW25+MG8P67QZA8wAiIgHgf7AIOBE4D8jYk1EPAf8Bmgs3UFEXB8RjRHRWF9f3/mjMDOzTqkk/OcCe0vaU9KWZDd0Z5a0eQo4HEDSULLwb03LP6nMtsBHgceqVbyZmW2cDsM/ItYCZwGzgMVk7+pZJGmipKNTs3OBL0haAEwHTo3sEyaTge2AR8leRG6MiIWb4DjMzKwT1N0+BdjY2BjNzc21LsOsR/MnfItH0ryIeNew+ob4E75mZgXk8DczKyCHv5lZATn8zcwKyOFvZlZA/j5/s26irq6OlStXVm171frDLwMHDmTFCn8rS2/j8DfrJlauXNkt355Z7b8eZt2Dh33MzArI4W9mVkAOfzOzAnL4m5kVkMPfzKyAHP5mZgXk8DczKyCHv5lZAflDXmbdRFw8ACbsUOsy3iUuHlDrEjaZan+ArTt+SG9DHP5m3YQueblbhockYkKtq9g0KjnfvfUP43jYx8ysgBz+ZmYF5PA3Mysgj/mbdSPd8Rs0Bw4cWOsSbBNw+Jt1E9W8qdhbb1Ja9XjYx8ysgBz+ZmYFVFH4SxohaYmkpZIuKLN+d0lNkuZLWijpqNy6/SU9KGmRpEck9a/mAZiZWed1OOYvqS8wGfgnoAWYK2lmRPwx1+wiYEZE/F9JDcDdwBBJ/YBbgM9HxAJJOwFrqn4UZmbWKZX0/A8GlkbEsohYDdwGHFPSJoC2z4DvACxP00cACyNiAUBEvBAR67petpmZdUUl4b8r8HRuviUty5sAnCSphazXPzYt3wcISbMkPSTp/HI7kHSGpGZJza2trZ06ADMz67xKwr/cG49L30M2CpgaEYOBo4CbJfUhG1YaBoxO/x4r6fB3bSzi+ohojIjG+vr6Th2AWZFIquin0rZWXJW8z78F2C03P5i3h3XajAFGAETEg+mm7qD02Psj4nkASXcDHwJmd7Fus0Lye/etWirp+c8F9pa0p6QtgZHAzJI2TwGHA0gaCvQHWoFZwP6Stkk3fw8F/oiZmdVUhz3/iFgr6SyyIO8LTImIRZImAs0RMRM4F/ixpK+RDQmdGlkXZaWk75G9gARwd0T8clMdjJmZVUbd7TKysbExmpuba12GmRnQc74qQ9K8iGistL0/4WtmVkAOfzOzAnL4m1mvVFdXV/FbY6vxttlKfurq6mp8Vt7mr3Q2s15p5cqV3W6svjt9tsI9fzOzAnL4m5kVkMPfzKyAHP5mZgXk8DczKyCHv5lZATn8zcwKyOFvZlZADn8zswJy+JuZFZDD38ysgPzdPmbWK8XFA2DCDrUu4x3i4gG1LmE9h7+Z9Uq65OVu+cVuMaHWVWQ87GNmVkAOfzOzAnL4m5kVkMPfzKyAHP5mZgXk8DczKyC/1dPMeq3u9DdzAQYOHFjrEtarqOcvaYSkJZKWSrqgzPrdJTVJmi9poaSjyqxfJem8ahVuZtaeiKjKTzW3tWLFihqflbd1GP6S+gKTgSOBBmCUpIaSZhcBMyLiQGAkcF3J+quBX3W9XDMzq4ZKev4HA0sjYllErAZuA44paRNA2+eWdwCWt62Q9BlgGbCo6+WamVk1VBL+uwJP5+Zb0rK8CcBJklqAu4GxAJK2BcYBl7S3A0lnSGqW1Nza2lph6WZmtrEqCf9yd0xKvzBjFDA1IgYDRwE3S+pDFvpXR8Sq9nYQEddHRGNENNbX11dSt5mZdUEl7/ZpAXbLzQ8mN6yTjAFGAETEg5L6A4OAjwCfk3Q5sCPwlqQ3IuLaLlduZmYbrZLwnwvsLWlP4BmyG7onlrR5CjgcmCppKNAfaI2IQ9oaSJoArHLwm5nVXofDPhGxFjgLmAUsJntXzyJJEyUdnZqdC3xB0gJgOnBqdLfvUjUzs/XU3TK6sbExmpuba12GmRmQvoO/m+VkOZLmRURjpe399Q5mZgXkr3cws8Kq9OsfKm3XE64Q2jj8zaywelJYV5uHfczMCsjhb2ZWQA5/M7MCcvibmRWQw9/MrIAc/mZmBeTwNzMrIIe/mVkBdbvv9pHUCjxZ6zoqMAh4vtZF9CI+n9Xl81k9PeVc7hERFf9BlG4X/j2FpObOfImStc/ns7p8Pqunt55LD/uYmRWQw9/MrIAc/hvv+loX0Mv4fFaXz2f19Mpz6TF/M7MCcs/fzKyAelT4S1on6WFJj0r6haQdq7TdIZIerdK2pkp6ItX5sKSzq7HdDezrMEkf31TbrwZJq3LTR0n6k6TdJU2Q9Jqkv9tA25B0VW7+PEkTNlvh3ZSknSVNk7RM0jxJD0o6Nv0uhKT/lWt7l6TD0vR9kpak38nFks6oUf2rOm7V4TZ2kXRHO+t3lHRmpe1Tm7bzs0DSXEkHdLXOakp/M/1T1dxmjwp/4PWIOCAi9gVWAF+pdUEb8G+pzgMi4oeVPkhS307u5zCgW4d/G0mHA9cAIyLiqbT4eeDcDTzkTeCzkgZtjvp6AmV/TupO4IGI2CsiDgJGAoNTkxZgfDubGB0RBwD/CHxX0pabtOBNJCKWR8Tn2mmyI3BmJ9q3GR0RHwSuA67oYpkASKrKH8yKiG9GxH9VY1ttelr45z0I7AogaTtJsyU9JOkRScek5UNSL+fHkhZJukfS1mndQelV/kFyLyKS+ku6MW1nvqThafmpku5MVxxPSDpL0tdTm99JqmuvWEmj0jYflfTd3PJV6VX998DHUl33p17dLEnvTe3OlvRHSQsl3SZpCPAl4GupN3dIFc9tVaXafgx8OiL+nFs1BThhA+duLdmNtq9thhJ7ik8CqyPiR20LIuLJiLgmzS4AXpL0Tx1sZzvgVWDdpimzcyTtkZ6/C9O/u6flf5+eW3PTc2RVWr7+Sl3SP0j6Q3oOLJS0N/Ad4O/TsitK2veVdGV6Li6UNLZMSeuzJT3miHSF9ZCk2yVtl5YfJekxSXMk/VDSXWn5BEnXS7oHuCnt84p0HAslfTG1e6+kB/T2aMYhqe3UNP+IpK+ltlMlfS5NH55y5xFJUyRtlZb/RdIluRz8QLsnPiJ6zA+wKv3bF7idrBcJ2Z+jHJCmBwFLAQFDyELkgLRuBnBSml4IHJqmrwAeTdPnAjem6Q8ATwH9gVPTdrcH6oGXgC+ldlcDX03TU4EngIfTz37ALmk79anWXwOfSe0DOD5NbwH8FqhP8ycAU9L0cmCrNL1j+ncCcF6t/186+D9bQ3aVtn/J8gnAecA3gUvy/79t08AA4C/ADqnthFofT43P5dnA1RtYdxhwF3AIcH9adhdwWJq+D1iSfu9fB75Yo2NYVWbZL4BT0vTpwJ25+kel6S/lnv9Dcs/Xa8h67ABbAlvn15dp/2Xg34F+ab4ud34a0/RXgUvT9CDgAWDbND8u/c72B54G9kzLpwN35X635wFbp/kzgIvS9FZAM7AnWdaMT8v7kmXLQcC9udrbnutTgc/l9rtPWn4Tb2fPX4CxafpM4Cft/V/0tJ7/1pIeBl4A6oB703IBl0paCPwX2av2zmndExHxcJqeBwyRtAPZSb0/Lb85t49hbfMR8RjZV03sk9Y1RcQrEdFKFv6/SMsfIfsFa5Mf9nkE+DBwX0S0RsRa4FbgE6ntOrJfRoD3A/sC96bjvIi3L+kXArdKOonsBa2nWEP2gjZmA+t/CJwiaUDpioh4meyXe5PdN+nJJE1OV69z25ZFxH+ndeWuBEdHxP7A7sB5kvbYTKV25GPAtDR9M9lzsG357Wl6WumDkgeBb0gaR/b1Bq93sK9PAT9Kz0MiYkVu3a2SWsgCvu1q6qNAA/Cb9Jw8BdiDrGO4LCKeSO2ml+xnZq6WI4CT0+N/D+wE7A3MBU5Tdi9rv4h4BVgG7CXpGkkjgJdLtvt+skx7PM3/lLezBOA/0r/zeGcmvUtPC//XIxuz3IPsVb5tuGY0Wa/6oLT+b2SvkJCNHbdZR9bzFlmPuxy1s//8tt7Kzb+Vtrsh7W3zjYhYl2u3KPfCsV9EHJHWfRqYTNYzmKcqjSVuBm8BxwMflvSN0pUR8SLZE/vM0nXJ98leOLbdZBX2HIuAD7XNRMRXgMPJfvfzJtHO2H/qvDwEfGQT1FgNFb//PCKmAUeTXc3MkvTJDh7S3nN/NFmPfBrZc62t/b2552RDRIyh/ec0ZMNq+X2OzW1jz4i4JyIeIAvuZ4CbJZ0cESuBD5JdiXwF+EmZ+tvTlkltWbdBPS38AYiIl8h6g+dJ2oJsWOC5iFiTxujb7dGkwHlJUlsPY3Ru9QNt85L2IeslLeliyb8HDpU0SNlN3VHA/WXaLQHqJX0s7X+LNKbZB9gtIpqA88luaG0HvEJ2qditRcRrwL8AoyWVuwL4HvBFyvyypp7ZDDZ85VAkvwb6S/pybtk2pY0i4h5gIFmIvIukbYADgT+XW18DvyW7cQ3Zc29Omv4dcFyaHln6IABJe5H1wH8IzAT2p/3nxT3Al9o6T6X3myJiDdkV90clDU01/KOk96X226RceIyshz4kPfSEdo5vFvDllFVI2kfStunK67mI+DFwA/AhZW9w6BMR/w78H3Iv9sljZKMX70vzn6d8lnSoR4Y/QETMJ7vBNZJsGKVRUjPZL89jFWziNGCyshu++UvF64C+kh4BfgacGhFvlttAJ2p9FrgQaEo1PxQR/79Mu9Vk43rflbSA7J7Bx8nGA29JNc0nG/d9kWzY6djufsMX1of4COAipRvyuXXPAz8nGw8t5yqysddCi2ww9zNu+TShAAABIklEQVRkHYknJP2B7LJ/XJnmk3h7yLDNrWnoYR4wNSLmbdKCy9tGUkvu5+tkHbnT0rDt54FzUtuvAl9Px/lesqHWUicAj6bj+gBwU0S8QDZM86ik0nft/ITs/tvC9Bw7sXSDabjmKrL7aa1k9/ump/p+B3wgtTkT+E9Jc8hGG8rV17bPPwIPpRvP/4+so3MY8LCk+WQvcj8gG7K+Lx3PVLLcyNf2Bll23Z7y4C3gR2wEf8LXzLqldIXyekSEpJFkN3+P6ehxm4uk7SJilSSRDRP9KSKurnVdleop48ZmVjwHAdemcH2R7J1A3ckXJJ1Cdv9xPlmPvsdwz9/MrIB67Ji/mZltPIe/mVkBOfzNzArI4W9mVkAOfzOzAnL4m5kV0P8ACee+WxMLcYYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a42a37588>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Comparing models\n",
    "# with assistance from:   https://machinelearningmastery.com/compare-machine-learning-algorithms-python-scikit-learn/\n",
    "\n",
    "\n",
    "models = [('RandomForest', rf_model),\n",
    "          ('KNN', knn_model),\n",
    "          ('GNB', GNB_model),\n",
    "          ('LogisticRegression', LR_model)\n",
    "         ]\n",
    "\n",
    "results = []\n",
    "names = []\n",
    "\n",
    "for name, model in models:\n",
    "    \n",
    "    kfold = KFold(n_splits=20, random_state=42)\n",
    "    cv_results = cross_val_score(model, train_data[key_features], train_labels, cv=kfold, scoring='accuracy')\n",
    "    results.append(cv_results)\n",
    "    names.append(name)\n",
    "    \n",
    "    msg = \"%s: %f (%f)\" % (name, cv_results.mean(), cv_results.std())\n",
    "    print(msg)\n",
    "\n",
    "# boxplot algorithm comparison\n",
    "fig = plt.figure()\n",
    "fig.suptitle('Algorithm Comparison')\n",
    "ax = fig.add_subplot(111)\n",
    "plt.boxplot(results)\n",
    "ax.set_xticklabels(names)\n",
    "plt.show()\n",
    "              "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "G1x4sdv9_9Yc"
   },
   "source": [
    "# 6. Post-Model Custom Logic\n",
    "\n",
    "As well as optimizing model hyper-parameters, we can also apply custom logic after the final classification. For simplification, our data is stored in rows of reps. This means our models predict on a repetition level. However, knowledge of the domain and the data informs us that repetitions belonging to the same set are performing the same exercise. So ideally, repetitions should all come to the same consensus as to which exercise is being performed. We couldn't identify a simple way to integrate this ensemble logic into our models, so we decided on a custom implementation that is applied after the model classification.\n",
    "\n",
    "This approach boosted the accuracy of KNN and RandomForest to over 96% !"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Qir_6-1l_9Yd"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "KNN Accuracy Original vs Ensemble: 94.90% vs 96.36%\n",
      "GNB Accuracy Original vs Ensemble: 86.28% vs 86.84%\n",
      "RF Accuracy Original vs Ensemble: 94.34% vs 96.35%\n"
     ]
    }
   ],
   "source": [
    "def set_consensus(predictions, set_ids):\n",
    "    \"\"\"\n",
    "    Takes in the original model predictions and their associated\n",
    "    setIDs. Polls the predictions within a set to establish a \n",
    "    consensus. Sets all predictions within a set to the consensus\n",
    "    label.\n",
    "    \n",
    "    Args:\n",
    "        predictions: predicted labels produced by model\n",
    "        set_ids: list of setIDs tied to the predictions\n",
    "    \n",
    "    Returns:\n",
    "        consensus_pred: new predictions\n",
    "    \"\"\"\n",
    "    \n",
    "    consensus_pred = predictions\n",
    "    \n",
    "    for i, setID in enumerate(set_ids):\n",
    "        set_ix = [i for i in range(len(set_ids)) if set_ids[i] == setID]\n",
    "        set_pred_mode = mode(consensus_pred[set_ix])[0][0]\n",
    "        consensus_pred[i] = set_pred_mode\n",
    "        \n",
    "    return consensus_pred\n",
    "\n",
    "\n",
    "consensus_pred = set_consensus(knn_predicts, dev_data['setID_ix'].tolist())\n",
    "correct_ct = 0\n",
    "for i, prediction in enumerate(consensus_pred):\n",
    "    if prediction == dev_labels[i]:\n",
    "        correct_ct += 1\n",
    "    \n",
    "print(\"KNN Accuracy Original vs Ensemble:\"\n",
    "      ,f\"{knn_model.score(dev_data[key_features], dev_labels)*100:.2f}% vs {correct_ct / len(dev_labels) * 100:.2f}%\")\n",
    "\n",
    "consensus_pred = set_consensus(GNB_predicts, dev_data['setID_ix'].tolist())\n",
    "correct_ct = 0\n",
    "for i, prediction in enumerate(consensus_pred):\n",
    "    if prediction == dev_labels[i]:\n",
    "        correct_ct += 1\n",
    "\n",
    "print(\"GNB Accuracy Original vs Ensemble:\"\n",
    "      ,f\"{GNB_model.score(dev_data[key_features], dev_labels)*100:.2f}% vs {correct_ct / len(dev_labels) * 100:.2f}%\")\n",
    "\n",
    "consensus_pred = set_consensus(rf_dev_predictions, dev_data['setID_ix'].tolist())\n",
    "correct_ct = 0\n",
    "for i, prediction in enumerate(consensus_pred):\n",
    "    if prediction == dev_labels[i]:\n",
    "        correct_ct += 1\n",
    "\n",
    "print(\"RF Accuracy Original vs Ensemble:\"\n",
    "      ,f\"{rf_model.score(dev_data[key_features], dev_labels)*100:.2f}% vs {correct_ct / len(dev_labels) * 100:.2f}%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Part 1 Summary  \n",
    "Overall, we are extremely excited about the results of this part of the project. We have met both of our goals for exercise classification, with accuracies greather than 96% and runtimes all under 1000ms. We can confidently say that it is completely possible to accurately classify an exercise in real-time using the OpenBarbell device. \n",
    "\n",
    "Please continue on to part 2 of our project where we cover lifter clustering."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "include_colab_link": true,
   "name": "w207_openbarbell_baseline_submission.ipynb",
   "provenance": [],
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}