Smarter-ML0101EN-Proj-Loan-Ayushman.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "# Classification algorithms: \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "import itertools\n",
    "import numpy as np\n",
    "import pylab as pl\n",
    "import scipy.optimize as opt\n",
    "import matplotlib.pyplot as plt\n",
    "from matplotlib.ticker import NullFormatter\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.ticker as ticker\n",
    "from sklearn import preprocessing\n",
    "from sklearn.metrics import classification_report, confusion_matrix, jaccard_similarity_score\n",
    "import itertools\n",
    "from sklearn import metrics\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### About dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "This dataset is about past loans. The __Loan_train.csv__ data set includes details of 346 customers whose loan are already paid off or defaulted. It includes following fields:\n",
    "\n",
    "| Field          | Description                                                                           |\n",
    "|----------------|---------------------------------------------------------------------------------------|\n",
    "| Loan_status    | Whether a loan is paid off on in collection                                           |\n",
    "| Principal      | Basic principal loan amount at the                                                    |\n",
    "| Terms          | Origination terms which can be weekly (7 days), biweekly, and monthly payoff schedule |\n",
    "| Effective_date | When the loan got originated and took effects                                         |\n",
    "| Due_date       | Since it’s one-time payoff schedule, each loan has one single due date                |\n",
    "| Age            | Age of applicant                                                                      |\n",
    "| Education      | Education of applicant                                                                |\n",
    "| Gender         | The gender of applicant                                                               |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "Lets download the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "data = \"https://raw.githubusercontent.com/ayushman1024/Machine-Learning-Deep-Learning-Algorithms/master/DATA/loan_train.csv\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Load Data From CSV File  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/8/2016</td>\n",
       "      <td>10/7/2016</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/8/2016</td>\n",
       "      <td>10/7/2016</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>9/8/2016</td>\n",
       "      <td>9/22/2016</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/9/2016</td>\n",
       "      <td>10/8/2016</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/9/2016</td>\n",
       "      <td>10/8/2016</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           0             0     PAIDOFF       1000     30       9/8/2016   \n",
       "1           2             2     PAIDOFF       1000     30       9/8/2016   \n",
       "2           3             3     PAIDOFF       1000     15       9/8/2016   \n",
       "3           4             4     PAIDOFF       1000     30       9/9/2016   \n",
       "4           6             6     PAIDOFF       1000     30       9/9/2016   \n",
       "\n",
       "    due_date  age             education  Gender  \n",
       "0  10/7/2016   45  High School or Below    male  \n",
       "1  10/7/2016   33              Bechalor  female  \n",
       "2  9/22/2016   27               college    male  \n",
       "3  10/8/2016   28               college  female  \n",
       "4  10/8/2016   29               college    male  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv(data)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(346, 10)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Convert to date time object "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-09-22</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           0             0     PAIDOFF       1000     30     2016-09-08   \n",
       "1           2             2     PAIDOFF       1000     30     2016-09-08   \n",
       "2           3             3     PAIDOFF       1000     15     2016-09-08   \n",
       "3           4             4     PAIDOFF       1000     30     2016-09-09   \n",
       "4           6             6     PAIDOFF       1000     30     2016-09-09   \n",
       "\n",
       "    due_date  age             education  Gender  \n",
       "0 2016-10-07   45  High School or Below    male  \n",
       "1 2016-10-07   33              Bechalor  female  \n",
       "2 2016-09-22   27               college    male  \n",
       "3 2016-10-08   28               college  female  \n",
       "4 2016-10-08   29               college    male  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['due_date'] = pd.to_datetime(df['due_date'])\n",
    "df['effective_date'] = pd.to_datetime(df['effective_date'])\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "# Data visualization and pre-processing\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Let’s see how many of each class is in our data set "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PAIDOFF       260\n",
       "COLLECTION     86\n",
       "Name: loan_status, dtype: int64"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['loan_status'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "260 people have paid off the loan on time while 86 have gone into collection \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# notice: installing seaborn might takes a few minutes\n",
    "# !conda install -c anaconda seaborn -y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAbjElEQVR4nO3de5wU5Z3v8c9XnBUVoiKjIogzIoqoZMBZjdclsLKo8XaMBuNR3HgO0WhcNvF4y3lpNr7WeCExyVGJuHLIJooasqBLEg1ROYqJF8ARwQshOuooIBA3xiAE8Hf+qJpJgz3MpXuma7q/79erXlP1VNVTv66ZZ35dT1c/pYjAzMwsa3YodQBmZmb5OEGZmVkmOUGZmVkmOUGZmVkmOUGZmVkmOUGZmVkmOUF1EUl7S7pP0uuSFkn6raQzi1T3aElzi1FXd5A0X1J9qeOw0iintiCpWtKzkl6QdHwXHufDrqq7J3GC6gKSBMwBnoyIAyLiCGACMKhE8exYiuOalWFbGAu8GhEjI+KpYsRkrXOC6hpjgL9ExA+bCyLizYj4PwCSekm6VdLzkpZI+nJaPjq92pgl6VVJ96YNHEnj07IFwH9rrlfSrpKmp3W9IOn0tPxCST+V9J/Arwp5MZJmSJoq6Yn0XfDfpcd8RdKMnO2mSlooaZmkf2mlrnHpO+jFaXx9ConNMq9s2oKkOuAW4GRJDZJ2bu3vWVKjpBvTdQsljZL0qKTfS7o43aaPpMfSfV9qjjfPcf9XzvnJ267KVkR4KvIEXA7ctp31k4D/nc7vBCwEaoHRwB9J3l3uAPwWOA7oDbwNDAUEPAjMTfe/Efjv6fzuwHJgV+BCoAno10oMTwENeaa/z7PtDOD+9NinAx8Ah6cxLgLq0u36pT97AfOBEenyfKAe6A88Ceyall8FXFfq35enrpvKsC1cCNyezrf69ww0Apek87cBS4C+QDXwXlq+I/CpnLpWAEqXP0x/jgOmpa91B2AucEKpf6/dNbnrpxtIuoOkcf0lIv6W5I9uhKTPp5vsRtLg/gI8FxFN6X4NQA3wIfBGRPwuLf8JScMmres0SVeky72Bwen8vIj4Q76YIqKj/ef/GREh6SVgdUS8lMayLI2xAThH0iSShjcAGE7SMJt9Ji17On0z/Dck/3isQpRJW2jW1t/zw+nPl4A+EfEn4E+SNkjaHfgzcKOkE4CPgYHA3sCqnDrGpdML6XIfkvPzZCdj7lGcoLrGMuCs5oWIuFRSf5J3h5C8G/pqRDyau5Ok0cDGnKIt/PV31NqgiQLOiojXtqnrKJIGkH8n6SmSd3TbuiIifp2nvDmuj7eJ8WNgR0m1wBXA30bE+2nXX+88sc6LiHNbi8vKTjm2hdzjbe/vebttBjiP5IrqiIjYJKmR/G3m2xFx13biKFv+DKprPA70lnRJTtkuOfOPApdIqgKQdJCkXbdT36tAraQh6XJug3gU+GpO//zI9gQYEcdHRF2eaXsNcns+RfJP4I+S9gZOyrPNM8Cxkg5MY91F0kGdPJ71DOXcFgr9e96NpLtvk6TPAvvn2eZR4Es5n20NlLRXB47RozlBdYFIOo/PAP5O0huSngN+RNJHDfBvwMvAYklLgbvYztVsRGwg6cb4efrB8Js5q28AqoAlaV03FPv1tEdEvEjSDbEMmA48nWebNSR9+DMlLSFp4MO6MUzrZuXcForw93wvUC9pIcnV1Kt5jvEr4D7gt2n3+izyX+2VpeYP5MzMzDLFV1BmZpZJTlBmZpZJTlBmZpZJTlBmZpZJmUhQ48ePD5LvNnjyVC5T0bh9eCqzqd0ykaDWrl1b6hDMMsvtwypVJhKUmZnZtpygzMwsk5ygzMwskzxYrJmVlU2bNtHU1MSGDRtKHUpF6927N4MGDaKqqqrTdThBmVlZaWpqom/fvtTU1JCOG2vdLCJYt24dTU1N1NbWdroed/GZWVnZsGEDe+65p5NTCUlizz33LPgq1gnKKsb+AwYgqSjT/gMGlPrl2HY4OZVeMX4H7uKzivHWqlU07TuoKHUNerepKPWYWet8BWVmZa2YV87tvXru1asXdXV1HHbYYZx99tmsX7++Zd3s2bORxKuv/vXxT42NjRx22GEAzJ8/n912242RI0dy8MEHc8IJJzB37tyt6p82bRrDhg1j2LBhHHnkkSxYsKBl3ejRozn44IOpq6ujrq6OWbNmbRVT89TY2FjIae0WvoIys7JWzCtnaN/V884770xDQwMA5513Hj/84Q/52te+BsDMmTM57rjjuP/++/nmN7+Zd//jjz++JSk1NDRwxhlnsPPOOzN27Fjmzp3LXXfdxYIFC+jfvz+LFy/mjDPO4LnnnmOfffYB4N5776W+vr7VmHqKNq+gJE2X9F76hMrmsm9KekdSQzqdnLPuGkkrJL0m6R+6KnAzs57g+OOPZ8WKFQB8+OGHPP3009xzzz3cf//97dq/rq6O6667jttvvx2Am2++mVtvvZX+/fsDMGrUKCZOnMgdd9zRNS+ghNrTxTcDGJ+n/LaIqEunXwBIGg5MAA5N97lTUq9iBWtm1pNs3ryZX/7ylxx++OEAzJkzh/Hjx3PQQQfRr18/Fi9e3K56Ro0a1dIluGzZMo444oit1tfX17Ns2bKW5fPOO6+lK2/dunUAfPTRRy1lZ555ZjFeXpdrs4svIp6UVNPO+k4H7o+IjcAbklYARwK/7XSEZmY9THMygOQK6qKLLgKS7r3JkycDMGHCBGbOnMmoUaParC9i+4OAR8RWd82VSxdfIZ9BXSbpAmAh8PWIeB8YCDyTs01TWvYJkiYBkwAGDx5cQBhm5cfto2fLlwzWrVvH448/ztKlS5HEli1bkMQtt9zSZn0vvPAChxxyCADDhw9n0aJFjBkzpmX94sWLGT58eHFfRAZ09i6+qcAQoA5YCXwnLc9343ve1B8R0yKiPiLqq6urOxmGWXly+yg/s2bN4oILLuDNN9+ksbGRt99+m9ra2q3uwMtnyZIl3HDDDVx66aUAXHnllVx11VUtXXcNDQ3MmDGDr3zlK13+Grpbp66gImJ187yku4HmeyCbgP1yNh0EvNvp6MzMCjR4n32K+r21wemdch01c+ZMrr766q3KzjrrLO677z6uuuqqrcqfeuopRo4cyfr169lrr734wQ9+wNixYwE47bTTeOeddzjmmGOQRN++ffnJT37CgDL88rja6tsESD+DmhsRh6XLAyJiZTr/z8BRETFB0qHAfSSfO+0LPAYMjYgt26u/vr4+Fi5cWMjrMGuTpKJ+UbeNtlO0oQzcPjrmlVdeaekOs9Jq5XfR7rbR5hWUpJnAaKC/pCbgemC0pDqS7rtG4MsAEbFM0oPAy8Bm4NK2kpOZmVk+7bmL79w8xfdsZ/t/Bf61kKDMzMw81JGZmWWSE5SZmWWSE5SZmWWSE5SZmWWSE5SZlbV9Bw0u6uM29h3UvpE9Vq1axYQJExgyZAjDhw/n5JNPZvny5SxbtowxY8Zw0EEHMXToUG644YaWryzMmDGDyy677BN11dTUsHbt2q3KZsyYQXV19VaP0Hj55ZcBWL58OSeffDIHHngghxxyCOeccw4PPPBAy3Z9+vRpeSTHBRdcwPz58/nc5z7XUvecOXMYMWIEw4YN4/DDD2fOnDkt6y688EIGDhzIxo0bAVi7di01NTUd+p20lx+3YWZlbeU7b3PUdY8Urb5nv5Vv7OytRQRnnnkmEydObBm1vKGhgdWrV3PhhRcydepUxo0bx/r16znrrLO48847W0aK6IgvfOELLaOcN9uwYQOnnHIK3/3udzn11FMBeOKJJ6iurm4Zfmn06NFMmTKlZby++fPnt+z/4osvcsUVVzBv3jxqa2t54403OPHEEznggAMYMWIEkDxbavr06VxyySUdjrkjfAVlZlZkTzzxBFVVVVx88cUtZXV1dSxfvpxjjz2WcePGAbDLLrtw++23c9NNNxXt2Pfddx9HH310S3IC+OxnP9vyQMS2TJkyhWuvvZba2loAamtrueaaa7j11ltbtpk8eTK33XYbmzdvLlrc+ThBmZkV2dKlSz/xSAzI/6iMIUOG8OGHH/LBBx90+Di53XZ1dXV89NFHrR67vdrzOI/Bgwdz3HHH8eMf/7jTx2kPd/GZmXWTbR+Lkau18u3J18VXqHwx5iu79tprOe200zjllFOKevxcvoIyMyuyQw89lEWLFuUt33Zcxddff50+ffrQt2/fLj12R/bfNsZ8j/M48MADqaur48EHH+z0sdriBGVmVmRjxoxh48aN3H333S1lzz//PEOHDmXBggX8+te/BpIHG15++eVceeWVRTv2F7/4RX7zm9/w85//vKXskUce4aWXXmrX/ldccQXf/va3aWxsBKCxsZEbb7yRr3/965/Y9hvf+AZTpkwpStz5uIvPzMragIH7tevOu47U1xZJzJ49m8mTJ3PTTTfRu3dvampq+N73vsdDDz3EV7/6VS699FK2bNnC+eefv9Wt5TNmzNjqtu5nnkmeATtixAh22CG5pjjnnHMYMWIEDzzwwFbPk7rzzjs55phjmDt3LpMnT2by5MlUVVUxYsQIvv/977fr9dXV1XHzzTdz6qmnsmnTJqqqqrjllltanhCc69BDD2XUqFHtfnR9R7XrcRtdzY8TsO7gx21UBj9uIzsKfdxGm118kqZLek/S0pyyWyW9KmmJpNmSdk/LayR9JKkhnX7Y3kDMzMxyteczqBnAttfH84DDImIEsBy4Jmfd7yOiLp0uxszMrBPaTFAR8STwh23KfhURzd/Qeobk0e5mZpmQhY8uKl0xfgfFuIvvS8Avc5ZrJb0g6f9JOr61nSRNkrRQ0sI1a9YUIQyz8uH20Xm9e/dm3bp1TlIlFBGsW7eO3r17F1RPQXfxSfoGyaPd702LVgKDI2KdpCOAOZIOjYhPfEU6IqYB0yD5ELiQOMzKjdtH5w0aNIimpiac2Eurd+/eDBpUWOdapxOUpInA54Cxkb5ViYiNwMZ0fpGk3wMHAb4Fycy6RVVVVcs4ctazdaqLT9J44CrgtIhYn1NeLalXOn8AMBR4vRiBmplZZWnzCkrSTGA00F9SE3A9yV17OwHz0vGZnknv2DsB+JakzcAW4OKI+EPeis3MzLajzQQVEefmKb6nlW1/Bvys0KDMzMw8Fp+ZmWWSE5SZmWWSE5SZmWWSE5SZmWWSE5SZmWWSE5SZmWWSE5SZmWWSE5SZmWWSE5SZmWWSE5SZmWWSE5SZmWWSE5SZmWWSE5SZmWWSE5SZmWVSuxKUpOmS3pO0NKesn6R5kn6X/twjLZekH0haIWmJpFFdFbyZmZWv9l5BzQDGb1N2NfBYRAwFHkuXAU4ieZLuUGASMLXwMM3MrNK0K0FFxJPAtk/GPR34UTr/I+CMnPJ/j8QzwO6SBhQjWDMzqxyFfAa1d0SsBEh/7pWWDwTeztmuKS3biqRJkhZKWrhmzZoCwjArP24fZl1zk4TylMUnCiKmRUR9RNRXV1d3QRhmPZfbh1lhCWp1c9dd+vO9tLwJ2C9nu0HAuwUcx8zMKlAhCephYGI6PxF4KKf8gvRuvs8Af2zuCjQzM2uvHduzkaSZwGigv6Qm4HrgJuBBSRcBbwFnp5v/AjgZWAGsB/6xyDGbmVkFaFeCiohzW1k1Ns+2AVxaSFBmZmYeScLMzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDKpXaOZ5yPpYOCBnKIDgOuA3YH/CTQ/p/raiPhFpyM0M7OK1OkEFRGvAXUAknoB7wCzSZ7/dFtETClKhGZmVpGK1cU3Fvh9RLxZpPrMzKzCFStBTQBm5ixfJmmJpOmS9si3g6RJkhZKWrhmzZp8m5hVLLcPsyIkKEl/A5wG/DQtmgoMIen+Wwl8J99+ETEtIuojor66urrQMMzKituHWXGuoE4CFkfEaoCIWB0RWyLiY+Bu4MgiHMPMzCpMMRLUueR070kakLPuTGBpEY5hZmYVptN38QFI2gU4EfhyTvEtkuqAABq3WWdmZtYuBSWoiFgP7LlN2fkFRWRmZoZHkjAzs4xygjIzs0xygjIzs0xygjIzs0xygjIzs0xygjIzs0wq6DZzs55EvaoY9G5T0eoys67lBGUVI7Zs4qjrHilKXc9+a3xR6jGz1rmLz8zMMskJyszMMskJyszMMskJyszMMskJyszMMskJyszMMqng28wlNQJ/ArYAmyOiXlI/4AGghuSZUOdExPuFHsvMzCpHsa6gPhsRdRFRny5fDTwWEUOBx9JlqzD7DxiApIKn/QcMaPtgZlZ2uuqLuqcDo9P5HwHzgau66FiWUW+tWkXTvoMKrqdYoz+YWc9SjCuoAH4laZGkSWnZ3hGxEiD9ude2O0maJGmhpIVr1qwpQhhm5cPtw6w4CerYiBgFnARcKumE9uwUEdMioj4i6qurq4sQhln5cPswK0KCioh305/vAbOBI4HVkgYApD/fK/Q4ZmZWWQpKUJJ2ldS3eR4YBywFHgYmpptNBB4q5DhmZlZ5Cr1JYm9gtqTmuu6LiEckPQ88KOki4C3g7AKPY2ZmFaagBBURrwOfzlO+DhhbSN1mZlbZPJKEmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZll8gnYXfVEXTMz60Gy+ARsX0GZmVkmdTpBSdpP0hOSXpG0TNI/peXflPSOpIZ0Orl44ZqZWaUopItvM/D1iFicPrRwkaR56brbImJK4eGZmVml6nSCioiVwMp0/k+SXgEGFiswMzOrbEX5DEpSDTASeDYtukzSEknTJe3Ryj6TJC2UtHDNmjXFCMOsbLh9mBUhQUnqA/wMmBwRHwBTgSFAHckV1nfy7RcR0yKiPiLqq6urCw3DrKy4fZgVmKAkVZEkp3sj4j8AImJ1RGyJiI+Bu4EjCw/TzMwqTSF38Qm4B3glIr6bU577La0zgaWdD8/MzCpVIXfxHQucD7wkqSEtuxY4V1IdEEAj8OWCIjQzs4pUyF18CwDlWfWLzodjZmaW8EgSZmaWSR6Lz7qMelUVZVwu9aoqQjRm1tM4QVmXiS2bOOq6Rwqu59lvjS9CNGbW07iLz8zMMskJyszMMskJyszMMskJyszMMskJysysm2Xx8epZ5Lv4zMy6WRYfr55FvoIyM7NMcoIyM7NMchefmZllcuQXJygzM8vkyC/u4jMzs0zqsgQlabyk1yStkHR1ofX5tkwzs8rSJV18knoBdwAnAk3A85IejoiXO1unb8s0M6ssXfUZ1JHAioh4HUDS/cDpQKcTVNbsP2AAb61aVXA9g/fZhzdXrixCROVNyvdsTMsit422FeuGhB16VZV121BEFL9S6fPA+Ij4H+ny+cBREXFZzjaTgEnp4sHAa0UPpP36A2tLePxCOPbSaCv2tRHR6U+LM9Q+yvl3lGXlHHu720ZXXUHlS+lbZcKImAZM66Ljd4ikhRFRX+o4OsOxl0ZXx56V9uHfUWk49kRX3STRBOyXszwIeLeLjmVmZmWoqxLU88BQSbWS/gaYADzcRccyM7My1CVdfBGxWdJlwKNAL2B6RCzrimMVScm7Ugrg2EujJ8feET35dTr20iha7F1yk4SZmVmhPJKEmZllkhOUmZllUsUkKEm9JL0gaW66XCvpWUm/k/RAejMHknZKl1ek62tKHPfukmZJelXSK5KOltRP0rw09nmS9ki3laQfpLEvkTSqxLH/s6RlkpZKmimpd1bPu6Tpkt6TtDSnrMPnWdLEdPvfSZrYna+hs9w2ShK720Y7VEyCAv4JeCVn+WbgtogYCrwPXJSWXwS8HxEHArel25XS94FHImIY8GmS13A18Fga+2PpMsBJwNB0mgRM7f5wE5IGApcD9RFxGMnNMhPI7nmfAWz75cEOnWdJ/YDrgaNIRlO5vrnhZpzbRjdy2+hA24iIsp9Ivof1GDAGmEvyReK1wI7p+qOBR9P5R4Gj0/kd0+1Uorg/Bbyx7fFJRhUYkM4PAF5L5+8Czs23XQliHwi8DfRLz+Nc4B+yfN6BGmBpZ88zcC5wV075VttlcXLbcNtoZ8wlaRuVcgX1PeBK4ON0eU/gvyJic7rcRPJHA3/94yFd/8d0+1I4AFgD/N+0C+bfJO0K7B0RK9MYVwJ7pdu3xJ7KfV3dKiLeAaYAbwErSc7jInrGeW/W0fOcmfPfAW4b3cxtY6vy7Sr7BCXpc8B7EbEotzjPptGOdd1tR2AUMDUiRgJ/5q+X0vlkJvb08v10oBbYF9iV5PJ/W1k8721pLdae9BrcNtw2ukJR20bZJyjgWOA0SY3A/SRdGd8DdpfU/EXl3KGYWoZpStfvBvyhOwPO0QQ0RcSz6fIskka5WtIAgPTneznbZ2WIqb8H3oiINRGxCfgP4Bh6xnlv1tHznKXz3x5uG6XhttHO81/2CSoiromIQRFRQ/JB5OMRcR7wBPD5dLOJwEPp/MPpMun6xyPtNO1uEbEKeFvSwWnRWJJHluTGuG3sF6R30nwG+GPzZXgJvAV8RtIuksRfY8/8ec/R0fP8KDBO0h7pu+RxaVkmuW24bRSge9pGKT4kLNUEjAbmpvMHAM8BK4CfAjul5b3T5RXp+gNKHHMdsBBYAswB9iDpf34M+F36s1+6rUgeFPl74CWSu4RKGfu/AK8CS4EfAztl9bwDM0k+D9hE8m7vos6cZ+BL6WtYAfxjqf/mO/D63Ta6N3a3jXYc20MdmZlZJpV9F5+ZmfVMTlBmZpZJTlBmZpZJTlBmZpZJTlBmZpZJTlAZJmmLpIZ0xOOfStqlle1+IWn3TtS/r6RZBcTXKKl/Z/c36yy3jcrg28wzTNKHEdEnnb8XWBQR381ZL5Lf4cet1dHF8TWSfM9hbSmOb5XLbaMy+Aqq53gKOFBSjZJn39wJLAb2a363lrPubiXPmvmVpJ0BJB0o6deSXpS0WNKQdPul6foLJT0k6RFJr0m6vvnAkuZIWpTWOakkr96sdW4bZcoJqgdIx986ieSb2QAHA/8eESMj4s1tNh8K3BERhwL/BZyVlt+bln+aZNyvfMO8HAmcR/IN/bMl1aflX4qII4B64HJJpR5J2Qxw2yh3TlDZtrOkBpLhXN4C7knL34yIZ1rZ542IaEjnFwE1kvoCAyNiNkBEbIiI9Xn2nRcR6yLiI5IBLI9Lyy+X9CLwDMmAj0MLfmVmhXHbqAA7tr2JldBHEVGXW5B0rfPn7eyzMWd+C7Az+Ye6z2fbDyRD0miS0ZePjoj1kuaTjA1mVkpuGxXAV1AVICI+AJoknQEgaadW7no6UVK/tG/+DOBpkqH9308b4DDgM90WuFkXc9vINieoynE+SXfEEuA3wD55tllAMrJyA/CziFgIPALsmO53A0lXhlk5cdvIKN9mbkBypxLJbbGXlToWsyxx2ygdX0GZmVkm+QrKzMwyyVdQZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSf8f4sNmiIERDx0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x216 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import seaborn as sns\n",
    "\n",
    "bins = np.linspace(df.Principal.min(), df.Principal.max(), 10)\n",
    "g = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\n",
    "g.map(plt.hist, 'Principal', bins=bins, ec=\"k\")\n",
    "\n",
    "g.axes[-1].legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAZwElEQVR4nO3de5BU5bnv8e9PGB0UvKCjjIzAqCigkgFne0MNonLYXvF4iTsexTqecDReQkWPt5yyku0u463U5HiJJFpYiaKGbNBNKhpUOBETL4AjgniLDjoKcolGOQoRfM4fvWb2AA3TM6yZXt3z+1St6l5vr/X28zL98vR61+p3KSIwMzPLmu2KHYCZmVk+TlBmZpZJTlBmZpZJTlBmZpZJTlBmZpZJTlBmZpZJTlApk7SXpEckvSdpvqS/SDojpbpHS5qZRl1dQdIcSfXFjsOKq5z6hKQqSS9JelXSMZ34Pms6q+5S4gSVIkkCZgB/ioh9I+JQ4Fygpkjx9CzG+5o1K8M+cTzwZkSMiIjn04jJtswJKl1jgH9ExC+aCyJiaUT8HwBJPSTdJukVSQsl/c+kfHRytDFN0puSHk46NpLGJWVzgf/aXK+knSQ9mNT1qqTTk/ILJf1W0n8Af9yWxkiaIuk+SbOTb7/fTt5ziaQprba7T9I8SYsl/WQLdY1NvjkvSOLrvS2xWckomz4hqQ64FThJUoOkXlv6XEtqlHRT8to8SSMlPS3pr5IuTrbpLenZZN/Xm+PN877/q9W/T97+VbYiwktKC3AFcOdWXp8I/O/k+Q7APKAWGA38ndy3yu2AvwBHA5XAh8BgQMDjwMxk/5uA/5Y83xV4G9gJuBBoAvpuIYbngYY8ywl5tp0CPJq89+nA58AhSYzzgbpku77JYw9gDjA8WZ8D1AN7AH8CdkrKrwFuKPbfy0vnL2XYJy4E7k6eb/FzDTQClyTP7wQWAn2AKmBFUt4T2LlVXe8CStbXJI9jgclJW7cDZgLHFvvv2lWLh4A6kaR7yHWqf0TEP5H7sA2XdFayyS7kOto/gJcjoinZrwEYBKwB3o+Id5Ly35Dr0CR1nSbpqmS9EhiQPJ8VEX/LF1NEtHfc/D8iIiS9DnwSEa8nsSxOYmwAzpE0kVyHqwaGkeuQzY5Iyl5IvgRvT+4/HOtmyqRPNGvrc/1k8vg60DsivgC+kLRW0q7A/wNuknQs8A3QH9gLWN6qjrHJ8mqy3pvcv8+fOhhzSXGCStdi4MzmlYi4VNIe5L4VQu5b0OUR8XTrnSSNBta1KtrAf/5ttjRZooAzI+KtTeo6nNwHP/9O0vPkvslt6qqIeCZPeXNc32wS4zdAT0m1wFXAP0XEp8nQX2WeWGdFxL9sKS4rW+XYJ1q/39Y+11vtO8B55I6oDo2IryU1kr/v/DQi7t9KHGXL56DS9RxQKemSVmU7tnr+NHCJpAoASQdI2mkr9b0J1EraL1lv3RGeBi5vNS4/opAAI+KYiKjLs2ytI27NzuQ6/98l7QX8c55tXgRGSdo/iXVHSQd08P2stJRzn9jWz/Uu5Ib7vpZ0HDAwzzZPA/+91bmt/pL2bMd7lDQnqBRFbtB4PPBtSe9Lehl4iNzYNMCvgDeABZIWAfezlaPYiFhLbvji98kJ4aWtXr4RqAAWJnXdmHZ7ChERr5EbflgMPAi8kGebleTG7qdKWkiuYw/pwjCtSMq5T6TwuX4YqJc0j9zR1Jt53uOPwCPAX5Jh9mnkP9orS80n5MzMzDLFR1BmZpZJTlBmZpZJTlBmZpZJTlBmZpZJXZqgxo0bF+R+w+DFS3dYOsT9xEs3XPLq0gS1atWqrnw7s5LkfmKW4yE+MzPLJCcoMzPLJCcoMzPLJE8Wa2Zl7+uvv6apqYm1a9cWO5RurbKykpqaGioqKgra3gnKzMpeU1MTffr0YdCgQSRzyVoXiwhWr15NU1MTtbW1Be3jIT4zK3tr165l9913d3IqIknsvvvu7TqKdYLqYgOrq5GUyjKwurrYzTErGU5Oxdfev4GH+LrYB8uX07R3TSp11XzclEo9ZmZZ5CMoM+t20hzJKHQ0o0ePHtTV1XHwwQdz9tln8+WXX7a8Nn36dCTx5pv/eUuoxsZGDj74YADmzJnDLrvswogRIzjwwAM59thjmTlz5kb1T548mSFDhjBkyBAOO+ww5s6d2/La6NGjOfDAA6mrq6Ouro5p06ZtFFPz0tjYuC3/rKnzEZSZdTtpjmRAYaMZvXr1oqGhAYDzzjuPX/ziF/zwhz8EYOrUqRx99NE8+uij/PjHP867/zHHHNOSlBoaGhg/fjy9evXi+OOPZ+bMmdx///3MnTuXPfbYgwULFjB+/Hhefvll+vXrB8DDDz9MfX39FmPKIh9BmZl1sWOOOYZ3330XgDVr1vDCCy/wwAMP8Oijjxa0f11dHTfccAN33303ALfccgu33XYbe+yxBwAjR45kwoQJ3HPPPZ3TgC7iBGVm1oXWr1/PH/7wBw455BAAZsyYwbhx4zjggAPo27cvCxYsKKiekSNHtgwJLl68mEMPPXSj1+vr61m8eHHL+nnnndcylLd69WoAvvrqq5ayM844I43mpcpDfGZmXaA5GUDuCOqiiy4CcsN7kyZNAuDcc89l6tSpjBw5ss36IrY4CXjL662vmivFIb6CEpSkRuALYAOwPiLqJfUFHgMGAY3AORHxaeeEaWZW2vIlg9WrV/Pcc8+xaNEiJLFhwwYkceutt7ZZ36uvvsrQoUMBGDZsGPPnz2fMmDEtry9YsIBhw4al24gu1p4hvuMioi4imlPwtcCzETEYeDZZNzOzAk2bNo0LLriApUuX0tjYyIcffkhtbe1GV+Dls3DhQm688UYuvfRSAK6++mquueaalqG7hoYGpkyZwve///1Ob0Nn2pYhvtOB0cnzh4A5wDXbGI+ZWacb0K9fqr8jHJBcKddeU6dO5dprN/5uf+aZZ/LII49wzTUb/3f6/PPPM2LECL788kv23HNPfv7zn3P88ccDcNppp/HRRx9x1FFHIYk+ffrwm9/8huoS/zG/2hrHBJD0PvApuTsf3h8RkyV9FhG7ttrm04jYLc++E4GJAAMGDDh06dKlqQVfiiSl+kPdQv5+VjQF/2ze/aRzLVmypGU4zIprC3+LvH2l0CG+URExEvhn4FJJxxYaTERMjoj6iKivqqoqdDezbsX9xGxzBSWoiPg4eVwBTAcOAz6RVA2QPK7orCDNzKz7aTNBSdpJUp/m58BYYBHwJDAh2WwC8ERnBWlmZt1PIRdJ7AVMT66n7wk8EhFPSXoFeFzSRcAHwNmdF6aZmXU3bSaoiHgP+Fae8tXA8Z0RlJmZmac6MjOzTHKCMrNuZ++aAanebmPvmgFtvufy5cs599xz2W+//Rg2bBgnnXQSb7/9NosXL2bMmDEccMABDB48mBtvvLHl5yNTpkzhsssu26yuQYMGsWrVqo3KpkyZQlVV1Ua3z3jjjTcAePvttznppJPYf//9GTp0KOeccw6PPfZYy3a9e/duuR3HBRdcwJw5czjllFNa6p4xYwbDhw9nyJAhHHLIIcyYMaPltQsvvJD+/fuzbt06AFatWsWgQYPa/TfJx3PxFWBgdTUfLF9e7DDMLCXLPvqQw294KrX6XvrXcVt9PSI444wzmDBhQsuM5Q0NDXzyySdceOGF3HfffYwdO5Yvv/ySM888k3vvvbdlloj2+M53vtMyw3mztWvXcvLJJ3PHHXdw6qmnAjB79myqqqpapl4aPXo0t99+e8tcfXPmzGnZ/7XXXuOqq65i1qxZ1NbW8v7773PiiSey7777Mnz4cCB3X6kHH3yQSy65pN0xb40TVAF8F1wz2xazZ8+moqKCiy++uKWsrq6OBx54gFGjRjF27FgAdtxxR+6++25Gjx7doQSVzyOPPMKRRx7ZkpwAjjvuuIL3v/3227n++uupra0FoLa2luuuu47bbruNX//61wBMmjSJO++8k+9973upxNzMQ3xmZp1s0aJFm90OA/LfJmO//fZjzZo1fP755+1+n9bDdnV1dXz11VdbfO9CFXIrjwEDBnD00Ue3JKy0+AjKzKxINr0lRmtbKt+afEN82ypfjPnKrr/+ek477TROPvnk1N7bR1BmZp3soIMOYv78+XnL582bt1HZe++9R+/evenTp0+nvnd79t80xny38th///2pq6vj8ccf7/B7bcoJysysk40ZM4Z169bxy1/+sqXslVdeYfDgwcydO5dnnnkGyN3U8IorruDqq69O7b2/+93v8uc//5nf//73LWVPPfUUr7/+ekH7X3XVVfz0pz+lsbERgMbGRm666SauvPLKzbb90Y9+xO23355K3OAhPjPrhqr779PmlXftrW9rJDF9+nQmTZrEzTffTGVlJYMGDeKuu+7iiSee4PLLL+fSSy9lw4YNnH/++RtdWj5lypSNLut+8cUXARg+fDjbbZc7xjjnnHMYPnw4jz322Eb3krr33ns56qijmDlzJpMmTWLSpElUVFQwfPhwfvaznxXUtrq6Om655RZOPfVUvv76ayoqKrj11ltb7g7c2kEHHcTIkSMLvm19Wwq63UZa6uvrY9NDxVKQ9i0yfLuNbqP9JxEo3X6SZb7dRnZ0xu02zMzMupQTlJmZZZITlJl1Cx4OL772/g2coMys7FVWVrJ69WonqSKKCFavXk1lZWXB+/gqPjMrezU1NTQ1NbFy5cpih9KtVVZWUlNT+EViTlAlbAc69mvzfAb068fSZctSqcssayoqKlrmkrPS4QRVwtaBJ7E1s7JV8DkoST0kvSppZrJeK+klSe9IekzS9p0XppmZdTftuUjiB8CSVuu3AHdGxGDgU+CiNAMzM7PuraAEJakGOBn4VbIuYAwwLdnkIWB8ZwRoZmbdU6FHUHcBVwPfJOu7A59FxPpkvQnon29HSRMlzZM0z1fQmOXnfmK2uTYTlKRTgBUR0Xq+9nyXjuX9gUFETI6I+oior6qq6mCYZuXN/cRsc4VcxTcKOE3SSUAlsDO5I6pdJfVMjqJqgI87L0wzM+tu2jyCiojrIqImIgYB5wLPRcR5wGzgrGSzCcATnRalmZl1O9sy1dE1wA8lvUvunNQD6YRkZmbWzh/qRsQcYE7y/D3gsPRDMjMz82SxZmaWUU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSW0mKEmVkl6W9JqkxZJ+kpTXSnpJ0juSHpO0feeHa2Zm3UUhR1DrgDER8S2gDhgn6QjgFuDOiBgMfApc1HlhmplZd9NmgoqcNclqRbIEMAaYlpQ/BIzvlAjNzKxbKugclKQekhqAFcAs4K/AZxGxPtmkCei/hX0nSponad7KlSvTiNms7LifmG2uoAQVERsiog6oAQ4DhubbbAv7To6I+oior6qq6nikZmXM/cRsc+26ii8iPgPmAEcAu0rqmbxUA3ycbmhmZtadFXIVX5WkXZPnvYATgCXAbOCsZLMJwBOdFaSZmXU/PdvehGrgIUk9yCW0xyNipqQ3gEcl/RvwKvBAJ8ZpZmbdTJsJKiIWAiPylL9H7nyUmZlZ6jyThJmZZZITlJmZZZITlJmZZZITlJmZZVLZJqiB1dVISmUxM7OuV8hl5iXpg+XLadq7JpW6aj5uSqUeMzMrXNkeQZmZWWlzgjIzs0xygjIzs0xygjIzs0xygjIzs0xygjIzs0xygjIzs0xygjIzs0xygjIzs0xygjIzs0xygjIzs0xqM0FJ2kfSbElLJC2W9IOkvK+kWZLeSR536/xwzcysuyjkCGo9cGVEDAWOAC6VNAy4Fng2IgYDzybrZmZmqWgzQUXEsohYkDz/AlgC9AdOBx5KNnsIGN9ZQZqZWffTrnNQkgYBI4CXgL0iYhnkkhiw5xb2mShpnqR5K1eu3LZozcqU+4nZ5gpOUJJ6A78DJkXE54XuFxGTI6I+Iuqrqqo6EqNZ2XM/MdtcQQlKUgW55PRwRPx7UvyJpOrk9WpgReeEaGZm3VEhV/EJeABYEhF3tHrpSWBC8nwC8ET64VlX2QG2etv79iwDq6uL3RwzKwOF3PJ9FHA+8LqkhqTseuBm4HFJFwEfAGd3TojWFdYBTXvXpFJXzcdNqdRjZt1bmwkqIuYC2sLLx6cbTjapR0Vq/+mq5/bp1dWjIpV6zMyyqJAjqG4vNnzN4Tc8lUpdL/3ruFTrMjMrV57qyMzMMskJyszMMskJyszMMskJyszMMskJyszMMskJyszMMskJyszMMskJyszMMskJyszMMqlsZ5JIc3oiMzPremWboNKensjMzLqWh/jMzCyTnKDMzCyTnKDMzCyTyvYcVHeQ6n2qfG8py5iB1dV8sHx5KnX12q4HX32zIZW6BvTrx9Jly1Kpy7bOCaqE+UIQK2cfLF+e6l2efcfo0tPmEJ+kByWtkLSoVVlfSbMkvZM87ta5YZqZWXdTyDmoKcCmX6+vBZ6NiMHAs8m6GQA7AJJSWQZWVxe7OWZWJG0O8UXEnyQN2qT4dGB08vwhYA5wTYpxWQlbBx5OMbNt1tGr+PaKiGUAyeOeW9pQ0kRJ8yTNW7lyZQffzqy8lUs/GVhdndrRs1mnXyQREZOByQD19fXR2e9nVorKpZ+kfWGDdW8dPYL6RFI1QPK4Ir2QzMzMOp6gngQmJM8nAE+kE46ZmVlOIZeZTwX+AhwoqUnSRcDNwImS3gFOTNbNzMxSU8hVfP+yhZeOTzkWMzOzFpmai89XAJmZWbNMTXXkK4DMzKxZphKUFY8nnjWzrHGCMsATz5pZ9mTqHJSZmVkzJygzM8skJygzM8skJygzM8skJyjLNN9bqvP594eWVb6KzzLN95bqfP79oWWVE5Slzr+pMrM0OEFZ6vybKjNLg89BmZlZJvkIyjItzeHC7XpUpHoif0C/fixdtiy1+spBqsO7Pbf3UHE7DKyu5oPly1OpKyufbScoy7S0hwvTuhgAfEFAPmn/vTxUXLhyvNjFQ3xmZpZJmTqCSnN4wMzMSlumEpSv/jIzs2bblKAkjQN+BvQAfhURN6cSlVknSPsIvZgn3tM8IW7tk+bFNtv1rOCb9V+nUlc56nCCktQDuAc4EWgCXpH0ZES8kVZwZmlK8wgdinuUXo4nxEvFNxm9cKcc/47bcpHEYcC7EfFeRPwDeBQ4PZ2wzMysu1NEdGxH6SxgXET8j2T9fODwiLhsk+0mAhOT1QOBtzoe7kb2AFalVFcWlFN7yqkt0PH2rIqIgg6z3E8K5vZkW6p9ZVvOQeUbhN0s20XEZGDyNrxP/jeX5kVEfdr1Fks5taec2gJd0x73k8K4PdmWdnu2ZYivCdin1XoN8PG2hWNmZpazLQnqFWCwpFpJ2wPnAk+mE5aZmXV3HR7ii4j1ki4DniZ3mfmDEbE4tcjalvpwSJGVU3vKqS1Q2u0p5djzcXuyLdX2dPgiCTMzs87kufjMzCyTnKDMzCyTMp+gJO0jabakJZIWS/pBUt5X0ixJ7ySPuxU71kJIqpT0sqTXkvb8JCmvlfRS0p7HkgtPSoakHpJelTQzWS/Z9khqlPS6pAZJ85KyzH/e3Feyz/2kfTKfoID1wJURMRQ4ArhU0jDgWuDZiBgMPJusl4J1wJiI+BZQB4yTdARwC3Bn0p5PgYuKGGNH/ABY0mq91NtzXETUtfpNRyl83txXss/9pD0ioqQW4Aly8/+9BVQnZdXAW8WOrQNt2RFYABxO7tfXPZPyI4Gnix1fO9pRk3wYxwAzyf2Iu5Tb0wjssUlZyX3e3FeytbiftH8phSOoFpIGASOAl4C9ImIZQPK4Z/Eia5/kML8BWAHMAv4KfBYR65NNmoD+xYqvA+4Crga+SdZ3p7TbE8AfJc1PpiCCEvu8ua9kkvtJO2XqflBbI6k38DtgUkR8ntZ098UQERuAOkm7AtOBofk269qoOkbSKcCKiJgvaXRzcZ5NS6I9iVER8bGkPYFZkt4sdkDt4b6SPe4nHVMSCUpSBbkO93BE/HtS/Imk6ohYJqma3DeskhIRn0maQ+58wa6Seibfpkpp2qhRwGmSTgIqgZ3JfVMs1fYQER8njyskTSc3c39JfN7cVzLL/aQDMj/Ep9zXvweAJRFxR6uXngQmJM8nkBtvzzxJVcm3QST1Ak4gd9J0NnBWslnJtCcirouImogYRG66q+ci4jxKtD2SdpLUp/k5MBZYRAl83txXssv9pIOKfaKtgBNxR5M77F0INCTLSeTGb58F3kke+xY71gLbMxx4NWnPIuCGpHxf4GXgXeC3wA7FjrUDbRsNzCzl9iRxv5Ysi4EfJeWZ/7y5r5TG4n5S+OKpjszMLJMyP8RnZmbdkxOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhNUGZA0I5mwcXHzpI2SLpL0tqQ5kn4p6e6kvErS7yS9kiyjihu9WddxXykt/qFuGZDUNyL+lkwH8wrwX4AXgJHAF8BzwGsRcZmkR4B7I2KupAHkpvfPNwGnWdlxXyktJTFZrLXpCklnJM/3Ac4H/m9E/A1A0m+BA5LXTwCGtZrhemdJfSLii64M2KxI3FdKiBNUiUum7j8BODIivkxmfH6L/LclgNyw7pER8VXXRGiWDe4rpcfnoErfLsCnSYcbQu52BDsC35a0m6SewJmttv8jcFnziqS6Lo3WrHjcV0qME1TpewroKWkhcCPwIvARcBO5u6k+A7wB/D3Z/gqgXtJCSW8AF3d9yGZF4b5SYnyRRJmS1Dsi1iTfCqcDD0bE9GLHZZY17ivZ5SOo8vVjSQ3k7qPzPjCjyPGYZZX7Skb5CMrMzDLJR1BmZpZJTlBmZpZJTlBmZpZJTlBmZpZJTlBmZpZJ/x8L0E1h/P2azQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x216 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "bins = np.linspace(df.age.min(), df.age.max(), 10)\n",
    "g = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\n",
    "g.map(plt.hist, 'age', bins=bins, ec=\"k\")\n",
    "\n",
    "g.axes[-1].legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "# Pre-processing:  Feature selection/extraction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Lets look at the day of the week people get the loan "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAZ+ElEQVR4nO3deZxU9bnn8c9XaC8quKO2dIBWEUTlNtjRuGAQEl8EN7wuMTEKGXMdjUsYw7glY3LjjHFhXBJXogZvRFxIxFxyo3GBCO4srYgY4lXEVlBgYoxRFPSZP+p0p4GCru4+1XWo+r5fr3p11alzfuc53fX0U+dXp34/RQRmZmZZs0WpAzAzM8vHBcrMzDLJBcrMzDLJBcrMzDLJBcrMzDLJBcrMzDLJBSplknaVdI+k1yXNlfSMpONTanuYpOlptNUZJM2UVF/qOKy0yiknJPWU9Jyk+ZKGFnE/Hxar7c2JC1SKJAmYBjwZEXtExAHAKUBNieLpWor9mjUpw5wYAbwaEYMjYlYaMdnGuUClazjwaUTc2rQgIt6MiJ8DSOoi6RpJL0h6SdJ/T5YPS842pkp6VdLkJLGRNDJZNhv4l6Z2JW0j6c6krfmSjkuWj5X0gKT/AP7QkYORNEnSLZJmJO9+v5zsc5GkSS3Wu0XSHEkLJf3bRto6MnnnPC+Jr3tHYrPNRtnkhKQ64GpglKQGSVtt7HUtaYmkK5Ln5kgaIukRSf8l6axkne6SHk+2XdAUb579/s8Wv5+8+VW2IsK3lG7A+cB1m3j+TOCHyf1/AuYAtcAw4K/k3lVuATwDHAZ0A94C+gEC7gemJ9tfAXwrub89sBjYBhgLNAI7biSGWUBDnttX8qw7Cbg32fdxwAfA/kmMc4G6ZL0dk59dgJnAoOTxTKAe2Bl4EtgmWX4RcFmp/16+Ff9WhjkxFrgxub/R1zWwBDg7uX8d8BLQA+gJvJcs7wps26Kt1wAljz9Mfh4JTEyOdQtgOnB4qf+unXVzF1ARSbqJXFJ9GhFfJPdiGyTpxGSV7cgl2qfA8xHRmGzXAPQFPgTeiIg/J8vvJpfQJG0dK2l88rgb0Du5/2hE/L98MUVEW/vN/yMiQtIC4N2IWJDEsjCJsQE4WdKZ5BKuGhhILiGbfClZ9lTyJnhLcv9wrMKUSU40ae11/dvk5wKge0T8DfibpNWStgf+Dlwh6XDgc6AXsCuwvEUbRya3+cnj7uR+P0+2M+bNigtUuhYCJzQ9iIhzJO1M7l0h5N4FnRcRj7TcSNIw4JMWiz7jH3+bjQ2WKOCEiPjTem0dRO6Fn38jaRa5d3LrGx8Rj+VZ3hTX5+vF+DnQVVItMB74YkT8Jen665Yn1kcj4hsbi8vKVjnmRMv9bep1vcncAU4ld0Z1QESskbSE/Lnz04i4bRNxlC1/BpWuJ4Buks5usWzrFvcfAc6WVAUgaW9J22yivVeBWkl7Jo9bJsIjwHkt+uUHFxJgRAyNiLo8t00l4qZsSy75/yppV+BredZ5FjhU0l5JrFtL2rud+7PNSznnREdf19uR6+5bI+kIoE+edR4B/luLz7Z6SdqlDfvYrLlApShyncajgS9LekPS88Bd5PqmAW4HXgHmSXoZuI1NnMVGxGpy3Re/Sz4QfrPF05cDVcBLSVuXp308hYiIF8l1PywE7gSeyrPOCnJ991MkvUQusQd0YphWIuWcEym8ricD9ZLmkDubejXPPv4A3AM8k3SzTyX/2V5ZavpAzszMLFN8BmVmZpnkAmVmZpnkAmVmZpnkAmVmZpnUqQVq5MiRQe47DL75Vgm3dnGe+FaBt7w6tUCtXLmyM3dntllynpjluIvPzMwyyQXKzMwyyQXKzMwyyYPFmlnZW7NmDY2NjaxevbrUoVS0bt26UVNTQ1VVVUHru0CZWdlrbGykR48e9O3bl2QsWetkEcGqVatobGyktra2oG3cxWdmZW/16tXstNNOLk4lJImddtqpTWexLlBWMfpUVyMptVuf6upSH5K1gYtT6bX1b+AuPqsYS5cvp3H3mtTaq3mnMbW2zGxDPoMys4pTirPpLl26UFdXx3777cdJJ53ERx991Pzcgw8+iCReffUfU0ItWbKE/fbbD4CZM2ey3XbbMXjwYPr378/hhx/O9OnT12l/4sSJDBgwgAEDBnDggQcye/bs5ueGDRtG//79qauro66ujqlTp64TU9NtyZIlHfm1pq6gMyhJ/wP4DrkhKRYA3waqgXuBHYF5wGkR8WmR4jQzS00pzqa32morGhoaADj11FO59dZbueCCCwCYMmUKhx12GPfeey8//vGP824/dOjQ5qLU0NDA6NGj2WqrrRgxYgTTp0/ntttuY/bs2ey8887MmzeP0aNH8/zzz7PbbrsBMHnyZOrr6zcaUxa1egYlqRdwPlAfEfsBXYBTgKuA6yKiH/AX4IxiBmpmVi6GDh3Ka6+9BsCHH37IU089xR133MG9995b0PZ1dXVcdtll3HjjjQBcddVVXHPNNey8884ADBkyhDFjxnDTTTcV5wA6SaFdfF2BrSR1BbYGlgHDyU0/DLkpnEenH56ZWXlZu3Ytv//979l///0BmDZtGiNHjmTvvfdmxx13ZN68eQW1M2TIkOYuwYULF3LAAQes83x9fT0LFy5sfnzqqac2d+WtWrUKgI8//rh52fHHH5/G4aWq1S6+iHhb0gRgKfAx8AdgLvB+RKxNVmsEeuXbXtKZwJkAvXv3TiNms7LjPCl/TcUAcmdQZ5yR63SaMmUK48aNA+CUU05hypQpDBkypNX2IjY6CHjz8y2vmtscu/haLVCSdgCOA2qB94EHgK/lWTXvbysiJgITAerr6zf9GzWrUM6T8pevGKxatYonnniCl19+GUl89tlnSOLqq69utb358+ezzz77ADBw4EDmzp3L8OHDm5+fN28eAwcOTPcgOlkhXXxfAd6IiBURsQb4DXAIsH3S5QdQA7xTpBjNzMrS1KlTOf3003nzzTdZsmQJb731FrW1tetcgZfPSy+9xOWXX84555wDwIUXXshFF13U3HXX0NDApEmT+O53v1v0YyimQq7iWwp8SdLW5Lr4RgBzgBnAieSu5BsDPFSsIM3M0tR7t91S/R5b7+RKubaaMmUKF1988TrLTjjhBO655x4uuuiidZbPmjWLwYMH89FHH7HLLrvws5/9jBEjRgBw7LHH8vbbb3PIIYcgiR49enD33XdTvZl/mVyt9WMCSPo34OvAWmA+uUvOe/GPy8znA9+KiE821U59fX3MmTOnozGbtYuk1C8tbiV/2jV0gfMkfYsWLWruDrPS2sjfIm+uFPQ9qIj4EfCj9Ra/DhzY5ujMzMwK4JEkzMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzKzi7F7TO9XpNnavaX14quXLl3PKKaew5557MnDgQEaNGsXixYtZuHAhw4cPZ++996Zfv35cfvnlzV9fmDRpEueee+4GbfXt25eVK1eus2zSpEn07NlznekzXnnlFQAWL17MqFGj2Guvvdhnn304+eSTue+++5rX6969e/N0HKeffjozZ87k6KOPbm572rRpDBo0iAEDBrD//vszbdq05ufGjh1Lr169+OST3LeMVq5cSd++fdv8N8nHExaaWcVZ9vZbHHTZw6m199xPRm7y+Yjg+OOPZ8yYMc0jljc0NPDuu+8yduxYbrnlFo488kg++ugjTjjhBG6++ebmUSLa4utf/3rzCOdNVq9ezVFHHcW1117LMcccA8CMGTPo2bNn89BLw4YNY8KECc1j9c2cObN5+xdffJHx48fz6KOPUltbyxtvvMFXv/pV9thjDwYNGgTk5pW68847Ofvss9sc86b4DMrMrMhmzJhBVVUVZ511VvOyuro6Fi9ezKGHHsqRRx4JwNZbb82NN97IlVdemdq+77nnHg4++ODm4gRwxBFHNE+G2JoJEyZw6aWXUltbC0BtbS2XXHIJ11xzTfM648aN47rrrmPt2rUba6ZdXKDMzIrs5Zdf3mA6DMg/Tcaee+7Jhx9+yAcffNDm/bTstqurq+Pjjz/e6L4LVchUHr179+awww7jV7/6Vbv3k4+7+MzMSmT9KTFa2tjyTcnXxddR+WLMt+zSSy/l2GOP5aijjkpt3z6DMjMrsn333Ze5c+fmXb7+uIuvv/463bt3p0ePHkXdd1u2Xz/GfFN57LXXXtTV1XH//fe3e1/rc4EyMyuy4cOH88knn/CLX/yiedkLL7xAv379mD17No899hiQm9Tw/PPP58ILL0xt39/85jd5+umn+d3vfte87OGHH2bBggUFbT9+/Hh++tOfsmTJEgCWLFnCFVdcwfe///0N1v3BD37AhAkTUokb3MVnZhWoutcXWr3yrq3tbYokHnzwQcaNG8eVV15Jt27d6Nu3L9dffz0PPfQQ5513Hueccw6fffYZp5122jqXlk+aNGmdy7qfffZZAAYNGsQWW+TOMU4++WQGDRrEfffdt85cUjfffDOHHHII06dPZ9y4cYwbN46qqioGDRrEDTfcUNCx1dXVcdVVV3HMMcewZs0aqqqquPrqq5tnB25p3333ZciQIQVPW9+agqbbSIunEbBS8nQblcvTbWRHW6bbcBefmZllUqYKVJ/q6tS+2d1nM59J0sys0mXqM6ily5en1gWT5nTOZrb529Ql3dY52vqRUqbOoMzMiqFbt26sWrWqzf8gLT0RwapVq+jWrVvB22TqDMrMrBhqampobGxkxYoVpQ6lonXr1o2amsJ7yVygzKzsVVVVNY8lZ5sPd/GZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUCZmVkmFVSgJG0vaaqkVyUtknSwpB0lPSrpz8nPHYodrJmZVY5Cz6BuAB6OiAHAPwOLgIuBxyOiH/B48tjMzCwVrRYoSdsChwN3AETEpxHxPnAccFey2l3A6GIFaWZmlaeQM6g9gBXALyXNl3S7pG2AXSNiGUDyc5d8G0s6U9IcSXP8LW6z/JwnZhsqpEB1BYYAt0TEYODvtKE7LyImRkR9RNT37NmznWGalTfnidmGCilQjUBjRDyXPJ5KrmC9K6kaIPn5XnFCNDOzStRqgYqI5cBbkvoni0YArwC/BcYky8YADxUlQjMzq0iFDhZ7HjBZ0pbA68C3yRW3+yWdASwFTipOiGbpUJeqVOcJU5eq1Noysw0VVKAiogGoz/PUiHTDMSue+GwNB132cGrtPfeTkam1ZWYb8kgSZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSQUXKEldJM2XND15XCvpOUl/lnSfpC2LF6aZmVWatpxBfQ9Y1OLxVcB1EdEP+AtwRpqBmZlZZSuoQEmqAY4Cbk8eCxgOTE1WuQsYXYwAzcysMhV6BnU9cCHwefJ4J+D9iFibPG4EeuXbUNKZkuZImrNixYoOBWtWrpwnZhtqtUBJOhp4LyLmtlycZ9XIt31ETIyI+oio79mzZzvDNCtvzhOzDXUtYJ1DgWMljQK6AduSO6PaXlLX5CyqBnineGGamVmlafUMKiIuiYiaiOgLnAI8ERGnAjOAE5PVxgAPFS1KMzOrOB35HtRFwAWSXiP3mdQd6YRkZmZWWBdfs4iYCcxM7r8OHJh+SGZmZh5JwszMMsoFyszMMskFyszMMskFyszMMskFyszMMskFyszMMskFyszMMskFyszMMskFyszMMskFyszMMskFyszMMskFyszMMskFyszMMskFyszMMskFqpP1qa5GUiq3PtXVpT4cM7OiadN8UNZxS5cvp3H3mlTaqnmnMZV2zMyyyGdQZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSa0WKElfkDRD0iJJCyV9L1m+o6RHJf05+blD8cM1M7NKUcgZ1Frg+xGxD/Al4BxJA4GLgccjoh/wePLYzMwsFa0WqIhYFhHzkvt/AxYBvYDjgLuS1e4CRhcrSDMzqzxt+gxKUl9gMPAcsGtELINcEQN22cg2Z0qaI2nOihUrOhatWZlynphtqOACJak78GtgXER8UOh2ETExIuojor5nz57tidGs7DlPzDZUUIGSVEWuOE2OiN8ki9+VVJ08Xw28V5wQzcysEhVyFZ+AO4BFEXFti6d+C4xJ7o8BHko/PDMzq1SFTFh4KHAasEBSQ7LsUuBK4H5JZwBLgZOKE6KZmVWiVgtURMwGtJGnR6QbjpmZlVqf6mqWLl+eSlu9d9uNN5cta9e2nvLdzMzWsXT5chp3r0mlrZp3Gtu9rYc6skzrU12NpFRu5STN30uf6upSH45ZXj6DskzLyju5rPHvxSqBz6DMzCyTyvYM6p8gtW6djnzIZx2jLlV+h29Wocq2QH0C7gIpA/HZGg667OFU2nruJyNTacfMOoe7+MzMLJNcoMzMLJNcoMzMLJNcoMzMLJNcoMzMLJNcoMzMLJNcoMzMLJNcoMzMLJNcoMzMLJNcoMzMLJPKdqgjMzNrnzTHwFSXqnZv6wJlZmbryMoYmO7iM6twTSP/e/JDyxqfQZlVOI/8b1nlMygzM8skFyhL3e41vVPrMjKzyuUuPkvdsrffysQHrGa2ectUgcrKpY1mVnp9qqtZunx5Km313m033ly2LJW2rPNkqkBl5dLGzUXT1VdpcAJb1ixdvtwXb1S4DhUoSSOBG4AuwO0RcWUqUVlBfPWVmZWzdl8kIakLcBPwNWAg8A1JA9MKzMwsLVn9rlef6urU4tq6S9eyuzipI2dQBwKvRcTrAJLuBY4DXkkjMDOztGS1tyHtbswsHmNHKCLat6F0IjAyIr6TPD4NOCgizl1vvTOBM5OH/YE/baLZnYGV7Qpo8+FjLB+tHefKiCjow9A25kkh+y4HPsbyUMgx5s2VjpxB5TsP3KDaRcREYGJBDUpzIqK+AzFlno+xfKR5nG3Jk7T3nVU+xvLQkWPsyBd1G4EvtHhcA7zTgfbMzMyadaRAvQD0k1QraUvgFOC36YRlZmaVrt1dfBGxVtK5wCPkLjO/MyIWdjCegrs4NmM+xvJRyuOshN+xj7E8tPsY232RhJmZWTF5sFgzM8skFygzM8ukzBQoSSMl/UnSa5IuLnU8aZP0BUkzJC2StFDS90odU7FI6iJpvqTppY6lGCRtL2mqpFeTv+fBnbjvss4TqJxcKfc8gY7nSiY+g0qGTVoMfJXc5esvAN+IiLIZlUJSNVAdEfMk9QDmAqPL6RibSLoAqAe2jYijSx1P2iTdBcyKiNuTK1i3joj3O2G/ZZ8nUDm5Uu55Ah3PlaycQTUPmxQRnwJNwyaVjYhYFhHzkvt/AxYBvUobVfok1QBHAbeXOpZikLQtcDhwB0BEfNoZxSlR9nkClZEr5Z4nkE6uZKVA9QLeavG4kTJ7QbYkqS8wGHiutJEUxfXAhcDnpQ6kSPYAVgC/TLpnbpe0TSftu6LyBMo6V8o9TyCFXMlKgSpo2KRyIKk78GtgXER8UOp40iTpaOC9iJhb6liKqCswBLglIgYDfwc667OgiskTKN9cqZA8gRRyJSsFqiKGTZJURS7hJkfEb0odTxEcChwraQm57qfhku4ubUipawQaI6LpHf1UcknYWfsu+zyBss+VSsgTSCFXslKgyn7YJOUmWbkDWBQR15Y6nmKIiEsioiYi+pL7Gz4REd8qcVipiojlwFuS+ieLRtB5U8yUfZ5A+edKJeQJpJMrmZjyvUjDJmXNocBpwAJJDcmySyPiP0sYk7XPecDkpEi8Dny7M3ZaIXkCzpVy0qFcycRl5mZmZuvLShefmZnZOlygzMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygMkLSjyWNT7G9AZIakiFG9kyr3Rbtz5RUn3a7Zq1xrlQOF6jyNRp4KCIGR8R/lToYswxzrmSUC1QJSfpBMrfPY0D/ZNm/SnpB0ouSfi1pa0k9JL2RDP+CpG0lLZFUJalO0rOSXpL0oKQdJI0CxgHfSebVuVDS+cm210l6Irk/ommIFUlHSnpG0jxJDyTjoCHpAEl/lDRX0iPJVAgtj2ELSXdJ+t+d9ouziuNcqUwuUCUi6QByw5wMBv4F+GLy1G8i4osR8c/kphk4I5lyYCa54flJtvt1RKwB/h24KCIGAQuAHyXfuL8VuC4ijgCeBIYm29YD3ZMEPgyYJWln4IfAVyJiCDAHuCBZ5+fAiRFxAHAn8H9aHEZXYDKwOCJ+mOKvx6yZc6VyZWKoowo1FHgwIj4CkNQ0ptp+yTus7YHu5Ia1gdy8MRcC08gNF/KvkrYDto+IPybr3AU8kGdfc4EDlJv87RNgHrnkGwqcD3wJGAg8lRsGjS2BZ8i9U90PeDRZ3gVY1qLd24D7I6JlIpqlzblSoVygSivfOFOTyM0e+qKkscAwgIh4SlJfSV8GukTEy0nStb6TiDXKjZz8beBp4CXgCGBPcu889wQejYhvtNxO0v7AwojY2DTNTwNHSPq/EbG6kFjM2sm5UoHcxVc6TwLHS9oqebd2TLK8B7As6TI4db1t/h2YAvwSICL+CvxFUlOXxGnAH8nvSWB88nMWcBbQELnBGJ8FDpW0F0DSl7838Cegp6SDk+VVkvZt0eYdwH8CD0jymx0rFudKhXKBKpFkSuv7gAZy897MSp76X+RmD30UeHW9zSYDO5BLvCZjgGskvQTUAT/ZyC5nAdXAMxHxLrC6aZ8RsQIYC0xJ2nkWGJBMK34icJWkF5NYD1nvOK4l1w3yK0l+PVnqnCuVy6OZb0YknQgcFxGnlToWsyxzrpQHn2puJiT9HPgaMKrUsZhlmXOlfPgMyszMMsn9oGZmlkkuUGZmlkkuUGZmlkkuUGZmlkkuUGZmlkn/H82S0fcuZqCmAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x216 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "df['dayofweek'] = df['effective_date'].dt.dayofweek\n",
    "bins = np.linspace(df.dayofweek.min(), df.dayofweek.max(), 10)\n",
    "g = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\n",
    "g.map(plt.hist, 'dayofweek', bins=bins, ec=\"k\")\n",
    "g.axes[-1].legend()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "We see that people who get the loan at the end of the week dont pay it off, so lets use Feature binarization to set a threshold values less then day 4 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>dayofweek</th>\n",
       "      <th>weekend</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>male</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-09-22</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>female</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           0             0     PAIDOFF       1000     30     2016-09-08   \n",
       "1           2             2     PAIDOFF       1000     30     2016-09-08   \n",
       "2           3             3     PAIDOFF       1000     15     2016-09-08   \n",
       "3           4             4     PAIDOFF       1000     30     2016-09-09   \n",
       "4           6             6     PAIDOFF       1000     30     2016-09-09   \n",
       "\n",
       "    due_date  age             education  Gender  dayofweek  weekend  \n",
       "0 2016-10-07   45  High School or Below    male          3        0  \n",
       "1 2016-10-07   33              Bechalor  female          3        0  \n",
       "2 2016-09-22   27               college    male          3        0  \n",
       "3 2016-10-08   28               college  female          4        1  \n",
       "4 2016-10-08   29               college    male          4        1  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['weekend'] = df['dayofweek'].apply(lambda x: 1 if (x>3)  else 0)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## Convert Categorical features to numerical values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Lets look at gender:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Gender  loan_status\n",
       "female  PAIDOFF        0.865385\n",
       "        COLLECTION     0.134615\n",
       "male    PAIDOFF        0.731293\n",
       "        COLLECTION     0.268707\n",
       "Name: loan_status, dtype: float64"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby(['Gender'])['loan_status'].value_counts(normalize=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "86 % of female pay there loans while only 73 % of males pay there loan\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Lets convert male to 0 and female to 1:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>dayofweek</th>\n",
       "      <th>weekend</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-09-22</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           0             0     PAIDOFF       1000     30     2016-09-08   \n",
       "1           2             2     PAIDOFF       1000     30     2016-09-08   \n",
       "2           3             3     PAIDOFF       1000     15     2016-09-08   \n",
       "3           4             4     PAIDOFF       1000     30     2016-09-09   \n",
       "4           6             6     PAIDOFF       1000     30     2016-09-09   \n",
       "\n",
       "    due_date  age             education  Gender  dayofweek  weekend  \n",
       "0 2016-10-07   45  High School or Below       0          3        0  \n",
       "1 2016-10-07   33              Bechalor       1          3        0  \n",
       "2 2016-09-22   27               college       0          3        0  \n",
       "3 2016-10-08   28               college       1          4        1  \n",
       "4 2016-10-08   29               college       0          4        1  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>dayofweek</th>\n",
       "      <th>weekend</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-09-22</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>5</td>\n",
       "      <td>7</td>\n",
       "      <td>7</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>36</td>\n",
       "      <td>college</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>6</td>\n",
       "      <td>8</td>\n",
       "      <td>8</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>11</td>\n",
       "      <td>13</td>\n",
       "      <td>13</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>900</td>\n",
       "      <td>7</td>\n",
       "      <td>2016-09-10</td>\n",
       "      <td>2016-09-16</td>\n",
       "      <td>26</td>\n",
       "      <td>college</td>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>33</td>\n",
       "      <td>40</td>\n",
       "      <td>40</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-11</td>\n",
       "      <td>2016-09-25</td>\n",
       "      <td>32</td>\n",
       "      <td>college</td>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>37</td>\n",
       "      <td>44</td>\n",
       "      <td>44</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-11</td>\n",
       "      <td>2016-09-25</td>\n",
       "      <td>24</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0            0             0     PAIDOFF       1000     30     2016-09-08   \n",
       "1            2             2     PAIDOFF       1000     30     2016-09-08   \n",
       "2            3             3     PAIDOFF       1000     15     2016-09-08   \n",
       "3            4             4     PAIDOFF       1000     30     2016-09-09   \n",
       "4            6             6     PAIDOFF       1000     30     2016-09-09   \n",
       "5            7             7     PAIDOFF       1000     30     2016-09-09   \n",
       "6            8             8     PAIDOFF       1000     30     2016-09-09   \n",
       "11          13            13     PAIDOFF        900      7     2016-09-10   \n",
       "33          40            40     PAIDOFF       1000     15     2016-09-11   \n",
       "37          44            44     PAIDOFF       1000     15     2016-09-11   \n",
       "\n",
       "     due_date  age             education  Gender  dayofweek  weekend  \n",
       "0  2016-10-07   45  High School or Below       0          3        0  \n",
       "1  2016-10-07   33              Bechalor       1          3        0  \n",
       "2  2016-09-22   27               college       0          3        0  \n",
       "3  2016-10-08   28               college       1          4        1  \n",
       "4  2016-10-08   29               college       0          4        1  \n",
       "5  2016-10-08   36               college       0          4        1  \n",
       "6  2016-10-08   28               college       0          4        1  \n",
       "11 2016-09-16   26               college       1          5        1  \n",
       "33 2016-09-25   32               college       1          6        1  \n",
       "37 2016-09-25   24              Bechalor       1          6        1  "
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby(['Gender']).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## One Hot Encoding  \n",
    "#### How about education?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "education             loan_status\n",
       "Bechalor              PAIDOFF        0.750000\n",
       "                      COLLECTION     0.250000\n",
       "High School or Below  PAIDOFF        0.741722\n",
       "                      COLLECTION     0.258278\n",
       "Master or Above       COLLECTION     0.500000\n",
       "                      PAIDOFF        0.500000\n",
       "college               PAIDOFF        0.765101\n",
       "                      COLLECTION     0.234899\n",
       "Name: loan_status, dtype: float64"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby(['education'])['loan_status'].value_counts(normalize=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Feature befor One Hot Encoding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>education</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>45</td>\n",
       "      <td>0</td>\n",
       "      <td>High School or Below</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>33</td>\n",
       "      <td>1</td>\n",
       "      <td>Bechalor</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>college</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>college</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>college</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Principal  terms  age  Gender             education\n",
       "0       1000     30   45       0  High School or Below\n",
       "1       1000     30   33       1              Bechalor\n",
       "2       1000     15   27       0               college\n",
       "3       1000     30   28       1               college\n",
       "4       1000     30   29       0               college"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[['Principal','terms','age','Gender','education']].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Use one hot encoding technique to conver categorical varables to binary variables and append them to the feature Data Frame "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>weekend</th>\n",
       "      <th>Bechalor</th>\n",
       "      <th>High School or Below</th>\n",
       "      <th>college</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>45</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>33</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n",
       "0       1000     30   45       0        0         0                     1   \n",
       "1       1000     30   33       1        0         1                     0   \n",
       "2       1000     15   27       0        0         0                     0   \n",
       "3       1000     30   28       1        1         0                     0   \n",
       "4       1000     30   29       0        1         0                     0   \n",
       "\n",
       "   college  \n",
       "0        0  \n",
       "1        0  \n",
       "2        1  \n",
       "3        1  \n",
       "4        1  "
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Feature = df[['Principal','terms','age','Gender','weekend']]\n",
    "Feature = pd.concat([Feature,pd.get_dummies(df['education'])], axis=1)\n",
    "Feature.drop(['Master or Above'], axis = 1,inplace=True)\n",
    "Feature.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Feature selection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Lets defind feature sets, X:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>weekend</th>\n",
       "      <th>Bechalor</th>\n",
       "      <th>High School or Below</th>\n",
       "      <th>college</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>45</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>33</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n",
       "0       1000     30   45       0        0         0                     1   \n",
       "1       1000     30   33       1        0         1                     0   \n",
       "2       1000     15   27       0        0         0                     0   \n",
       "3       1000     30   28       1        1         0                     0   \n",
       "4       1000     30   29       0        1         0                     0   \n",
       "\n",
       "   college  \n",
       "0        0  \n",
       "1        0  \n",
       "2        1  \n",
       "3        1  \n",
       "4        1  "
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = Feature\n",
    "X[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "What are our lables?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF'],\n",
       "      dtype=object)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = df['loan_status'].values\n",
    "y[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## Normalize Data "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Data Standardization give data zero mean and unit variance (technically should be done after train test split )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.51578458,  0.92071769,  2.33152555, -0.42056004, -1.20577805,\n",
       "        -0.38170062,  1.13639374, -0.86968108],\n",
       "       [ 0.51578458,  0.92071769,  0.34170148,  2.37778177, -1.20577805,\n",
       "         2.61985426, -0.87997669, -0.86968108],\n",
       "       [ 0.51578458, -0.95911111, -0.65321055, -0.42056004, -1.20577805,\n",
       "        -0.38170062, -0.87997669,  1.14984679],\n",
       "       [ 0.51578458,  0.92071769, -0.48739188,  2.37778177,  0.82934003,\n",
       "        -0.38170062, -0.87997669,  1.14984679],\n",
       "       [ 0.51578458,  0.92071769, -0.3215732 , -0.42056004,  0.82934003,\n",
       "        -0.38170062, -0.87997669,  1.14984679]])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Xn is normalised X \n",
    "\n",
    "Xn = preprocessing.StandardScaler().fit(X).transform(X.astype(float))\n",
    "Xn[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "# Classification "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Using the following algorithm:\n",
    "- K Nearest Neighbor(KNN)\n",
    "- Decision Tree\n",
    "- Support Vector Machine\n",
    "- Logistic Regression\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# K Nearest Neighbor(KNN)\n",
    "Notice: You should find the best k to build the model with the best accuracy.  \n",
    "**warning:** You should not use the __loan_test.csv__ for finding the best k, however, you can split your train_loan.csv into train and test to find the best __k__."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train set: (276, 8) (276,)\n",
      "Test set: (70, 8) (70,)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split( Xn, y, test_size=0.2, random_state=4)\n",
    "print ('Train set:', X_train.shape,  y_train.shape)\n",
    "print ('Test set:', X_test.shape,  y_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Ks is number of K used to find best K value\n",
    "Ks = 14\n",
    "mean_acc = np.zeros((Ks-1))\n",
    "std_acc = np.zeros((Ks-1))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "for n in range(1,Ks):\n",
    "    #Train Model and Predict  \n",
    "    neigh = KNeighborsClassifier(n_neighbors = n).fit(X_train,y_train)\n",
    "    yhat=neigh.predict(X_test)\n",
    "\n",
    "    mean_acc[n-1] = metrics.accuracy_score(y_test, yhat)\n",
    "    std_acc[n-1]=np.std(yhat==y_test)/np.sqrt(yhat.shape[0])\n",
    "    best_k = mean_acc.argmax()+1\n",
    "    \n",
    "def knn_():\n",
    "    return [neigh, best_k]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.05613914, 0.05673323, 0.05399492, 0.05548619, 0.05125253,\n",
       "       0.05399492, 0.04904333, 0.05125253, 0.05125253, 0.05613914,\n",
       "       0.05477226, 0.05315141, 0.05477226])"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "std_acc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0.67142857, 0.65714286, 0.71428571, 0.68571429, 0.75714286,\n",
       "       0.71428571, 0.78571429, 0.75714286, 0.75714286, 0.67142857,\n",
       "       0.7       , 0.72857143, 0.7       ])"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mean_acc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3hUVfrA8e+ZkgqEKkgRULpCAkZFYW0gRaUrBFFp9q67Kru66g9117Z2BJVqoXeRDjYQEdDQO4IUaQmpk+nn90cSjJAyydw7987kfJ4nD2Qyc+87kJl3zrnveY+QUqIoiqIoZmMxOgBFURRFKY5KUIqiKIopqQSlKIqimJJKUIqiKIopqQSlKIqimJLN6AC0Urt2bdmkSROjw1AURVHKadOmTaellHXOvT1iElSTJk3YuHGj0WEoiqIo5SSEOFTc7WqKT1EURTEllaAURVEUU1IJSlEURTGliLkGpSiKojWPx8ORI0dwOp1GhxIRYmJiaNiwIXa7PaD7qwSlKIpSgiNHjlC1alWaNGmCEMLocMKalJK0tDSOHDlC06ZNA3qMmuJTFEUpgdPppFatWio5aUAIQa1atco1GlUJSlEUpRQqOWmnvP+WKkEpiqIopqQSlKIoisnNmzcPIQS7du0yOpSQUglKUUzO6/OT4/IaHYZioGnTptG5c2emT5+u63l8Pp+uxy8vlaAUxeRy3T5yXV4y8zyoHbArn5ycHNauXcuECRPOS1BvvPEGbdu2JTExkVGjRgGwb98+unbtSmJiIh06dGD//v18++233HrrrWcf98gjjzB58mQgv03c6NGj6dy5M7NmzeLTTz/liiuuIDExkQEDBuBwOAA4ceIE/fr1IzExkcTERH788Uf+/e9/895775097nPPPcf777+v2XNXZeaKYmI+v8Tlyf9U6/T48PslCbF2LBZ14T7Unlj6BKnHUzU9ZlK9JN7t8W6p95k/fz49evSgRYsW1KxZk19++YUOHTqwZMkS5s+fz/r164mLiyM9PR2AIUOGMGrUKPr164fT6cTv93P48OFSzxETE8OaNWsASEtL49577wXg+eefZ8KECTz66KM89thjXHfddcybNw+fz0dOTg7169enf//+PP744/j9fqZPn87PP/+swb9MPpWgFMXEHG4vRcdMbp+fdIebGnFRWFWSqhSmTZvGE088AUBKSgrTpk2jQ4cOrFy5kuHDhxMXFwdAzZo1yc7O5ujRo/Tr1w/ITzyBGDRo0Nm/b9u2jeeff56MjAxycnLo3r07AKtXr+azzz4DwGq1kpCQQEJCArVq1eLXX3/lxIkTtG/fnlq1amn23FWCUhSTklKS5zn/moDPL0nLdVEjLgq7Vc3Sh0pZIx09pKWlsXr1arZt24YQAp/PhxCCN954AynleWXbJU0B22w2/H7/2e/PXYsUHx9/9u/Dhg1j/vz5JCYmMnnyZL799ttSY7znnnuYPHkyx48fZ8SIEeV8hqVTv92KYlIOt4+SLjlJCWdy3TiLSWBK5Jg9ezZ33303hw4d4uDBgxw+fJimTZuyZs0aunXrxsSJE89eI0pPT6datWo0bNiQ+fPnA+ByuXA4HDRu3JgdO3bgcrnIzMxk1apVJZ4zOzubCy+8EI/Hw5dffnn29i5dujB27Fggv5giKysLgH79+rF06VI2bNhwdrSlFZWgFMWEpJQ43KUnHwlk5nlwuFWFX6SaNm3a2em6QgMGDGDq1Kn06NGD3r17k5ycTFJSEm+99RYAn3/+Oe+//z7t2rXjmmuu4fjx4zRq1IiBAwfSrl07hgwZQvv27Us858svv8xVV13FTTfdRKtWrc7e/t577/HNN9/Qtm1bLr/8crZv3w5AVFQUN9xwAwMHDsRqtWr6/EWkVAUlJydLtWGhEiny3D6ynJ6A7x8bZaVaTGANOJXA7dy5k9atWxsdhqn5/X46dOjArFmzaN68eZn3L+7fVAixSUqZfO591QhKUUyovKOiPLePDIdblaErIbVjxw6aNWtGly5dAkpO5aWKJBTFZFxeH15/+RONy+snPTe/wk+VoSuh0KZNGw4cOKDb8dUISlFMxuGqeOGD1y9Jy3Xj9fnLvrOimJxKUIpiIh6fH3eQycUvJekONy6vqvBTwptKUIpiIsGMnoqSEjIdHvLKqARUFDNTCUpRTMLnlzg1HPVIIMvpUY1mlbClEpSimESuTuuZcl1eMh2q0awWTmQ5Nf3Syrp16872zyvJuHHjaNu2LUlJSXTu3JkdO3aU+zwHDx5k6tSpJf78+uuvR8vlPipBKYoJ+P0Sp47TcU6vjzMOD/4KVAcq5vDtt98ybNiwYn+2dOlSevToUerj77jjDrZu3UpqairPPPMMTz31VLljKCtBaU0lKEUxAYfHh96pw1PQaNanklTEWbVqFV27di31PtWqVTv799zc3LN9/N5+++2zPfS2bt3KZZddhsPh4LvvviMpKYmkpCTat29PdnY2o0aN4ocffiApKYl33nmHvLw8UlJSaNeuHYMGDSIvL0/T56XWQSmKwfLbGoXmOlFho9nqsVFE2dTn00hw+vRp7HY7CQkJZd53zJgxvP3227jdblavXg3AE088wfXXX8+8efN49dVX+fjjj4mLi+Ott95izJgxdOrUiZycHGJiYnjttdd46623WLRoEZCf3OLi4tiyZQtbtmyhQ4cOmj439RuqKAbL85TcFFYPUkKGQzWaDRdXXXUVSUlJ3HPPPSxcuPDsqGbZsmUALF++nG7dugV0rIcffpj9+/fz+uuv88orrwBgsViYPHkyd911F9dddx2dOnUCoFOnTjz11FO8//77ZGRkYLOdP575/vvvufPOOwFo164d7dq10+Ipn6USlKIYrKymsHoobDSbqyr8TG/9+vWkpqYyfvx4evfuTWpqKqmpqWc7hy9ZsuTs9afhw4eTlJTEzTffXOoxU1JSznY8B9i7dy9VqlTh2LFjZ28bNWoU48ePJy8vj44dO7Jr165ij3Xulh9aUglKUQzk9PgMvSaU4/KWqymtYi5SSrZs2UJSUhIAkyZNIjU1lcWLF59337179579+9dff322d15mZiaPP/4433//PWlpacyePRuA/fv307ZtW5599lmSk5PZtWsXVatWJTs7++xxrr322rNbcmzbto0tW7Zo+vzUNShFMZARo6dz5bn/3Epez0/DkaButcB2qA2VTZs20b59+4D+3z788ENWrlyJ3W6nRo0aTJkyBYAnn3yShx56iBYtWjBhwgRuuOEGrr32Wt59912++eYbrFYrbdq0oWfPnlgsFmw2G4mJiQwbNowHH3yQ4cOH065dO5KSkrjyyis1fX5quw1FMYjb6+eMw210GGfZLILqaiv5vzD7dhuvvPIKzZo1IyUlxehQAlae7TbUCEpRDGK2jQa9fkl6rpvqcXa1lXyYeP75540OQVcqQSkRweH2EhcVPr/OXp8fl9d8Hcf9UnIm1x3S7TrsFgsJcWqzReV84fOKVpQSuL1+sp1eBILYKG23nNZLrgmuPZVEQkgLN3x+H/E+KzaTjtqklOranEbKe0nJnL8RilIOhR27s50ePGGwD5LPL3GpNUh/kWfSf4+YmBjS0tJUH0MNSClJS0sjJibwQhM1glLCWtEO4IVre2qafEdZh9ure1ujcOP0+KlqrgI5ABo2bMiRI0c4deqU0aFEhJiYGBo2bBjw/VWCUsLauR3AfX5JltND9bgogyIqnZTStKMFI/mlxOX1EW0z1xSt3W6nadOmRodRaek6xSeE6CGE2C2E2CeEGFXMzy8SQnwjhPhVCLFFCHFzkZ/9s+Bxu4UQ3fWMUwlPJXUAd3n9pquQK+Rwh7atUThxus0/PauElm4JSghhBcYAPYE2wGAhRJtz7vY8MFNK2R5IAT4qeGybgu8vBXoAHxUcT1HOyiulA3iO04vbZFVy+U1h1eipJC6vT20HovyFniOoK4F9UsoDUko3MB3oc859JFDYAz4BKGwE1QeYLqV0SSl/A/YVHE9RgPw3+9I2+Cu8HmWmNzyX149fDZ9KJEHTHYWV8KdngmoAHC7y/ZGC24p6CbhTCHEEWAw8Wo7HIoS4TwixUQixUV3ErFycHn+ZU2V+KU3VZ041Zi1bnhphKkXomaCKK6M69y1lMDBZStkQuBn4XAhhCfCxSCk/kVImSymT69SpE3TASvgIdHt0l9dvisTg8vrwmmg0Z1Zev8QbBksFlNDQM0EdARoV+b4hf07hFRoJzASQUq4DYoDaAT5WqaTK2wE8x2X89SiHS40MAqWqHJVCeiaoDUBzIURTIUQU+UUPC8+5z+9AFwAhRGvyE9SpgvulCCGihRBNgebAzzrGqoSRikwDZeS5Dbse5fH5catRQcBUglIK6bYOSkrpFUI8AiwDrMBEKeV2IcRoYKOUciHwd+BTIcST5E/hDZP5S7a3CyFmAjsAL/CwlFL91ioVfrOXMr9ookZ86NdHqdFT+UiZP0qOsavC3cpObbehhJVMhyeoSq/4aBtVokO3Pt3nl5zOcYXsfJEi2mYx7WJrRXslbbehevEpYaNoW6OKynV5cYWwlDnQYg7lr9xev6mWCCjGUAlKCRtadYfIzPOEpFt3SZ0ulLKpNVEKqASlhAm/X2q2RqbwepTeHKV0ulDKptZEKSpBKWGhtLZGFeHx+cnWcRFvflsjNb0XDK9fhsX2KYp+VIJSTE+vHnYOtw+nTiXNeR7VFFYLev3/KOFBJSjF9Jwe/XrYZTn1uR6lmsJqIz/Rq0xfWakEpZienlNlUkKGw63pm2B5O12U5d31r/P3FQ/h9Do1O2a4kDK/XZVSOakNCxVTC0UPO69fku3yUi3GrsnxtBw9nXKc5O31/8Htc/N75kEm955JfFQVzY4fDtSi3cpLjaAUUwtVF4Y8ja5Hub1+TS/sf7l1Em6fm793fI4fj3zPwLm3cMaZrtnxw4FaE1V5qQSlmFaoe9hl5XmC7qSt5XSkx+dh8uaPub5xV56++nnG3zqVrSdT6T+rOydzj2t2HrOTqP58lZVKUIpphbqHnQQy8jwVvh7l9fk1vV6yeN8Cjuf+wYikBwHo2aw3X/SZx8GMA/SZ2ZXDWYc0O5fZqWq+wPgjbLsSlaAUU/L5ZUhbEhU9b5azYqOgXI0r9yamjqVxQlO6NOl+9rZrG9/IrAFfk56XRp8ZXdibvlvTc5qVWhNVNq/PT7rDTbrDbchrRw8qQSmm5HB7DevC4PT4yt3FwO+XuDT8lL/t5GbWH/uR4Yn3Y7X8tUAguX5H5t6+DLffQ9+ZN7H1ZKpm5zUzNc1XMrc3Pzn5/DK/U4rDExGjTpWgFNORUhr+ZpTt9JTrE3uuxgl1QupYYm1xDL50aLE/v7ROOxYOXEWMLYb+s7qz/uhaDc9uTk61JqpYTo+vYKnEn7dJ8tt5mWE36WCoBKWYjsNtfBeGwhd4IG+IWifUtLzTzNs1g9vbDCEhpnqJ97u4RjMWDlpF3fh6pMztxeqDyzWLwYzUmqjzOdze/N/TEn6e4/KSpWNLL72pBKWYjlm6MPj8kqy8sj+Bap1Qp26dhNPnZETiA2Xet0HVRswbuIJLarRg6ILbWLhnjnaBmFAkTFtpJdvpITuA66V5bp/mi9FDRSUoxVScHp9ubY0qwun1lVk6rmVC9fq9TN7yCZ0bXU+r2m0CekyduAuYc9tS2tdL5oHFdzN12xTN4jEbl1oThZSSzDxPuX7vXF4/ZxyesPu3UwlKMRUzzpnnOL0lXo/SOqEu27+Io9lHGFlQWh6ohJjqTO+/iOsu6sJTKx7g41/e1ywmszH6+qSRpJRkVLAAwuP7s5AiXKgEpZhGKNoaVYQEMkr49Kl1Qp2QOpaG1S6i28W3lPuxcfY4pvSZza3N+/Hid8/yxo8vh+W0Tlkqa4Ly+SXpue6gFq/7/JK0XFfYlOyrBKWYRqgX5paHX8rzLjZrnVB3nt7Gj0e+Z1i7+84rLQ9UlDWKj2/+nMGXDuXt9f/h39/+A78MjzejQPn8EnclK5bw+vyk57o1+X2TEs7khsdaKdUsVjEFb4jbGlWEy+sn1+UlPjr/ZaN1Qp2YOo5YWyxD2g4P6jhWi5W3bxpLtehqfPzLB2S5s3j7prHYLJHzcnd6fUTZKsfna7fXT0aeW9NCnMJZgWoxEBtl3ka8kfMbq4Q1rbsw6CXH5cVutSAEmibUM850Zu+cSv9WKdSIqRn08YQQvHTt6yRE1+CNdaPJdWfzUc8pRNuiNYjWeE6Pj6rRNoQQRoeiK6fHR1YpZeTBynJ68ElJlWhzpoLK8RFEMfW1CJ/GXRj0lpnnIaeC7ZBKMm3bFPK8eeUujiiNEIKnOv6Tl697k6/3LeDuBQPI9eRqdnwjVYY1UWWtcdJKriv/PGakElQlkeX0km3SBXtGtjWqCL+Umo6efH4fkzZ/TMcGnWlTp61mxy10b4dHeLfbJ/xw+BtS5txKpjND83MYobztqMJJVoBrnLTyZzcKc70SVYKqBDw+P06PD4fbR6aj4t269WCGtkZGW/nbEg5nHdJ09HSulEvv4tNbviT1xCb6z+7OqdwTup0rVNw+f1iVTAdCSkmmw2NI8nV58wsxzLRWSiWoSqDodJTT6yPDREkqz2N8WyOjTUgdS/0qDejZrLeu57mleV8+7zuXA2f20WdmV45k/a7r+UIhkjpL+P2SMw4PTgOr67x+SVqu2zRbdqgEFeGcHt9501HugpJVM3z6zDVxaXko7E7byfe/r2Zo4n0hqbK7vnFXZg5YxOm8U/SZ2YV96Xt0P6eeImX07fNL0h1uU6xP8sv8WMxQyq8SVITLKWEhqbdg0Z+Rn5TM1tbICBNTxxFtjWbIZcGVlpfHFfWvZu5ty3D53PSZ2ZVtJzeH7Nxai4Q1UR4TfWAsJCVkONyGj1BVgopgDre31F96oz8pmbGtUShluTKZtfNL+rYcSO24OiE992UXJLJg4EpibNH0n92dn4/+GNLzaymcR1Eur48zuW5TflAr7OhfVi9KPakEFaH8flni6Kkooz4pub1+U7Y1CqXp2z/D4cnVtTiiNJfUaM6CgauoE3cBg+b24ttDKw2JI1iuMN0nKq+waMnoQMqQbWAFsEpQESrX7Q24+MCIT0pGfiozA7/0M2nzx1xxYUfa1W1vWBwNq13E/NtXcHGNZtw1vz+L9s4zLJaKkoTfmqjcgn2azJ6cChlVAawSVATy+WWFylRD9UnJ6/OH3RuK1lYfXM5vGfsZ2d6Y0VNRdeLrMve2ZSTVvZz7vr6Tads/MzqkcgunNVFZTk9AsxtmY0QFsDn7WyhByXFWfOGrw+3D74dqsfq1kdG6rdH+M3t5d/3rvHTda9SKra3psfUyIXUsdePrcUuzfkaHAhRs1zFgESO/SuHJ5ffz2ZZPsYrQ9GizWWyMvu7NoEaShWuirBbztj4q3McpnD+cFVYAV4+LCsm/tUpQEcbt9Qe9jsLp9eF3SKrH2TVPUn4d2hr976dXmbtrBtnuTCb1mmn6/mz70vfwzcHlPH31v7Fb7UaHc1a8PZ4pvWfz6prn2Z22M2Tn3fDHT3zyywd82HNiUMfJ8/hM21PO75dk5HlMUUYerMIK4BpxdmxWfSfhzPm/qVSYVlMHen1Scnh8ms67n8j5g4V75tAk4WKW7l/EF1sncle7kRqeQXuTNo/DbrFzd1vzxRlti2b09W+G9Jx/X/EQ83bPJNeTS7w9vsLHcZo0QXl9fjLyPKYqIw9WYQVw9dgoXbvKq2tQEcTp8Wn6CU3rtVJSSs2LIz7bOgGv38uX/eZz3UVdeOG7p9mbvlvTc2gpx53NjB1f0KfFbdSJr2t0OKbQv1UKDk8uy/cvCuo4ZlwTFY672AZKSnRvMqtrghJC9BBC7BZC7BNCjCrm5+8IIVILvvYIITKK/MxX5GcL9YwzEkgpdWkuqeVaKa3bGrl9bj7fMoEbm3TnkhrNea/7J8Ta4nh4yXDcPrd2J9LQjB1fkOPOZoQJiiPM4uqGnalfpQFzds0I+lhmWhNVuMYpDCvgAyZ1rkPULUEJIazAGKAn0AYYLIRoU/Q+UsonpZRJUsok4ANgbpEf5xX+TEqpb5OyCOBw69eVQau1Ug6NiyMW7Z3LScdx7mn/EAD1qtTnfzd9xJaTv/Lmupc1PZcW/NLPpNRxtK+XTId6VxgdjmlYhIW+LQfy7aEVpOWdDupYZlkTlecuqHgzOpAwp+cI6kpgn5TygJTSDUwH+pRy/8HANB3jiVh+vyRX53VFwa6Vcnp8mk9zTEgdy8XVm3F9465nb+vZrDd3tR3Jhxv+x9rD32t6vmB9//tq9p3Zw8ikh4wOxXQGtE7B6/eycM+coI4jAafH2Gm+nII1Tkrw9ExQDYDDRb4/UnDbeYQQjYGmwOoiN8cIITYKIX4SQvQt4XH3Fdxn46lTp7SKO+zklGNRbrAqulZK69HTr8c3sumPnxmR9AAW8ddf45eue52LazTj0aUjyHCe0fS8wZjw61hqx11Ar+b9jQ7FdNrUbkvLWm2Yu3N60Mcysn9cltNT6Vt4aUnPBFVc6VdJb6MpwGwpZdHfrIuklMnAHcC7QohLzjuYlJ9IKZOllMl16oS2l5lZeH3+kC9SLO+qcrfXr3l57YTUscTbqzCozV3n/SzeHs9HPSdz0nGCp1c+Yoopn4MZB1j52xLubntPxGy7riUhBANapbDhj584lHkwqGMZsU+UlJIMhzusFgyHAz0T1BGgUZHvGwLHSrhvCudM70kpjxX8eQD4FjCuH4yJGbUivTyryrWu3DvlOMnCPbMZ2GYIVaOrFXufxLodePaaF/lq71xm7vhS0/NXxKTN47BarNxt8hJ4I/VrNRCAeWFWLFG4j1M4L8A1Kz0T1AaguRCiqRAiivwkdF41nhCiJVADWFfkthpCiOiCv9cGOgE7dIw1LLm8PkNfFIHsK6VHW6Mvtk7E7XMzoowmqw9d/iTXNLyWf33zJAczDmgaQ3nkunOYtv0zbm3ej3pV6hsWh9k1qtaYq+pfw5xd04Me9YZqJGOmfZwikW4JSkrpBR4BlgE7gZlSyu1CiNFCiKJVeYOB6fKvv5GtgY1CiM3AN8BrUkqVoM6Ro0NZeXmVtVbKofEnWY/Pw5TNn3B94640r9my1PtaLVY+6DEem8XGQ0uG4fEZc+F69s5pZLkyDetaHk76t0phb/outp/aEtRx/FLi0nlnWjPu4xRpdF0HJaVcLKVsIaW8REr5asFtL0gpFxa5z0tSylHnPO5HKWVbKWViwZ8T9IwzHOW5fabZrqKktVJ+v8Sp8SfZxfsWcDz3jzJHT4UaVG3Em10/5JfjG3hn/WuaxhIIKSUTN4+j3QXtSb6wY8jPH256teiPzWJjzi4tiiX0G9WYeR+nSKI6SYQhKQPb6ymUilsrpXVbI4CJqWNpnNCULk26B/yY3i0GMLDNnbz782sh35hv7eHv2J22gxFJD5q+R6AZ1IytxY1NujN/90x8/uA+3Oi1JsrpCY99nCKBSlBhKFfHRbnBKLpWSo+2RltPprL+2I8MT7wfq6V8nbZfvf5/NKrWmIeXjiDLlalpXKUZnzqGmrG16dvy9pCdM9z1bzWIP3KOse7omqCOo8eaqFyXl8w8lZxCRSWoMOPzSxwmGz2dK9vp5YzDo/narAmpY4m1xTH40qHlfmzV6GqM6TGRY9lH+NfqJ7UNrAS/Zx5i+YHF3NV2BDG2mJCcMxJ0u/gW4u1VmKvBNJ+W1XzZYbqPUzhTCSrM5LgqvtdTKGld1ZSWd5p5u2Zwe+s7SIipXqFjJNfvyFMd/8XsXdM0efMry+TNHyMQDG13r+7niiRx9jhubtaHRXvn4fQ6gzqWx+cPutmxlJJMh0fzxeZK2cpMUEKIR4QQNUIRjFI6j89v6Cp5I03dOgmXzxVwcURJHr/yGa64sCPPrnqc3zMPaRTd+RweB1O3TaZns97Ur9pQt/NEqgGtU8hyZbLqt6VBH8sZxDKH/AW4nqD3WFMqJpARVD1ggxBiZkF3cnWl1yBmKCs3gtfvZfKWT+jc6Hpa1W5T9gNKYbPY+LDnJCSSR5eOCPpCfEnm7ZpBhuuM6rtXQZ0bXU+duLraTPNVcOTjK1hC4VZrnAxTZoKSUj4PNAcmAMOAvUKI/xTXekjRj9Pjq7QvlGX7F3E0+4hm64gaJzThtRvfY/2xH/lgw1uaHLMoKSUTUsfSpnZbOjbopMkxK9vHQpvFRt+Wt7HityVkOjPKfkApKrImyluwxsksSzkqq4CuQRUsoj1e8OUlv/PDbCHEGzrGphRRmS/Ojk/9iIbVLqLbxbdodswBrVLo13Igb657mV+Ob9DsuADrjq5hx+mtjNSotDzaZqF6bFSxzS0jWf9WKbh9br7eNz/oYzndgX+4c3vzNxk0Y6VsZRPINajHhBCbgDeAtUBbKeWDwOXAAJ3jU8jvZVdZV6vvOLWVdUd+YFi7+8pdWl4aIQSv3fge9arU56HFw8h152h27Am/fkT16Br0azVIk+PF2K1E2SxUj6tcSSqp7uVcXL2ZJot2XV4f/gBeQ06PjwxHZG8yGE4CGUHVBvpLKbtLKWdJKT0AUko/cKuu0Sn4/eZblBtKE1PHEWON4Y7Lhml+7ISY6ozpMZHfsw7y/Lf/0OSYR7MPs3T/VwxpO5w4e1zQx7MIQYw9PzFXtiQlhKB/qxR+PPw9f+QcDepYEsrsCelwqzVOZhNIgloMpBd+I4SoKoS4CkBKuVOvwJR8uSHc68lszjjTmbNrGgNaD6ZmbC1dztGxYWceu+Jppm2fwld75pb9gDJM2fwpEsmwxPs0iA5io/46aqxsSWpA6xQkknm7ZgZ9rNLWROW4vGRX0iIkMwskQY0Fis5/5BbcpujM55eVen+ZadumkOfN073J6t87PkdS3ct5euUjHMs+UuHjOL1Ovtg2ie4X30Kjao01iS3Wfv60ZmVKUk2rX0L7esnM1WALjpLWRGXmqU0GzSqQBCWKdhovmNqz6ReSUijHGR6LcvXg8/uYtPljOjboTJs6bXU9l91q56Oek3H5XDy27F78smLVkvN3zyI977RmpeXRNgtWS/FpKMpmISHOXimS1IBWKWw7tZndaXr43bYAACAASURBVMFP2BQdRRVuMlhZ1xaGg0AS1IGCQgl7wdfjgHGb61QSbq+/Ui8OXPHbYg5nHQrZFhUX12jGqzf8jzWHv2XcpvfK/fj80vKPaFmrDZ0aXadJTDHFjJ6KirZZK0WS6tPiNqzCqsmaqMLefP6CNU6RtsngrB1TeXf96xX+kGU2gSSoB4BrgKPk75J7FaDNBLtSospcGAEw4dex1K/SgJ7Nepd9Z40MvnQotzTrw3/XvsjWk6nleuyGY+vYejKVEYkPaFJaXrQ4ojSVIUnVia/LtRfdyNxdM4LuTu4vaGKc7oi8NU7v//wmjy4byWs/vsSjS0catv+ZlgJZqHtSSpkipbxASllXSnmHlPJkKIILtQyH+2wnbiM5Pb5KvUPn7rSd/HD4G4Ym3ofNErrZZCEEb3YdQ63Y2jy0ZBgOjyPgx05IHUu16ARua3OHJrGcWxxRmsqQpPq3SuFw1iE2HFtX9p3LkO2MrGUbUkpe+eF5/rP2Bfq3GsSoa15izq7p3LPojqB7GRotkHVQMUKIh4UQHwkhJhZ+hSK4UPP4JNlOL6dyXOS4vAGtm9CalLLSVxNNTB1HtDWaIZcND/m5a8bW4oMeE9ibvpv/+/6fAT3meM4xvt43n8GXDiXeHq9JHMUVR5Qm0pNUz2a9iLXFMkeDYolI4pd+nl39OB9u/B93t7uXD3tM5ImrnuW/N77LsgOLGDK/LznubKPDrLBApvg+J78fX3fgO6AhEL7POABS5u/7cjrHRWaeJ+huyOXhMOleT6GS5cpk1s4v6dtyILXj6hgSw98uuoEHL3+CKVs+Yfn+r8u8/5Qt4/H5fYxIfECT85dWHFH64yI3SVWJqkr3S27lqz1zImLqSgsen4dHlo7gsy2f8ugV/+D1G9/DIvLf0ocn3s+HPSby05E13D7nZs4408s4mjkFkqCaSSn/DeRKKacAtwD6llWZRP6GZz7Sct1kONzl7udVXn6/JFfjTf7CzfTtn+Hw5IasOKIko655icvqJPLEigc4mXu8xPu5vC4+3zqBrk170rh6U03OHci1p5JE26xUi43MJNW/5SDSnWl8c2iF0aEYLs+bx4ivUpi7awbPdRrNc51fPu/a522tBzOx13R2nNpKv5ndOJHzh0HRVlwgCarw40qGEOIyIAFooltEJuXy+slweEjLceHUaSvpnEq8KBfypysmpo7jigs70q5ue0NjibZF81HPyTjcOTy+7L4Sq6IW7pnDacdJRrbXJqEGWhxRmhh7ZCap65vcRI2YmiHZy8vMctzZDJnXl5W/LeG1G9/j0SufLvG+3S+5lS/6zuP3rIP0mdlV1y1m9BBIgvqkYD+o54GFwA7gdV2jMjGvX5KZ5+F0jptcDa9TeX3+Sr0oF2DVb8s4mHlAszf7YLWo1YqXrnudbw6tYGJq8WvTJ6R+RLMaLbj2ohs1OWd5iiNKE4lJKsoaRa8WA1i2f1FYX1cJRnpeGrfN7sn6o2v5sMeEgDqW/O2iG5g9YDEZzjP0mXkje9J2hSBSbZSaoIQQFiBLSnlGSvm9lPLigmq+j0MUn2n5ZX6PvNM5LrKcnqCrgip7WTnAxNSx1I2vxy3N+hkdyllD293LTU1v5uUfnmPn6W1/+dkvf/xM6olNDE964Ozcf7DKWxxRmsIkFUkGtBpEnjePJfu+MjqUkDuec4y+M29i5+ltTOo1gwGtBwf82A4XXsm8gcvx+f30ndmVzSd+0TFS7ZT6qiroGvFIiGIJS5L8DdFO57jIdHhwV2Dhn8vri7gFg+W1L30P3xxawd3t7sVuNc+bqhCCt7uNpWp0Ag8uHvaXst0JqWOpElWVQW3u1ORcUdaKFUeUJsZuJSGCktQV9a+mYbWLKt0036GM3+g9owtHsw/zZb8FdLuk/FvPtK59GQsGrSQuKp4Bs3uw7sgaHSLVViAf+1YIIf4hhGgkhKhZ+KV7ZGHI6fVxxuEmPbd87VMq6065RU3aPA67xc7dbUcaHcp56sRdwHvdPmFX2nZeXfM8ACdzj7NwzxxS2txFlaiqmpxHq+m9c0VSkrIIC/1bDuK731dxKveE0eGExK7TO+g980ay3VnMum0xnYPoVNK0+iUsHLiKC6vUZ/DcXqz8bamGkWovkAQ1AngY+B7YVPC1Uc+gwp3H5y+4TuUqc+FvntsXcSvayyvblcWMHV/Qp8Vt1Imva3Q4xerStDsjkx7k01/HsPrgcj7fOhGP38PwJG1Ky7UojihNJCWp/q1S8Es/C/bMNjoU3f16fCP9Zt0EwNzbl9Oh3hVBH7N+1YbMu30FLWq1ZtjC25m/e1bQx9RLIJ0kmhbzdXEoggt3Pn/pC3+lrNx7PRWaseMLctzZjDBJcURJnv/bq7Ss1YbHl93HlM2fckOTblxSo7kmx9Zr9FRUpCSpVrXbcGmddppsZGhmaw9/z22ze1I1qhoLBq6ide1LNTt27bg6zL5tCckXXsWDi4fyxVZz9l4IpJPE3cV9hSK4SFHSwt/cEC3KlVIyZuPb/GTCOWe/9DNp8zja10vW5NOhnmJtsYztOZlM1xlOOo5rulZLy+KI0kRKkurfahC/Ht/IgTP7jA5FFysOLGHIvD40qNqIBYNW0qS69mOCatEJTO23kBuadOMfKx/mo43vaH6OYAUyxXdFka+/AS8BoevgGUGKLvw9k+vGEaLR08rflvLyD88xcO4tLN1vruqn7w6tYv+ZvZptUaG3NnXa8r+uY+nfahA3NummyTH1KI4oTYzdSrWY8E5S/VoORCA02SfKbObtmsHwrwbSsnYb5g1czoVVGuh2rjh7HJN7z6R3iwGM/uFfvLb2JcN7kRYlyhuMECIB+FxKaaoklZycLDduDO7S2KlsV8S1GfJLP12+uIo8j4OasbXYfOIX3u32Cbdr1NQ0WHfO70/qiU1sGrmHaFu00eEYIiHWruv1p5LkuX1kOcO3bVD/Wd05nnOMtcO2aNJB3gw+2zKeZ1c9RseGnfms92yqRlcLyXl9fh/PrHqUL7dNYnji/bx6w9sBLZ0QAi6oGhP0+YUQm6SUyefeXpHFGw5Am4l3RXdzd81g5+ltjOr0IrMGLObqhn/j0WUjmZg6zujQOJhxgFW/LeXutvdU2uSkd3FEaWKjwnskNaBVCgcy9pF6YpPRoWjiww3/45lVj9KlaQ+m9lsQsuQEYLVYeavrGB64/HEmbf6Yx5beg9dv/PXxQK5BfSWEWFjwtQjYDSzQPzQlWG6fm9d//D/aXpBE7xa3ER9VhS/6zqPHJbfyr2+e5L2f3zB0OD9p8zisFit3tzNfaXmoxNi1WeBbUeGcpG5t3o8oa1TYr4mSUvKfNS/wyprn6dvydib1mkGsLTbkcQghePFv/2XUNS8xe9c07lk02PDtOgJ5dbwF/K/g67/AtVLKUbpGpWjisy3jOZx1iH91Gn12uB5ji+HTW6ZyW6vB/Hfti7z8w3OGJKlcdw5Tt03h1ub9qFelfsjPbxZxUaHb76oksVFWqsYYH0d5JcRUp2vTHszfPdsUn/Yrwi/9jFr9BO9veJO72o5kTI9Jhi5UF0LwxFXP8uoNb7N0/yLunN+PXHeOYfEEkqB+B9ZLKb+TUq4F0oQQTXSNSglajjubd9a/RqdG13F9465/+Zndauf9HuMZnng/H216h2dWPYrPH9o+gLN2TiXbnWV413Ijhbo4ojRxUbawTFL9W6VwynGCNYe/NTqUcvP4PDy6dCRTtnzCQ5c/yRtdPsBqMWa691wjkx7k/e7jWXfkB0O36wgkQc0Civbh8RXcppjYx798QFreKZ7rNLrYC8gWYeE/N7zD41c+w+dbJ/Dw0uEh22dHSsnE1HG0u6A9yRd2DMk5zSgUa5/KIxyTVNemPakWncCcneE1zef0Orln0R3M2TWdf3Uazb//9qrpCj0GthnC+Funsu3UZvrP6l7qtjN6CSRB2aSU7sJvCv4epV9ISrBOO04xdtO73NKsDx0uvLLE+wkh+Gen/+P5zq8wf/cshn+V34hTb2sOf8ue9J2MSHpQsxeluV7aZRMif2NCsylMUnarJWRfwYwiY2wx3NKsL4v3LcDhcWj4L6GfHHc2Q+b3ZdmBRfz3xnd57MqnTZecCvVs1psv+szjUGZ+L8BQb9cRyCvklBDibEm5EKIPcFq/kJRgvf/zGzg8uTx7zUsB3f+RK/7Om10+ZNVvSxk8tzfZrixd45uQ+hE1Y2vTt+Xtmh0z3HaSjbVbTfumFBdlo2Z8VMi+asQF93l3QKsUcj05LD9Q9u7HRjvjTOf2OTfz05E1fNhjIsMT7zc6pDJd2/hGZvZfxBlnesi36wgkQT0A/EsI8bsQ4nfgWcD8/6qV1OGsQ0ze8gmD2txFi1qtAn7cXe1GMvbmyWz84ycGzO5JWp4+n0EOZR5k+YHF3NV2BDG24NdPQP5IJNpmJT46fKanzFAcYRZWi8Burfho8uqGf6Ne/IWmX7R7IucP+s3sxvZTW5jQaxq3lWO7DKMl1+/I3NuX4fF76TvrJrac+DUk5w2kF99+KWVHoA1wqZTyGillQP1FhBA9hBC7hRD7hBDnVf4JId4RQqQWfO0RQmQU+dlQIcTegq+h5XlSldmb615BIPjH1c+V+7F9Ww5kUq+Z7EnbQd+ZN/FHzlHN45u8+WMEgqHt7tXsmIXriOKjbUG90YWKmYojzCKYcnurxUrfVgNZfXAZ6XlpGkalnd8zD+XvaJt1kC/7zqfHJb2MDqncLq3TjoUDVxFri2XA7B4haZ0WyDqo/wghqkspc6SU2UKIGkKIVwJ4nBUYA/QkP7kNFkK0KXofKeWTUsokKWUS8AEwt+CxNYEXgauAK4EXC3b1VUqx8/R2Zu34kuFJD9CgaqMKHeOmi3sytf9C/sg5Su8ZXTiYcUCz+BweB9O2TaFns97Ur9pQk2Oeu9C1WozN9FN9ZiuOMINoW3D/JgNapeD1e/lqz1yNItLOnrRd9Jl5IxnOM8wesJi/XXSD0SFV2MU1mrFw0CrqxtcjZW4vVv22TNfzBfKxpaeU8uzIRkp5Brg5gMddCeyTUh4oKKyYDvQp5f6DgWkFf+8OrJBSphecbwXQI4BzVmqv//gSVaKq8tgVTwd1nGsa/o3ZA5aQ686h94wu7Dy9XZP45u6aTobrjKZ99859s7dZLaae6jNrcYTRgp3mu6xOIs1rtjLdot3NJ36h78yu+Px+5g1cXmrRUrhoULUR8wauoFnNlgxdcDuztutX1B3Ib4RVCHG2D40QIhYIpC9NA+Bwke+PFNx2HiFEY6ApsLo8jxVC3CeE2CiE2Hjq1KkAQopcG46tY+n+RTyc/BQ1Y2sFfbykepczb+AKLELQb+ZN/HJ8Q1DHk1IyIXUsbWq3pWODTkHHV6i4LuBmnuozc3GE0YKZ5hNCMKBVCuuP/cjhrNBWmpVk3ZE1DJjdg7ioeBYMWknr2pcZHZJm6sRdwJzbltK+XjLL9y/X7TyB/EZ8AawSQowUQowgfzTzWQCPK+5VWFLLghRgtpSycLVoQI+VUn4ipUyWUibXqVMngJAik5SSV9f8mzpxdbm3wyOaHbdlrdYsGLSahJjq3D77ZtYc/q7Cx1p3dA07T2/jnvYPafYGHW0r+VqOWaf6QrWtRjgKdpqvX6tBAMzfNVOLcIKy8relDJ7biwur1GfhwFU0rX6J0SFpLiGmOjMGLGLcrfr19QykSOIN4BWgNXAp8LKU8vUAjn0EKHohpCFwrIT7pvDn9F55H1vprTq4jJ+OruWpjv8k3h6v6bEbJzRh4cBVNKx2EUPm9WHZ/kUVOs6EXz+iRkzNs28iWijtWo4Zp/qirBZsJh3ZmUGw03yNE5pwxYUdmWNwNd/83bMYtvB2WtRqzbzbV2h2vdWM4uxxuna/COi3QUq5VEr5Dynl34EcIcSYAB62AWguhGgqhIgiPwktPPdOQoiWQA1gXZGblwHdCgoyagDdCm5TzuGXfv6z5gWaJFzMnZeN0OUcdatcyLzbl9OmTltGfJVS7nn+I1m/s2T/Qu64bJhmTTAtQpT5idtsU32qOKJswTbP7d86hV1p29lxaqtGEZXPF1sn8uDioSRfeBWzb1tC7bjKO7OjhYB+G4QQSUKI14UQB8kfTZW5UktK6QUeIT+x7ARmSim3CyFGF134S35xxHRZpGOplDIdeJn8JLcBGF1wm3KOebtmsOP0Vp695gVdm0zWjK3FrAGLuapBJx5eMoLJmz8J+LFTtnwKwLDE+zSLJy7AN3uzTPWp4ojAxAQ5zde7xQBsFpsh28F/tPEd/rHyYW5o0o2p/RZSLToh5DFEmhI3LBRCtCB/1DMYSANmAP+QUjYOXXiBq4wbFrp9bjpPTqRadALLh/wY0AZjwcrz5nH/13ex/MDXPNdpNI9eWXrFYJ43j8s/bc5VDa5hUm/trg3UqRKNJcC1RLkuLzkh2r24JHFRVqqG6bYWoXYm143b5y/7jiW4c35/dp7eyoaRu0PympBS8sa60byz/jV6txjAhz0mEmWtHN3gjNywcBfQBeglpewspfyA/Eaxikl8vnUCv2cd5F+dR4fkhQgQa4tlwq3T6N9qEK+ufYFXfni+1O065u+eRbozTdPS8mibJeDkBOaY6lPFEYGLDnKab0CrQRzNPsJPR9dqFFHJ/NLPc98+xTvrX2PIZcMZ23NKpUlOoVDab8IA4DjwjRDiUyFEF8KvJ2fEynXn8M7617im4bXc0PimkJ7bbrXzYY+J3N3uXj7c+D+eXf04fnn+J978ruVjaVmrDZ0aXafZ+StyLcfIqT5VHFE+wU7zdbvkVuLs8czVucO51+/lsWX3MjF1HA9c/jhvdR1jmu0yIkWJrxop5Twp5SCgFfAt8CRQVwgxVgjRLUTxKSX4+JcPOO04yXOdi99OQ28WYeH1G9/j0Sv+wWdbPuWRpSPO265jw7F1bD2ZykgNu5ZbLWUXRxTHZrVQxaCtJFRxRPlYLIKoIBJ6vD2emy/pzVd75+LyujSM7E9Or5N7F93B7J1TGXXNS7z4t/+q9W06CKTMPFdK+aWU8lbyy71TAbWjroHS8k7z0aZ3uLlZby6/8CrD4hBC8Fznl3mu02jm7prBiK9S/rJdx/jUj0iIrs4ADZtiBjNVFhcV+qk+VRxRMTFBTon2b51CpiuD1Qe1L/7Ndedw1/z+LNn/Fa/e8DZPXPWsSk46Kdcrp6D10MdSyhv1CkgpW3m309Dbo1c+zes3vs/K35YwZF5fctzZ/JFzlK/3zifl0rs1W5slCP5aTqin+lTniIoJNqlfe9GN1Iqto3k13xlnOgPn3sKPR77n/e7jK/WO0KFgrpWMSpmOZP3OpM0fM7DNnbSs1drocM4amngvVaOr8ujSe7htdk/a10vGL/2MSHxAs3NE26zlKo4oTuFUX7YzNFV9qjiiYgqn+SpazWez2OjTcgBfbp1ElitTk5Lvk7nHGTS3F/vP7GH8rVPp2ax32Q9SgqLmHsLMWz+9mr+dRsfyb6eht/6tUpjUawY7T29j0uaP6dq0J42rN9Xs+DFR2vy6hmqqTxVHBCfYab4BrVJw+Vx8vW9B0LEczsrfLuNQ5m980WeeSk4hol49YWR32k5m7viCYYn307DaRUaHU6xul9zC1H4LaVGzNU9c9Yxmx61ocURJQjHVp4ojghPsNF+HelfSJOHioKv59qbvps+MLqTnpTGz/yKubayucISKSlBh5LW1LxJvr8LjV2r3xq+HTo2u5fuhv2hawKH1VJneVX2qOCJ4wVbzCSHo12oQaw5/y/GcirXy3Hoylb4zb8Lj9zLv9uUk1+9Y4XiU8lOvoDCx8dhPLNn/FQ8lP6nJdhrhRIviiOLoOdWniiO0ocU0n0Qyf3f59yxaf3Qt/Wd1J8YWw4KBK2lTp21QsSjlpxJUGJBS8krBdhr3tdduO41woUVxREn0mupTxRHaiLZZgvr/aVazBe0u6MDccnY4X31wOSlze1E3vh4LB63i4hrNgohCqSiVoMLA6oPL+enoGp68ahTxUVWMDifk9LyWo8dUn10VR2jGEuQWHAADWg9iy8lf2Zu+O6D7L9wzh6ELbqNZzZbMH7iSBlUblf0gRRfqVWRyfunnP2tfoHFCU+5sq892GmZmtQiidL6Wo/VUX6Cd1pXABDvN17fF7ViEJaBtYqZum8IDi++mQ70rmHPbUrVdhsFUgjK5+btnsv3UFp695oVK2YQyVG/2CbF2Tab6VHGE9oKd5qtb5UI6N7qeubtmlNrY+ONf3uepFQ9w3UVdmNb/K7VdhgmoV5KJuX1uXv9xNJfWaUfflgONDifkBME3Dg2U1SI0mepTxRHas2gwiu7fKoVDmb+x6Y/15/1MSskbP77Mi989y63N+zGlz2zi7HFBnU/RhkpQJvbF1okcyvyNf3UK3XYaZqJncURx4qJsQZU1gyqO0Euw03y3NOtDjDXmvNZHfunnhe+e5u31/2HwpUP5+ObPK+VMhVlVvne9MFG4ncbVDf/GjU0qZ/N4Ixa6Vgtiqk8VR+gn2Gm+qtHVuOnim1mwZ87Zrvtev5cnlz/Ap7+O4f4Oj/L2TWPDfrsMAVgiaASvXk0m9emvYzjlOMHznV+ulFNGoSiOKOm8FZ3qU8UR+hEi+N+HAa1TSM87zXe/r8LldXH/13cyY8fnPHP1C7x07eth/zoTQPW4KGrGR0VMklLNYk0oPS+NMRvfpuclvQzdTsNIRr7Zx0XZcHn85WpUqooj9Bdjt+LyVnwr+BubdKd6dA2mbpvE+F/H8O2hlbxy/Vvc0/5hDaM0RmFyKkziNeOjSM914y+lKCQcqFdUgcV7FzMx9eO/7GdklPc3vEmuJ8c022mEWiiLI0pS3qm+GFUcobtgp/mirFH0atGfxfsW8v3vq3mv26cRmZwgfyYgEkZSKkEVmLNjDqNWP84VE1rx/s9vkunMMCSOI1m/Myl1HLe3HkKr2m0MicFo0fbQFkcUp7xTfXGqOEJ3Wkzz3d3uHhpVa8ynt3zJoEvv1Cgy4xSXnAoVJimrwa+lYIjS1gWEk+TkZLlx48YKP15KycKdK3lvw1t8c3A5VaKqMrTdvdzX/hHqVrlQw0hL9+TyB5izaxo/Dttq2o7leqtRwgvOCGdy3WVO9dmtFmrGq8qvUHB6fGTmeYwOwxSEgOqxZb9WfH7JGYcbn1/793oh4IKqMRocR2ySUiafe7s53gVMQAjBNY2uZVq/Bawc8hNdm/Zg7KZ3uWJiK55e+Qi/ZezXPYbdaTuZseNzhpt4Ow292QwqjihJIFN9qrQ8dIKd5osUQgT+Qc5qEdSIC8+RlHneCUzksgsSGXfzZ/w4bCsplw5l5o4v6DS5Hfd9fSdbTvyq23lf//El4uzxPGby7TT0ZLY9lMqa6hMCYuzqZRQqWkzzhbvC5FSe9lzhmqQq9/90GZpUv5g3urzPhpG7eDj5Kb45uIJuU68hZW4v1hz+rtS2KeW16Y/1LN63kIcuf5JasbU1O2440WtbjWCVtoBXFUeEXrCLdsNZRZJTIatFUDPMkpRKUAG4IL4ez3V+mU337OG5TqPZfmort83uwc3Tr2XJvoX4ZcVLX+HP7TRqx13A/R0e1Sjq8BNt4jf7kqb6VHFE6FXWab5gklMhS5glKZWgyqFadAKPXvk0P4/cyes3vk96XhrDvxrEdZ91YPr2z3H73BU67jeHVrDuyA+VdjuNQmZe6FrcVJ/qHGEMIQTRBi9DCDUtklOhcEpS6tVVAbG2WIYm3svaYVsYd/MUoqzRPLH8PjpObMMnv3xArjsn4GP5pZ//rHmBi6o14a62I3WM2txsGuz7o7dzp/rMOB1ZWURXout+WianQuGSpCrP/7IObBYbfVsOZOWQn5jabwGNq1/MC989Q/KElry57hXS8k6XeYyFe2az7dTmSrudRiGzFUeUpHCqTxVHGKuyTPMJATU1Tk6FCpOUzcRJSq2DKuJUtivo1iAbj/3Ehxv/x9L9i4i1xXFn2+Hc3+GxYsvG3T43105pT5w9npV3/lQpO5ZDfnFEnarRpr3+dC6H24vXL6kWYzc6lEot0+HB6fUZHYZuCpOT3tPI/oJ1Ut4KrJNS66DCTHL9jkzuPYvv7v6FXi36M2nzx3ScdCmPL7uPPWm7/nLfqdsmczDzAP/qrP92GjaLwKzv/2YujihOXJSNKlGqjaXRInmaL1TJCfJHUjVMOpKK3P9hg7Ws1Zr3u3/KT8O3MzzxfhbumcO1n7Vn+MKBbPpjPbmeXN7+6b90bNCZLk266x5P1Rg7NeKiTJmkzFwcURKjWzEpkTvNF8rkVMisSUpN8RWhxRRfSdLyTjMxdRwTfv2IDNcZGlVrzOGsQywa9A3J9Tvqcs5C0TYL1ePyr295fH7OONyY5b/dZhHUqhJtdBhKmIq0aT6LENSIsxtWHVre6T41xRchasXW5umrn2fTPXv4v+texy999Gs5UPfkBBAf/ed0lN1qMdVIKk5NlSlBiKRpPqOTE5hvJKVGUEXoOYIqjpRS92svRUdPRZlhJBVuxRGK+UgpOZXtItzfxcyQnIry+yUZeR48ZTRKViOoCBaKN+aio6ei7FYLNQ0eScVEhVdxhGI+kbBo12zJCQpHUnbD1yaa519E0VyMzVrqL5jN4CSlFroqWgjnaT6LyN+zyUzJqZAQxicpXc8shOghhNgthNgnhBhVwn0GCiF2CCG2CyGmFrndJ4RILfhaqGeckSo+uuwEUJikQr3zpt1qMfzTmRIZom0W01xTLY/C5GTmbg5GJyndrlALIazAGOAm4AiwQQixUEq5o8h9mgP/BDpJKc8IIS4ocog8KWWSXvFFuhibNeBPZTarhRpxds44PCG7BqdGT4pWhBBEW61hVc0XDsmpUGGSOuMo+5qU1vRMi1cC+6SUB6SUbmA60Oec+9wLjJFSngGQUp7UMZ5KJZDRU1GFSSoUIynVZJuk4gAAEARJREFUJkjRWjhN84VTcipk1EhKz7M1AA4X+f5IwW1FtQBaCCHWCiF+EkL0KPKzGCHExoLb+xZ3AiHEfQX32Xjq1Cltow9j5Rk9FRWqJKX2UFK0Fi7TfOGYnAoZkaT0PFNx/wPnzh/ZgObA9cBgYLwQonrBzy4qKDu8A3hXCHHJeQeT8hMpZbKUMrlOnTraRR7myjt6KspmtVAzXt9rUmoPJUVr4VDNZ7WEb3IqVJikStrAU2t6nuUI0KjI9w2BY8XcZ4GU0iOl/A3YTX7CQkp5rODPA8C3QHsdY40YMfaKjZ6KKnwh6ZGk1B5Kil6iTbwVfLhuuV4cIQTVQ5Sk9DzDBqC5EKKpECIKSAHOrcabD9wAIISoTf6U3wEhRA0hRHSR2zsBO1BKJYAqJax7Ki+9Pu2FY989JTyYdZovkpJTocIkFW3V9/WsW4KSUnqBR4BlwE5gppRyuxBitBCid8HdlgFpQogdwDfA01LKNKA1sFEIsbng9teKVv8pxYu2WzV9EWj9whLC3J9ylfBmxmm+SExOhYQQJMTpu+WManVURKhbHWlJALWqROvyQvAVNJD0VWC/mKJio6xqDyVFVy6vjwyHx+gwgIJZiLgo1fk+AKrVUYSLidJ29FSUVp8CVXGEordom9UU03w2lZw0oRJUBBBAvM5dwQs/DVY0SaniCCVUjJ7msxV8oFPJKXjqHSMC6Dl6KsoSRJJSxRFKqBi5CFwlJ22pBBXmBIR0+/HCJFWe/WJUcYQSSkZN86nkpD31rhHmYqOsIX9BlHdTs1jVOUIJsVBP8xVuBKqSk7ZUggpjobj2VJLyJCnVGFYJtVBO89kLW4Sp5KQ5laDCmBGjp6ICSVJRqjhCMUCopvkKk5OaIdCHeucIU0aOnooqTFIlNZCMVcURikFidB65R6nkpDuVoMJUXLTNNFMKJW0PrYojFCPp+bsXZbVQXSUn3al3jzAkhPkWvRbXil8VRyhG0muaTyWn0FEJKgzFRZln9FTUuUlKFUcoRtN6mi/appJTKKkEFWaEgHgTX9cpTFLx0TZVHKEYLkbDcvNom4WEWJWcQkm9g4SZuCib6V8gQgjNtv1QlGBE2Sya7GumkpMxVIIKI2YfPSmKGUUHuSYqxmalelyUSk4GUAkqjMSHwehJUcwmmGm+GJtV9z2PlJKpBBUmhFANVxWlIio6zaeSk/FUggoTavSkKBVX3mm+GLtKTmagElQYsAihRk+KEoTyTPPF2K0kxKrkZAYqQYWB+Gi14FVRghHoNF9slEpOZqISlMlZhFALXhVFA2V1OI+NslItRiUnM1EJyuTU6ElRtFHaHlEqOZmTSlAmpkZPiqKdkqb54lRyMi2VoEysSrSq3FMULZ07zRcXZaWqSk6mpRKUSVmEUHspKYrGik7zxUfbVHIyOdUwzaRULztF0V7hNF9slFW9xsKAGkEVEWWSzfWsFjV6UhS9VI+zq+QUJszxjmwSCbF2asZHadqivyLUi0dR9HPuzs+Keal3wnPYrRYS4izE+6zkunw4vb6Qnt9qEZpvsqYoihKOVIIqga1oonL7cHpCk6jU6ElRFCWfGuuWwWbN36isdpVoYqOs6Fn0rUZPiqIof1IJKkBWi6BajJ1aOiYqNXpSFEX5k0pQ5VSYqGpXiSZOw0RlU6MnRVGUv1AJqoIsFkFVDRNVvBo9KYqi/IV6VwxSYaKKj7Lh8PhwuL1IWb5jqNGToijK+VSC0ojFIqgSbSPObi13olKjJ0VRlPOpd0aNFSaq+CgrDreP3DISld1qUaMnRVGUYuh6DUoI0UMIsVsIsU8IMaqE+wwUQuwQQmwXQkwtcvtQIcTegq+hesapByEE8dE26lSJpmqMrcTdPOOjVXJSFEUpjm4jKCGEFRgD3AQcATYIIRZKKXcUuU9z4J9AJynlGSHEBQW31wReBJIBCWwqeOwZveLVixCCuCgbsXYreR4fuS4f/oIhld1qKXUTNUVRlMpMzxHUlcA+KeUBKaUbmA70Oec+9wJjChOPlPJkwe3dgRVSyvSCn60AeugYq+4KE1XtKlFUi7FjEUKNnhRFUUqhZ4JqABwu8v2RgtuKagG0EEKsFUL8JIToUY7HIoS4TwixUQix8dSpUxqGrh9R0Oq/TtVoNXpSFEUphZ4JqriLLueWC9iA5sD1wGBgvBCieoCPRUr5iZQyWUqZXKdOnSDDVRRFUcxEzwR1BGhU5PuGwLFi7rNASumRUv4G7CY/YQXyWEVRFCWC6ZmgNgDNhRBNhRBRQAqw8Jz7zAduABBC1CZ/yu8AsAzoJoSoIYSoAXQruE1RFEWpJHSr4pNSeoUQj5CfWKzARCnldiHEaGCjlHIhfyaiHYAPeFpKmQYghHiZ/CQHMFpKma5XrIqiKIr5CFnevjwmlZycLDdu3Gh0GIqiKEo5CSE2SSmTz71dNYtVFEVRTEklKEVRFMWUVIJSFEVRTEklKEVRFMWUVIJSFEVRTCliqviEEKeAQ0bHUU61gdNGB6GjSH5+kfzcILKfXyQ/NwjP59dYSnleO6CISVDhSAixsbjSykgRyc8vkp8bRPbzi+TnBpH1/NQUn6IoimJKKkEpiqIopqQSlLE+MToAnUXy84vk5waR/fwi+blBBD0/dQ1KURRFMSU1glIURVFMSSUoRVEUxZRUgjKAEKKREOIbIcROIcR2IcTjRsekNSGEVQjxqxBikdGxaE0IUV0IMVsIsavg//Bqo2PSihDiyYLfyW1CiGlCiBijYwqGEGKiEOKkEGJbkdtqCiFWCCH2FvxZw8gYg1HC83uz4HdzixBiXsEu5WFJJShjeIG/SylbAx2Bh4UQbQyOSWuPAzuNDkIn7wFLpZStgEQi5HkKIRoAjwHJUsrLyN/HLcXYqII2Gehxzm2jgFVSyubAqoLvw9Vkzn9+K4DLpJTtgD3AP0MdlFZUgjKAlPIPKeUvBX/PJv8NroGxUWlHCNEQuAUYb3QsWhNCVAOuBSYASCndUsoMY6PSlA2IFULYgDjgmMHxBEVK+T1w7manfYApBX+fAvQNaVAaKu75SSmXSym9Bd/+BDQMeWAaUQnKYEKIJkB7YL2xkWjqXeAZwG90IDq4GDgFTCqYwhwvhIg3OigtSCmPAm8BvwN/AJlSyuXGRqWLulLKPyD/wyJwgcHx6GkEsMToICpKJSgDCSGqAHOAJ6SUWUbHowUhxK3ASSnlJqNj0YkN6ACMlVK2B3IJ7ymiswquxfQBmgL1gXghxJ3GRqVUlBDiOfIvJ3xpdCwVpRKUQf6/vXONsaq64vjvD2qNj9RGjRmjKRYFk6ZhIhlfWPHZhyK1KiE60cEQIxonRsVYo6HGtARrMH6glRaNiqhAjUY/GEmDgG9AEcUnoPggUUEFrBSJOv9+2Puaw8kMc0cv3ntn1i+ZzDnr7L3P/5w7c9ZZZ5+7lqTdSc7pftsP11tPDRkFjJX0HjAXOEXSnPpKqinrgfW2KxHvQySH1R84DVhne6Ptr4GHgePrrGlX8ImkFoD8e0Od9dQcSR3AGKDdTfxl13BQdUCSSHMYb9q+rd56aont620fYnsIaYL9Sdv95i7c9sfAh5KGZ9OpwBt1lFRLPgCOlbRX/hs9lX7yAkiJx4COvNwBPFpHLTVH0u+A64Cxtv9Xbz0/hHBQ9WEUcCEpuliZf86ot6igajqB+yW9CrQCU+uspybkqPAhYAWwinR9aOq0OZIeBJ4HhktaL2kiMA04XdIa4PS83pT0cHwzgH2B/+Rry8y6ivwBRKqjIAiCoCGJCCoIgiBoSMJBBUEQBA1JOKggCIKgIQkHFQRBEDQk4aCCIAiChiQcVNB0SLKk6YX1yZJuqtHY90g6rxZj9bKfcTkT+qKSfUg+vs6CbYakCb2MN0nSRb20mSBpRg/bvuyD/O+FpJZKdntJJxUz3Uv6i6QFkn4iaa6kI3a1nqDxCQcVNCPbgXMkHVBvIUUkDe5D84nA5bZP7mbbBuBKSXtUO5jtmbZn92H/NSMnlq2Gq4FZ3fS/gfTdwLNtbwfuIOVyDAY44aCCZuQb0hdIrypvKEdAlcgg37EvkTRf0mpJ0yS1S1omaZWkoYVhTpP0dG43JvcfnOvsLM91di4tjLtI0gOkL7eW9Zyfx39N0i3ZNgU4AZgp6dZujm8jqQxER3mDpKGSnpD0UtZ4ZLbfJGlyXm7LGp/Pml8rDHFw7r9G0t9KY0+XtELSQkkHZlurpBcKtYV+lu2LJU2VtITkTMflY3xF0lPdHBPAucATpX1eA5wBnGV7WzY/nT+Dah1f0E8JBxU0K38H2iX9tA99RpDqVP2KlMljmO2jSWVBOgvthgCjSSVDZioV7ZtIyu7dBrQBl0g6LLc/GrjB9g41vSQdDNwCnELKONEm6WzbNwMvkvKkXduD1mnANd1EZf8COm2PBCYD/+im793AJNvHAd+WtrUC4/M5GC/p0GzfG1hh+yhgCfDnbJ8NXJdrC60q2AH2sz3a9nRgCvBb2yOAsWVB+VxtyhFShVHAJOD3tr97xGi7C1hL+ryCAUw4qKApydnfZ5MK7FXL8lyLazvwDlApJbGK5JQqzLfdZXsN8C5wJPAb4CJJK0mlUfYHKvMky2yv62Z/bcDinHy1klX6xCqPbx2wDLigYlPKfn888O+s459AS7GfUvXUfW0/l00PlIZeaHuL7a9IOQR/nu1dwLy8PAc4ITv//WwvyfZ7S/rnFZafBe6RdAmp0GGZFlJkWGQtINK5LbOBlFE9GMBECB00M7eT8sbdXbB9Q77xyglPi/M4xbv3rsJ6Fzv+L5Tzf5l0Ie20vaC4QdJJpJIb3aFej2DnTCXlxqs8MhsEbLbdupM+ve2zeA6+pedrQDU50L47btuTJB1DijpXSmq1/Vmh7TagXD7+E6AdWCjpM9vFF0b2zH2CAUxEUEHTYvtzYD7p8VuF94CRefkPwO7fY+hxkgblealfAG8DC4DLlMqkIGmYei9UuBQYLemA/KjufNLjs6qw/RYpyhmT178A1kkalzVI0ohSn03AfyUdm03VlmwfBFTm7i4AnrG9Bdgk6dfZfmFP+iUNtb3U9hTgU+DQUpPV7BilVvSuBs4B5kgqOt5hwOtVag/6KRFBBc3OdOCKwvos4FFJy0gvGvQU3eyMt0kX4oNIczlfSbqTdIFdkSOzjfRSKtz2R5KuBxaRIpvHbfe1tMNfgZcL6+3AHZJuJDnfucArpT4TgVmStgKLgS1V7Gcr8EtJL+X247O9gzQPtxfpcefFPfS/Nb8aLtJ530GT7a2S3pF0uO21pW3LJV0MPCbpZOBLYFul6m0wcIls5kHQz5C0T+WlA0l/AlpsX1lnWUj6IzDS9o29tLsK+ML2XT+OsqBRiQgqCPofZ+bIbTfgfWBCfeUkbD8iaf8qmm4G7tvVeoLGJyKoIAiCoCGJlySCIAiChiQcVBAEQdCQhIMKgiAIGpJwUEEQBEFDEg4qCIIgaEj+Dx34R5+Z2hkBAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(range(1,Ks),mean_acc,'g')\n",
    "plt.fill_between(range(1,Ks),mean_acc - 1 * std_acc,mean_acc + 1 * std_acc, alpha=0.10)\n",
    "plt.legend(('Accuracy ', '+/- 3xstd'))\n",
    "plt.ylabel('Accuracy ')\n",
    "plt.xlabel('Number of Neighbors (K)')\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The best value of K = 7  with maximum accuracy of  0.7857\n"
     ]
    }
   ],
   "source": [
    "acc_kmean = mean_acc.max() # K-Mean accuracy\n",
    "best_k = mean_acc.argmax()+1 # Best K for this model\n",
    "print(\"The best value of K =\",best_k,\" with maximum accuracy of \", round(acc_kmean,4) )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Decision Tree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.tree import DecisionTreeClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Creating Train and Test sets\n",
    "X_trainset, X_testset, y_trainset, y_testset = train_test_split(X, y, test_size=0.3, random_state=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Creating DecisionTree Model\n",
    "dTree = DecisionTreeClassifier(criterion=\"entropy\", max_depth = 6)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DecisionTreeClassifier(class_weight=None, criterion='entropy', max_depth=6,\n",
       "                       max_features=None, max_leaf_nodes=None,\n",
       "                       min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "                       min_samples_leaf=1, min_samples_split=2,\n",
       "                       min_weight_fraction_leaf=0.0, presort=False,\n",
       "                       random_state=None, splitter='best')"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Fitting Tree\n",
    "dTree.fit(X_trainset,y_trainset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Prediction\n",
    "predTree = dTree.predict(X_testset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7019230769230769"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "acc_dtree = metrics.accuracy_score(y_testset, predTree)\n",
    "acc_dtree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "def dtr_():\n",
    "    return [dTree]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Support Vector Machine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn import svm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "Xi = np.asarray(X)\n",
    "yi = np.asarray(y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF'],\n",
       "      dtype=object)"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "yi[0:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train set: (276, 8) (276,)\n",
      "Test set: (70, 8) (70,)\n"
     ]
    }
   ],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split( Xi, yi, test_size=0.2, random_state=4)\n",
    "print ('Train set:', X_train.shape,  y_train.shape)\n",
    "print ('Test set:', X_test.shape,  y_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\svm\\base.py:193: FutureWarning: The default value of gamma will change from 'auto' to 'scale' in version 0.22 to account better for unscaled features. Set gamma explicitly to 'auto' or 'scale' to avoid this warning.\n",
      "  \"avoid this warning.\", FutureWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,\n",
       "    decision_function_shape='ovr', degree=3, gamma='auto_deprecated',\n",
       "    kernel='rbf', max_iter=-1, probability=False, random_state=None,\n",
       "    shrinking=True, tol=0.001, verbose=False)"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf = svm.SVC(kernel='rbf')\n",
    "clf.fit(X_train, y_train) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF']    ['PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'COLLECTION' 'PAIDOFF']\n"
     ]
    }
   ],
   "source": [
    "yhat = clf.predict(X_test)\n",
    "print(y[0:5],\"  \",yhat[0:5])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_confusion_matrix(cm, classes,\n",
    "                          normalize=False,\n",
    "                          title='Confusion matrix',\n",
    "                          cmap=plt.cm.Blues):\n",
    "    \"\"\"\n",
    "    This function prints and plots the confusion matrix.\n",
    "    \"\"\"\n",
    "    if normalize:\n",
    "        cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\n",
    "        print(\"Normalized confusion matrix\")\n",
    "    else:\n",
    "        print('Confusion matrix, without normalization')\n",
    "\n",
    "    print(cm)\n",
    "\n",
    "    plt.imshow(cm, interpolation='nearest', cmap=cmap)\n",
    "    plt.title(title)\n",
    "    plt.colorbar()\n",
    "    tick_marks = np.arange(len(classes))\n",
    "    plt.xticks(tick_marks, classes, rotation=45)\n",
    "    plt.yticks(tick_marks, classes)\n",
    "\n",
    "    fmt = '.2f' if normalize else 'd'\n",
    "    thresh = cm.max() / 2.\n",
    "    for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):\n",
    "        plt.text(j, i, format(cm[i, j], fmt),\n",
    "                 horizontalalignment=\"center\",\n",
    "                 color=\"white\" if cm[i, j] > thresh else \"black\")\n",
    "\n",
    "    plt.tight_layout()\n",
    "    plt.ylabel('True label')\n",
    "    plt.xlabel('Predicted label')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "  COLLECTION       0.25      0.07      0.11        15\n",
      "     PAIDOFF       0.79      0.95      0.86        55\n",
      "\n",
      "    accuracy                           0.76        70\n",
      "   macro avg       0.52      0.51      0.48        70\n",
      "weighted avg       0.67      0.76      0.70        70\n",
      "\n",
      "Confusion matrix, without normalization\n",
      "[[52  3]\n",
      " [14  1]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAaMAAAEXCAYAAAAAziuXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd5xcdbnH8c83CaRIDYFQFIJIQOSSEIoIAWlG0EjRUBS9ICWKoijSBK7GAioIIopwQRS8UoJopEkNoIAYSDCUSJMSBAIhFKUETMJz//j9BodlMzu7mZlzduf75jWvzJxz5pxnk2We+ZXz/BQRmJmZFalf0QGYmZk5GZmZWeGcjMzMrHBORmZmVjgnIzMzK9yAogMwM7PW6b/cWhEL59d9fMx/9pqI2KmJIQFORmZmbSUWzmfgenvWffxrM08f1sRw3uRkZGbWVgQq3wiNk5GZWTsRIBUdxds4GZmZtRu3jMzMrHBuGZmZWbE8ZmRmZmXQ4JaRpMeAl4BFwMKI2FTSUGAyMAJ4DNgzIl5Y3DnKlx7NzKx5RGoZ1fuo33YRMToiNs2vjwamRsS6wNT8erGcjMzM2opSy6jeR8/tCpyXn58H7FbrYCcjM7N2072W0TBJ06seEzs5YwDXSppRtX94RMwByH+uUiskjxmZmbWb7rV45lV1vS3OVhHxlKRVgOsk3d/dkNwyMjNrK2r4mFFEPJX/nAtMATYHnpG0GkD+c26tczgZmZm1k0oFhgaNGUl6h6RlK8+BccC9wGXAvvmwfYFLa53H3XRmZm1F0K+hH/3DgSlKiWsAcEFEXC3pDuBiSQcAjwN71DqJk5GZWbvp17j7jCLiEWBUJ9ufA3ao9zxORmZm7aRyn1HJOBmZmbUb16YzM7NiuTadmZmVgVtGZmZWOLeMzMysUEtec64pnIzMzNqNW0ZmZlY4t4zMzKxYnk1nZmZl4JaRmZkVyhUYzMyseO6mMzOzMnA3nZmZFc4tIzMzK5xbRmZmVih5zMjMzMrALSMzMyuanIzMzKxIwsnIzMyKpvwoGScjM7O2IreMzMyseE5GZmZWOCcjMzMrnJORmZkVyxMYzMysaCrpBIby1YQwKxFJgyVdLumfkn6zBOfZR9K1jYytKJK2lvRA0XFYz0mq+9EqTkbWJ0j6lKTpkl6WNEfSVZLGNuDUE4DhwEoRsUdPTxIR50fEuAbE01SSQtJ7ah0TETdHxHqtiskaz8nIrAkkHQacCpxAShxrAj8Ddm3A6dcCHoyIhQ04V68nyV37fYCTkVmDSVoe+DbwxYj4XUS8EhELIuLyiDgiHzNQ0qmSnsqPUyUNzPu2lfSEpK9JmptbVZ/N+74FfAPYK7e4DpA0SdKvq64/IrcmBuTX+0l6RNJLkh6VtE/V9luq3relpDty998dkras2neTpO9IujWf51pJwxbz81fiP7Iq/t0kfUTSg5Kel3RM1fGbS7pN0ov52J9KWjrv+1M+7K788+5Vdf6jJD0N/LKyLb9nnXyNMfn16pLmSdp2if5hrXnUzUeLOBlZb/cBYBAwpcYxxwJbAKOBUcDmwHFV+1cFlgfWAA4ATpe0YkR8k9TamhwRy0TEObUCkfQO4DRg54hYFtgSmNnJcUOBK/OxKwGnAFdKWqnqsE8BnwVWAZYGDq9x6VVJfwdrkJLn2cCngU2ArYFvSHp3PnYR8FVgGOnvbgfgCwARsU0+ZlT+eSdXnX8oqZU4sfrCEfEwcBRwvqQhwC+BcyPiphrxWsHcMjJrvJWAeV10o+0DfDsi5kbEs8C3gM9U7V+Q9y+IiD8ALwM9HRN5A9hQ0uCImBMRszo55qPAQxHxfxGxMCIuBO4HPlZ1zC8j4sGImA9cTEqki7MAOD4iFgAXkRLNjyPipXz9WcBGABExIyL+kq/7GPC/wAfr+Jm+GRGv53jeIiLOBh4CpgGrkZK/lVRlNp2TkVljPQcM62IsY3VgdtXr2Xnbm+fokMxeBZbpbiAR8QqwF/B5YI6kKyWtX0c8lZjWqHr9dDfieS4iFuXnlWTxTNX++ZX3Sxop6QpJT0v6F6nl12kXYJVnI+K1Lo45G9gQ+ElEvN7FsVYwJyOzxrsNeA3YrcYxT5G6mCrWzNt64hVgSNXrVat3RsQ1EfEhUgvhftKHdFfxVGJ6socxdccZpLjWjYjlgGPoemQgau2UtAxpAsk5wKTcDWll5jEja3OPAfeQxlGm520nkT4c7yaN+6zQnRNGxD9J4ySn54H7IZKWkrSzpBPzYRcCx0laOU8E+Abw68WdswszgW0krZknT3y9skPScEm75LGj10ndfYs6OccfgJFK09EHSNoL2AC4oocxdceywL+Al3Or7eAO+58B3v22d9X2Y2BGRBxIGgs7c0kClDRI0u2S7pI0K08ksUaRW0ZmANuRxj82za+vI3XvbAQ8SNWHe70i4hTgMNKkhGeBfwCHAL/Ph3yXlPzuJiXDO/O2bouI64DJ+VwzeGsC6Qd8jdTyeZ40FvOFTs7xHDA+H/sccCQwPiLm9SSmbjqcNDniJVKrbXKH/ZOA8/Jsuz27OpmkXYGdSF2TkP4dxijPIuyh14HtI2IU6XdlJ0lbLMH5rIMyJiNF1GyBmzXSY6QktLgP3d1JN5kuyQeZ9SF5ht4twMERMa3oePqCpVZeJ4btfmLXB2ZPnz1hRkRs2vWRS8YtI2ulAK4ltSgmdrJ/f+CqlkZkpSSpv6SZwFzgOieixvFsuiUkaZGkmZLulfSb/I2psm93pRsP16/aNkLSvfn5tko3F/5V0gOS/iRpfIfzT5R0f37crqpSMko3IT6Qrz9T0oQOMVUeI5r999DLbQWMAXYGvghsU7XvWGAhcH4BcVnJRMSiiBgNvBPYXNKGRcfUZwjUT3U/WqU3lfaYn385kXQ+qY/6lLzvk6Sm/N6kPu/O3BwR4/P7RwO/lzQ/IqbmxPQ5YGxEzFO6m/z3kjaPiMoU230iYnqHc74Zk9WlMoNtLmmywubAn4B9SWMoO9DFzC1rLxHxoqSbSONS9xYcTp/RyhZPvXpNy6iDm4H3wJvTSrci3Tm/dz1vjoiZpBIyh+RNRwFHVAaQI+JO4DzSt3drjHeQZnJVno8jfbjsRPr734V0P421uTzrcYX8fDCwI2nGpTVIo7vpcrfqXyVdkV+vLWmapIckTVYuOVVLb2oZAW8WatwZuDpv2g24OiIqdbjG5GTSlTuBI/Lz95HGMapNJ31jrzhfUuWGwh3yjKjBuV8b4NGI2L2TeCdSGR/RgE00aMU6Qut71h6xFr+76FwABgzoz4UX/44TTjz1qgfvmcbAgUvz3PMvzAWYdvsMDv7yEbVO1WeNWn/NokMohfXWfy+Pz36MwYOHMGjQIFZYYUVWXW31sRuPafoYeqk9/vhjPDdvXmOaNI1vGB0K3Acsl1//APhRRFwk6UxSY+GMWifoTcmo+oP/ZtINdpC66E7Nzy/Kr+tJRl39c4i3dhn1qJsuIs4CzgLoN2SVGLhel7Nl+6SngC32fevv4sD19uS/Jpz8tmPb9e/oxlt+XHQIVmLbjX1/w87VyG46Se8klbg6HjhM6eTbk24hgNTLNIk+lIze9sGvVFhye1ItsAD6AyHpyDrOtzEpkwP8jVRU8oaq/WPydjOzPqMHs+SGSar+In5W/pJdcSrpXrlKN/xKwItVJbae4K2lrjrVm5JRZyYAv4qIz1U2SPojMJZ042OnJG0E/A9wYN50IvADSTtFxHN5gsN+QOO+ipiZlUQ3k9G8xd1nlCd/zY2IGfrPsiGdnbzLiUm9PRl9Evh+h22/JTUPf9Bh+9aS/kqqKzYX+HJETAWIiMskrQH8ObewXgI+HRFzmhq9mVkBGthNtxWwi6SPkJYxWY7UUlpB0oDcOnonddSC7DXJKCLeVrU4IrbtZNtpVS83zNtuIq1XU+v8Z7CYPs3OrrO4mMzMSq9BuSgivk4u4ZVbRodHxD6SfkPqubqINBHs0q7O1VundpuZWQ+1oALDUaTJDH8njSHVXJgSelHLyMzMGkDNuek190DdlJ8/QrqpvW5ORmZmbURACQswOBmZmbWX1hZArZeTkZlZmylhLnIyMjNrN24ZmZlZseSWkZmZFUxAvxauU1QvJyMzszbjlpGZmRXOY0ZmZlYsjxmZmVnR0k2v5ctGTkZmZm3FN72amVkJlDAXORmZmbUbt4zMzKxYnsBgZmZF8wQGMzMrhRLmIicjM7N245aRmZkVroS5yMnIzKytNGnZ8SXlZGRm1ka87LiZmZWAKzCYmVkJlDAXORmZmbUbt4zMzKxYrsBgZmZFcwUGMzMrBScjMzMrXAlzkZORmVm7ccvIzMwKJYl+/ZyMzMysYCVsGDkZmZm1m34lzEZORmZmbaaEucjJyMysnchVu83MrAxKOH/BycjMrN24ZWRmZoUrYS5yMjIzaycCRPmy0WKTkaTlar0xIv7V+HDMzKzZetuY0Swg4C0ptPI6gDWbGJeZmTWDetlKrxHxrlYGYmZmrVHCXES/eg6StLekY/Lzd0rapLlhmZlZM4hUgaHeR5fnkwZJul3SXZJmSfpW3r62pGmSHpI0WdLStc7TZTKS9FNgO+AzedOrwJldRmhmZqUk1f+ow+vA9hExChgN7CRpC+AHwI8iYl3gBeCAWiepp2W0ZUR8DngNICKeB2pmODMzKy/lcaN6Hl2J5OX8cqn8CGB74JK8/Txgt1rnqScZLZDUL58cSSsBb9TxPjMzK5nutIpyLhomaXrVY+Lbz6n+kmYCc4HrgIeBFyNiYT7kCWCNWnHVc5/R6cBvgZVzX+CewLfq/cHNzKxculm1e15EbFrrgIhYBIyWtAIwBXhvZ4fVOkeXySgifiVpBrBj3rRHRNzb1fvMzKycmjWZLiJelHQTsAWwgqQBuXX0TuCpWu+tazYd0B9YAPy7G+8xM7MSauSYkaSVc4sISYNJDZf7gBuBCfmwfYFLa52nntl0xwIXAquTstsFkr7eZYRmZlY6aWp3/Y86rAbcKOlu4A7guoi4AjgKOEzS34GVgHNqnaSeMaNPA5tExKsAko4HZgDfqytMMzMrjwZXYIiIu4GNO9n+CLB5veepJxnN7nDcAOCRei9gZmblUsYKDLUKpf6INPvhVWCWpGvy63HALa0Jz8zMGq1X1aYDKjPmZgFXVm3/S/PCMTOzZqqMGZVNrUKpNQebzMysd+ptLSMAJK0DHA9sAAyqbI+IkU2My8zMmqR8qai+e4bOBX5Jin9n4GLgoibGZGZmTSI1tmp3o9STjIZExDUAEfFwRBxHquJtZma9UIOrdjdEPVO7X1fqYHxY0ueBJ4FVmhuWmZk1S68cMwK+CiwDfJk0drQ8sH8zgzIzs+YpYS6qq1DqtPz0Jf6zwJ6ZmfVCorVjQfWqddPrFGqU/I6Ijzcloj5s5DprcM7F3yk6DCupQUv3LzoEK7GG3RvU4rGgetVqGf20ZVGYmVnL9Koxo4iY2spAzMysNcq4DlA9ExjMzKyPEL2sZWRmZn1Tr6pN15GkgRHxejODMTOz5itjMqpnpdfNJd0DPJRfj5L0k6ZHZmZmDZcqKzRu2fFGqWcc6zRgPPAcQETchcsBmZn1Wg1edrwh6umm6xcRsztkyEVNisfMzJpIQP8S9tPVk4z+IWlzICT1B74EPNjcsMzMrFl669Tug0lddWsCzwDX521mZtYLlXBmd1216eYCe7cgFjMzazK1eJ2ietWz0uvZdFKjLiImNiUiMzNrqhLmorq66a6vej4I2B34R3PCMTOzZivh/IW6uukmV7+W9H/AdU2LyMzMmkbQO7vpOrE2sFajAzEzs9YoYS6qa8zoBf4zZtQPeB44uplBmZlZk7T4ZtZ61UxGSne6jgKezJveiIjFLrhnZmblJ8qXjWre+5QTz5SIWJQfTkRmZr1YGjMqXzmgem7EvV3SmKZHYmZmLVHGZLTYbjpJAyJiITAWOEjSw8ArpMQaEeEEZWbWC/W2xfVuB8YAu7UoFjMza7JKN13Z1EpGAoiIh1sUi5mZNZt639TulSUdtridEXFKE+IxM7Mm6203vfYHloESzgE0M7Me6Y3ddHMi4tsti8TMzFqihA2jrseMzMysLxH9SvjxXisZ7dCyKMzMrCVEL2sZRcTzrQzEzMxaoDfWpjMzs76njLPp6ikHZGZmfUSlm67eR5fnk94l6UZJ90maJenQvH2opOskPZT/XLHWeZyMzMzaTD+p7kcdFgJfi4j3AlsAX5S0AWmpoakRsS4wlS6WHnIyMjNrM41sGUXEnIi4Mz9/CbgPWAPYFTgvH3YeXZSW85iRmVkbEd1uhQyTNL3q9VkRcVan55ZGABsD04DhETEHUsKStEqtizgZmZm1E3W7ave8iNi0y9NKywC/Bb4SEf/qbmVwd9OZmbUZdeNR1/mkpUiJ6PyI+F3e/Iyk1fL+1YC5tc7hZGRm1kZSbbrGTWBQagKdA9zXoYD2ZcC++fm+wKW1zuNuOjOzNtPgu4y2Aj4D3CNpZt52DPB94GJJBwCPA3vUOomTkZlZm2nkPa8RcQuLz291l5VzMjIzayvqdcuOm5lZH9ODqd0t4WRkZtZm3DIyM7PClS8VORmZmbWX7t/02hJORmZmbcRjRmZmVgpuGZmZWeHKl4qcjMzM2oqA/m4ZmZlZ0UqYi5yMzMzai1AJO+qcjMzM2oxbRmZmVqg0tbt82cjJyMysncgtIzMzKwEnIzMzK5wnMJiZWaHSsuNFR/F2TkZmZm3GLSMzMyucx4zMzKxwbhmZmVmhPGZkZmYl4HJAZmZWtJLe9FrGBf+sDzrh64cwfouRfOajW75t3wXn/ISxI4fy4vPPFRCZlc3nDtyfNVdfhU1Gb1h0KH2WuvFoFScja4mPfPxTnHzOb962/Zk5TzD91psYvvo7C4jKyugz++7HpVdcXXQYfVYaM1Ldj1ZxMrKWGL3Zliy3/Ipv2/6TE47l4CO+VcplkK0YY7fehqFDhxYdRp9WxpaRx4ysMLdMvYphw1dj3fe6O8aspUr43a+pLSNJq0q6SNLDkv4m6Q+SRkp6n6QbJD0o6SFJ/6P81VjSfpJ+2sm5HpM0rMO2/SQ9K2lm1WODvG9kvt7fJd0n6WJJe1Ud97KkB/LzX0naVtIVVefeTdLdku6XdI+k3ar2nSvpSUkD8+thkh5r0l9jn/Ta/Fc574yTOfDQY4oOxaztqBv/tUrTWkY5uUwBzouIvfO20cBw4Fzg4Ii4VtIQ4LfAF4DTe3CpyRFxSIdrDwKuBA6LiMvztu2AZyNidH59E3B4REzPr7etev8o4IfAhyLiUUlrA9dJeiQi7s6HLQL2B87oQcxt78nHH2POE4+z3y5bA/Ds00+x/+7bcvYl17PSysMLjs6sbytjr3gzu+m2AxZExJmVDRExU9IBwK0RcW3e9qqkQ4Cb6Fky6syngNsqiShf58ZuvP9w4ISIeDS/91FJ3wOOAD6TjzkV+KqksxsUc1tZZ70NuOIvD775esJ2o/j5b29ghaErFRiVWXsoYS5qajLaEJjRyfb3ddweEQ9LWkbScj24zl6Sxla9/kCNa9frfaSWUbXpwBerXj8O3EJKTpezGJImAhPzy5fHjhz6wBLE1ZutDSwLDBg7cuhC4Km8fV7+87/Gb7HufcDCIoKzUqn8riw1eCktIP2uzKv9lrawVsPOVMJsVMQEBgGxmH2L215LZ910PTjNW3QWY2fbTgAuI3UJdioizgLOWtKA+iJJ0yNi06LjsHLy70dzpFly5ctGzZzAMAvYZDHb3/ILJundwMsR8VKTr92d93f8n2AM8LfqDRHxd2AmsOcSXMvMrHVyBYZ6H63SzGR0AzBQ0kGVDZI2Ax4CxkraMW8bDJwGnNjAa18AbCnpo1XX3knSf9X5/h8CX5c0Ir93BHAMcHInxx5PGmMyM+sVynifUdOSUUQEsDvwoTy1exYwidT/uytwnKQHgHuAO4Dq6dz7SXqi6lG5Pf/uqm2n5G3V07VnStoyIuYD44Ev5anjfwP2A+bWGftM4Cjgckn3k8aEjszbOx47C7izG3819h/uvrRa/PvRLCXMRko5w8zM2sEGG42J8y//Y93Hjxmx3IxWjN25AoOZWZtpt/uMzMysZFo9FlQvJyMz6/Mk9YuIN4qOozRKmI1ctdtKT9L6edalWbdIGgMQEW9I8uddVsbadP7HsVKTtBPppuI1io7FeqXjJE0FJ6RqjbzPSNIvJM2VdG/VtqGSrsuzma+T9Pb1YzrwP4yVlqTxwNeBz0XE3yW9o+iYrNfZE3hB0uXghFTR4Jnd5wI7ddh2NDA1ItYFpubXNbX9P4qVU65TOAWYFhHX5xuPp0h6b6GBWelJ2krSGEkrRsRCYA/gFUlXgRNStzJRHdkoIv4EPN9h867Aefn5ecBudMETGKx0JL0nt4TGA+dLmk36Zb48Iu4rODwrMUlrkL6pDwduljQTuBg4CDhV0iURMaGSkNp1UkM3x4KGSZpe9fqsXHOzluERMQcgIuZIWqWri7TvtwMrJUkfAc6V9O6IuIa0HMgpwDMRcVo+pn+RMVp5RcSTwEnAH4E/kMYaDwKuBqaRKsJMzse2aSLq9pjRvIjYtOrRlMoYTkZWGpI+DHwH+EZEPCJpcF736iPAxyR9Ih/alh8itniSVqqUDcsflpcDI4FLgC+T6koGMBt4v6TVi4q1DFpQDegZSasB5D+7LMXmbjorBUlrAT8DToyIGyStCfxU0sSImCppT+ACSYMi4vxio7Uyyd25x+XnjwKPAceS6lHuSVoj69KIWCRpCtAvIuqqU9lnNX/G9mXAvsD385+XdvUGJyMrhYiYnT8oPpDHiI4DpkTE07lv/xpJ+wGnS7qMtOSICyu2OUkfInXjHgDcDWyUn18E7A30ByYAgyVdFxFepA/o18B6QJIuBLYljS09AXyTlIQuzit7P06aRFKTk5EVKs+aGxgRz0bE4ZKOJlVw/01EnPyfw9QvIi6XdENEvFJcxFYymwOTIuLm/PrmXKX/ZOC4iPi2pOHAdsC1RQVZNo1sGEXEJxeza4funMfJyAqT15s6ElhK0mPAn0nrWr0KbJ3Xn/pb7l6p/P/zaiHBWlmtCqxMWsMMgIh4TtJFwH/n19+VNNRfYqq4HJBZkicrnAp8m7Tu1RWkLpYf5VlzdwHfAEbBm+tj4a45k7SRpK/ml1elTVq+w2EzgNUkDQOIiI73wbStyrLjLgdkbS/fcLgDcExETI2IZ4ALgdNJffsHRMR3gWeAQyUNLDBcKxFJI4FhwLg8hngjsAFwpKQVqg7difT5Nr/lQZZdSZcddzedtVy+4XAw8B5I9w3lrri7gdtJ/fvnRMQhklaOiNeLjNfKIdcpPBz4EmnSwpeAl0kz5iYDx0saBNwH7A/s5a65zpWwl87JyAozi/SNlpyIBkTEQklXA3tKWiYiXo6IZ4sN08ogd+v+EPhCRNyXZ1yKlJAE7AJsBowD/g18wtU6aihhNnIyspaQNAp4F7A8aTzoV8Dtkk6KiCNyDTFIHyaQ7g0xqySiyaSqCjfnmZWvSqrMoDsEWDYifgHcvLjzWEVrx4Lq5WRkTZdL/PyQVJ5lY2ARcA3wAeBWSUOAl4Cngc8DEyLitYLCtRKR9AHgNFJZqD1I3XMnAU9FxPyckAI4Onf3nl1ctL2Hlx23tpOnb38DODAi/pwTz/rAmaQPkfcDn8jblgU+HhF/KypeK49cof1dpN+dmyX9BTgLODy3qOfkhHQr8F3goSLj7S3Kuuy4Z9NZUygZRmoR/TUi/gwQEa9GxJ3AF4EdgTUi4tcRcRzwXScigze/xJxD+oIyK48pPk+qrvAu4IhK7bP8O3VjRDxRXMS9TAuK03WXk5E1y7BceuVIYEVJX+yw/yHSFN0NKht8D5EBSBpHugftcOAXEfF8ntzSPyL+CRwIrA5MkrRqkbH2Vr7PyNqCpJWBmyR9IiIuJ60vM07SwZVjIuJF4A7ghWKitBLbBTg2t6b7QWpp51mX/XJCOhgYSOrqtW4q431GTkbWcHk69jeBYyXtEhFXkcaIdq4kJEmfArYGHi0uUiuTqpJP61Rt7lh5Y628tMgLwGfzDdPWTSXspfMEBmuOiLhE0kLSjYhExGX5s2Y/STuSulk+5X5+A5C0DfAa6abnq4CxuSjuvKqbooeQlrP+NTDf3bo91OIWT73cMrKGkLSzpMMkbVjZFhG/J82kO76qhXQBafXNiRFxb0HhWonkMaJz+c+X4+nAEGAXScMiYlHeviupjqE/t5ZY+dpGbhnZEsu15nYGPg1sJmkBafD51YiYkpcJn5QXxrtY0vUu02Lw5qy5ScCn89T/lYF7gOtJa+R8QNItwFDS8uF7tv3CeEuosux42fgbhvVYpY8/It4gda08CHyVVI7lKOBkSSMi4hLSGNKXJC3rRGQAeSbcJGBaTkTDgduAD0fERcAvSNW3P0Tq1p3g1nRjlK9d5JaRLZn+5LI9EXGVpM8CB0XEgZIOBU4gtZSuIVVcGBcRrqJsFS+TFlLcUNJRwHjSsvOXAETE7aQxpDOLC7FvKmPLyMnIeiQv97y/pLuA+/P40I+Aj0jaFTgU+DBpMbyxwN+diKwiT9V+WdKlwAJSwdMnIuKsqmM+DvwLuCG3vq1Bylibzt101m25lP/xpJVZ3wHsKmkMqXT/WOB3pBIut+RqCz/zrDkDyF1xRETkhPQicBmp/twzuXWNpAmklvVsJ6ImKGE/nZORdYukoaSCp9+JiJ+QaoUNAdbOHyxfAaaSEhMAVRW5rU1VlYe6Pd9jVp2QXgauJn25GS1pMmkCzO4R4XpzTVDCXORkZN2T64N9DPi+pOUi4h+kbpZheULDbFJVha2qbmK0NhfJPNJyD5Mk7VHZnu8jegG4iLS8yDKksUevR9QE3am+4JVerdQi4kpJbwAz8uSEIcCv8k2IL+ZxgBm+KdEq8vT+iIjL89T/M3Kr6GKg0g33UWA54JMR8a+iYm0HZRwzcjKyHsmz5w4GrgVWzaX8h+QKyhcUHZ+Vg6T1SNUSHpfUL9eWuzr/7pyRq3NcLGki6Qbp7Z2IWqB8ucjJyHouIq7PNy3eKGk734xo1SQtD+wLrCDpe8uiLTUAAAWYSURBVBHxD0n9OySk0ySNJy20+NGIeLDQoNtECXORx4xsyeQSP8cAV+dvvmX8PbcC5OravwdeBL4qac1c2qd/Hie6GvgyMAr4RETcVWC4baWMY0ZORrbEIuJSYJuIeMPjRCZppKTNJK1LqjP3c1JVjq/kihwLcuHTQ0iTFd4fEXcXGXN76c5qRl7PyHqZPD3X2pykXUj3mU0i3QQ9DXgJ+CVp1uWXc1fdnsCxwMMR8VpB4balSm26srWMPGZkZg0haSvgRGCfiJiRt/0MuAH4IPC/wGeBW4C1SOWh7ikoXCsZt4zMrFFWBX4eETMkDQSIiC+Qks9lpIUULyPVm/uwE1Fx3DIysz5H0qb56cbAZsAPI+J1SUtFxALSJIUpwBoRcYekuyPi9aLitXLeZ+SWkZn1WJ7a/wvSUuFXAP+QtGuevr1A0oCckPoBywM4ERXMFRjMrC+R9EHgx6QxommSBgGPANuRks+UiFiYS/+sDjxbXLRW0eqac/VyMjKzntoE+ElOREtHxGuSzgL2J1Vy/wqp+Ol4UsLyTdFlUcJs5GRkZt2Sa8oFsDbwz7x5Qe6amyfpNNKNrGNJS4j/PCIeLihc64THjMys16u6sXkKsIWkTSrb8hjRa6SJDFMj4lonovIp45iRk5GZ9dQ00rTtvXJCeiOPEe1Fqkn3fLHh2eKUcT0jd9OZWY9ExCuSzgYOAE6RdAfwGjABmBARjxcaoC1e+XrpnIzMrOci4klJJ5GqLOwIzAF2cfXtcivjmJFc19LMrH1IuhoY1o23zIuInZoVT4WTkZmZFc4TGMzMrHBORmZmVjgnIzMzK5yTkZmZFc7JyMzMCudkZG1L0iJJMyXdK+k3koYswbm2lXRFfr6LpKNrHLuCpC/04BqTJB1e7/YOx5wraUI3rjVC0r3djdGsp5yMrJ3Nj4jREbEh8G/g89U7lXT7/5GIuCwivl/jkBWAbicjs77MycgsuRl4T24R3CfpZ8CdwLskjZN0m6Q7cwtqGQBJO0m6X9ItwMcrJ5K0n6Sf5ufDJU2RdFd+bAl8H1gnt8pOyscdIekOSXdL+lbVuY6V9ICk64H1uvohJB2Uz3OXpN92aO3tKOlmSQ9KGp+P7y/ppKprf25J/yLNesLJyNqepAHAzqTlDiB96P8qIjYGXgGOA3aMiDHAdOCwvJDc2cDHgK2BVRdz+tOAP0bEKGAMMAs4Gng4t8qOkDQOWBfYHBgNbCJpG0mbAHuTlvP+OKkSdld+FxGb5evdR6obVzEC+CDwUeDM/DMcAPwzIjbL5z9I0tp1XMesoVybztrZYEkz8/ObgXNIK5LOjoi/5O1bABsAtyrV018auA1YH3g0Ih4CkPRrYGIn19ge+G+AiFgE/FPSih2OGZcff82vlyElp2VJq6W+mq9xWR0/04aSvkvqClwGuKZq38UR8QbwkKRH8s8wDtioajxp+Xxt15azlnIysnY2PyJGV2/ICeeV6k3AdRHxyQ7HjQYaVUtLwPci4n87XOMrPbjGucBuEXGXpP2Abav2dTxX5Gt/KSKqkxaSRnTzumZLxN10ZrX9BdhK0nsAJA2RNBK4H1hb0jr5uE8u5v1TgYPze/tLWg54idTqqbgG2L9qLGoNSasAfwJ2lzRY0rKkLsGuLAvMkbQUsE+HfXtI6pdjfjfwQL72wfl4JI2U9I46rmPWUG4ZmdUQEc/mFsaFkgbmzcdFxIOSJgJXSppHWmRuw05OcShwlqQDgEXAwRFxm6Rb89Tpq/K40XuB23LL7GXg0xFxp6TJwExgNqkrsSv/Q1r0bjZpDKw66T0A/BEYDnw+Il6T9HPSWNKdShd/Ftitvr8ds8Zx1W4zMyucu+nMzKxwTkZmZlY4JyMzMyuck5GZmRXOycjMzArnZGRmZoVzMjIzs8L9P5nUWolfswjAAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Compute confusion matrix\n",
    "cnf_matrix = confusion_matrix(y_test, yhat, labels=['PAIDOFF','COLLECTION'])\n",
    "np.set_printoptions(precision=2)\n",
    "\n",
    "print (classification_report(y_test, yhat))\n",
    "\n",
    "# Plot non-normalized confusion matrix\n",
    "plt.figure()\n",
    "plot_confusion_matrix(cnf_matrix, classes=['PAIDOFF','COLLECTION'],normalize= False,  title='Confusion matrix')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "def svm_():\n",
    "    return [clf]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train set: (276, 8) (276,)\n",
      "Test set: (70, 8) (70,)\n"
     ]
    }
   ],
   "source": [
    "# Train Test split\n",
    "X_train, X_test, y_train, y_test = train_test_split( Xn, y, test_size=0.2, random_state=4)\n",
    "print ('Train set:', X_train.shape,  y_train.shape)\n",
    "print ('Test set:', X_test.shape,  y_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(C=0.01, class_weight=None, dual=False, fit_intercept=True,\n",
       "                   intercept_scaling=1, l1_ratio=None, max_iter=100,\n",
       "                   multi_class='warn', n_jobs=None, penalty='l2',\n",
       "                   random_state=None, solver='liblinear', tol=0.0001, verbose=0,\n",
       "                   warm_start=False)"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Model\n",
    "LR = LogisticRegression(C=0.01, solver='liblinear').fit(X_train,y_train)\n",
    "LR"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['COLLECTION', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF',\n",
       "       'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF',\n",
       "       'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF',\n",
       "       'COLLECTION', 'PAIDOFF', 'COLLECTION', 'PAIDOFF', 'PAIDOFF',\n",
       "       'PAIDOFF', 'COLLECTION', 'PAIDOFF', 'PAIDOFF', 'COLLECTION',\n",
       "       'COLLECTION', 'PAIDOFF', 'COLLECTION', 'PAIDOFF', 'PAIDOFF',\n",
       "       'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF',\n",
       "       'COLLECTION', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'COLLECTION',\n",
       "       'PAIDOFF', 'PAIDOFF', 'COLLECTION', 'PAIDOFF', 'PAIDOFF',\n",
       "       'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF',\n",
       "       'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF',\n",
       "       'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF',\n",
       "       'COLLECTION', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF',\n",
       "       'PAIDOFF', 'PAIDOFF'], dtype=object)"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Prediction\n",
    "yhat = LR.predict(X_test)\n",
    "yhat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "yhat_prob = LR.predict_proba(X_test)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "def lgr_():\n",
    "    return [LR]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model Evaluation using Test set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import jaccard_similarity_score\n",
    "from sklearn.metrics import f1_score\n",
    "from sklearn.metrics import log_loss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, download and load the test set:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Load Test set for evaluation "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/8/2016</td>\n",
       "      <td>10/7/2016</td>\n",
       "      <td>50</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>300</td>\n",
       "      <td>7</td>\n",
       "      <td>9/9/2016</td>\n",
       "      <td>9/15/2016</td>\n",
       "      <td>35</td>\n",
       "      <td>Master or Above</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>21</td>\n",
       "      <td>21</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/10/2016</td>\n",
       "      <td>10/9/2016</td>\n",
       "      <td>43</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>24</td>\n",
       "      <td>24</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/10/2016</td>\n",
       "      <td>10/9/2016</td>\n",
       "      <td>26</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>35</td>\n",
       "      <td>35</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>800</td>\n",
       "      <td>15</td>\n",
       "      <td>9/11/2016</td>\n",
       "      <td>9/25/2016</td>\n",
       "      <td>29</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           1             1     PAIDOFF       1000     30       9/8/2016   \n",
       "1           5             5     PAIDOFF        300      7       9/9/2016   \n",
       "2          21            21     PAIDOFF       1000     30      9/10/2016   \n",
       "3          24            24     PAIDOFF       1000     30      9/10/2016   \n",
       "4          35            35     PAIDOFF        800     15      9/11/2016   \n",
       "\n",
       "    due_date  age             education  Gender  \n",
       "0  10/7/2016   50              Bechalor  female  \n",
       "1  9/15/2016   35       Master or Above    male  \n",
       "2  10/9/2016   43  High School or Below  female  \n",
       "3  10/9/2016   26               college    male  \n",
       "4  9/25/2016   29              Bechalor    male  "
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df = pd.read_csv('https://raw.githubusercontent.com/ayushman1024/Machine-Learning-Deep-Learning-Algorithms/master/DATA/loan_test.csv')\n",
    "test_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_df['due_date'] = pd.to_datetime(test_df['due_date'])\n",
    "test_df['effective_date'] = pd.to_datetime(test_df['effective_date'])\n",
    "test_df['dayofweek'] = test_df['effective_date'].dt.dayofweek\n",
    "test_df['weekend'] = test_df['dayofweek'].apply(lambda x: 1 if (x>3)  else 0)\n",
    "test_df.groupby(['Gender'])['loan_status'].value_counts(normalize=True)\n",
    "test_df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)\n",
    "test_df.groupby(['education'])['loan_status'].value_counts(normalize=True)\n",
    "\n",
    "Feature = test_df[['Principal','terms','age','Gender','weekend']]\n",
    "Feature = pd.concat([Feature,pd.get_dummies(test_df['education'])], axis=1)\n",
    "Feature.drop(['Master or Above'], axis = 1,inplace=True)\n",
    "\n",
    "X = Feature\n",
    "# Xn is normalised X \n",
    "Xn = preprocessing.StandardScaler().fit(X).transform(X.astype(float))\n",
    "Xi = np.asarray(X)\n",
    "y = test_df['loan_status'].values\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LabelEncoder()"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Making Label Encoder\n",
    "le = preprocessing.LabelEncoder()\n",
    "le.fit(['COLLECTION', 'PAIDOFF'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Declare evaluation data\n",
    "eval = {}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\metrics\\classification.py:635: DeprecationWarning: jaccard_similarity_score has been deprecated and replaced with jaccard_score. It will be removed in version 0.23. This implementation has surprising behavior for binary and multiclass classification tasks.\n",
      "  'and multiclass classification tasks.', DeprecationWarning)\n"
     ]
    }
   ],
   "source": [
    "# KNN\n",
    "knn_model = knn_()[0]\n",
    "knn_k = knn_()[1]\n",
    "\n",
    "yhat_knn = knn_model.predict(Xn)\n",
    "j = jaccard_similarity_score(y, yhat_knn)\n",
    "f = f1_score(y, yhat_knn, average='weighted') \n",
    "l = 'NA'\n",
    "eval[\"KNN\"] = [j, f, l]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Decision Tree\n",
    "dtr_model = dtr_()[0]\n",
    "\n",
    "yhat_dtr = dtr_model.predict(X)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\metrics\\classification.py:635: DeprecationWarning: jaccard_similarity_score has been deprecated and replaced with jaccard_score. It will be removed in version 0.23. This implementation has surprising behavior for binary and multiclass classification tasks.\n",
      "  'and multiclass classification tasks.', DeprecationWarning)\n"
     ]
    }
   ],
   "source": [
    "j = jaccard_similarity_score(y, yhat_dtr)\n",
    "f = f1_score(y, yhat_dtr, average='weighted') \n",
    "l = 'NA'\n",
    "eval[\"Decision Tree\"] = [j, f, l]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\metrics\\classification.py:635: DeprecationWarning: jaccard_similarity_score has been deprecated and replaced with jaccard_score. It will be removed in version 0.23. This implementation has surprising behavior for binary and multiclass classification tasks.\n",
      "  'and multiclass classification tasks.', DeprecationWarning)\n"
     ]
    }
   ],
   "source": [
    "# Support Vector Machine\n",
    "svm_model = svm_()[0]\n",
    "\n",
    "yhat_svm = svm_model.predict(Xi)\n",
    "j = jaccard_similarity_score(y, yhat_svm)\n",
    "f = f1_score(y, yhat_svm, average='weighted') \n",
    "l = 'NA'\n",
    "eval[\"SVM\"] = [j, f, l]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\metrics\\classification.py:635: DeprecationWarning: jaccard_similarity_score has been deprecated and replaced with jaccard_score. It will be removed in version 0.23. This implementation has surprising behavior for binary and multiclass classification tasks.\n",
      "  'and multiclass classification tasks.', DeprecationWarning)\n",
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\metrics\\classification.py:1437: UndefinedMetricWarning: F-score is ill-defined and being set to 0.0 in labels with no predicted samples.\n",
      "  'precision', 'predicted', average, warn_for)\n"
     ]
    }
   ],
   "source": [
    "# Logistic Regression\n",
    "lgr_model = lgr_()[0]\n",
    "\n",
    "yhat_lgr = lgr_model.predict(X)\n",
    "yhat_lgr_le = le.transform(yhat_lgr)\n",
    "yhat_lgr_prob = lgr_model.predict_proba(X)\n",
    "\n",
    "j = jaccard_similarity_score(y, yhat_lgr)\n",
    "f = f1_score(y, yhat_lgr, average='weighted') \n",
    "l = log_loss(y, yhat_lgr_le)\n",
    "\n",
    "eval[\"LogisticRegression\"] = [j, f, l]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [],
   "source": [
    "evalDF = pd.DataFrame.from_dict(eval,orient='index', columns=['Jaccard','F1-score','LogLoss'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Jaccard</th>\n",
       "      <th>F1-score</th>\n",
       "      <th>LogLoss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>KNN</td>\n",
       "      <td>0.685185</td>\n",
       "      <td>0.645381</td>\n",
       "      <td>NA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>Decision Tree</td>\n",
       "      <td>0.814815</td>\n",
       "      <td>0.809951</td>\n",
       "      <td>NA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>SVM</td>\n",
       "      <td>0.759259</td>\n",
       "      <td>0.671764</td>\n",
       "      <td>NA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>LogisticRegression</td>\n",
       "      <td>0.259259</td>\n",
       "      <td>0.106754</td>\n",
       "      <td>25.5843</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                     Jaccard  F1-score  LogLoss\n",
       "KNN                 0.685185  0.645381       NA\n",
       "Decision Tree       0.814815  0.809951       NA\n",
       "SVM                 0.759259  0.671764       NA\n",
       "LogisticRegression  0.259259  0.106754  25.5843"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "evalDF"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Report\n",
    "You should be able to report the accuracy of the built model using different evaluation metrics:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "| Algorithm          | Jaccard | F1-score | LogLoss |\n",
    "|--------------------|---------|----------|---------|\n",
    "| KNN                | ?       | ?        | NA      |\n",
    "| Decision Tree      | ?       | ?        | NA      |\n",
    "| SVM                | ?       | ?        | NA      |\n",
    "| LogisticRegression | ?       | ?        | ?       |"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}