Machine Learning with Python.ipynb

{
    "nbformat_minor": 2, 
    "cells": [
        {
            "source": "<a href=\"https://www.bigdatauniversity.com\"><img src=\"https://ibm.box.com/shared/static/cw2c7r3o20w9zn8gkecaeyjhgw3xdgbj.png\" width=\"400\" align=\"center\"></a>\n\n<h1 align=\"center\"><font size=\"5\">Classification with Python</font></h1>", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "In this notebook we try to practice all the classification algorithms that we learned in this course.\n\nWe load a dataset using Pandas library, and apply the following algorithms, and find the best one for this specific dataset by accuracy evaluation methods.\n\nLets first load required libraries:", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 1, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [], 
            "source": "import itertools\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom matplotlib.ticker import NullFormatter\nimport pandas as pd\nimport numpy as np\nimport matplotlib.ticker as ticker\nfrom sklearn import preprocessing\nimport seaborn as sns\n%matplotlib inline"
        }, 
        {
            "source": "### About dataset", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "This dataset is about past loans. The __Loan_train.csv__ data set includes details of 346 customers whose loan are already paid off or defaulted. It includes following fields:\n\n| Field          | Description                                                                           |\n|----------------|---------------------------------------------------------------------------------------|\n| Loan_status    | Whether a loan is paid off on in collection                                           |\n| Principal      | Basic principal loan amount at the                                                    |\n| Terms          | Origination terms which can be weekly (7 days), biweekly, and monthly payoff schedule |\n| Effective_date | When the loan got originated and took effects                                         |\n| Due_date       | Since it\u2019s one-time payoff schedule, each loan has one single due date                |\n| Age            | Age of applicant                                                                      |\n| Education      | Education of applicant                                                                |\n| Gender         | The gender of applicant                                                               |", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "Lets download the dataset", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 2, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "--2019-05-22 04:30:49--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_train.csv\nResolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.193\nConnecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.193|:443... connected.\nHTTP request sent, awaiting response... 200 OK\nLength: 23101 (23K) [text/csv]\nSaving to: \u2018loan_train.csv\u2019\n\n100%[======================================>] 23,101      --.-K/s   in 0.002s  \n\n2019-05-22 04:30:49 (10.8 MB/s) - \u2018loan_train.csv\u2019 saved [23101/23101]\n\n"
                }
            ], 
            "source": "!wget -O loan_train.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_train.csv"
        }, 
        {
            "source": "### Load Data From CSV File  ", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 3, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 3, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/8/2016</td>\n      <td>10/7/2016</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>2</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/8/2016</td>\n      <td>10/7/2016</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>3</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>9/8/2016</td>\n      <td>9/22/2016</td>\n      <td>27</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>4</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/9/2016</td>\n      <td>10/8/2016</td>\n      <td>28</td>\n      <td>college</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>6</td>\n      <td>6</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/9/2016</td>\n      <td>10/8/2016</td>\n      <td>29</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           0             0     PAIDOFF       1000     30       9/8/2016   \n1           2             2     PAIDOFF       1000     30       9/8/2016   \n2           3             3     PAIDOFF       1000     15       9/8/2016   \n3           4             4     PAIDOFF       1000     30       9/9/2016   \n4           6             6     PAIDOFF       1000     30       9/9/2016   \n\n    due_date  age             education  Gender  \n0  10/7/2016   45  High School or Below    male  \n1  10/7/2016   33              Bechalor  female  \n2  9/22/2016   27               college    male  \n3  10/8/2016   28               college  female  \n4  10/8/2016   29               college    male  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "df = pd.read_csv('loan_train.csv')\ndf.head()"
        }, 
        {
            "execution_count": 4, 
            "cell_type": "code", 
            "metadata": {
                "scrolled": true
            }, 
            "outputs": [
                {
                    "execution_count": 4, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "(346, 10)"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "df.shape"
        }, 
        {
            "source": "Let's clean the data by deleting the first two unnamed columns", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 5, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 5, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/8/2016</td>\n      <td>10/7/2016</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/8/2016</td>\n      <td>10/7/2016</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>9/8/2016</td>\n      <td>9/22/2016</td>\n      <td>27</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/9/2016</td>\n      <td>10/8/2016</td>\n      <td>28</td>\n      <td>college</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/9/2016</td>\n      <td>10/8/2016</td>\n      <td>29</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "  loan_status  Principal  terms effective_date   due_date  age  \\\n0     PAIDOFF       1000     30       9/8/2016  10/7/2016   45   \n1     PAIDOFF       1000     30       9/8/2016  10/7/2016   33   \n2     PAIDOFF       1000     15       9/8/2016  9/22/2016   27   \n3     PAIDOFF       1000     30       9/9/2016  10/8/2016   28   \n4     PAIDOFF       1000     30       9/9/2016  10/8/2016   29   \n\n              education  Gender  \n0  High School or Below    male  \n1              Bechalor  female  \n2               college    male  \n3               college  female  \n4               college    male  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "df = df.iloc[:, 2:]\n\ndf.head()"
        }, 
        {
            "execution_count": 6, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 6, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "(346, 8)"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "df.shape"
        }, 
        {
            "source": "Let's check for NaN values", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 7, 
            "cell_type": "code", 
            "metadata": {
                "scrolled": true
            }, 
            "outputs": [
                {
                    "execution_count": 7, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "loan_status       0\nPrincipal         0\nterms             0\neffective_date    0\ndue_date          0\nage               0\neducation         0\nGender            0\ndtype: int64"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "df.isna().sum()\n\n# Great! No NaN values"
        }, 
        {
            "source": "### Convert to date time object ", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 8, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 8, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>2016-09-08</td>\n      <td>2016-09-22</td>\n      <td>27</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>28</td>\n      <td>college</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>29</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "  loan_status  Principal  terms effective_date   due_date  age  \\\n0     PAIDOFF       1000     30     2016-09-08 2016-10-07   45   \n1     PAIDOFF       1000     30     2016-09-08 2016-10-07   33   \n2     PAIDOFF       1000     15     2016-09-08 2016-09-22   27   \n3     PAIDOFF       1000     30     2016-09-09 2016-10-08   28   \n4     PAIDOFF       1000     30     2016-09-09 2016-10-08   29   \n\n              education  Gender  \n0  High School or Below    male  \n1              Bechalor  female  \n2               college    male  \n3               college  female  \n4               college    male  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "df['due_date'] = pd.to_datetime(df['due_date'])\ndf['effective_date'] = pd.to_datetime(df['effective_date'])\ndf.head()"
        }, 
        {
            "source": "# Data visualization and pre-processing\n\n", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "Let\u2019s see how many of each class is in our data set ", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 9, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 9, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "PAIDOFF       260\nCOLLECTION     86\nName: loan_status, dtype: int64"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "df['loan_status'].value_counts()"
        }, 
        {
            "source": "260 people have paid off the loan on time while 86 have gone into collection \n", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "Lets plot some columns to underestand data better:", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 10, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "# notice: installing seaborn might takes a few minutes\n# !conda install -c anaconda seaborn -y"
        }, 
        {
            "execution_count": 11, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "display_data", 
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAG4xJREFUeJzt3XucFOWd7/HPV5wVFaIioyKIMyKKqGTAWY3XJbCyqPF2jAbjUdx4DtFoXDbxeMt5aTa+1nghMclRibhyyCaKGrKgSxINUTmKiRfAEcELITrqKCAQN8YgBPB3/qiaSYM9zKV7pmu6v+/Xq15T9VTVU7+umWd+XU9XP6WIwMzMLGt2KHUAZmZm+ThBmZlZJjlBmZlZJjlBmZlZJjlBmZlZJjlBmZlZJjlBdRFJe0u6T9LrkhZJ+q2kM4tU92hJc4tRV3eQNF9SfanjsNIop7YgqVrSs5JekHR8Fx7nw66quydxguoCkgTMAZ6MiAMi4ghgAjCoRPHsWIrjmpVhWxgLvBoRIyPiqWLEZK1zguoaY4C/RMQPmwsi4s2I+D8AknpJulXS85KWSPpyWj46vdqYJelVSfemDRxJ49OyBcB/a65X0q6Spqd1vSDp9LT8Qkk/lfSfwK8KeTGSZkiaKumJ9F3w36XHfEXSjJztpkpaKGmZpH9ppa5x6TvoxWl8fQqJzTKvbNqCpDrgFuBkSQ2Sdm7t71lSo6Qb03ULJY2S9Kik30u6ON2mj6TH0n1fao43z3H/V875yduuylZEeCryBFwO3Lad9ZOA/53O7wQsBGqB0cAfSd5d7gD8FjgO6A28DQwFBDwIzE33vxH47+n87sByYFfgQqAJ6NdKDE8BDXmmv8+z7Qzg/vTYpwMfAIenMS4C6tLt+qU/ewHzgRHp8nygHugPPAnsmpZfBVxX6t+Xp66byrAtXAjcns63+vcMNAKXpPO3AUuAvkA18F5aviPwqZy6VgBKlz9Mf44DpqWvdQdgLnBCqX+v3TW566cbSLqDpHH9JSL+luSPboSkz6eb7EbS4P4CPBcRTel+DUAN8CHwRkT8Li3/CUnDJq3rNElXpMu9gcHp/LyI+EO+mCKio/3n/xkRIeklYHVEvJTGsiyNsQE4R9IkkoY3ABhO0jCbfSYtezp9M/w3JP94rEKUSVto1tbf88Ppz5eAPhHxJ+BPkjZI2h34M3CjpBOAj4GBwN7Aqpw6xqXTC+lyH5Lz82QnY+5RnKC6xjLgrOaFiLhUUn+Sd4eQvBv6akQ8mruTpNHAxpyiLfz1d9TaoIkCzoqI17ap6yiSBpB/J+kpknd027oiIn6dp7w5ro+3ifFjYEdJtcAVwN9GxPtp11/vPLHOi4hzW4vLyk45toXc423v73m7bQY4j+SK6oiI2CSpkfxt5tsRcdd24ihb/gyqazwO9JZ0SU7ZLjnzjwKXSKoCkHSQpF23U9+rQK2kIelyboN4FPhqTv/8yPYEGBHHR0Rdnml7DXJ7PkXyT+CPkvYGTsqzzTPAsZIOTGPdRdJBnTye9Qzl3BYK/XvejaS7b5OkzwL759nmUeBLOZ9tDZS0VweO0aM5QXWBSDqPzwD+TtIbkp4DfkTSRw3wb8DLwGJJS4G72M7VbERsIOnG+Hn6wfCbOatvAKqAJWldNxT79bRHRLxI0g2xDJgOPJ1nmzUkffgzJS0haeDDujFM62bl3BaK8Pd8L1AvaSHJ1dSreY7xK+A+4Ldp9/os8l/tlaXmD+TMzMwyxVdQZmaWSU5QZmaWSU5QZmaWSU5QZmaWSZlIUOPHjw+S7zZ48lQuU9G4fXgqs6ndMpGg1q5dW+oQzDLL7cMqVSYSlJmZ2bacoMzMLJOcoMzMLJM8WKyZlZVNmzbR1NTEhg0bSh1KRevduzeDBg2iqqqq03U4QZlZWWlqaqJv377U1NSQjhtr3SwiWLduHU1NTdTW1na6HnfxmVlZ2bBhA3vuuaeTUwlJYs899yz4KtYJyirG/gMGIKko0/4DBpT65dh2ODmVXjF+B+7is4rx1qpVNO07qCh1DXq3qSj1mFnrfAVlZmWtmFfO7b167tWrF3V1dRx22GGcffbZrF+/vmXd7NmzkcSrr/718U+NjY0cdthhAMyfP5/ddtuNkSNHcvDBB3PCCScwd+7creqfNm0aw4YNY9iwYRx55JEsWLCgZd3o0aM5+OCDqauro66ujlmzZm0VU/PU2NhYyGntFr6CMrOyVswrZ2jf1fPOO+9MQ0MDAOeddx4//OEP+drXvgbAzJkzOe6447j//vv55je/mXf/448/viUpNTQ0cMYZZ7DzzjszduxY5s6dy1133cWCBQvo378/ixcv5owzzuC5555jn332AeDee++lvr6+1Zh6ijavoCRNl/Re+oTK5rJvSnpHUkM6nZyz7hpJKyS9JukfuipwM7Oe4Pjjj2fFihUAfPjhhzz99NPcc8893H///e3av66ujuuuu47bb78dgJtvvplbb72V/v37AzBq1CgmTpzIHXfc0TUvoITa08U3Axifp/y2iKhLp18ASBoOTAAOTfe5U1KvYgVrZtaTbN68mV/+8pccfvjhAMyZM4fx48dz0EEH0a9fPxYvXtyuekaNGtXSJbhs2TKOOOKIrdbX19ezbNmyluXzzjuvpStv3bp1AHz00UctZWeeeWYxXl6Xa7OLLyKelFTTzvpOB+6PiI3AG5JWAEcCv+10hGZmPUxzMoDkCuqiiy4Cku69yZMnAzBhwgRmzpzJqFGj2qwvYvuDgEfEVnfNlUsXXyGfQV0m6QJgIfD1iHgfGAg8k7NNU1r2CZImAZMABg8eXEAYZuXH7aNny5cM1q1bx+OPP87SpUuRxJYtW5DELbfc0mZ9L7zwAocccggAw4cPZ9GiRYwZM6Zl/eLFixk+fHhxX0QGdPYuvqnAEKAOWAl8Jy3Pd+N73tQfEdMioj4i6qurqzsZhll5cvsoP7NmzeKCCy7gzTffpLGxkbfffpva2tqt7sDLZ8mSJdxwww1ceumlAFx55ZVcddVVLV13DQ0NzJgxg6985Std/hq6W6euoCJidfO8pLuB5nsgm4D9cjYdBLzb6ejMzAo0eJ99ivq9tcHpnXIdNXPmTK6++uqtys466yzuu+8+rrrqqq3Kn3rqKUaOHMn69evZa6+9+MEPfsDYsWMBOO2003jnnXc45phjkETfvn35yU9+woAy/PK42urbBEg/g5obEYelywMiYmU6/8/AURExQdKhwH0knzvtCzwGDI2ILdurv76+PhYuXFjI6zBrk6SiflG3jbZTtKEM3D465pVXXmnpDrPSauV30e620eYVlKSZwGigv6Qm4HpgtKQ6ku67RuDLABGxTNKDwMvAZuDStpKTmZlZPu25i+/cPMX3bGf7fwX+tZCgzMzMPNSRmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZW1fQcNLurjNvYd1L6RPVatWsWECRMYMmQIw4cP5+STT2b58uUsW7aMMWPGcNBBBzF06FBuuOGGlq8szJgxg8suu+wTddXU1LB27dqtymbMmEF1dfVWj9B4+eWXAVi+fDknn3wyBx54IIcccgjnnHMODzzwQMt2ffr0aXkkxwUXXMD8+fP53Oc+11L3nDlzGDFiBMOGDePwww9nzpw5LesuvPBCBg4cyMaNGwFYu3YtNTU1HfqdtJcft2FmZW3lO29z1HWPFK2+Z7+Vb+zsrUUEZ555JhMnTmwZtbyhoYHVq1dz4YUXMnXqVMaNG8f69es566yzuPPOO1tGiuiIL3zhCy2jnDfbsGEDp5xyCt/97nc59dRTAXjiiSeorq5uGX5p9OjRTJkypWW8vvnz57fs/+KLL3LFFVcwb948amtreeONNzjxxBM54IADGDFiBJA8W2r69OlccsklHY65I3wFZWZWZE888QRVVVVcfPHFLWV1dXUsX76cY489lnHjxgGwyy67cPvtt3PTTTcV7dj33XcfRx99dEtyAvjsZz/b8kDEtkyZMoVrr72W2tpaAGpra7nmmmu49dZbW7aZPHkyt912G5s3by5a3Pk4QZmZFdnSpUs/8UgMyP+ojCFDhvDhhx/ywQcfdPg4ud12dXV1fPTRR60eu73a8ziPwYMHc9xxx/HjH/+408dpD3fxmZl1k20fi5GrtfLtydfFV6h8MeYru/baaznttNM45ZRTinr8XL6CMjMrskMPPZRFixblLd92XMXXX3+dPn360Ldv3y49dkf23zbGfI/zOPDAA6mrq+PBBx/s9LHa4gRlZlZkY8aMYePGjdx9990tZc8//zxDhw5lwYIF/PrXvwaSBxtefvnlXHnllUU79he/+EV+85vf8POf/7yl7JFHHuGll15q1/5XXHEF3/72t2lsbASgsbGRG2+8ka9//euf2PYb3/gGU6ZMKUrc+biLz8zK2oCB+7XrzruO1NcWScyePZvJkydz00030bt3b2pqavje977HQw89xFe/+lUuvfRStmzZwvnnn7/VreUzZszY6rbuZ55JngE7YsQIdtghuaY455xzGDFiBA888MBWz5O68847OeaYY5g7dy6TJ09m8uTJVFVVMWLECL7//e+36/XV1dVx8803c+qpp7Jp0yaqqqq45ZZbWp4QnOvQQw9l1KhR7X50fUe163EbXc2PE7Du4MdtVAY/biM7Cn3cRptdfJKmS3pP0tKcslslvSppiaTZknZPy2skfSSpIZ1+2N5AzMzMcrXnM6gZwLbXx/OAwyJiBLAcuCZn3e8joi6dLsbMzKwT2kxQEfEk8Idtyn4VEc3f0HqG5NHuZmaZkIWPLipdMX4HxbiL70vAL3OWayW9IOn/STq+tZ0kTZK0UNLCNWvWFCEMs/Lh9tF5vXv3Zt26dU5SJRQRrFu3jt69exdUT0F38Un6Bsmj3e9Ni1YCgyNinaQjgDmSDo2IT3xFOiKmAdMg+RC4kDjMyo3bR+cNGjSIpqYmnNhLq3fv3gwaVFjnWqcTlKSJwOeAsZG+VYmIjcDGdH6RpN8DBwG+BcnMukVVVVXLOHLWs3Wqi0/SeOAq4LSIWJ9TXi2pVzp/ADAUeL0YgZqZWWVp8wpK0kxgNNBfUhNwPcldezsB89LxmZ5J79g7AfiWpM3AFuDiiPhD3orNzMy2o80EFRHn5im+p5Vtfwb8rNCgzMzMPBafmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllUrsSlKTpkt6TtDSnrJ+keZJ+l/7cIy2XpB9IWiFpiaRRXRW8mZmVr/ZeQc0Axm9TdjXwWEQMBR5LlwFOInmS7lBgEjC18DDNzKzStCtBRcSTwLZPxj0d+FE6/yPgjJzyf4/EM8DukgYUI1gzM6schXwGtXdErARIf+6Vlg8E3s7Zrikt24qkSZIWSlq4Zs2aAsIwKz9uH2Zdc5OE8pTFJwoipkVEfUTUV1dXd0EYZj2X24dZYQlqdXPXXfrzvbS8CdgvZ7tBwLsFHMfMzCpQIQnqYWBiOj8ReCin/IL0br7PAH9s7go0MzNrrx3bs5GkmcBooL+kJuB64CbgQUkXAW8BZ6eb/wI4GVgBrAf+scgxm5lZBWhXgoqIc1tZNTbPtgFcWkhQZmZmHknCzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyqV2jmecj6WDggZyiA4DrgN2B/wk0P6f62oj4RacjNDOzitTpBBURrwF1AJJ6Ae8As0me/3RbREwpSoRmZlaRitXFNxb4fUS8WaT6zMyswhUrQU0AZuYsXyZpiaTpkvbIt4OkSZIWSlq4Zs2afJuYVSy3D7MiJChJfwOcBvw0LZoKDCHp/lsJfCfffhExLSLqI6K+urq60DDMyorbh1lxrqBOAhZHxGqAiFgdEVsi4mPgbuDIIhzDzMwqTDES1LnkdO9JGpCz7kxgaRGOYWZmFabTd/EBSNoFOBH4ck7xLZLqgAAat1lnZmbWLgUlqIhYD+y5Tdn5BUVkZmaGR5IwM7OMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMKug2c7OeRL2qGPRuU9HqMrOu5QRlFSO2bOKo6x4pSl3Pfmt8Ueoxs9a5i8/MzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDKp4NvMJTUCfwK2AJsjol5SP+ABoIbkmVDnRMT7hR7LzMwqR7GuoD4bEXURUZ8uXw08FhFDgcfSZasw+w8YgKSCp/0HDGj7YGZWdrrqi7qnA6PT+R8B84GruuhYllFvrVpF076DCq6nWKM/mFnPUowrqAB+JWmRpElp2d4RsRIg/bnXtjtJmiRpoaSFa9asKUIYZuXD7cOsOAnq2IgYBZwEXCrphPbsFBHTIqI+Iuqrq6uLEIZZ+XD7MCtCgoqId9Of7wGzgSOB1ZIGAKQ/3yv0OGZmVlkKSlCSdpXUt3keGAcsBR4GJqabTQQeKuQ4ZmZWeQq9SWJvYLak5rrui4hHJD0PPCjpIuAt4OwCj2NmZhWmoAQVEa8Dn85Tvg4YW0jdZmZW2TyShJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZfIJ2F31RF0zM+tBsvgEbF9BmZlZJnU6QUnaT9ITkl6RtEzSP6Xl35T0jqSGdDq5eOGamVmlKKSLbzPw9YhYnD60cJGkeem62yJiSuHhmZlZpep0goqIlcDKdP5Pkl4BBhYrMDMzq2xF+QxKUg0wEng2LbpM0hJJ0yXt0co+kyQtlLRwzZo1xQjDrGy4fZgVIUFJ6gP8DJgcER8AU4EhQB3JFdZ38u0XEdMioj4i6qurqwsNw6ysuH2YFZigJFWRJKd7I+I/ACJidURsiYiPgbuBIwsP08zMKk0hd/EJuAd4JSK+m1Oe+y2tM4GlnQ/PzMwqVSF38R0LnA+8JKkhLbsWOFdSHRBAI/DlgiI0M7OKVMhdfAsA5Vn1i86HY2ZmlvBIEmZmlkkei8+6jHpVFWVcLvWqKkI0ZtbTOEFZl4ktmzjqukcKrufZb40vQjRm1tO4i8/MzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMrJtl8fHqWeS7+MzMulkWH6+eRb6CMjOzTHKCMjOzTHIXn5mZZXLkFycoMzPL5Mgv7uIzM7NM6rIEJWm8pNckrZB0daH1+bZMM7PK0iVdfJJ6AXcAJwJNwPOSHo6Ilztbp2/LNDOrLF31GdSRwIqIeB1A0v3A6UCnE1TW7D9gAG+tWlVwPYP32Yc3V64sQkTlTcr3bEzLIreNthXrhoQdelWVddtQRBS/UunzwPiI+B/p8vnAURFxWc42k4BJ6eLBwGtFD6T9+gNrS3j8Qjj20mgr9rUR0elPizPUPsr5d5Rl5Rx7u9tGV11B5UvpW2XCiJgGTOui43eIpIURUV/qODrDsZdGV8eelfbh31FpOPZEV90k0QTsl7M8CHi3i45lZmZlqKsS1PPAUEm1kv4GmAA83EXHMjOzMtQlXXwRsVnSZcCjQC9gekQs64pjFUnJu1IK4NhLoyfH3hE9+XU69tIoWuxdcpOEmZlZoTyShJmZZZITlJmZZVLFJChJvSS9IGluulwr6VlJv5P0QHozB5J2SpdXpOtrShz37pJmSXpV0iuSjpbUT9K8NPZ5kvZIt5WkH6SxL5E0qsSx/7OkZZKWSpopqXdWz7uk6ZLek7Q0p6zD51nSxHT730ma2J2vobPcNkoSu9tGO1RMggL+CXglZ/lm4LaIGAq8D1yUll8EvB8RBwK3pduV0veBRyJiGPBpktdwNfBYGvtj6TLAScDQdJoETO3+cBOSBgKXA/URcRjJzTITyO55nwFs++XBDp1nSf2A64GjSEZTub654Wac20Y3ctvoQNuIiLKfSL6H9RgwBphL8kXitcCO6fqjgUfT+UeBo9P5HdPtVKK4PwW8se3xSUYVGJDODwBeS+fvAs7Nt10JYh8IvA30S8/jXOAfsnzegRpgaWfPM3AucFdO+VbbZXFy23DbaGfMJWkblXIF9T3gSuDjdHlP4L8iYnO63ETyRwN//eMhXf/HdPtSOABYA/zftAvm3yTtCuwdESvTGFcCe6Xbt8Seyn1d3Soi3gGmAG8BK0nO4yJ6xnlv1tHznJnz3wFuG93MbWOr8u0q+wQl6XPAexGxKLc4z6bRjnXdbUdgFDA1IkYCf+avl9L5ZCb29PL9dKAW2BfYleTyf1tZPO9taS3WnvQa3DbcNrpCUdtG2Sco4FjgNEmNwP0kXRnfA3aX1PxF5dyhmFqGaUrX7wb8oTsDztEENEXEs+nyLJJGuVrSAID053s522dliKm/B96IiDURsQn4D+AYesZ5b9bR85yl898ebhul4bbRzvNf9gkqIq6JiEERUUPyQeTjEXEe8ATw+XSzicBD6fzD6TLp+scj7TTtbhGxCnhb0sFp0ViSR5bkxrht7Bekd9J8Bvhj82V4CbwFfEbSLpLEX2PP/HnP0dHz/CgwTtIe6bvkcWlZJrltuG0UoHvaRik+JCzVBIwG5qbzBwDPASuAnwI7peW90+UV6foDShxzHbAQWALMAfYg6X9+DPhd+rNfuq1IHhT5e+AlkruEShn7vwCvAkuBHwM7ZfW8AzNJPg/YRPJu76LOnGfgS+lrWAH8Y6n/5jvw+t02ujd2t412HNtDHZmZWSaVfRefmZn1TE5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QGSZpi6SGdMTjn0rapZXtfiFp907Uv6+kWQXE1yipf2f3N+sst43K4NvMM0zShxHRJ52/F1gUEd/NWS+S3+HHrdXRxfE1knzPYW0pjm+Vy22jMvgKqud4CjhQUo2SZ9/cCSwG9mt+t5az7m4lz5r5laSdASQdKOnXkl6UtFjSkHT7pen6CyU9JOkRSa9Jur75wJLmSFqU1jmpJK/erHVuG2XKCaoHSMffOonkm9kABwP/HhEjI+LNbTYfCtwREYcC/wWclZbfm5Z/mmTcr3zDvBwJnEfyDf2zJdWn5V+KiCOAeuBySaUeSdkMcNsod05Q2bazpAaS4VzeAu5Jy9+MiGda2eeNiGhI5xcBNZL6AgMjYjZARGyIiPV59p0XEesi4iOSASyPS8svl/Qi8AzJgI9DC35lZoVx26gAO7a9iZXQRxFRl1uQdK3z5+3sszFnfguwM/mHus9n2w8kQ9JoktGXj46I9ZLmk4wNZlZKbhsVwFdQFSAiPgCaJJ0BIGmnVu56OlFSv7Rv/gzgaZKh/d9PG+Aw4DPdFrhZF3PbyDYnqMpxPkl3xBLgN8A+ebZZQDKycgPws4hYCDwC7JjudwNJV4ZZOXHbyCjfZm5AcqcSyW2xl5U6FrMscdsoHV9BmZlZJvkKyszMMslXUGZmlklOUGZmlklOUGZmlklOUGZmlklOUGZmlkn/H+LDZoiBEQ8dAAAAAElFTkSuQmCC\n", 
                        "text/plain": "<matplotlib.figure.Figure at 0x7f6623c30f60>"
                    }, 
                    "metadata": {}
                }
            ], 
            "source": "import seaborn as sns\n\nbins = np.linspace(df.Principal.min(), df.Principal.max(), 10)\ng = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\ng.map(plt.hist, 'Principal', bins=bins, ec=\"k\")\n\ng.axes[-1].legend()\nplt.show()"
        }, 
        {
            "execution_count": 12, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "output_type": "display_data", 
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAGfZJREFUeJzt3XuQVOW57/HvTxgdFbygo4yMwKgoopIBZ3tDDYJy2N49XuKOR7GOJx4Naqjo8ZZTVrLdZbyVmhwvkUQLK1HUmA26SUWDCidi4gVwRBBv0UFHQS7RKAchgs/5o9fMHqBhembWTK/u+X2qVnWvt1e/61lMvzy93vX2uxQRmJmZZc02xQ7AzMwsHycoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCeolEjaU9Ijkt6XNE/SXySdkVLdoyXNSKOu7iBptqT6YsdhxVdO7UJSlaSXJb0m6Zgu3M/qrqq71DhBpUCSgOnAnyJin4g4FDgXqClSPL2LsV+z1sqwXYwF3oqIERHxQhox2dY5QaVjDPCPiPhFc0FELImI/wMgqZek2yS9KmmBpP+ZlI9OzjaekPSWpIeTRo2k8UnZHOC/NtcraUdJDyZ1vSbptKT8Qkm/lfQfwB87czCSpki6T9Ks5Jvvt5N9LpY0pdV290maK2mRpJ9soa5xybfm+Ul8fToTm5WUsmkXkuqAW4ETJTVI2n5Ln21JjZJuSl6bK2mkpGck/VXSJck2fSQ9l7z3jeZ48+z3f7X698nbxspaRHjp5AJcAdy5ldcvBv538nw7YC5QC4wG/k7uG+U2wF+Ao4FK4CNgCCDgcWBG8v6bgP+WPN8FeAfYEbgQaAL6bSGGF4CGPMvxebadAjya7Ps04AvgkCTGeUBdsl2/5LEXMBsYnqzPBuqB3YE/ATsm5dcANxT77+Wle5YybBcXAncnz7f42QYagUuT53cCC4C+QBWwPCnvDezUqq73ACXrq5PHccDk5Fi3AWYAxxb779qdi7uCuoCke8g1qH9ExD+R+6ANl3RWssnO5BrZP4BXIqIpeV8DMBhYDXwQEe8m5b8h15hJ6jpV0lXJeiUwMHk+MyL+li+miGhvn/l/RERIegP4NCLeSGJZlMTYAJwj6WJyja0aGEauMTY7Iil7MfkCvC25/2ysByqTdtGsrc/2U8njG0CfiPgS+FLSWkm7AP8PuEnSscA3wABgT2BZqzrGJctryXofcv8+f+pgzCXHCSodi4Azm1ciYqKk3cl9I4TcN6DLI+KZ1m+SNBpY16poA//5N9nSJIkCzoyItzep63ByH/r8b5JeIPctblNXRcSzecqb4/pmkxi/AXpLqgWuAv4pIj5Luv4q88Q6MyL+ZUtxWVkrx3bRen9b+2xvtf0A55E7ozo0Ir6W1Ej+9vPTiLh/K3GUNV+DSsfzQKWkS1uV7dDq+TPApZIqACTtL2nHrdT3FlArad9kvXUjeAa4vFWf/IhCAoyIYyKiLs+ytUa4NTuRa/h/l7Qn8M95tnkJGCVpvyTWHSTt38H9Wekp53bR2c/2zuS6+76WdBwwKM82zwD/vdW1rQGS9mjHPkqeE1QKItdhfDrwbUkfSHoFeIhcvzTAr4A3gfmSFgL3s5Wz14hYS67r4vfJxeAlrV6+EagAFiR13Zj28RQiIl4n1/WwCHgQeDHPNivI9dtPlbSAXKMe2o1hWhGVc7tI4bP9MFAvaS65s6m38uzjj8AjwF+SrvYnyH+2V7aaL8qZmZllis+gzMwsk5ygzMwsk5ygzMwsk5ygzMwsk7o1QY0fPz7I/Y7Bi5dyXTrN7cRLD1gK0q0JauXKld25O7OS5HZiluMuPjMzyyQnKDMzyyQnKDMzyyRPFmtmZefrr7+mqamJtWvXFjuUHq2yspKamhoqKio69H4nKDMrO01NTfTt25fBgweTzB9r3SwiWLVqFU1NTdTW1naoDnfxmVnZWbt2LbvttpuTUxFJYrfdduvUWawTVDcaVF2NpFSWQdXVxT4cs0xzciq+zv4N3MXXjT5ctoymvWpSqavmk6ZU6jEzyyqfQZlZ2Uuz96LQHoxevXpRV1fHwQcfzNlnn82aNWtaXps2bRqSeOut/7wNVGNjIwcffDAAs2fPZuedd2bEiBEccMABHHvsscyYMWOj+idPnszQoUMZOnQohx12GHPmzGl5bfTo0RxwwAHU1dVRV1fHE088sVFMzUtjY2Nn/lm7nM+gzKzspdl7AYX1YGy//fY0NDQAcN555/GLX/yCH/7whwBMnTqVo48+mkcffZQf//jHed9/zDHHtCSlhoYGTj/9dLbffnvGjh3LjBkzuP/++5kzZw6777478+fP5/TTT+eVV16hf//+ADz88MPU19dvMaZS4DMoM7Mudswxx/Dee+8BsHr1al588UUeeOABHn300YLeX1dXxw033MDdd98NwC233MJtt93G7rvvDsDIkSOZMGEC99xzT9ccQJE4QZmZdaH169fzhz/8gUMOOQSA6dOnM378ePbff3/69evH/PnzC6pn5MiRLV2CixYt4tBDD93o9fr6ehYtWtSyft5557V05a1atQqAr776qqXsjDPOSOPwupS7+MzMukBzMoDcGdRFF10E5Lr3Jk2aBMC5557L1KlTGTlyZJv1RWx9EvCI2GjUXDl08RWUoCQ1Al8CG4D1EVEvqR/wGDAYaATOiYjPuiZMM7PSki8ZrFq1iueff56FCxciiQ0bNiCJW2+9tc36XnvtNQ488EAAhg0bxrx58xgzZkzL6/Pnz2fYsGHpHkSRtaeL77iIqIuI5pR8LfBcRAwBnkvWzcxsC5544gkuuOAClixZQmNjIx999BG1tbUbjcDLZ8GCBdx4441MnDgRgKuvvpprrrmmpeuuoaGBKVOm8P3vf7/Lj6E7daaL7zRgdPL8IWA2cE0n4zEzS93A/v1T/e3gwGSkXHtNnTqVa6/d+Lv8mWeeySOPPMI112z83+cLL7zAiBEjWLNmDXvssQc///nPGTt2LACnnnoqH3/8MUcddRSS6Nu3L7/5zW+oLrMf8Kutfk0ASR8An5G7E+L9ETFZ0ucRsUurbT6LiF3zvPdi4GKAgQMHHrpkyZLUgi81klL9oW4hfzvrdh366bzbSboWL17c0h1mxbWFv0VB7aTQLr5RETES+GdgoqRjCw0uIiZHRH1E1FdVVRX6NrMexe3EbHMFJaiI+CR5XA5MAw4DPpVUDZA8Lu+qIM3MrOdpM0FJ2lFS3+bnwDhgIfAUMCHZbALwZFcFaWZmPU8hgyT2BKYl4+t7A49ExNOSXgUel3QR8CFwdteFaWZmPU2bCSoi3ge+lad8FTC2K4IyMzPzVEdmZpZJTlBmVvb2qhmY6u029qoZWNB+ly1bxrnnnsu+++7LsGHDOPHEE3nnnXdYtGgRY8aMYf/992fIkCHceOONLT8bmTJlCpdddtlmdQ0ePJiVK1duVDZlyhSqqqo2uoXGm2++CcA777zDiSeeyH777ceBBx7IOeecw2OPPdayXZ8+fVpuyXHBBRcwe/ZsTj755Ja6p0+fzvDhwxk6dCiHHHII06dPb3ntwgsvZMCAAaxbtw6AlStXMnjw4Hb9TQrhufgKMKi6mg+XLSt2GGbWQUs//ojDb3g6tfpe/tfxbW4TEZxxxhlMmDChZdbyhoYGPv30Uy688ELuu+8+xo0bx5o1azjzzDO59957W2aKaI/vfOc7LbOcN1u7di0nnXQSd9xxB6eccgoAs2bNoqqqqmX6pdGjR3P77be3zNc3e/bslve//vrrXHXVVcycOZPa2lo++OADTjjhBPbZZx+GDx8O5O4t9eCDD3LppZe2O+ZCOUEVIK17yfguuGY9x6xZs6ioqOCSSy5pKaurq+OBBx5g1KhRjBs3DoAddtiBu+++m9GjR3coQeXzyCOPcOSRR7YkJ4Djjjuu4PfffvvtXH/99dTW1gJQW1vLddddx2233cavf/1rACZNmsSdd97J9773vVRizsddfGZmXWDhwoWb3RID8t8qY99992X16tV88cUX7d5P6267uro6vvrqqy3uu1CF3M5j4MCBHH300S0Jqyv4DMrMrBtteluM1rZUvjX5uvg6K1+M+cquv/56Tj31VE466aRU99/MZ1BmZl3goIMOYt68eXnL586du1HZ+++/T58+fejbt2+X7rs97980xny389hvv/2oq6vj8ccf7/C+tsYJysysC4wZM4Z169bxy1/+sqXs1VdfZciQIcyZM4dnn30WyN3Y8IorruDqq69Obd/f/e53+fOf/8zvf//7lrKnn36aN954o6D3X3XVVfz0pz+lsbERgMbGRm666SauvPLKzbb90Y9+xO23355K3JtyF5+Zlb3qAXsXNPKuPfW1RRLTpk1j0qRJ3HzzzVRWVjJ48GDuuusunnzySS6//HImTpzIhg0bOP/88zcaWj5lypSNhnW/9NJLAAwfPpxttsmdV5xzzjkMHz6cxx57bKP7Sd17770cddRRzJgxg0mTJjFp0iQqKioYPnw4P/vZzwo6vrq6Om655RZOOeUUvv76ayoqKrj11ltb7hDc2kEHHcTIkSMLvnV9exR0u4201NfXx6anjaUgrdtk1HzS5NttlL8O3W6jtVJtJ1ni221kR3fcbsPMzKxbOUGZmVkmOUGZWVlyF3jxdfZv4ARlZmWnsrKSVatWOUkVUUSwatUqKisrO1yHR/GZWdmpqamhqamJFStWFDuUHq2yspKamo4PDHOCKlHb0bFfneczsH9/lixdmkpdZllQUVHRMo+clS4nqBK1DlIdsm5mljUFX4OS1EvSa5JmJOu1kl6W9K6kxyRt23VhmplZT9OeQRI/ABa3Wr8FuDMihgCfARelGZiZmfVsBSUoSTXAScCvknUBY4Ankk0eAk7vigDNzKxnKvQM6i7gauCbZH034POIWJ+sNwED8r1R0sWS5kqa6xE1Zvm5nZhtrs0EJelkYHlEtJ67Pd/wsbw/OIiIyRFRHxH1VVVVHQzTrLy5nZhtrpBRfKOAUyWdCFQCO5E7o9pFUu/kLKoG+KTrwjQzs56mzTOoiLguImoiYjBwLvB8RJwHzALOSjabADzZZVGamVmP05mpjq4BfijpPXLXpB5IJyQzM7N2/lA3ImYDs5Pn7wOHpR+SmZmZJ4s1M7OMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMajNBSaqU9Iqk1yUtkvSTpLxW0suS3pX0mKRtuz5cMzPrKQo5g1oHjImIbwF1wHhJRwC3AHdGxBDgM+CirgvTzMx6mjYTVOSsTlYrkiWAMcATSflDwOldEqGZmfVIBV2DktRLUgOwHJgJ/BX4PCLWJ5s0AQO28N6LJc2VNHfFihVpxGxWdtxOzDZXUIKKiA0RUQfUAIcBB+bbbAvvnRwR9RFRX1VV1fFIzcqY24nZ5to1ii8iPgdmA0cAu0jqnbxUA3ySbmhmZtaTFTKKr0rSLsnz7YHjgcXALOCsZLMJwJNdFaSZmfU8vdvehGrgIUm9yCW0xyNihqQ3gUcl/RvwGvBAF8ZpZmY9TJsJKiIWACPylL9P7nqUmZlZ6jyThJmZZZITlJmZZZITlJmZZZITlJmZZVLZJqhB1dVISmUxM7PuV8gw85L04bJlNO1Vk0pdNZ80pVKPmZkVrmzPoMzMrLQ5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSa1maAk7S1plqTFkhZJ+kFS3k/STEnvJo+7dn24ZmbWUxRyBrUeuDIiDgSOACZKGgZcCzwXEUOA55J1MzOzVLSZoCJiaUTMT55/CSwGBgCnAQ8lmz0EnN5VQZqZWc/TrmtQkgYDI4CXgT0jYinkkhiwxxbec7GkuZLmrlixonPRmpUptxOzzRWcoCT1AX4HTIqILwp9X0RMjoj6iKivqqrqSIxmZc/txGxzBSUoSRXkktPDEfHvSfGnkqqT16uB5V0TopmZ9USFjOIT8ACwOCLuaPXSU8CE5PkE4Mn0w7PusB20edv7QpZB1dXFPhQzKyOF3PJ9FHA+8IakhqTseuBm4HFJFwEfAmd3TYjW1dYBTXvVdLqemk+aOh+MmVmizQQVEXMAbeHlsemGk03qVZHKf77qvW1q/4mrV0Uq9ZiZZVUhZ1A9Xmz4msNveLrT9bz8r+NTqae5LjOzcuapjszMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJPKdiaJtKYnMjOz4ijbBJXW9ETgaYXMzIrBXXxmZpZJTlBmZpZJTlBmZpZJZXsNqtylOQjE95ayrBlUXc2Hy5Z1up7tt+nFV99sSCEiGNi/P0uWLk2lLiuME1SJ8iAQK2cfLluW2l2e06inuS7rXm128Ul6UNJySQtblfWTNFPSu8njrl0bppmZ9TSFXIOaAmz6Ffta4LmIGAI8l6xbD7cdICmVZVB1dbEPx8yKrM0uvoj4k6TBmxSfBoxOnj8EzAauSTEuK0HrwN0pZpaajo7i2zMilgIkj3tsaUNJF0uaK2nuihUrOrg7s/JWDu1kUHV1amfQZtANgyQiYjIwGaC+vj66en9mpagc2klaAxvAZ9CW09EzqE8lVQMkj8vTC8nMzKzjCeopYELyfALwZDrhmJmZ5RQyzHwq8BfgAElNki4CbgZOkPQucEKybmZmlppCRvH9yxZeGptyLGZmZi0yNRefRwGZmVmzTE115FFAZmbWLFMJyoojrYlnPemsmaXJCcpSm3jWk86aWZoydQ3KzMysmROUmZllkhOUmZllkhOUmZllkhOUZZLvLdU9/NtDyzKP4rNM8r2luod/e2hZ5gRlqUnr91TNdZlZz+YEZalJ6/dU4N9UmZmvQZmZWUb5DMoyKc3uwm16VaRyEX9g//4sWbo0hYjKU6pdvL239fRbBRhUXc2Hy5alUlcWP99OUJZJaXcXpjEQwIMAti7tv5mn32pbuQ9ycRefmZllUqbOoNLsIjAzs9KWqQTlUWBmZtasUwlK0njgZ0Av4FcRcXMqUZmlqBzvd5XmxXErTFqDbQC26V3BN+u/TqWuctbhBCWpF3APcALQBLwq6amIeDOt4MzSUI73u0rr4ri71Av3jQfudLvODJI4DHgvIt6PiH8AjwKnpROWmZn1dIqIjr1ROgsYHxH/I1k/Hzg8Ii7bZLuLgYuT1QOAtzsebovdgZUp1JMFPpZs6uixrIyIdp9qdVE7Af9NsqqnH0tB7aQz16DydcZulu0iYjIwuRP72XzH0tyIqE+zzmLxsWRTdx9LV7QT8N8kq3wshelMF18TsHer9Rrgk86FY2ZmltOZBPUqMERSraRtgXOBp9IJy8zMeroOd/FFxHpJlwHPkBtm/mBELEotsq1LvSukiHws2VQux1IuxwE+lqzqsmPp8CAJMzOzruS5+MzMLJOcoMzMLJMyn6Ak7S1plqTFkhZJ+kFS3k/STEnvJo+7FjvWtkiqlPSKpNeTY/lJUl4r6eXkWB5LBp1knqRekl6TNCNZL8njAJDUKOkNSQ2S5iZlJfMZczvJtnJpK93dTjKfoID1wJURcSBwBDBR0jDgWuC5iBgCPJesZ906YExEfAuoA8ZLOgK4BbgzOZbPgIuKGGN7/ABY3Gq9VI+j2XERUdfqNx2l9BlzO8m2cmor3ddOIqKkFuBJcvP/vQ1UJ2XVwNvFjq2dx7EDMB84nNyvsHsn5UcCzxQ7vgLir0k+jGOAGeR+uF1yx9HqeBqB3TcpK9nPmNtJdpZyaivd3U5K4QyqhaTBwAjgZWDPiFgKkDzuUbzICpec6jcAy4GZwF+BzyNifbJJEzCgWPG1w13A1cA3yfpulOZxNAvgj5LmJdMOQel+xgbjdpIl5dRWurWdZOp+UFsjqQ/wO2BSRHyR1rT33S0iNgB1knYBpgEH5tuse6NqH0knA8sjYp6k0c3FeTbN9HFsYlREfCJpD2CmpLeKHVBHuJ1kSxm2lW5tJyWRoCRVkGt0D0fEvyfFn0qqjoilkqrJfdMqGRHxuaTZ5K4X7CKpd/KNqhSmjBoFnCrpRKAS2Inct8RSO44WEfFJ8rhc0jRys/WX1GfM7SSTyqqtdHc7yXwXn3JfAR8AFkfEHa1eegqYkDyfQK7PPdMkVSXfCJG0PXA8uQuns4Czks0yfywRcV1E1ETEYHJTXD0fEedRYsfRTNKOkvo2PwfGAQspoc+Y20k2lVNbKUo7KfZFtwIuyh1N7vR3AdCQLCeS68d9Dng3eexX7FgLOJbhwGvJsSwEbkjK9wFeAd4DfgtsV+xY23FMo4EZpXwcSdyvJ8si4EdJecl8xtxOsr+UelspRjvxVEdmZpZJme/iMzOznskJyszMMskJyszMMskJyszMMskJyszMMskJyszMMskJyszMMskJqsRJmp5M3LioefJGSRdJekfSbEm/lHR3Ul4l6XeSXk2WUcWN3qx7uJ2UJv9Qt8RJ6hcRf0umhHkV+C/Ai8BI4EvgeeD1iLhM0iPAvRExR9JAclP855uE06ysuJ2UppKYLNa26gpJZyTP9wbOB/5vRPwNQNJvgf2T148HhrWa4XonSX0j4svuDNisCNxOSpATVAlLpu8/HjgyItYksz6/Tf5bE0CuS/fIiPiqeyI0Kz63k9Lla1ClbWfgs6TRDSV3S4IdgG9L2lVSb+DMVtv/EbiseUVSXbdGa1YcbiclygmqtD0N9Ja0ALgReAn4GLiJ3N1UnwXeBP6ebH8FUC9pgaQ3gUu6P2Szbud2UqI8SKIMSeoTEauTb4bTgAcjYlqx4zLLEreT7PMZVHn6saQGcvfS+QCYXuR4zLLI7STjfAZlZmaZ5DMoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLpP8PlTlGZbaTvVAAAAAASUVORK5CYII=\n", 
                        "text/plain": "<matplotlib.figure.Figure at 0x7f661c1ab4a8>"
                    }, 
                    "metadata": {}
                }
            ], 
            "source": "bins = np.linspace(df.age.min(), df.age.max(), 10)\ng = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\ng.map(plt.hist, 'age', bins=bins, ec=\"k\")\n\ng.axes[-1].legend()\nplt.show()"
        }, 
        {
            "source": "# Pre-processing:  Feature selection/extraction", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "### Lets look at the day of the week people get the loan ", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 13, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "output_type": "display_data", 
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAGepJREFUeJzt3XmcVPW55/HPV2gvIriC2tIBWkQQldtgR+OCQUh4EdzwuoTEKGTMdTQuYQyDSzImN84YF8YlcSVq8EbEhUTMJTcaVIjgztKCiCFebbEVFJgYYxQFfeaPOt1poKGr6VPU6erv+/WqV1edOud3ntNdTz91fnXq91NEYGZmljU7FDsAMzOzprhAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlApUTS3pLuk/S6pAWSnpV0ckptD5U0M422tgdJcyRVFzsOK75SygtJ3SU9L2mRpCEF3M+HhWq7rXGBSoEkATOApyJiv4g4FBgDVBQpno7F2K9ZYyWYF8OBVyNiUETMTSMm2zoXqHQMAz6NiNvrF0TEmxHxcwBJHSRdJ+lFSYsl/fdk+dDkbGO6pFclTU2SGkkjk2XzgH+pb1fSzpLuTtpaJOmkZPk4SQ9J+g/gD605GElTJN0maXbyzvfLyT6XSZrSaL3bJM2XtFTSv22hrRHJu+aFSXxdWhObtSklkxeSqoBrgVGSaiTttKXXtqRaSVclz82XNFjSY5L+S9K5yTpdJD2RbLukPt4m9vs/G/1+msyxkhYRvrXyBlwE3LCV588Bfpjc/ydgPlAJDAX+Su4d5Q7As8DRQCfgLaAvIOBBYGay/VXAt5L7uwHLgZ2BcUAdsMcWYpgL1DRx+0oT604B7k/2fRLwAXBIEuMCoCpZb4/kZwdgDjAweTwHqAa6AU8BOyfLLwGuKPbfy7ftcyvBvBgH3Jzc3+JrG6gFzkvu3wAsBroC3YH3kuUdgV0atfUaoOTxh8nPEcDk5Fh3AGYCxxT777o9b+4KKgBJt5BLqE8j4ovkXmgDJZ2arLIruST7FHghIuqS7WqA3sCHwBsR8edk+b3kkpmkrRMlTUgedwJ6JvdnRcT/ayqmiGhpn/l/RERIWgK8GxFLkliWJjHWAKdLOodcspUDA8glY70vJcueTt4A70jun421QyWSF/Wae23/Nvm5BOgSEX8D/iZpnaTdgL8DV0k6Bvgc6AHsDaxq1MaI5LYoedyF3O/nqW2Muc1xgUrHUuCU+gcRcb6kbuTeEULuHdCFEfFY440kDQU+abToM/7xN9nSIIkCTomIP23S1uHkXvRNbyTNJfcublMTIuLxJpbXx/X5JjF+DnSUVAlMAL4YEX9Juv46NRHrrIj4xpbispJWinnReH9be21vNX+AM8idUR0aEesl1dJ0/vw0Iu7YShwlzZ9BpeNJoJOk8xot69zo/mPAeZLKACQdIGnnrbT3KlApqU/yuHESPAZc2KhPflA+AUbEkIioauK2tSTcml3IJf5fJe0NfK2JdZ4DjpK0fxJrZ0kHbOP+rO0p5bxo7Wt7V3LdfeslHQv0amKdx4D/1uizrR6S9mrBPto8F6gURK7DeDTwZUlvSHoBuIdcvzTAncArwEJJLwN3sJWz14hYR67r4nfJh8FvNnr6SqAMWJy0dWXax5OPiHiJXNfDUuBu4Okm1llNrt9+mqTF5JK6/3YM04qolPMihdf2VKBa0nxyZ1OvNrGPPwD3Ac8mXe3Tafpsr2TVfyhnZmaWKT6DMjOzTHKBMjOzTHKBMjOzTHKBMjOzTNquBWrkyJFB7nsMvvlWqrdWc5741g5uedmuBWrNmjXbc3dmbZLzxCzHXXxmZpZJLlBmZpZJLlBmZpZJHizWzErO+vXrqaurY926dcUOpV3r1KkTFRUVlJWVbdP2LlBmVnLq6uro2rUrvXv3Jhk/1raziGDt2rXU1dVRWVm5TW24i8/MSs66devYc889XZyKSBJ77rlnq85iXaCs5PUqL0dSq2+9ysuLfSjWAi5Oxdfav4G7+KzkrVi1irp9K1rdTsU7dSlEY2b58hmUmZW8tM6iW3I23aFDB6qqqjj44IM57bTT+Oijjxqee/jhh5HEq6/+Yxqo2tpaDj74YADmzJnDrrvuyqBBg+jXrx/HHHMMM2fO3Kj9yZMn079/f/r3789hhx3GvHnzGp4bOnQo/fr1o6qqiqqqKqZPn75RTPW32tra1vxaCy6vMyhJ/wP4DrkhKpYA3wbKgfuBPYCFwJkR8WmB4jQz22ZpnUXXy+dseqeddqKmpgaAM844g9tvv52LL74YgGnTpnH00Udz//338+Mf/7jJ7YcMGdJQlGpqahg9ejQ77bQTw4cPZ+bMmdxxxx3MmzePbt26sXDhQkaPHs0LL7zAPvvsA8DUqVOprq7eYkxtQbNnUJJ6ABcB1RFxMNABGANcA9wQEX2BvwBnFzJQM7O2asiQIbz22msAfPjhhzz99NPcdddd3H///XltX1VVxRVXXMHNN98MwDXXXMN1111Ht27dABg8eDBjx47llltuKcwBFEm+XXwdgZ0kdQQ6AyuBYeSmIIbcNM6j0w/PzKxt27BhA7///e855JBDAJgxYwYjR47kgAMOYI899mDhwoV5tTN48OCGLsGlS5dy6KGHbvR8dXU1S5cubXh8xhlnNHTlrV27FoCPP/64YdnJJ5+cxuEVVLNdfBHxtqRJwArgY+APwALg/YjYkKxWB/RoantJ5wDnAPTs2TONmM1KjvOk9NQXA8idQZ19dq6Tadq0aYwfPx6AMWPGMG3aNAYPHtxsexFbHwQ8Ija6aq4UuviaLVCSdgdOAiqB94GHgK81sWqTv72ImAxMBqiurs57mHWz9sR5UnqaKgZr167lySef5OWXX0YSn332GZK49tprm21v0aJFHHjggQAMGDCABQsWMGzYsIbnFy5cyIABA9I9iCLLp4vvK8AbEbE6ItYDvwGOBHZLuvwAKoB3ChSjmVlJmD59OmeddRZvvvkmtbW1vPXWW1RWVm50BV5TFi9ezJVXXsn5558PwMSJE7nkkksauu5qamqYMmUK3/3udwt+DNtTPlfxrQC+JKkzuS6+4cB8YDZwKrkr+cYCjxQqSDOz1ui5zz6pfo+tZ3KlXEtNmzaNSy+9dKNlp5xyCvfddx+XXHLJRsvnzp3LoEGD+Oijj9hrr7342c9+xvDhwwE48cQTefvttznyyCORRNeuXbn33nspL7Evk6u5fk0ASf8GfB3YACwid8l5D/5xmfki4FsR8cnW2qmuro758+e3NmazFpGU2hd188iXVg9f4DxpvWXLljV0h1lxbeFvkVee5PU9qIj4EfCjTRa/DhyWz/ZmZmYt5ZEkzMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzKzk7VvRM9XpNvatyG84qlWrVjFmzBj69OnDgAEDGDVqFMuXL2fp0qUMGzaMAw44gL59+3LllVc2fIVhypQpXHDBBZu11bt3b9asWbPRsilTptC9e/eNptB45ZVXAFi+fDmjRo1i//3358ADD+T000/ngQceaFivS5cuDVNynHXWWcyZM4fjjz++oe0ZM2YwcOBA+vfvzyGHHMKMGTManhs3bhw9evTgk09y3yxas2YNvXv3btHfJB+esNDMSt7Kt9/i8CseTa29538ystl1IoKTTz6ZsWPHNoxaXlNTw7vvvsu4ceO47bbbGDFiBB999BGnnHIKt956a8NIES3x9a9/vWGU83rr1q3juOOO4/rrr+eEE04AYPbs2XTv3r1h+KWhQ4cyadKkhvH65syZ07D9Sy+9xIQJE5g1axaVlZW88cYbfPWrX2W//fZj4MCBQG5uqbvvvpvzzjuvxTHny2dQZmYFMHv2bMrKyjj33HMbllVVVbF8+XKOOuooRowYAUDnzp25+eabufrqq1Pb93333ccRRxzRUJwAjj322IYJEZszadIkLr/8ciorKwGorKzksssu47rrrmtYZ/z48dxwww1s2LBhS820mguUmVkBvPzyy5tNiQFNT5XRp08fPvzwQz744IMW76dxt11VVRUff/zxFvedr3ym8+jZsydHH300v/rVr7Z5P81xF5+Z2Xa06bQYjW1p+dY01cXXWk3F2NSyyy+/nBNPPJHjjjsu1f3X8xmUmVkBHHTQQSxYsKDJ5ZuOtfj666/TpUsXunbtWtB9t2T7TWNsajqP/fffn6qqKh588MFt3tfWuECZmRXAsGHD+OSTT/jFL37RsOzFF1+kb9++zJs3j8cffxzITWx40UUXMXHixNT2/c1vfpNnnnmG3/3udw3LHn30UZYsWZLX9hMmTOCnP/0ptbW1ANTW1nLVVVfx/e9/f7N1f/CDHzBp0qRU4t6Uu/jMrOSV9/hCXlfetaS95kji4YcfZvz48Vx99dV06tSJ3r17c+ONN/LII49w4YUXcv755/PZZ59x5plnbnRp+ZQpUza6rPu5554DYODAgeywQ+684vTTT2fgwIE88MADG80ndeutt3LkkUcyc+ZMxo8fz/jx4ykrK2PgwIHcdNNNeR1fVVUV11xzDSeccALr16+nrKyMa6+9tmGG4MYOOuggBg8enPfU9S2R13QbafE0AlYMnm6j/fF0G9nRmuk23MVnZmaZlKkC1au8PLVvevcqsZklzczam0x9BrVi1apUumKAVKd3NrO2Z2uXc9v20dqPkDJ1BmVmloZOnTqxdu3aVv+DtG0XEaxdu5ZOnTptcxuZOoMyM0tDRUUFdXV1rF69utihtGudOnWiomLbe8VcoMys5JSVlTWMI2dtl7v4zMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzMwsk/IqUJJ2kzRd0quSlkk6QtIekmZJ+nPyc/dCB2tmZu1HvmdQNwGPRkR/4J+BZcClwBMR0Rd4InlsZmaWimYLlKRdgGOAuwAi4tOIeB84CbgnWe0eYHShgjQzs/YnnzOo/YDVwC8lLZJ0p6Sdgb0jYiVA8nOvpjaWdI6k+ZLm+1vdZk1znphtLp8C1REYDNwWEYOAv9OC7ryImBwR1RFR3b17920M06y0OU/MNpdPgaoD6iLi+eTxdHIF611J5QDJz/cKE6KZmbVHzRaoiFgFvCWpX7JoOPAK8FtgbLJsLPBIQSI0M7N2Kd/BYi8EpkraEXgd+Da54vagpLOBFcBphQnRrHXUoSyV+cHUoSyFaMwsX3kVqIioAaqbeGp4uuGYpS8+W8/hVzza6nae/8nIFKIxs3x5JAkzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8ukvAuUpA6SFkmamTyulPS8pD9LekDSjoUL08zM2puWnEF9D1jW6PE1wA0R0Rf4C3B2moGZmVn7lleBklQBHAfcmTwWMAyYnqxyDzC6EAGamVn7lO8Z1I3ARODz5PGewPsRsSF5XAf0aGpDSedImi9p/urVq1sVrFmpcp6Yba7ZAiXpeOC9iFjQeHETq0ZT20fE5Iiojojq7t27b2OYZqXNeWK2uY55rHMUcKKkUUAnYBdyZ1S7SeqYnEVVAO8ULkwzM2tvmj2DiojLIqIiInoDY4AnI+IMYDZwarLaWOCRgkVpZmbtTmu+B3UJcLGk18h9JnVXOiGZmZnl18XXICLmAHOS+68Dh6UfkpmZmUeSMDOzjHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKB2o56lZcjKZVbr/LyYh+OmVlBtWg+KGudFatWUbdvRSptVbxTl0o7ZmZZ5TMoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLpGYLlKQvSJotaZmkpZK+lyzfQ9IsSX9Ofu5e+HDNzKy9yOcMagPw/Yg4EPgScL6kAcClwBMR0Rd4InlsZmaWimYLVESsjIiFyf2/AcuAHsBJwD3JavcAowsVpJmZtT8t+gxKUm9gEPA8sHdErIRcEQP22sI250iaL2n+6tWrWxetWYlynphtLu8CJakL8GtgfER8kO92ETE5Iqojorp79+7bEqNZyXOemG0urwIlqYxccZoaEb9JFr8rqTx5vhx4rzAhmplZe5TPVXwC7gKWRcT1jZ76LTA2uT8WeCT98MzMrL3KZ8LCo4AzgSWSapJllwNXAw9KOhtYAZxWmBDNzKw9arZARcQ8QFt4eni64ZiZWTH0Ki9nxapVqbTVc599eHPlyla34ynfzcyMFatWUbdvRSptVbxTl0o7HurIMqlXeTmSUrmVorR+P73Ky4t9KGZb5DMoy6QsvpvLkrR+P6X4u7HS4TMoMzPLpJI9g/onSK17J60P/Cx/6lDmd/dm7VzJFqhPwF1EbVh8tp7Dr3g0lbae/8nIVNoxs+3LXXxmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJJTvUkZmZ5S/N8S/VoSyVdlygzMwsk+NfuovPrB2rH/Xfkx9aFvkMyqwd86j/lmU+gzIzs0xygbLU7FvRM7XuIjMzd/FZala+/VbmPmQ1s7YrUwUqi5c5mtn216u8nBWrVrW6nZ777MObK1emEJEVQ6YKVBYvc8yq+quv0uAktqxZsWpVKhdv+MKNtq1VBUrSSOAmoANwZ0RcnUpU1ixffWVmpW6bL5KQ1AG4BfgaMAD4hqQBaQVmZtZaWf2eV6/y8lRi6tyhY0lfmNSaM6jDgNci4nUASfcDJwGvpBGYmVlrZbWnIc0uzCweX1oUEdu2oXQqMDIivpM8PhM4PCIu2GS9c4Bzkof9gD9tpdluwJptCqht8PG1bfkc35qIaPEHoC3Mk3xjact8fG1bc8eXV5605gyqqXPCzapdREwGJufVoDQ/IqpbEVOm+fjatkIeX0vypNCxZIGPr21L6/ha80XdOuALjR5XAO+0LhwzM7Oc1hSoF4G+kiol7QiMAX6bTlhmZtbebXMXX0RskHQB8Bi5y8zvjoilrYwn7y6ONsrH17Zl6fiyFEsh+PjatlSOb5svkjAzMyskDxZrZmaZ5AJlZmaZlJkCJWmkpD9Jek3SpcWOJ02SviBptqRlkpZK+l6xY0qbpA6SFkmaWexYCkHSbpKmS3o1+TseUaQ4nCdtXCnnStp5konPoJJhk5YDXyV3+fqLwDcioiRGpZBUDpRHxEJJXYEFwOhSOT4ASRcD1cAuEXF8seNJm6R7gLkRcWdy1WrniHh/O8fgPCkBpZwraedJVs6gGoZNiohPgfphk0pCRKyMiIXJ/b8By4AexY0qPZIqgOOAO4sdSyFI2gU4BrgLICI+3d7FKeE8aeNKOVcKkSdZKVA9gLcaPa6jxF6Y9ST1BgYBzxc3klTdCEwEPi92IAWyH7Aa+GXSNXOnpJ2LEIfzpO0r5VxJPU+yUqDyGjaprZPUBfg1MD4iPih2PGmQdDzwXkQsKHYsBdQRGAzcFhGDgL8Dxfj8x3nShrWDXEk9T7JSoEp+2CRJZeSSbmpE/KbY8aToKOBESbXkupyGSbq3uCGlrg6oi4j6d/PTySViMeJwnrRdpZ4rqedJVgpUSQ+bpNxkK3cByyLi+mLHk6aIuCwiKiKiN7m/25MR8a0ih5WqiFgFvCWpX7JoOMWZVsZ50oaVeq4UIk8yMeV7gYZNypKjgDOBJZJqkmWXR8R/FjEma5kLgalJYXgd+Pb2DsB5Ym1AqnmSicvMzczMNpWVLj4zM7ONuECZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUBlgKQfS5qQYnv9JdUkw430SavdRu3PkVSddrtmW+M8aX9coErTaOCRiBgUEf9V7GDMMsp5knEuUEUi6QfJvD6PA/2SZf8q6UVJL0n6taTOkrpKeiMZAgZJu0iqlVQmqUrSc5IWS3pY0u6SRgHjge8kc+tMlHRRsu0Nkp5M7g+vH2ZF0ghJz0paKOmhZCw0JB0q6Y+SFkh6LJkOofEx7CDpHkn/e7v94qxdcZ60by5QRSDpUHJDnQwC/gX4YvLUbyLiixHxz+SmGjg7mXZgDrkh+km2+3VErAf+HbgkIgYCS4AfJd+6vx24ISKOBZ4ChiTbVgNdkiQ+GpgrqRvwQ+ArETEYmA9cnKzzc+DUiDgUuBv4P40OoyMwFVgeET9M8ddjBjhPLCNDHbVDQ4CHI+IjAEn146kdnLzL2g3oQm5IG8jNHTMRmEFu6JB/lbQrsFtE/DFZ5x7goSb2tQA4VLkJ4D4BFpJLwCHARcCXgAHA07mh0NgReJbcu9WDgVnJ8g7Aykbt3gE8GBGNk9EsTc6Tds4FqniaGmNqCrkZRF+SNA4YChART0vqLenLQIeIeDlJvOZ3ErFeudGTvw08AywGjgX6kHv32QeYFRHfaLydpEOApRGxpSmbnwGOlfR/I2JdPrGYbQPnSTvmLr7ieAo4WdJOyTu2E5LlXYGVSbfBGZts8+/ANOCXABHxV+Avkuq7Jc4E/kjTngImJD/nAucCNZEbiPE54ChJ+wMk/fkHAH8Cuks6IlleJumgRm3eBfwn8JAkv9GxQnCetHMuUEWQTGv9AFBDbu6buclT/4vcDKKzgFc32WwqsDu55Ks3FrhO0mKgCvjJFnY5FygHno2Id4F19fuMiNXAOGBa0s5zQP9kSvFTgWskvZTEeuQmx3E9ua6QX0nya8lS5Twxj2beRkg6FTgpIs4sdixmWeU8KS0+5WwDJP0c+BowqtixmGWV86T0+AzKzMwyyf2hZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSf8feZ3K8s9z83MAAAAASUVORK5CYII=\n", 
                        "text/plain": "<matplotlib.figure.Figure at 0x7f661c01fb38>"
                    }, 
                    "metadata": {}
                }
            ], 
            "source": "df['dayofweek'] = df['effective_date'].dt.dayofweek\nbins = np.linspace(df.dayofweek.min(), df.dayofweek.max(), 10)\ng = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\ng.map(plt.hist, 'dayofweek', bins=bins, ec=\"k\")\ng.axes[-1].legend()\nplt.show()\n"
        }, 
        {
            "execution_count": 14, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 14, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n      <th>dayofweek</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>male</td>\n      <td>3</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>female</td>\n      <td>3</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>2016-09-08</td>\n      <td>2016-09-22</td>\n      <td>27</td>\n      <td>college</td>\n      <td>male</td>\n      <td>3</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "  loan_status  Principal  terms effective_date   due_date  age  \\\n0     PAIDOFF       1000     30     2016-09-08 2016-10-07   45   \n1     PAIDOFF       1000     30     2016-09-08 2016-10-07   33   \n2     PAIDOFF       1000     15     2016-09-08 2016-09-22   27   \n\n              education  Gender  dayofweek  \n0  High School or Below    male          3  \n1              Bechalor  female          3  \n2               college    male          3  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "df.head(3)"
        }, 
        {
            "source": "We see that people who get the loan at the end of the week dont pay it off, so lets use Feature binarization to set a threshold values less then day 4 ", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 15, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 15, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n      <th>dayofweek</th>\n      <th>weekend</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>male</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>female</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>2016-09-08</td>\n      <td>2016-09-22</td>\n      <td>27</td>\n      <td>college</td>\n      <td>male</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>28</td>\n      <td>college</td>\n      <td>female</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>29</td>\n      <td>college</td>\n      <td>male</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "  loan_status  Principal  terms effective_date   due_date  age  \\\n0     PAIDOFF       1000     30     2016-09-08 2016-10-07   45   \n1     PAIDOFF       1000     30     2016-09-08 2016-10-07   33   \n2     PAIDOFF       1000     15     2016-09-08 2016-09-22   27   \n3     PAIDOFF       1000     30     2016-09-09 2016-10-08   28   \n4     PAIDOFF       1000     30     2016-09-09 2016-10-08   29   \n\n              education  Gender  dayofweek  weekend  \n0  High School or Below    male          3        0  \n1              Bechalor  female          3        0  \n2               college    male          3        0  \n3               college  female          4        1  \n4               college    male          4        1  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "df['weekend'] = df['dayofweek'].apply(lambda x: 1 if (x>3)  else 0)\ndf.head()"
        }, 
        {
            "source": "## Convert Categorical features to numerical values", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "Lets look at gender:", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 16, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 16, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "Gender  loan_status\nfemale  PAIDOFF        0.865385\n        COLLECTION     0.134615\nmale    PAIDOFF        0.731293\n        COLLECTION     0.268707\nName: loan_status, dtype: float64"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "df.groupby(['Gender'])['loan_status'].value_counts(normalize=True)"
        }, 
        {
            "source": "86 % of female pay there loans while only 73 % of males pay there loan\n", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "Lets convert male to 0 and female to 1:\n", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 17, 
            "cell_type": "code", 
            "metadata": {
                "scrolled": true, 
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 17, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n      <th>dayofweek</th>\n      <th>weekend</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>0</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>1</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>2016-09-08</td>\n      <td>2016-09-22</td>\n      <td>27</td>\n      <td>college</td>\n      <td>0</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>28</td>\n      <td>college</td>\n      <td>1</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>29</td>\n      <td>college</td>\n      <td>0</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "  loan_status  Principal  terms effective_date   due_date  age  \\\n0     PAIDOFF       1000     30     2016-09-08 2016-10-07   45   \n1     PAIDOFF       1000     30     2016-09-08 2016-10-07   33   \n2     PAIDOFF       1000     15     2016-09-08 2016-09-22   27   \n3     PAIDOFF       1000     30     2016-09-09 2016-10-08   28   \n4     PAIDOFF       1000     30     2016-09-09 2016-10-08   29   \n\n              education  Gender  dayofweek  weekend  \n0  High School or Below       0          3        0  \n1              Bechalor       1          3        0  \n2               college       0          3        0  \n3               college       1          4        1  \n4               college       0          4        1  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)\ndf.head()"
        }, 
        {
            "source": "## One Hot Encoding  \n#### How about education?", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 18, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 18, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "education             loan_status\nBechalor              PAIDOFF        0.750000\n                      COLLECTION     0.250000\nHigh School or Below  PAIDOFF        0.741722\n                      COLLECTION     0.258278\nMaster or Above       COLLECTION     0.500000\n                      PAIDOFF        0.500000\ncollege               PAIDOFF        0.765101\n                      COLLECTION     0.234899\nName: loan_status, dtype: float64"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "df.groupby(['education'])['loan_status'].value_counts(normalize=True)"
        }, 
        {
            "source": "#### Feature befor One Hot Encoding", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 19, 
            "cell_type": "code", 
            "metadata": {
                "scrolled": true, 
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 19, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>education</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>45</td>\n      <td>0</td>\n      <td>High School or Below</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>33</td>\n      <td>1</td>\n      <td>Bechalor</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>15</td>\n      <td>27</td>\n      <td>0</td>\n      <td>college</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>28</td>\n      <td>1</td>\n      <td>college</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>29</td>\n      <td>0</td>\n      <td>college</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "   Principal  terms  age  Gender             education\n0       1000     30   45       0  High School or Below\n1       1000     30   33       1              Bechalor\n2       1000     15   27       0               college\n3       1000     30   28       1               college\n4       1000     30   29       0               college"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "df[['Principal','terms','age','Gender','education']].head()"
        }, 
        {
            "source": "#### Use one hot encoding technique to conver categorical varables to binary variables and append them to the feature Data Frame ", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 20, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 20, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>weekend</th>\n      <th>Bechalor</th>\n      <th>High School or Below</th>\n      <th>Master or Above</th>\n      <th>college</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>45</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>33</td>\n      <td>1</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>15</td>\n      <td>27</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>28</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>29</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n0       1000     30   45       0        0         0                     1   \n1       1000     30   33       1        0         1                     0   \n2       1000     15   27       0        0         0                     0   \n3       1000     30   28       1        1         0                     0   \n4       1000     30   29       0        1         0                     0   \n\n   Master or Above  college  \n0                0        0  \n1                0        0  \n2                0        1  \n3                0        1  \n4                0        1  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "Feature = df[['Principal','terms','age','Gender','weekend']]\nFeature = pd.concat([Feature,pd.get_dummies(df['education'])], axis=1)\nFeature.head()"
        }, 
        {
            "execution_count": 21, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 21, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>weekend</th>\n      <th>Bechalor</th>\n      <th>High School or Below</th>\n      <th>college</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>45</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>33</td>\n      <td>1</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>15</td>\n      <td>27</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>28</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>29</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n0       1000     30   45       0        0         0                     1   \n1       1000     30   33       1        0         1                     0   \n2       1000     15   27       0        0         0                     0   \n3       1000     30   28       1        1         0                     0   \n4       1000     30   29       0        1         0                     0   \n\n   college  \n0        0  \n1        0  \n2        1  \n3        1  \n4        1  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "Feature.drop(['Master or Above'], axis = 1,inplace=True)  # This is because exactly 50% of 'masters or above' default and 50% don't default\nFeature.head()"
        }, 
        {
            "source": "### Feature selection", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "Lets defind feature sets, X:", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 22, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 22, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>weekend</th>\n      <th>Bechalor</th>\n      <th>High School or Below</th>\n      <th>college</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>45</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>33</td>\n      <td>1</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>15</td>\n      <td>27</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>28</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>29</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n0       1000     30   45       0        0         0                     1   \n1       1000     30   33       1        0         1                     0   \n2       1000     15   27       0        0         0                     0   \n3       1000     30   28       1        1         0                     0   \n4       1000     30   29       0        1         0                     0   \n\n   college  \n0        0  \n1        0  \n2        1  \n3        1  \n4        1  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "X = Feature\nX[0:5]"
        }, 
        {
            "source": "What are our labels?", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 23, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "Y = df['loan_status']"
        }, 
        {
            "source": "## Normalize Data ", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "Data Standardization give data zero mean and unit variance (technically should be done after train test split )", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 24, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 24, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "array([[ 0.51578458,  0.92071769,  2.33152555, -0.42056004, -1.20577805,\n        -0.38170062,  1.13639374, -0.86968108],\n       [ 0.51578458,  0.92071769,  0.34170148,  2.37778177, -1.20577805,\n         2.61985426, -0.87997669, -0.86968108],\n       [ 0.51578458, -0.95911111, -0.65321055, -0.42056004, -1.20577805,\n        -0.38170062, -0.87997669,  1.14984679],\n       [ 0.51578458,  0.92071769, -0.48739188,  2.37778177,  0.82934003,\n        -0.38170062, -0.87997669,  1.14984679],\n       [ 0.51578458,  0.92071769, -0.3215732 , -0.42056004,  0.82934003,\n        -0.38170062, -0.87997669,  1.14984679]])"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "X= preprocessing.StandardScaler().fit(X).transform(X)\nX[0:5]"
        }, 
        {
            "source": "##### Let's convert y to numerical values. PAIDOFF will be 0, COLLECTION will be 1", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 25, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "y = Y.copy(deep=True)\ny.replace(to_replace=['PAIDOFF','COLLECTION'], value=[0, 1], inplace=True)"
        }, 
        {
            "execution_count": 26, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 26, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "0    0\n1    0\n2    0\n3    0\n4    0\nName: loan_status, dtype: int64"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "y[:5]"
        }, 
        {
            "source": "Now Let's shuffle the data and  split X and y into training and testing sets. in a ratio of 70% training and 30% testing<br>\nI use 70% for training because it's a small data set and I don't want the model to overfit on the training data.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 27, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "from sklearn.model_selection import train_test_split"
        }, 
        {
            "execution_count": 28, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.27, random_state=47, shuffle=True)"
        }, 
        {
            "source": "# Classification ", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "Now, it is your turn, use the training set to build an accurate model. Then use the test set to report the accuracy of the model\nYou should use the following algorithm:\n- K Nearest Neighbor(KNN)\n- Decision Tree\n- Support Vector Machine\n- Logistic Regression\n\n\n\n__ Notice:__ \n- You can go above and change the pre-processing, feature selection, feature-extraction, and so on, to make a better model.\n- You should use either scikit-learn, Scipy or Numpy libraries for developing the classification algorithms.\n- You should include the code of the algorithm in the following cells.", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "# K Nearest Neighbor(KNN)\nNotice: You should find the best k to build the model with the best accuracy.  \n**warning:** You should not use the __loan_test.csv__ for finding the best k, however, you can split your train_loan.csv into train and test to find the best __k__.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "Let's confirm the shapes and dimensions of our Feature matrix-X and target variable-y", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 29, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "The shape of train_loan.csv is(346, 10).\nThe shape of X is(346, 8), and the dimension of X is2.\nThe shape of y is(346,), and the dimension of y is 1.\n"
                }
            ], 
            "source": "print('The shape of train_loan.csv is{}.'.format(df.shape))\nprint('The shape of X is{}, and the dimension of X is{}.'.format(X.shape, X.ndim))\nprint('The shape of y is{}, and the dimension of y is {}.'.format(y.shape, y.ndim))"
        }, 
        {
            "execution_count": 30, 
            "cell_type": "code", 
            "metadata": {
                "scrolled": true
            }, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "X_train shape is (252, 8)\nX_test shape is (94, 8)\ny_train shape is (252,)\ny_test shape is (94,)\n"
                }
            ], 
            "source": "# Let's confirm the shapes of our Training and Testing Sets\n\nprint('X_train shape is',X_train.shape)\nprint('X_test shape is',X_test.shape)\nprint('y_train shape is',y_train.shape)\nprint('y_test shape is',y_test.shape)"
        }, 
        {
            "execution_count": 31, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "X_train\n [[-1.31458942 -0.95911111  0.17588281  2.37778177  0.82934003 -0.38170062\n   1.13639374 -0.86968108]\n [ 0.51578458  0.92071769  0.17588281 -0.42056004  0.82934003 -0.38170062\n   1.13639374 -0.86968108]\n [ 0.51578458  0.92071769  0.50752015 -0.42056004  0.82934003 -0.38170062\n  -0.87997669  1.14984679]\n [-1.31458942  0.92071769 -0.81902922 -0.42056004  0.82934003 -0.38170062\n  -0.87997669  1.14984679]\n [-5.89052444 -1.96168647 -0.3215732  -0.42056004  0.82934003 -0.38170062\n  -0.87997669  1.14984679]]\n\ny_train\n 115    0\n16     0\n309    1\n17     0\n8      0\nName: loan_status, dtype: int64\n"
                }
            ], 
            "source": "# Let's just view first 5 rows of X_train and y_train\n\nprint('X_train\\n',X_train[:5])\nprint()\nprint('y_train\\n',y_train[:5])"
        }, 
        {
            "execution_count": 32, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "# Importing the Knn classifier\n# Importing Sklearn metrics to evaluate the model\n\nfrom sklearn.neighbors import KNeighborsClassifier\nfrom sklearn import metrics"
        }, 
        {
            "source": "Now, we would define a simple function that takes k(an int), X_train, X_test, y_train, y_test \nand outputs the best K in the range 2 to k(inclusive) that produces the best accuracy on test data", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 33, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "accuracy_dict = {}\n\ndef knn_classifier(k, X_train, y_train, X_test, y_test):\n    \"\"\"Method takes k an int, and X_train and y_train 2D arrays,\n    as well as test data x_test and y_test, also 2D arrays, and returns,\n    The best K value, from 1 to K, and its test_accuracy score\"\"\"\n    \n    for i in range(1, k+1):\n        # Let's instantiate a model and train it in one line\n        knn_model = KNeighborsClassifier(n_neighbors=i).fit(X_train, y_train)\n        # Let's make predictions on the training data\n        predictX = knn_model.predict(X_train)\n        # Let's make predictions on the test data \n        y_hat = knn_model.predict(X_test)\n        # Let's measure accuracy of predictions on training data\n        train_acc = metrics.accuracy_score(y_train, predictX)\n        # Let's measure accuracy of predictions on training data\n        test_acc = metrics.accuracy_score(y_test, y_hat)\n        accuracy_dict[i] = test_acc\n    key_max_acc = [0,0]\n    for key, value in accuracy_dict.items():\n        if value > key_max_acc[1]:\n            key_max_acc[0] = key\n            key_max_acc[1] = value\n        \n    return key_max_acc"
        }, 
        {
            "source": "Let's call the knn_classifier method above with k = 10, to see the best k from 1 to 10", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 34, 
            "cell_type": "code", 
            "metadata": {
                "scrolled": true
            }, 
            "outputs": [], 
            "source": "best_k_and_test_accuracy = knn_classifier(10, X_train, y_train, X_test, y_test)"
        }, 
        {
            "execution_count": 35, 
            "cell_type": "code", 
            "metadata": {
                "scrolled": true
            }, 
            "outputs": [
                {
                    "execution_count": 35, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "[4, 0.77659574468085102]"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "best_k_and_test_accuracy"
        }, 
        {
            "execution_count": 36, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "knn_model = KNeighborsClassifier(n_neighbors=4).fit(X_train, y_train)"
        }, 
        {
            "source": "Plotting the various K values and corresponding scores from 1 to 15.\nThis shows K = 4 as the best K value for highest test accuracy score", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 37, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "display_data", 
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAm8AAAGJCAYAAADcw9SKAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzs3Xd8FHX+x/HXhxAggPTeq4UmSMAK2MGCioWigiAo3J3+PNTzvDs9Pbt39t5BsYuHig3wFLDTQZqUBKVIExOBAIHk+/vjO4Fl2ZBNSLLZ5P18PPKAnf3uzGdmZ2c+M/Od+ZhzDhERERGJD+ViHYCIiIiIRE/Jm4iIiEgcUfImIiIiEkeUvImIiIjEESVvIiIiInFEyZuIiIhIHCkzyZuZDTUzZ2ZtwoZ3M7MtZjbXzOoEw1YFbe+IMJ67zMyFDZsatH8lQvsRwXstCneOCo+Z3R4+T1F+rkUwb0MPcfonB+M5/VDGc4gxFMq85HOatwfTzOuv0GIyszrBdDsV4LNtgnh2mFn1woqptDKzPsHyOi7GcdxnZjsLaVy3mdlqM8sys+8KY5xSfCKtk2b2nZl9GoNY7jOzPRGG9zOzTDObaGYViymWqsFyubE4plcYysc6gFgysxOAT4ClQB/n3G9hTUab2ePOuU1RjvIyM7vPObe4UAOV4vALcDywshin+QIQutE8B7gFuARYEzK8MGOqA9wGrAAW5POzVwT/VgL6A88XYlxSwplZT+B24G7gYyA9pgFJYRkOZMU6CAAzGwiMAz4ABjrndsc4pBKrzCZvZtYL+BCYD5ztnPs9rMk04DjgZuCGKEY5D2gE3AlcVIihSjFwzu0CivVMgnNuDSFJmpkdGfx3nnNuRXHGkhczM2AwMAe/nl9BCUzezKxi8F1K4Tsq+Pcp59y6Qx1ZrL6rYF1OdM5lRnivQqThpZlzblGsYwAwsyHAS8DbwBDn3AFn5WSfMnPZNJSZnYE/4zYT6B0hcQO/U30a+KOZNY5itNuBe4ALzSy5ADGNNbM1ZpZsZt8El6Z+NLNzgvevDy7n/m5m75tZ3bDPVzOzJ8xsnZntCj47OthQhbbrYmZfmtlOM1trZrcC+7UJ2pU3s7+Z2dJgfOvM7EEzq1SAeTvczCaY2cZguj+b2TtmFn7wUDmYh81mtsnMXjWzGvmZTzNLMLM0M7sl5DMdg1PiX4WNa42Z/Tv4/wGXTUO+k5xllmFmy81sVIR5PN38pfedZrbC/OXysWa2Kr/LKzdmdljwHfwUXFZYaWY3hX7HZlbdzJ4KLm3tMrMNZjbZ/CXPI4ElQdNxtu+y7MAoJn8y0BwYC7wGnGhmrSPEmGhmt4SsN5vM7KPQtmbWwMyeDZbtrmB9GGtmCcH7b5rZ0gjj3u/yju27BNQ3+PyvwE/Be0ea2WvBb2ZHsKweN7NqEcZ7upn9L/htbTOzeeZ3JJjZZ2b2bYTPHBlM+4rw9/JiZgPMbEawPv0WzG/jkPc/N7OvI3yuuZllh65/wff6ZvCb2Wlmsy3YZuQRw43Bd7TDfLeRGWZ27kHafwc8E7xcG8z7zcF7NczsaTNbH6yXS83smrDP5/pdHWSabczsddu33VhpZv8JazPMzH4IWdfGmFm9sDbrzewFMxtlZsuA3cBpId/hCDN72Mx+AXaaWVK0y9bM2pnZB8G0d5r/bb6Vy3y/ambpwd/LduC2LT/LsU/wG9oSLJ+x4eu2+d/Z22a2NWj3EnBYhOWc2+8qP9PYFrR7zswutnx2FzCzq/DblnHA5XklbmZ2ajCNUyK8N878NqVc8HqomU0PvsffzWyWmQ2IIqbxZrYwwvBZZvZh2LCGZvaimf0SrIuLLNiGhLRpamZvhLRZa35fXrAuKM65MvEHDAUccB2wE3+5KimXtquAV4G6wFbgmZD37vKLbb/2U4GvgIr4DdKnIe+NCKbbIo/4xgK/A4uBK4E+wJdBrA8CE/GX1a4M2r0d8tlyQdvt+LOEZwKPBtO9J6RdHeA3/A58AHAB8DWwOsI8vRmM75/A6cC1QBrwbkibFsE0huYxb8uAGfgzkr2AS4PlWyF4/+RgPKnA40H81wI7gJcLMJ8fAJ+HvL4OyAAygSrBsCOCz52V27yEfCdLgJHAGcDrQbtTQtq1A3YFsV2Av6T4A/AzsKoA62ibCO9VAL4HNgbL5nT85c9M4O6QduOAtcAwoCdwIfAIcAz+cueAYBq34c8sHwfUjiK2scG06gAdg3H8K6yN4dfTTOA+oDfQL/iOTgxZB1cF8/F/wKnB+vBOyPrwJrA0Qgzfsf9vq08Qx1p8YnEmcF7w3mn4s+Dn49e54UAKMDVsnP3xl4z+Fyyb04HRwK3B+xcG0+gU9rmHgS3ksg0Ji++4kGF/BrKBZ4GzgEHAcvxvpHLYetA6bHz/CNazWsHrVsCv+LP+lwbLe1wwP71DPncfsDPk9fDgO/o7cAp+u/J3/NmO3OalPfBAENfZwXrTCH/15nv8dvK6IIangnb/jOa7ymV6bYPlm4Lfhp6KX6dDtwf/F4xzXLAsRwbLY1Ho9wKsD6Y7P+Q7bgEcGRLTO8Fy6AckRrNs8ev7T/ht6IX49ewy4I0I870aeC4Yz5/x27DQdTm/yzEFeAi/TRodrBfPhv0WZ+C3938IPjcuiCN8ncztdxXtNLYAo4LPjQmWyX7TyOU7vg/YA1yD/008DViU20rDb19fChteJViG94YM+1cQ3xnB3/3B93h5SJuqQcw3hgwbDyyMMO1ZwIchr2vjt2kr8fvnM4DHgnkaFtLuW2AhMBC/bR6A7zpTL9p9xH5xFORD8fjHvg2iw/f3qXiQtquAV4P/34nf0LUOXueavAX/Hx5Mo2fwOj/J297PBcM6BcN+BBJChj+EP3pMCF6fS4QkKlgxdgF1gtd3B/PSLGxl3xw6T0CPYHxDwsZ3WTC8c/C6RaTphn2mTtDmYBvqk4M2L4cNfwKfvFo+53M0PvGrGLx+D79h2Ma+De+oYBlWzW1eQr6T0EStYrC8ngsZ9jqwiWDnGwxrGMS+qgDraKTk7Sr8xuDYsOF3BvNaI3i9gpBENsJ4cnZYl+cjrpwN4vshw+bik20LGXZ2MO6rDzKuf+M32O0P0ia/ydsbUcxDefxO2wFHBcMSgHX4nW/EnUbQZjXwZMiwSvgd+yN5THO/5A2ogd9pPxXW7vBgmYwKXh8WtLstrN0S4L8hr18L4q8e1m468F3I6/Dk7QXgm2i//5DPXRPMT4OQYRcHwwaGtX0Vf8BUPb/fVdD+bfzBYt1c3q8QfAefhg3P+Y6vDhm2Plh/6+TyWzhgWUSzbIEmwefPjGIdeC9seM5+IuegJr/L8dmwdi8Av4e87hu0uyCs3Reh62Qev6u8pnEeEbbtwOTwaeSybO5j3z55cgHWx3vxB9ehifrlwfja5fKZcvhtwRvA1yHDDyV5ux+/b2kW1u4N/LbD8NuRLODK/M5nbn9l8bLpR0Br4G9Rtn8A/8P/V5Ttx+KPou/Od2Sw3Tk3PeR1zqWjz5xzWWHDy+MTBPBZfDZ+ZQn1Kn4jd3zw+nj8hufnnAbOue34syWh+uCTvHfNXz4tb/4S5+SQ6UXrV/wR3H1mdpWZtT1I24/CXv+AT5bqh0w3mvn8Ar+DPSE4dd4LmITfSZ8atDkVmOmc25ZH/BnOuS9yXjjfR2c50CykzXHAx865jJB2vwDf5DHu/OiDX69mR/hOKgHdg3YzgavN7K9mdkzOpYO8mFm50PGGfe4i/MYt9G7ql/EJb4+QYWfik5AxB5nUmfgDncLsZzMhfICZVTKzW81fVt+JT9SnBG8fEfzbEf8bet4FW9twwe/ueeByM6sSDL4EqIU/e5YfPYDKwGth32FK8NczmOZW4H38jihnfrrhk43Q76AP/re7PcI60c1y7+IwEzg2uFR4as5lwgLqid9WvBM2/FUgiX3rZY4DvqtcnIlPeHK7WawD/jt4NXSgc+4zYAP+Nx/qS+fc5lzG9V6EYdEs2/X47jUPmNlwi9CNIMTbYa/fDP7N2WbldzlG2lYeFnIp9nj8Ae0HuUw3GnlN47hgGuH7j/H5mEYW/irYaWZ2aaQG5rvChG6bcrqJjMMf6Jwf0nwwMNuF3DRo/tL2eDNbh98+7caf/TqCwtEH30d+Xdi6Mgmf4LcMtiNzgFvM7E9m1u5QJ1oWk7fRwIvAbRb02TgY51w6/mzBIDNrH0X7LPylxpPM7Kx8xpYWNq6cjrPhd8HmDM/ZONcCtrgDO/+uD3kf/I5qQ4Tphg+rh0+GtuFX9Jy/jcH7tXOfhf0FO8Uz8Ecr9wLLzCzFzP4QofmWsNc585Pf+ZyPTxpPAboA1fA/ri+AU4If/8nB67yEL/ucuEJ3jA3Zt2xCRVrWBVUPv7HZHfaXk+znfCcj8Z1+RwKzgQ1m9p+D7Mhz3BM23o9D3rsCf4T7pfl+OTWC97PZdwdqTgwb3MHvEKvN/nfSFoZfIgx7EH/n7lj8JbXu+A027PvucpZZXvE8h9+BDgpejwKmO+eW5P6RiHL6Yn3Fgd9jW/b/Xb0CtDGznJ37YPzv42PwOzT8+n51hHHdid+218wljufxlxx7AJ8BW8z3QW2Sz/khiGFj2MElHPibzBHpu9pPMG/VOfj3kjPeSONbn8/p7vdetMvW+X5Zp+Lv2v4PsMJ8f9fhEaax37YgOGjeDuT0dczvcsxrW9kQ2OScyz5YHHmIdhrhBz753e71w1+9esXMLo7w/gb2/w4uAggStNn43wZm1gDfXWJczgfNrBa+S0Qb4EbgJKAbPonNd//tXNTDX3UIX1dyDmJzftfn439vtwKLzPdL3q/Pcn6UxbtNHf5HWRG418x2OuceyeMzj+P7KdyF70+Rl7fxd6nexb5OvkVpC1DLDrxTqkHw76/Bv7+w7yxWqPBhv+Iv+fWI0Bb85YSoOedSgCHBSno0/vLLU2a2yjn3ST5GFdV8OuecmU3Db1i34u/e/M3MPsd/Jyfi+zNGk7xF4xf27ZhDRVrWBfUr/vL55bm8nwLg/M03NwE3mVlLfJ+uu/GXXm47yPgfZ/8j5nTwnWzxiW45Im+ULzGza5xzO/CXk+ubWXmXe4fjzezbYeVmJ/7gIVxtwg5wApHOmg3An1G7N2eABc9xDIuFvOJxzq03s/eAkeY77p+A70KQXzm/w0vxZ2/Dhd44NQW/477czGbi5+etnPXeOZdlZun4syMP5zK9iGeagh36k8CTZlYbf+bgQfylwvAzVnnZAtQ1s3JhiUL4tmfv5PMaYTBvaRz8e8lJLBpEeK8Bvm9RtNPd7738LFvn3HL8d1QO6IzfT7xgZimhZ+wJ2xYEZ3Gr4Pvb5cxPfpZjXn7JZXyFuU3KmYaFJXD5moZzbqeZ9cWfgXvdzHY550LP5p2G74eYI/TRSePwZz7r4X9Xjv2vzPTCL8OznHPzcgaaWaTtS7iDbYfWh7z+Fb++5XY1bwmA83doX42/MtIef+n8fvxyHJfLZ3NVFs+85Wy8huKTrIctwt2DYe0z8Dv9C/BZe17jd/ij/mMonseGTMN/l5eEDb8Mf5Yu5xEY3wLHBTtkYO9GpG/Y5z7FH5VUd87NivBXoMcEOG8ecH0wqEM+RxHtfIJPzLrj+8l9HgybjT/avT1of8AdfQX0HXC2mVXOGWBmDfFJYmH5FH+Z8rdcvpPwo2Scc6nOufvxl1tzlnXO0XNSWNu1YePLSS6G4Jf5cPyZzNC/m/CXLfoFbSfjDwiHHWQ+JuPPSh91kDY/AY1DLs8QtG95kM/sFRwkJOGPfkOFx7UQfyAyIoqj36eAZHzSsxl4N5pYwkzH909slct3uCynYXAG5g180tYXf3AQ/hDwT/EHQz/kMr48n5HlnPvVOfca8F/y/3sE/5usyL51IMdlwbzOLMA4wa8n/SIk3DkW4hOe/e6UNrPT8MnDtAJON0e+lq1zLts5Nwd/dgcOXJb9w17nxJ1zJ3NhL8dvg/Gdl8t0C8N3wTTC9x/h2+c8BfvYc/Db6HfMrHfIe/PDln3o1ZCcRG0g/gzcJOdc6FWQnG3y3u8rOEPXJ4qwfgKaWsgdtmbWgf27zIBfV9oDK3JZV7ZHmN9F+JvudlGw312ZPPMG7D26ugy/8j0VZPsH66vzPP6HeWaU4//I/O3+vfNsfOg+wV+Kecb8I0QW4U/jjsDfdZNzBP4w8Edgspndjl9x/oLfOITGPtXM3gDGm9lD+DuKsvHJw9nAX0N3NAdj/kn+jwJv4TvTJ+AT5z3sS6oKez4Jxp2I70tyfzBfWWY2HZ/QTQ/OFhWGu/AdjieZ2QP4depW/Jmq8MsWBTUGf4nyCzN7EL/zqoi/HHAe/kaMLDObhT8oWYRPVE/H95V6NBjPGvwZnsvM7Ef8GbmV7sAHVOcYgr954KXwN4L1+y9BXK/jN2IfAk8EZ/2m4g8CTgbGO+e+wXdBGBDMR86Z7Hr4ndaQ4MzSW/iDn1fN7DH8zvhmcjmTFC448zoZn5Qtxd+A1B9/MBXaLsvMRuMvoUw2s+fxR9HtgWrOubtC2k41s8X49ek/ES7dRxPXlqCrxoNm1gjfJ2Yr/gzTKcAnzrnQs5+v4Lt5PI7fMYQ/h/Dv+DsUp5rZU/i772rib3Rq5JyLeFBqZmPxN9h8F/x7JH7nNym/84TvmzcDeCmYpx/x6+Pl+BsuCvog31vw29rvzOxe/JnlpsCpzrmhzrlMM/sX8KiZjcGvM83wZ5kXE9YXrgDyXLZm1h3f3eBt/NmgRPy2KBO/7ofqambP4pPko4I4P3XO5RxAFvZy/BCf8I0JDiRT8Ylgm4N+Kn8mhkzj7/jfWWhfsnxt+5xzW82sD37bPcHMzgk7exnpMxvNbBL+QLIx/iaIUDkHTM8F25sa+CsQ68n7DOFb+LNp48zsSfxl4r9y4FnQ+/B3G39lZo/g93OH4Z9C0NU519/8o4DG47eTP+KXTX/8mb0pFIQrpDsfSvofudzJFyy8j/EdJy8Nhq0iuNs0rO0VwThc2PCpBHebhg3vldOe6O42XRNhuAPuymte8P26nsCfgs3En20ZTdhddPgdWM4jSNbik4x/RZincvhb1ucHbdOD//+bfXc+tSDvu03r4Tu3L8MnClvwR5mhjzI4ORjP6bnMZ4v8zmfQdj3+iOuwkGGjg3HeHtb2gHk5yHcylQMfOXEG/rECu/A7mpH4ztlzD3UdDXm/Mj5RXBZM51f8Duaf7Lsj96EgjnR8n8X5wB/DxnMJ/qaX3US4wy2k3fHB+385SMwP4n87jUN+T7fhLwtm4vsCTiTksRf4jeCLwfezi+CWf/a/o7o/fie8A9/R9xRyvyvupAhx1cd3/k4L1rmX8Zc7I93Rd2awTm7HJ1NziXA3Lv53kp3b9xOh/QGPCgmGnx9Mbyv+N7EcfyffERHGsYCwx0WEvd8cn9ivC5b3OnwSNjCkTfjdpiPwO7VN+N92Cr7PVtU85ueAu02D4TXw3UPWBzEsBa7JZVkc8F0dZHqH4xOjnG4cK4D7w9oMw3ek3xXMz1jCHr0QxPVChPEf9M7rvJYtPlkYF3x/GUGcXwCnRZjvvviEMh1/8PQKvt9coSxHfD/M/b4b/OXCt/Hbgd/wv7FLwtdJovxdHWQa74RN46qg3QHrc9j47gP2RBheC7/d2kZwN24e48l5/FE6ER7dg+/zuiBYh5bhL10+AGwLaXPA3abB8IH4y5478P22exF2t2nQri5+v/RT8N1twO8nQu8gfyEY17Yg1m+Bi6L9PYT/5WzwRaQQmVlV/M7mI+dcpA7MEmeCs5q/OefOiHUsEh+CM0mfAD2cc1/l1b40MLMX8d2F6jhVSSgyZfayqUhhMrPH8Y8GWYd/eOl1+Mssjx7sc1KyBXfpdsFfnu9K8XSDEIkLZjYC3zViKb4bx9n4s6F3KHErWkreRApHJXzfuvr40+Yz8JeB81v8XUqWFvikfAv+UvvkgzcXKVMy8I+daYnvMpGC7xue2126Ukh02VREREQkjpTJR4WIiIiIxCslbyIiIiJxpNT2eatTp45r0aJFrMMQERERydPs2bM3O+fqRtO21CZvLVq0YNasWbEOQ0RERCRPZvZTtG112VREREQkjih5ExEREYkjSt5ERERE4oiSNxEREZE4ouRNREREJI4oeRMRERGJI0reREREROKIkjcRERGROKLkTURERCSOKHkTERERiSNK3kRERETiiJI3ERERkTii5E1EREQkjih5ExEREYkjSt5ERERE4oiSNxEREZE4ouRNREREJI4oeRMRERGJI0reREREROKIkjcRERGROKLkTURERCSOKHkTERERiSNK3kRERETiiJI3ERERkTii5E1EREQkjih5ExEREYkjxZq8mVkfM/vRzFaY2c0R3n/YzOYFf8vMLC3kvX+b2SIzW2Jmj5mZFWfsIiIiIiVB+eKakJklAE8CZwBrgJlm9oFzbnFOG+fc6JD21wJdgv+fAJwIdAre/groBUwtluBFRERESojiPPPWHVjhnEtxzmUCbwLnH6T9IOCN4P8OqARUACoCicCGIoxVREREpEQqzuStMbA65PWaYNgBzKw50BL4HMA59y3wBfBL8DfJObckwueuNrNZZjZr06ZNhRy+iIiISOwVZ/IWqY+ay6XtQGC8cy4LwMzaAEcBTfAJ36lm1vOAkTn3nHMu2TmXXLdu3UIKW0RERKTkKM7kbQ3QNOR1E2BdLm0Hsu+SKUA/4Dvn3Dbn3DbgE+C4IolSREREpAQrzuRtJtDWzFqaWQV8gvZBeCMzOwKoCXwbMvhnoJeZlTezRPzNCgdcNhUREREp7YoteXPO7QGuASbhE6+3nXOLzOwOMzsvpOkg4E3nXOgl1fHASuAHYD4w3zk3sZhCFxERESkxbP8cqfRITk52s2bNinUYIiIiInkys9nOueRo2hbbc95EpJhlZMCECZCaCq1aQb9+kJQU66hEROQQKXkTKY1mzsT17Ytt2Pc4RFe/PjZxInTrFsPARETkUKm2qUhps2PH3sRtUb1WPHPsRSyu1xLbsAHXty/s2BHrCEVE5BAoeRMpbSZMwDZsYF6Dtpx7xSPcd/Iw+g55eG8Cx4QJsY5QREQOgZI3kdImJYVsjL/3uQZXzv/EsxLKM61Fl73vi4hI/FLyJlLKuJatuP30kSyu35py2Vl7hy9s0MaXNGnVKmaxiYjIodMNCyKlzCO1juaVrtW5asZ/uWDRF0xv0YUVdZrxbsfTab0rjev79Yt1iCIicgiUvImUImO+TuXRaau4pHkl/jb2I8pt3ED7jak4IKFiRR7rfB4156xn2IktYx2qiIgUkJI3kVLivblr+dfExZzZrj73XnYM5Yam+psTUlKwVq245/zzSXt3Cf+auJgalRPp16VJrEMWEZECUPImUgp8vnQDN74zn+Na1eKxQV0on1DOP5D30kv3tikPPDaoC8PGzOTGdxZQPSmRU4+sH7ugRUSkQHTDgkicm7lqC394dQ5HNazG80OSqZSYkGvbSokJPDekK+0aVuMPr85hRuqWYoxUREQKg5I3kTi2eN3vXDl2Jo1rJDF2WDcOq5SY52cOq5TI2GHdaFwzieFjZ7JoXXoxRCoiIoVFyZtInFq1eTtDXppB1YrlGTfiWGpXrRj1Z2tXrci44cdStVJ5rnhpJqs2by/CSEVEpDApeROJQxt+38nlL35PVnY244Z3p3GN/Becb1wjiXHDjyUrO5vLX/yeDb/vLIJIRUSksCl5E4kzaRmZDHlxBlu2ZzJ2WHfa1DuswONqU68qY4d157ftmQx+8XvSMjILMVIRESkKSt5E4khG5h6uHDuT1M3beX5IMkc3rXHI4zy6aQ2eG5LMqs0ZDBs7k4zMPYUQqYiIFBUlbyJxInNPNqNencO81Wk8NqgzJ7apU2jjPrFNHR4b1Jn5q9MYOW42mXuyC23cIiJSuJS8icSBrGzH9W/PY/qyTdzTryN9OjQs9Gn06dCQey/syJfLN3P92/PIynaFPg0RETl0ekivSAnnnOP2Dxbx4YJfuPmsIxnYvVmRTWtAt2akZezm3k+WUj0pkbsu6ICZFdn0REQk/5S8iZRwD09ZxrjvfmJkz1aM6tW6yKc3sldrtmRk8uy0FGpVqcANZx5R5NMUEZHoKXkTKcFe+iqVxz5fQf/kJtx81pHFNt2b+xxJ2vbdPP75CmpUrsDwk1TIXkSkpFDyJlJC/XfOGu74cDG929fnnn4di/XypZlxd78OpO/YzZ0fLqZGUiIXdVUhexGRkkA3LIiUQJ8t3sBfxi/ghNa1eXRgUGi+mJVPKMejgzpzYpva3PTuAj5bvKHYYxARkQMpeRMpYb5P+ZU/vT6H9o2q8VweheaLWsXyCTw7OJkOjarxp9fn8H3KrzGLRUREPCVvIiXIonXpjHh5Fo1rJjFmaDeqVox9z4aqFcszZlh3mtRMYsTLs1i4VoXsRURiScmbSAmRunk7V7w0g8MqlefV4fkrNF/UalWpwLjhx3JYpfIMHTODVBWyFxGJGSVvIiXA+vSdDH7xe7KyHa8MP5ZGBSg0X9Qa1Uhi3IhjyXZw+Qvfsz5dhexFRGJByZtIjKVlZDLkpe/5bXsmL1/ZnTb1qsY6pFy1rluVl4d1Jy1DhexFRGJFyZtIDGVk7mHY2Jms2pzB80OS6dTk0AvNF7WOTarz/BXJ/LQlg6FjZrJ9lwrZi4gUJyVvIjGSuSebkeNmM391Go8N6sIJhVhovqid0LoOjw/qwoI1aYx6dTa79mTFOiQRkTJDyZtIDGRlO0a/PY8vl2/mvgs70adDg1iHlG+92zfgvos6+UL2b81XIXsRkWIS++cQiJQxzjlufX8hHy34hb+ddST9uzWNdUgF1j+5KekZu7n74yVUr5zI3SpkLyJS5JS8iRSzh6Ys4/Xvf2ZUr9aMLIZC80Xtqp6t2JKRydNTV1KrcgVu7K1C9iIiRUnJm0gxevGrVB7/fAUDuzXlr31KT5JzU+8jSMvI5IkvVlCjciIjerSKdUgiIqWWkjeRYvLu7DXc+eFi+rRvwN3FXGi+qJkZd13QkbSM3dz10RJqVK7AxSpkLyJSJHTDgkgxmLJ4Aze9u4AT29Tm0UGdSShXehK3HAnljEcQnEoXAAAgAElEQVQGduakNnX467sLmKJC9iIiRULJm0gR+y4oNN+hUTWeHZxMxfKxKzRf1Hwh+650aFydP70+h+9UyF5EpNApeRMpQgvXpnPVy7NoWjOJMcO6l4hC80WtSsXyjB3ajWa1KquQvYhIEVDyJlJEUjZt21toftzwY6lVpUKsQyo2NatUYNzw7lRPSuSKl2aQsmlbrEMSESk1lLyJFAFfaH4GDhg3omQWmi9qDasnMW54dwAGvziDX9J3xDgiEZHSQcmbSCH7bbsv2p6+YzcvD+tO67olt9B8UWtVtyovX9md9B27GfLiDH7brkL2IiKHSsmbSCHavssXmv9piy8037FJ9ViHFHMdGlfnhZxC9mNVyF5E5FApeRMpJLv2ZDHq1dksWJPG44O6cHzr2rEOqcQ4rlVtnrz0GBauTWfkOBWyFxE5FEreRApBVrbj+rfm+0LzF3Wid/v4KzRf1M5oV5/7L+rEVys2M/qteSpkLyJSQKX/uQUiRcw5xy3vLeSjH37hH2cfRf/k+C00X9Qu7tqEtIxM7vpoCdWTfuCeUlZpQkSkOCh5EzlED0z+kTdm/MwfTm7NVT1V0zMvI3q04reMTJ78YiU1K1fgpj5HxjokEZG4ouRN5BC88GUKT36xkkHdm3JT79JTaL6o3XjmEfyWsZunpvoETkmviEj0lLyJFND42Wu466MlnN2xAXddoMt/+WFm3Hl+B9IzdnP3x0uoXjlRl5tFRKJUrDcsmFkfM/vRzFaY2c0R3n/YzOYFf8vMLC3kvWZmNtnMlpjZYjNrUZyxi4SavGg9f313ASe1qcPDA0pnofmillDOeGjA0fRoW4eb313ApEXrYx2SiEhcKLbkzcwSgCeBs4B2wCAzaxfaxjk32jnX2TnXGXgc+G/I268A/3HOHQV0BzYWT+Qi+/t25a9c88ZcOjSuzrODu5bqQvNFrWL5BJ65vCudmtTg2jfm8u1KFbIXEclLcZ556w6scM6lOOcygTeB8w/SfhDwBkCQ5JV3zk0BcM5tc85lFHXAIuEWrk3nqldm0axWZcYO7UaVMlBovqhVqVieMUO70bxWZa56ZRY/rFEhexGRgynO5K0xsDrk9Zpg2AHMrDnQEvg8GHQ4kGZm/zWzuWb2n+BMXvjnrjazWWY2a9OmTYUcvpR1OYXmqyclMm54d2qWoULzRc0Xsj/WF7IfM4OVKmQvIpKr4kzeInUKyu0pnQOB8c65nMewlwd6ADcC3YBWwNADRubcc865ZOdcct26dQ89YpHAL+k7GPziDADGDe9Ow+plr9B8UWtQvRKvjjiWcgaDX/iedWkqZC8iEklxJm9rgNDbyZoA63JpO5DgkmnIZ+cGl1z3AO8BxxRJlCJhfKH5Gb7Q/JXdaVWGC80XtZZ1qjB2WHe27tzD4Be/Z4sK2YuIHKA4k7eZQFsza2lmFfAJ2gfhjczsCKAm8G3YZ2uaWc7ptFOBxUUcrwjbdu1h6NiZ/LwlgxeuSKZDYxWaL2o5hezX/LaDYWNmsE2F7EVE9lNsyVtwxuwaYBKwBHjbObfIzO4ws/NCmg4C3nTOuZDPZuEvmf7PzH7AX4J9vrhil7Jp154sRo2bzcK16Tx56TEc10qF5ovLsTmF7Nf9zshxs1TIXkQkhIXkSKVKcnKymzVrVqzDkDiVle249o05fPzDeh645Ggu7tok1iGVSe/OXsMN78znrA4NeOLSY/Q8PREptcxstnMuOZq2xfqQXpF44AvN/8DHP6znlnOOUuIWQxd1bcKt57bjk4Xr+ceEHyitB5siIvmhh1SJhPn3pB95Y8Zq/nRKa0b0UM3NWBt+UkvSMjJ5/PMV1KhcgZvPUiF7ESnblLyJhHhu+kqenrqSS49txo1nqtB8SXH9GYezZXsmz0xbSc3KiYzs1TrWIYmIxIySN5HA27NWc8/HSzmnY0PuPL+DCs2XIGbGHed3IH3Hbu79ZCk1K1egfzcVsheRsknJm5RdGRkwYQKkpjKp9uHc/HMVerStw0MDjlbH+BIooZzxUP/O/L5zDzf/dwHVErLps+QrSE2FVq2gXz9IKmUPTw5ZR0vtPIpIvil5k7Jp5kxc377Yhg1806wj117yLzr9tpJnzqujQvMlWIXy5Xjm8mO47JH/8X9vzGXs2/dzws8/AODq18cmToRu3WIcZSEJWUdzlLp5FJEC0aNCpOzZsQPXsiW2YQPvHdWLm86+jgZbf+X9V66nRvXKWGqqzm6UZDt28NsRHRhwxg2srlGf/gsmM2DBFNptTPXJTWn4/kLW0UX1WvFlyy70TJ1TuuZRRPaTn0eFKHmTsuf119k15Aoe7DGY57pfCGYkZO1h4iujabcxFV57DS69NNZRSm5efx0uu4zpzTszZMCdYAbOkeCyMecgIQHKxflTkLKzISsLZ0aWlQMzymft4QOtoyKlVn6SN102lTIlO9sx8cffeGDEM6yu0WDv8KyE8kxv0cXvGFNSYhih5Cn4fhY3aO0TNwAzuv+0kGPWLYVeveCkE2MYYCH46mv4ZhpzGh/Jt82PBmCP1lERCSh5kzLjy+WbuO+TpSza1YJ2u1Zy16Qnuf30kexJKE/5rD30XDXXN2ylZ7uVaMH30zN1Dg/0GLz3+7v1ixd8YjOqD/SO82fB/ToH7hrH4not6TvkYbISymPZ2VpHRQTQZVMpAxauTef+T5fy5fLNNKmZxI0nt6Tv+SeQsGE9i+u1ZHqLLvRcNVf9ieJFSH+wUvv9hc3j33pfww8N2vDR2Os4styO0jGPIrIf9XlDyZvA6i0ZPDD5R96ft46alRO55tS2XH5cM383qe7ki29l4fsLmce0SlU59apnabFtE+NHHU+57t1jHZ2IFDL1eZMybcv2TB7/fDmvfvcTCeWMP53SmpG9WlOtUuK+Rt26+bMXEyb4/kOtWmF6hlb8KAvfX8g81khJ4e81ErlxTRveogGDYh2biMSUzrxJqZGRuYeXvkrl2WkpbM/cQ//kpvz59MNpUL1SrEMTOWTOOQY89x0/rt/K5zf0onbVirEOSUQKUX7OvMX5/fQisCcrmzdm/MzJ/5nKA5OXcVzr2kwe3ZP7LuqkxE1KDTPj7gs6sH3XHu75eGmswxGRGNJlU4lbzjkmL97Avz9dyspN2+navCZPXXYMyS1qxTo0kSLRtv5hXN2zFU9NXcklyU04rlXtWIckIjGgM28Sl2at2sLFz3zLyHGzAXhucFfGjzpeiZuUetee2pYmNZO45b2FZO7JjnU4IhIDSt4krizfsJURL8/i4me+Zc1vGdx3YUcm/bknZ7ZvgJmKyUvpl1QhgTvOb8+Kjdt4/ks9rFekLNJlU4kL69N38vCUZbwzezVVKpTnL72P4MoTW5JUQUXkpew59cj69GnfgMc/X855Rzeiaa3KsQ5JRIqRkjcp0X7fuZtnpq7kpa9Tycp2DD2hJdec2oZaVSrEOjSRmPpn33ZMf2gT/3x/IS8N7aYzzyJliJI3KZF27cli3Lc/8cQXK0jL2M0FnRtxw5lH6AyDSKBRjSSuP+Nw7vpoCZMWradPh4axDklEiklUyZuZ3Qe86JxbXsTxSBmXne14f/5aHpi0jLVpO+jRtg5/7XMkHRpXj3VoIiXO0BNa8O6ctfxr4mJOaluXqhV1PC5SFkR7w8IZwFIzm25mQ8ysFD3GXEoC5xzTlm3inMe/YvRb86lROZFXhx/LuOHHKnETyUX5hHLcdUEHfknfySNTlsU6HBEpJlElb865rkAXYC7wILDezJ4zs2OLMjgpG35Yk87lL37PFS/NYNuu3Tw6sDMTrzmJk9rWiXVoIiVe1+Y1GdS9GWO+WcXidb/HOhwRKQZRPyrEObfAOXcd0AgYATQBvjKzhWb2ZzOrVlRBSun0868ZXPvGXPo+8RVLftnKbX3b8dn1vTi/c2PKlVPna5Fo/bXPEdRISuQf7/1AdnbpLHkoIvsU5DlvDsgK/gVIB64BVpvZxYUVmJRev27bxe0fLOK0h6by2eINXHtqG6b95WSGndiSiuX16A+R/KpRuQJ/P/so5v6cxpszV8c6HBEpYlH3bjWzDsCVwGWAAa8ANzjnlgbv3wQ8AYwvgjilFMjI3MMLX6by3PQUduzOYkC3pvz5tLbUq6b6oyKH6sJjGvP2rNXc/+lSzmxfnzoqXC9SakV15s3MZgLzgI7AtUBj59yNOYlbYCxQr9AjlLi3OyubV7/7iV7/mcpDU5ZxYpvaTPpzT+7p11GJm0ghMTPu7teBjMw93PPxkliHIyJFKNozb5OB/s651NwaOOc26i5UCeWcY9Ki9fz70x9J2bydbi1q8szlXenavGasQxMpldrU84Xrn/xiJZd0bcrxrVW4XqQ0Mufy7txqZuWCtllhwxMA55wrcdWRk5OT3axZs2IdRpk1I3UL936yhLk/p9GmXlX+2udITj+qnp4CL1LEdmRmceYj06iQUI5PrutJhfIqYS0SD8xstnMuOZq20f6qxwPXRxj+Z+CdaAOT0m/Zhq2MeHkm/Z/9lnVpO7j/oo58el0PzmhXX4mbSDFIqpDAHed1YOWm7SpcL1JKRXvZtAdwW4Thk4CbCi8ciVe/pO/g4SnLGD97DVUqluemPkcw7AQVjheJhVOOrMdZHRrw2P+W07dTI5rVVlk5kdIk2uStKpAZYfgeQM93K8PSd+zm6akrGfN1Ks7BlSe25E+ntKGmCseLxNQ/+7Zj+rJN/PODhYxR4XqRUiXa5G0h0B+4M2x4f2BxoUYkcWHn7n2F43/fuZt+nRsz+ozDVThepIRoWD2J0UHh+k8XruesjipcL1JaRJu83QWMN7MWwOfBsNOAy4EBhR+WxFxGBkyYAKmp0KoV9OsHSUlkZTvem7uWh6b4wvG9Dq/LX/scSbtGOgErUtKEFq7vcbgK14uUFlH9kp1z75vZJcAtwJBg8HxggHNuQlEFJzEycyaub19sw4a9g7Lr12fa8+O5f2U2S9dvpWPj6vzn4k6c0Eb1R0VKqvIJ5bi7XwcuevobHp6yjFvPbRfrkESkEER9GOacew94rwhjkZJgx469iduieq34smUXGvy+ibeO7s23X6fTrGYlHh/UhXM6NlT9UZE4cEyzoHD916lceExj2jeqHuuQROQQ6Ry67G/ChL2J23lDHiIrwa8i1XZu5fYpz3Dp6EFUOPq0GAcpIvnx195HMmnhev4xYSH//cMJOvASiXPRlscqb2Z/M7MFZpZmZhmhf0UdpBSjFP9cqC9bdtmbuAEMn/EeQ+d8SIVVem6USLypXjmRf5xzFPNWp/HGzJ9jHY6IHKJoH9J7O/An4EWgIv6u01eBDPSct9KlVSsAuq5ZDEH1jfJZezhj5Yz93heR+NKvS2OOa1WL+z9ZyuZtu2IdjogcgmiTt0uBkc65R/HPdnvbOXc1cAdwUlEFJzHQrx+ufn3mND4KzBg6830+eGU07Tam4urX93edikjcMTPuuqAjO3Zncc9HKlwvEs+iTd4aAD8E/98G5PR4/RA4q7CDkhhKSiLzvQ948dgLOXHVPG7//Pm9iZtNnAhJSbGOUEQKqE29qozs2Zr/zl3LNys3xzocESmgaJO3NfgEDiAF/4w3gK6Azr+XMu8nNGRj5RqMOqUt3HknvPYalpoK3brFOjQROUTXnNqGZrUqc8t7C9m1JyvW4YhIAUR7t+lEoDcwA3gCeMXMrgRaAo8XUWwSA9nZjmenr6Rdw2qcdPXZoJI6IqVKpcQE7ji/PUPHzOT56Slcc2rbWIckIvkU7UN6bwj5/xtmtg44EVjmnBtfVMFJ8fvf0o2s3LSdRwd2Vi1EkVLq5CPqcXbHBjz++QrOO7qxCteLxJk8L5uaWaKZvWJme28zdM5Nc87do8St9Hlm2koa10jiHNVBFCnV/nlue8qXM259fyEuuLNcROJDnsmbc243cB6gX3cpN2vVFmb/9BtX9WhJ+YRou0OKSDxqUL0S1595BNOWbeKThetjHY6I5EO0e+j38QmclGLPTEuhZuVE+ndrGutQRKQYXHF8c9o1rMa/Ji5i2649sQ5HRKIUbfK2FLjNzN40s7+Y2R9D/6KdmJn1MbMfzWyFmd0c4f2HzWxe8LfMzNLC3q9mZmvN7IlopynRWbFxK58t2cCQ41tQuYKqpomUBTmF6zdu3cVDk5fFOhwRiVK0e+n/wz8SpFfwF8oBT+U1AjNLAJ4EzsA/emSmmX3gnFu8d0TOjQ5pfy3QJWw0dwLTooxZ8uG56SlUSizHkOObxzoUESlGXZrV5NLuzRj7jS9c36GxCteLlHRRnXlzzjU8yF+jKKfVHVjhnEtxzmUCbwLnH6T9IOCNnBdm1hWoD0yOcnoSpfXpO5kwdy39k5tSu2rFWIcjIsXspt5HUqtKBf7x3kKystW9WaSkK85e6Y2B1SGv1wTDDmBmzfHPkPs8eF0OeBD4SxHHWCaN+TqVrGzHVT1Ut1SkLKpeOZFbzmnH/NVpvDFDhetFSrqoLpua2b8P9r5zLpri9JEeGpbbId5AYLxzLufx338EPnbOrT7Ys8fM7GrgaoBmzZpFEZL8vnM3r33/M+d0akTTWnrWk0hZdX7nRrw9azX3f7qU3u0bUPcwnYUXKamiPfPWI+zvVGAkPlGKtjD9GiD0NsYmwLpc2g4k5JIpcDxwjZmtAh4AhpjZfeEfcs4955xLds4l161bN8qwyrbXv/+Zbbv2MLKnzrqJlGVmxp0XdGDX7mzu+ViF60VKsmgrLBwfPszMqgBjgE+jnNZMoK2ZtQTW4hO0SyOM9wigJvBtyPQvC3l/KJDsnDvgblXJn117snjpq1ROalNHnZRFhNZ1qzKqVyse+3wFl3Rtwglt6sQ6JBGJoMB93pxz24E7gNuibL8HuAaYBCwB3nbOLTKzO8ws9Blyg4A3nR75XeTem7uWjVt3MbKXzrqJiPfHU1S4XqSks0PJkczsROBD51zNwgupcCQnJ7tZs2bFOowSKzvbcfrD00hKTODDa09SHVMR2WvqjxsZOmYmN5xxONeepsL1IsXBzGY755KjaRvtDQvhD+I1oCFwBXp0R1z6bMkGUjZt57FBXZS4ich+Tj6iHud0bMjjX6zgvM6NaF67SqxDEpEQ0T6k99aw19nAJuAd/KVTiTPPTk+hSc0kzu7QINahiEgJdOu57Zi2bBP/fH8RY4d100GeSAlS0If0NnbOdXbOXe+cS8t7DFKSzNxbgL6VCtCLSEQNqlfihjMPZ9qyTXz8gwrXi5QkUe25zaxcUN4qfHhC8ABdiSPPTltJzcqJXJLcJNahiEgJNvi45rRv5AvXb925O9bhiEgg2sRrPHB9hOF/xl86lTixfMNWPluykStOUAF6ETk4X7i+I5u27eKhKSpcL1JS5OchvZGe5zaJ6B/SKyXAvgL0LWIdiojEgc5Na3D5sc15+ZtVLFybHutwRITok7eqQGaE4XuAaoUXjhSlX9J38N68tQxIbkqtKhViHY6IxIkbex9BrSoV+ceEH1S4XqQEiDZ5Wwj0jzC8P7C48MKRojTm61VkOxihAvQikg/VkxK59dyjmL8mnddVuF4k5qLt9HQXMN7MWgCfB8NOAy4HBhR+WFLY0nfs5vXvf+acjg1VgF5E8u28o33h+n9/upTe7etT77BKsQ5JpMyK9lEh7wOXAEcDY4O/TsAA59yEogpOCk9OAfqrVYBeRArAzLjz/KBw/UcqXC8SS1E/5sM5955zLtk5lxj8JStxiw+79mTx0tep9GirAvQiUnCt6lZl1MmteW/eOr5esTnW4YiUWdE+5+2EoI5p+PATzez4wg9LCtOEOWvZtHUXI3u2jnUoIhLn/nhya5rXrsytKlwvEjPRnnl7FKgfYXjd4D0pobKzHc9NT6F9o2qc2KZ2rMMRkThXKTGBO87vQMrm7Tw7LSXW4YiUSdEmb+2AeRGGzweOKrxwpLBNWbKBlM3bGdWrtWoTikih6HV4Xc7t1JAnvljBqs3bYx2OSJkTbfK2A6gXYXhD/LPepARyzvHMtJU0rZXEWSpALyKF6NZz21EhoRy3vr8Q5/TsN5HiFG3y9hlwj5kdljPAzKrhHyEypSgCk0M366ffmPtzmgrQi0ihq1+tEjeeeThfLt/MRz/8EutwRMqUaPfofwFaAT+Z2RQzmwKkBsNuLKrg5NA8M3UltapU4JKuTWMdioiUQoOPb0HHxtW5Y+JiFa4XKUbRPudtNdARuA34GVgd/L+Tc06P2y6Blm3Yyv+WbuSK41uQVCEh1uGISCmUUM64u18HNm3bxYOTVbhepLhEW2EB59xW4PHw4WZ2knPuq0KNSg7Zc9NTSEpMYMjxzWMdioiUYp2a1GDwcc155dtVXHRMEzo20bMkRYpagTpCmVldM7vRzJYC0wo5JjlEv6Tv4P15axnQrSk1VYBeRIrYjb2PoHbVivzjPRWuFykOUSdv5p1lZv8F1gBXAK8DHYoqOCmYl75KJdvB8JNaxjoUESkDqlVK5JZzjmLBmnRe//6nWIcjUurlmbyZWQszuwPf1+0FICX43ADn3B3OORW5K0FUgF5EYuG8oxtxUps6/PvTH9m4dWeswxEp1Q6avAV3lS7A31U6AmjqnNPdpSXYa9//xPbMLEb2UgF6ESk+ZsYd57dn155s7vpQx/QiRSmvM2+nAM8DtzvnJjnnsoshJimgnbuzGPP1Knq0rUP7Ruo0LCLFq1Xdqvzh5NZ8MH8dXy1X4XqRopJX8tYDqAHMMbNvzewPZlarGOKSAnhvri9AP6qXCtCLSGz84eTWtKhdmVvfX8jO3SpcL1IUDpq8Oee+dc4Nx5fBegl/k8IvwedON7OqRR+iRCMrKEDfoXE1TmitAvQiEhuVEhO484IOpKpwvUiRifYhvdudc887544DuuKf93YrsNHMJhRlgBKdKYt9AfqRPVWAXkRiq0fbuvQ9uhFPTl1BqgrXixS6fD/nzTm30Dn3Z6ARMAzQLY0xllOAvlmtyipALyIlwq3nHEXFhHL8U4XrRQpdvpM3M+tnZpWdc7udc28553oXRWASvZmrfmPe6jSu6tFSBehFpESoV60SN/Y+gi+Xb+bDBSpcL1KYCrKnfxnQ6Z0S5JlpvgD9xSpALyIlyOXHNfeF6z9czO8qXC9SaAqSvKlDVQny4/qtfK4C9CJSAiWUM+7p15Fft+3iwUk/xjockVJD19jinArQi0hJ1rFJdQYf15xx3/3EgjVpsQ5HpFQoSPJ2AbCusAOR/FMBehGJBzfkFK6fsFCF60UKQVTJm5l9bGbVAZxz/3PO7QyGH2ZmHxdlgJK7l75KxaEC9CJSslWrlMit57bjh7XpvKbC9SKHLNozb72BihGGVwLOKLxwJFrpGb4A/bmdVIBeREq+vp0a0qNtHf7z6Y9s/F2F60UORV6F6duZWTv8TQqH57wO/joCQ9El1Jh4NShAf3VPFaAXkZLPF67vwK6sbO78SIXrRQ5F+TzeXwi44G9a2HsGZAJ/LoK45CByCtD3PLyuCtCLSNxoWacKfzy5NY98tpz+yU3o0bZurEMSiUt5XTY9CmiPT9R6Au1C/loDNZxzzxRphHKACXPXsnnbLkbprJuIxJlRvVrTsk4Vbn1PhetFCuqgZ96ccz8CmFmSc25X8YQkB5NTgL5j4+ocrwL0IhJnKiUmcOf5Hbj8xe95espSRv82D1JToVUr6NcPkpJiHaJIiRftDQtnmdmpOS/M7CYzW2Fm75uZznsXoymL15O6eTsje7VSAXoRiUsnta3DeU0r8vQXy0m99ia49Va47DJcy5Ywc2aswxMp8aJN3u4CKgCY2dHAncArQC3gwaIJTcI553h6WkpQgL5hrMMRESmYHTu45YE/UnH3Lkafcz1PH3sRi+u1xDZswPXtCzt2xDpCkRIt2uStBbA0+P+FwPvOuTuA64AziyAuiWBG6hbmr07jqp6tSCins24iEqcmTKDequVcPvdj5jU+kvtPHsZ5Qx7em8AxYUKsIxQp0aJN3jKBnIeJnQZ8Fvx/C1CtsIOSyJ6dnkLtKhW4pGuTWIciIlJwKSkAHLYrY++gPQnlmd6iy37vi0hk0SZvXwP3m9lfgO5ATlWFtsDaoghM9re3AP0JLaiUqAL0IhLHWvk75U9OnU1C1h4/zDmapG/c730RiSza5O1afDWFEcB1zrk1wfDzgP8VRWCyv2enryQpMYHBx6kAvYjEuX79cPXr025jKhNfGc0fvnmLOhlp3H76SFLbdPR3nYpIrvJ6SC8AzrlVRCiD5Zy7trADkgOtS9vBB/PWMfj45ipALyLxLykJmzgR17cv7Tak0m5jKhct+oJLBv+HwYPv593dRn09MUQkV9GeecPMEs3sXDO7zsyqBcOa5vxfio4K0ItIqdOtG5aaCq+9BnfeSZvH72fsNafw2x5jyIszSMvIjHWEIiVWVGfezKwFMAWoj79xYSLwO3ADkASMLJrwJD1jN2/M+Jm+nRrSpKYK0ItIKZKUBJdeuvfl0cBzQ5IZNmYmV46dyasjjqVyhah2UyJlSrRn3h7F37RQGwh9AM8E/N2nUkT2FaBvHetQRESK3Ilt6vDYoM7MW53GqFfnkLknO9YhiZQ40SZvJwL3Oud2hw3/CWgU7cTMrI+Z/RhUZ7g5wvsPm9m84G+ZmaUFwzub2bdmtsjMFpjZgGinGc98AfpUeh5el3aNdHVaRMqGPh0acu+FHZm+bBPXvz2PrGwX65BESpRoz0cnBH/hmgBboxmBmSUAT+JvfFgDzDSzD5xzi3PaOOdGh7S/Fgge+kMGMMQ5t9zMGgGzzWyScy4tyvjj0n/nrGXztkxG9dJt8yJStgzo1oy0jN3c+8lSalauwB3nt1dJQJFAtGfepuAfF5LDmVkV4Dbg0yjH0R1Y4ZxLcc5lAm8C5x+k/SDgDQDn3DLn3PLg/+uAjUCprqmale14/ssUOjWpzvGtVIBeRK8nA+YAAB0ISURBVMqekb1aM7JXK8Z99xMPT1kW63BESoxoz7zdCEw1swX45729AhyOP+s2OMpxNAZWh7xeAxwbqaGZNQdaAp9HeK87vs7qygjvXQ1cDdCsWbMowyqZJi/yBeifvPQYHW2KSJl1c58jSdu+m8c+X0GNyhW4Unfdi0T9nLefzawTPlHrij9j9xbwsnMuqsumQKQMJLeODAOB8c65rP1GYNYQGAdc4Zw7oBerc+454DmA5OTkuO0k4ZzjmWkraV67Mn06NIh1OCIiMWNm3N2vA+k7dnPHh4upUTmRC49RiUAp2w562dTMXjKzwwCcc9ucc08750Y45650zj2Rj8QN/Jm2piGvmwDrcmk7kOCSaUgs1YCPgFucc9/lY7px5/vULcxfk85VPVSAXkSkfEI5Hh3UmRPb1OYv4xfw2eINsQ5JJKby6vN2Bf45boVhJtDWzFqaWQV8gvZBeCMzOwKoCXwbMqwC/rEkrzjn3imkeEqsZ6etpHaVClysAvQiIgBULJ/As4OTad+oGn96fQ7fp/wa65BEYiav5K3QTvs45/YA1wCTgCXA2865RWZ2h5mdF9J0EPCmcy70smd/oCcwNORRIp0LK7aSZOn63/nix00MVQF6EZH9VK1YnrHDutOkZhIjXp7FonXpsQ5JJCZs/xwp7E2zbKC+c25T8YVUOJKTk92sWbNiHUa+Xf/WPD5ZuJ5v/3YqNSqrjqmISLh1aTu4+OlvyMzK5p1RJ9CyTpVYhyRyyMxstnMuOZq20TwqZL2ZZR3s7xDjlcDatB18MH8dA7s3VeImIpKLRjWSGDfiWLIdXP7C96xP3xnrkESKVTR3m14NlOqH4ZYUOQXoR/TQQ3lFRA6mdd2qvDysOwOf+5YhL33P2yOP10GvlBnRJG8TnXMbizySMi6nAP15RzeicY3CukdERKT06tikOs9fkczQMTMZNnYmr6mQvZQReV02jdtnpcWbcd+tIiMzi6t76qybiEi0Tmhdh8cHdWH+6jRGjputQvZSJhTb3aaSu527sxj7zSp6HV6XoxqqAL2ISH70bt+A+y7qxJfLNzNaheylDDjo+WXnXLS1T+UQvDtnTVCAvnWsQxERiUv9k5uSnrGbuz9eQvWkRO6+oINKC0qppc4BMZaV7Xh+egpHN6nOca1qxTocEZG4dVXPVmzJyOTpqSupVbkCN/Y+ItYhiRQJJW8xNnnRelb9msFTl6kAvYjIobqp9xGkZWTyxBcrqFE5UXfvS6mk5C2GQgvQ926vAvQiIofKzLjrgo6kZezmro+WULNyBS5SqUEpZdSnLYa+S1EBehGRwpZQznhkYGdOalOHm95dwBQVspdSRslbDD07fSV1qqoAvYhIYfv/9u48uqry3OP49yEDJCCTTDInMkTEAQlgHRAHBKygqLU4z9h7tVpbrdg79F47obgcam0VRbSiUmulyrWAIiiiCASoghCmhCHMMkMCmZ77xzl4Y64gCcnZZ5/8PmtlkbPPe9gPawP7l73fdz+RRva96dmuCXe9tpDP1MheEojCW0CWbdrDh2pALyJSaxrWT+alm/vQsXk6d7ycw5INamQviUHhLSBjZ+WRnprE9Wd2CroUEZGE1axhKq/c1pfGaSnc9OI88rbtC7okkWOm8BaAgp2FkQb0fTqqF5+ISC07oUkar9zWF4Abxs1j0+6igCsSOTYKbwF4cfYaDLjt3IygSxERqRMyWzbi5Vv7sruohBvHzWPn/uKgSxKpNoW3GNtVWMzE+WpALyISaz3bNeH5G7NZu6OQW16az/6DpUGXJFItCm8xNuGztZEG9OfpwZEiIrH2vROP5w/X9GLxht38aMICDpaWBV2SSJUpvMXQgZIyxn+yhgHdW5LVRg3oRUSCcPHJbXgk2sj+p3/5XI3sJXTUYSGG3lxQwPb9xdzZXw3oRUSCdFXv9uwqLObX7y6jcVoKvx2uRvYSHgpvMVJW7jz/cR6ndWiqBvQiInHg9nMz2VlYzDMzV9O8YQoPDMoKuiSRo6LwFiPTvtzM2u2FjBqcpZ/uRETixP0Xd2dnYQnPzFxNs/RUNbKXUFB4i4FDDeg7H5/OxWpALyISN8yMX13Wk93RRvZN0lL4QXaHoMsSOSKFtxiYk7edLwp285vhPdWAXkQkziTVMx7/4WnsOVDCqLcW0yQtRT9oS1zTatMYeO6jPFo0SuXKM9SAXkQkHtVPTuLZ63tzSrsm3P36IuasViN7iV8Kb7Vs2aY9fLRiG7ecnaEG9CIicaxh/WTG39yHTs3TuePPOSwuUCN7iU8Kb7Xs6wb0/dSAXkQk3kUa2fejSVoKN42fx2o1spc4pPBWiw41oL+mb0eapKcEXY6IiByFNk0aMOH2fhhw47h5bNylRvYSXxTeatG42fmRBvTnqAG9iEiYZLRoyMu39mVPUQk3jJvLDjWylzii8FZLdu4vZuK89Qw7vS1t1YBeRCR0erZrwgs3ZVOws4hbxs9jnxrZS5xQeKslEz5bS1FJGSP764GPIiJh1S/zeJ659gyWbNzDna/kqJG9xAWFt1pwoKSMlz5dw/lqQC8iEnoX9WjNo1eeyiertvOTif9UI3sJnMJbLfjroQb056kBvYhIIriyd3v+49IeTFmymX+btBh3BTgJjjos1LCycuf5WZEG9P0y1IBeRCRR3HZOBrsKi3l6xiqaNUzlwcFqZC/BUHirYVOXbGbdjkJ+cYka0IuIJJqfDuzGjv3F/OnD1TRLT2Fkf91hkdhTeKtBhxrQZ7RoyMAe6osnIpJozIyHL+vJ7qISfvuPXJqmpXJ1HzWyl9hSeKtBc1ZvZ/GG3fx2+ClqQC8ikqCS6hmPX306u4tKGPXWFzROS2FwT/3ALrGjBQs16NlZkQb0V5zRLuhSRESkFqUm1+O5G3pzWoem3PP6Ij5d/VXQJUkdovBWQ5Zu3MMsNaAXEakz0lMjjew7t0jnjpdz+KJgV9AlSR2h8FZDxs5aTUM1oBcRqVOapkca2TdrmMrN4+ezaqsa2UvtU3irAQU7C5n8xSY1oBcRqYNaN27AhNv6Uc/gxnFz1cheap3CWw144eNIA/pb1YBeRKRO6hxtZL/3QKka2Uut02rT6ioshEmT2Jm3jr8UnsKwU09QA3oRkTrs5LZNGHdzH24YN5ebx8/jtetOpdE/JkN+PmRmwvDhkKbzRKhEz/XxdgwV3qpj/nx86FBsyxZeOWsEReeeysj/HgldnoU+fYKuTkREAtI3ozl/vO4MRv45h5H3juXF135Bg7ISALx1a2zyZJ0nwqLCuf6QeDmGum1aVUVFXx/MhSd050/9rqLP+iVkLV+IDx0KRZrrICJSl13YuTFjPnqeT9uexC0/+C/+eOZVLG2VgW3ZovNEWFQ413/ZKpNn+10ZV8dQ4a2qJk36+mBedf2jFKU2YGHbrK8PKpMmBV2hiIgEadIkrpjzNnd+9iZzOp3Go+fdzNAbn9B5Ikyi5/o5HXpy6U1PMnrALQyLo2Oo8FZVeXkAfJzRi/J6kee5lSUlM6tzr2+8LyIidVT0PNDswN6vN5UlJTPm3Bs4kJyq80QIFK3O55kzf8DNVz+M14tEpdI4OtcrvFVVZiYA/fMXklxWCkByWSn91yz6xvsiIlJHfct5wrycmV36csEdz/LXpt0pK/cgK5TDKC0r5y/z13F+8amMOe8mem3MJSkOz/Xmnph/gbKzsz0nJ6fmf+OiIjwjctl0aasMZnXuRf81i+ixNT8ykTE/Py5WooiISEAOc57Y1aARoweO5IsWGWS1OY4HB2cxoHtLzNQLO2juzvRlW3l0ai4rt+7jjPaNeXDsL+i3eHbMzvVmtsDds49qbCzDm5kNBp4CkoAX3H10pfefAM6PvkwHWrl70+h7NwH/Hn3v1+7+8pH2VWvhDeJ6BYqIiMSBw5wneOcd3m3QnjHTlrN2eyH9Mprz0CUncXqHpgEWW7ctWLuD0VNymb9mJ5ktG/LzQVkMOrk1lpMT03N9XIY3M0sCVgADgQJgPnCNuy89zPgfA73c/VYzaw7kANmAAwuA3u6+83D7q9XwBpGVJpMmRe57x9GzX0REJE4c4TxRXFrOxPnreGr6SrbvL+aSU9rwwKAsMlo0DLjoumPV1n08OjWX95ZuoeVx9bnvom5cnd2e5KQKM8pieK6P1/D2PeC/3H1Q9PVDAO7+u8OM/xT4pbu/b2bXAAPc/c7oe88BH7r764fbX62HNxERkWO072Apz8/K4/mP8yguLWdE3w7ce2E3Wh5XP+jSEtaWPQd4cvpK3shZT1pKEnf2z+S2czNITw320bdVCW+xrLQdsL7C6wKg37cNNLNOQAYw4wifbfctnxsJjATo2LHjsVcsIiJSixrVT+a+gd247syO/P6Dlbw+bz1vLdzA7edmMrJ/Jo3q61n6NWXPgRLGfpTHC7PzKCt3bjizEz++oAvHNwpfUI7l34pvm5F5uMt+I4A33b2sKp9197HAWIhceatOkSIiIrHW6rgG/PryU7j17Awee285v/9gJa/NXcs9F3ZlRJ+OpCbr4RDVdbC0jFc/W8fTM1ays7CEYae15f6Lu9Px+PSgS6u2WIa3AqBDhdftgY2HGTsCuKvSZwdU+uyHNVibiIhI4DJbNuKP1/Vm0bqdjJ6Sy3++/SXjZufzwKDufP+UE7QytQrKy53JX2zksfeWs35HEed0acGoIVn0bNck6NKOWSznvCUTWbBwIbCByIKFa939y0rjugPTgAyPFhddsLAAOCM6bCGRBQs7Drc/zXkTEZEwc3c+XL6N0VNyWb5lL6e2b8KoIVmcdWKLoEuLe7NXfsXoqctYsmEPPU5ozKghWfTv1jLoso4oLue8uXupmd1NJJglAS+6+5dm9jCQ4+7vRIdeA0z0CqnS3XeY2a+IBD6Ah48U3ERERMLOzDg/qxX9u7XkrYUFPP7+Cq59fi4DurfkwcFZnHRC46BLjDtLNuzmkam5fLzyK9o3S+PJH57OsNPaUq9eYl2x1EN6RUREQuBASRkvf7qGZ2auYu/BUob3asfPLu5Ou6Z6TNX6HYU89t5y3v7nRpqlp3D3BV25/syO1E9OCrq0oxaXjwqJNYU3ERFJRLsLS/jjh6sY/+kaAG76XifuOr8LTdNTgy0sADv2F/P0jJVM+GwtSfWM287J4M7zTqRxg5SgS6syhTcU3kREJLFt3FXE4++v4G8LC2hUP5l/HdCFW87uTIOU8Fxtqq7C4lJenJ3Pcx/lsb+4lKuzO/CTi7rRpkmDoEurNoU3FN5ERKRuyN28h0em5DJz+TbaNG7ATwd248re7UlKsHleEGkc/9cFBTzx/gq27j3IwB6teXBwd7q0Oi7o0o6ZwhsKbyIiUrfMWb2d0VNz+Xz9Lrq1bsTPB2Vx4UmtEuLxIu7Oe0u38OjUXFZv20/vTs14aEgW2Z2bB11ajVF4Q+FNRETqHndnypLNjJm2nPyv9tO3c3NGXZLFGR2bBV1ateWs2cHvpuSyYO1OTmzZkAcHZzGwR+uECKUVKbyh8CYiInVXSVk5E+ev56npK/lq30GG9GzD/YO6c2LLRkGXdtRWbd3LI1OX8/7SLbRuHGkcf1XvSo3jE4jCGwpvIiIi+w+W8sLH+YydtZoDpeWM6NOBey/sSqvG8Tuxf/PuAzw5fQVv5KynYWoyPxpwIreenUFaamIvxFB4Q+FNRETkkG17D/L0jJW8NncdKUn1uOPcDO7on8lxcfRIjT0HSnj2w9W8+El+tHF8Z+6+oAvNG9aNR6AovKHwJiIiUtmar/Yz5r3lvPvFJo5vmMqPL+jCtf06Bdr4/mBpGa/MWcsfZq5iV2EJl5/elp9d3J0OzcPbOL46FN5QeBMRETmcz9fvYvSUXObkbadj83TuH9SdS085IaZtpMrLnbc/38Bj01awYVcR53ZtwYODE6NxfHUovKHwJiIiciTuzkcrIo3vczfv5ZR2kcb3Z3ep3cb37s6slV8xekouyzbt4eS2jXloyEmc07V29xvvFN5QeBMRETkaZeXO3xdt4PH3I1fA+ndryajBWfRoW/ON7xcX7Gb01GV8smo7HZqncf/F3Rl6auI1jq8OhTcU3kRERKriQMn/zT3bc6CEy09vx08HdquRuWfrtkcax7/z+Uaafz3XLlyN42ubwhsKbyIiItWxu7CEP320mvGf5OMON0Yb3zerxqrP7fsO8vSMVbw6N9I4/vZzMrnzvPha5RovFN5QeBMRETkWG3cV8eT0Fby5oICG9ZP5lwEncstZR/e8tcLiUsZ9nM9zs/IoKimLNo7vSus4fr5c0BTeUHgTERGpCcs372XMtFymL9tKm8YNuG9gV648oz3JBw/ApEmQnw+ZmTB8OCWp9XkjZz1PTl/Jtr0HGXRyax4YlEWXVuHp7BAUhTcU3kRERGrS3LxI4/tF63bR9bgkHvj7EwzMeQ8DHJjaZwhjht1D3r4ysjs146FLsujdKXEax9c2hTcU3kRERGqauzNt0ToeeX46+U3acPLm1XTeuYHVx3cgt1UGXXZt5MEfDeKi0zokXOP42laV8JaY3V1FRESkxpkZg3M/4b2xd3L3pxP5snUm757Un9yWnbl39qtMHfsjBi6dreBWyxTeRERE5Ojl5ZFSXkaj4iI4FNLMSCs5SLKXQ15esPXVAQpvIiIicvQyMwHon7+Q5LJSAJLLSum/ZtE33pfaozlvIiIicvSKivCMDGzLFpa2ymBW5170X7OIHlvz8datsfx8SEsLusrQqcqct+TaLkZEREQSSFoaNnkyPnQoPbbk02NrPkAkuE2erOAWAwpvIiIiUjV9+kSusE2aFJnjlpmJDR+u4BYjCm8iIiJSdWlpcO21QVdRJ2nBgoiIiEiIKLyJiIiIhIjCm4iIiEiIKLyJiIiIhIjCm4iIiEiIKLyJiIiIhIjCm4iIiEiIKLyJiIiIhIjCm4iIiEiIKLyJiIiIhIjCm4iIiEiIKLyJiIiIhIjCm4iIiEiIKLyJiIiIhIjCm4iIiEiIKLyJiIiIhIjCm4iIiEiIKLyJiIiIhIjCm4iIiEiIKLyJiIiIhIjCm4iIiEiIKLyJiIiIhIjCm4iIiEiIxDS8mdlgM1tuZqvMbNRhxlxtZkvN7Esze63C9kej25aZ2e/NzGJXuYiIiEh8SI7VjswsCXgGGAgUAPPN7B13X1phTFfgIeBsd99pZq2i288CzgZOjQ6dDZwHfBir+kVERETiQSyvvPUFVrl7nrsXAxOByyqNuQN4xt13Arj71uh2BxoAqUB9IAXYEpOqRUREROJILMNbO2B9hdcF0W0VdQO6mdknZvaZmQ0GcPc5wExgU/Rrmrsvi0HNIiIiInElZrdNgW+bo+aVXicDXYEBQHvgYzPrCbQATopuA3jfzPq7+6xv7MBsJDASoGPHjjVXuYiIiEiciOWVtwKgQ4XX7YGN3zLmbXcvcfd8YDmRMDcc+Mzd97n7PmAKcGblHbj7WHfPdvfsli1b1sofQkRERCRIsQxv84GuZpZhZqnACOCdSmP+DpwPYGYtiNxGzQPWAeeZWbKZpRBZrKDbpiIiIlLnxCy8uXspcDcwjUjwesPdvzSzh81sWHTYNGC7mS0lMsftAXffDrwJrAYWA58Dn7v75FjVLiIiIhIvzL3ytLPEkJ2d7Tk5OUGXISIiIvKdzGyBu2cfzVh1WBAREREJEYU3ERERkRBReBMREREJkYSd82Zm24C1QdeRAFoAXwVdhBwTHcPw0zEMNx2/8IvFMezk7kf1nLOEDW9SM8ws52gnUEp80jEMPx3DcNPxC794O4a6bSoiIiISIgpvIiIiIiGi8CbfZWzQBcgx0zEMPx3DcNPxC7+4Ooaa8yYiIiISIrryJiIiIhIiCm/yrcysg5nNNLNlZvalmd0bdE1SdWaWZGaLzOx/gq5Fqs7MmprZm2aWG/23+L2ga5KqMbP7ov+HLjGz182sQdA1yZGZ2YtmttXMllTY1tzM3jezldFfmwVZo8KbHE4p8DN3Pwk4E7jLzHoEXJNU3b3AsqCLkGp7Cpjq7lnAaehYhoqZtQPuAbLdvSeQBIwItio5Ci8BgyttGwV84O5dgQ+irwOj8Cbfyt03ufvC6Pd7iZw02gVblVSFmbUHvg+8EHQtUnVm1hjoD4wDcPdid98VbFVSDclAmpklA+nAxoDrke/g7rOAHZU2Xwa8HP3+ZeDymBZVicKbfCcz6wz0AuYGW4lU0ZPAz4HyoAuRaskEtgHjo7e+XzCzhkEXJUfP3TcAjwHrgE3Abnd/L9iqpJpau/smiFzcAFoFWYzCmxyRmTUC/gb8xN33BF2PHB0zuxTY6u4Lgq5Fqi0ZOAP4k7v3AvYT8K0aqZrovKjLgAygLdDQzK4PtipJBApvclhmlkIkuL3q7m8FXY9UydnAMDNbA0wELjCzCcGWJFVUABS4+6Er3m8SCXMSHhcB+e6+zd1LgLeAswKuSapni5mdABD9dWuQxSi8ybcyMyMy12aZuz8edD1SNe7+kLu3d/fORCZIz3B3/cQfIu6+GVhvZt2jmy4ElgZYklTdOuBMM0uP/p96IVp0ElbvADdFv78JeDvAWkgOcucS184GbgAWm9k/o9t+4e7/CLAmkbrmx8CrZpYK5AG3BFyPVIG7zzWzN4GFRFbwLyLOntQv/5+ZvQ4MAFqYWQHwS2A08IaZ3UYklP8guArVYUFEREQkVHTbVERERCREFN5EREREQkThTURERCREFN5EREREQkThTURERCREFN5ERKrBzGab2ZNB1yEidY/Cm4iEmpm9ZGb/U2nbpWZWaGa/CaouEZHaovAmIgnFzG4g0tbtIXf/t6DrERGpaQpvIpIwzOxe4AXgdnd/6jBj7jKzjWZWr9L2N8zsb9Hvu5rZO2a2xcz2mdkCMxvyHfsuMLOfVNr2jVurZlbfzMaY2QYz229m88zsogrvp5rZH8xsk5kdNLP1unooIpUpvIlIQjCzXwG/A65w91eOMHQi0AK4oMJnGwNDgQnRTccB7xJpLN6LSB/Dt82s6zGW+WciredGAKcCrwLvmlnP6Pv3Reu4GugWHbfyGPcpIglGvU1FJBEMBL4PXOru7x5poLtvN7NpwHXA9OjmK4AiIoENd19IpB/lIQ+b2TDgSiI9DqvMzLoR6YfY3t03Rjc/ZWYDgZHAPUAnYDkw2yO9C9cCn1RnfyKSuHTlTUQSwRJgNfBLM2t6aKOZDYje9jz09cPoWxOAK8ysQfT1dcBf3b04+rlGZvaYmS0zs11mtg84Heh4DDX2BgxYUbEmYBBwYnTMeCAbWG5mT5vZkMq3d0VEdOVNRBLBJmAYMAOYbmYD3X0nMJdI6Dpkc/TXt4GxwFAzmw2cDzxcYdwTRG6rPgCsAgqJ3OJMPUIN5UTCWUUpFb6vB5QRCXFllcYVArj7fDPrDAyO7n8CkGNmg6NX4kREFN5EJDG4+wYzG0AkwH0QDXDbiYSvymMPmNlbRK64tQcKgNkVhpwDvOTubwGYWTqQCSw+QgnbgBMOvTCzNCLz1uZENy0EkoBW7v7xEf4ce4A3gDfM7JVoXRlA3hH2LSJ1iC7Hi0jCcPdNwAAiV8hmmFmLIwyfAAwhMt/s1UpXtlYQua3ay8wOLSyo/x27nwHcYGb9zexkIrdAkyrUtgz4C/BnM7vSzDLMrI+Z/dzMLgcws/vNbISZZUUXR1wD7AY2/v/diUhdpfAmIgnF3bcQuQ0KMNPMWh1m6ExgK5DF/60yPeReYCeRxQLvArOAT79j17+JjpsMTIv+/l9UGnMj8AowhsjChHeIrD5dG31/H/AgkBP96gkMdvcD37FvEalDTNMoRERERMJDV95EREREQkThTURERCREFN5EREREQkThTURERCREFN5EREREQkThTURERCREFN5EREREQkThTURERCREFN5EREREQuR/ARI5hGeu/pdBAAAAAElFTkSuQmCC\n", 
                        "text/plain": "<matplotlib.figure.Figure at 0x7f661c164860>"
                    }, 
                    "metadata": {}
                }
            ], 
            "source": "import matplotlib.pyplot as plt\nplt.figure(figsize=(10,6))\nu = list(accuracy_dict.keys())\nv = list(accuracy_dict.values())\nplt.scatter(x=u, y=v, linewidths=2, edgecolor='red')\nplt.plot(u,v)\nplt.xlabel('K-values', fontsize=14)\nplt.ylabel('Test-Accuracy', fontsize=14)\nplt.title('KNN model showing Test-Accuracy levels for corresponding K-values', fontsize=16)\nplt.show()"
        }, 
        {
            "source": "#### Therefore The best Test accuracy Score for KNN model is  = 0.7765 and The best k = 4", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "# Decision Tree", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "First let's import Decision tree classifier from Sklearn", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 38, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "from sklearn.tree import DecisionTreeClassifier"
        }, 
        {
            "source": "Let's view our X and y variables", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 39, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 39, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "array([[ 0.51578458,  0.92071769,  2.33152555, -0.42056004, -1.20577805,\n        -0.38170062,  1.13639374, -0.86968108],\n       [ 0.51578458,  0.92071769,  0.34170148,  2.37778177, -1.20577805,\n         2.61985426, -0.87997669, -0.86968108],\n       [ 0.51578458, -0.95911111, -0.65321055, -0.42056004, -1.20577805,\n        -0.38170062, -0.87997669,  1.14984679],\n       [ 0.51578458,  0.92071769, -0.48739188,  2.37778177,  0.82934003,\n        -0.38170062, -0.87997669,  1.14984679],\n       [ 0.51578458,  0.92071769, -0.3215732 , -0.42056004,  0.82934003,\n        -0.38170062, -0.87997669,  1.14984679]])"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "X[0:5]"
        }, 
        {
            "execution_count": 40, 
            "cell_type": "code", 
            "metadata": {
                "scrolled": true
            }, 
            "outputs": [
                {
                    "execution_count": 40, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "0    0\n1    0\n2    0\n3    0\n4    0\nName: loan_status, dtype: int64"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "y[:5]"
        }, 
        {
            "source": "Let's confirm the shapes of our variables", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 41, 
            "cell_type": "code", 
            "metadata": {
                "scrolled": false
            }, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "X_train shape is (252, 8)\nX_test shape is (94, 8)\ny_train shape is (252,)\ny_test shape is (94,)\n"
                }
            ], 
            "source": "print('X_train shape is',X_train.shape)\nprint('X_test shape is',X_test.shape)\nprint('y_train shape is',y_train.shape)\nprint('y_test shape is',y_test.shape)"
        }, 
        {
            "source": "Let's print out the first five rows of X and y", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 42, 
            "cell_type": "code", 
            "metadata": {
                "scrolled": true
            }, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "X_train [[-1.31458942 -0.95911111  0.17588281  2.37778177  0.82934003 -0.38170062\n   1.13639374 -0.86968108]\n [ 0.51578458  0.92071769  0.17588281 -0.42056004  0.82934003 -0.38170062\n   1.13639374 -0.86968108]\n [ 0.51578458  0.92071769  0.50752015 -0.42056004  0.82934003 -0.38170062\n  -0.87997669  1.14984679]\n [-1.31458942  0.92071769 -0.81902922 -0.42056004  0.82934003 -0.38170062\n  -0.87997669  1.14984679]\n [-5.89052444 -1.96168647 -0.3215732  -0.42056004  0.82934003 -0.38170062\n  -0.87997669  1.14984679]]\n\ny_train 115    0\n16     0\n309    1\n17     0\n8      0\nName: loan_status, dtype: int64\n"
                }
            ], 
            "source": "print('X_train', X_train[:5])\nprint()\nprint('y_train', y_train[:5])"
        }, 
        {
            "source": "Next let's define a simple method that takes in our parameters and creates a model with the best test accuracy", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 43, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "accuracy_dict={}\ndef decision_tree(max_depth, X_train, y_train, X_test, y_test):\n    \"\"\"Method takes max_depth an int, and X_train and y_train 2D/1D arrays,\n    as well as test data x_test and y_test, also arrays, and returns,\n    The best max_depth value, from 1 to max_depth, and its best test_accuracy score\"\"\"\n    \n    for i in range(1, max_depth+1):\n        # Let's instantiate a model\n        decision_model = DecisionTreeClassifier(criterion='entropy', max_depth = i)\n        # Let's train the model\n        decision_model.fit(X_train, y_train)\n        # Let's make prediction on the test data\n        y_hat = decision_model.predict(X_test)\n        # Let's measure accuracy of predictions on test data\n        test_acc = round(metrics.accuracy_score(y_test, y_hat),2)\n        accuracy_dict[i] = test_acc\n        \n    key_max_acc = [0,0]\n    for key, value in accuracy_dict.items():\n        if value > key_max_acc[1]:\n            key_max_acc[0] = key\n            key_max_acc[1] = value\n        \n    return key_max_acc "
        }, 
        {
            "execution_count": 44, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "decision_tree_best_depth_and_acc = decision_tree(10, X_train, y_train, X_test, y_test)"
        }, 
        {
            "execution_count": 45, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 45, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "[1, 0.76000000000000001]"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "decision_tree_best_depth_and_acc"
        }, 
        {
            "execution_count": 46, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 46, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "DecisionTreeClassifier(class_weight=None, criterion='entropy', max_depth=1,\n            max_features=None, max_leaf_nodes=None,\n            min_impurity_decrease=0.0, min_impurity_split=None,\n            min_samples_leaf=1, min_samples_split=2,\n            min_weight_fraction_leaf=0.0, presort=False, random_state=None,\n            splitter='best')"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "decision_model = DecisionTreeClassifier(criterion='entropy', max_depth = 1)\ndecision_model.fit(X_train, y_train)"
        }, 
        {
            "source": "Plotting the various max_depth values and corresponding acc scores from 1 to 10. This shows max_depth=1 as the best value for highest test accuracy score", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 47, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "display_data", 
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnEAAAGJCAYAAADlpGXRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzs3Xl8VNXdx/HPLwkJS9iTAWQNCokIAgqodSnEDbUutIob2t22T9un7rWL1r3WVm2rfbTWWlux7uJW6wa4CwIKKDsCYd9B9oQk5/nj3OAwTJIJTHIzk+/79ZpXMnf93Tt37vzuOeeea845RERERCS1ZIQdgIiIiIjUnZI4ERERkRSkJE5EREQkBSmJExEREUlBSuJEREREUpCSOBEREZEUlDZJnJndaGYueFWa2SYzm2Jmt5lZ53pa5yNmNrWOMa6vj1jirMsl8BreELE0FmZ2n5nNreM8RcG+OukA1jsyWMYh+7uMA1j3Acdfh3WNTOCY25XE9Z1lZj/cj/muCGK5N1mxSGLM7Lhg3w8KO5Z0Frufzaxd8H5M2LE1ZmbWLdhP59TDstsEOUBRzPBBwTqP25/lZiUnvEbjC2Bk8H9b4AjgR8BlZjbSOTctyeu7BWhRh+kfAl5KcgzVOSbq/xbABOBW4D9Rw2c3UCwSniX4Y6EhPusP2fu4GwHcDpwBbAyGVSZxfWcBQ4AH6jjfhcHf0WZ2uXOuIokxiTRGW/HfzYVhB9KEtQF+A0wH6lSYUJN0S+LKnXOTot6/Zmb3A+8AT5pZYTJP2M65z+s4/XJgebLWX8u69uwHM8sN/v08Zv/EZWbNnXNJKzGR8ASfY62feZLW9UX0usysW/Dvx8651Q0RQ23M7GBgKDAeOBEoBt4INaiAmbVwzu0MO450ZWY5QJmL6eHezDKAZs650nAiq3/B716DnAekYaVNdWp1nHObgWuBg4GTq4abWXMzu9PMlplZqZnNMLPTY+c3s++b2admtsvM1pjZM2bWNhi3V3VqUGT9kJmtDKZfamZ/ixq/T3WqmRWY2fNmtsXMtprZS7HVbkFR68/M7HYzW2dma83sL8FJ6YCY2Q+D5R9hZu+a2U7gp8G4lmZ2t5mtCPbRx2Z2cpxl/MjM5gTTLDazyxNY7+qgqvv6YL9uNrPfBuPODpa3JdjfbWLm7RPsp63BNOPMrCBmmo5m9pSZbQ8+j2uqiaPAzJ4O1r/dzP4T/NAnzLwbzGxR8LmvNrNXzKxjzKSdgli3m9nnZva9OMu62MxmBftyaXDMZAbjWptZuZl9PWr6e4LP75SoYX8zs/HB//tUpwbx3Wpm1wb7ZqOZPWpmrWNiOcLMJgfb9KmZnWxmn5lZXUu+qttvJ5vZh2a2Mziu7zWz5lHjI0Fcq4MYFpvZPcG4PwLfBQbal1W1f0xgtRfiSwO/C2zmy1K56Lgs+L7NCda7ysz+Hf19M7NTzex9M9thvunGmxZUk5jZH81sYcwy96nOCo6535j/Xq8GlgXDi83sv8F2bzWzqRanesfM+prZs8Hnt9389/PsYNy8qn0VM88LZjahup1jZoODZa4Itm2mmX0/Zppzgm0ZFnwPt5vZAjO7NM7yrgmOsW1m9jQQ+52oLo4u5s+va4PjY1b098XM2prZg8Fxs9PMPjCz42OWMd38+fhKMysBdgK5VZ+PmZ1iZtOBXcApwTydzOyfZrY+WO5bZnZ4zHL/N9i/u4L1v2nB+ce+rBo7x/y5a1uw/VfH2cYzgs9sVzDN3THHWNL2czXHX9X++b6ZLTGzL8z/FuXHzNvXzCYG+2O+mZ0XbPPztXyGVcv/qfnf2a1m9oCZZZnZ8GD8NjN71cw6Rc2XZf68Njs4Bpea2cNm1iFqmlOC7Yk+F3YKPo/7aoorJsZLzZ9XdpjZG0Cvaqar+sxLzZ+7fxwz/vlgn1wcHFs7zWy8mfUOxrcj+H4D4+zLc1a7qMW0MbN/BfupxMyuTWgjnHNp8QJuBNZXMy4H2A3cGDXsZWAtvrr1FHxVZzkwKGqaX+NP+Pfhq2m/Dvwd6BqMfwSYGjX9w/hi0vOBrwJjgAerizGIaxEwL5jnG8BnwAqgQ9R0DlgarO9U4Jog1msT3De5wTK+FWfcD4NxC4DL8VVgAwHDl1CsBi4L9tG/gDLg0Kj5rwdKgZuAk4J9VgZ8r5aYVuNLJZ8MtummII67gI+Ac4BLgS3AH6Pma4H/MswCzgte84ASoG3UdP8F1gPfwVe7fRCsb27UNBFgJTAFODeYbjLwOZAdTFMUxHVSDdtyGb4q/wfACcHneD/QLRg/MljGPOA6/MXE2GDYwKjlnBkMeyjYJ7/EH7fR2z8VuCfq/TT8j9MtUcPmERzr8eIP9v1SYBxwGv47sAO4O2qa1sC6qM9iDDA/2KcPJHjcnRusu3OccSPxx/A/gv+/G6zvoahpngM+CZYzHPgm8OdgXE/ghSCmo4NXzwRi+gyYEPz/d3wilxMzzR+AiuDvKcExNhZoHYw/Kxj/fLBvzgDuBIYH4/8ILIxZZrtgX4yJGrYZWIVvYnEGcFYw/DvAVcF+ORG4LVjfaVHz9gA24L8HY/DfvauAnwbjf44/vzWLOd7LgEtq2D9n4b/DX8OfC64Njo0fRU1zTrAtc4Gr8cfzk/hzZVHUdN/ky+/0qcA9+O+gI+o8GyeGtvjv4DLg+/jS0v9h7/P3f/BV9JcF++5VfDIWff6eHuzfD4BRwbblBJ/PZnzV4reDfdwbf56cB8wBLgJOx58D1xOcW4Jl7AKuwJ/jzwm2b2AwflCwfcuD7T0VuDsYdnFUbEcHn+nT+O/gz4DtwBP1sZ+Jf/xNx58HxuPPPZcCm4B/R02TFeynufjvwbn4Y24l8Hwt37XpwWf4arAvrwy2+b5g3OhgmWtitrsF/hw4OtjHF+DPA5Niln8//hjPD94/H8TaKsHz0/BgX/4z2G/X439HHHBO1HS3BZ/59fjv2Y34c1f0vnwef6zNi9quBcF+ywxeVef3a/nynJUZdcwswjd5Ohm4Nxh2Sq3bkcjGpsKLGpK4YPwq4P7g/xODHfTVmGneAZ6OOuj3+mGLs8xH2DuJ+4zgJJpIjPgEqhzoHTWsG/5E+4uoYQ54J2ZZz8ce1DWsN5Ek7gcxw88Ihh8VM3wy8Gjwfwd8AvHzmGnuBJbWEtNq/MnAoobNDLa9W9SwPwMlUe8vD6bpHjWsd7AfrwjeHxHEfnbUNO3w7UKik7jfB3FEJ3/5wDbgu8H7RJK4h4DHahhflcT9MmpYc/wPyY1Rw6YD/42Z9wZ8IhcJ3t9TdczhE61y/EnxrWBYJFjXidXFH2zzHCAjatgDwJKo91fhT1z5UcNOCJaVjCTuU2BczLCvB9vTM3i/nJoTjoeA6YnEEkx/eBDPZcH7k9n3hN01iOGGGpYzBxhfw/i6JHELgcwalpWB/yF9Enguavj9+CSuXTXzdQ6Om1FRw67EXxS1THB/WbDuO/FV4lXDq5KLK6OGtcInIddFDZsPPB6zzKepPYn7Of47fnA1449m3+93M/wP8JMx36fNRH2/oz6feOf/q/Df/a5Rw1oG35dfBe9vreWzr/pBfirOdkefe14FPmbv899lwbz9kr2fqzn+pgfb1jJq2I3Azqj3F+MTncKoYYXBsESSuFVA86hhrwdxHB417Jf4c41Vs5xM4LA487XCJ/vP4hPQCuC4RI7tqFg+jBn2e6LOCfjvURnws5jp/gwsiHr/fDBf/6hh/YNhFwTvuxFzvok5Zv4c891bTALn2rSvTo1iUf+fhD943w+KbrPMLAt/RTIkmOYY/BXBP+qwjunANWb2P2bWN4Hph+FPjouqBjjfbu59IPZOlddj3s/GHxTJ8p+Y9yfhG8VPq2EfHY9PRp6OM0336CLyakx0wREbWAjMD/ZB9LCDzKzq8xuGT16riqYJ9t8Uvtxnw/Bf6P9ETbMZf3NH7Da+CmyPin0TMCNqGxMxHTjHfJXqEPNtbOLZ8xk631ZtEcFnGFSjHI4/+UZ7Ev9DelTw/l1gkPmqz2PxpVd/B44ys2x8olVO7e1fxjvnom8ymA10jdrPQ/EnuHVRMb+D/1E8IMFx0R94Kua4mciXV6bg9+sNQXVP7wSXbdHLjPksLsTvm2eD9xPwpQDRVaonBDHE/d6bWRd8YvxIIvEk4FUX007XfDXyX81sGT4R242/uo8+pxTjf0Tjfh7Ot0F8BfhW1OBv4pOcHdUFY2a5ZvY7M1uM//HajS/5j3c+iz6et+OTqKrjuS3QB19aGu256tYdpRh411Xf5ngYvvR/z01izrndwbJjz5vvOd9WM9Y259zbMcNOAt4D1kQdk2X4kryq88F04AQzu8PMvhJME8+4mPfPAYX2ZdOQYcAzMee/J4O/x8bMW1/7Gfz+iT4eZgPNzSwveD8UmOecmxcVwzx84piI993e7asXAhucczNjhuUQVQVsZt8ws4/MbAv+O/tZMGrPcRjsi2/hk90H8QUu7yUYF/jPoLb99lX8BUK837hDgv1fZb5zripOgv/nB+tJRPTn7PAXi7X+xjeJJM58O5uO+BM2QB5fXqlGv24EugfTVB1Qq+qwqp/gM/IbgHnm2y9cUMP0XaJiirYGX8oVLfZkXYZPoJIlNo48fPuA2H30C77cR1Vf9M9jpnk1GF41XXXibVO8YVn4H1ZIbJ91BjY658pjplkb8z4P/8MWu41fSSD2aPfjq4MvxieTq823dYr9ftX0GXbGX2jEblvV+6ptexe/L47BJ9Hv4pPOMvwPzfH4C4PttcRc237ujE8QYyWji5yq4+bf7L3fq+5grdr338YnWr8FPg/ayJxZy7LPjllm9En5Anxpe0XQFqU1vlnFmfblzT+1fe/357xQk3jHclUV2634WoOhwbDo73vHBGL4O3B6kBQeib9IeLiWef6CLxH6M74EeSjwJ+Kfa2o6nrsEf2O/c7Hv46lt27oA62IuQiD+eTPe/q0ujjx8tVrs+WAUwTHpnHsG32b4NPzF9jozu8v2bZ9c3XZ3CaZtHxtbkGyWxtmG+trP1S0b9j4vxTsPxBuW6PJrXKeZFeOP99n4C6yj+bI9e+xxOAlf2JCDPw8nJPgM2lL7fqs6V61g72Oiqj1g9O9EvH2+li8/o9rs1298ut2dWp0R+G39MHi/Ef+h1NQXzIbgbxcS/OEKror/F/hf841hrwUeM7OZzrl4XTyswhcTx+rElz9oDcXFvN+IL84dHWfayqhpwLcb2hRnujnJCW0vq/BVXrE64dtpgC9l7WBmWTGJXCRmno34k8Dv4iwv3tV7XEFJyp3AnWbWE1+0fxP+ivmRBBezGv8ZxMZYVZq5MVjXOvN93R2PLzV60jlXaWYfRA0bn2jstcQTryQ1P86wuqo6bq7EJ6GxloLfVuAH5vuCOwL4FfCsmR3inFtazbLfwiceVTYBmNkx+IuSXsQ/Vs8GHmPv7/2yONOtjxpfnV1Adsyw2B/mKnt978w3Kj8BuNA590TU8GYx822oJQbwJdHr8G3mCvAlKh9WN3FQCvsN4Fbn3D1Rwy+uZT3xVCVhscdz7Pt4atu2VUC+mVlMSVa882bsea2m4RvxpcHxGpTvKa1yzj0APGC+/9Hz8VVw64A7oqavbrtXOedKzWxT7DRBqU5OnG2oyYHs50SsxreRjpXPl9+VZDsXX6r1raoBZhYvBvDtN/PxBQn34psB1Sr4DL6g9v1W9Vl8lahjIEp0aXG8fR7BX9jXm7QviQuuuH+HL7J9Mxg8Hn+Fsc05NzX2FUzzIb691zf3Z71BcfE1+H1cVM1kk4EjLerOSjPrii8JqkuxcH0Yjy/K3RRnH30cTPMe/mqhc7z9mEBp0P6YDBwT7CcAzKwX/oe7ap99hC9ROiNqmrb4aprYbewPzIwT+4L9Cc45V+KcuwWfAPSrw3yl+BK182JGjcZXJ0yOGvYu/sp0KL5kieDv1/ClLfESo7qagt/Pe5I2MzsBf/V6QJxzq/AJ/iHVHDdrY6Z3zvfx+Ct81UbV3dv7XKk65zbHLKvqJHsh/vt8Mv6iLvq1iC+rVN/G7++43/ugmnJOdeMDy4HOMXee7XNXdzVaBn/3dHcRfAax84/HV+FX+3kEFxf/xN80chG1Nw3JxCef0evOwf+o1klQqrQQnxxH+3qcyWONB46voQr9I3yys6dUNqjiGsWBnTfH48/V8+Mck/tchDvnVjvn/oRvdB/7XR8V8/7r+CR6S/B+Mvvu16oL5oS34QD3cyKm4KuBC6sGBP8n0lxof7Ug6hgM7HMhYWZH4NvTXYm/UDnVzL5bh/VMofb99g7+fNCpmnNVdJdAfc1sT6FM8H9f/PEK+5ZyJkW6lcRlmdnRwf+tgSPxd961BEZGtT15A3gNeMPMfodvYN8G3xanuXPuF865zWZ2C3Bb0NboFfyJ4wzgJufcitiVm9l7+LYQn+Gv9L6Pb4T6Uey0gUfwjXj/a2Y34Ntx3Yi/2v/r/u6EJHkZfwCPD/bRHHzj2CMAnHM3BKVCtwH3m+8W5T38MVUIfMU5F68U70A9iL9T61UzuwlfBXkzvmT170FsH5vZ68DfzXfzsQ5fDRxbunYnvoptvJn9BX9V2xl/19KbzrlnSYCZ/SNY/0f4huOn4IvZJ9Zx224AXjSzB4Fn8Pv6euAvMYnNO/hjaxP+BgHwidvt+OPu/TquN56/4ffZf4LPOBd/bG4gOR32XoXf1mx8u5Sd+FKys/A32aw2s4/x35E5+OPqx/htnh4sYy7wfTM7H39FvCa6rWQV8120nAe84Jx7M874x4DrzKyjc26lmd0N3Bi0X3oT34D6HOAnzrmt+JKaF8zsOXySVIYvBX3dOfcWvq3WPcA/zOz/gEPxVZSJWIq/w+12M6vEJ603sG9VzW/xP/rvmdkd+GP3cHxfmdFdLDyMvyO6Ani0phU758rN7C18u95V+JKHK/DVR/vjduBhM1uJP9+OZO/OoKvzAP74nmhmN+OT7D5AF+fcTc65SWb2Cn7/Xof/7v0U/929fT9jBV8d9x3gbfPds5TgS3mOBWY75/5mZncF076HL6U5Bv89ja3KOzY4jl7Fb/e5+BL6Kjfh22Q/iU+u++A/0yedc3Wtwdjf/ZyIJ/Hn1xfM7NfBsJvwJXTJ7Lg72hvAt4LzzgT8xfdeSXFwcfEv/Hn6oWDYH4C7zeyNGkrqo/0Wf+7/J75px1Biap6ccyvM7Pf435IifM1NM3yyf6RzLjq5XA08F7WfbsffoVrVznkd/pi52MyW4s8bnyQQZ81cgndyNPYX/gfGBa9KfP3yVPztwfHujsvBH4wLg525Gv+FOyNmuh/g6+ZLg2meAtoE4x5h77tTf4//Ud0arH8icHxMjOtjlt8bX7++FX9n1MtAn5hpHP4HJHZ7q70bN2baRO5OzYozrjn+QFwU7KNV+GT21Jjpvo0/GHfhD9IPqeEu3WCe1fhqm+hhT+Ab2tYYH/6E93Kwv7YG+68gZr48fCK0I4j7OvxdnHNjpuuOPxmsDeJfjP9hLgzGJ3J36veDbd6ET9qnA5dGja+6O/WQmPkmAWNjho0JjrcyfGnejcTcvYjvXsMBL8YczzvxPzbR01Z3d2rsvo+3n4/EJ6al+ETqdHyScUeCx121d6cG44fjT9JVn+On+FLzFsH4/wv2xfbguHodGBI1f2vgcfxFjyOqK5aY9ZwUjD+9mvF9ibpDG39hcAW+UXLVcT+WqK5Igs90El8e86+z9x18VV3f7MAngoOJf3fqr+PE0x+fJOwIjscfE/+O1774Y/+LYB9OA86Ms7zPgJcT/My647vn2YZvnnAjvqSjPGqaqrsmu8XMO52oLmKCYdcFx9t2fPvEqnmrvTs1mK8L/nu5PjiuZxHcMR6Mb4u/0FgffAYfEHWurS6eYPg++zLmvPFAsO2l+O/gk8ARUZ/r21FxzQb+N2r+qjsNz8Ff0G8Pjp99uoPCl5x/wpe/LXfHHGNJ289Uf3dq7HL2WWdwnL0V7OdF+FKxj4BHavkM4y0/3nG81zrx37+bgv22Dd8soOrO8jHBNL/Dn2+j7yTOwZ9D3qSaO13jxPgtfJu6nQQlwFWfX8x0l+FrSkrxF7LvAz+MGv98sN5Lgn20C39uiz3nn8OX+YQLPpeqY+a4mGmfxyepNW6DBROLiNTI/B3Xc/H9XT0edjxSOzPrjk8Ez3cJlizL/jP/rNJP8All2E1i6oX5u8sX4S9A9ulQuiky3/FxrnOu3p9RHSvdqlNFJEnM7Hr8VeoyfFXnr4L/Y2/Ll0YmaI9XhG8zpM9M9puZXYEv7V2ELyH9Ob4k6bEw4xJPSZyIVKeqveFB+OqGt4GrXA39jEmjMRxfnbcQX7Uf292OSKKqupbqhm/k/yHwTRdzA1JjEnTvVO2Nm+n0fVB1qoiIiKQN889R/lkNkwx2zk2vYXzKUBInIiIiaSNoC1rTE4Nmub27B0lZaZvE5eXluV69eoUdhoiIiEitpk2btt45V6cO1dO2TVyvXr2YOnVq7ROKiIiIhMzMSuo6T9o/sUFEREQkHSmJExEREUlBSuJEREREUpCSOBEREZEUpCROREREJAUpiRMRERFJQUriRERERFKQkjgRERGRFKQkTkRERCQFKYkTERERSUFK4kRERERSkJI4ERERkRSkJE5EREQkBSmJExEREUlBSuJEREREUpCSOBEREZEUpCROREREJAUpiRMRERFJQUriRERERFKQkjgRERGRFKQkTkRERCQFKYkTERERSUFK4kRERERSkJI4ERERkRSkJE5EREQkBSmJExEREUlBSuJEREREUpCSOBEREZEUpCROREREJAUpiRMRERFJQUriRERERFKQkjgRERGRFKQkTkRERCQFKYkTERERSUFZYQeQsnbsgHHjYPFi6N0bRo2CFi3Cjip50n37oGlso4iIpK0GTeLMbCTwJyATeMg5d0fM+HuAEcHblkDEOdcuGNcDeAjoDjjgdOfckgYKfW9TpuDOPBNbs2bPINepE/bSSzB0aCghJVW6bx80jW0UEZG0Zs65hlmRWSYwHzgZWA5MAS50zs2uZvqfAoOdc98J3r8F3Oace8PMcoFK59yO6tY3ZMgQN3Xq1CRvBbBzJ66gAFuzhlmR3rxbMJgTFn9Mv7WLfRKweHFql+ak+/ZB09hGERFJKWY2zTk3pC7zNGRJ3DBgoXNuEYCZPQGcDcRN4oALgd8E0/YDspxzbwA457bVf7jVGDduz4//WZfeTUVmFncefym3vHE/PTevgn+9CMUnhhbeAZswHpp3ZsnAwdxw8o+ozMxMr+2DuNv4h+Mv4cV/XUG/NYt9FetFF4UdpYiISI0aMonrCiyLer8cOCrehGbWEygAJgSD+gKbzey5YPibwHXOuYqY+S4DLgPo0aNHUoPfY9EiAN4tGExFpt99lZmZ/GrkT/z4xcDfJ9fPuhtELlxw215D0mv7IN42lmdm8U6vwfRbu3jPZywiItKYNWQSZ3GGVVeXewHwTFSSlgUcDwwGlgJPAt8C/r7Xwpx7EHgQfHXqgYccR+/eAJyw+GN+f/wlVGRmkVlRzh2v3kuvTavgNzfAySfXy6obxBtvwE03s6R9F64b+dP02z7YaxuvOf1yMCOropwTlnzixwefsYiISGPWkG3ijgFudM6dGrz/BYBz7rdxpv0E+LFz7oPg/dHAHc654cH7S4CjnXM/rm59DdEmbnakgHd6DeaEJZ+kT3uqdN8+2Gsbrz/5hzx6xNd47PFfcuzSmemzjSIiklL2p01cQ/YTNwXoY2YFZpaNL217MXYiMysE2gMfxszb3szyg/fFVN+Wrn61aIG99BKuUyf6rV3MDz967ssE56WXUv/HP923D/baxgtmvAbAiraR9NpGERFJew1WEgdgZqcDf8R3MfKwc+42M7sZmOqcezGY5kaguXPuuph5TwbuwlfLTgMuc86VVbeueiuJq7Jzp28Av2hRevYxlu7bB75E7rlxHPNZCwa3Me6//NT020YREUkJ+1MS16BJXEOq9yRO0sYvnpvJSzNW8fH1J5OdpYeYiIhIw2vs1akijdKIwgjbSsuZsmRj2KGIiIgkTEmcNHnHHpJHdlYGE+auDTsUERGRhCmJkyavVU4WR/fuyEQlcSIikkKUxIkAxYX5LFq/ncXrt4cdioiISEKUxIkAxUWdAFSlKiIiKUNJnAjQo2NLDonkqkpVRERShpI4kUBxUYTJizewrbQ87FBERERqpSROJDCiMMLuCsd7C9aHHYqIiEitlMSJBIb0ak/r5llMmLsm7FBERERqpSROJNAsM4MT+uYzcd46KivT80kmIiKSPpTEiUQpLoywbmsps1ZuCTsUERGRGimJE4kyvDAfM3U1IiIijZ+SOJEoHXNzGNS9ndrFiYhIo6ckTiRGcWGEGcu/YN3W0rBDERERqZaSOJEYI4oiALw1T1WqIiLSeCmJE4lx2EFt6NQmR+3iRESkUVMSJxLDzCguivDugvWUlVeGHY6IiEhcSuJE4hhRGGFbaTlTl2wMOxQREZG4lMSJxHHsIXlkZ2aoSlVERBotJXEicbTKyeKo3h2UxImISKOlJE6kGicWRVi0fjtL1m8POxQREZF9KIkTqUZxUSdAT28QEZHGSUmcSDV6dGzJwfmtmKj+4kREpBFSEidSgxMP7cSkRRvYVloedigiIiJ7URInUoMRhRF2VzjeW7A+7FBERET2oiROpAZDerWndfMsJqpdnIiINDJK4kRq0CwzgxP65DNx3loqK13Y4YiIiOyhJE6kFsVFEdZuLWXWyi1hhyIiIrKHkjiRWgwvzMdMXY2IiEjjoiROpBYdc3MY2K0dE9TViIiINCJK4kQSUFwUYcayzazbWhp2KCIiIoCSOJGEFBdFAHhLpXEiItJIKIkTScBhB7WhU5scPb1BREQaDSVxIgkwM0YURnh3/nrKyivDDkdERERJnEiiRhRF2FpaztQlG8MORUREREmcSKKOOySP7MwMdTUiIiKNgpI4kQS1ysniqN4d1NWIiIg0CkriROqguCjConXbWbJ+e9ihiIhIE6ckTqQOqroaUZWqiIiETUmcSB307NiKg/NbqasREREJnZI4kToqLoowedFGtpeWhx1TM6zHAAAgAElEQVSKiIg0YUriROpoRFGEsopK3lu4PuxQRESkCVMSJ1JHQ3t1oHVOFhPmqEpVRETCoyROpI6aZWZwQt98Js5bi3Mu7HBERKSJUhInsh9GFEVYu7WUWSu3hB2KiIg0UVlhByCSioYX5mMG4+espX/XtmGH0zTt2AHjxsHixdC7N4waBS1ahB2ViEiDadCSODMbaWbzzGyhmV0XZ/w9ZjY9eM03s80x49uY2Qozu6/hohbZV15uDgO7tdPTG8IyZQqud28YMwauvx4uvhhXUABTpoQdmYhIg2mwJM7MMoG/AKcB/YALzaxf9DTOuSucc4Occ4OAe4HnYhZzC/B2Q8QrUpvioggzl29m/bbSsENpWnbuxJ15JrZmDbMivXngqG8wO1KArVmDO/NM2Lkz7AhFRBpEQ5bEDQMWOucWOefKgCeAs2uY/kLg8ao3ZnYk0Al4vV6jFElQcVEE5+CteevCDqVpGTduTwJ31qV3c8fwb3PmpffsSeQYNy7sCEVEGkRDJnFdgWVR75cHw/ZhZj2BAmBC8D4DuAu4pqYVmNllZjbVzKauW6cfVqlfhx3UhkjrHCbMXRN2KE3LokUAvFswmIpM36y3IjOLd3oN3mu8iEi6a8gkzuIMq65/hguAZ5xzFcH7/wFecc4tq2Z6vzDnHnTODXHODcnPzz+AUEVqZ2YUF0V4d/56dldUhh1O09G7NwADV86Dqi5enKP3xuV7jRcRSXcNmcQtB7pHve8GrKxm2guIqkoFjgF+YmZLgD8Al5rZHfURpEhdjCiKsLW0nClLNoYdStMxahSuUydmdCkEMy6Y/iq5ZTv4v2NGs7tzF3+XqohIE9CQSdwUoI+ZFZhZNj5RezF2IjMrBNoDH1YNc85d7Jzr4ZzrBVwN/Ms5t8/drSIN7bhD8sjOzGDiXN2l2mBatMC9+CL/PvIMhi39lDteu4/fvnof0w8q4q93/lvdjIhIk9FgSZxzrhz4CfAaMAd4yjk3y8xuNrOzoia9EHjCqSt8SQGtcrI4qncHxiuJa1DvtO3F0jYRLjnuYLjlFs68/od87bAIf5yzg89WfBF2eCIiDcLSNVcaMmSImzp1athhSBPwj/cXc9NLs3n7muH07Ngq7HCahO/9cwrTl33BB9cVk53lr0U3bS/j1D++Q/uW2bzwk2Np3iwz5ChFRBJnZtOcc0PqMo8euyVygIqLIgBMUGlcg1i+aQcT5q7l/KHd9iRwAO1bZfO7bxzOvDVbueeN+SFGKCLSMJTEiRygnh1b0Tu/lZK4BvL4R0sBuHBYj33GjSiKcOGwHjz47iLdbCIiaU9JnEgSFBdGmLxoI9tLy8MOJa2VlVfy5JRlFBdF6Na+Zdxpfn3GoXRv35Krnpqhz0NE0pqSOJEkKD40QllFJe8tXB92KGnttVmrWb+tjDFH96x2mlY5WfzhvIEs27SD216Z04DRiYg0LCVxIkkwtFcHWudkqauRevbopBJ6dGjJCX1q7sx7WEEHLju+N/+evJSJ8/SZiEh6UhInkgTNMjM4vm8eE+auJV3v+A7b/DVb+WjxRi46qgcZGfEeALO3K07uS2Gn1vz8mZls3lHWABGKiDQsJXEiSVJc1Im1W0uZtXJL2KGkpccmlZCdlcHoId1rnxho3iyTu0YPZOP2Mq5/YVY9Ryci0vCUxIkkyfDCfMzU1Uh92F5azrMfr+CMAV3o0Co74fn6d23L5Sf14aUZK3lxRnVP+RMRSU1K4kSSJC83h8O7tVMSVw9emL6SbaXljDl6325FavPDrx7MoO7tuP75z1izZVc9RCciEg4lcSJJVFwYYcbyzazfVhp2KGnDOcejk0o4tEsbjujRvs7zZ2VmcNfogZSWV/DzZ2eqzaKIpA0lcSJJdOKhEZyDt+atCzuUtPHx0s3MWbWFMUf3wKz2GxriOTg/l1+cdihvzVvH4x8tS3KEIiLhUBInkkSHHdSGSOscdTWSRI9NKiE3J4tzBnU9oOVccnRPjj2kI7f+ZzZLN+xIUnQiIuFREieSRGbGiMII78xfx+6KyrDDSXkbt5fx8sxVfP2IrrTKyTqgZWVkGL8/dyCZGcZVT0+nolLVqiKS2pTEiSTZiKIIW0vL9ezOJHh66jLKKiprfEJDXRzUrgU3nXUYU5Zs4qF3FyVlmSIiYVESJ5Jkx/XJIzszQ1WqB6iy0vHY5KUMK+hA306tk7bcUYO7cuphnbjr9fnMW701acsVEWloCSVxZnaHmfWp72BE0kFuThZH9e6grkYO0DsL1rF0446klcJVMTNuHzWANi2yuOLJ6ZSVq9pbRFJToiVxJwNzzewdM7vUzFrUZ1AiqW5EYYTP122nZMP2sENJWWMnLSUvN5uRh3VO+rI75uZw+6gBzF61hXsnLEj68kVEGkJCSZxz7khgMPAJcBew2sweNLOj6jM4kVRVXBQB9PSG/bVi804mzF3D+UO7k51VP60+TjmsM+ce2Y2/TFzIx0s31cs6RETqU8JnR+fcTOfcz4CDgO8B3YD3zOwzM7vczNrUV5AiqaZXXit657dSErefHp+8FAdcOKzuT2ioixvO7EeXti24+qkZ7CyrqNd1iYgk2/5c4jqgIvgL8AXwE2CZmZ2brMBEUl1xYYTJizayvbQ87FBSSll5JU9MWcaJRRG6tW9Zr+tq07wZvz/vcBat387vXp1br+sSEUm2hJM4M+tvZncDK4AHgDnAAOfcsc65Q4DbgPvqJ0yR1FNcFKGsopL3Fq4PO5SU8tqs1azfVsrFSb6hoTpfOTiPbx/bi0c+WML7+qxEJIUkenfqFGA6MAD4KdDVOXe1cy760vURIJL0CEVS1JBeHWidk6WuRupo7KQSundowVf75DfYOn8+soiD81tx9dMz+GLn7gZbr4jIgUi0JO51oI9z7mTn3FPOuX3Ocs65tYDuWhUJZGdlcHzfPCbOW6uHridowZqtTF68kYuP6klGxv49J3V/NG+Wyd2jB7F2ayk3vTSrwdYrInIgEk3irgeWxg40s0wz27MM51xpsgITSQcjCiOs2VLKrJVbwg4lJYydVEJ2ZgbnHdmtwdc9sHs7fjziEJ77eAWvfra6wdcvIlJXiSZxzwBXxhl+OfB08sIRSS/DC9XVSKK2l5bz3McrOH1AZzrm5oQSw0+LD6F/1zb8atynrN+ma1IRadwSTeKOB16NM/w14LjkhSOSXvJb5zCwezslcQl4YfpKtpaWc8kxDXNDQzzNMjO4e/QgtpaW84vnPlU1uIg0aokmcblAWZzh5YD6hxOpQXFhhBnLN7NBJTvVcs4xdlIJRZ1bc0SP9qHG0rdTa649tZA3Zq/h2Y9XhBqLiEhNEk3iPgNGxxk+GpidvHBE0k9xUQTn4K1568IOpdH6ZNlmZq/awiXH9MSs4W5oqM53ji1gWEEHbnpxFss37Qg7HBGRuBJN4m4FbjCzv5vZxcHrYeDXwTgRqcZhB7Uhv3WOqlRrMPbDEnJzsjhnUNewQwEgI8O467yBVDrHNU/PpLJS1aoi0vgk+uzUF4DzgIH4/uAeAQ4HznfOjauv4ETSQUaGUVwY4Z3569hdURl2OI3Opu1lvPzpKkYN7kqrnKyww9mje4eW3HBmPz5ctIF/frgk7HBERPZRl2enPu+cG+Kcaxa8hiiBE0nMiKIIW0vLmbpED1qP9fS0ZZSVVzKmgZ7QUBejh3SnuCjCHf+dy8K128IOR0RkL/vz7FQRqaPj+uTRLNOYOE9VqtEqKx2PTV7KsF4dKOzcOuxw9mFm3PGNAbTMzuSqp6ZTrpJUEWlEEn3sVpaZ/cLMZprZZjPbEf2q7yBFUl1uThZHFXRk/Jw1YYfSqLy7cD0lG3Zw8dE9wg6lWpHWzbn1nAHMWP4F//fW52GHIyKyR6IlcTcCPwb+DuQAtwBjgR3AtfUSmUiaKS6K8Pm67SzdoOueKo9+WEJebjYj+3cOO5QanXF4F84edBB/Hr+AT5d/EXY4IiJA4kncRcAPnHN/wvcN95Rz7jLgZtTZr0hCiouqnt6g0jiAFZt3MmHuGkYP6U5OVmbY4dTq5rP60zE3myufms6u3RVhhyMiknAS1xn4NPh/G9A2+P9l4LRkByWSjnrltaJ3XivGq6sRAJ74aCkOuOioxluVGq1ty2bcee5AFqzdxl2vzws7HBGRhJO45fhEDmARcGLw/5GAuqEXSdCIogiTF21ke2l52KGEqqy8ksc/WkZxYYRu7VuGHU7Cvto3nzFH9+Ch9xYzadGGsMMRkSYu0STuJeDU4P/7gNvNbA7wKPDP+ghMJB2dWBShrKKS9xeuDzuUUL0+ezXrt5U2ym5FavPL0w+lR4eWXP30DLY18WRcRMKVaGe/Vznnbgn+fxw4CZ/AjXHOXVOP8YmklSG9OpCbk9XkuxoZO6mE7h1acELf/LBDqbOW2Vncdd5AVm7eya0v66mDIhKeWpM4M2tmZv8ys95Vw5xzbzvnbnfOPVO/4Ymkl+ysDI7vk8eEuWtxrmk+ymnBmq1MWrSRi4b1JDMj/Oek7o8hvTrwg68ezBNTlulGFREJTa1JnHNuN3AW0DR/cUSSrLgowpotpcxauSXsUELx2OSlZGdmMHpIt7BDOSCXn9SHos6tufaZT9m4vSzscESkCUq0TdwL+ERORA7Q8ELf1cjEJniX6vbScp6dtpzTB3SmY25O2OEckJysTO4ePYgvdpZx/fOfNdmSVREJT6JJ3FzgN2b2hJldY2b/E/2qzwBF0k1+6xwGdmvLhCbYLu7FGSvZWlqekjc0xNPvoDZccXJf/vPpKl6csTLscESkiclKcLr/xXcl8tXgFc0B/5fMoETS3YiiCH8av4AN20pTvkQqUc45Hv2whKLOrTmyZ/uww0maH5xwMG/OXsP1z3/GUQUd6dy2edghiUgTkejdqV1qeB1U30GKpJsTizrhHLw1b13YoTSYT5ZtZvaqLYw5uidmqXlDQzyZGcZdowexu8Jx7bMzVa0qIg0m0epUEUmiww5qQ37rnCZVpTp2UgmtsjM5Z3DXsENJuoK8Vvzy9CLemb+OxyYvDTscEWkiEqpONbM7axrvnLs2weWMBP4EZAIPOefuiBl/DzAieNsSiDjn2pnZIOB+oA1QAdzmnHsykXWKNEYZGcaIwnz++9lqdldU0iwzva+nNm0v4+WZqzh/SHdycxJtxZFaxhzdk9dnr+G2/8zhuEPy6JXXKuyQRCTNJfrLcXzMqxj4AXAZcFwiCzCzTOAv+Get9gMuNLN+0dM4565wzg1yzg0C7gWeC0btAC51zh0GjAT+aGbtEoxdpFEqLoqwdVc5U5dsCjuUevf0tGWUlVemzQ0N8ZgZd557OM0yjauenkFFpapVRaR+Jdom7piY1xDgIOB14KEE1zUMWOicW+ScKwOeAM6uYfoLgceD9c93zi0I/l8JrAVSr6t3kSjH9cmnWaal/dMbKisdj01eytBe7Sns3DrscOpVl7YtuPns/kwr2cSD7ywKOxwRSXP7XYfjnNsO3Az8JsFZugLLot4vD4btw8x6AgXAhDjjhgHZwOdxxl1mZlPNbOq6dU2nwbikptycLI4q6MiENO8v7t2F6ynZsCOtS+GinT3oIE4f0Jm735jHnFVNs0NnEWkYB9oQpy2+nVoi4t2OVl19wwXAM865ir0WYNYF/8zWbzvnKvdZmHMPOueGOOeG5OeroE4avxFFERau3cbSDTvCDqXejJ1UQsdW2Yzs3znsUBqEmXHrOQNo2yKbK56cTml5Re0ziYjsh4SSuNjOfc3sx2Z2K75K9PUE17Uc6B71vhtQXe+YFxBUpUbF0Ab4D/Br59ykBNcp0qgVF/mnN6Tr8zdXbt7J+DlrOH9od3KyMsMOp8F0aJXN774xgLmrt/KnNxeEHY6IpKlEbxO7PuZ9JbAOeBpfpZqIKUAfMysAVuATtYtiJzKzQqA98GHUsGxgHPAv59zTCa5PpNEryGtF77xWTJi3jm8dWxB2OEn3+EdLccCFw3qEHUqDO/HQTpw/pDsPvP05Jx7aKa06OBaRxmF/O/vtGtxFeqVzbnOCyygHfgK8BswBnnLOzTKzm80s+rmsFwJPuL17zBwNnAB8y8ymB69BCW2hSCM3oijCpM83sL20POxQkmp3RSVPTFnGiMII3Tu0DDucUPz6a4fSpW0LrnpqOjvK0uvzFZHwJVqdmhF0ERI7PNPMEm5X55x7xTnX1zl3sHPutmDYDc65F6OmudE5d13MfGOdc82quh8JXtMTXa9IY1ZcFKGsopL3F64PO5Sken3WGtZtLeWSJnJDQzytmzfjrtEDKdm4g9++MjfscEQkzSSagD0DXBln+OX4KlUR2U9De3UgNycr7boaeXTSErq1b8EJfZv2TUZH9+7Id48t4NFJJbwzX3fNi0jy1KWz31fjDH+NBDv7FZH4srMyOL5PHhPnrkub524uXLuVSYs2ctFRPcjMSJ/npO6vq08t5JBILtc+M5MvduwOOxwRSROJJnG5QFmc4eUk3sWIiFRjRFGE1Vt2MTtN+hUbO2kp2ZkZjB7SvfaJm4DmzTK5Z/Qg1m8r5caXZoUdjoikiUSTuM/wNxfEGg3MTl44Ik3T8EJf5ThhTupXqe4oK+fZacs5bUBn8nJzwg6n0RjQrS0/Le7DuE9W8Mqnq8IOR0TSQKJJ3K3ADWb2dzO7OHg9DPw6GCciByDSujkDu7VlQhq0i3tx+kq2lpY36RsaqvM/Iw7m8G5t+dW4T1m7dVfY4YhIiku0i5EXgPOAgcAjwetw4Hzn3Lj6Ck6kKRlRFGH6ss1s2FYadij7zTnHo5NKKOrcWv2ixdEsM4O7Rw9kR1kFv3zu07RpAyki4ahL9yDPB4+0aha8hiiBE0me4qIIzsHbKXwH4/Rlm5m1cgsXH90TM93QEM8hkdZcO7KIN+es5empy8MOR0RSWKL9xH3FzI6NM/xYMzsm+WGJND39D2pLXm4O4+embpXq2ElLaZWdyajBXcMOpVH79ld6cXTvDtz88myWbUzf5+aKSP1KtCTuT0CnOMPzg3EicoAyMozionzemb+O3RWVYYdTZ5u2l/HSzJWMOqIruTmJPtGvacrIMP5w3kAArn56BpWVqlYVkbpLNInrB8R7QsIM4NDkhSPStBUXRdi6q5xpJZvCDqXOnpm2nLLySsbohoaEdGvfkhvO7MfkxRt5+P3FYYcjIiko0SRuJxCJM7wLvq84EUmC4/rk0yzTmJhiVaqVlY7HJpcwtFd7ijqr68hEnXdkN046tBN3vjaPBWu2hh2OiKSYRJO4N4Hbzax11QAza4PvXuSN+ghMpCnKzcliWEGHlGsX997C9SzZsEOlcHVkZvz26wPIzcniyqdmpGQ1uoiEJ9Ek7hqgN1BiZm+Y2RvA4mDY1fUVnEhTVFzUiYVrt6VUg/exk0ro2Cqbkf07hx1KyslvncPto/rz6YovuG/CwrDDEZEUkmg/ccuAAcBvgKXAsuD/w51zS+svPJGmp7jIt1yYkCKlcSs37+TNOWsYPbQ7OVmZYYeTkkb278LXB3flvokLmbl8c9jhiEiKqEs/cVudc/c6577rnPuOc+4+59wWMzuuPgMUaWoK8lpRkNcqZapUn/hoKQ64aFiPsENJab856zAirXO44snp7NpdEXY4IpICEk7ioplZvpldbWZzgbeTHJNIkzeiMMKkRRvYUda47xvaXVHJ41OWMaIwQvcOLcMOJ6W1bdGMO889nM/Xbef3r80LOxwRSQEJJ3HmnWZmzwHLgW8C/wb611dwIk3ViYdGKCuv5P2FG8IOpUavz1rDuq2ljDlapXDJcHyffL55TE/+/t5iPvh8fdjhiEgjV2sSZ2a9zOxmfFu4h4BFwXznO+duds7NqecYRZqcob06kJuT1ejbxY2dVELXdi34at94PRDJ/rjutEMpyGvFNU/PZOumLfDYY3DrrfDvf8POnWGHJyKNSI1JXHAX6kz8XajfA7o753Q3qkg9y87K4LhD8pg4d22jfUj6wrVb+XDRBi4+ugeZGXpOarK0yM7krtEDWbV5J7d893YYMwauvx4uvhhXUABTpoQdoog0ErWVxI0A/gbc6Jx7zTmnToxEGkhxUYTVW3Yxe9WWsEOJa+ykpTTLNEYP6R52KGnniPzm/GjGyzzV93gePvIsHjjqG8yOFGBr1uDOPFMlciIC1J7EHQ+0Az42sw/N7Edm1qEB4hJp8oYX5QM0yqc37Cgr59mPl3P6gC7k5eaEHU76GTeOn73+EL03LOfmE7/PHcO/zVmX3rMnkWPcuLAjFJFGoMYkzjn3oXPuu/jHaz2Mv5lhVTDfSWaWW/8hijRNkdbNObxb20bZLu7F6SvZuqtcT2ioL4sWkV1ZzvBFU8F8VXV5Zhbv9Bq8Z7yISKKd/W53zv3NOXc0cCRwL3A9sNbMdEkoUk9GFEb4ZNlmNmwrDTuUPZxzjJ1cQmGn1gzp2T7scNJT794AnPvZeDIqfZ9xGZUVnLDkk73Gi0jTVud+4pxznznnLgcOAr4NqHMokXpy4qERnIO3568LO5Q9Ziz/gs9WbGHMMT0x0w0N9WLUKFynTvRbu5gX/nUF3TavocXuUtrv3ILr1AlGjQo7QhFpBOqcxJnZKDNr6Zzb7Zx70jl3an0EJiLQ/6C25OXmNKoq1Uc/LKFVdiajBncNO5T01aIF9tJLuE6dGLBmEY89+SucGdecfS28+CK0aBF2hCLSCOzPExv+Cegp1yINICPDGFGYzzvz17G7IvybwzfvKOPlmSs5Z3BXcnOywg4nvQ0dii1eDI89Rs+rfsyvCuC9rocxtkJ98omItz9JnOpPRBpQcVGELbvKmVayKexQeGbackrLK3VDQ0Np0QIuugh+/Wsu+sk3+GrffG57ZQ6L128POzIRaQT269mpItJwjuuTR7NMC72rkcpKx9hJJQzp2Z5Du7QJNZamyMy489zDycnK5MqnplPeCEpmRSRc+5PEnQOsTHYgIhJf6+bNGFbQIfR2ce9/vp4lG3ZwyTEqhQtLpzbNueWc/nyydDN/fUfdjIg0dQklcWb2ipm1BXDOjXfO7QqGtzazV+ozQBHxXY0sWLuNZRt3hBbDox+W0LFVNiP7q0lsmM4aeBBfO7wLf3xzPrNWfhF2OCISokRL4k4F4nXL3hw4OXnhiEg8xUW+MXtYpXGrvtjJm3PWcN6Q7uRkZYYSg3zplrP7065lNlc+OYPS8oqwwxGRkNSYxJlZPzPrh7+ZoW/V++A1APgWqloVqXe983MpyGsVWhL3+EfLcMDFR/UIZf2yt/atsrnzG4czb81W7nljQdjhiEhIausj4DPABa+3Y8YZUAZcXg9xiUiMEYURxk4uYUdZOS2zG657j90VlTzx0VKG982newf17d1YjCiKcOGwHvz1nc856dAIQ3rpsdYiTU1t1amHAofhE7YTgH5Rr4OBds65B+o1QhEBfJVqWXkl7y/c0KDrfWP2GtZuLVW3Io3Qr844lG7tW3DV0zPYXloedjgi0sBqTOKcc/Occ3OBFs6594P3Va/FVTc4iEj9G1bQgVbZmQ1epTp2Ugld27VgeKE6mW1scnOyuOu8QSzduIPbX5kTdjgi0sASvbHhNDMrrnpjZtea2UIze8HM8uspNhGJkp2VwfF98nlr3lqccw2yzoVrt/HB5xu46KgeZGaon+/GaFhBB75/fG8em7yUt+Y1nseziUj9SzSJuxXIBjCzgcAtwL+ADsBd9ROaiMQqLoqw6otdzFm1tUHW99jkEpplGucP7d4g65P9c+XJfenbKZefPzuTzTvKwg5HRBpIoklcL2Bu8P/XgRecczcDPwNOqYe4RCSO4UW+4HvC3DX1vq4dZeU8M205p/XvQl5uvB6GpLFo3iyTu0cPYsO2Mm54YVbY4YhIA0k0iSsDqm5LOxF4M/h/I6Dn74g0kEjr5hzerW2DtIt7acZKtu4q1w0NKaJ/17b87MQ+vDhjJS/PVM9PIk1Boknc+8DvzOwaYBhQ9ZSGPsCK+ghMROIbURjhk2Wb2bi9/qrNnHM8OqmEwk6tGdqrfb2tR5LrR8MPZmD3dvz6+c9Yu0X3nYmku0STuJ/in87wPeBnzrnlwfCzgPH1EZiIxFdcFME5eHt+/ZXGzVj+BZ+t2MKYo3tgphsaUkVWZgZ3jx7Irt0V/PzZmQ12A4yIhCOhJM45t8Q5d7JzrtA5d3/U8J86535Yf+GJSKwBXduSl5vD+Dn1l8SNnVRCy+xMzhnctd7WIfXj4PxcrhtZxMR563hyyrKwwxGRepRoSRxm1szMvmZmPzOzNsGw7lX/i0jDyMgwRhTm8878dZRXVCZ9+Zt3lPHSjJWMGtyV1s2bJX35Uv8uPaYXxx7SkVtens3SDTvCDkdE6klCSZyZ9QJmA//GdymSF4y6Cvh9fQQmItUrLoqwZVc500o2JX3Zz0xbTml5pW5oSGEZGcbvzx1IhhlXPz2DikpVq4qko0RL4v6Ev7mhI7Azavg4/N2qItKAjuuTR7NMS/pdqpWVjscmL2VIz/Yc2kWF7KnsoHYtuPGsw/hoyUYefm9x2OGISD1INIk7Fvitc253zPAS4KBEV2ZmI81sXvC0h+vijL/HzKYHr/lmtjlq3DfNbEHw+mai6xRJR62bN2Norw5JT+Le/3w9i9dvVylcmvj6EV059bBO/P61ecxb3TAdRItIw0k0icsMXrG6AQmdGcwsE/gLcBrQD7jQzPpFT+Ocu8I5N8g5Nwi4F3gumLcD8BvgKHwXJ78xM/V7IE1acVGEBWu3sWxj8to8jZ1UQodW2Zw2oHPSlinhMTNuHzWA1s2zuPKp6ZSVJ78NpYiEJ9Ek7g18NyNVnJm1widWrya4jGHAQufcIudcGfAEcHYN018IPB78fyrwhnNuo3NuUxDPyATXK5KWiov8A+knJul5mau+2Mmbc9Yyekh3crLiXbNJKlDVhBkAACAASURBVOqYm8Nvvz6AWSu3cN+EBWGHIyJJlGgSdzVwqpnNxPcX9y9gEVAA/DzBZXQFou93Xx4M24eZ9QyWPaGu84o0Fb3zc+nVsWXSuhp5/KNlVDrHxUf1SMrypPE45bDOnHtkN/7y1udMX7a59hlEJCUk2k/cUuBw4H7gn8AC4BZgsHNudYLritdjaHW3TF0APOOcq6jLvGZ2mZlNNbOp69atSzAskdQ1oijCh4s2sKOs/ICWs7uikic+WspX++bTvUPL2meQlHPDmf3o3KY5Vz41nZ1lFbXPICKNXo1JnJk9bGatAZxz25xz9zvnvuec+45z7j7nXF1ayi4Huke97wZU94C/C/iyKjXheZ1zDzrnhjjnhuTn59chNJHUdGJRJ8rKK/lg4YYDWs6bs9ewdmspl+iGhrTVpnkzfn/u4Sxat53fvTo37HBEJAlqK4n7JtAiSeuaAvQxswIzy8Ynai/GTmRmhUB74MOowa8Bp5hZ++CGhlOCYSJN2rCCDrTKzmTCAbaLe3RSCV3btWB4YSRJkUlj9JVD8vjWV3rxyAdLeH/h+rDDEZEDVFsSl7SHJjrnyoGf4JOvOcBTzrlZZnazmZ0VNemFwBMu6qF/zrmN+OrbKcHr5mCYSJOWnZXBcX3ymDh37X4/J3Ph2m188PkGLjqqB5kZek5quvv5yCJ657fimqdnsGVXbK9RIpJKEmkTl7Suvp1zrzjn+jrnDnbO3RYMu8E592LUNDc65/bpQ84597Bz7pDg9Y9kxSSS6oqLIqz6YhdzVu1fP2CPTS6hWaZx/tDutU8sKa9FdiZ3jx7Emq2l3PTi7LDDEZEDkEgSt9rMKmp61XuUIlKtEYX739XIzrIKnp22nJH9u5CXm5Ps0KSRGtS9HT8efjDPfryc12Ylem+aiDQ2WQlMcxmge9JFGqlIm+YM6NqWCXPX8uMRh9Rp3pdmrGTLrnLd0NAE/aS4DxPmreWXz33KkT3bK4kXSUGJJHEvOeeS+2wfEUmqEUUR7p2wgI3by+jQKjvh+R6dVELfTrkM7aUHoDQ12VkZ3D16EF+79z1++dyn/PWSIzFTm0iRVFJbdWrS2sOJSP05sSiCc/D2/MSvt2Ys28ynK75gzNE99ePdRPXt1JprTink9dlreO7jFWGHIyJ11GB3p4pI/RnQtS15uTlMmJt4J9djJ5XQMjuTUYP18JOm7DvHFTCsoAM3vjiLFZt3hh2OiNRBjUmccy5DVakijV9GhjG8MJ+3562lvKL2h5xv3lHGizNWcs7grrRu3qwBIpTGKjPDuOu8gVQ6xzVPz6CyUhUwIqki0WenikgjV1wUYcuucqaVbKp12memLae0vJIxR+mGBoHuHVpy/df68cHnG/jXh0vCDkdEEqQkTiRNHN8nj6wMq/XpDZWVjscmL+XInu3pd1CbBopOGrvzh3anuCjCb/87l8/XbQs7HBFJgJI4kTTRunkzhhV0YOL/t3fn4VGVdxvHv7/MJCTsa8IuiSwBWSVsWkFQKiqgWEURVIoK+pa21tq61La+Vltf3LBqVRBxA5SioNRWi4KyKBgWQQiIIWGHBJGdQEjyvH/MYEMKQmKYkzPcn+vikjnzzJn7ODDcOc9Z1nx/ift03U6yvznAsO5NI5RM/MDMeOSqdiTEBbhz6vJTmpYXEW+pxIlEkT6piazN2c+mbw+ecMzrCzdQu0ocl7ZtEMFk4geJ1eN5+Mp2LN+0m+c+Xud1HBE5CZU4kSjSO/X7796wfc8hZq3O4Zq0xsTHBiIZTXzi8vYNuKJjQ5766GtWbtnjdRwR+R4qcSJRJKVuFZrVqczsE0ypTvl8I0XOMbSrTmiQE3twYFvqVI3jzqlfcOiI7qwoUlGpxIlEETOjd2oin67bycH8gmOeO1JYxBvpG+nVsh5N61T2KKH4QY3KsYy5ugNrc/bzxKy1XscRkRNQiROJMn1SE8kvKOLTzJ3HLP8wI4ecvYd1WRE5Jb1a1mNot6aMn5fFoqydJ3+BiEScSpxIlOmaXJvKcYH/utTI64s20KhmwnfHzYmczH2XtaZp7crcNW05+w8XnPwFIhJRKnEiUaZSMMAFLeoyZ00uzoWuvr9ux34WZO7k+m5NCcTobnpyaqpUCvL4NR3YsiuPh9/L8DqOiJSgEicShfqkJrJtzyHWbN8HwKSFG4kNGIPTmnicTPwmrVltRvY8mymfbzrpNQhFJLJU4kSiUO9WoSnT2WtyycsvZNqSTfRr24B61Sp5nEz86Fd9W5Bavxq/fWsFuw7kex1HRMJU4kSiUGL1eNo2qMrseauY+fAL7D1UwLBOSV7HEp+qFAzwxOCO7D6Yz/3vrPQ6jlQUBw/CpEnw0EMweTLk5Xmd6IyjEicSjdLT6fOvySzbb4zbYrTcsYEuF54L6eleJxOfatOwOndc3JL3Vmzj3eVbvY4jXktPx6WkwLBh8Pvfw9ChuORkfcdEmEqcSLTJy8MNGECfFR9TFBMgs25TLspcRExODm7AAP20LGU2qmcK5zatye9nrGT7nkNexxGvhL9jLCeHVYkpPN/tJ2QkJmP6jok4lTiRaDN9OpaTQ6CwEMJnp47retV3X7JMn+5xQPGrYCCGxwd3JL+giLvfWvHd2c9yhgl/x3xRvwUDbnqSRy78KQNufFLfMR5QiROJNllZACxI7ggWupxIYSDI3GadjnlepCyS61bhvstS+WTtDiZ/vtHrOOKBQ+uyeeXc/txw7UMUxYTuwVwYCDK17cWhAfqOiRiVOJFok5ICQM/spQQLQxdoDRYW0HP9smOeFymrYd3P4oIWdXn4vdVs2HnA6zgSIQcOFzB+bhYXHG7PH/veRuM9OcQUhu+t6xzT2vdlWYOW+o6JIIvW3eFpaWlu8eLFXscQiby8PFxyaFojIzGZuc060XP9MtrkZuOSkrDsbEhI8Dql+Ny2PXn8+Mm5tEqqxpujeugi0lFs76EjvPbZBl6cl8Wug0c4L7kWP3/uXrovn8vq8HdMqx3r+WPf2/m2Sk0mjjqfLq0aeB3bd8xsiXMurVSvUYkTiULp6d8deHyUS0rCZs6ELl08DCbRZMayLdzx5hfcc2kqt/U62+s4Us52H8znpQXreXlBNnsPFdC7VT1G92lO57NqH/c7ZmtyKsNGPMn2fJhwUxd6nF3Hw/T+oxJXjEqcnPHy8kIHGGdlhaY3Bg3SHjgpV845/mfSUj5ancu7Pz+f1PrVvY4k5eCb/YcZPy+L1z/bwIH8Qi45J4nRvVvQrnGNYwce5zsmt8AYOn4RG789yPgb0+jZsp43G+FDKnHFqMSJiJx+O/cf5pKx86hXrRLv/Ox84oI61Nqvtu85xAtz1zHl840cLiiif/uGjO7dnFb1q5VqPTv3H2bYhM9Zl7uf5284lz6putD4qShLidPfNhERKbM6VSvxyFXtWL1tL099tNbrOFIGm749yO+mf0nPMXN49bMN9G/fkA/v7MXTQzqVusBB6M/ElFu70ap+NUa9toT3V24/DakFIOh1ABER8beL2yQxOK0xz328jj6pSXQ+q5bXkeQUZH9zgL/NyWT6si2YwTVpTbi919k0qV35B6+7ZuU4Xr+lG8Mnfs7PJi9l7LUdGdChYTmkluI0nSoiIj/YvkNH6Dd2HnHBGN77xY+oHKd9BBXV2px9PDsnk5nLtxIbiGFI16aM6pVCgxrlf8zs/sMFjJiYzuIN3/LYNR246tzG5f4e0ULTqSIi4olq8bE8PrgD63ce4JF/rfE6jhzHyi17uO21Jfz4ybnMysjh1gtSmH93Hx4YeM5pKXAAVSsFeXlEF7qn1OHXf1/Om+m6QHR50o9KIiJSLrqn1GHE+clMmJ9N3zZJXNBCZyZWBEs37uKZ2ZnMXpNLtfggv+jTnJ+en0ytKnERef/KcUFeGt6FUa8t4e63viS/oIgbejSLyHtHO02niohIuTl0pJD+T89n/6ECPrijJzUqx3od6Yy1MGsnz8zOZH7mN9SqHMvNP0rmhh7NqJHgzWdyuKCQn01ayoerc7n/8tbccoHu7FCcplNFRMRT8bEBnhjcgR37D/PAzFVexznjOOeYu3YHg5//jOvGLWTN9n3cd1kq8+/uw+g+LTwrcACVggH+NrQzl7atz0PvreZvH2d6liVaaDpVRETKVfvGNfl5n+aM/fBrftwmiUvb6RZMp5tzjo9W5/L0nEyWb9pNgxrx/O/Ac7i2SxPiYwNex/tOXDCGp4d04s6pyxnz/lccKXD84qLmmOm2bWWhEiciIuXuZ72bM3tNLvdN/5LOzWqRWC3e60hRqajI8f6q7Tw9O5PV2/bSuFYCfx7Ujp90bkSlYMUpb8UFAzE8eW1H4oIxPPnhWvILC7nrx61U5MpAJU5ERMpdbCCGJwZ34LK/zue+t79k/I1p+ke6HBUUFjFzxVaenbOOzNz9pNStwuPXdGBgx4bEBir+kVKBGGPMT9oTG4jh2TnrOHykiN9d3lp/RkpJJU5ERE6L5onVuLtfKn/6RwZ/X7KZwWlNvI7ke/kFRUxftpm/fbyODTsP0iqpGk8P6cRl7RoQiPFXAYqJMf48qC1xAePF+dnkFxbxwIBziPHZdnhJJU5ERE6bn57XjFkZ23lwZgY9UuqUy90AzkSHjhTy98WbeP6TLLbszqNdoxq8cENn+rZO8nXpMTMeGHgOccEYxs/LJr+giD8PaufrbYoklTgRETltYmKMx67pQL+x8/jNtOVMvqW7/oEuhYP5BUxetJFxc7PI3XeYc5vW5KFBbbmwZb2omXo0M+67rDWVggGemZPJkULHmKvb+27PohdU4kRE5LRqXKsyfxjQht9OW8HET9dz84+SvY5U4e07dITXFm5gwrxsdh7Ip0dKHcZe15EeKXWiprwVZ2bcdUkr4oIxPDFrLfmFRTwxuIMvju/zkkqciIicdtd0bsy/V23n/95fQ6+WdWmeWM3rSBXSnoNHmPhpNhMXrGdP3hEubFWP0b2bk9asttfRIuIXF7UgLhjDI/9aw5GCIv46pBNxQRW5E9EdG0REJCJ27DvMJWPn0rhWAm/dfp72shSzc/9hXpyfzWufbWD/4QJ+3CaJ0X2a075xTa+jeeKl+dk8+I8MLkpN5Nmh51aoa92dLmW5Y4P2xImISETUq1aJh69sy+2TlvLsnEzuuLil15E8l7P3EOPmZjFp0QYOFxRxebsGjO7TnNT61b2O5qkRP0omLhjD/TNWcuurixl/Y9oZUeRKSyVOREQi5tJ2DRjUqRHPzM6kT7PqtF/0EWRnQ0oKDBoECQleRyw/Bw/C9OnH3b7Nuw7ywidZvLl4E4VFjis7NuJ/ep/N2fWqehy64hjW/SziAjHc/fYKfjoxnQnD06gcp9pSXESnU82sH/AUEABedM49cpwxg4EHAAcsd85dH14+Bric0P1eZwG/dN8TXtOpIiIV0568I1wy5kOqbtvMPyaMJr4gHwCXlITNnAldunicsBykp+MGDMBycr5b5JKS2DBlOs/trMJbSzdjBld3bsLtvc6maR1deuVEpi/bzK+nLqfzWbV4aXgXqsV7d//X06ks06kRK3FmFgDWAn2BzUA6MMQ5l1FsTAtgKtDHObfLzBKdc7lmdh7wKNAzPHQ+cK9z7uMTvZ9KnIhIBZWXx9zzLufGS37NlSvnkPrNenpmL6VNbnaoyGVn+3uPXF4eLjkZy8lhVWIK85I70ezbLXzQ8jzeaXMhsXFBhnRtysieKTSs6ePtjKB/rNjKL9/4gnaNavDKiK7USIi+IlfRj4nrCmQ657IAzOwN4Aogo9iYW4FnnXO7AJxzueHlDogH4gADYoEcRETEf6ZPp+cXc+jftAszzrkQzBjT80Zu/Xw69ffvhPHvQefOXqcsuyVLoHFXtqfWYXzXQRTFBMA54gvyuSV9OrcM70viwEu9Tukr/duHbic2evJShr64kNdGdKNWlTivY3kukiWuEbCp2OPNQLcSY1oCmNkCQlOuDzjn3nfOfWZmc4BthErcM8651SXfwMxGAiMBmjZtWv5bICIiP1xWFgAtd2yA1qEJlqKYAC90vzr0/FZga8YJXuwHCXDxqGMXmTFy0TTuXDAFLkrxJpbPXXJOfcbdkMao15cwZPxCXr+lG3WrVvI6lqciWeKOd3XCknO5QaAFcCHQGJhnZm2BukDr8DKAWWbW0zk395iVOTcOGAeh6dTyiy4iIuUmJVRiLl73OU+dP4TCQJBAYQGT3/gdrb7ZAC+Mg6t/4nHIH2DaWzBqJF/VPYvrr3uYwkCQYGEB/b5eGHo+RSWurHqnJjLhpjRufXUxQ8YtZNIt3UisHu91LM9E8pi4HoT2rF0SfnwvgHPuL8XGPA8sdM69HH78EXAPoVIX75z7U3j5H4BDzrkxJ3o/HRMnIlJBFTtmLCMxmbnNOtFz/bKoPCYuKrevAliYtZMRL6dTv3o8k27tRoMa/v//WZZj4iJ5pcV0oIWZJZtZHHAd8G6JMTOA3gBmVpfQ9GoWsBHoZWZBM4sFegH/NZ0qIiI+kJCAzZyJS0qiTW42t33+9n8KzsyZ/i840b59FUD3lDq8dnNXduw7zLUvLGTzroNeR/JEpC8xchkwltDxbi855x42sweBxc65dy10Q7jHgX5AIfCwc+6N8JmtfyN0dqoD3nfO3fl976U9cSIiFVxeXug6allZ0XmduGjfvgpg+abd3DBhEdXiY5l8azfOqlPF60hlVqEvMRJpKnEiIiLRb+WWPdwwYRFxwRgm39rdtxdMrujTqSIiIiLlqm2jGkwZ2Z3CIse1Lyxkbc4+ryNFjEqciIiI+Fpq/eq8MbIHMQbXjVtIxta9XkeKCJU4ERER8b3miVWZOqoH8cEYhoxfyIrNu72OdNqpxImIiEhUaFa3Cm+O6kG1+CBDxy9iyYZdXkc6rVTiREREJGo0qV2ZqaN6UKdqHDdOWMSirJ1eRzptVOJEREQkqjSsmcDUUT2oXyOe4RPTWZD5jdeRTguVOBEREYk6idXjeWNkD5rWrsyIl9P5+KtcryOVO5U4ERERiUr1qlViysjuNE+syshXl/BhRo7XkcqVSpyIiIhErdpV4ph8S3daN6zOba8v4V9fbvM6UrlRiRMREZGoVqNyLK/f3JWOTWoyesoy3vlii9eRyoVKnIiIiES9avGxvDKiK12a1eKON7/g74s3eR3pB1OJExERkTNClUpBJg7vyo+a1+U301YwedFGryP9ICpxIiIicsZIiAsw/sY0+qQmct/0L3nl0/VeRyozlTgRERE5o8THBnh+WGcuOSeJP767ivFzs7yOVCYqcSIiInLGiQvG8Mz159K/fQMe/udqnpn9tdeRSi3odQARERERL8QGYhh7bUfiAjE89u+15BcU8au+LTEzr6OdEpU4EREROWMFAzE8ek0HYgMx/HV2JocLi7inX6ovipxKnIiIiJzRAjHGX65qR2zQeOGTLPILivhD/zYVvsipxImIiMgZLybG+NMVbYkLBHhpQTZHCot4sG8KMTNmQHY2pKTAoEGQkOB11O+oxImIiIgAZsbv+7emUmwMz328jvxXJ/GXGY8ScEUAuKQkbOZM6NLF46QhKnEiIiIiYWbGb3s2JW7sEzzV8Qp2XBVPly2ruDBrCW1ysnEDBmDZ2RVij5wuMSIiIiJSjM2Ywa8+GM9Ni2cyp3kXxvQazsAbnyQjMRnLyYHp072OCKjEiYiIiBwrK3Tx3wb7v/luUUEgyNxmnY553msqcSIiIiLFpaQA0DN7KcHCAgCChQX0XL/smOe9Zs45rzOcFmlpaW7x4sVexxARERG/ycvDJYemTjMSk5nbrBM91y+jTW526OSG03BMnJktcc6lleY1OrFBREREpLiEBGzmTNyAAbTJyaZNbjZQ7OzUCnBSA6jEiYiIiPy3Ll1Ce9ymTw8dA5eSguk6cSIiIiI+kJAA11/vdYoT0okNIiIiIj6kEiciIiLiQypxIiIiIj6kEiciIiLiQypxIiIiIj6kEiciIiLiQypxIiIiIj6kEiciIiLiQypxIiIiIj6kEiciIiLiQypxIiIiIj6kEiciIiLiQypxIiIiIj6kEiciIiLiQypxIiIiIj6kEiciIiLiQypxIiIiIj6kEiciIiLiQxEtcWbWz8y+MrNMM7vnBGMGm1mGma0ys8nFljc1s3+b2erw880ilVtERESkoglG6o3MLAA8C/QFNgPpZvaucy6j2JgWwL3A+c65XWaWWGwVrwIPO+dmmVlVoChS2UVEREQqmkjuiesKZDrnspxz+cAbwBUlxtwKPOuc2wXgnMsFMLM2QNA5Nyu8fL9z7mDkoouIiIhULJEscY2ATcUebw4vK64l0NLMFpjZQjPrV2z5bjN728yWmdmj4T17xzCzkWa22MwW79ix47RshIiIiEhFEMkSZ8dZ5ko8DgItgAuBIcCLZlYzvPwC4C6gC5ACDP+vlTk3zjmX5pxLq1evXvklFxEREalgIlniNgNNij1uDGw9zph3nHNHnHPZwFeESt1mYFl4KrYAmAGcG4HMIiIiIhVSJEtcOtDCzJLNLA64Dni3xJgZQG8AM6tLaBo1K/zaWmZ2dPdaHyADERERkTNUxEpceA/aaOADYDUw1Tm3ysweNLOB4WEfADvNLAOYA/zGObfTOVdIaCr1IzP7ktDU7PhIZRcRERGpaMy5koelRYe0tDS3ePFir2OIiIiInJSZLXHOpZXmNbpjg4iIiIgPqcSJiIiI+JBKnIiIiIgPqcSJiIiI+JBKnIiIiIgPqcSJiIiI+JBKnIiIiIgPRe114sxsB7DB6xxRoC7wjdch5AfRZ+h/+gz9TZ+f/0XiMzzLOVeqG79HbYmT8mFmi0t78UGpWPQZ+p8+Q3/T5+d/FfUz1HSqiIiIiA+pxImIiIj4kEqcnMw4rwPID6bP0P/0GfqbPj//q5CfoY6JExEREfEh7YkTERER8SGVODkuM2tiZnPMbLWZrTKzX3qdSUrPzAJmtszM/uF1Fik9M6tpZtPMbE3472IPrzNJ6ZjZr8LfoSvNbIqZxXudSb6fmb1kZrlmtrLYstpmNsvMvg7/t5aXGY9SiZMTKQB+7ZxrDXQHfmZmbTzOJKX3S2C11yGkzJ4C3nfOpQId0GfpK2bWCPgFkOacawsEgOu8TSWn4GWgX4ll9wAfOedaAB+FH3tOJU6Oyzm3zTm3NPz7fYT+8WjkbSopDTNrDFwOvOh1Fik9M6sO9AQmADjn8p1zu71NJWUQBBLMLAhUBrZ6nEdOwjk3F/i2xOIrgFfCv38FuDKioU5AJU5OysyaAZ2ARd4mkVIaC/wWKPI6iJRJCrADmBieEn/RzKp4HUpOnXNuC/AYsBHYBuxxzv3b21RSRknOuW0Q2skBJHqcB1CJk5Mws6rAW8Adzrm9XueRU2Nm/YFc59wSr7NImQWBc4HnnHOdgANUkCkcOTXh46auAJKBhkAVMxvmbSqJJipxckJmFkuowE1yzr3tdR4plfOBgWa2HngD6GNmr3sbSUppM7DZOXd0D/g0QqVO/ONiINs5t8M5dwR4GzjP40xSNjlm1gAg/N9cj/MAKnFyAmZmhI7FWe2ce8LrPFI6zrl7nXONnXPNCB1IPds5pz0APuKc2w5sMrNW4UUXARkeRpLS2wh0N7PK4e/Ui9DJKX71LnBT+Pc3Ae94mOU7Qa8DSIV1PnAD8KWZfRFedp9z7p8eZhI50/wcmGRmcUAW8FOP80gpOOcWmdk0YCmhM/6XUUGv/C//YWZTgAuBuma2Gfgj8Agw1cxuJlTOr/Eu4X/ojg0iIiIiPqTpVBEREREfUokTERER8SGVOBEREREfUokTERER8SGVOBEREREfUokTERER8SGVOBGRH8DM0szMhe8xfLrfa7iZ7T/d7yMi/qASJyJSAZnZejO7y+scIlJxqcSJiIiI+JBKnIhUWGb2sZk9Z2aPm9m3ZrbDzH5pZpXM7Fkz221mG83shmKvecTMvjKzvPDerDFmFh9+zsxslpl9GL6XJWZW1cy+NrNnTjFTPzNbY2aHzGwe0PI4Y84zs0/M7KCZbQlvQ/US2/W8mT1lZrvCvx41s5ijzwNnAY+Gp2pdifVfZGYrzeyAmc0xs+TS/98VEb9TiRORim4osA/oRuj+hWOBGcBaIA14BXjRzBqGxx8ARgCtgf8BrgN+B+BC9xm8CegIHJ2q/CuQD/zmZEHMrEn4vWeF1/E0MKbEmHbAvwndMLsDcFV47EvH2a4YoAcwChgJ3BF+7ipgM/Ag0CD866hKwL3hbewB1ASeP1l2EYk+uneqiFRY4T1SlZxzPcKPDcgFPnPODQwviyVU3K53zk07zjpuA+5yzjUvtuxK4E1CBey3QFfn3PJTyPNn4GqgVbgQYmb3A38Ckp1z683sVeCIc+7mYq/rSOjm50nOudzwdjU8znpuc841Dj9eDzzjnHus2HqGAxOBVOfcV+FlQ8PL4p1zRSfbBhGJHtoTJyIV3YqjvwkXnlzgy2LLjgC7gEQAM7vazOab2fbwmZxPAk2Lr9A5NwOYDNwP3H8qBS6sNbDQHfvT72clxnQGhpnZ/qO/gAXh584uNu5462lUfNr1BA4fLXBhW4FYQnvkROQMEvQ6gIjISRwp8didYFmMmXUH3gD+F/gVsBsYCDxWfHD4GLkuQCHQnFNnpzAmBniRUHksaUsp3utECko8PloE9UO5yBlGJU5Eosn5wBbn3J+OLjCzs44z7lFCx5b1BT4ws3865945hfVnAD8xMyu2F617iTFLgXOcc5knWVe346xnq3Nub/hxPhA4hUwicobST24iEk3WEpqSHGpmKWZ2OzCk+AAz60foRIJhzrk5wAOEToyofwrrfx5oBow1s1ZmdjVwW4kx/wd0DZ992snMmptZfzN7ocS4hiXW8xuO3Xu3HrjAzBqZWd1TyCYiZxiVOBGJGs65mYT2so0ldCxdpvFBmAAAAMJJREFUX+APR583s3rAy8BDzrlF4cWPAKuAiUcvO/I9699I6MzRfsByQlO295QYswLoSajsfRIe9xcgp8TqJhHa07YIGA9M4NgS9wegCbAO2HGSTReRM5DOThURibDw2akrnXOjvc4iIv6lPXEiIiIiPqQSJyISFj6Obf8JfumCuiJSoWg6VUQkzMwSgRNdp22vcy43knlERL6PSpyIiIiID2k6VURERMSHVOJEREREfEglTkRERMSHVOJEREREfOj/Aeg1AM5y7UnBAAAAAElFTkSuQmCC\n", 
                        "text/plain": "<matplotlib.figure.Figure at 0x7f661c17e128>"
                    }, 
                    "metadata": {}
                }
            ], 
            "source": "import matplotlib.pyplot as plt\nplt.figure(figsize=(10,6))\nu = list(accuracy_dict.keys())\nv = list(accuracy_dict.values())\nplt.scatter(x=u, y=v, linewidths=2, edgecolor='red')\nplt.plot(u,v)\nplt.xlabel('max_depth', fontsize=14)\nplt.ylabel('Test-Accuracy', fontsize=14)\nplt.title('Decision Tree model showing Test-Accuracy and corresponding max_depth', fontsize=15)\nplt.show()"
        }, 
        {
            "source": "#### Therefore The best Test accuracy Score for Decision_tree model is = 0.76 and the best max_depth is 1", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "# Support Vector Machine", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "First let's import SVM classifier from Sklearn", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 48, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "from sklearn import svm"
        }, 
        {
            "source": "Next let's confirm our X and y variables shape and view first 5 rows", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 49, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "X_train shape is: (252, 8)\nX_test shape is: (94, 8)\ny_train shape is: (252,)\ny_test shape is: (94, 8)\n"
                }
            ], 
            "source": "print('X_train shape is:',X_train.shape)\nprint('X_test shape is:',X_test.shape)\nprint('y_train shape is:',y_train.shape)\nprint('y_test shape is:',X_test.shape)"
        }, 
        {
            "execution_count": 50, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "X_train\n [[-1.31458942 -0.95911111  0.17588281  2.37778177  0.82934003 -0.38170062\n   1.13639374 -0.86968108]\n [ 0.51578458  0.92071769  0.17588281 -0.42056004  0.82934003 -0.38170062\n   1.13639374 -0.86968108]\n [ 0.51578458  0.92071769  0.50752015 -0.42056004  0.82934003 -0.38170062\n  -0.87997669  1.14984679]\n [-1.31458942  0.92071769 -0.81902922 -0.42056004  0.82934003 -0.38170062\n  -0.87997669  1.14984679]\n [-5.89052444 -1.96168647 -0.3215732  -0.42056004  0.82934003 -0.38170062\n  -0.87997669  1.14984679]]\n\ny_train\n 115    0\n16     0\n309    1\n17     0\n8      0\nName: loan_status, dtype: int64\n"
                }
            ], 
            "source": "# Let's just view first 5 rows of X_train and y_train\n\nprint('X_train\\n',X_train[:5])\nprint()\nprint('y_train\\n',y_train[:5])"
        }, 
        {
            "source": "Now let's instantiate the SVM model and pass in the kernel parameter<br>\nRemember kernel could be \u2018linear\u2019, \u2018poly\u2019, \u2018rbf\u2019, \u2018sigmoid\u2019.<br>\nI choose sigmoid first but will try them all to compare results", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "Now let's define a simple method that will compile and predict our data using the svm model and also compare accuracy based on the kernel type", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 51, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "def svm_classifier(X_train, y_train, X_test, y_test):\n    \"\"\"Method takes X_train and y_train 2D/1D arrays,\n    as well as test data x_test and y_test, also arrays, and returns,\n    The best Kernel value, and its test_accuracy score\"\"\"\n    \n    kernels = ['linear','poly','rbf','sigmoid']\n    acc = []\n    for i in kernels:\n        svm_model = svm.SVC(kernel=i)\n        # Let's train the model\n        svm_model.fit(X_train, y_train)\n        # let's mae a prediction on test data\n        y_hat = svm_model.predict(X_test)\n        # Let's measure accuracy on test data prediction\n        test_acc = metrics.accuracy_score(y_test, y_hat)\n        print(i,test_acc)\n        acc.append(test_acc)\n    j = acc.index(max(acc))\n    return [kernels[j], round(max(acc),2)] "
        }, 
        {
            "execution_count": 52, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "linear 0.755319148936\npoly 0.744680851064\nrbf 0.776595744681\nsigmoid 0.691489361702\n"
                }
            ], 
            "source": "best_kernel_max_acc = svm_classifier(X_train, y_train, X_test, y_test)"
        }, 
        {
            "execution_count": 53, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 53, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "['rbf', 0.78000000000000003]"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "best_kernel_max_acc"
        }, 
        {
            "execution_count": 54, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 54, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,\n  decision_function_shape='ovr', degree=3, gamma='auto', kernel='rbf',\n  max_iter=-1, probability=False, random_state=None, shrinking=True,\n  tol=0.001, verbose=False)"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "svm_model = svm.SVC(kernel='rbf')\nsvm_model.fit(X_train, y_train)"
        }, 
        {
            "source": "#### Therefore the best kernel for svm model is the linear kernel and the best test accuracy is 0.78", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": null, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": ""
        }, 
        {
            "source": "# Logistic Regression", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "As usual, let's first import The LinearRegression Classifier for this exercise", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 55, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "from sklearn.linear_model import LogisticRegression"
        }, 
        {
            "source": "Let's confirm the shapes and few rows of our Training and Testing sets", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 56, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "X_train shape is (252, 8)\nX_test shape is (94, 8)\ny_train shape is (252,)\ny_test shape is (94,)\n"
                }
            ], 
            "source": "print('X_train shape is',X_train.shape)\nprint('X_test shape is',X_test.shape)\nprint('y_train shape is',y_train.shape)\nprint('y_test shape is',y_test.shape)"
        }, 
        {
            "execution_count": 57, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "X_train\n [[-1.31458942 -0.95911111  0.17588281  2.37778177  0.82934003 -0.38170062\n   1.13639374 -0.86968108]\n [ 0.51578458  0.92071769  0.17588281 -0.42056004  0.82934003 -0.38170062\n   1.13639374 -0.86968108]\n [ 0.51578458  0.92071769  0.50752015 -0.42056004  0.82934003 -0.38170062\n  -0.87997669  1.14984679]\n [-1.31458942  0.92071769 -0.81902922 -0.42056004  0.82934003 -0.38170062\n  -0.87997669  1.14984679]\n [-5.89052444 -1.96168647 -0.3215732  -0.42056004  0.82934003 -0.38170062\n  -0.87997669  1.14984679]]\n\ny_train\n 115    0\n16     0\n309    1\n17     0\n8      0\nName: loan_status, dtype: int64\n"
                }
            ], 
            "source": "# Let's just view first 5 rows of X_train and y_train\n\nprint('X_train\\n',X_train[:5])\nprint()\nprint('y_train\\n',y_train[:5])"
        }, 
        {
            "source": "Now let's define a simple method that will compile and predict our data using the LogisticRegression model <br>and also compare accuracy based on the common numerical optimizers available, such as \u2018newton-cg\u2019, \u2018lbfgs\u2019, \u2018liblinear\u2019, \u2018sag\u2019, \u2018saga\u2019", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 58, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "def logistic_regression_classifier(X_train, y_train, X_test, y_test):\n    \"\"\"Method takes X_train and y_train 2D/1D arrays,\n    as well as test data x_test and y_test, also arrays, and returns,\n    The best Kernel value, and its test_accuracy score\"\"\"\n    \n    solvers = ['newton-cg','lbfgs','liblinear','sag','saga']\n    acc = []\n    for i in solvers:\n        # Let's instantiate and train the model in one line\n        logistic_model = LogisticRegression(C=0.01, solver=i).fit(X_train,y_train)\n        # Let's make a prediction on test data\n        y_hat = logistic_model.predict(X_test)\n        # let's measure accuracy on test data prediction\n        test_acc = metrics.accuracy_score(y_test, y_hat)\n        print(i,test_acc)\n        acc.append(test_acc)\n    j = acc.index(max(acc))\n    return [solvers[j], round(max(acc),2)] "
        }, 
        {
            "execution_count": 59, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "newton-cg 0.755319148936\nlbfgs 0.755319148936\nliblinear 0.712765957447\nsag 0.755319148936\nsaga 0.755319148936\n"
                }
            ], 
            "source": "best_logistic_solver_and_acc = logistic_regression_classifier(X_train, y_train, X_test, y_test)"
        }, 
        {
            "execution_count": 60, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 60, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "['newton-cg', 0.76000000000000001]"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "best_logistic_solver_and_acc"
        }, 
        {
            "execution_count": 61, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "logistic_model = LogisticRegression(C=0.01, solver='newton-cg').fit(X_train,y_train)"
        }, 
        {
            "source": "We can see that four out of five solvers:- newton-cg, lbfgs, sag, saga all output exact same accuracy, <br>so we choose newton-cg since its first on the list.<br>\n##### Therefore The best accuracy is 0.76 and the best solver for logistic regression is newton-cg", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "# Model Evaluation using Test set", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 62, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "from sklearn.metrics import jaccard_similarity_score\nfrom sklearn.metrics import f1_score\nfrom sklearn.metrics import log_loss"
        }, 
        {
            "source": "First, download and load the test set:", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 63, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "--2019-05-22 04:30:52--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_test.csv\nResolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.193\nConnecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.193|:443... connected.\nHTTP request sent, awaiting response... 200 OK\nLength: 3642 (3.6K) [text/csv]\nSaving to: \u2018loan_test.csv\u2019\n\n100%[======================================>] 3,642       --.-K/s   in 0s      \n\n2019-05-22 04:30:52 (634 MB/s) - \u2018loan_test.csv\u2019 saved [3642/3642]\n\n"
                }
            ], 
            "source": "!wget -O loan_test.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_test.csv"
        }, 
        {
            "source": "### Load Test set for evaluation ", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 64, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 64, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1</td>\n      <td>1</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/8/2016</td>\n      <td>10/7/2016</td>\n      <td>50</td>\n      <td>Bechalor</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>5</td>\n      <td>5</td>\n      <td>PAIDOFF</td>\n      <td>300</td>\n      <td>7</td>\n      <td>9/9/2016</td>\n      <td>9/15/2016</td>\n      <td>35</td>\n      <td>Master or Above</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>21</td>\n      <td>21</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/10/2016</td>\n      <td>10/9/2016</td>\n      <td>43</td>\n      <td>High School or Below</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>24</td>\n      <td>24</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/10/2016</td>\n      <td>10/9/2016</td>\n      <td>26</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>35</td>\n      <td>35</td>\n      <td>PAIDOFF</td>\n      <td>800</td>\n      <td>15</td>\n      <td>9/11/2016</td>\n      <td>9/25/2016</td>\n      <td>29</td>\n      <td>Bechalor</td>\n      <td>male</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           1             1     PAIDOFF       1000     30       9/8/2016   \n1           5             5     PAIDOFF        300      7       9/9/2016   \n2          21            21     PAIDOFF       1000     30      9/10/2016   \n3          24            24     PAIDOFF       1000     30      9/10/2016   \n4          35            35     PAIDOFF        800     15      9/11/2016   \n\n    due_date  age             education  Gender  \n0  10/7/2016   50              Bechalor  female  \n1  9/15/2016   35       Master or Above    male  \n2  10/9/2016   43  High School or Below  female  \n3  10/9/2016   26               college    male  \n4  9/25/2016   29              Bechalor    male  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "test_df = pd.read_csv('loan_test.csv')\ntest_df.head()"
        }, 
        {
            "source": "let's clean test data remove the first two unnnamed columns", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 65, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "test_df = test_df.iloc[:,2:]"
        }, 
        {
            "execution_count": 66, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 66, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/8/2016</td>\n      <td>10/7/2016</td>\n      <td>50</td>\n      <td>Bechalor</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>PAIDOFF</td>\n      <td>300</td>\n      <td>7</td>\n      <td>9/9/2016</td>\n      <td>9/15/2016</td>\n      <td>35</td>\n      <td>Master or Above</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/10/2016</td>\n      <td>10/9/2016</td>\n      <td>43</td>\n      <td>High School or Below</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/10/2016</td>\n      <td>10/9/2016</td>\n      <td>26</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>PAIDOFF</td>\n      <td>800</td>\n      <td>15</td>\n      <td>9/11/2016</td>\n      <td>9/25/2016</td>\n      <td>29</td>\n      <td>Bechalor</td>\n      <td>male</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "  loan_status  Principal  terms effective_date   due_date  age  \\\n0     PAIDOFF       1000     30       9/8/2016  10/7/2016   50   \n1     PAIDOFF        300      7       9/9/2016  9/15/2016   35   \n2     PAIDOFF       1000     30      9/10/2016  10/9/2016   43   \n3     PAIDOFF       1000     30      9/10/2016  10/9/2016   26   \n4     PAIDOFF        800     15      9/11/2016  9/25/2016   29   \n\n              education  Gender  \n0              Bechalor  female  \n1       Master or Above    male  \n2  High School or Below  female  \n3               college    male  \n4              Bechalor    male  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "test_df.head()"
        }, 
        {
            "execution_count": 67, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 67, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "(54, 8)"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "# Let's check the shape\ntest_df.shape"
        }, 
        {
            "source": "Let's convert categorical to numeric data", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 68, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "test_df.loan_status.replace(to_replace=['PAIDOFF','COLLECTION'],value=[0,1],inplace=True)"
        }, 
        {
            "execution_count": 69, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "test_df.Gender.replace(to_replace=['male','female'],value=[0,1],inplace=True)"
        }, 
        {
            "execution_count": 70, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 70, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/8/2016</td>\n      <td>10/7/2016</td>\n      <td>50</td>\n      <td>Bechalor</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>0</td>\n      <td>300</td>\n      <td>7</td>\n      <td>9/9/2016</td>\n      <td>9/15/2016</td>\n      <td>35</td>\n      <td>Master or Above</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>0</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/10/2016</td>\n      <td>10/9/2016</td>\n      <td>43</td>\n      <td>High School or Below</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>0</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/10/2016</td>\n      <td>10/9/2016</td>\n      <td>26</td>\n      <td>college</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>0</td>\n      <td>800</td>\n      <td>15</td>\n      <td>9/11/2016</td>\n      <td>9/25/2016</td>\n      <td>29</td>\n      <td>Bechalor</td>\n      <td>0</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "   loan_status  Principal  terms effective_date   due_date  age  \\\n0            0       1000     30       9/8/2016  10/7/2016   50   \n1            0        300      7       9/9/2016  9/15/2016   35   \n2            0       1000     30      9/10/2016  10/9/2016   43   \n3            0       1000     30      9/10/2016  10/9/2016   26   \n4            0        800     15      9/11/2016  9/25/2016   29   \n\n              education  Gender  \n0              Bechalor       1  \n1       Master or Above       0  \n2  High School or Below       1  \n3               college       0  \n4              Bechalor       0  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "test_df.head()"
        }, 
        {
            "execution_count": 71, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 71, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "education             loan_status\nBechalor              0              1.000000\nHigh School or Below  0              0.523810\n                      1              0.476190\nMaster or Above       0              1.000000\ncollege               0              0.826087\n                      1              0.173913\nName: loan_status, dtype: float64"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "# Lets investigate the relationship between education and loan status\n\ntest_df.groupby(['education'])['loan_status'].value_counts(normalize=True)"
        }, 
        {
            "execution_count": 72, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "# Let's convert date to date-time-object like we did in loan_train_data\n\ntest_df['dayofweek'] = df['effective_date'].dt.dayofweek"
        }, 
        {
            "execution_count": 73, 
            "cell_type": "code", 
            "metadata": {
                "scrolled": false
            }, 
            "outputs": [
                {
                    "execution_count": 73, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n      <th>dayofweek</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/8/2016</td>\n      <td>10/7/2016</td>\n      <td>50</td>\n      <td>Bechalor</td>\n      <td>1</td>\n      <td>3</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>0</td>\n      <td>300</td>\n      <td>7</td>\n      <td>9/9/2016</td>\n      <td>9/15/2016</td>\n      <td>35</td>\n      <td>Master or Above</td>\n      <td>0</td>\n      <td>3</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>0</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/10/2016</td>\n      <td>10/9/2016</td>\n      <td>43</td>\n      <td>High School or Below</td>\n      <td>1</td>\n      <td>3</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>0</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/10/2016</td>\n      <td>10/9/2016</td>\n      <td>26</td>\n      <td>college</td>\n      <td>0</td>\n      <td>4</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>0</td>\n      <td>800</td>\n      <td>15</td>\n      <td>9/11/2016</td>\n      <td>9/25/2016</td>\n      <td>29</td>\n      <td>Bechalor</td>\n      <td>0</td>\n      <td>4</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "   loan_status  Principal  terms effective_date   due_date  age  \\\n0            0       1000     30       9/8/2016  10/7/2016   50   \n1            0        300      7       9/9/2016  9/15/2016   35   \n2            0       1000     30      9/10/2016  10/9/2016   43   \n3            0       1000     30      9/10/2016  10/9/2016   26   \n4            0        800     15      9/11/2016  9/25/2016   29   \n\n              education  Gender  dayofweek  \n0              Bechalor       1          3  \n1       Master or Above       0          3  \n2  High School or Below       1          3  \n3               college       0          4  \n4              Bechalor       0          4  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "test_df.head()"
        }, 
        {
            "execution_count": 74, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 74, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n      <th>dayofweek</th>\n      <th>weekend</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/8/2016</td>\n      <td>10/7/2016</td>\n      <td>50</td>\n      <td>Bechalor</td>\n      <td>1</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>0</td>\n      <td>300</td>\n      <td>7</td>\n      <td>9/9/2016</td>\n      <td>9/15/2016</td>\n      <td>35</td>\n      <td>Master or Above</td>\n      <td>0</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>0</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/10/2016</td>\n      <td>10/9/2016</td>\n      <td>43</td>\n      <td>High School or Below</td>\n      <td>1</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>0</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/10/2016</td>\n      <td>10/9/2016</td>\n      <td>26</td>\n      <td>college</td>\n      <td>0</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>0</td>\n      <td>800</td>\n      <td>15</td>\n      <td>9/11/2016</td>\n      <td>9/25/2016</td>\n      <td>29</td>\n      <td>Bechalor</td>\n      <td>0</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "   loan_status  Principal  terms effective_date   due_date  age  \\\n0            0       1000     30       9/8/2016  10/7/2016   50   \n1            0        300      7       9/9/2016  9/15/2016   35   \n2            0       1000     30      9/10/2016  10/9/2016   43   \n3            0       1000     30      9/10/2016  10/9/2016   26   \n4            0        800     15      9/11/2016  9/25/2016   29   \n\n              education  Gender  dayofweek  weekend  \n0              Bechalor       1          3        0  \n1       Master or Above       0          3        0  \n2  High School or Below       1          3        0  \n3               college       0          4        1  \n4              Bechalor       0          4        1  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "# Let's define the weekend and day of week variables as we did before\n\ntest_df['weekend'] = test_df['dayofweek'].apply(lambda x: 1 if (x>3)  else 0)\ntest_df.head()"
        }, 
        {
            "source": "### Defining our Testing Data feature set ", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 75, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 75, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>weekend</th>\n      <th>Bechalor</th>\n      <th>High School or Below</th>\n      <th>Master or Above</th>\n      <th>college</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>50</td>\n      <td>1</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>300</td>\n      <td>7</td>\n      <td>35</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>43</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>26</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>800</td>\n      <td>15</td>\n      <td>29</td>\n      <td>0</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n0       1000     30   50       1        0         1                     0   \n1        300      7   35       0        0         0                     0   \n2       1000     30   43       1        0         0                     1   \n3       1000     30   26       0        1         0                     0   \n4        800     15   29       0        1         1                     0   \n\n   Master or Above  college  \n0                0        0  \n1                1        0  \n2                0        0  \n3                0        1  \n4                0        0  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "Feature = test_df[['Principal','terms','age','Gender','weekend']]\nFeature = pd.concat([Feature,pd.get_dummies(test_df['education'])], axis=1)\nFeature.head()"
        }, 
        {
            "execution_count": 76, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "# deleting MasterorAbove columnlike we did before\n\ndel(Feature['Master or Above'])"
        }, 
        {
            "execution_count": 77, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 77, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>weekend</th>\n      <th>Bechalor</th>\n      <th>High School or Below</th>\n      <th>college</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>50</td>\n      <td>1</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>300</td>\n      <td>7</td>\n      <td>35</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>43</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>26</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>800</td>\n      <td>15</td>\n      <td>29</td>\n      <td>0</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n0       1000     30   50       1        0         1                     0   \n1        300      7   35       0        0         0                     0   \n2       1000     30   43       1        0         0                     1   \n3       1000     30   26       0        1         0                     0   \n4        800     15   29       0        1         1                     0   \n\n   college  \n0        0  \n1        0  \n2        0  \n3        1  \n4        0  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "Feature.head()"
        }, 
        {
            "execution_count": 78, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 78, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "Principal               0\nterms                   0\nage                     0\nGender                  0\nweekend                 0\nBechalor                0\nHigh School or Below    0\ncollege                 0\ndtype: int64"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "# Checking for any NaN values\n\nFeature.isna().sum()"
        }, 
        {
            "source": "### Normalizing our Test Data predictor variables", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 79, 
            "cell_type": "code", 
            "metadata": {
                "scrolled": true
            }, 
            "outputs": [
                {
                    "execution_count": 79, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "array([[ 0.49362588,  0.92844966,  3.05981865,  1.97714211, -4.12310563,\n         2.39791576, -0.79772404, -0.86135677],\n       [-3.56269116, -1.70427745,  0.53336288, -0.50578054, -4.12310563,\n        -0.41702883, -0.79772404, -0.86135677],\n       [ 0.49362588,  0.92844966,  1.88080596,  1.97714211, -4.12310563,\n        -0.41702883,  1.25356634, -0.86135677],\n       [ 0.49362588,  0.92844966, -0.98251057, -0.50578054,  0.24253563,\n        -0.41702883, -0.79772404,  1.16095912],\n       [-0.66532184, -0.78854628, -0.47721942, -0.50578054,  0.24253563,\n         2.39791576, -0.79772404, -0.86135677]])"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "X = Feature\nX_Test_Train= preprocessing.StandardScaler().fit(X).transform(X)\nX_Test_Train[0:5]"
        }, 
        {
            "source": "### Defining our Target variable for the Test data", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 80, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 80, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "0    0\n1    0\n2    0\n3    0\n4    0\nName: loan_status, dtype: int64"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "Y_Test_True = test_df.loan_status\nY_Test_True[:5]"
        }, 
        {
            "execution_count": 81, 
            "cell_type": "code", 
            "metadata": {
                "scrolled": false
            }, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "X_Test_Train shape is (54, 8)\nY_Test_True shape is (54,)\n"
                }
            ], 
            "source": "print('X_Test_Train shape is',X_Test_Train.shape)\nprint('Y_Test_True shape is',Y_Test_True.shape)"
        }, 
        {
            "source": "### Models Prediction on Test Data", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "knn_classifier_model", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 82, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "knn_yhat = knn_model.predict(X_Test_Train)"
        }, 
        {
            "source": "decision_tree_model", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 83, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "decision_yhat = decision_model.predict(X_Test_Train)"
        }, 
        {
            "source": "logistic_regression_model ", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 84, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "logistic_yhat = logistic_model.predict(X_Test_Train)"
        }, 
        {
            "source": "svm_classifier_model", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 85, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "svm_yhat = svm_model.predict(X_Test_Train)"
        }, 
        {
            "source": "# Report\nYou should be able to report the accuracy of the built model using different evaluation metrics:", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "| Algorithm          | Jaccard | F1-score | LogLoss |\n|--------------------|---------|----------|---------|\n| KNN                | ?       | ?        | NA      |\n| Decision Tree      | ?       | ?        | NA      |\n| SVM                | ?       | ?        | NA      |\n| LogisticRegression | ?       | ?        | ?       |", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 86, 
            "cell_type": "code", 
            "metadata": {
                "scrolled": true
            }, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stderr", 
                    "text": "/opt/conda/envs/DSX-Python35/lib/python3.5/site-packages/sklearn/metrics/classification.py:1135: UndefinedMetricWarning: F-score is ill-defined and being set to 0.0 in labels with no predicted samples.\n  'precision', 'predicted', average, warn_for)\n"
                }, 
                {
                    "execution_count": 86, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Jaccard</th>\n      <th>F1_Score</th>\n      <th>LogLoss</th>\n      <th>Accuracy</th>\n      <th>Sum_per_Model</th>\n    </tr>\n    <tr>\n      <th>Algorithm</th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n      <th></th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>KNN</th>\n      <td>0.6667</td>\n      <td>0.6152</td>\n      <td>11.5130</td>\n      <td>0.6667</td>\n      <td>-9.5645</td>\n    </tr>\n    <tr>\n      <th>Decision_Tree</th>\n      <td>0.7407</td>\n      <td>0.6304</td>\n      <td>8.9545</td>\n      <td>0.7407</td>\n      <td>-6.8426</td>\n    </tr>\n    <tr>\n      <th>SVM</th>\n      <td>0.7407</td>\n      <td>0.6304</td>\n      <td>8.9545</td>\n      <td>0.7407</td>\n      <td>-6.8426</td>\n    </tr>\n    <tr>\n      <th>Logistic_Regression</th>\n      <td>0.7407</td>\n      <td>0.6304</td>\n      <td>8.9545</td>\n      <td>0.7407</td>\n      <td>-6.8426</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "                     Jaccard  F1_Score  LogLoss  Accuracy  Sum_per_Model\nAlgorithm                                                               \nKNN                   0.6667    0.6152  11.5130    0.6667        -9.5645\nDecision_Tree         0.7407    0.6304   8.9545    0.7407        -6.8426\nSVM                   0.7407    0.6304   8.9545    0.7407        -6.8426\nLogistic_Regression   0.7407    0.6304   8.9545    0.7407        -6.8426"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "Report={'Algorithm':['KNN','Decision_Tree','SVM','Logistic_Regression'],\n       'Jaccard':[jaccard_similarity_score(Y_Test_True, knn_yhat),\n                  jaccard_similarity_score(Y_Test_True, decision_yhat),\n                  jaccard_similarity_score(Y_Test_True, svm_yhat),\n                  jaccard_similarity_score(Y_Test_True, logistic_yhat)],\n       'F1_Score':[f1_score(Y_Test_True, knn_yhat, average='weighted'),\n                   f1_score(Y_Test_True, decision_yhat,average='weighted'),\n                   f1_score(Y_Test_True, svm_yhat,average='weighted'),\n                   f1_score(Y_Test_True, logistic_yhat,average='weighted')],\n       'LogLoss':[log_loss(Y_Test_True, knn_yhat),\n                  log_loss(Y_Test_True, decision_yhat),\n                  log_loss(Y_Test_True, svm_yhat),\n                  log_loss(Y_Test_True, logistic_yhat)],\n       'Accuracy':[metrics.accuracy_score(Y_Test_True, knn_yhat),\n                   metrics.accuracy_score(Y_Test_True, decision_yhat),\n                   metrics.accuracy_score(Y_Test_True, svm_yhat),\n                   metrics.accuracy_score(Y_Test_True, logistic_yhat)]}\n\nReport_df = pd.DataFrame(Report, columns=['Algorithm', 'Jaccard','F1_Score','LogLoss','Accuracy'])\nReport_df = Report_df.set_index('Algorithm')\n\n# To find the cumulative average performance per model, we sum the total for all models and find the percent for each model \n\nReport_df['Sum_per_Model'] = Report_df.apply(lambda x: x.Jaccard + x.F1_Score + x.Accuracy - x.LogLoss, axis=1)\nReport_df = round(Report_df, 4)\nReport_df.head()"
        }, 
        {
            "source": "### From the above table, the best performing model is tied to the trio of Decision Tree, SVM and Logistic_Regression<br>\n### KNN performs slightly worse than the rest", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 1, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "# The code was removed by Watson Studio for sharing."
        }, 
        {
            "source": "<h2>Want to learn more?</h2>\n\nIBM SPSS Modeler is a comprehensive analytics platform that has many machine learning algorithms. It has been designed to bring predictive intelligence to decisions made by individuals, by groups, by systems \u2013 by your enterprise as a whole. A free trial is available through this course, available here: <a href=\"http://cocl.us/ML0101EN-SPSSModeler\">SPSS Modeler</a>\n\nAlso, you can use Watson Studio to run these notebooks faster with bigger datasets. Watson Studio is IBM's leading cloud solution for data scientists, built by data scientists. With Jupyter notebooks, RStudio, Apache Spark and popular libraries pre-packaged in the cloud, Watson Studio enables data scientists to collaborate on their projects without having to install anything. Join the fast-growing community of Watson Studio users today with a free account at <a href=\"https://cocl.us/ML0101EN_DSX\">Watson Studio</a>\n\n<h3>Thanks for completing this lesson!</h3>\n\n<h4>Author:  <a href=\"https://ca.linkedin.com/in/saeedaghabozorgi\">Saeed Aghabozorgi</a></h4>\n<p><a href=\"https://ca.linkedin.com/in/saeedaghabozorgi\">Saeed Aghabozorgi</a>, PhD is a Data Scientist in IBM with a track record of developing enterprise level applications that substantially increases clients\u2019 ability to turn data into actionable knowledge. He is a researcher in data mining field and expert in developing advanced analytic methods like machine learning and statistical modelling on large datasets.</p>\n\n<hr>\n\n<p>Copyright &copy; 2018 <a href=\"https://cocl.us/DX0108EN_CC\">Cognitive Class</a>. This notebook and its source code are released under the terms of the <a href=\"https://bigdatauniversity.com/mit-license/\">MIT License</a>.</p>", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }
    ], 
    "metadata": {
        "kernelspec": {
            "display_name": "Python 3.5", 
            "name": "python3", 
            "language": "python"
        }, 
        "language_info": {
            "mimetype": "text/x-python", 
            "nbconvert_exporter": "python", 
            "version": "3.5.5", 
            "name": "python", 
            "file_extension": ".py", 
            "pygments_lexer": "ipython3", 
            "codemirror_mode": {
                "version": 3, 
                "name": "ipython"
            }
        }
    }, 
    "nbformat": 4
}