income-prediction.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "id": "86abac76",
      "metadata": {
        "id": "86abac76"
      },
      "source": [
        "# Income Prediction\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "60c95671",
      "metadata": {
        "id": "60c95671"
      },
      "source": [
        "## Prepare the dataset\n",
        "\n",
        "We will use the [Census Income dataset](https://archive.ics.uci.edu/ml/datasets/Adult). The task for this dataset is to predict whether an individual's income exceeds $50,000 per year or not based on a set of attributes/features (including age, occupation, education, and other factors). The dataset was constructed from the 1994 US Census database."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "29a5e8cb",
      "metadata": {
        "id": "29a5e8cb",
        "outputId": "61c3542c-f4eb-4b30-8290-28df381984bb",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 546
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "       age   workclass      education       marital-status         occupation  \\\n",
              "22278   27     Private   Some-college             Divorced       Adm-clerical   \n",
              "8950    27     Private      Bachelors        Never-married     Prof-specialty   \n",
              "7838    25     Private     Assoc-acdm   Married-civ-spouse              Sales   \n",
              "16505   46     Private        5th-6th   Married-civ-spouse   Transport-moving   \n",
              "19140   45     Private           11th             Divorced   Transport-moving   \n",
              "12319   29     Private      Bachelors   Married-civ-spouse     Prof-specialty   \n",
              "28589   42   Local-gov        HS-grad            Separated      Other-service   \n",
              "10000   34     Private   Some-college             Divorced       Adm-clerical   \n",
              "28530   60   Local-gov      Assoc-voc             Divorced     Prof-specialty   \n",
              "24237   19     Private   Some-college        Never-married       Adm-clerical   \n",
              "\n",
              "         relationship                 race      sex  capital-gain  \\\n",
              "22278       Unmarried                White   Female             0   \n",
              "8950    Not-in-family                White   Female             0   \n",
              "7838          Husband                White     Male             0   \n",
              "16505         Husband   Amer-Indian-Eskimo     Male             0   \n",
              "19140   Not-in-family                White     Male             0   \n",
              "12319       Own-child                White     Male             0   \n",
              "28589   Not-in-family                Black   Female             0   \n",
              "10000   Not-in-family                White   Female             0   \n",
              "28530       Unmarried                White     Male             0   \n",
              "24237       Own-child                White   Female             0   \n",
              "\n",
              "       capital-loss  hours-per-week  native-country   class  \n",
              "22278             0              44   United-States   <=50K  \n",
              "8950              0              40   United-States   <=50K  \n",
              "7838              0              40   United-States   <=50K  \n",
              "16505          1902              40   United-States   <=50K  \n",
              "19140          2824              76   United-States    >50K  \n",
              "12319             0              75   United-States   <=50K  \n",
              "28589             0              60             NaN   <=50K  \n",
              "10000             0              50   United-States   <=50K  \n",
              "28530             0              40   United-States    >50K  \n",
              "24237             0              40   United-States   <=50K  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-a553267d-0013-45b1-83c4-8179f087e46d\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>age</th>\n",
              "      <th>workclass</th>\n",
              "      <th>education</th>\n",
              "      <th>marital-status</th>\n",
              "      <th>occupation</th>\n",
              "      <th>relationship</th>\n",
              "      <th>race</th>\n",
              "      <th>sex</th>\n",
              "      <th>capital-gain</th>\n",
              "      <th>capital-loss</th>\n",
              "      <th>hours-per-week</th>\n",
              "      <th>native-country</th>\n",
              "      <th>class</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>22278</th>\n",
              "      <td>27</td>\n",
              "      <td>Private</td>\n",
              "      <td>Some-college</td>\n",
              "      <td>Divorced</td>\n",
              "      <td>Adm-clerical</td>\n",
              "      <td>Unmarried</td>\n",
              "      <td>White</td>\n",
              "      <td>Female</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>44</td>\n",
              "      <td>United-States</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8950</th>\n",
              "      <td>27</td>\n",
              "      <td>Private</td>\n",
              "      <td>Bachelors</td>\n",
              "      <td>Never-married</td>\n",
              "      <td>Prof-specialty</td>\n",
              "      <td>Not-in-family</td>\n",
              "      <td>White</td>\n",
              "      <td>Female</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>40</td>\n",
              "      <td>United-States</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7838</th>\n",
              "      <td>25</td>\n",
              "      <td>Private</td>\n",
              "      <td>Assoc-acdm</td>\n",
              "      <td>Married-civ-spouse</td>\n",
              "      <td>Sales</td>\n",
              "      <td>Husband</td>\n",
              "      <td>White</td>\n",
              "      <td>Male</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>40</td>\n",
              "      <td>United-States</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>16505</th>\n",
              "      <td>46</td>\n",
              "      <td>Private</td>\n",
              "      <td>5th-6th</td>\n",
              "      <td>Married-civ-spouse</td>\n",
              "      <td>Transport-moving</td>\n",
              "      <td>Husband</td>\n",
              "      <td>Amer-Indian-Eskimo</td>\n",
              "      <td>Male</td>\n",
              "      <td>0</td>\n",
              "      <td>1902</td>\n",
              "      <td>40</td>\n",
              "      <td>United-States</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19140</th>\n",
              "      <td>45</td>\n",
              "      <td>Private</td>\n",
              "      <td>11th</td>\n",
              "      <td>Divorced</td>\n",
              "      <td>Transport-moving</td>\n",
              "      <td>Not-in-family</td>\n",
              "      <td>White</td>\n",
              "      <td>Male</td>\n",
              "      <td>0</td>\n",
              "      <td>2824</td>\n",
              "      <td>76</td>\n",
              "      <td>United-States</td>\n",
              "      <td>&gt;50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>12319</th>\n",
              "      <td>29</td>\n",
              "      <td>Private</td>\n",
              "      <td>Bachelors</td>\n",
              "      <td>Married-civ-spouse</td>\n",
              "      <td>Prof-specialty</td>\n",
              "      <td>Own-child</td>\n",
              "      <td>White</td>\n",
              "      <td>Male</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>75</td>\n",
              "      <td>United-States</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>28589</th>\n",
              "      <td>42</td>\n",
              "      <td>Local-gov</td>\n",
              "      <td>HS-grad</td>\n",
              "      <td>Separated</td>\n",
              "      <td>Other-service</td>\n",
              "      <td>Not-in-family</td>\n",
              "      <td>Black</td>\n",
              "      <td>Female</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>60</td>\n",
              "      <td>NaN</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10000</th>\n",
              "      <td>34</td>\n",
              "      <td>Private</td>\n",
              "      <td>Some-college</td>\n",
              "      <td>Divorced</td>\n",
              "      <td>Adm-clerical</td>\n",
              "      <td>Not-in-family</td>\n",
              "      <td>White</td>\n",
              "      <td>Female</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>50</td>\n",
              "      <td>United-States</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>28530</th>\n",
              "      <td>60</td>\n",
              "      <td>Local-gov</td>\n",
              "      <td>Assoc-voc</td>\n",
              "      <td>Divorced</td>\n",
              "      <td>Prof-specialty</td>\n",
              "      <td>Unmarried</td>\n",
              "      <td>White</td>\n",
              "      <td>Male</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>40</td>\n",
              "      <td>United-States</td>\n",
              "      <td>&gt;50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24237</th>\n",
              "      <td>19</td>\n",
              "      <td>Private</td>\n",
              "      <td>Some-college</td>\n",
              "      <td>Never-married</td>\n",
              "      <td>Adm-clerical</td>\n",
              "      <td>Own-child</td>\n",
              "      <td>White</td>\n",
              "      <td>Female</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>40</td>\n",
              "      <td>United-States</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-a553267d-0013-45b1-83c4-8179f087e46d')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-a553267d-0013-45b1-83c4-8179f087e46d button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-a553267d-0013-45b1-83c4-8179f087e46d');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 19
        }
      ],
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "seed = 0\n",
        "np.random.seed(0)\n",
        "data = pd.read_csv('https://archive.ics.uci.edu/ml/machine-learning-databases/adult/adult.data', header=None)\n",
        "data.columns = ['age', 'workclass', 'fnlwgt', 'education', 'education-num', \n",
        "                'marital-status', 'occupation', 'relationship', 'race', 'sex', \n",
        "                'capital-gain', 'capital-loss', 'hours-per-week', 'native-country',\n",
        "                'class']\n",
        "\n",
        "data = data.drop(['fnlwgt', 'education-num'], axis=1)\n",
        "\n",
        "data = data.replace(' ?', np.nan)\n",
        "\n",
        "data.sample(10, random_state=seed)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "db452ae8",
      "metadata": {
        "id": "db452ae8",
        "outputId": "22432eaf-74fc-4500-aaab-1891fed0a673",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(32561, 13)"
            ]
          },
          "metadata": {},
          "execution_count": 20
        }
      ],
      "source": [
        "data.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "6209c67f",
      "metadata": {
        "id": "6209c67f",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "1c96127d-d08b-4b99-c4aa-a568f8e9f4e6"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Number of NaN values\n",
            "age                  0\n",
            "workclass         1836\n",
            "education            0\n",
            "marital-status       0\n",
            "occupation        1843\n",
            "relationship         0\n",
            "race                 0\n",
            "sex                  0\n",
            "capital-gain         0\n",
            "capital-loss         0\n",
            "hours-per-week       0\n",
            "native-country     583\n",
            "class                0\n",
            "dtype: int64\n",
            "\n",
            "\n",
            "Number of Nan Values after categoricalImputer\n",
            "age               0\n",
            "workclass         0\n",
            "education         0\n",
            "marital-status    0\n",
            "occupation        0\n",
            "relationship      0\n",
            "race              0\n",
            "sex               0\n",
            "capital-gain      0\n",
            "capital-loss      0\n",
            "hours-per-week    0\n",
            "native-country    0\n",
            "class             0\n",
            "dtype: int64\n"
          ]
        }
      ],
      "source": [
        "from sklearn_pandas import CategoricalImputer\n",
        "from sklearn.preprocessing import LabelEncoder\n",
        "print(\"Number of NaN values\")\n",
        "print(data.isna().sum())\n",
        "print(\"\\n\")\n",
        "# Using the most frequent strategy for replacing NaN values\n",
        "categoricalImputer = CategoricalImputer(strategy = 'most_frequent')\n",
        "data['workclass'] = categoricalImputer.fit_transform(data['workclass'])\n",
        "data['native-country'] = categoricalImputer.fit_transform(data['native-country'])\n",
        "data['occupation'] = categoricalImputer.fit_transform(data['occupation'])\n",
        "print(\"Number of Nan Values after categoricalImputer\")\n",
        "print(data.isna().sum())\n"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Finding features to exclude by checking correlation with class(income)\n",
        "# We'll copy data to dataPreprocessing to find features to ignore\n",
        "# Encoding the categorical data\n",
        "encoder = LabelEncoder()\n",
        "dataPreprocessing = data.copy()\n",
        "dataPreprocessing[['class', 'workclass', 'education', 'marital-status', 'occupation', 'relationship', 'race', 'sex', 'native-country']] = dataPreprocessing[['class', 'workclass', 'education', 'marital-status', 'occupation', 'relationship', 'race', 'sex', 'native-country']].apply(encoder.fit_transform)\n",
        "dataPreprocessing.sample(10, random_state=seed)\n",
        "\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 363
        },
        "id": "28_eYpw7YDXa",
        "outputId": "0e8fd227-56fa-4963-cab8-73f472b51e29"
      },
      "id": "28_eYpw7YDXa",
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "       age  workclass  education  marital-status  occupation  relationship  \\\n",
              "22278   27          3         15               0           0             4   \n",
              "8950    27          3          9               4           9             1   \n",
              "7838    25          3          7               2          11             0   \n",
              "16505   46          3          4               2          13             0   \n",
              "19140   45          3          1               0          13             1   \n",
              "12319   29          3          9               2           9             3   \n",
              "28589   42          1         11               5           7             1   \n",
              "10000   34          3         15               0           0             1   \n",
              "28530   60          1          8               0           9             4   \n",
              "24237   19          3         15               4           0             3   \n",
              "\n",
              "       race  sex  capital-gain  capital-loss  hours-per-week  native-country  \\\n",
              "22278     4    0             0             0              44              38   \n",
              "8950      4    0             0             0              40              38   \n",
              "7838      4    1             0             0              40              38   \n",
              "16505     0    1             0          1902              40              38   \n",
              "19140     4    1             0          2824              76              38   \n",
              "12319     4    1             0             0              75              38   \n",
              "28589     2    0             0             0              60              38   \n",
              "10000     4    0             0             0              50              38   \n",
              "28530     4    1             0             0              40              38   \n",
              "24237     4    0             0             0              40              38   \n",
              "\n",
              "       class  \n",
              "22278      0  \n",
              "8950       0  \n",
              "7838       0  \n",
              "16505      0  \n",
              "19140      1  \n",
              "12319      0  \n",
              "28589      0  \n",
              "10000      0  \n",
              "28530      1  \n",
              "24237      0  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-976e7ed5-8f18-4633-84af-67d806c43369\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>age</th>\n",
              "      <th>workclass</th>\n",
              "      <th>education</th>\n",
              "      <th>marital-status</th>\n",
              "      <th>occupation</th>\n",
              "      <th>relationship</th>\n",
              "      <th>race</th>\n",
              "      <th>sex</th>\n",
              "      <th>capital-gain</th>\n",
              "      <th>capital-loss</th>\n",
              "      <th>hours-per-week</th>\n",
              "      <th>native-country</th>\n",
              "      <th>class</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>22278</th>\n",
              "      <td>27</td>\n",
              "      <td>3</td>\n",
              "      <td>15</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>4</td>\n",
              "      <td>4</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>44</td>\n",
              "      <td>38</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8950</th>\n",
              "      <td>27</td>\n",
              "      <td>3</td>\n",
              "      <td>9</td>\n",
              "      <td>4</td>\n",
              "      <td>9</td>\n",
              "      <td>1</td>\n",
              "      <td>4</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>40</td>\n",
              "      <td>38</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7838</th>\n",
              "      <td>25</td>\n",
              "      <td>3</td>\n",
              "      <td>7</td>\n",
              "      <td>2</td>\n",
              "      <td>11</td>\n",
              "      <td>0</td>\n",
              "      <td>4</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>40</td>\n",
              "      <td>38</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>16505</th>\n",
              "      <td>46</td>\n",
              "      <td>3</td>\n",
              "      <td>4</td>\n",
              "      <td>2</td>\n",
              "      <td>13</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1902</td>\n",
              "      <td>40</td>\n",
              "      <td>38</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19140</th>\n",
              "      <td>45</td>\n",
              "      <td>3</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>13</td>\n",
              "      <td>1</td>\n",
              "      <td>4</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>2824</td>\n",
              "      <td>76</td>\n",
              "      <td>38</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>12319</th>\n",
              "      <td>29</td>\n",
              "      <td>3</td>\n",
              "      <td>9</td>\n",
              "      <td>2</td>\n",
              "      <td>9</td>\n",
              "      <td>3</td>\n",
              "      <td>4</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>75</td>\n",
              "      <td>38</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>28589</th>\n",
              "      <td>42</td>\n",
              "      <td>1</td>\n",
              "      <td>11</td>\n",
              "      <td>5</td>\n",
              "      <td>7</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>60</td>\n",
              "      <td>38</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10000</th>\n",
              "      <td>34</td>\n",
              "      <td>3</td>\n",
              "      <td>15</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>4</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>50</td>\n",
              "      <td>38</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>28530</th>\n",
              "      <td>60</td>\n",
              "      <td>1</td>\n",
              "      <td>8</td>\n",
              "      <td>0</td>\n",
              "      <td>9</td>\n",
              "      <td>4</td>\n",
              "      <td>4</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>40</td>\n",
              "      <td>38</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24237</th>\n",
              "      <td>19</td>\n",
              "      <td>3</td>\n",
              "      <td>15</td>\n",
              "      <td>4</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>4</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>40</td>\n",
              "      <td>38</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-976e7ed5-8f18-4633-84af-67d806c43369')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-976e7ed5-8f18-4633-84af-67d806c43369 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-976e7ed5-8f18-4633-84af-67d806c43369');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Scalling the numerical data using MinMaxScaler\n",
        "from sklearn.preprocessing import MinMaxScaler\n",
        "scaler = MinMaxScaler()\n",
        "dataPreprocessing[['age', 'capital-gain', 'capital-loss', 'hours-per-week']] = scaler.fit_transform(dataPreprocessing[['age', 'capital-gain', 'capital-loss', 'hours-per-week']])\n",
        "dataPreprocessing.sample(10, random_state=seed)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 363
        },
        "id": "xTv-s77hYNF7",
        "outputId": "3d574639-1831-49ee-93c8-480da35f4e6e"
      },
      "id": "xTv-s77hYNF7",
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "            age  workclass  education  marital-status  occupation  \\\n",
              "22278  0.136986          3         15               0           0   \n",
              "8950   0.136986          3          9               4           9   \n",
              "7838   0.109589          3          7               2          11   \n",
              "16505  0.397260          3          4               2          13   \n",
              "19140  0.383562          3          1               0          13   \n",
              "12319  0.164384          3          9               2           9   \n",
              "28589  0.342466          1         11               5           7   \n",
              "10000  0.232877          3         15               0           0   \n",
              "28530  0.589041          1          8               0           9   \n",
              "24237  0.027397          3         15               4           0   \n",
              "\n",
              "       relationship  race  sex  capital-gain  capital-loss  hours-per-week  \\\n",
              "22278             4     4    0           0.0      0.000000        0.438776   \n",
              "8950              1     4    0           0.0      0.000000        0.397959   \n",
              "7838              0     4    1           0.0      0.000000        0.397959   \n",
              "16505             0     0    1           0.0      0.436639        0.397959   \n",
              "19140             1     4    1           0.0      0.648301        0.765306   \n",
              "12319             3     4    1           0.0      0.000000        0.755102   \n",
              "28589             1     2    0           0.0      0.000000        0.602041   \n",
              "10000             1     4    0           0.0      0.000000        0.500000   \n",
              "28530             4     4    1           0.0      0.000000        0.397959   \n",
              "24237             3     4    0           0.0      0.000000        0.397959   \n",
              "\n",
              "       native-country  class  \n",
              "22278              38      0  \n",
              "8950               38      0  \n",
              "7838               38      0  \n",
              "16505              38      0  \n",
              "19140              38      1  \n",
              "12319              38      0  \n",
              "28589              38      0  \n",
              "10000              38      0  \n",
              "28530              38      1  \n",
              "24237              38      0  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-ba46ebbc-d1f4-4454-8b75-e74a438efe68\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>age</th>\n",
              "      <th>workclass</th>\n",
              "      <th>education</th>\n",
              "      <th>marital-status</th>\n",
              "      <th>occupation</th>\n",
              "      <th>relationship</th>\n",
              "      <th>race</th>\n",
              "      <th>sex</th>\n",
              "      <th>capital-gain</th>\n",
              "      <th>capital-loss</th>\n",
              "      <th>hours-per-week</th>\n",
              "      <th>native-country</th>\n",
              "      <th>class</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>22278</th>\n",
              "      <td>0.136986</td>\n",
              "      <td>3</td>\n",
              "      <td>15</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>4</td>\n",
              "      <td>4</td>\n",
              "      <td>0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.438776</td>\n",
              "      <td>38</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8950</th>\n",
              "      <td>0.136986</td>\n",
              "      <td>3</td>\n",
              "      <td>9</td>\n",
              "      <td>4</td>\n",
              "      <td>9</td>\n",
              "      <td>1</td>\n",
              "      <td>4</td>\n",
              "      <td>0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.397959</td>\n",
              "      <td>38</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7838</th>\n",
              "      <td>0.109589</td>\n",
              "      <td>3</td>\n",
              "      <td>7</td>\n",
              "      <td>2</td>\n",
              "      <td>11</td>\n",
              "      <td>0</td>\n",
              "      <td>4</td>\n",
              "      <td>1</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.397959</td>\n",
              "      <td>38</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>16505</th>\n",
              "      <td>0.397260</td>\n",
              "      <td>3</td>\n",
              "      <td>4</td>\n",
              "      <td>2</td>\n",
              "      <td>13</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.436639</td>\n",
              "      <td>0.397959</td>\n",
              "      <td>38</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19140</th>\n",
              "      <td>0.383562</td>\n",
              "      <td>3</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>13</td>\n",
              "      <td>1</td>\n",
              "      <td>4</td>\n",
              "      <td>1</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.648301</td>\n",
              "      <td>0.765306</td>\n",
              "      <td>38</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>12319</th>\n",
              "      <td>0.164384</td>\n",
              "      <td>3</td>\n",
              "      <td>9</td>\n",
              "      <td>2</td>\n",
              "      <td>9</td>\n",
              "      <td>3</td>\n",
              "      <td>4</td>\n",
              "      <td>1</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.755102</td>\n",
              "      <td>38</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>28589</th>\n",
              "      <td>0.342466</td>\n",
              "      <td>1</td>\n",
              "      <td>11</td>\n",
              "      <td>5</td>\n",
              "      <td>7</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.602041</td>\n",
              "      <td>38</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10000</th>\n",
              "      <td>0.232877</td>\n",
              "      <td>3</td>\n",
              "      <td>15</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>4</td>\n",
              "      <td>0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>38</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>28530</th>\n",
              "      <td>0.589041</td>\n",
              "      <td>1</td>\n",
              "      <td>8</td>\n",
              "      <td>0</td>\n",
              "      <td>9</td>\n",
              "      <td>4</td>\n",
              "      <td>4</td>\n",
              "      <td>1</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.397959</td>\n",
              "      <td>38</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24237</th>\n",
              "      <td>0.027397</td>\n",
              "      <td>3</td>\n",
              "      <td>15</td>\n",
              "      <td>4</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>4</td>\n",
              "      <td>0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.397959</td>\n",
              "      <td>38</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-ba46ebbc-d1f4-4454-8b75-e74a438efe68')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-ba46ebbc-d1f4-4454-8b75-e74a438efe68 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-ba46ebbc-d1f4-4454-8b75-e74a438efe68');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Checking correlation with income\n",
        "print(\"\\n Checking correlation with income \\n\")\n",
        "abs(dataPreprocessing.corrwith(dataPreprocessing['class'])).sort_values(ascending=False)[1:]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BtGM60AJYXnt",
        "outputId": "b089bf89-93e4-48b2-e6ed-a177d7607591"
      },
      "id": "BtGM60AJYXnt",
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            " Checking correlation with income \n",
            "\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "relationship      0.250918\n",
              "age               0.234037\n",
              "hours-per-week    0.229689\n",
              "capital-gain      0.223329\n",
              "sex               0.215980\n",
              "marital-status    0.199307\n",
              "capital-loss      0.150526\n",
              "education         0.079317\n",
              "race              0.071846\n",
              "occupation        0.034625\n",
              "native-country    0.023058\n",
              "workclass         0.002693\n",
              "dtype: float64"
            ]
          },
          "metadata": {},
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# we find workclass has the least correlation of 0.002693 which can be ignored\n",
        "# similarly we can drop native-country as well which has a correlation of 0.023058\n",
        "data = data.drop(['workclass', 'native-country'], axis=1)\n",
        "data.sample(10, random_state=seed)\n",
        "\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 363
        },
        "id": "oDpmOfP1YYEd",
        "outputId": "cbed7d4d-8115-416f-b4e6-bb3249ee9694"
      },
      "id": "oDpmOfP1YYEd",
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "       age      education       marital-status         occupation  \\\n",
              "22278   27   Some-college             Divorced       Adm-clerical   \n",
              "8950    27      Bachelors        Never-married     Prof-specialty   \n",
              "7838    25     Assoc-acdm   Married-civ-spouse              Sales   \n",
              "16505   46        5th-6th   Married-civ-spouse   Transport-moving   \n",
              "19140   45           11th             Divorced   Transport-moving   \n",
              "12319   29      Bachelors   Married-civ-spouse     Prof-specialty   \n",
              "28589   42        HS-grad            Separated      Other-service   \n",
              "10000   34   Some-college             Divorced       Adm-clerical   \n",
              "28530   60      Assoc-voc             Divorced     Prof-specialty   \n",
              "24237   19   Some-college        Never-married       Adm-clerical   \n",
              "\n",
              "         relationship                 race      sex  capital-gain  \\\n",
              "22278       Unmarried                White   Female             0   \n",
              "8950    Not-in-family                White   Female             0   \n",
              "7838          Husband                White     Male             0   \n",
              "16505         Husband   Amer-Indian-Eskimo     Male             0   \n",
              "19140   Not-in-family                White     Male             0   \n",
              "12319       Own-child                White     Male             0   \n",
              "28589   Not-in-family                Black   Female             0   \n",
              "10000   Not-in-family                White   Female             0   \n",
              "28530       Unmarried                White     Male             0   \n",
              "24237       Own-child                White   Female             0   \n",
              "\n",
              "       capital-loss  hours-per-week   class  \n",
              "22278             0              44   <=50K  \n",
              "8950              0              40   <=50K  \n",
              "7838              0              40   <=50K  \n",
              "16505          1902              40   <=50K  \n",
              "19140          2824              76    >50K  \n",
              "12319             0              75   <=50K  \n",
              "28589             0              60   <=50K  \n",
              "10000             0              50   <=50K  \n",
              "28530             0              40    >50K  \n",
              "24237             0              40   <=50K  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-5be68726-5629-4037-97e0-770423b22cf2\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>age</th>\n",
              "      <th>education</th>\n",
              "      <th>marital-status</th>\n",
              "      <th>occupation</th>\n",
              "      <th>relationship</th>\n",
              "      <th>race</th>\n",
              "      <th>sex</th>\n",
              "      <th>capital-gain</th>\n",
              "      <th>capital-loss</th>\n",
              "      <th>hours-per-week</th>\n",
              "      <th>class</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>22278</th>\n",
              "      <td>27</td>\n",
              "      <td>Some-college</td>\n",
              "      <td>Divorced</td>\n",
              "      <td>Adm-clerical</td>\n",
              "      <td>Unmarried</td>\n",
              "      <td>White</td>\n",
              "      <td>Female</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>44</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8950</th>\n",
              "      <td>27</td>\n",
              "      <td>Bachelors</td>\n",
              "      <td>Never-married</td>\n",
              "      <td>Prof-specialty</td>\n",
              "      <td>Not-in-family</td>\n",
              "      <td>White</td>\n",
              "      <td>Female</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>40</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7838</th>\n",
              "      <td>25</td>\n",
              "      <td>Assoc-acdm</td>\n",
              "      <td>Married-civ-spouse</td>\n",
              "      <td>Sales</td>\n",
              "      <td>Husband</td>\n",
              "      <td>White</td>\n",
              "      <td>Male</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>40</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>16505</th>\n",
              "      <td>46</td>\n",
              "      <td>5th-6th</td>\n",
              "      <td>Married-civ-spouse</td>\n",
              "      <td>Transport-moving</td>\n",
              "      <td>Husband</td>\n",
              "      <td>Amer-Indian-Eskimo</td>\n",
              "      <td>Male</td>\n",
              "      <td>0</td>\n",
              "      <td>1902</td>\n",
              "      <td>40</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19140</th>\n",
              "      <td>45</td>\n",
              "      <td>11th</td>\n",
              "      <td>Divorced</td>\n",
              "      <td>Transport-moving</td>\n",
              "      <td>Not-in-family</td>\n",
              "      <td>White</td>\n",
              "      <td>Male</td>\n",
              "      <td>0</td>\n",
              "      <td>2824</td>\n",
              "      <td>76</td>\n",
              "      <td>&gt;50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>12319</th>\n",
              "      <td>29</td>\n",
              "      <td>Bachelors</td>\n",
              "      <td>Married-civ-spouse</td>\n",
              "      <td>Prof-specialty</td>\n",
              "      <td>Own-child</td>\n",
              "      <td>White</td>\n",
              "      <td>Male</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>75</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>28589</th>\n",
              "      <td>42</td>\n",
              "      <td>HS-grad</td>\n",
              "      <td>Separated</td>\n",
              "      <td>Other-service</td>\n",
              "      <td>Not-in-family</td>\n",
              "      <td>Black</td>\n",
              "      <td>Female</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>60</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10000</th>\n",
              "      <td>34</td>\n",
              "      <td>Some-college</td>\n",
              "      <td>Divorced</td>\n",
              "      <td>Adm-clerical</td>\n",
              "      <td>Not-in-family</td>\n",
              "      <td>White</td>\n",
              "      <td>Female</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>50</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>28530</th>\n",
              "      <td>60</td>\n",
              "      <td>Assoc-voc</td>\n",
              "      <td>Divorced</td>\n",
              "      <td>Prof-specialty</td>\n",
              "      <td>Unmarried</td>\n",
              "      <td>White</td>\n",
              "      <td>Male</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>40</td>\n",
              "      <td>&gt;50K</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24237</th>\n",
              "      <td>19</td>\n",
              "      <td>Some-college</td>\n",
              "      <td>Never-married</td>\n",
              "      <td>Adm-clerical</td>\n",
              "      <td>Own-child</td>\n",
              "      <td>White</td>\n",
              "      <td>Female</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>40</td>\n",
              "      <td>&lt;=50K</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-5be68726-5629-4037-97e0-770423b22cf2')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-5be68726-5629-4037-97e0-770423b22cf2 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-5be68726-5629-4037-97e0-770423b22cf2');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Encoding class and sex with LabelEncoder\n",
        "encoder = LabelEncoder()\n",
        "data[['class','sex']] = data[['class','sex']].apply(encoder.fit_transform)\n",
        "data.sample(10, random_state=seed)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 363
        },
        "id": "qT2mHtV_Ie-q",
        "outputId": "02ff0d1f-d707-43af-e306-3872a0085bac"
      },
      "id": "qT2mHtV_Ie-q",
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "       age      education       marital-status         occupation  \\\n",
              "22278   27   Some-college             Divorced       Adm-clerical   \n",
              "8950    27      Bachelors        Never-married     Prof-specialty   \n",
              "7838    25     Assoc-acdm   Married-civ-spouse              Sales   \n",
              "16505   46        5th-6th   Married-civ-spouse   Transport-moving   \n",
              "19140   45           11th             Divorced   Transport-moving   \n",
              "12319   29      Bachelors   Married-civ-spouse     Prof-specialty   \n",
              "28589   42        HS-grad            Separated      Other-service   \n",
              "10000   34   Some-college             Divorced       Adm-clerical   \n",
              "28530   60      Assoc-voc             Divorced     Prof-specialty   \n",
              "24237   19   Some-college        Never-married       Adm-clerical   \n",
              "\n",
              "         relationship                 race  sex  capital-gain  capital-loss  \\\n",
              "22278       Unmarried                White    0             0             0   \n",
              "8950    Not-in-family                White    0             0             0   \n",
              "7838          Husband                White    1             0             0   \n",
              "16505         Husband   Amer-Indian-Eskimo    1             0          1902   \n",
              "19140   Not-in-family                White    1             0          2824   \n",
              "12319       Own-child                White    1             0             0   \n",
              "28589   Not-in-family                Black    0             0             0   \n",
              "10000   Not-in-family                White    0             0             0   \n",
              "28530       Unmarried                White    1             0             0   \n",
              "24237       Own-child                White    0             0             0   \n",
              "\n",
              "       hours-per-week  class  \n",
              "22278              44      0  \n",
              "8950               40      0  \n",
              "7838               40      0  \n",
              "16505              40      0  \n",
              "19140              76      1  \n",
              "12319              75      0  \n",
              "28589              60      0  \n",
              "10000              50      0  \n",
              "28530              40      1  \n",
              "24237              40      0  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-830e670c-5c93-4011-958c-8960f592d5d0\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>age</th>\n",
              "      <th>education</th>\n",
              "      <th>marital-status</th>\n",
              "      <th>occupation</th>\n",
              "      <th>relationship</th>\n",
              "      <th>race</th>\n",
              "      <th>sex</th>\n",
              "      <th>capital-gain</th>\n",
              "      <th>capital-loss</th>\n",
              "      <th>hours-per-week</th>\n",
              "      <th>class</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>22278</th>\n",
              "      <td>27</td>\n",
              "      <td>Some-college</td>\n",
              "      <td>Divorced</td>\n",
              "      <td>Adm-clerical</td>\n",
              "      <td>Unmarried</td>\n",
              "      <td>White</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>44</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8950</th>\n",
              "      <td>27</td>\n",
              "      <td>Bachelors</td>\n",
              "      <td>Never-married</td>\n",
              "      <td>Prof-specialty</td>\n",
              "      <td>Not-in-family</td>\n",
              "      <td>White</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>40</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7838</th>\n",
              "      <td>25</td>\n",
              "      <td>Assoc-acdm</td>\n",
              "      <td>Married-civ-spouse</td>\n",
              "      <td>Sales</td>\n",
              "      <td>Husband</td>\n",
              "      <td>White</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>40</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>16505</th>\n",
              "      <td>46</td>\n",
              "      <td>5th-6th</td>\n",
              "      <td>Married-civ-spouse</td>\n",
              "      <td>Transport-moving</td>\n",
              "      <td>Husband</td>\n",
              "      <td>Amer-Indian-Eskimo</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>1902</td>\n",
              "      <td>40</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19140</th>\n",
              "      <td>45</td>\n",
              "      <td>11th</td>\n",
              "      <td>Divorced</td>\n",
              "      <td>Transport-moving</td>\n",
              "      <td>Not-in-family</td>\n",
              "      <td>White</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>2824</td>\n",
              "      <td>76</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>12319</th>\n",
              "      <td>29</td>\n",
              "      <td>Bachelors</td>\n",
              "      <td>Married-civ-spouse</td>\n",
              "      <td>Prof-specialty</td>\n",
              "      <td>Own-child</td>\n",
              "      <td>White</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>75</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>28589</th>\n",
              "      <td>42</td>\n",
              "      <td>HS-grad</td>\n",
              "      <td>Separated</td>\n",
              "      <td>Other-service</td>\n",
              "      <td>Not-in-family</td>\n",
              "      <td>Black</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>60</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10000</th>\n",
              "      <td>34</td>\n",
              "      <td>Some-college</td>\n",
              "      <td>Divorced</td>\n",
              "      <td>Adm-clerical</td>\n",
              "      <td>Not-in-family</td>\n",
              "      <td>White</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>50</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>28530</th>\n",
              "      <td>60</td>\n",
              "      <td>Assoc-voc</td>\n",
              "      <td>Divorced</td>\n",
              "      <td>Prof-specialty</td>\n",
              "      <td>Unmarried</td>\n",
              "      <td>White</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>40</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24237</th>\n",
              "      <td>19</td>\n",
              "      <td>Some-college</td>\n",
              "      <td>Never-married</td>\n",
              "      <td>Adm-clerical</td>\n",
              "      <td>Own-child</td>\n",
              "      <td>White</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>40</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-830e670c-5c93-4011-958c-8960f592d5d0')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-830e670c-5c93-4011-958c-8960f592d5d0 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-830e670c-5c93-4011-958c-8960f592d5d0');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Scaling numerical data using MinMaxScaler\n",
        "scaler = MinMaxScaler()\n",
        "data[['age', 'capital-gain', 'capital-loss', 'hours-per-week']] = scaler.fit_transform(data[['age', 'capital-gain', 'capital-loss', 'hours-per-week']])\n",
        "data.sample(10, random_state=seed)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 363
        },
        "id": "B_9zR52wJEua",
        "outputId": "7a7552df-1289-41b5-d221-835ae365cbaf"
      },
      "id": "B_9zR52wJEua",
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "            age      education       marital-status         occupation  \\\n",
              "22278  0.136986   Some-college             Divorced       Adm-clerical   \n",
              "8950   0.136986      Bachelors        Never-married     Prof-specialty   \n",
              "7838   0.109589     Assoc-acdm   Married-civ-spouse              Sales   \n",
              "16505  0.397260        5th-6th   Married-civ-spouse   Transport-moving   \n",
              "19140  0.383562           11th             Divorced   Transport-moving   \n",
              "12319  0.164384      Bachelors   Married-civ-spouse     Prof-specialty   \n",
              "28589  0.342466        HS-grad            Separated      Other-service   \n",
              "10000  0.232877   Some-college             Divorced       Adm-clerical   \n",
              "28530  0.589041      Assoc-voc             Divorced     Prof-specialty   \n",
              "24237  0.027397   Some-college        Never-married       Adm-clerical   \n",
              "\n",
              "         relationship                 race  sex  capital-gain  capital-loss  \\\n",
              "22278       Unmarried                White    0           0.0      0.000000   \n",
              "8950    Not-in-family                White    0           0.0      0.000000   \n",
              "7838          Husband                White    1           0.0      0.000000   \n",
              "16505         Husband   Amer-Indian-Eskimo    1           0.0      0.436639   \n",
              "19140   Not-in-family                White    1           0.0      0.648301   \n",
              "12319       Own-child                White    1           0.0      0.000000   \n",
              "28589   Not-in-family                Black    0           0.0      0.000000   \n",
              "10000   Not-in-family                White    0           0.0      0.000000   \n",
              "28530       Unmarried                White    1           0.0      0.000000   \n",
              "24237       Own-child                White    0           0.0      0.000000   \n",
              "\n",
              "       hours-per-week  class  \n",
              "22278        0.438776      0  \n",
              "8950         0.397959      0  \n",
              "7838         0.397959      0  \n",
              "16505        0.397959      0  \n",
              "19140        0.765306      1  \n",
              "12319        0.755102      0  \n",
              "28589        0.602041      0  \n",
              "10000        0.500000      0  \n",
              "28530        0.397959      1  \n",
              "24237        0.397959      0  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-01545469-6463-440d-a5fd-4ba6b6d1069a\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>age</th>\n",
              "      <th>education</th>\n",
              "      <th>marital-status</th>\n",
              "      <th>occupation</th>\n",
              "      <th>relationship</th>\n",
              "      <th>race</th>\n",
              "      <th>sex</th>\n",
              "      <th>capital-gain</th>\n",
              "      <th>capital-loss</th>\n",
              "      <th>hours-per-week</th>\n",
              "      <th>class</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>22278</th>\n",
              "      <td>0.136986</td>\n",
              "      <td>Some-college</td>\n",
              "      <td>Divorced</td>\n",
              "      <td>Adm-clerical</td>\n",
              "      <td>Unmarried</td>\n",
              "      <td>White</td>\n",
              "      <td>0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.438776</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8950</th>\n",
              "      <td>0.136986</td>\n",
              "      <td>Bachelors</td>\n",
              "      <td>Never-married</td>\n",
              "      <td>Prof-specialty</td>\n",
              "      <td>Not-in-family</td>\n",
              "      <td>White</td>\n",
              "      <td>0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.397959</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7838</th>\n",
              "      <td>0.109589</td>\n",
              "      <td>Assoc-acdm</td>\n",
              "      <td>Married-civ-spouse</td>\n",
              "      <td>Sales</td>\n",
              "      <td>Husband</td>\n",
              "      <td>White</td>\n",
              "      <td>1</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.397959</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>16505</th>\n",
              "      <td>0.397260</td>\n",
              "      <td>5th-6th</td>\n",
              "      <td>Married-civ-spouse</td>\n",
              "      <td>Transport-moving</td>\n",
              "      <td>Husband</td>\n",
              "      <td>Amer-Indian-Eskimo</td>\n",
              "      <td>1</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.436639</td>\n",
              "      <td>0.397959</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19140</th>\n",
              "      <td>0.383562</td>\n",
              "      <td>11th</td>\n",
              "      <td>Divorced</td>\n",
              "      <td>Transport-moving</td>\n",
              "      <td>Not-in-family</td>\n",
              "      <td>White</td>\n",
              "      <td>1</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.648301</td>\n",
              "      <td>0.765306</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>12319</th>\n",
              "      <td>0.164384</td>\n",
              "      <td>Bachelors</td>\n",
              "      <td>Married-civ-spouse</td>\n",
              "      <td>Prof-specialty</td>\n",
              "      <td>Own-child</td>\n",
              "      <td>White</td>\n",
              "      <td>1</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.755102</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>28589</th>\n",
              "      <td>0.342466</td>\n",
              "      <td>HS-grad</td>\n",
              "      <td>Separated</td>\n",
              "      <td>Other-service</td>\n",
              "      <td>Not-in-family</td>\n",
              "      <td>Black</td>\n",
              "      <td>0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.602041</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10000</th>\n",
              "      <td>0.232877</td>\n",
              "      <td>Some-college</td>\n",
              "      <td>Divorced</td>\n",
              "      <td>Adm-clerical</td>\n",
              "      <td>Not-in-family</td>\n",
              "      <td>White</td>\n",
              "      <td>0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.500000</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>28530</th>\n",
              "      <td>0.589041</td>\n",
              "      <td>Assoc-voc</td>\n",
              "      <td>Divorced</td>\n",
              "      <td>Prof-specialty</td>\n",
              "      <td>Unmarried</td>\n",
              "      <td>White</td>\n",
              "      <td>1</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.397959</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24237</th>\n",
              "      <td>0.027397</td>\n",
              "      <td>Some-college</td>\n",
              "      <td>Never-married</td>\n",
              "      <td>Adm-clerical</td>\n",
              "      <td>Own-child</td>\n",
              "      <td>White</td>\n",
              "      <td>0</td>\n",
              "      <td>0.0</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.397959</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-01545469-6463-440d-a5fd-4ba6b6d1069a')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-01545469-6463-440d-a5fd-4ba6b6d1069a button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-01545469-6463-440d-a5fd-4ba6b6d1069a');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 27
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Removing the duplicate values from data\n",
        "print(\"\\n Before removing duplicates:\", data.duplicated().sum())\n",
        "data = data[~data.duplicated()]\n",
        "print(\"\\n After removing duplicates:\", data.duplicated().sum())\n",
        "print(\"\\n Data shape : \", data.shape)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "h9YxnbQhPhmi",
        "outputId": "28d52d02-8ff8-43a5-a797-a5718896bb44"
      },
      "id": "h9YxnbQhPhmi",
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            " Before removing duplicates: 5133\n",
            "\n",
            " After removing duplicates: 0\n",
            "\n",
            " Data shape :  (27428, 11)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# creating numerical features from categorical features\n",
        "data = pd.get_dummies(data)\n"
      ],
      "metadata": {
        "id": "lNITNHqYJZUs"
      },
      "id": "lNITNHqYJZUs",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "encoded = list(data.columns)\n",
        "print(\"{} total features after one-hot encoding.\".format(len(encoded)))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9Ievz3rhJt-6",
        "outputId": "b8dc5cfa-dd26-49df-bef9-149f855f107b"
      },
      "id": "9Ievz3rhJt-6",
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "54 total features after one-hot encoding.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "id": "680c6d90",
      "metadata": {
        "id": "680c6d90"
      },
      "source": [
        "## Model training\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "b0b608b1",
      "metadata": {
        "id": "b0b608b1",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "845f2ac9-aedc-452d-879d-799aab15afa1"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "The best params for the model are {'bootstrap': True, 'max_depth': 9, 'max_features': 4, 'min_samples_leaf': 4, 'n_estimators': 181}\n"
          ]
        }
      ],
      "source": [
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.model_selection import GridSearchCV\n",
        "from sklearn.ensemble import RandomForestClassifier\n",
        "from sklearn.metrics import accuracy_score\n",
        "\n",
        "\n",
        "#We split the data into train(60%) and test(40%) sets\n",
        "X_train, X_test, y_train, y_test = train_test_split(data[['age', 'sex', 'capital-gain', 'capital-loss', 'hours-per-week',\n",
        "       'education_ 10th', 'education_ 11th', 'education_ 12th',\n",
        "       'education_ 1st-4th', 'education_ 5th-6th', 'education_ 7th-8th',\n",
        "       'education_ 9th', 'education_ Assoc-acdm', 'education_ Assoc-voc',\n",
        "       'education_ Bachelors', 'education_ Doctorate', 'education_ HS-grad',\n",
        "       'education_ Masters', 'education_ Preschool', 'education_ Prof-school',\n",
        "       'education_ Some-college', 'marital-status_ Divorced',\n",
        "       'marital-status_ Married-AF-spouse',\n",
        "       'marital-status_ Married-civ-spouse',\n",
        "       'marital-status_ Married-spouse-absent',\n",
        "       'marital-status_ Never-married', 'marital-status_ Separated',\n",
        "       'marital-status_ Widowed', 'occupation_ Adm-clerical',\n",
        "       'occupation_ Armed-Forces', 'occupation_ Craft-repair',\n",
        "       'occupation_ Exec-managerial', 'occupation_ Farming-fishing',\n",
        "       'occupation_ Handlers-cleaners', 'occupation_ Machine-op-inspct',\n",
        "       'occupation_ Other-service', 'occupation_ Priv-house-serv',\n",
        "       'occupation_ Prof-specialty', 'occupation_ Protective-serv',\n",
        "       'occupation_ Sales', 'occupation_ Tech-support',\n",
        "       'occupation_ Transport-moving', 'relationship_ Husband',\n",
        "       'relationship_ Not-in-family', 'relationship_ Other-relative',\n",
        "       'relationship_ Own-child', 'relationship_ Unmarried',\n",
        "       'relationship_ Wife', 'race_ Amer-Indian-Eskimo',\n",
        "       'race_ Asian-Pac-Islander', 'race_ Black', 'race_ Other',\n",
        "       'race_ White']], \n",
        "                                                    data['class'], \n",
        "                                                    test_size=0.4, \n",
        "                                                    random_state=seed)\n",
        "\n",
        "# We can now find the will find the optimal hyperparameters for a Random Forest Classifier using GridSearchCV\n",
        "param_grid = {'bootstrap': [True], \n",
        "              'max_depth': list(range(5,10)),\n",
        "              'max_features': list(range(1,5)),\n",
        "              'min_samples_leaf': [3, 4, 5],\n",
        "              'n_estimators' : list(range(10, 201, 19))\n",
        "             }\n",
        "rfc = RandomForestClassifier(random_state=seed)\n",
        "clf = GridSearchCV(rfc, param_grid)\n",
        "clf.fit(X_train, y_train)\n",
        "print(\"The best params for the model are\", clf.best_params_)\n"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# We initialise the Random Forest Classifier with the optimal hyperparameters that was returned from the GridSearchCV\n",
        "randomForestClassifier = RandomForestClassifier(bootstrap= True, max_depth= 9, max_features=4, min_samples_leaf=4, n_estimators= 181, random_state=seed)\n",
        "randomForestClassifier.fit(X_train, y_train)"
      ],
      "metadata": {
        "id": "LMRmRyVKUrZl",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "fa7aeecb-a479-4ec6-9fe7-a653ca75875a"
      },
      "id": "LMRmRyVKUrZl",
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "RandomForestClassifier(max_depth=9, max_features=4, min_samples_leaf=4,\n",
              "                       n_estimators=181, random_state=0)"
            ]
          },
          "metadata": {},
          "execution_count": 32
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# We can further find the feature importance of the features and drop unimportant features to speed up the process of training.\n",
        "feat_imp = pd.DataFrame(zip(X_train.columns.tolist(), randomForestClassifier.feature_importances_ * 100), columns=['feature', 'importance'])\n",
        "feat_imp"
      ],
      "metadata": {
        "id": "qG_FEq_rUsCU",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1711
        },
        "outputId": "59a2b43c-de89-4bf7-ff5c-193b610858e0"
      },
      "id": "qG_FEq_rUsCU",
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                  feature  importance\n",
              "0                                     age    6.180114\n",
              "1                                     sex    2.427851\n",
              "2                            capital-gain   20.691384\n",
              "3                            capital-loss    4.859182\n",
              "4                          hours-per-week    4.534605\n",
              "5                         education_ 10th    0.258531\n",
              "6                         education_ 11th    0.334049\n",
              "7                         education_ 12th    0.025507\n",
              "8                      education_ 1st-4th    0.046661\n",
              "9                      education_ 5th-6th    0.107470\n",
              "10                     education_ 7th-8th    0.316123\n",
              "11                         education_ 9th    0.143142\n",
              "12                  education_ Assoc-acdm    0.036556\n",
              "13                   education_ Assoc-voc    0.039295\n",
              "14                   education_ Bachelors    2.821674\n",
              "15                   education_ Doctorate    1.157801\n",
              "16                     education_ HS-grad    1.287929\n",
              "17                     education_ Masters    2.204806\n",
              "18                   education_ Preschool    0.009901\n",
              "19                 education_ Prof-school    1.747047\n",
              "20                education_ Some-college    0.141338\n",
              "21               marital-status_ Divorced    1.296469\n",
              "22      marital-status_ Married-AF-spouse    0.013492\n",
              "23     marital-status_ Married-civ-spouse   16.485585\n",
              "24  marital-status_ Married-spouse-absent    0.046881\n",
              "25          marital-status_ Never-married    4.747362\n",
              "26              marital-status_ Separated    0.142447\n",
              "27                marital-status_ Widowed    0.187004\n",
              "28               occupation_ Adm-clerical    0.227148\n",
              "29               occupation_ Armed-Forces    0.000000\n",
              "30               occupation_ Craft-repair    0.123483\n",
              "31            occupation_ Exec-managerial    3.042685\n",
              "32            occupation_ Farming-fishing    0.490195\n",
              "33          occupation_ Handlers-cleaners    0.254694\n",
              "34          occupation_ Machine-op-inspct    0.202000\n",
              "35              occupation_ Other-service    1.227182\n",
              "36            occupation_ Priv-house-serv    0.006806\n",
              "37             occupation_ Prof-specialty    1.035219\n",
              "38            occupation_ Protective-serv    0.031390\n",
              "39                      occupation_ Sales    0.151915\n",
              "40               occupation_ Tech-support    0.107877\n",
              "41           occupation_ Transport-moving    0.166167\n",
              "42                  relationship_ Husband   11.053866\n",
              "43            relationship_ Not-in-family    2.570095\n",
              "44           relationship_ Other-relative    0.369219\n",
              "45                relationship_ Own-child    2.598962\n",
              "46                relationship_ Unmarried    1.547725\n",
              "47                     relationship_ Wife    1.933992\n",
              "48               race_ Amer-Indian-Eskimo    0.035651\n",
              "49               race_ Asian-Pac-Islander    0.067192\n",
              "50                            race_ Black    0.192228\n",
              "51                            race_ Other    0.031962\n",
              "52                            race_ White    0.242138"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-adcdc2fa-c1a0-468d-8b14-2e96c0d0e2a0\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>feature</th>\n",
              "      <th>importance</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>age</td>\n",
              "      <td>6.180114</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>sex</td>\n",
              "      <td>2.427851</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>capital-gain</td>\n",
              "      <td>20.691384</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>capital-loss</td>\n",
              "      <td>4.859182</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>hours-per-week</td>\n",
              "      <td>4.534605</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>education_ 10th</td>\n",
              "      <td>0.258531</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>education_ 11th</td>\n",
              "      <td>0.334049</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>education_ 12th</td>\n",
              "      <td>0.025507</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>education_ 1st-4th</td>\n",
              "      <td>0.046661</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>education_ 5th-6th</td>\n",
              "      <td>0.107470</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10</th>\n",
              "      <td>education_ 7th-8th</td>\n",
              "      <td>0.316123</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11</th>\n",
              "      <td>education_ 9th</td>\n",
              "      <td>0.143142</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>12</th>\n",
              "      <td>education_ Assoc-acdm</td>\n",
              "      <td>0.036556</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>13</th>\n",
              "      <td>education_ Assoc-voc</td>\n",
              "      <td>0.039295</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14</th>\n",
              "      <td>education_ Bachelors</td>\n",
              "      <td>2.821674</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>15</th>\n",
              "      <td>education_ Doctorate</td>\n",
              "      <td>1.157801</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>16</th>\n",
              "      <td>education_ HS-grad</td>\n",
              "      <td>1.287929</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>17</th>\n",
              "      <td>education_ Masters</td>\n",
              "      <td>2.204806</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>18</th>\n",
              "      <td>education_ Preschool</td>\n",
              "      <td>0.009901</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>19</th>\n",
              "      <td>education_ Prof-school</td>\n",
              "      <td>1.747047</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>20</th>\n",
              "      <td>education_ Some-college</td>\n",
              "      <td>0.141338</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>21</th>\n",
              "      <td>marital-status_ Divorced</td>\n",
              "      <td>1.296469</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>22</th>\n",
              "      <td>marital-status_ Married-AF-spouse</td>\n",
              "      <td>0.013492</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>23</th>\n",
              "      <td>marital-status_ Married-civ-spouse</td>\n",
              "      <td>16.485585</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>24</th>\n",
              "      <td>marital-status_ Married-spouse-absent</td>\n",
              "      <td>0.046881</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25</th>\n",
              "      <td>marital-status_ Never-married</td>\n",
              "      <td>4.747362</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>26</th>\n",
              "      <td>marital-status_ Separated</td>\n",
              "      <td>0.142447</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>27</th>\n",
              "      <td>marital-status_ Widowed</td>\n",
              "      <td>0.187004</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>28</th>\n",
              "      <td>occupation_ Adm-clerical</td>\n",
              "      <td>0.227148</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>29</th>\n",
              "      <td>occupation_ Armed-Forces</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>30</th>\n",
              "      <td>occupation_ Craft-repair</td>\n",
              "      <td>0.123483</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>31</th>\n",
              "      <td>occupation_ Exec-managerial</td>\n",
              "      <td>3.042685</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>32</th>\n",
              "      <td>occupation_ Farming-fishing</td>\n",
              "      <td>0.490195</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>33</th>\n",
              "      <td>occupation_ Handlers-cleaners</td>\n",
              "      <td>0.254694</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>34</th>\n",
              "      <td>occupation_ Machine-op-inspct</td>\n",
              "      <td>0.202000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>35</th>\n",
              "      <td>occupation_ Other-service</td>\n",
              "      <td>1.227182</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>36</th>\n",
              "      <td>occupation_ Priv-house-serv</td>\n",
              "      <td>0.006806</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>37</th>\n",
              "      <td>occupation_ Prof-specialty</td>\n",
              "      <td>1.035219</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>38</th>\n",
              "      <td>occupation_ Protective-serv</td>\n",
              "      <td>0.031390</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>39</th>\n",
              "      <td>occupation_ Sales</td>\n",
              "      <td>0.151915</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>40</th>\n",
              "      <td>occupation_ Tech-support</td>\n",
              "      <td>0.107877</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>41</th>\n",
              "      <td>occupation_ Transport-moving</td>\n",
              "      <td>0.166167</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>42</th>\n",
              "      <td>relationship_ Husband</td>\n",
              "      <td>11.053866</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>43</th>\n",
              "      <td>relationship_ Not-in-family</td>\n",
              "      <td>2.570095</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>44</th>\n",
              "      <td>relationship_ Other-relative</td>\n",
              "      <td>0.369219</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>45</th>\n",
              "      <td>relationship_ Own-child</td>\n",
              "      <td>2.598962</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>46</th>\n",
              "      <td>relationship_ Unmarried</td>\n",
              "      <td>1.547725</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>47</th>\n",
              "      <td>relationship_ Wife</td>\n",
              "      <td>1.933992</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>48</th>\n",
              "      <td>race_ Amer-Indian-Eskimo</td>\n",
              "      <td>0.035651</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>49</th>\n",
              "      <td>race_ Asian-Pac-Islander</td>\n",
              "      <td>0.067192</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50</th>\n",
              "      <td>race_ Black</td>\n",
              "      <td>0.192228</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>51</th>\n",
              "      <td>race_ Other</td>\n",
              "      <td>0.031962</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>52</th>\n",
              "      <td>race_ White</td>\n",
              "      <td>0.242138</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-adcdc2fa-c1a0-468d-8b14-2e96c0d0e2a0')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-adcdc2fa-c1a0-468d-8b14-2e96c0d0e2a0 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-adcdc2fa-c1a0-468d-8b14-2e96c0d0e2a0');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 33
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# since race had the least importance of we can drop the columns as this will make the training process faster\n",
        "X_train = X_train.drop(['race_ White', 'race_ Black', 'race_ Other', 'race_ Asian-Pac-Islander', 'race_ Amer-Indian-Eskimo'], axis=1)\n",
        "X_test = X_test.drop(['race_ White', 'race_ Black', 'race_ Other', 'race_ Asian-Pac-Islander', 'race_ Amer-Indian-Eskimo'], axis=1)\n",
        "print(len(X_train.columns))"
      ],
      "metadata": {
        "id": "aMs-Rg7OWREC",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "489b8341-e44a-458c-d5f8-8e2de8b6c521"
      },
      "id": "aMs-Rg7OWREC",
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "48\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# we now perform k(5)-fold cross validation on the training data and print the accuracy of each fold\n",
        "randomForestClassifier = RandomForestClassifier(bootstrap= True, max_depth= 9, max_features=4, min_samples_leaf=3, random_state=seed)\n",
        "k = 5\n",
        "X_train['class'] = y_train\n",
        "folds = np.array_split(X_train.sample(frac=1, random_state=seed), indices_or_sections=k)\n",
        "for i in range(len(folds)):\n",
        "  print(\"Number of samples in fold \" + str(i) + \" is \" + str(len(folds[i])))\n",
        "  print(\"Number of samples in each class in fold \"+ str(i) + \" is \\n\" + str(folds[i]['class'].value_counts()))\n",
        "  print(\"*\"*20)\n",
        "accuracies = []\n",
        "for i in range(len(folds)):\n",
        "  x_test = folds[i].iloc[:, 0:48]\n",
        "  y_test_fold = folds[i].iloc[:, 48]\n",
        "  trainData = []\n",
        "  for j in range(len(folds)):\n",
        "    if j != i:\n",
        "      trainData.append(folds[j])\n",
        "  traiDataFrame = pd.concat(trainData)\n",
        "  randomForestClassifier.fit(traiDataFrame.iloc[:, 0:48], traiDataFrame.iloc[:,48])\n",
        "  y_pred = randomForestClassifier.predict(x_test)\n",
        "  accuracies.append(round(accuracy_score(y_test_fold, y_pred),2))\n",
        "print(\"The acuracies across the folds are\", accuracies)"
      ],
      "metadata": {
        "id": "dclBwV1QUcQv",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "1a21507e-85d0-48b3-fe85-094dd5ca60c2"
      },
      "id": "dclBwV1QUcQv",
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Number of samples in fold 0 is 3292\n",
            "Number of samples in each class in fold 0 is \n",
            "0    2490\n",
            "1     802\n",
            "Name: class, dtype: int64\n",
            "********************\n",
            "Number of samples in fold 1 is 3291\n",
            "Number of samples in each class in fold 1 is \n",
            "0    2491\n",
            "1     800\n",
            "Name: class, dtype: int64\n",
            "********************\n",
            "Number of samples in fold 2 is 3291\n",
            "Number of samples in each class in fold 2 is \n",
            "0    2493\n",
            "1     798\n",
            "Name: class, dtype: int64\n",
            "********************\n",
            "Number of samples in fold 3 is 3291\n",
            "Number of samples in each class in fold 3 is \n",
            "0    2482\n",
            "1     809\n",
            "Name: class, dtype: int64\n",
            "********************\n",
            "Number of samples in fold 4 is 3291\n",
            "Number of samples in each class in fold 4 is \n",
            "0    2505\n",
            "1     786\n",
            "Name: class, dtype: int64\n",
            "********************\n",
            "The acuracies across the folds are [0.85, 0.85, 0.86, 0.85, 0.85]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "id": "b5f713eb",
      "metadata": {
        "id": "b5f713eb"
      },
      "source": [
        "## Evaluation\n",
        "\n",
        "We will evaluate the model on the following metrics using the test set: \n",
        "- Overall accuracy\n",
        "- Precision\n",
        "- Recall\n",
        "- F1 score"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "2986a2bc",
      "metadata": {
        "id": "2986a2bc",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a7af0fb8-6b1c-4e07-89ac-ef404e3015c3"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Accuracy using the test data is  0.85\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       0.85      0.97      0.91      8261\n",
            "           1       0.86      0.47      0.60      2711\n",
            "\n",
            "    accuracy                           0.85     10972\n",
            "   macro avg       0.85      0.72      0.76     10972\n",
            "weighted avg       0.85      0.85      0.83     10972\n",
            "\n"
          ]
        }
      ],
      "source": [
        "from sklearn.metrics import classification_report\n",
        "\n",
        "# we can now use the model to predict on the test set and print the accuracy, precision, recall, f1-score and support\n",
        "y_pred = randomForestClassifier.predict(X_test)\n",
        "print(\"Accuracy using the test data is \",round(accuracy_score(y_test, y_pred), 2))\n",
        "print(classification_report(y_test, y_pred))"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.13"
    },
    "colab": {
      "provenance": [],
      "collapsed_sections": []
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}