Lab4 - Using several classifiers and tuning parameters - ML.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# Using several classifiers and tuning parameters - Parameters grid"
      ],
      "metadata": {
        "id": "JY9hkClNeRSd"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Model selection** is the process of selecting one final machine learning model from among a collection of candidate machine learning models for a training dataset. \n",
        "**Model Selection** is a critical step in your machine learning model building. Choosing the right model can greatly impact the performance of your machine learning model, and choosing the wrong model, can leave you with unacceptable results.\n",
        "\n",
        "We're going to use the *model selection* features of scikit-learn, and comparison of several classification methods.\n",
        "\n"
      ],
      "metadata": {
        "id": "PYKRe52NeXIA"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "aLhhxwi1cy_5",
        "outputId": "d626fe47-afab-46d3-9721-c2dff5954b9a"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "http://scikit-learn.org/stable/auto_examples/model_selection/plot_grid_search_digits.html\n",
            "@author: scikit-learn.org and Claudio Sartori\n",
            "\n"
          ]
        }
      ],
      "source": [
        "\"\"\"\n",
        "http://scikit-learn.org/stable/auto_examples/model_selection/plot_grid_search_digits.html\n",
        "@author: scikit-learn.org and Claudio Sartori\n",
        "\"\"\"\n",
        "import warnings\n",
        "warnings.filterwarnings('ignore') # uncomment this line to suppress warnings\n",
        "\n",
        "# A few imports from the sklearn library\n",
        "from sklearn import datasets\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.model_selection import GridSearchCV\n",
        "from sklearn.metrics import classification_report\n",
        "from sklearn.svm import SVC\n",
        "from sklearn.linear_model import Perceptron\n",
        "from sklearn.neural_network import MLPClassifier\n",
        "from sklearn.tree import DecisionTreeClassifier\n",
        "from sklearn.naive_bayes import GaussianNB\n",
        "from sklearn.neighbors import KNeighborsClassifier\n",
        "from sklearn.ensemble import AdaBoostClassifier, RandomForestClassifier\n",
        "\n",
        "print(__doc__) # print information included in the triple quotes at the beginning\n",
        "\n",
        "\n",
        "# Loading a standard dataset\n",
        "# For this lab we are going to use the iris dataset\n",
        "\n",
        "#dataset = datasets.load_digits()\n",
        "#dataset = datasets.fetch_olivetti_faces()\n",
        "#dataset = datasets.fetch_covtype()\n",
        "dataset = datasets.load_iris()\n",
        "#dataset = datasets.load_wine()\n",
        "#dataset = datasets.load_breast_cancer()"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now, prepare the environment, specifying attributes, targets."
      ],
      "metadata": {
        "id": "o1-_VQ8vWkbD"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "X = dataset.data\n",
        "y = dataset.target\n",
        "ts = 0.3 # Fraction of the test data (between 0.2 and 0.5); ts = test size\n",
        "random_state = 42"
      ],
      "metadata": {
        "id": "TIQGvaNvWuHv"
      },
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Split the dataset into train and test, and print the shapes."
      ],
      "metadata": {
        "id": "fESQiHyaXcv4"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = ts, random_state = random_state)\n",
        "\n",
        "print(X_train.shape)\n",
        "print(X_test.shape)\n",
        "print(y_train.shape)\n",
        "print(y_test.shape)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LcaqQCZNXfng",
        "outputId": "5e4a92a2-2666-47d9-e25e-e6596cdeda4c"
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(105, 4)\n",
            "(45, 4)\n",
            "(105,)\n",
            "(45,)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "In order to ease the remainder of the session."
      ],
      "metadata": {
        "id": "RNkwHp4jX4xx"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "model_lbls = [\n",
        "              'dt', \n",
        "              'nb', \n",
        "              'lp', \n",
        "              'svc', \n",
        "             'knn',\n",
        "             'adb',\n",
        "             'rf',\n",
        "            ]\n",
        "\n",
        "# Set the parameters by cross-validation\n",
        "tuned_param_dt = [{'max_depth': [*range(1,20)]}]\n",
        "tuned_param_nb = [{'var_smoothing': [10, 1, 1e-1, 1e-2, 1e-3, 1e-4, 1e-5, 1e-6, 1e-07, 1e-8, 1e-9, 1e-10]}]\n",
        "tuned_param_lp = [{'early_stopping': [True]}]\n",
        "tuned_param_svc = [{'kernel': ['rbf'], \n",
        "                    'gamma': [1e-3, 1e-4],\n",
        "                    'C': [1, 10, 100, 1000],\n",
        "                    },\n",
        "                    {'kernel': ['linear'],\n",
        "                     'C': [1, 10, 100, 1000],                     \n",
        "                    },\n",
        "                   ]\n",
        "tuned_param_knn =[{'n_neighbors': [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]}]\n",
        "tuned_param_adb = [{'n_estimators':[20,30,40,50],\n",
        "                   'learning_rate':[0.5,0.75,1,1.25,1.5]}]\n",
        "tuned_param_rf = [{'max_depth': [*range(5,15)],\n",
        "                   'n_estimators':[*range(10,100,10)]}]\n",
        "\n",
        "models = {\n",
        "    'dt': {'name': 'Decision Tree       ',\n",
        "           'estimator': DecisionTreeClassifier(), \n",
        "           'param': tuned_param_dt,\n",
        "          },\n",
        "    'nb': {'name': 'Gaussian Naive Bayes',\n",
        "           'estimator': GaussianNB(),\n",
        "           'param': tuned_param_nb\n",
        "          },\n",
        "    'lp': {'name': 'Linear Perceptron   ',\n",
        "           'estimator': Perceptron(),\n",
        "           'param': tuned_param_lp,\n",
        "          },\n",
        "    'svc':{'name': 'Support Vector      ',\n",
        "           'estimator': SVC(), \n",
        "           'param': tuned_param_svc\n",
        "          },\n",
        "    'knn':{'name': 'K Nearest Neighbor ',\n",
        "           'estimator': KNeighborsClassifier(),\n",
        "           'param': tuned_param_knn\n",
        "       },\n",
        "       'adb':{'name': 'AdaBoost           ',\n",
        "           'estimator': AdaBoostClassifier(),\n",
        "           'param': tuned_param_adb\n",
        "          },\n",
        "    'rf': {'name': 'Random forest       ',\n",
        "           'estimator': RandomForestClassifier(),\n",
        "           'param': tuned_param_rf\n",
        "          }\n",
        "\n",
        "}\n",
        "\n",
        "scores = ['precision', 'recall']"
      ],
      "metadata": {
        "id": "AR53t13IYl_D"
      },
      "execution_count": 11,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Group the outputs with a simple function, having as parameter the fitted model and using the components of the fitted model to inspect the results of the search with the parameters grid.\n",
        "\n",
        "The components are:<br>\n",
        "`model.best_params_`<br>\n",
        "`model.cv_results_['mean_test_score']`<br>`\n",
        "model.cv_results_['std_test_score']`<br>\n",
        "`model.cv_results_['params']`\n",
        "\n",
        "The classification_report() is generated by the function imported above from sklearn.metrics, which takes as argument the true and the predicted test labels."
      ],
      "metadata": {
        "id": "Na3p47hMY4zy"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def print_results(model):\n",
        "    print(\"Best parameters set found on train set:\")\n",
        "    print()\n",
        "    # if best is linear there is no gamma parameter\n",
        "    print(model.best_params_)\n",
        "    print()\n",
        "    print(\"Grid scores on train set:\")\n",
        "    print()\n",
        "    means = model.cv_results_['mean_test_score']\n",
        "    stds = model.cv_results_['std_test_score']\n",
        "    params = model.cv_results_['params']\n",
        "    for mean, std, params_tuple in zip(means, stds, params):\n",
        "        print(\"%0.3f (+/-%0.03f) for %r\"\n",
        "              % (mean, std * 2, params_tuple))\n",
        "    print()\n",
        "    print(\"Detailed classification report for the best parameter set:\")\n",
        "    print()\n",
        "    print(\"The model is trained on the full train set.\")\n",
        "    print(\"The scores are computed on the full test set.\")\n",
        "    print()\n",
        "    y_true, y_pred = y_test, model.predict(X_test)\n",
        "    print(classification_report(y_true, y_pred))\n",
        "    print()"
      ],
      "metadata": {
        "id": "LY5UY5BuZSMg"
      },
      "execution_count": 12,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Loop on scores and model labels"
      ],
      "metadata": {
        "id": "GTnkGJ13aoU0"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Grid search** is a tuning technique that attempts to compute the optimum values of hyperparameters.\n",
        "\n",
        " It is an exhaustive search that is performed on a the specific parameter values of a model. \n",
        "\n",
        "The model is also known as an estimator. Grid search exercise can save us time, effort and resources."
      ],
      "metadata": {
        "id": "tcbYgI8sdDKA"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "results_short = {}\n",
        "\n",
        "for score in scores:\n",
        "    print('='*40)\n",
        "    print(\"# Tuning hyper-parameters for %s\" % score)\n",
        "    print()\n",
        "    \n",
        "  #'%s_macro' % score ## is a string formatting expression\n",
        "  # the parameter after % is substituted in the string placeholder %s\n",
        "    for model in model_lbls:\n",
        "          print('-'*40)\n",
        "          print(\"Trying model {}\".format(models[model]['name']))\n",
        "          \n",
        "          # Activate the grid search\n",
        "          clf = GridSearchCV(models[model]['estimator'], models[model]['param'], cv=5,\n",
        "                            scoring='%s_macro' % score, \n",
        "                            return_train_score = False,\n",
        "                            n_jobs = 2, # this allows using multi-cores\n",
        "                            )\n",
        "          clf.fit(X_train, y_train)\n",
        "          print_results(clf)\n",
        "          results_short[model] = clf.best_score_\n",
        "    print(\"Summary of results for {}\".format(score))\n",
        "    print(\"Estimator\")\n",
        "    for m in results_short.keys():\n",
        "        print(\"{}\\t - score: {:4.2}%\".format(models[m]['name'], results_short[m]))\n",
        "          \n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WmCkOI_Iarsd",
        "outputId": "a40358dd-1d7f-4e4c-c906-6e96d3fdc594"
      },
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "========================================\n",
            "# Tuning hyper-parameters for precision\n",
            "\n",
            "----------------------------------------\n",
            "Trying model Decision Tree       \n",
            "Best parameters set found on train set:\n",
            "\n",
            "{'max_depth': 4}\n",
            "\n",
            "Grid scores on train set:\n",
            "\n",
            "0.491 (+/-0.009) for {'max_depth': 1}\n",
            "0.924 (+/-0.070) for {'max_depth': 2}\n",
            "0.941 (+/-0.072) for {'max_depth': 3}\n",
            "0.943 (+/-0.042) for {'max_depth': 4}\n",
            "0.933 (+/-0.048) for {'max_depth': 5}\n",
            "0.943 (+/-0.042) for {'max_depth': 6}\n",
            "0.943 (+/-0.042) for {'max_depth': 7}\n",
            "0.943 (+/-0.042) for {'max_depth': 8}\n",
            "0.943 (+/-0.042) for {'max_depth': 9}\n",
            "0.943 (+/-0.042) for {'max_depth': 10}\n",
            "0.943 (+/-0.042) for {'max_depth': 11}\n",
            "0.943 (+/-0.042) for {'max_depth': 12}\n",
            "0.943 (+/-0.042) for {'max_depth': 13}\n",
            "0.943 (+/-0.042) for {'max_depth': 14}\n",
            "0.943 (+/-0.042) for {'max_depth': 15}\n",
            "0.943 (+/-0.042) for {'max_depth': 16}\n",
            "0.943 (+/-0.042) for {'max_depth': 17}\n",
            "0.943 (+/-0.042) for {'max_depth': 18}\n",
            "0.943 (+/-0.042) for {'max_depth': 19}\n",
            "\n",
            "Detailed classification report for the best parameter set:\n",
            "\n",
            "The model is trained on the full train set.\n",
            "The scores are computed on the full test set.\n",
            "\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       1.00      1.00      1.00        19\n",
            "           1       1.00      1.00      1.00        13\n",
            "           2       1.00      1.00      1.00        13\n",
            "\n",
            "    accuracy                           1.00        45\n",
            "   macro avg       1.00      1.00      1.00        45\n",
            "weighted avg       1.00      1.00      1.00        45\n",
            "\n",
            "\n",
            "----------------------------------------\n",
            "Trying model Gaussian Naive Bayes\n",
            "Best parameters set found on train set:\n",
            "\n",
            "{'var_smoothing': 0.001}\n",
            "\n",
            "Grid scores on train set:\n",
            "\n",
            "0.762 (+/-0.352) for {'var_smoothing': 10}\n",
            "0.920 (+/-0.131) for {'var_smoothing': 1}\n",
            "0.911 (+/-0.150) for {'var_smoothing': 0.1}\n",
            "0.933 (+/-0.090) for {'var_smoothing': 0.01}\n",
            "0.941 (+/-0.072) for {'var_smoothing': 0.001}\n",
            "0.941 (+/-0.072) for {'var_smoothing': 0.0001}\n",
            "0.941 (+/-0.072) for {'var_smoothing': 1e-05}\n",
            "0.941 (+/-0.072) for {'var_smoothing': 1e-06}\n",
            "0.941 (+/-0.072) for {'var_smoothing': 1e-07}\n",
            "0.941 (+/-0.072) for {'var_smoothing': 1e-08}\n",
            "0.941 (+/-0.072) for {'var_smoothing': 1e-09}\n",
            "0.941 (+/-0.072) for {'var_smoothing': 1e-10}\n",
            "\n",
            "Detailed classification report for the best parameter set:\n",
            "\n",
            "The model is trained on the full train set.\n",
            "The scores are computed on the full test set.\n",
            "\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       1.00      1.00      1.00        19\n",
            "           1       1.00      0.92      0.96        13\n",
            "           2       0.93      1.00      0.96        13\n",
            "\n",
            "    accuracy                           0.98        45\n",
            "   macro avg       0.98      0.97      0.97        45\n",
            "weighted avg       0.98      0.98      0.98        45\n",
            "\n",
            "\n",
            "----------------------------------------\n",
            "Trying model Linear Perceptron   \n",
            "Best parameters set found on train set:\n",
            "\n",
            "{'early_stopping': True}\n",
            "\n",
            "Grid scores on train set:\n",
            "\n",
            "0.667 (+/-0.654) for {'early_stopping': True}\n",
            "\n",
            "Detailed classification report for the best parameter set:\n",
            "\n",
            "The model is trained on the full train set.\n",
            "The scores are computed on the full test set.\n",
            "\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       0.66      1.00      0.79        19\n",
            "           1       1.00      0.23      0.38        13\n",
            "           2       1.00      1.00      1.00        13\n",
            "\n",
            "    accuracy                           0.78        45\n",
            "   macro avg       0.89      0.74      0.72        45\n",
            "weighted avg       0.85      0.78      0.73        45\n",
            "\n",
            "\n",
            "----------------------------------------\n",
            "Trying model Support Vector      \n",
            "Best parameters set found on train set:\n",
            "\n",
            "{'C': 1000, 'gamma': 0.001, 'kernel': 'rbf'}\n",
            "\n",
            "Grid scores on train set:\n",
            "\n",
            "0.322 (+/-0.349) for {'C': 1, 'gamma': 0.001, 'kernel': 'rbf'}\n",
            "0.171 (+/-0.240) for {'C': 1, 'gamma': 0.0001, 'kernel': 'rbf'}\n",
            "0.940 (+/-0.108) for {'C': 10, 'gamma': 0.001, 'kernel': 'rbf'}\n",
            "0.322 (+/-0.349) for {'C': 10, 'gamma': 0.0001, 'kernel': 'rbf'}\n",
            "0.964 (+/-0.063) for {'C': 100, 'gamma': 0.001, 'kernel': 'rbf'}\n",
            "0.940 (+/-0.108) for {'C': 100, 'gamma': 0.0001, 'kernel': 'rbf'}\n",
            "0.978 (+/-0.055) for {'C': 1000, 'gamma': 0.001, 'kernel': 'rbf'}\n",
            "0.964 (+/-0.063) for {'C': 1000, 'gamma': 0.0001, 'kernel': 'rbf'}\n",
            "0.970 (+/-0.052) for {'C': 1, 'kernel': 'linear'}\n",
            "0.964 (+/-0.063) for {'C': 10, 'kernel': 'linear'}\n",
            "0.964 (+/-0.063) for {'C': 100, 'kernel': 'linear'}\n",
            "0.964 (+/-0.063) for {'C': 1000, 'kernel': 'linear'}\n",
            "\n",
            "Detailed classification report for the best parameter set:\n",
            "\n",
            "The model is trained on the full train set.\n",
            "The scores are computed on the full test set.\n",
            "\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       1.00      1.00      1.00        19\n",
            "           1       1.00      1.00      1.00        13\n",
            "           2       1.00      1.00      1.00        13\n",
            "\n",
            "    accuracy                           1.00        45\n",
            "   macro avg       1.00      1.00      1.00        45\n",
            "weighted avg       1.00      1.00      1.00        45\n",
            "\n",
            "\n",
            "----------------------------------------\n",
            "Trying model K Nearest Neighbor \n",
            "Best parameters set found on train set:\n",
            "\n",
            "{'n_neighbors': 1}\n",
            "\n",
            "Grid scores on train set:\n",
            "\n",
            "0.963 (+/-0.043) for {'n_neighbors': 1}\n",
            "0.947 (+/-0.061) for {'n_neighbors': 2}\n",
            "0.950 (+/-0.054) for {'n_neighbors': 3}\n",
            "0.950 (+/-0.054) for {'n_neighbors': 4}\n",
            "0.956 (+/-0.052) for {'n_neighbors': 5}\n",
            "0.953 (+/-0.060) for {'n_neighbors': 6}\n",
            "0.963 (+/-0.043) for {'n_neighbors': 7}\n",
            "0.963 (+/-0.043) for {'n_neighbors': 8}\n",
            "0.957 (+/-0.049) for {'n_neighbors': 9}\n",
            "0.947 (+/-0.061) for {'n_neighbors': 10}\n",
            "\n",
            "Detailed classification report for the best parameter set:\n",
            "\n",
            "The model is trained on the full train set.\n",
            "The scores are computed on the full test set.\n",
            "\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       1.00      1.00      1.00        19\n",
            "           1       1.00      1.00      1.00        13\n",
            "           2       1.00      1.00      1.00        13\n",
            "\n",
            "    accuracy                           1.00        45\n",
            "   macro avg       1.00      1.00      1.00        45\n",
            "weighted avg       1.00      1.00      1.00        45\n",
            "\n",
            "\n",
            "----------------------------------------\n",
            "Trying model AdaBoost           \n",
            "Best parameters set found on train set:\n",
            "\n",
            "{'learning_rate': 1, 'n_estimators': 20}\n",
            "\n",
            "Grid scores on train set:\n",
            "\n",
            "0.928 (+/-0.067) for {'learning_rate': 0.5, 'n_estimators': 20}\n",
            "0.923 (+/-0.059) for {'learning_rate': 0.5, 'n_estimators': 30}\n",
            "0.923 (+/-0.059) for {'learning_rate': 0.5, 'n_estimators': 40}\n",
            "0.923 (+/-0.059) for {'learning_rate': 0.5, 'n_estimators': 50}\n",
            "0.937 (+/-0.041) for {'learning_rate': 0.75, 'n_estimators': 20}\n",
            "0.937 (+/-0.041) for {'learning_rate': 0.75, 'n_estimators': 30}\n",
            "0.928 (+/-0.067) for {'learning_rate': 0.75, 'n_estimators': 40}\n",
            "0.937 (+/-0.041) for {'learning_rate': 0.75, 'n_estimators': 50}\n",
            "0.942 (+/-0.087) for {'learning_rate': 1, 'n_estimators': 20}\n",
            "0.937 (+/-0.041) for {'learning_rate': 1, 'n_estimators': 30}\n",
            "0.942 (+/-0.087) for {'learning_rate': 1, 'n_estimators': 40}\n",
            "0.937 (+/-0.041) for {'learning_rate': 1, 'n_estimators': 50}\n",
            "0.942 (+/-0.087) for {'learning_rate': 1.25, 'n_estimators': 20}\n",
            "0.935 (+/-0.071) for {'learning_rate': 1.25, 'n_estimators': 30}\n",
            "0.942 (+/-0.087) for {'learning_rate': 1.25, 'n_estimators': 40}\n",
            "0.935 (+/-0.071) for {'learning_rate': 1.25, 'n_estimators': 50}\n",
            "0.937 (+/-0.087) for {'learning_rate': 1.5, 'n_estimators': 20}\n",
            "0.937 (+/-0.086) for {'learning_rate': 1.5, 'n_estimators': 30}\n",
            "0.931 (+/-0.083) for {'learning_rate': 1.5, 'n_estimators': 40}\n",
            "0.931 (+/-0.083) for {'learning_rate': 1.5, 'n_estimators': 50}\n",
            "\n",
            "Detailed classification report for the best parameter set:\n",
            "\n",
            "The model is trained on the full train set.\n",
            "The scores are computed on the full test set.\n",
            "\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       1.00      1.00      1.00        19\n",
            "           1       1.00      1.00      1.00        13\n",
            "           2       1.00      1.00      1.00        13\n",
            "\n",
            "    accuracy                           1.00        45\n",
            "   macro avg       1.00      1.00      1.00        45\n",
            "weighted avg       1.00      1.00      1.00        45\n",
            "\n",
            "\n",
            "----------------------------------------\n",
            "Trying model Random forest       \n",
            "Best parameters set found on train set:\n",
            "\n",
            "{'max_depth': 6, 'n_estimators': 10}\n",
            "\n",
            "Grid scores on train set:\n",
            "\n",
            "0.937 (+/-0.087) for {'max_depth': 5, 'n_estimators': 10}\n",
            "0.952 (+/-0.062) for {'max_depth': 5, 'n_estimators': 20}\n",
            "0.952 (+/-0.062) for {'max_depth': 5, 'n_estimators': 30}\n",
            "0.952 (+/-0.062) for {'max_depth': 5, 'n_estimators': 40}\n",
            "0.959 (+/-0.074) for {'max_depth': 5, 'n_estimators': 50}\n",
            "0.952 (+/-0.062) for {'max_depth': 5, 'n_estimators': 60}\n",
            "0.959 (+/-0.074) for {'max_depth': 5, 'n_estimators': 70}\n",
            "0.952 (+/-0.062) for {'max_depth': 5, 'n_estimators': 80}\n",
            "0.952 (+/-0.062) for {'max_depth': 5, 'n_estimators': 90}\n",
            "0.962 (+/-0.047) for {'max_depth': 6, 'n_estimators': 10}\n",
            "0.943 (+/-0.088) for {'max_depth': 6, 'n_estimators': 20}\n",
            "0.954 (+/-0.028) for {'max_depth': 6, 'n_estimators': 30}\n",
            "0.962 (+/-0.047) for {'max_depth': 6, 'n_estimators': 40}\n",
            "0.943 (+/-0.088) for {'max_depth': 6, 'n_estimators': 50}\n",
            "0.952 (+/-0.062) for {'max_depth': 6, 'n_estimators': 60}\n",
            "0.952 (+/-0.062) for {'max_depth': 6, 'n_estimators': 70}\n",
            "0.952 (+/-0.062) for {'max_depth': 6, 'n_estimators': 80}\n",
            "0.952 (+/-0.062) for {'max_depth': 6, 'n_estimators': 90}\n",
            "0.952 (+/-0.062) for {'max_depth': 7, 'n_estimators': 10}\n",
            "0.954 (+/-0.028) for {'max_depth': 7, 'n_estimators': 20}\n",
            "0.943 (+/-0.042) for {'max_depth': 7, 'n_estimators': 30}\n",
            "0.952 (+/-0.062) for {'max_depth': 7, 'n_estimators': 40}\n",
            "0.943 (+/-0.042) for {'max_depth': 7, 'n_estimators': 50}\n",
            "0.962 (+/-0.047) for {'max_depth': 7, 'n_estimators': 60}\n",
            "0.952 (+/-0.062) for {'max_depth': 7, 'n_estimators': 70}\n",
            "0.952 (+/-0.062) for {'max_depth': 7, 'n_estimators': 80}\n",
            "0.952 (+/-0.062) for {'max_depth': 7, 'n_estimators': 90}\n",
            "0.943 (+/-0.088) for {'max_depth': 8, 'n_estimators': 10}\n",
            "0.952 (+/-0.062) for {'max_depth': 8, 'n_estimators': 20}\n",
            "0.943 (+/-0.088) for {'max_depth': 8, 'n_estimators': 30}\n",
            "0.944 (+/-0.087) for {'max_depth': 8, 'n_estimators': 40}\n",
            "0.952 (+/-0.062) for {'max_depth': 8, 'n_estimators': 50}\n",
            "0.952 (+/-0.062) for {'max_depth': 8, 'n_estimators': 60}\n",
            "0.943 (+/-0.042) for {'max_depth': 8, 'n_estimators': 70}\n",
            "0.952 (+/-0.062) for {'max_depth': 8, 'n_estimators': 80}\n",
            "0.943 (+/-0.088) for {'max_depth': 8, 'n_estimators': 90}\n",
            "0.937 (+/-0.041) for {'max_depth': 9, 'n_estimators': 10}\n",
            "0.952 (+/-0.062) for {'max_depth': 9, 'n_estimators': 20}\n",
            "0.952 (+/-0.062) for {'max_depth': 9, 'n_estimators': 30}\n",
            "0.943 (+/-0.042) for {'max_depth': 9, 'n_estimators': 40}\n",
            "0.944 (+/-0.087) for {'max_depth': 9, 'n_estimators': 50}\n",
            "0.952 (+/-0.062) for {'max_depth': 9, 'n_estimators': 60}\n",
            "0.952 (+/-0.062) for {'max_depth': 9, 'n_estimators': 70}\n",
            "0.943 (+/-0.042) for {'max_depth': 9, 'n_estimators': 80}\n",
            "0.952 (+/-0.062) for {'max_depth': 9, 'n_estimators': 90}\n",
            "0.941 (+/-0.032) for {'max_depth': 10, 'n_estimators': 10}\n",
            "0.944 (+/-0.087) for {'max_depth': 10, 'n_estimators': 20}\n",
            "0.943 (+/-0.042) for {'max_depth': 10, 'n_estimators': 30}\n",
            "0.952 (+/-0.062) for {'max_depth': 10, 'n_estimators': 40}\n",
            "0.934 (+/-0.122) for {'max_depth': 10, 'n_estimators': 50}\n",
            "0.952 (+/-0.062) for {'max_depth': 10, 'n_estimators': 60}\n",
            "0.952 (+/-0.062) for {'max_depth': 10, 'n_estimators': 70}\n",
            "0.943 (+/-0.042) for {'max_depth': 10, 'n_estimators': 80}\n",
            "0.943 (+/-0.042) for {'max_depth': 10, 'n_estimators': 90}\n",
            "0.952 (+/-0.062) for {'max_depth': 11, 'n_estimators': 10}\n",
            "0.952 (+/-0.062) for {'max_depth': 11, 'n_estimators': 20}\n",
            "0.952 (+/-0.062) for {'max_depth': 11, 'n_estimators': 30}\n",
            "0.952 (+/-0.062) for {'max_depth': 11, 'n_estimators': 40}\n",
            "0.952 (+/-0.062) for {'max_depth': 11, 'n_estimators': 50}\n",
            "0.943 (+/-0.042) for {'max_depth': 11, 'n_estimators': 60}\n",
            "0.952 (+/-0.062) for {'max_depth': 11, 'n_estimators': 70}\n",
            "0.952 (+/-0.062) for {'max_depth': 11, 'n_estimators': 80}\n",
            "0.952 (+/-0.062) for {'max_depth': 11, 'n_estimators': 90}\n",
            "0.951 (+/-0.099) for {'max_depth': 12, 'n_estimators': 10}\n",
            "0.935 (+/-0.071) for {'max_depth': 12, 'n_estimators': 20}\n",
            "0.952 (+/-0.062) for {'max_depth': 12, 'n_estimators': 30}\n",
            "0.952 (+/-0.062) for {'max_depth': 12, 'n_estimators': 40}\n",
            "0.943 (+/-0.042) for {'max_depth': 12, 'n_estimators': 50}\n",
            "0.952 (+/-0.062) for {'max_depth': 12, 'n_estimators': 60}\n",
            "0.952 (+/-0.062) for {'max_depth': 12, 'n_estimators': 70}\n",
            "0.952 (+/-0.062) for {'max_depth': 12, 'n_estimators': 80}\n",
            "0.952 (+/-0.062) for {'max_depth': 12, 'n_estimators': 90}\n",
            "0.952 (+/-0.062) for {'max_depth': 13, 'n_estimators': 10}\n",
            "0.952 (+/-0.062) for {'max_depth': 13, 'n_estimators': 20}\n",
            "0.943 (+/-0.042) for {'max_depth': 13, 'n_estimators': 30}\n",
            "0.952 (+/-0.062) for {'max_depth': 13, 'n_estimators': 40}\n",
            "0.943 (+/-0.088) for {'max_depth': 13, 'n_estimators': 50}\n",
            "0.943 (+/-0.042) for {'max_depth': 13, 'n_estimators': 60}\n",
            "0.952 (+/-0.062) for {'max_depth': 13, 'n_estimators': 70}\n",
            "0.943 (+/-0.042) for {'max_depth': 13, 'n_estimators': 80}\n",
            "0.952 (+/-0.062) for {'max_depth': 13, 'n_estimators': 90}\n",
            "0.952 (+/-0.062) for {'max_depth': 14, 'n_estimators': 10}\n",
            "0.956 (+/-0.052) for {'max_depth': 14, 'n_estimators': 20}\n",
            "0.952 (+/-0.062) for {'max_depth': 14, 'n_estimators': 30}\n",
            "0.952 (+/-0.062) for {'max_depth': 14, 'n_estimators': 40}\n",
            "0.944 (+/-0.087) for {'max_depth': 14, 'n_estimators': 50}\n",
            "0.952 (+/-0.062) for {'max_depth': 14, 'n_estimators': 60}\n",
            "0.943 (+/-0.042) for {'max_depth': 14, 'n_estimators': 70}\n",
            "0.952 (+/-0.062) for {'max_depth': 14, 'n_estimators': 80}\n",
            "0.952 (+/-0.062) for {'max_depth': 14, 'n_estimators': 90}\n",
            "\n",
            "Detailed classification report for the best parameter set:\n",
            "\n",
            "The model is trained on the full train set.\n",
            "The scores are computed on the full test set.\n",
            "\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       1.00      1.00      1.00        19\n",
            "           1       1.00      1.00      1.00        13\n",
            "           2       1.00      1.00      1.00        13\n",
            "\n",
            "    accuracy                           1.00        45\n",
            "   macro avg       1.00      1.00      1.00        45\n",
            "weighted avg       1.00      1.00      1.00        45\n",
            "\n",
            "\n",
            "Summary of results for precision\n",
            "Estimator\n",
            "Decision Tree       \t - score: 0.94%\n",
            "Gaussian Naive Bayes\t - score: 0.94%\n",
            "Linear Perceptron   \t - score: 0.67%\n",
            "Support Vector      \t - score: 0.98%\n",
            "K Nearest Neighbor \t - score: 0.96%\n",
            "AdaBoost           \t - score: 0.94%\n",
            "Random forest       \t - score: 0.96%\n",
            "========================================\n",
            "# Tuning hyper-parameters for recall\n",
            "\n",
            "----------------------------------------\n",
            "Trying model Decision Tree       \n",
            "Best parameters set found on train set:\n",
            "\n",
            "{'max_depth': 7}\n",
            "\n",
            "Grid scores on train set:\n",
            "\n",
            "0.667 (+/-0.000) for {'max_depth': 1}\n",
            "0.920 (+/-0.065) for {'max_depth': 2}\n",
            "0.937 (+/-0.071) for {'max_depth': 3}\n",
            "0.938 (+/-0.040) for {'max_depth': 4}\n",
            "0.929 (+/-0.045) for {'max_depth': 5}\n",
            "0.938 (+/-0.040) for {'max_depth': 6}\n",
            "0.948 (+/-0.065) for {'max_depth': 7}\n",
            "0.948 (+/-0.065) for {'max_depth': 8}\n",
            "0.938 (+/-0.040) for {'max_depth': 9}\n",
            "0.938 (+/-0.040) for {'max_depth': 10}\n",
            "0.938 (+/-0.040) for {'max_depth': 11}\n",
            "0.938 (+/-0.040) for {'max_depth': 12}\n",
            "0.938 (+/-0.040) for {'max_depth': 13}\n",
            "0.938 (+/-0.040) for {'max_depth': 14}\n",
            "0.938 (+/-0.040) for {'max_depth': 15}\n",
            "0.938 (+/-0.040) for {'max_depth': 16}\n",
            "0.938 (+/-0.040) for {'max_depth': 17}\n",
            "0.938 (+/-0.040) for {'max_depth': 18}\n",
            "0.938 (+/-0.040) for {'max_depth': 19}\n",
            "\n",
            "Detailed classification report for the best parameter set:\n",
            "\n",
            "The model is trained on the full train set.\n",
            "The scores are computed on the full test set.\n",
            "\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       1.00      1.00      1.00        19\n",
            "           1       1.00      1.00      1.00        13\n",
            "           2       1.00      1.00      1.00        13\n",
            "\n",
            "    accuracy                           1.00        45\n",
            "   macro avg       1.00      1.00      1.00        45\n",
            "weighted avg       1.00      1.00      1.00        45\n",
            "\n",
            "\n",
            "----------------------------------------\n",
            "Trying model Gaussian Naive Bayes\n",
            "Best parameters set found on train set:\n",
            "\n",
            "{'var_smoothing': 0.001}\n",
            "\n",
            "Grid scores on train set:\n",
            "\n",
            "0.716 (+/-0.224) for {'var_smoothing': 10}\n",
            "0.918 (+/-0.134) for {'var_smoothing': 1}\n",
            "0.908 (+/-0.152) for {'var_smoothing': 0.1}\n",
            "0.927 (+/-0.092) for {'var_smoothing': 0.01}\n",
            "0.937 (+/-0.071) for {'var_smoothing': 0.001}\n",
            "0.937 (+/-0.071) for {'var_smoothing': 0.0001}\n",
            "0.937 (+/-0.071) for {'var_smoothing': 1e-05}\n",
            "0.937 (+/-0.071) for {'var_smoothing': 1e-06}\n",
            "0.937 (+/-0.071) for {'var_smoothing': 1e-07}\n",
            "0.937 (+/-0.071) for {'var_smoothing': 1e-08}\n",
            "0.937 (+/-0.071) for {'var_smoothing': 1e-09}\n",
            "0.937 (+/-0.071) for {'var_smoothing': 1e-10}\n",
            "\n",
            "Detailed classification report for the best parameter set:\n",
            "\n",
            "The model is trained on the full train set.\n",
            "The scores are computed on the full test set.\n",
            "\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       1.00      1.00      1.00        19\n",
            "           1       1.00      0.92      0.96        13\n",
            "           2       0.93      1.00      0.96        13\n",
            "\n",
            "    accuracy                           0.98        45\n",
            "   macro avg       0.98      0.97      0.97        45\n",
            "weighted avg       0.98      0.98      0.98        45\n",
            "\n",
            "\n",
            "----------------------------------------\n",
            "Trying model Linear Perceptron   \n",
            "Best parameters set found on train set:\n",
            "\n",
            "{'early_stopping': True}\n",
            "\n",
            "Grid scores on train set:\n",
            "\n",
            "0.719 (+/-0.443) for {'early_stopping': True}\n",
            "\n",
            "Detailed classification report for the best parameter set:\n",
            "\n",
            "The model is trained on the full train set.\n",
            "The scores are computed on the full test set.\n",
            "\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       0.66      1.00      0.79        19\n",
            "           1       1.00      0.23      0.38        13\n",
            "           2       1.00      1.00      1.00        13\n",
            "\n",
            "    accuracy                           0.78        45\n",
            "   macro avg       0.89      0.74      0.72        45\n",
            "weighted avg       0.85      0.78      0.73        45\n",
            "\n",
            "\n",
            "----------------------------------------\n",
            "Trying model Support Vector      \n",
            "Best parameters set found on train set:\n",
            "\n",
            "{'C': 1000, 'gamma': 0.001, 'kernel': 'rbf'}\n",
            "\n",
            "Grid scores on train set:\n",
            "\n",
            "0.505 (+/-0.299) for {'C': 1, 'gamma': 0.001, 'kernel': 'rbf'}\n",
            "0.371 (+/-0.152) for {'C': 1, 'gamma': 0.0001, 'kernel': 'rbf'}\n",
            "0.937 (+/-0.111) for {'C': 10, 'gamma': 0.001, 'kernel': 'rbf'}\n",
            "0.505 (+/-0.299) for {'C': 10, 'gamma': 0.0001, 'kernel': 'rbf'}\n",
            "0.955 (+/-0.080) for {'C': 100, 'gamma': 0.001, 'kernel': 'rbf'}\n",
            "0.937 (+/-0.111) for {'C': 100, 'gamma': 0.0001, 'kernel': 'rbf'}\n",
            "0.973 (+/-0.075) for {'C': 1000, 'gamma': 0.001, 'kernel': 'rbf'}\n",
            "0.955 (+/-0.080) for {'C': 1000, 'gamma': 0.0001, 'kernel': 'rbf'}\n",
            "0.963 (+/-0.071) for {'C': 1, 'kernel': 'linear'}\n",
            "0.955 (+/-0.080) for {'C': 10, 'kernel': 'linear'}\n",
            "0.955 (+/-0.080) for {'C': 100, 'kernel': 'linear'}\n",
            "0.955 (+/-0.080) for {'C': 1000, 'kernel': 'linear'}\n",
            "\n",
            "Detailed classification report for the best parameter set:\n",
            "\n",
            "The model is trained on the full train set.\n",
            "The scores are computed on the full test set.\n",
            "\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       1.00      1.00      1.00        19\n",
            "           1       1.00      1.00      1.00        13\n",
            "           2       1.00      1.00      1.00        13\n",
            "\n",
            "    accuracy                           1.00        45\n",
            "   macro avg       1.00      1.00      1.00        45\n",
            "weighted avg       1.00      1.00      1.00        45\n",
            "\n",
            "\n",
            "----------------------------------------\n",
            "Trying model K Nearest Neighbor \n",
            "Best parameters set found on train set:\n",
            "\n",
            "{'n_neighbors': 1}\n",
            "\n",
            "Grid scores on train set:\n",
            "\n",
            "0.954 (+/-0.060) for {'n_neighbors': 1}\n",
            "0.935 (+/-0.075) for {'n_neighbors': 2}\n",
            "0.936 (+/-0.073) for {'n_neighbors': 3}\n",
            "0.935 (+/-0.078) for {'n_neighbors': 4}\n",
            "0.945 (+/-0.067) for {'n_neighbors': 5}\n",
            "0.944 (+/-0.069) for {'n_neighbors': 6}\n",
            "0.954 (+/-0.060) for {'n_neighbors': 7}\n",
            "0.954 (+/-0.060) for {'n_neighbors': 8}\n",
            "0.944 (+/-0.072) for {'n_neighbors': 9}\n",
            "0.935 (+/-0.075) for {'n_neighbors': 10}\n",
            "\n",
            "Detailed classification report for the best parameter set:\n",
            "\n",
            "The model is trained on the full train set.\n",
            "The scores are computed on the full test set.\n",
            "\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       1.00      1.00      1.00        19\n",
            "           1       1.00      1.00      1.00        13\n",
            "           2       1.00      1.00      1.00        13\n",
            "\n",
            "    accuracy                           1.00        45\n",
            "   macro avg       1.00      1.00      1.00        45\n",
            "weighted avg       1.00      1.00      1.00        45\n",
            "\n",
            "\n",
            "----------------------------------------\n",
            "Trying model AdaBoost           \n",
            "Best parameters set found on train set:\n",
            "\n",
            "{'learning_rate': 1, 'n_estimators': 20}\n",
            "\n",
            "Grid scores on train set:\n",
            "\n",
            "0.920 (+/-0.065) for {'learning_rate': 0.5, 'n_estimators': 20}\n",
            "0.911 (+/-0.057) for {'learning_rate': 0.5, 'n_estimators': 30}\n",
            "0.911 (+/-0.057) for {'learning_rate': 0.5, 'n_estimators': 40}\n",
            "0.911 (+/-0.057) for {'learning_rate': 0.5, 'n_estimators': 50}\n",
            "0.929 (+/-0.045) for {'learning_rate': 0.75, 'n_estimators': 20}\n",
            "0.929 (+/-0.045) for {'learning_rate': 0.75, 'n_estimators': 30}\n",
            "0.920 (+/-0.065) for {'learning_rate': 0.75, 'n_estimators': 40}\n",
            "0.929 (+/-0.045) for {'learning_rate': 0.75, 'n_estimators': 50}\n",
            "0.939 (+/-0.088) for {'learning_rate': 1, 'n_estimators': 20}\n",
            "0.929 (+/-0.045) for {'learning_rate': 1, 'n_estimators': 30}\n",
            "0.939 (+/-0.088) for {'learning_rate': 1, 'n_estimators': 40}\n",
            "0.929 (+/-0.045) for {'learning_rate': 1, 'n_estimators': 50}\n",
            "0.939 (+/-0.088) for {'learning_rate': 1.25, 'n_estimators': 20}\n",
            "0.930 (+/-0.068) for {'learning_rate': 1.25, 'n_estimators': 30}\n",
            "0.939 (+/-0.088) for {'learning_rate': 1.25, 'n_estimators': 40}\n",
            "0.930 (+/-0.068) for {'learning_rate': 1.25, 'n_estimators': 50}\n",
            "0.929 (+/-0.089) for {'learning_rate': 1.5, 'n_estimators': 20}\n",
            "0.929 (+/-0.089) for {'learning_rate': 1.5, 'n_estimators': 30}\n",
            "0.919 (+/-0.087) for {'learning_rate': 1.5, 'n_estimators': 40}\n",
            "0.919 (+/-0.087) for {'learning_rate': 1.5, 'n_estimators': 50}\n",
            "\n",
            "Detailed classification report for the best parameter set:\n",
            "\n",
            "The model is trained on the full train set.\n",
            "The scores are computed on the full test set.\n",
            "\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       1.00      1.00      1.00        19\n",
            "           1       1.00      1.00      1.00        13\n",
            "           2       1.00      1.00      1.00        13\n",
            "\n",
            "    accuracy                           1.00        45\n",
            "   macro avg       1.00      1.00      1.00        45\n",
            "weighted avg       1.00      1.00      1.00        45\n",
            "\n",
            "\n",
            "----------------------------------------\n",
            "Trying model Random forest       \n",
            "Best parameters set found on train set:\n",
            "\n",
            "{'max_depth': 10, 'n_estimators': 20}\n",
            "\n",
            "Grid scores on train set:\n",
            "\n",
            "0.929 (+/-0.089) for {'max_depth': 5, 'n_estimators': 10}\n",
            "0.938 (+/-0.040) for {'max_depth': 5, 'n_estimators': 20}\n",
            "0.956 (+/-0.077) for {'max_depth': 5, 'n_estimators': 30}\n",
            "0.946 (+/-0.064) for {'max_depth': 5, 'n_estimators': 40}\n",
            "0.938 (+/-0.040) for {'max_depth': 5, 'n_estimators': 50}\n",
            "0.938 (+/-0.087) for {'max_depth': 5, 'n_estimators': 60}\n",
            "0.946 (+/-0.064) for {'max_depth': 5, 'n_estimators': 70}\n",
            "0.946 (+/-0.064) for {'max_depth': 5, 'n_estimators': 80}\n",
            "0.946 (+/-0.064) for {'max_depth': 5, 'n_estimators': 90}\n",
            "0.927 (+/-0.120) for {'max_depth': 6, 'n_estimators': 10}\n",
            "0.946 (+/-0.064) for {'max_depth': 6, 'n_estimators': 20}\n",
            "0.938 (+/-0.040) for {'max_depth': 6, 'n_estimators': 30}\n",
            "0.946 (+/-0.064) for {'max_depth': 6, 'n_estimators': 40}\n",
            "0.937 (+/-0.091) for {'max_depth': 6, 'n_estimators': 50}\n",
            "0.946 (+/-0.064) for {'max_depth': 6, 'n_estimators': 60}\n",
            "0.938 (+/-0.087) for {'max_depth': 6, 'n_estimators': 70}\n",
            "0.946 (+/-0.064) for {'max_depth': 6, 'n_estimators': 80}\n",
            "0.946 (+/-0.064) for {'max_depth': 6, 'n_estimators': 90}\n",
            "0.938 (+/-0.087) for {'max_depth': 7, 'n_estimators': 10}\n",
            "0.938 (+/-0.040) for {'max_depth': 7, 'n_estimators': 20}\n",
            "0.946 (+/-0.064) for {'max_depth': 7, 'n_estimators': 30}\n",
            "0.955 (+/-0.053) for {'max_depth': 7, 'n_estimators': 40}\n",
            "0.946 (+/-0.064) for {'max_depth': 7, 'n_estimators': 50}\n",
            "0.946 (+/-0.064) for {'max_depth': 7, 'n_estimators': 60}\n",
            "0.946 (+/-0.064) for {'max_depth': 7, 'n_estimators': 70}\n",
            "0.946 (+/-0.064) for {'max_depth': 7, 'n_estimators': 80}\n",
            "0.946 (+/-0.064) for {'max_depth': 7, 'n_estimators': 90}\n",
            "0.938 (+/-0.087) for {'max_depth': 8, 'n_estimators': 10}\n",
            "0.937 (+/-0.071) for {'max_depth': 8, 'n_estimators': 20}\n",
            "0.929 (+/-0.120) for {'max_depth': 8, 'n_estimators': 30}\n",
            "0.946 (+/-0.064) for {'max_depth': 8, 'n_estimators': 40}\n",
            "0.946 (+/-0.064) for {'max_depth': 8, 'n_estimators': 50}\n",
            "0.946 (+/-0.064) for {'max_depth': 8, 'n_estimators': 60}\n",
            "0.946 (+/-0.064) for {'max_depth': 8, 'n_estimators': 70}\n",
            "0.946 (+/-0.064) for {'max_depth': 8, 'n_estimators': 80}\n",
            "0.946 (+/-0.064) for {'max_depth': 8, 'n_estimators': 90}\n",
            "0.938 (+/-0.040) for {'max_depth': 9, 'n_estimators': 10}\n",
            "0.946 (+/-0.064) for {'max_depth': 9, 'n_estimators': 20}\n",
            "0.938 (+/-0.087) for {'max_depth': 9, 'n_estimators': 30}\n",
            "0.946 (+/-0.064) for {'max_depth': 9, 'n_estimators': 40}\n",
            "0.956 (+/-0.077) for {'max_depth': 9, 'n_estimators': 50}\n",
            "0.946 (+/-0.064) for {'max_depth': 9, 'n_estimators': 60}\n",
            "0.946 (+/-0.064) for {'max_depth': 9, 'n_estimators': 70}\n",
            "0.946 (+/-0.064) for {'max_depth': 9, 'n_estimators': 80}\n",
            "0.946 (+/-0.064) for {'max_depth': 9, 'n_estimators': 90}\n",
            "0.946 (+/-0.064) for {'max_depth': 10, 'n_estimators': 10}\n",
            "0.956 (+/-0.053) for {'max_depth': 10, 'n_estimators': 20}\n",
            "0.938 (+/-0.040) for {'max_depth': 10, 'n_estimators': 30}\n",
            "0.946 (+/-0.064) for {'max_depth': 10, 'n_estimators': 40}\n",
            "0.956 (+/-0.077) for {'max_depth': 10, 'n_estimators': 50}\n",
            "0.937 (+/-0.091) for {'max_depth': 10, 'n_estimators': 60}\n",
            "0.946 (+/-0.064) for {'max_depth': 10, 'n_estimators': 70}\n",
            "0.946 (+/-0.064) for {'max_depth': 10, 'n_estimators': 80}\n",
            "0.946 (+/-0.064) for {'max_depth': 10, 'n_estimators': 90}\n",
            "0.930 (+/-0.068) for {'max_depth': 11, 'n_estimators': 10}\n",
            "0.955 (+/-0.053) for {'max_depth': 11, 'n_estimators': 20}\n",
            "0.946 (+/-0.064) for {'max_depth': 11, 'n_estimators': 30}\n",
            "0.946 (+/-0.064) for {'max_depth': 11, 'n_estimators': 40}\n",
            "0.946 (+/-0.064) for {'max_depth': 11, 'n_estimators': 50}\n",
            "0.946 (+/-0.064) for {'max_depth': 11, 'n_estimators': 60}\n",
            "0.946 (+/-0.064) for {'max_depth': 11, 'n_estimators': 70}\n",
            "0.946 (+/-0.064) for {'max_depth': 11, 'n_estimators': 80}\n",
            "0.946 (+/-0.064) for {'max_depth': 11, 'n_estimators': 90}\n",
            "0.938 (+/-0.087) for {'max_depth': 12, 'n_estimators': 10}\n",
            "0.938 (+/-0.087) for {'max_depth': 12, 'n_estimators': 20}\n",
            "0.938 (+/-0.040) for {'max_depth': 12, 'n_estimators': 30}\n",
            "0.946 (+/-0.064) for {'max_depth': 12, 'n_estimators': 40}\n",
            "0.946 (+/-0.064) for {'max_depth': 12, 'n_estimators': 50}\n",
            "0.946 (+/-0.064) for {'max_depth': 12, 'n_estimators': 60}\n",
            "0.938 (+/-0.087) for {'max_depth': 12, 'n_estimators': 70}\n",
            "0.946 (+/-0.064) for {'max_depth': 12, 'n_estimators': 80}\n",
            "0.946 (+/-0.064) for {'max_depth': 12, 'n_estimators': 90}\n",
            "0.946 (+/-0.064) for {'max_depth': 13, 'n_estimators': 10}\n",
            "0.930 (+/-0.068) for {'max_depth': 13, 'n_estimators': 20}\n",
            "0.946 (+/-0.064) for {'max_depth': 13, 'n_estimators': 30}\n",
            "0.946 (+/-0.064) for {'max_depth': 13, 'n_estimators': 40}\n",
            "0.938 (+/-0.087) for {'max_depth': 13, 'n_estimators': 50}\n",
            "0.946 (+/-0.064) for {'max_depth': 13, 'n_estimators': 60}\n",
            "0.946 (+/-0.064) for {'max_depth': 13, 'n_estimators': 70}\n",
            "0.946 (+/-0.064) for {'max_depth': 13, 'n_estimators': 80}\n",
            "0.946 (+/-0.064) for {'max_depth': 13, 'n_estimators': 90}\n",
            "0.938 (+/-0.040) for {'max_depth': 14, 'n_estimators': 10}\n",
            "0.946 (+/-0.064) for {'max_depth': 14, 'n_estimators': 20}\n",
            "0.946 (+/-0.064) for {'max_depth': 14, 'n_estimators': 30}\n",
            "0.938 (+/-0.087) for {'max_depth': 14, 'n_estimators': 40}\n",
            "0.938 (+/-0.040) for {'max_depth': 14, 'n_estimators': 50}\n",
            "0.946 (+/-0.064) for {'max_depth': 14, 'n_estimators': 60}\n",
            "0.938 (+/-0.040) for {'max_depth': 14, 'n_estimators': 70}\n",
            "0.938 (+/-0.040) for {'max_depth': 14, 'n_estimators': 80}\n",
            "0.946 (+/-0.064) for {'max_depth': 14, 'n_estimators': 90}\n",
            "\n",
            "Detailed classification report for the best parameter set:\n",
            "\n",
            "The model is trained on the full train set.\n",
            "The scores are computed on the full test set.\n",
            "\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       1.00      1.00      1.00        19\n",
            "           1       1.00      1.00      1.00        13\n",
            "           2       1.00      1.00      1.00        13\n",
            "\n",
            "    accuracy                           1.00        45\n",
            "   macro avg       1.00      1.00      1.00        45\n",
            "weighted avg       1.00      1.00      1.00        45\n",
            "\n",
            "\n",
            "Summary of results for recall\n",
            "Estimator\n",
            "Decision Tree       \t - score: 0.95%\n",
            "Gaussian Naive Bayes\t - score: 0.94%\n",
            "Linear Perceptron   \t - score: 0.72%\n",
            "Support Vector      \t - score: 0.97%\n",
            "K Nearest Neighbor \t - score: 0.95%\n",
            "AdaBoost           \t - score: 0.94%\n",
            "Random forest       \t - score: 0.96%\n"
          ]
        }
      ]
    }
  ]
}