Cap11_Classificacao_Ocorrencia_Cancer.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Cap11-Classificacao-Ocorrencia-Cancer.ipynb",
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "code",
      "metadata": {
        "id": "659EGVz5DGGX",
        "colab_type": "code",
        "outputId": "9dff5a7d-320d-443c-aa00-c5cc0c667602",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        }
      },
      "source": [
        "# activate R magic\n",
        "%load_ext rpy2.ipython"
      ],
      "execution_count": 159,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "The rpy2.ipython extension is already loaded. To reload it, use:\n",
            "  %reload_ext rpy2.ipython\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xykAIe2KQzyN",
        "colab_type": "text"
      },
      "source": [
        "#**Prevendo a Ocorrência de Câncer**\n",
        "\n",
        "Problema de Negócio: Previsão de Ocorrência de Câncer de Mama\n",
        "\n",
        "[link text](http://archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+%28Diagnostic%29)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "swiFZffzRnwk",
        "colab_type": "text"
      },
      "source": [
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "p3i8RZRaRO6C",
        "colab_type": "text"
      },
      "source": [
        "##**Etapa 1** - Coletando Dados\n",
        "\n",
        "Os dados do câncer da mama incluem 569 observações de biópsias de câncer, cada um com 32 características (variáveis). \n",
        "\n",
        "#### Variável Target: **O diagnóstico é codificado como \"M\" para indicar maligno ou \"B\" para indicar benigno.** "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SJos4F58RSoQ",
        "colab_type": "code",
        "outputId": "0acd8868-df03-462d-d31d-1401a6bde4f6",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 604
        }
      },
      "source": [
        "%%R\n",
        "  dados <- read.csv(\"dataset.csv\", stringsAsFactors = FALSE) # Indica a não considerar nenhuma variável numérica como sendo do tipo fator\n",
        "  str(dados)"
      ],
      "execution_count": 160,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "'data.frame':\t569 obs. of  32 variables:\n",
            " $ id               : int  87139402 8910251 905520 868871 9012568 906539 925291 87880 862989 89827 ...\n",
            " $ diagnosis        : chr  \"B\" \"B\" \"B\" \"B\" ...\n",
            " $ radius_mean      : num  12.3 10.6 11 11.3 15.2 ...\n",
            " $ texture_mean     : num  12.4 18.9 16.8 13.4 13.2 ...\n",
            " $ perimeter_mean   : num  78.8 69.3 70.9 73 97.7 ...\n",
            " $ area_mean        : num  464 346 373 385 712 ...\n",
            " $ smoothness_mean  : num  0.1028 0.0969 0.1077 0.1164 0.0796 ...\n",
            " $ compactness_mean : num  0.0698 0.1147 0.078 0.1136 0.0693 ...\n",
            " $ concavity_mean   : num  0.0399 0.0639 0.0305 0.0464 0.0339 ...\n",
            " $ points_mean      : num  0.037 0.0264 0.0248 0.048 0.0266 ...\n",
            " $ symmetry_mean    : num  0.196 0.192 0.171 0.177 0.172 ...\n",
            " $ dimension_mean   : num  0.0595 0.0649 0.0634 0.0607 0.0554 ...\n",
            " $ radius_se        : num  0.236 0.451 0.197 0.338 0.178 ...\n",
            " $ texture_se       : num  0.666 1.197 1.387 1.343 0.412 ...\n",
            " $ perimeter_se     : num  1.67 3.43 1.34 1.85 1.34 ...\n",
            " $ area_se          : num  17.4 27.1 13.5 26.3 17.7 ...\n",
            " $ smoothness_se    : num  0.00805 0.00747 0.00516 0.01127 0.00501 ...\n",
            " $ compactness_se   : num  0.0118 0.03581 0.00936 0.03498 0.01485 ...\n",
            " $ concavity_se     : num  0.0168 0.0335 0.0106 0.0219 0.0155 ...\n",
            " $ points_se        : num  0.01241 0.01365 0.00748 0.01965 0.00915 ...\n",
            " $ symmetry_se      : num  0.0192 0.035 0.0172 0.0158 0.0165 ...\n",
            " $ dimension_se     : num  0.00225 0.00332 0.0022 0.00344 0.00177 ...\n",
            " $ radius_worst     : num  13.5 11.9 12.4 11.9 16.2 ...\n",
            " $ texture_worst    : num  15.6 22.9 26.4 15.8 15.7 ...\n",
            " $ perimeter_worst  : num  87 78.3 79.9 76.5 104.5 ...\n",
            " $ area_worst       : num  549 425 471 434 819 ...\n",
            " $ smoothness_worst : num  0.139 0.121 0.137 0.137 0.113 ...\n",
            " $ compactness_worst: num  0.127 0.252 0.148 0.182 0.174 ...\n",
            " $ concavity_worst  : num  0.1242 0.1916 0.1067 0.0867 0.1362 ...\n",
            " $ points_worst     : num  0.0939 0.0793 0.0743 0.0861 0.0818 ...\n",
            " $ symmetry_worst   : num  0.283 0.294 0.3 0.21 0.249 ...\n",
            " $ dimension_worst  : num  0.0677 0.0759 0.0788 0.0678 0.0677 ...\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6qYkZBR1f_Ab",
        "colab_type": "text"
      },
      "source": [
        "## ATENÇÃO! Temos escalas numéricas diferentes para algumas variáveis.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_ed7jTLFS2m1",
        "colab_type": "text"
      },
      "source": [
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OBkfbNyVSeFC",
        "colab_type": "text"
      },
      "source": [
        "##**Etapa 2** - Pré-Processamento\n",
        "\n",
        "\n",
        "Excluindo a coluna ID Independentemente do método de aprendizagem de máquina, deve sempre ser excluídas variáveis de ID. É uma variável que apresenta a identificação de cada registro.\n",
        "\n",
        "Caso contrário, isso pode levar a resultados errados porque o ID \n",
        "pode ser usado para unicamente \"prever\" cada exemplo.  \n",
        "\n",
        "Im modelo  que inclui um identificador pode sofrer de superajuste (overfitting), e será muito difícil usá-lo para generalizar outros dados."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TD7r7PGSSlZi",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%%R\n",
        "  dados$id = NULL\n",
        "  # Removendo a variável ID do conjunto de dados de análise"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YA9oSKbNTCda",
        "colab_type": "text"
      },
      "source": [
        "### Ajustando o label da variável alvo"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rj-FM5X1TBYq",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 586
        },
        "outputId": "62b4b37d-1671-43f2-ce5f-ffcac375263a"
      },
      "source": [
        "%%R\n",
        "  dados$diagnosis = sapply(dados$diagnosis,                                          # utilizar a função sapply(), selecionar a coluna \"diagnosis\" em dados, atualizando a mesma coluna\n",
        "                           function(x){ ifelse (x =='M', 'Maligno', 'Benigno')})     # criar uma função ifelse se o valor for igual a M, é Maligno. Caso contrário, benigno.\n",
        "  str(dados)"
      ],
      "execution_count": 163,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "'data.frame':\t569 obs. of  31 variables:\n",
            " $ diagnosis        : chr  \"Benigno\" \"Benigno\" \"Benigno\" \"Benigno\" ...\n",
            " $ radius_mean      : num  12.3 10.6 11 11.3 15.2 ...\n",
            " $ texture_mean     : num  12.4 18.9 16.8 13.4 13.2 ...\n",
            " $ perimeter_mean   : num  78.8 69.3 70.9 73 97.7 ...\n",
            " $ area_mean        : num  464 346 373 385 712 ...\n",
            " $ smoothness_mean  : num  0.1028 0.0969 0.1077 0.1164 0.0796 ...\n",
            " $ compactness_mean : num  0.0698 0.1147 0.078 0.1136 0.0693 ...\n",
            " $ concavity_mean   : num  0.0399 0.0639 0.0305 0.0464 0.0339 ...\n",
            " $ points_mean      : num  0.037 0.0264 0.0248 0.048 0.0266 ...\n",
            " $ symmetry_mean    : num  0.196 0.192 0.171 0.177 0.172 ...\n",
            " $ dimension_mean   : num  0.0595 0.0649 0.0634 0.0607 0.0554 ...\n",
            " $ radius_se        : num  0.236 0.451 0.197 0.338 0.178 ...\n",
            " $ texture_se       : num  0.666 1.197 1.387 1.343 0.412 ...\n",
            " $ perimeter_se     : num  1.67 3.43 1.34 1.85 1.34 ...\n",
            " $ area_se          : num  17.4 27.1 13.5 26.3 17.7 ...\n",
            " $ smoothness_se    : num  0.00805 0.00747 0.00516 0.01127 0.00501 ...\n",
            " $ compactness_se   : num  0.0118 0.03581 0.00936 0.03498 0.01485 ...\n",
            " $ concavity_se     : num  0.0168 0.0335 0.0106 0.0219 0.0155 ...\n",
            " $ points_se        : num  0.01241 0.01365 0.00748 0.01965 0.00915 ...\n",
            " $ symmetry_se      : num  0.0192 0.035 0.0172 0.0158 0.0165 ...\n",
            " $ dimension_se     : num  0.00225 0.00332 0.0022 0.00344 0.00177 ...\n",
            " $ radius_worst     : num  13.5 11.9 12.4 11.9 16.2 ...\n",
            " $ texture_worst    : num  15.6 22.9 26.4 15.8 15.7 ...\n",
            " $ perimeter_worst  : num  87 78.3 79.9 76.5 104.5 ...\n",
            " $ area_worst       : num  549 425 471 434 819 ...\n",
            " $ smoothness_worst : num  0.139 0.121 0.137 0.137 0.113 ...\n",
            " $ compactness_worst: num  0.127 0.252 0.148 0.182 0.174 ...\n",
            " $ concavity_worst  : num  0.1242 0.1916 0.1067 0.0867 0.1362 ...\n",
            " $ points_worst     : num  0.0939 0.0793 0.0743 0.0861 0.0818 ...\n",
            " $ symmetry_worst   : num  0.283 0.294 0.3 0.21 0.249 ...\n",
            " $ dimension_worst  : num  0.0677 0.0759 0.0788 0.0678 0.0677 ...\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MHbXqDrHmH8B",
        "colab_type": "text"
      },
      "source": [
        "### Muitos classificadores requerem que as variáveis sejam do tipo fator"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0aYMgt0EmHkf",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 70
        },
        "outputId": "921fbf01-85b4-4179-c5af-6181b8d6a2f4"
      },
      "source": [
        "%%R \n",
        "  table(dados$diagnosis)\n",
        "  # Criar a tabela de Cntingência da Variável Alvo com valores de \"benigno\" e \"maligno\""
      ],
      "execution_count": 166,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\n",
            "Benigno Maligno \n",
            "    357     212 \n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xBFoEm0nxTp6",
        "colab_type": "text"
      },
      "source": [
        "### Fatorando a variável target diagnóstico "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9KRbIaiYma68",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%%R\n",
        "  dados$diagnosis <- factor(dados$diagnosis, levels = c(\"Benigno\", \"Maligno\"), labels = c(\"Benigno\", \"Maligno\"))\n",
        "  # a variável target que era caracter agora passa a ser fator"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OfeAURyaTLha",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "93a307ec-e891-4c53-e75a-0bd9d0b6ad14"
      },
      "source": [
        "%%R\n",
        "  str(dados$diagnosis)"
      ],
      "execution_count": 168,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            " Factor w/ 2 levels \"Benigno\",\"Maligno\": 1 1 1 1 1 1 1 2 1 1 ...\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fpTeVvB8TP9v",
        "colab_type": "text"
      },
      "source": [
        "###Verificando a proporção das categorias de Câncer"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "QcDrgIa_TR8D",
        "colab_type": "code",
        "outputId": "e27093e8-2a52-4824-f8e8-f694f9bf81fd",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 70
        }
      },
      "source": [
        "%%R\n",
        "  round(prop.table(table(dados$diagnosis)) * 100, digits = 1)"
      ],
      "execution_count": 169,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\n",
            "Benigno Maligno \n",
            "   62.7    37.3 \n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7equqC1yTbhr",
        "colab_type": "text"
      },
      "source": [
        "###Medidas de Tendência Central\n",
        "\n",
        "Detectamos um problema de escala entre os dados, que então precisam ser normalizados. \n",
        "\n",
        "O cálculo de distância feito pelo kNN é dependente das medidas de escala nos dados de entrada."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GoESvX7wTX1E",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 141
        },
        "outputId": "7d659f5c-465b-4f77-c893-700688d03f2a"
      },
      "source": [
        "%%R\n",
        "  summary(dados[c(\"radius_mean\", \"area_mean\", \"smoothness_mean\")])"
      ],
      "execution_count": 170,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "  radius_mean      area_mean    smoothness_mean \n",
            " Min.   : 6.98   Min.   : 144   Min.   :0.0526  \n",
            " 1st Qu.:11.70   1st Qu.: 420   1st Qu.:0.0864  \n",
            " Median :13.37   Median : 551   Median :0.0959  \n",
            " Mean   :14.13   Mean   : 655   Mean   :0.0964  \n",
            " 3rd Qu.:15.78   3rd Qu.: 783   3rd Qu.:0.1053  \n",
            " Max.   :28.11   Max.   :2501   Max.   :0.1634  \n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VZWYAeY4TpHM",
        "colab_type": "text"
      },
      "source": [
        "### Criando uma função de **Normalização** para converter os dados numéricos na mesma escala\n",
        "\n",
        "\n",
        "#### Muitos algoritmos esperam receber os mesmos dados na mesma escala e com uma distribuição normal. Média igual a 0 e desvio-padrão igual a 1. \n",
        "\n",
        "#### Na maioria das vezes os dados não virão na mesma escala e nem com uma distribuição normal. Cabe então identificar e normalizar se necessário. \n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KIeWxFifTtCM",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%%R \n",
        "  normalizar <- function(x) { # \"Padronização\"\n",
        "  return ((x - min(x)) / (max(x) - min(x))) # a função pega o valor \"x\" que rece e subtrai do valor mínimo.\n",
        "}                                           # dividirá pelo valor máximo de \"x\" subtraído deo mínimo de \"x\""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1qI1JfFoTy_r",
        "colab_type": "text"
      },
      "source": [
        "#### Testando a função de normalização - **os resultados devem ser idênticos**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "s6cAKTJ_UAZG",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "aa53b1d7-817b-434e-b4a6-811370c3e33b"
      },
      "source": [
        "%%R\n",
        "  normalizar(c(1, 2, 3, 4, 5)) # valores unitários coincidem com a dezena - confirma a normalização \n",
        "  "
      ],
      "execution_count": 135,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[1] 0.00 0.25 0.50 0.75 1.00\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tE7KrfTRyJsH",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "dd39e47f-6d4d-4ead-aa36-d973ca38d6ce"
      },
      "source": [
        "%%R\n",
        "  normalizar(c(10, 20, 30, 40, 50)) # valores em dezena concidem com unitários - confirma a normalização "
      ],
      "execution_count": 136,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[1] 0.00 0.25 0.50 0.75 1.00\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Rb9GtBGYUCVr",
        "colab_type": "text"
      },
      "source": [
        "###Normalizando os dados com lapply() \n",
        "\n",
        "#### lapply() percorrerá o conjunto \"dados\" da coluna 2 à 31, aplicando a função criada normalizar(). \n",
        "\n",
        "#### a saída sera um novo dataset chamado dados_norm com as variáveis numéricas em escala. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "w6v8457cUPHZ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%%R\n",
        "  dados_norm <- as.data.frame(lapply(dados[2:31], normalizar))\n",
        "  head(dados_norm)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ebz5nn2QYivG",
        "colab_type": "text"
      },
      "source": [
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MV3x6XhjUWis",
        "colab_type": "text"
      },
      "source": [
        "##**Etapa 3** - Treinando o modelo com KNN\n",
        "\n",
        "### Um dos algoritmos de Machine Learning mais simples que existem "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OJYcoN13Ua2J",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%%R\n",
        "  # Carregando o pacote library class para rodar o KNN\n",
        "  install.packages(\"class\")\n",
        "  library(class)\n",
        "  # ?knn"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QCwsvr2jUnMf",
        "colab_type": "text"
      },
      "source": [
        "### Criando dados de treino e dados de teste"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "vz2eVcYEUqbh",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%%R\n",
        "  #notação de índice  \n",
        "  dados_treino <- dados_norm[1:469, ]  # Selecionando 469 linhas e todas as suas colunas, aplicando no dataset \"treino\"\n",
        "  dados_teste <- dados_norm[470:569, ] # Selecionando linhas restantes e todas as suas colunas, aplicando no dataset \"teste\""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gvven4C1UvIi",
        "colab_type": "text"
      },
      "source": [
        "### Criando Labels para os dados de treino e de teste\n",
        "\n",
        "Labels são as palavras | valores | siglas usadas para aplicação da **variável target**.\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6ITsBmcYU0Q8",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "5e69813a-6fb9-446d-c487-e1f5a3386238"
      },
      "source": [
        "%%R\n",
        "  dados_treino_labels <- dados[1:469, 1] # Aplicando a identificação dos labels à PRIMEIRA COLUNHA e suas 469 linhas\n",
        "  length(dados_treino_labels) # Retornar comprimento dos dados de Treino\n"
      ],
      "execution_count": 175,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[1] 469\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "107ewYY22SLg",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "1ddb0f5b-0056-423e-b6d0-e5ee396b8cac"
      },
      "source": [
        " %%R \n",
        "  dados_teste_labels <- dados[470:569, 1] # Aplicando a identificação dos labels à PRIMEIRA COLUNHA e suas 100 linhas restantes\n",
        "  length(dados_teste_labels) # Retornar comprimento dos dados de Teste"
      ],
      "execution_count": 176,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[1] 100\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dQ09vfg8U8dS",
        "colab_type": "text"
      },
      "source": [
        "### Criando Modelo kNN"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "x0teEEquVAg3",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%%R\n",
        "  modelo_knn_v1 <- knn(train = dados_treino,   # Chamar a função knn indicando dados de treino\n",
        "                     test = dados_teste,       # indicar dados de teste \n",
        "                     cl = dados_treino_labels, # aplicar slicing de lables anterior\n",
        "                     k = 21)                   # este k = 21 indica que o modelo observará 21 pontos de dados mais próximos de cada ponto de dado, com base na distância euclidiana"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IMTg2qsDVLre",
        "colab_type": "text"
      },
      "source": [
        "### A **função knn() retorna um objeto do tipo fator** com as previsões para cada exemplo no dataset de teste"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "u-_u8LHdVRAQ",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        },
        "outputId": "dd458001-7727-4a27-aa07-41ac45b8514f"
      },
      "source": [
        "%%R\n",
        "  summary(modelo_knn_v1)"
      ],
      "execution_count": 143,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Benigno Maligno \n",
            "     63      37 \n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "K99p87QVYha-",
        "colab_type": "text"
      },
      "source": [
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yaUfLlh0VVGb",
        "colab_type": "text"
      },
      "source": [
        "##**Etapa 4** - Avaliando e Interpretando o Modelo"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ntpa0tBiVYjL",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%%R\n",
        "  install.packages(\"gmodels\")\n",
        "  library(gmodels)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pgsx_n-dVdoC",
        "colab_type": "text"
      },
      "source": [
        "### Criando uma tabela cruzada dos dados previstos x dados atuais. \n",
        "### Usaremos amostra com 100 observações: length(dados_teste_labels)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "h0tq5SU8WAId",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 586
        },
        "outputId": "4d58d04b-4e08-47f9-ab80-baad833457f8"
      },
      "source": [
        "%%R\n",
        "  # Criando matriz de confusão \n",
        "  CrossTable(x = dados_teste_labels, # Selecionar labels de Teste\n",
        "             y = modelo_knn_v1,      # Aplicar Movedelo V1 anteriormente criado\n",
        "             prop.chisq = FALSE)     # Não imprimir estatísticas de Qui-Quadrado"
      ],
      "execution_count": 180,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\n",
            " \n",
            "   Cell Contents\n",
            "|-------------------------|\n",
            "|                       N |\n",
            "|           N / Row Total |\n",
            "|           N / Col Total |\n",
            "|         N / Table Total |\n",
            "|-------------------------|\n",
            "\n",
            " \n",
            "Total Observations in Table:  100 \n",
            "\n",
            " \n",
            "                   | modelo_knn_v1 \n",
            "dados_teste_labels |   Benigno |   Maligno | Row Total | \n",
            "-------------------|-----------|-----------|-----------|\n",
            "           Benigno |        61 |         0 |        61 | \n",
            "                   |     1.000 |     0.000 |     0.610 | \n",
            "                   |     0.968 |     0.000 |           | \n",
            "                   |     0.610 |     0.000 |           | \n",
            "-------------------|-----------|-----------|-----------|\n",
            "           Maligno |         2 |        37 |        39 | \n",
            "                   |     0.051 |     0.949 |     0.390 | \n",
            "                   |     0.032 |     1.000 |           | \n",
            "                   |     0.020 |     0.370 |           | \n",
            "-------------------|-----------|-----------|-----------|\n",
            "      Column Total |        63 |        37 |       100 | \n",
            "                   |     0.630 |     0.370 |           | \n",
            "-------------------|-----------|-----------|-----------|\n",
            "\n",
            " \n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "y3obmf8FKZ1M",
        "colab_type": "text"
      },
      "source": [
        "###**Interpretando os Resultados da Matriz de Confusão acima:**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Zh28S4ebWHJs",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# A tabela cruzada mostra 4 possíveis valores, que representam:\n",
        "\n",
        "TP | FP\n",
        "FN | TN\n",
        "\n",
        "# Temos:\n",
        "# Cenário 1: Célula Benigno (Observado) x Benigno (Previsto) - 61 casos - true positive \n",
        "# Cenário 2: Célula Maligno (Observado) x Benigno (Previsto) - 00 casos - false positive (o modelo errou)\n",
        "# Cenário 3: Célula Benigno (Observado) x Maligno (Previsto) - 02 casos - false negative (o modelo errou) não conseguiu compreender tudo que existe de relação entre os dados\n",
        "# Cenário 4: Célula Maligno (Observado) x Maligno (Previsto) - 37 casos - true negative \n",
        "\n",
        "# Lendo a Confusion Matrix (Perspectiva de ter ou não a doença):\n",
        "\n",
        "# True Negative  = nosso modelo previu que a pessoa NÃO tinha a doença e os dados mostraram que realmente a pessoa NÃO tinha a doença\n",
        "# False Positive = nosso modelo previu que a pessoa tinha a doença e os dados mostraram que NÃO, a pessoa tinha a doença\n",
        "# False Negative = nosso modelo previu que a pessoa NÃO tinha a doença e os dados mostraram que SIM, a pessoa tinha a doença\n",
        "# True Positive = nosso modelo previu que a pessoa tinha a doença e os dados mostraram que SIM, a pessoa tinha a doença\n",
        "\n",
        "# ------------------------------------------------------------------------\n",
        "\n",
        "# Erros podem ter consequências diferentes\n",
        "# Falso Positivo - Erro Tipo I \n",
        "# Falso Negativo - Erro Tipo II\n",
        "\n",
        "# Taxa de acerto deste  Modelo: 98% (acertou 98 em 100)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Tu7yToPcYflm",
        "colab_type": "text"
      },
      "source": [
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aLaZo4ntWJC0",
        "colab_type": "text"
      },
      "source": [
        "##**Etapa 5** - Otimizando a Performance do Modelo"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yZ-GF5-aWQN4",
        "colab_type": "text"
      },
      "source": [
        "### Usando a função scale() para padronizar na mesma escala do z-score "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jG5PX1d-WN7Q",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%%R\n",
        "  #?scale() a função faz uma escala e centralização dos objetos. scale(conjuntod de dados)\n",
        "  dados_z <- as.data.frame(scale(dados[-1])) # -1 é para excluir justamente a coluna que temos a variável alvo e aplicar em dados_Z"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J6PR97cyWZ26",
        "colab_type": "text"
      },
      "source": [
        "### Confirmando transformação realizada com sucesso\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0hMZ1KTLWmi7",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        },
        "outputId": "e1b32118-1602-46d7-ef3d-28a053a9bbac"
      },
      "source": [
        "%%R\n",
        "  summary(dados_z$area_mean)\n",
        "  # temos a média igual a 0"
      ],
      "execution_count": 182,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. \n",
            " -1.453  -0.667  -0.295   0.000   0.363   5.246 \n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Bi3c1_tXM83N",
        "colab_type": "text"
      },
      "source": [
        "### Fazendo uma nova divisão em dados de Treino e de Teste - **Novamente**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lFMt-0uiWqTt",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%%R \n",
        " # Dados de Treino\n",
        " dados_treino <- dados_z[1:469, ]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9FNg730ZWowc",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%%R\n",
        "  # Dados de Teste\n",
        "  dados_teste <- dados_z[470:569, ]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pzoEvjGzNN1b",
        "colab_type": "text"
      },
      "source": [
        "### Criando Labels para os dados de treino e de teste - **Novamente**\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xEwHVpKVNNSD",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "46b1c2c4-bae2-4c49-81d6-96578baa24b0"
      },
      "source": [
        "%%R\n",
        "  dados_treino_labels <- dados[1:469, 1] # Aplicando a identificação dos labels à PRIMEIRA COLUNHA e suas 469 linhas\n",
        "  length(dados_treino_labels) # Retornar comprimento dos dados de Treino"
      ],
      "execution_count": 186,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[1] 469\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "l8eKFtqGNM-A",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "bbadbf84-73ea-469f-b49e-6abdbad1f68d"
      },
      "source": [
        " %%R \n",
        "  dados_teste_labels <- dados[470:569, 1] # Aplicando a identificação dos labels à PRIMEIRA COLUNHA e suas 100 linhas restantes\n",
        "  length(dados_teste_labels) # Retornar comprimento dos dados de Teste"
      ],
      "execution_count": 187,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[1] 100\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tS8YGhc1XwPQ",
        "colab_type": "text"
      },
      "source": [
        "### Reclassificando como modelo kNN Versão 2"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "nEGaKY3HX14a",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%%R\n",
        "  modelo_knn_v2 <- knn(train = dados_treino,   # Chamar a função knn 2 indicandos os novos dados de treino\n",
        "                     test = dados_teste,       # indicar novos dados de teste\n",
        "                     cl = dados_treino_labels, # Aplicar slicing de labels anteriores\n",
        "                     k = 21)                   # este k = 21 indica que o modelo observará 21 pontos de dados mais próximos de cada ponto de dado, com base na distância euclidiana\n",
        "  "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ArhXm_WMYAvt",
        "colab_type": "text"
      },
      "source": [
        "### Criando uma tabela cruzada dos dados previstos x dados atuais\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xtoKlrVUYG3Y",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 586
        },
        "outputId": "f769af6a-3780-43dd-c1fd-e4491a744894"
      },
      "source": [
        "%%R\n",
        "  CrossTable(x = dados_teste_labels, y = modelo_knn_v2, prop.chisq = FALSE)\n",
        "  # É possível notar que a quantidade de erros aumentou"
      ],
      "execution_count": 190,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\n",
            " \n",
            "   Cell Contents\n",
            "|-------------------------|\n",
            "|                       N |\n",
            "|           N / Row Total |\n",
            "|           N / Col Total |\n",
            "|         N / Table Total |\n",
            "|-------------------------|\n",
            "\n",
            " \n",
            "Total Observations in Table:  100 \n",
            "\n",
            " \n",
            "                   | modelo_knn_v2 \n",
            "dados_teste_labels |   Benigno |   Maligno | Row Total | \n",
            "-------------------|-----------|-----------|-----------|\n",
            "           Benigno |        61 |         0 |        61 | \n",
            "                   |     1.000 |     0.000 |     0.610 | \n",
            "                   |     0.924 |     0.000 |           | \n",
            "                   |     0.610 |     0.000 |           | \n",
            "-------------------|-----------|-----------|-----------|\n",
            "           Maligno |         5 |        34 |        39 | \n",
            "                   |     0.128 |     0.872 |     0.390 | \n",
            "                   |     0.076 |     1.000 |           | \n",
            "                   |     0.050 |     0.340 |           | \n",
            "-------------------|-----------|-----------|-----------|\n",
            "      Column Total |        66 |        34 |       100 | \n",
            "                   |     0.660 |     0.340 |           | \n",
            "-------------------|-----------|-----------|-----------|\n",
            "\n",
            " \n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wu2Y2uA2YVjQ",
        "colab_type": "text"
      },
      "source": [
        "### Experimentar diferentes valores para *K*"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DZ50Dom-Yd_E",
        "colab_type": "text"
      },
      "source": [
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "epRf3zRyY2s7",
        "colab_type": "text"
      },
      "source": [
        "##**Etapa 6** - Construindo um Modelo com Algoritmo **Support Vector Machine (SVM)**\n",
        "\n",
        "Trabalhar com dados não linearmente separáveis. \n",
        "\n",
        "Pode ser aplicado em **Regressão** e **Classificação**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mgiF4CuuZE4Y",
        "colab_type": "text"
      },
      "source": [
        "### Preparar dataset"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "16KLrh4PZAo0",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%%R\n",
        "set.seed(40) "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5uaOmA-5Y2bk",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 586
        },
        "outputId": "6cefa079-9ebc-452b-f009-521f74769ad7"
      },
      "source": [
        "%%R\n",
        "  dados <- read.csv(\"dataset.csv\", stringsAsFactors = FALSE) # Carregar novamente o conjunto de dados\n",
        "  dados$id = NULL # Remover a coluna ID\n",
        "  str(dados)"
      ],
      "execution_count": 193,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "'data.frame':\t569 obs. of  31 variables:\n",
            " $ diagnosis        : chr  \"B\" \"B\" \"B\" \"B\" ...\n",
            " $ radius_mean      : num  12.3 10.6 11 11.3 15.2 ...\n",
            " $ texture_mean     : num  12.4 18.9 16.8 13.4 13.2 ...\n",
            " $ perimeter_mean   : num  78.8 69.3 70.9 73 97.7 ...\n",
            " $ area_mean        : num  464 346 373 385 712 ...\n",
            " $ smoothness_mean  : num  0.1028 0.0969 0.1077 0.1164 0.0796 ...\n",
            " $ compactness_mean : num  0.0698 0.1147 0.078 0.1136 0.0693 ...\n",
            " $ concavity_mean   : num  0.0399 0.0639 0.0305 0.0464 0.0339 ...\n",
            " $ points_mean      : num  0.037 0.0264 0.0248 0.048 0.0266 ...\n",
            " $ symmetry_mean    : num  0.196 0.192 0.171 0.177 0.172 ...\n",
            " $ dimension_mean   : num  0.0595 0.0649 0.0634 0.0607 0.0554 ...\n",
            " $ radius_se        : num  0.236 0.451 0.197 0.338 0.178 ...\n",
            " $ texture_se       : num  0.666 1.197 1.387 1.343 0.412 ...\n",
            " $ perimeter_se     : num  1.67 3.43 1.34 1.85 1.34 ...\n",
            " $ area_se          : num  17.4 27.1 13.5 26.3 17.7 ...\n",
            " $ smoothness_se    : num  0.00805 0.00747 0.00516 0.01127 0.00501 ...\n",
            " $ compactness_se   : num  0.0118 0.03581 0.00936 0.03498 0.01485 ...\n",
            " $ concavity_se     : num  0.0168 0.0335 0.0106 0.0219 0.0155 ...\n",
            " $ points_se        : num  0.01241 0.01365 0.00748 0.01965 0.00915 ...\n",
            " $ symmetry_se      : num  0.0192 0.035 0.0172 0.0158 0.0165 ...\n",
            " $ dimension_se     : num  0.00225 0.00332 0.0022 0.00344 0.00177 ...\n",
            " $ radius_worst     : num  13.5 11.9 12.4 11.9 16.2 ...\n",
            " $ texture_worst    : num  15.6 22.9 26.4 15.8 15.7 ...\n",
            " $ perimeter_worst  : num  87 78.3 79.9 76.5 104.5 ...\n",
            " $ area_worst       : num  549 425 471 434 819 ...\n",
            " $ smoothness_worst : num  0.139 0.121 0.137 0.137 0.113 ...\n",
            " $ compactness_worst: num  0.127 0.252 0.148 0.182 0.174 ...\n",
            " $ concavity_worst  : num  0.1242 0.1916 0.1067 0.0867 0.1362 ...\n",
            " $ points_worst     : num  0.0939 0.0793 0.0743 0.0861 0.0818 ...\n",
            " $ symmetry_worst   : num  0.283 0.294 0.3 0.21 0.249 ...\n",
            " $ dimension_worst  : num  0.0677 0.0759 0.0788 0.0678 0.0677 ...\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XNOB6b2uQ9Fx",
        "colab_type": "text"
      },
      "source": [
        "### Criando uma coluna de Índice para separar os dados de Treino e de Teste"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BO5QrwLNQxB7",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 604
        },
        "outputId": "f0a30870-d37a-4e7f-cf8c-17cef53fd34d"
      },
      "source": [
        "%%R\n",
        "  dados[,'index'] <- ifelse(runif(nrow(dados)) < 0.8,1,0)\n",
        "  str(dados)"
      ],
      "execution_count": 195,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "'data.frame':\t569 obs. of  32 variables:\n",
            " $ diagnosis        : chr  \"B\" \"B\" \"B\" \"B\" ...\n",
            " $ radius_mean      : num  12.3 10.6 11 11.3 15.2 ...\n",
            " $ texture_mean     : num  12.4 18.9 16.8 13.4 13.2 ...\n",
            " $ perimeter_mean   : num  78.8 69.3 70.9 73 97.7 ...\n",
            " $ area_mean        : num  464 346 373 385 712 ...\n",
            " $ smoothness_mean  : num  0.1028 0.0969 0.1077 0.1164 0.0796 ...\n",
            " $ compactness_mean : num  0.0698 0.1147 0.078 0.1136 0.0693 ...\n",
            " $ concavity_mean   : num  0.0399 0.0639 0.0305 0.0464 0.0339 ...\n",
            " $ points_mean      : num  0.037 0.0264 0.0248 0.048 0.0266 ...\n",
            " $ symmetry_mean    : num  0.196 0.192 0.171 0.177 0.172 ...\n",
            " $ dimension_mean   : num  0.0595 0.0649 0.0634 0.0607 0.0554 ...\n",
            " $ radius_se        : num  0.236 0.451 0.197 0.338 0.178 ...\n",
            " $ texture_se       : num  0.666 1.197 1.387 1.343 0.412 ...\n",
            " $ perimeter_se     : num  1.67 3.43 1.34 1.85 1.34 ...\n",
            " $ area_se          : num  17.4 27.1 13.5 26.3 17.7 ...\n",
            " $ smoothness_se    : num  0.00805 0.00747 0.00516 0.01127 0.00501 ...\n",
            " $ compactness_se   : num  0.0118 0.03581 0.00936 0.03498 0.01485 ...\n",
            " $ concavity_se     : num  0.0168 0.0335 0.0106 0.0219 0.0155 ...\n",
            " $ points_se        : num  0.01241 0.01365 0.00748 0.01965 0.00915 ...\n",
            " $ symmetry_se      : num  0.0192 0.035 0.0172 0.0158 0.0165 ...\n",
            " $ dimension_se     : num  0.00225 0.00332 0.0022 0.00344 0.00177 ...\n",
            " $ radius_worst     : num  13.5 11.9 12.4 11.9 16.2 ...\n",
            " $ texture_worst    : num  15.6 22.9 26.4 15.8 15.7 ...\n",
            " $ perimeter_worst  : num  87 78.3 79.9 76.5 104.5 ...\n",
            " $ area_worst       : num  549 425 471 434 819 ...\n",
            " $ smoothness_worst : num  0.139 0.121 0.137 0.137 0.113 ...\n",
            " $ compactness_worst: num  0.127 0.252 0.148 0.182 0.174 ...\n",
            " $ concavity_worst  : num  0.1242 0.1916 0.1067 0.0867 0.1362 ...\n",
            " $ points_worst     : num  0.0939 0.0793 0.0743 0.0861 0.0818 ...\n",
            " $ symmetry_worst   : num  0.283 0.294 0.3 0.21 0.249 ...\n",
            " $ dimension_worst  : num  0.0677 0.0759 0.0788 0.0678 0.0677 ...\n",
            " $ index            : num  0 0 0 1 1 0 1 0 1 1 ...\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Hlk1vDYCZVv0",
        "colab_type": "text"
      },
      "source": [
        "#### Dados de Treino com chamada pelo Índice = 1"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kSybQPH-ZbcG",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%%R\n",
        "  trainset <- dados[dados$index==1,] # Obter dados de treino para o objeto trainset"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "o1V9I_dcZZNc",
        "colab_type": "text"
      },
      "source": [
        "#### Dados de Teste com chamada pelo Índice = 0\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fC3q2JJyZecP",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%%R\n",
        "  testset <- dados[dados$index==0,] # Obter dados de teste para o objeto testset"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tbBJguzdST1g",
        "colab_type": "text"
      },
      "source": [
        "####Obter o Index de treino"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Zsd_fC7uSZ4P",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%%R\n",
        "trainColNum <- grep(\"index\", names(trainset)) # Obter o índice apenas do conjunto de dados de treino"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cqzkKSHpZVkF",
        "colab_type": "text"
      },
      "source": [
        "### Remover o índice dos datasets"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gexdOBZRTpUm",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%%R\n",
        "  trainset <- trainset[,-trainColNum]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7lboB-coTpJM",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%%R\n",
        "  testset <- testset[, -trainColNum]"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QVZPOx4uUBhq",
        "colab_type": "text"
      },
      "source": [
        "###Obter Índice de coluna da variável target no conjunto de dados"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GkIFa8WhUSp4",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%%R\n",
        "typeColNum <- grep(\"diag\", names(dados)) # grep() pesquisa pelo nome de variável que começa com \"diag\""
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mURktnnOZuEP",
        "colab_type": "text"
      },
      "source": [
        "---"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-p8GpcmNalas",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%%R\n",
        " install.packages(\"e1071\") \n",
        " library(e1071)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8w-lFOUqZskh",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "#?svw"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NZrUzaT1ZtZH",
        "colab_type": "text"
      },
      "source": [
        "## Criar Modelo\n",
        "\n",
        "### Ajustamos Kernel para radial, já que este conjunto não tem um plano linear que pode ser desenhado\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TzsQHpSwayue",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%%R\n",
        "  modelo_svm_v1 <- svm(diagnosis ~ .,            # passar variável target relacionada com todas as outras\n",
        "                     data = trainset,            # indicar dataset de treino\n",
        "                     type = 'C-classification',  # ajustar tipo para modelo de classificação\n",
        "                     kernel = 'radial')          # kernel é a base do algoritmo svm"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "frDhK_pxa2Lt",
        "colab_type": "text"
      },
      "source": [
        "#Previsões"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "E0OdKlmGbBDO",
        "colab_type": "text"
      },
      "source": [
        "### Previsões aplicadas no dataset de TREINO"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "h408RS8ya9vP",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%%R\n",
        "  pred_train <- predict(modelo_svm_v1, trainset) "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UuCRobPGbHJJ",
        "colab_type": "text"
      },
      "source": [
        "### Calcular o percentual de previssões acertadas com dataset de treino"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VA--_I04bMZ1",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "51276ffe-bae3-4b7a-aef6-0cb10ec3e9d5"
      },
      "source": [
        "%%R\n",
        "  mean_pred_train <- mean(pred_train == trainset$diagnosis, n = 6)  \n",
        "  mean_pred_train \n",
        "  # com dados de históricos de treino o modelo acertou 98.47 das vezes, praticamente o mesmo com kNN\n",
        "  # mudar de algoritmo não foi tão positivo, provavelmente estamos no limite do dataset"
      ],
      "execution_count": 209,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[1] 0.988688\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xxmhjE4rbP08",
        "colab_type": "text"
      },
      "source": [
        "### Previsões nos dados de Teste"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "siRZc7ZkbSAy",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%%R\n",
        "  pred_test <- predict(modelo_svm_v1, testset) "
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "blLsEDG3be8m",
        "colab_type": "text"
      },
      "source": [
        "### Percentual de previsões corretas com dataset de teste"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dP9inJWsbfsi",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "5073e1fc-a940-4e08-a21a-ef48eb82c133"
      },
      "source": [
        "%%R\n",
        "  mean(pred_test == testset$diagnosis)  "
      ],
      "execution_count": 211,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[1] 0.96063\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5E2HzioWbiYu",
        "colab_type": "text"
      },
      "source": [
        "### Matriz de Confusão "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "img-zfs7bkpS",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 88
        },
        "outputId": "e726f2d4-0d62-45e1-af0d-07a33632e813"
      },
      "source": [
        "%%R \n",
        "  table(pred_test, testset$diagnosis)"
      ],
      "execution_count": 212,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "         \n",
            "pred_test  B  M\n",
            "        B 82  3\n",
            "        M  2 40\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ndYRifSMbLph",
        "colab_type": "text"
      },
      "source": [
        "---"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yyWDHHlIbp2f",
        "colab_type": "text"
      },
      "source": [
        "##**Etapa 7** - Construindo um Modelo com Algoritmo Random Forest\n",
        "\n",
        "### Quando algumas variáveis não tem um nível de relevância alto, elas afetam a Performance do modelo. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gwuFpGmubvDt",
        "colab_type": "text"
      },
      "source": [
        "### Criando o modelo"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TvVMaRC6b0Gw",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%%R\n",
        "  install.packages(\"rpart\")\n",
        "  library(rpart)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XlGIAR7VaAdE",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%%R\n",
        "  modelo_rf_v1 = rpart(diagnosis ~ .,                       # Chamar função rpart() e passar a variável target relacionado com as demais preditoras\n",
        "                       data = trainset,                     # Indicar qual é o dataset\n",
        "                       control = rpart.control(cp = .0005)) # Ajustar rpart.controlo com nível de precisão .0005, o nívlel das folhas"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Pd8JMLXobzVr",
        "colab_type": "text"
      },
      "source": [
        "###Previsões nos dados de teste"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-Ur6T45Pb9jb",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "%%R\n",
        "  tree_pred = predict(modelo_rf_v1, testset, type='class')"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a5gIY7dkb8sw",
        "colab_type": "text"
      },
      "source": [
        "### Calcular percentual de acertos com dados de Teste\n",
        "\n",
        "#### Tivemos uma performance pior do que com a aplicação do kNN"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9x23yj2mcEG0",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        },
        "outputId": "b6457e41-c7bb-482e-ab5f-dcbe54cc5ee3"
      },
      "source": [
        "%%R\n",
        "  mean(tree_pred==testset$diagnosis) \n"
      ],
      "execution_count": 215,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[1] 0.92126\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mdkeXNJDbu1u",
        "colab_type": "text"
      },
      "source": [
        "### Matriz de Confusão \n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "h0h_E8dgbt5C",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 88
        },
        "outputId": "1db5c3ac-dedc-4f73-b061-3eedbb6fd374"
      },
      "source": [
        "%%R\n",
        "  table(tree_pred, testset$diagnosis)"
      ],
      "execution_count": 216,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "         \n",
            "tree_pred  B  M\n",
            "        B 79  5\n",
            "        M  5 38\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}