Breast Cancer SVM Classifier .ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "a6643b44",
   "metadata": {},
   "source": [
    "In this project, we will be working on a breast cancer dataset and building a Support Vector Machine Model to classify whether a certain cell is at a benign or malignant stage. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3af12b33",
   "metadata": {},
   "source": [
    "### 1. Importing necessary libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "5c1bfecc",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "#Train test split of data\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "#Modelling\n",
    "from sklearn import svm\n",
    "\n",
    "#Evaluation\n",
    "from sklearn.metrics import classification_report"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90b3bd4a",
   "metadata": {},
   "source": [
    "### 2. Dataset description"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b560c182",
   "metadata": {},
   "source": [
    "There are 700 records and each record has 11 characteristics. The columns/characteristics include 9 predictors, the sample ID, and class of the cell. The fields in each record are:<br>\n",
    "\n",
    "| Field name | Description |\n",
    "| -- | -- |\n",
    "| ID | Identifier |\n",
    "|Clump|ClumpThickness|<br>\n",
    "|UnifSize|Uniformity of cell size|<br>\n",
    "|UnifShape|Uniformity of cell shape|<br>\n",
    "|MargAdh|Marginal Adhesion|<br>\n",
    "|SingEpiSize|Single Epithelial Cell Size|<br>\n",
    "|BareNuc|Bare Nuclei|<br>\n",
    "|BlandChrom|Bland Chromatin|<br>\n",
    "|NormNucl|Normal Nucleoli|<br>\n",
    "|Mit|Mitosis|<br>\n",
    "|Class|Benign or malignant|<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d57de19",
   "metadata": {},
   "source": [
    "### 3. Importing and analysing dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "5185511b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>Clump</th>\n",
       "      <th>UnifSize</th>\n",
       "      <th>UnifShape</th>\n",
       "      <th>MargAdh</th>\n",
       "      <th>SingEpiSize</th>\n",
       "      <th>BareNuc</th>\n",
       "      <th>BlandChrom</th>\n",
       "      <th>NormNucl</th>\n",
       "      <th>Mit</th>\n",
       "      <th>Class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1000025</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1002945</td>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>7</td>\n",
       "      <td>10</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1015425</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1016277</td>\n",
       "      <td>6</td>\n",
       "      <td>8</td>\n",
       "      <td>8</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1017023</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        ID  Clump  UnifSize  UnifShape  MargAdh  SingEpiSize BareNuc  \\\n",
       "0  1000025      5         1          1        1            2       1   \n",
       "1  1002945      5         4          4        5            7      10   \n",
       "2  1015425      3         1          1        1            2       2   \n",
       "3  1016277      6         8          8        1            3       4   \n",
       "4  1017023      4         1          1        3            2       1   \n",
       "\n",
       "   BlandChrom  NormNucl  Mit  Class  \n",
       "0           3         1    1      2  \n",
       "1           3         2    1      2  \n",
       "2           3         1    1      2  \n",
       "3           3         7    1      2  \n",
       "4           3         1    1      2  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cellDataFrame = pd.read_csv('cell_samples.csv')\n",
    "cellDataFrame.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "ef83292e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(699, 11)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cellDataFrame.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "c73bfddb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ID             699\n",
       "Clump          699\n",
       "UnifSize       699\n",
       "UnifShape      699\n",
       "MargAdh        699\n",
       "SingEpiSize    699\n",
       "BareNuc        699\n",
       "BlandChrom     699\n",
       "NormNucl       699\n",
       "Mit            699\n",
       "Class          699\n",
       "dtype: int64"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#count records under each attribute and check if there are any missing attributes\n",
    "cellDataFrame.count()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "a6d47b2c",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2    458\n",
       "4    241\n",
       "Name: Class, dtype: int64"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Counting the number of malignant and benign cells in the dataset2\n",
    "cellDataFrame['Class'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb4f1f03",
   "metadata": {},
   "source": [
    "2 implies Benign and 4 implies Malignant"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "4e9fcbc7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ID              int64\n",
       "Clump           int64\n",
       "UnifSize        int64\n",
       "UnifShape       int64\n",
       "MargAdh         int64\n",
       "SingEpiSize     int64\n",
       "BareNuc        object\n",
       "BlandChrom      int64\n",
       "NormNucl        int64\n",
       "Mit             int64\n",
       "Class           int64\n",
       "dtype: object"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cellDataFrame.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2c0ab94",
   "metadata": {},
   "source": [
    "### 4. Split the dataset based on the classes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "83444758",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>Clump</th>\n",
       "      <th>UnifSize</th>\n",
       "      <th>UnifShape</th>\n",
       "      <th>MargAdh</th>\n",
       "      <th>SingEpiSize</th>\n",
       "      <th>BareNuc</th>\n",
       "      <th>BlandChrom</th>\n",
       "      <th>NormNucl</th>\n",
       "      <th>Mit</th>\n",
       "      <th>Class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>1017122</td>\n",
       "      <td>8</td>\n",
       "      <td>10</td>\n",
       "      <td>10</td>\n",
       "      <td>8</td>\n",
       "      <td>7</td>\n",
       "      <td>10</td>\n",
       "      <td>9</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>1041801</td>\n",
       "      <td>5</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>1044572</td>\n",
       "      <td>8</td>\n",
       "      <td>7</td>\n",
       "      <td>5</td>\n",
       "      <td>10</td>\n",
       "      <td>7</td>\n",
       "      <td>9</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>1047630</td>\n",
       "      <td>7</td>\n",
       "      <td>4</td>\n",
       "      <td>6</td>\n",
       "      <td>4</td>\n",
       "      <td>6</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>1050670</td>\n",
       "      <td>10</td>\n",
       "      <td>7</td>\n",
       "      <td>7</td>\n",
       "      <td>6</td>\n",
       "      <td>4</td>\n",
       "      <td>10</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         ID  Clump  UnifSize  UnifShape  MargAdh  SingEpiSize BareNuc  \\\n",
       "5   1017122      8        10         10        8            7      10   \n",
       "12  1041801      5         3          3        3            2       3   \n",
       "14  1044572      8         7          5       10            7       9   \n",
       "15  1047630      7         4          6        4            6       1   \n",
       "18  1050670     10         7          7        6            4      10   \n",
       "\n",
       "    BlandChrom  NormNucl  Mit  Class  \n",
       "5            9         7    1      4  \n",
       "12           4         4    1      4  \n",
       "14           5         5    4      4  \n",
       "15           4         3    1      4  \n",
       "18           4         1    2      4  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# combination of over-sampling the minority class and under-sampling the majority class can \n",
    "# achieve better classifier performance\n",
    "# here the minority class is the malignant cells \n",
    "malignantDataFrame = cellDataFrame[cellDataFrame['Class']==4][0:200]\n",
    "malignantDataFrame.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "2166203f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>Clump</th>\n",
       "      <th>UnifSize</th>\n",
       "      <th>UnifShape</th>\n",
       "      <th>MargAdh</th>\n",
       "      <th>SingEpiSize</th>\n",
       "      <th>BareNuc</th>\n",
       "      <th>BlandChrom</th>\n",
       "      <th>NormNucl</th>\n",
       "      <th>Mit</th>\n",
       "      <th>Class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1000025</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1002945</td>\n",
       "      <td>5</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>7</td>\n",
       "      <td>10</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1015425</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1016277</td>\n",
       "      <td>6</td>\n",
       "      <td>8</td>\n",
       "      <td>8</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "      <td>7</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1017023</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        ID  Clump  UnifSize  UnifShape  MargAdh  SingEpiSize BareNuc  \\\n",
       "0  1000025      5         1          1        1            2       1   \n",
       "1  1002945      5         4          4        5            7      10   \n",
       "2  1015425      3         1          1        1            2       2   \n",
       "3  1016277      6         8          8        1            3       4   \n",
       "4  1017023      4         1          1        3            2       1   \n",
       "\n",
       "   BlandChrom  NormNucl  Mit  Class  \n",
       "0           3         1    1      2  \n",
       "1           3         2    1      2  \n",
       "2           3         1    1      2  \n",
       "3           3         7    1      2  \n",
       "4           3         1    1      2  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "benignDataFrame = cellDataFrame[cellDataFrame['Class']==2][0:200]\n",
    "benignDataFrame.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d87a93df",
   "metadata": {},
   "source": [
    "### 5. Modify dataset based on requirements"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "50295a71",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ID             int64\n",
       "Clump          int64\n",
       "UnifSize       int64\n",
       "UnifShape      int64\n",
       "MargAdh        int64\n",
       "SingEpiSize    int64\n",
       "BareNuc        int32\n",
       "BlandChrom     int64\n",
       "NormNucl       int64\n",
       "Mit            int64\n",
       "Class          int64\n",
       "dtype: object"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#convert 'BareNuc' from object datatype to int datatype\n",
    "cellDataFrame = cellDataFrame[pd.to_numeric(cellDataFrame['BareNuc'], errors='coerce').notnull()]\n",
    "cellDataFrame['BareNuc'] = cellDataFrame['BareNuc'].astype('int')\n",
    "cellDataFrame.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "07260ddb",
   "metadata": {},
   "source": [
    "### 7. Remove unwated columns\n",
    "We will remove columns that won't help is in prediction (ID and class)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "ec5a79a0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['ID', 'Clump', 'UnifSize', 'UnifShape', 'MargAdh', 'SingEpiSize',\n",
       "       'BareNuc', 'BlandChrom', 'NormNucl', 'Mit', 'Class'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cellDataFrame.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "6a897c4a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 5,  1,  1,  1,  2,  1,  3,  1,  1],\n",
       "       [ 5,  4,  4,  5,  7, 10,  3,  2,  1],\n",
       "       [ 3,  1,  1,  1,  2,  2,  3,  1,  1],\n",
       "       [ 6,  8,  8,  1,  3,  4,  3,  7,  1],\n",
       "       [ 4,  1,  1,  3,  2,  1,  3,  1,  1]], dtype=int64)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "featureDataFrame = cellDataFrame[['Clump', 'UnifSize', 'UnifShape', 'MargAdh', 'SingEpiSize',\n",
    "       'BareNuc', 'BlandChrom', 'NormNucl', 'Mit']]\n",
    "\n",
    "#convert featureDataFrame into numpy n-dimensional array\n",
    "#independent variable\n",
    "X=np.asarray(featureDataFrame)\n",
    "\n",
    "#dependent variable\n",
    "y=np.asarray(cellDataFrame['Class'])\n",
    "\n",
    "X[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9429f4fc",
   "metadata": {},
   "source": [
    "### 8. Divide the data into train/test set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "1a7ab60d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(546, 9)\n",
      "(137, 9)\n",
      "(546,)\n",
      "(137,)\n"
     ]
    }
   ],
   "source": [
    "'''\n",
    "cellDataFrame (100) --> Train (80 rows) / Test (20 rows)\n",
    "\n",
    "Train(X,y) ## X is a 2D array an y is a 1D array\n",
    "Test(X, y)\n",
    "'''\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X,y, test_size=0.2, random_state=4) #random state is used to generate a random number \n",
    "\n",
    "print(X_train.shape)\n",
    "print(X_test.shape)\n",
    "print(y_train.shape)\n",
    "print(y_test.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d1e1e6c",
   "metadata": {},
   "source": [
    "### 9. Modelling"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "88170e2d",
   "metadata": {},
   "source": [
    "SVC - Support Vector Classifier - those data points near the hyperplane whose perpendicular distance to the hyperplane <br>\n",
    "If we sum that distance up of all the points near the hyperplane and maximize it such data points would be called SVC <br>\n",
    "The SVM algorithm offers a choice of kernel functions for performing its processing. Basically, mapping data into a higher dimensional space is called kerneling. <br>\n",
    "The mathematical function used for the trandformation is known as kernel function, and can be of different types, such as, \n",
    "\n",
    "1. Linear\n",
    "2. Polynomial\n",
    "3. Radial Basis Function (RBF)\n",
    "4. Sigmoid\n",
    "\n",
    "Each of these functions has its characteristics, its pros and cons, and its equations, but as there's no easy way of knowing which function performs best with any given dataset, we usually choose different functions and compare the results. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8b54c815",
   "metadata": {},
   "source": [
    "C- The Regularization parameter - tells the SVM optimization how much you want to avoid misclassifying each training example. Here C is the penalty parameter, which represents misclassification or error term. The misclassification or error term tells the SVM optimization how much error is bearable. This is how you can control the trade-off between decision boundary and misclassification term. A smaller value of C creates a small-margin hyperplane and a larger value of C creates a larger-margin hyperplane."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "0c4a7b5d",
   "metadata": {},
   "outputs": [],
   "source": [
    "classifier = svm.SVC(kernel='linear', gamma='auto', C=2)\n",
    "classifier.fit(X_train, y_train)\n",
    "\n",
    "y_predict = classifier.predict(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2c3a42da",
   "metadata": {},
   "source": [
    "### 10. Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "507314ec",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           2       1.00      0.94      0.97        90\n",
      "           4       0.90      1.00      0.95        47\n",
      "\n",
      "    accuracy                           0.96       137\n",
      "   macro avg       0.95      0.97      0.96       137\n",
      "weighted avg       0.97      0.96      0.96       137\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(classification_report(y_test, y_predict))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d81dc4b6",
   "metadata": {},
   "source": [
    "precision = true_positive/(true_positive + false_positive)\n",
    "\n",
    "recall = true_positive/(true_positive + false_negative) = true_positive/total_actual_positive\n",
    "\n",
    "F1: harmonical mean of precision = 2*((precision * recall)/(precision + recall))\n",
    "\n",
    "support: how many instances of the class were there"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26c86796",
   "metadata": {},
   "source": [
    "### Hyperparameter tuning for SVM model - C, gamma, epsilon, kernel \n",
    "\n",
    "##### 1. GridSearchCV\n",
    "##### 2. RandomizedSearchCV"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}