08_test_and_train_split.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d9592ef5-d4b3-4f54-9751-29ddd0636de3",
   "metadata": {},
   "source": [
    "### Introduction\n",
    "\n",
    "When we created our machine learning model in the notebook  [03 first steps with machine learning](./03_first_steps_with_machine_learning.ipynb) we called `fit(X, Y)` passing in the complete data set:\n",
    "\n",
    "```python\n",
    "X, y = iris.data, iris.target\n",
    "\n",
    "classifier = LinearSVC()\n",
    "\n",
    "model = classifier.fit(X, y)\n",
    "```\n",
    "\n",
    "The problem we are faced with now is how do we test our model's accuracy?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b9426c3c-255a-4d5d-a438-290b96f530d5",
   "metadata": {},
   "source": [
    "### Setup random state\n",
    "\n",
    "Many methods in Scikit-learn use randomness in their operation.  These methods usually have the parameter `random_state` allowing you to provide a fixed value which will give reproduceable results for others trying your code.\n",
    "\n",
    "Rather than setting the random state on each method, we can set it globally in Scikit-learn using the following code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "4a043ebb-5f6f-44a7-9b04-7d7c61aa7681",
   "metadata": {},
   "outputs": [],
   "source": [
    "# set the random state to one to ensure everyone running the code \n",
    "# gets the same results\n",
    "import numpy as np\n",
    "np.random.seed(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78d5366a-459f-410f-8b68-3df0776551fe",
   "metadata": {},
   "source": [
    "### Splitting our dataset\n",
    "\n",
    "Typically we will want to split our dataset so that we use a portion of the data for training the model and a portion of the dataset is held back to test the model.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "423bf934-3551-4711-9a45-f4184ec2a3ef",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training data size :  (100, 4) (100,)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.datasets import load_iris\n",
    "iris = load_iris()\n",
    "\n",
    "X, y = iris.data, iris.target\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "X_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.33)\n",
    "\n",
    "print('Training data size : ', X_train.shape, y_train.shape)\n",
    "\n",
    "# Exercise: uncomment the next line and print the size of your test data\n",
    "# print('Test data size : ',???.???, ???.???)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "96c38119-546c-4bd4-99d3-a7c61439b2ab",
   "metadata": {},
   "source": [
    "### Fitting the model\n",
    "\n",
    "Now that we have split the data, we can train the model using **only** the training data: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "d1433c0e-b5a1-4bc8-b6c6-ced0cd49699d",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/conda/lib/python3.9/site-packages/sklearn/svm/_base.py:1206: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "from sklearn.svm import LinearSVC\n",
    "classifier = LinearSVC()\n",
    "\n",
    "# Train the model (using just the training data)\n",
    "model = classifier.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "324289f5-3c12-4e9e-8659-510ad5ec712c",
   "metadata": {},
   "source": [
    "### Scoring the model\n",
    "\n",
    "Finally, we can test the accuracy of the model by using the [score](https://scikit-learn.org/stable/modules/generated/sklearn.svm.LinearSVC.html?highlight=linear%20svc#sklearn.svm.LinearSVC.score) function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "81bd79f5-db46-4fc2-9a9f-43ae02fe27f2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.92"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2e91f390-e32f-4ada-a7c9-ec5c8604009f",
   "metadata": {},
   "source": [
    "We will discuss more about quantifying the quality of predictions in the next notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "44a03a8b-936c-451f-b088-9f2770e5a686",
   "metadata": {},
   "source": [
    "### Summary\n",
    "\n",
    "In this notebook we covered the basics of splitting our dataset into train and test data.\n",
    "\n",
    "**Exercise:**\n",
    "\n",
    "Try retraining your model with the parameters `penalty='l1', dual=False` and obtain the score.\n",
    "\n",
    "The score when using the default parameter values (`classifier = LinearSVC()`) in Fitting the model was `0.92`.  Did you get a score of `0.9` with the parameters `penalty='l1', dual=False`?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f38393e4-1af3-4241-911e-be4417094ea9",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "### Navigation\n",
    "\n",
    "[Previous](./07_estimator_parameters.ipynb) | [Home](./00-README-FIRST.ipynb) | [Next](./09_model_performance.ipynb) notebook"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "06069662-51a0-48e1-831c-dc76ceef91ac",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}