11_svm/11_03_okcupid.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# OKCupid Profiles\n",
    "\n",
    "<img src=\"http://www.echoexaminer.com/wp-content/uploads/2015/12/OkCupid.jpg\"/ width=500>\n",
    "\n",
    "Included in this dataset are many different OKCupid profiles.  Using this data, let's see who can get the highest cross_validated score in predicting male/female based on the features below. Because there are so many features, you may have better luck using SVMs to classify the data.\n",
    "\n",
    "**Steps:**\n",
    "1. View the data. Think about some of the features that might be relevant. \n",
    "2. Start simple!! Use only one or two features first, and then work your way up.\n",
    "3. Score your classifier using 10 fold cross validation (Try using KFold or StratifiedKFold to ensure shuffling of the data)\n",
    "4. Report your score, let's make a competition out of this!\n",
    "\n",
    "**Some useful hints:**\n",
    "* Try using `pandas.get_dummies( data ) ` to convert string based features into many dummy features\n",
    "* Remember the distinction between categorical and continuous features\n",
    "* Using CountVectorizors here may be useful (think back to Naive Bayes examples)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn import svm\n",
    "\n",
    "%matplotlib inline\n",
    "pd.options.display.max_columns=1000\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>age</th>\n",
       "      <th>body_type</th>\n",
       "      <th>diet</th>\n",
       "      <th>drinks</th>\n",
       "      <th>drugs</th>\n",
       "      <th>education</th>\n",
       "      <th>essay0</th>\n",
       "      <th>essay1</th>\n",
       "      <th>essay2</th>\n",
       "      <th>essay3</th>\n",
       "      <th>essay4</th>\n",
       "      <th>essay5</th>\n",
       "      <th>essay6</th>\n",
       "      <th>essay7</th>\n",
       "      <th>essay8</th>\n",
       "      <th>essay9</th>\n",
       "      <th>ethnicity</th>\n",
       "      <th>height</th>\n",
       "      <th>income</th>\n",
       "      <th>job</th>\n",
       "      <th>last_online</th>\n",
       "      <th>location</th>\n",
       "      <th>offspring</th>\n",
       "      <th>orientation</th>\n",
       "      <th>pets</th>\n",
       "      <th>religion</th>\n",
       "      <th>sex</th>\n",
       "      <th>sign</th>\n",
       "      <th>smokes</th>\n",
       "      <th>speaks</th>\n",
       "      <th>status</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>46766</td>\n",
       "      <td>42</td>\n",
       "      <td>fit</td>\n",
       "      <td>NaN</td>\n",
       "      <td>socially</td>\n",
       "      <td>NaN</td>\n",
       "      <td>graduated from college/university</td>\n",
       "      <td>what am i to me? i am a mother, artist, friend...</td>\n",
       "      <td>currently, i work in the healthcare field, go ...</td>\n",
       "      <td>making people laugh and mixing colors!</td>\n",
       "      <td>NaN</td>\n",
       "      <td>if i look at my bookcase, i have a lot of trea...</td>\n",
       "      <td>my son&lt;br /&gt;\\nthe ocean&lt;br /&gt;\\nbooks&lt;br /&gt;\\nsk...</td>\n",
       "      <td>how to create a story, how to draw a character...</td>\n",
       "      <td>~debating whether i want to drive to the city ...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>you are curious about me, are creative, want t...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>64</td>\n",
       "      <td>-1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2012-07-01-00-08</td>\n",
       "      <td>san francisco, california</td>\n",
       "      <td>has a kid, but doesn&amp;rsquo;t want more</td>\n",
       "      <td>straight</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>f</td>\n",
       "      <td>cancer and it&amp;rsquo;s fun to think about</td>\n",
       "      <td>NaN</td>\n",
       "      <td>english</td>\n",
       "      <td>single</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>52438</td>\n",
       "      <td>19</td>\n",
       "      <td>athletic</td>\n",
       "      <td>anything</td>\n",
       "      <td>rarely</td>\n",
       "      <td>never</td>\n",
       "      <td>working on college/university</td>\n",
       "      <td>outgoing, love sports, and full time student.</td>\n",
       "      <td>student, aiming for a biology major.</td>\n",
       "      <td>running, soccer, talking, joking around, liste...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>the girl, sister carrie, always running, red h...</td>\n",
       "      <td>1. chips&lt;br /&gt;\\n2. phone&lt;br /&gt;\\n3. sports&lt;br /...</td>\n",
       "      <td>my future</td>\n",
       "      <td>relieved from school, playing soccer till nigh...</td>\n",
       "      <td>ask me</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>68</td>\n",
       "      <td>-1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2012-05-29-17-42</td>\n",
       "      <td>san francisco, california</td>\n",
       "      <td>NaN</td>\n",
       "      <td>straight</td>\n",
       "      <td>NaN</td>\n",
       "      <td>catholicism</td>\n",
       "      <td>m</td>\n",
       "      <td>NaN</td>\n",
       "      <td>no</td>\n",
       "      <td>english, spanish</td>\n",
       "      <td>single</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>26151</td>\n",
       "      <td>43</td>\n",
       "      <td>average</td>\n",
       "      <td>NaN</td>\n",
       "      <td>socially</td>\n",
       "      <td>never</td>\n",
       "      <td>NaN</td>\n",
       "      <td>it's hard to know exactly what to write, but h...</td>\n",
       "      <td>working (i love my job), spending time with fa...</td>\n",
       "      <td>being honest, listening (for some reason compl...</td>\n",
       "      <td>my sarcastic wit, red hair and green eyes.</td>\n",
       "      <td>books: i love to read horror, thrillers, sci-f...</td>\n",
       "      <td>books, my mp3 player, gps (i have an amazing a...</td>\n",
       "      <td>if i could have one super power i would want t...</td>\n",
       "      <td>hanging out with friends, watching scary movie...</td>\n",
       "      <td>i'm not.</td>\n",
       "      <td>you have a sense of humor and can just as easi...</td>\n",
       "      <td>white</td>\n",
       "      <td>64</td>\n",
       "      <td>-1</td>\n",
       "      <td>other</td>\n",
       "      <td>2012-06-10-14-37</td>\n",
       "      <td>pinole, california</td>\n",
       "      <td>has a kid</td>\n",
       "      <td>straight</td>\n",
       "      <td>NaN</td>\n",
       "      <td>christianity</td>\n",
       "      <td>f</td>\n",
       "      <td>NaN</td>\n",
       "      <td>trying to quit</td>\n",
       "      <td>english</td>\n",
       "      <td>single</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>7733</td>\n",
       "      <td>29</td>\n",
       "      <td>fit</td>\n",
       "      <td>anything</td>\n",
       "      <td>socially</td>\n",
       "      <td>never</td>\n",
       "      <td>graduated from college/university</td>\n",
       "      <td>still single, still picky&lt;br /&gt;\\n&lt;br /&gt;\\n&lt;stro...</td>\n",
       "      <td>i strive to be some sort of errant intellectua...</td>\n",
       "      <td>finding the humor in anything.&lt;br /&gt;\\napplying...</td>\n",
       "      <td>my irreverence with respect to long held socia...</td>\n",
       "      <td>books: &lt;i&gt;a short history of nearly everything...</td>\n",
       "      <td>1. a breathable mixture of oxygen and nitrogen...</td>\n",
       "      <td>both the distant future of humanity and the ve...</td>\n",
       "      <td>developing talent in tranquility or character ...</td>\n",
       "      <td>i snowboard goofy and can't whistle</td>\n",
       "      <td>you are still reading this and have somehow re...</td>\n",
       "      <td>white</td>\n",
       "      <td>74</td>\n",
       "      <td>-1</td>\n",
       "      <td>science / tech / engineering</td>\n",
       "      <td>2012-06-29-01-09</td>\n",
       "      <td>san francisco, california</td>\n",
       "      <td>doesn&amp;rsquo;t have kids, but wants them</td>\n",
       "      <td>straight</td>\n",
       "      <td>likes dogs and likes cats</td>\n",
       "      <td>agnosticism and laughing about it</td>\n",
       "      <td>m</td>\n",
       "      <td>pisces and it&amp;rsquo;s fun to think about</td>\n",
       "      <td>no</td>\n",
       "      <td>english (fluently), spanish (poorly), c++ (flu...</td>\n",
       "      <td>single</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>52145</td>\n",
       "      <td>26</td>\n",
       "      <td>NaN</td>\n",
       "      <td>mostly vegetarian</td>\n",
       "      <td>socially</td>\n",
       "      <td>NaN</td>\n",
       "      <td>graduated from masters program</td>\n",
       "      <td>i am moving to san francisco in a few days and...</td>\n",
       "      <td>i just became the program director for a non-p...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>some of my favorite books in the last year hav...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>you think you can make me laugh :)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>64</td>\n",
       "      <td>-1</td>\n",
       "      <td>education / academia</td>\n",
       "      <td>2012-06-26-23-30</td>\n",
       "      <td>san francisco, california</td>\n",
       "      <td>NaN</td>\n",
       "      <td>straight</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>f</td>\n",
       "      <td>sagittarius</td>\n",
       "      <td>no</td>\n",
       "      <td>english, spanish (okay)</td>\n",
       "      <td>single</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  age body_type               diet    drinks  drugs  \\\n",
       "0       46766   42       fit                NaN  socially    NaN   \n",
       "1       52438   19  athletic           anything    rarely  never   \n",
       "2       26151   43   average                NaN  socially  never   \n",
       "3        7733   29       fit           anything  socially  never   \n",
       "4       52145   26       NaN  mostly vegetarian  socially    NaN   \n",
       "\n",
       "                           education  \\\n",
       "0  graduated from college/university   \n",
       "1      working on college/university   \n",
       "2                                NaN   \n",
       "3  graduated from college/university   \n",
       "4     graduated from masters program   \n",
       "\n",
       "                                              essay0  \\\n",
       "0  what am i to me? i am a mother, artist, friend...   \n",
       "1      outgoing, love sports, and full time student.   \n",
       "2  it's hard to know exactly what to write, but h...   \n",
       "3  still single, still picky<br />\\n<br />\\n<stro...   \n",
       "4  i am moving to san francisco in a few days and...   \n",
       "\n",
       "                                              essay1  \\\n",
       "0  currently, i work in the healthcare field, go ...   \n",
       "1               student, aiming for a biology major.   \n",
       "2  working (i love my job), spending time with fa...   \n",
       "3  i strive to be some sort of errant intellectua...   \n",
       "4  i just became the program director for a non-p...   \n",
       "\n",
       "                                              essay2  \\\n",
       "0             making people laugh and mixing colors!   \n",
       "1  running, soccer, talking, joking around, liste...   \n",
       "2  being honest, listening (for some reason compl...   \n",
       "3  finding the humor in anything.<br />\\napplying...   \n",
       "4                                                NaN   \n",
       "\n",
       "                                              essay3  \\\n",
       "0                                                NaN   \n",
       "1                                                NaN   \n",
       "2         my sarcastic wit, red hair and green eyes.   \n",
       "3  my irreverence with respect to long held socia...   \n",
       "4                                                NaN   \n",
       "\n",
       "                                              essay4  \\\n",
       "0  if i look at my bookcase, i have a lot of trea...   \n",
       "1  the girl, sister carrie, always running, red h...   \n",
       "2  books: i love to read horror, thrillers, sci-f...   \n",
       "3  books: <i>a short history of nearly everything...   \n",
       "4  some of my favorite books in the last year hav...   \n",
       "\n",
       "                                              essay5  \\\n",
       "0  my son<br />\\nthe ocean<br />\\nbooks<br />\\nsk...   \n",
       "1  1. chips<br />\\n2. phone<br />\\n3. sports<br /...   \n",
       "2  books, my mp3 player, gps (i have an amazing a...   \n",
       "3  1. a breathable mixture of oxygen and nitrogen...   \n",
       "4                                                NaN   \n",
       "\n",
       "                                              essay6  \\\n",
       "0  how to create a story, how to draw a character...   \n",
       "1                                          my future   \n",
       "2  if i could have one super power i would want t...   \n",
       "3  both the distant future of humanity and the ve...   \n",
       "4                                                NaN   \n",
       "\n",
       "                                              essay7  \\\n",
       "0  ~debating whether i want to drive to the city ...   \n",
       "1  relieved from school, playing soccer till nigh...   \n",
       "2  hanging out with friends, watching scary movie...   \n",
       "3  developing talent in tranquility or character ...   \n",
       "4                                                NaN   \n",
       "\n",
       "                                essay8  \\\n",
       "0                                  NaN   \n",
       "1                               ask me   \n",
       "2                             i'm not.   \n",
       "3  i snowboard goofy and can't whistle   \n",
       "4                                  NaN   \n",
       "\n",
       "                                              essay9 ethnicity  height  \\\n",
       "0  you are curious about me, are creative, want t...       NaN      64   \n",
       "1                                                NaN       NaN      68   \n",
       "2  you have a sense of humor and can just as easi...     white      64   \n",
       "3  you are still reading this and have somehow re...     white      74   \n",
       "4                 you think you can make me laugh :)       NaN      64   \n",
       "\n",
       "   income                           job       last_online  \\\n",
       "0      -1                           NaN  2012-07-01-00-08   \n",
       "1      -1                           NaN  2012-05-29-17-42   \n",
       "2      -1                         other  2012-06-10-14-37   \n",
       "3      -1  science / tech / engineering  2012-06-29-01-09   \n",
       "4      -1          education / academia  2012-06-26-23-30   \n",
       "\n",
       "                    location                                offspring  \\\n",
       "0  san francisco, california   has a kid, but doesn&rsquo;t want more   \n",
       "1  san francisco, california                                      NaN   \n",
       "2         pinole, california                                has a kid   \n",
       "3  san francisco, california  doesn&rsquo;t have kids, but wants them   \n",
       "4  san francisco, california                                      NaN   \n",
       "\n",
       "  orientation                       pets                           religion  \\\n",
       "0    straight                        NaN                                NaN   \n",
       "1    straight                        NaN                        catholicism   \n",
       "2    straight                        NaN                       christianity   \n",
       "3    straight  likes dogs and likes cats  agnosticism and laughing about it   \n",
       "4    straight                        NaN                                NaN   \n",
       "\n",
       "  sex                                      sign          smokes  \\\n",
       "0   f  cancer and it&rsquo;s fun to think about             NaN   \n",
       "1   m                                       NaN              no   \n",
       "2   f                                       NaN  trying to quit   \n",
       "3   m  pisces and it&rsquo;s fun to think about              no   \n",
       "4   f                               sagittarius              no   \n",
       "\n",
       "                                              speaks  status  \n",
       "0                                            english  single  \n",
       "1                                   english, spanish  single  \n",
       "2                                            english  single  \n",
       "3  english (fluently), spanish (poorly), c++ (flu...  single  \n",
       "4                            english, spanish (okay)  single  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#path_to_repo = \"\"\n",
    "data = pd.read_csv(\"profiles.csv\")\n",
    "data.head()\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Your code here!\n",
    "y = np.array([1 if x==\"m\" else 0 for x in data.sex.tolist()])\n",
    "X = data[['height']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn import svm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.83399999999999996"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model = svm.SVC(kernel=\"rbf\")\n",
    "model.fit(X,y)\n",
    "model.score(X,y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.cross_validation import cross_val_score\n",
    "from sklearn.cross_validation import KFold\n",
    "from sklearn.cross_validation import StratifiedShuffleSplit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.83299999999999996"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.mean(cross_val_score(model,X,y,cv=KFold(len(X),n_folds=10,shuffle=True)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn import grid_search"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "C_range = np.logspace(-3,10,5)\n",
    "gamma_range = np.logspace(-3,10,5)\n",
    "param_grid = dict(C=C_range,gamma=gamma_range)\n",
    "\n",
    "cv = StratifiedShuffleSplit(y, n_iter=5, test_size=0.2)\n",
    "gridmodel = grid_search.GridSearchCV(svm.SVC(),param_grid=param_grid,cv=cv)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=StratifiedShuffleSplit(labels=[0 1 ..., 0 1], n_iter=5, test_size=0.2, random_state=None),\n",
       "       error_score='raise',\n",
       "       estimator=SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0, degree=3, gamma=0.0,\n",
       "  kernel='rbf', max_iter=-1, probability=False, random_state=None,\n",
       "  shrinking=True, tol=0.001, verbose=False),\n",
       "       fit_params={}, iid=True, loss_func=None, n_jobs=1,\n",
       "       param_grid={'C': array([  1.00000e-03,   1.77828e+00,   3.16228e+03,   5.62341e+06,\n",
       "         1.00000e+10]), 'gamma': array([  1.00000e-03,   1.77828e+00,   3.16228e+03,   5.62341e+06,\n",
       "         1.00000e+10])},\n",
       "       pre_dispatch='2*n_jobs', refit=True, score_func=None, scoring=None,\n",
       "       verbose=0)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gridmodel.fit(X,y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.849\n",
      "{'C': 10000000000.0, 'gamma': 10000000000.0}\n"
     ]
    }
   ],
   "source": [
    "print gridmodel.best_score_\n",
    "print gridmodel.best_params_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}