course1/02_data_description_and_representation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "31ac894e",
   "metadata": {},
   "source": [
    "# Module 2 - describing and representing tabulated data \n",
    "-------------------------------------------------------------------------------------------------------\n",
    "\n",
    "## Table of Content <a id='toc'></a>\n",
    "\n",
    "\n",
    "\n",
    "1. **[Basic description - common summary statistics](#descr1)**\n",
    "   <br>\n",
    "\n",
    "2. **[Basic representation - plotting one column](#plt1)**  \n",
    "   [Interlude: multi-panel figures](#multi)  \n",
    "   [Exercise 1 - histograms](#ex1)  \n",
    "   <br>\n",
    "\n",
    "3. **[Accounting for categories in the data](#descr.3)**  \n",
    "   3.1 [The `.groupby()` method](#descr.3.1)  \n",
    "   3.2 [The `hue` argument and `sns.catplot()`](#descr.3.2)  \n",
    "   [Exercise 2 - representing categories](#ex2)  \n",
    "   <br>\n",
    "   \n",
    "4. **[Representing the relationship between 2 numerical variables](#plot2)**\n",
    "   <br>\n",
    "\n",
    "5. **[Writing plots to disk](#writing)**\n",
    "   <br>\n",
    "   \n",
    "6. **[Free form exercise](#exo)**\n",
    "   <br>\n",
    "\n",
    "**Supplementary Material:**  \n",
    "   * [Annex - seaborn themes + python graph gallery](#gallery)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dedb6823",
   "metadata": {},
   "source": [
    "<br>\n",
    "\n",
    "## Introduction\n",
    "\n",
    "Before we begin this chapter in earnest, let's import some libraries and read the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ba8fbd05",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "import numpy as np\n",
    "\n",
    "print(\"All modules loaded successfully!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a2a0a1bf",
   "metadata": {},
   "source": [
    "> Note: if you are missing some of the above modules, you should install them.\n",
    ">\n",
    ">    * Installation with **pip**: `pip install --user pandas matplotlib seaborn numpy` \n",
    ">    * Installation with **conda**: `conda install -c conda-forge pandas matplotlib seaborn numpy`\n",
    "\n",
    "<br>\n",
    "<br>\n",
    "\n",
    "This is a configuration to make the plots work better when presenting online... you do **not need to run this cell**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8de4a210",
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "plt.rcParams['figure.figsize'] = 10, 10\n",
    "plt.rc(\"font\", size=20)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c48cf705",
   "metadata": {},
   "source": [
    "<br>\n",
    "<br>\n",
    "<br>\n",
    "\n",
    "[Back to ToC](#toc)\n",
    "\n",
    "\n",
    "## 1. Basic description - common summary statistics <a id='descr1'></a>\n",
    "-----------------------------------------------------------------------------------------\n",
    "\n",
    "When doing exploratory analysis of a dataset, it is often useful get some basic statistics on a per-column basis (since rows will typically represent the samples and columns the explanatory variables).\n",
    "\n",
    "Pandas has a number of methods that can be applied both on an entire DataFrame or on individual columns (Series):\n",
    "* **`describe()`**: prints summary statistics for all **numeric columns** (count, mean, min, max, std, quantiles).\n",
    "* **`count()`**: returns the count of values (in each column, if applied to a DataFrame), that are not NaN\n",
    "  (i.e. missing values - Not a Number).\n",
    "* **`value_counts()`**: returns the count of each value. Useful for discrete variables (e.g. factors).\n",
    "* **`min()`**, **`max()`**, **`mean()`**, **`std()`**, **`quantile()`**, **`sum()`**: returns the min,\n",
    "  max, mean, etc.\n",
    "\n",
    "When applied to a DataFrame, these method compute values for all columns, skipping columns for which they cannot be applied (e.g. standard deviation can only be computed for numeric columns).\n",
    "\n",
    "> Note: by default, all these statistics are computed by columns (e.g. max value of all rows in a given column), but they can also be computed by row by adding the `axis=1` argument. E.g. `df.max(axis=1)` returns the max value among columns for each row.\n",
    "\n",
    "<br>\n",
    "\n",
    "**Example:** in the Titanic dataset, count the number of people that embarked at different ports.  \n",
    "The Port of Embarkation information is given in the column `Embarked`: C = Cherbourg; Q = Queenstown; S = Southampton."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "176a60f0",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load the Titanic data set.\n",
    "df = pd.read_csv(\"data/titanic.csv\") \n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8f80e062",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.Embarked.value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e182433c",
   "metadata": {},
   "source": [
    "<br>\n",
    "\n",
    "**Example:** compute the mean, median, min and max passenger fare."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "97a25d1f",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"mean Fare\", df.Fare.mean())\n",
    "print(\"median Fare\", df.Fare.median())\n",
    "print(\"min Fare\", df.Fare.min())\n",
    "print(\"max Fare\", df.Fare.max())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eef0bde8",
   "metadata": {},
   "source": [
    "* The **`.describe()`** method is a shortcut to compute all of the above statistics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9c8a9979",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.Fare.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f566032e",
   "metadata": {},
   "source": [
    "<br>\n",
    "\n",
    "Of course we can combine this with selections:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bf54d888",
   "metadata": {},
   "outputs": [],
   "source": [
    "mask_male = df.Sex == \"male\"\n",
    "print(\"median fare of male passengers  :\", df.Fare[mask_male].median())\n",
    "print(\"median fare of female passengers:\", df.Fare[~mask_male].median())\n",
    "print(\"count of female passengers under 18:\", df.Age[~mask_male & (df.Age <= 18)].count())  # Does not include NaN."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aa0c87c9",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-warning\">\n",
    "\n",
    "**Pro-tips:** \n",
    "* Note the use of **`~`** to reverse the mask!\n",
    "* You can use the operators **&** (AND) and **|** (OR) to combine several conditions.\n",
    "* **Important:** when combining several conditions, do not forget the parenthesis!\n",
    "  * `(df.Sex == \"male\") & (df.Age <= 18)` is correct.\n",
    "  * `df.Sex == \"male\" & df.Age <= 18` will raise an error: the parenthesis are missing.\n",
    "\n",
    "<div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6bd7d355",
   "metadata": {},
   "source": [
    "<br>\n",
    "\n",
    "These methods can be used on **whole DataFrames**: by default they apply to each column (`axis=0`).  \n",
    "\n",
    "**Example:**\n",
    "* Let's compute the mean values for each column of the Titanic data set.  \n",
    "  Note: one could use `df.mean(axis=1)` to compute a mean per row, but it would here be senseless."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a53e1a7d",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.mean()  # Raises a TypeError or a FutureWarning, depending on the version of pandas."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "360cdaaa",
   "metadata": {},
   "source": [
    "Depending on the version of pandas, we get either a **`TypeError`** (newer versions) or a **`FutureWarning`** (older versions), letting us know that in the future automatic dropping of non-numerical columns will raise an error.  \n",
    "We can correct by explicitly selecting columns that contain numeric values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a339b7df",
   "metadata": {},
   "outputs": [],
   "source": [
    "df[[\"Age\", \"Pclass\", \"Family\", \"Fare\"]].mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2d1559c3",
   "metadata": {},
   "source": [
    "<br>\n",
    "\n",
    "<div class=\"alert alert-block alert-success\">\n",
    "\n",
    "### Micro-Exercise 1\n",
    "    \n",
    "* In the Titanic dataset, compute the mean fare for each passenger class (`Pclass`).\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8a66da5e",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "457d94a3",
   "metadata": {},
   "source": [
    "<br>\n",
    "\n",
    "Another useful method is **`pd.crosstab()`** to create contingency tables (multivariate frequency distribution of variables). Such tables allow e.g. to look at the relationship between categorical variables:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "17d07d13",
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.crosstab(df.Pclass, df.Survived)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "483f71ee",
   "metadata": {},
   "source": [
    "<br>\n",
    "\n",
    "<div class=\"alert alert-block alert-success\">\n",
    "\n",
    "### Micro-Exercise 2\n",
    "    \n",
    "* Using the dataset of single cell data `data/pbmc_data.countMatrix.50.txt.zip`:\n",
    "  1. Compute the sum for each column.\n",
    "  2. Normalize each column by dividing its values by the column-wise sum.\n",
    "    \n",
    "You can load the dataset with:\n",
    "    \n",
    "```py\n",
    "df_cell = pd.read_table(\"data/pbmc_data.countMatrix.50.txt.zip\", sep=\" \", index_col=0)\n",
    "```\n",
    "  \n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d3c7718e",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "5fc3823b",
   "metadata": {},
   "source": [
    "<br>\n",
    "<br>\n",
    "\n",
    "A very useful method of DataFrame to get an **overview of a dataset** is **`df.describe()`**:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "feeaea09",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "29884a43",
   "metadata": {},
   "source": [
    "* The `df.describe()` method gives information about all numerical columns in the dataset at once\n",
    "  (note that by default, non-numerical columns are absent).\n",
    "* It is very useful not only to get a first impression on the dataset, but also to catch eventual errors\n",
    "  in the data: a negative number where there should be only positive values, missing values (NAs), ...\n",
    "\n",
    "\n",
    "> *Note:* by default, `.describe()` only gives a summary of **numeric columns**.\n",
    "> To include all columns in the summary table:\n",
    ">\n",
    "> ```py\n",
    "> df.describe(include=\"all\")\n",
    "> ```\n",
    "\n",
    "\n",
    "<br>\n",
    "\n",
    "**BUT** the most eagle-eyed among you will have spotted a problem here... what is it? how to solve it ?\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "200c94e1",
   "metadata": {},
   "outputs": [],
   "source": [
    "# %load solutions/solution_describe.py"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d06c82dc",
   "metadata": {},
   "source": [
    "<br>\n",
    "\n",
    "Anyway, back to the matter at hand.\n",
    "\n",
    "**`.describe()`** gives access to some of the most commonly used summary statistics:\n",
    "* (arithmetic) **mean**: ${\\displaystyle \\bar{x}={\\frac {1}{n}}\\sum _{i=1}^{n}x_{i}}$  or,\n",
    "  for coders: `sum(x) / len(x)`\n",
    "* **standard deviation** (a.k.a. std, stdev): this corresponds to the average of the absolute difference to the mean. It is the **square root of the variance**.\n",
    "* **minimum** and **maximum**: smallest and biggest value among the data. Looking at them can help detect outliers.\n",
    "* **quartiles**: they correspond to the value such that\n",
    "    * 25% (first quartile, Q1), \n",
    "    * 50% (second quartile, Q2, median), or\n",
    "    * 75% (second quartile, Q3)\n",
    "    \n",
    "  of the values are lower than them. They are less sensitive than the mean to outlier values.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4512c7d3",
   "metadata": {},
   "source": [
    "<br>\n",
    "<br>\n",
    "<br>\n",
    "\n",
    "[Back to ToC](#toc)\n",
    "\n",
    "## 2. Basic representation - plotting one column <a id='plt1'></a>\n",
    "-----------------------------------------------------------\n",
    "\n",
    "\n",
    "It is all nice to be able to display numerical summary numbers, but often data is best explored visually! To this end, there are a number of plotting libraries in python. Here we will mostly be using **matplotlib** and **seaborn**, which are two very popular plotting libraries in python.\n",
    "\n",
    "* **[matplotlib](https://matplotlib.org)** can be seen as a base library, which defines a lot of\n",
    "  low-level plotting functions.\n",
    "* **[seaborn](https://seaborn.pydata.org)** is built on top of matplotlib, and provides more high-level\n",
    "  functions. It's designed to interface very well with DataFrames.\n",
    "\n",
    "We will not delve in all the details of these two libraries, but will instead try to convey elements about their general philosophies and look at their most common functions/arguments. This should give you a head start to get the most out of their very well done tutorial and galleries:\n",
    " * [matplotlib](https://matplotlib.org) -\n",
    "   [tutorial](https://matplotlib.org/stable/tutorials/introductory/pyplot.html#sphx-glr-tutorials-introductory-pyplot-py) -\n",
    "   [gallery](https://matplotlib.org/stable/gallery/index.html)\n",
    " * [seaborn](https://seaborn.pydata.org) -\n",
    "   [tutorial](https://seaborn.pydata.org/tutorial.html) -\n",
    "   [gallery](https://seaborn.pydata.org/examples/index.html)\n",
    "\n",
    "<br>\n",
    "\n",
    "> Note: `pandas` also proposes plotting functions, which are basically also using `matplotlib`. \n",
    "\n",
    "<br>\n",
    "\n",
    "\n",
    "\n",
    "The simplest way to represent the distribution of a set of values using **seaborn** is to use its **`sns.displot()`** function (`displot` stands for \"distribution plot\").\n",
    "\n",
    "**Example:** let's plot the distribution of passengers fares from the Titanic dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d8d9a2d4",
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.__version__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a500bed8",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.isin([np.inf]).sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "607e5c3d",
   "metadata": {},
   "outputs": [],
   "source": [
    "import seaborn as sns\n",
    "\n",
    "sns.displot(df.Fare)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f98312f",
   "metadata": {},
   "source": [
    "Ok, that is nice, but tiny... Le'ts make it larger:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cdb1ecb0",
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.displot(df.Fare, height=7)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bfd4613a",
   "metadata": {},
   "source": [
    "Above, you should now see a histogram (in blue):\n",
    "* The *x-axis* corresponds to the fare paid by each passenger.\n",
    "* The *y-axis* corresponds to the **count**: the number of values falling in a given *bin*\n",
    "  (i.e. a bar of the histogram).\n",
    "\n",
    "By using the **`kind`** argument, one can change the type of plot:\n",
    "* `kind=\"hist\"`: histogram (this is the default).\n",
    "* `kind=\"kde\"`: density line (kernel density estimation).\n",
    "* `kind=\"ecdf\"`: cumulative distribution function.\n",
    "\n",
    "<br>\n",
    "\n",
    "**Example:**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e566d95b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Note: kde stands for 'kernel density estimation', which is the method used to compute the density line.\n",
    "sns.displot(df.Fare, kind=\"kde\", height=7)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a568d5c1",
   "metadata": {},
   "source": [
    "> *Note:* the y-axis is now in **density** units."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ce705713",
   "metadata": {},
   "source": [
    "Naturally, **`sns.displot()`** has many options allowing to combine plot types and pimp your plot.  \n",
    "For instance, using `kde=True` will superimpose a density line:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c7808c1b",
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.displot(df.Age, kde=True, color=\"teal\", height=7)\n",
    "\n",
    "# Note: some attributes of the KDE line can be changed via the `line_kws` argument, but not its color.\n",
    "sns.displot(df.Age, kde=True, color=\"darkorange\", height=7, line_kws={\"ls\": \"--\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "63f987c6",
   "metadata": {},
   "source": [
    "> *Note:* having both the histogram and the density line switches the y-axis from density to count,\n",
    "  which is sometimes more interpretable."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "70f0303e",
   "metadata": {},
   "source": [
    "\n",
    "`sns.displot()` is what we could call a **figure-level** function:\n",
    " * It has several *kind* of representation.\n",
    " * It does not play nice in **multi-panel** figures.\n",
    "\n",
    "<br>\n",
    "<br>\n",
    "\n",
    "\n",
    "[Back to ToC](#toc)\n",
    "\n",
    "### Interlude: multi-panel figures <a id='multi'></a>\n",
    "\n",
    "**Multi-panel figures** can be created using `matplotlib` command and concepts.  \n",
    "To make it simple, we call **`fig, axes = plt.subplots(nb_rows, nb_cols)`**, which creates:\n",
    " * **`fig`**: a `figure` object which controls the whole multi-panel figure\n",
    " * **`axes`**: a list containing individual `axe` objects though which we can plot on individual\n",
    "   panels of the figure.\n",
    "   \n",
    "   > **What is a matplotlib `axe`?**  \n",
    "   > Axes class represents one (sub-)plot in a figure. It contains the plotted data, axis ticks,\n",
    "     labels, title, legend, etc. Its methods are the main interface for manipulating the plot\n",
    "     in matplotlib. See [here](https://matplotlib.org/stable/api/axes_api.html) for details.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "34a73437",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "# Create a list of 100 evenly spaced points between 0 and 10.\n",
    "x = np.linspace(0, 10, 100) \n",
    "\n",
    "# Create a figure with multiple panels: 1 row, 2 columns.\n",
    "fig, axes = plt.subplots(1, 2, figsize=(10, 8))\n",
    "\n",
    "print(axes)  # axes is a list of two axes objects.\n",
    "\n",
    "# Plotting on the first axe => the left panel.\n",
    "axes[0].plot(x, np.sin(x), label=\"sin\")\n",
    "axes[0].set_title(\"sine plot\")\n",
    "axes[0].set_xlabel(\"value\")\n",
    "axes[0].set_ylabel(\"sine/cosine value\")\n",
    "axes[0].legend()\n",
    "\n",
    "# Plotting on the second axe => the right panel.\n",
    "axes[1].plot(x, np.cos(x), label=\"cos\", color=\"orange\")\n",
    "axes[1].set_title(\"cosine plot\")\n",
    "axes[1].set_xlabel(\"value\")\n",
    "axes[1].set_ylabel(\"sine/cosine value\")\n",
    "axes[1].legend()\n",
    "\n",
    "plt.tight_layout()  # This creates a more harmonious organization of the different panels.\n",
    "                    # E.g. it avoids that the labels of one plot overlap with the other plot.\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7cfc7dc0",
   "metadata": {},
   "source": [
    "<br>\n",
    "\n",
    "### Back to data representation ...\n",
    "\n",
    "The different values of the **`kind`** argument (`histogram`, `kde`, `ecdf`) of `sns.displot()` can all be called individually using their own functions:\n",
    "* **`histplot()`**: histogram.\n",
    "* **`kdeplot()`**: kernel density estimation line.\n",
    "* **`ecdfplot()`**: cumulative distribution function.\n",
    "\n",
    "In contrast to `sns.displot()` which is **figure-level**, these more-specialized functions are **axe-level**: they play nicely in a multiple panel context.  \n",
    "They interface with `matplotlib`'s axes using the **`ax`** argument.\n",
    "\n",
    "**Example:** plotting the passenger age and fare from the Titanic dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d6959f69",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Creating a figure with multiple panels: 1 row, 2 columns.\n",
    "fig, axes = plt.subplots(1, 2)\n",
    "\n",
    "# Plotting on the first axe == left panel.\n",
    "sns.histplot(df.Age, ax=axes[0])\n",
    "\n",
    "# Plotting on the second axe == right panel.\n",
    "sns.kdeplot(df.Fare, ax=axes[1])\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73998c1f",
   "metadata": {},
   "source": [
    "<br>\n",
    "\n",
    "In **histogram**, a different number of *bins* shows different aspects of the distribution of your data and so it is important to choose one that is appropriate to your sample size and data.  \n",
    "By default, seaborn automatically infers the number of *bins*. You may stick with the original or change it.\n",
    "\n",
    "<br>\n",
    "\n",
    "**Example:** plotting the same data with 4 different bin sizes.\n",
    "Some notes about this example:\n",
    "* In the example below, we are creating a figure with a 2 x 2 subplot organization.\n",
    "* `axes` is therefore a list of list of axes (a 2 x 2 matrix), to represent the 2-dimensional\n",
    "  nature of our panels.\n",
    "  ```\n",
    "  [[<axe>,<axe>],\n",
    "   [<axe>,<axe>]]\n",
    "   ```\n",
    "* We can specify where each plot goes on the figure with the ax argument.\n",
    "  * `ax=axes[0,0]` is the top left corner.\n",
    "  * `ax=axes[0,1]` is upper right.\n",
    "  * `ax=axes[1,0]` is lower left.\n",
    "  * etc..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a9d2e010",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's play with a different dataset for this example.\n",
    "df_census = pd.read_table(\"data/census1880_fractions.csv\", sep=\",\")\n",
    "\n",
    "# Set up a figure with multiple panels, here a 2 by 2 grid.\n",
    "f, axes = plt.subplots(2, 2, figsize=(14, 7))\n",
    "\n",
    "# Plot a simple histogram with bin size determined automatically.\n",
    "sns.histplot(\n",
    "    df_census[\"0-14 y.o.\"],\n",
    "    kde=False,\n",
    "    color=\"b\",\n",
    "    ax=axes[0,0]\n",
    ").set_title(\"automatic\")\n",
    "\n",
    "# Plot a simple histogram with a number of bins of 5, 10 and 1000.\n",
    "sns.histplot(df_census[\"0-14 y.o.\"], bins=5    , kde=False, color=\"b\", ax=axes[0,1]).set_title(\"5 bins\")\n",
    "sns.histplot(df_census[\"0-14 y.o.\"], bins=10   , kde=False, color=\"b\", ax=axes[1,0]).set_title(\"10 bins\")\n",
    "sns.histplot(df_census[\"0-14 y.o.\"], bins=1000 , kde=False, color=\"b\", ax=axes[1,1]).set_title(\"1000 bins\")\n",
    "\n",
    "plt.tight_layout()  # This makes the panels margins and setup more graceful."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2fb03cab",
   "metadata": {},
   "source": [
    "<br>\n",
    "\n",
    "To recap, **when to use `sns.histplot` or `sns.displot`**: \n",
    " * `histplot`: gives a lower-level kind of plot which is easier to manipulate in multi-panel figures.\n",
    " * `displot`: high-level function, with a lot of capabilities, but does not play nice if it is not the top figure.\n",
    "     "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "262124e1",
   "metadata": {},
   "source": [
    "\n",
    "<br>\n",
    "\n",
    "We can combine graphical elements coming from `matplotlib` and `seaborn` quite seamlessly.\n",
    "\n",
    "Here is a more advanced example where we represent the the **mean**, the **median** and the **mode** of a distribution on top of an histogram."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "acc451a7",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define a small function that plots a distribution, along with its mean, median, and mode.\n",
    "\n",
    "def plot_with_mean_median_mode(dat, ax):\n",
    "    \"\"\"Plots the specified data `dat` onto the specified axe `ax`.\n",
    "    Arguments:\n",
    "     * dat: a pandas series.\n",
    "     * ax : a matplotlib axe.\n",
    "    \"\"\"\n",
    "    mode = dat.mode()[0]  # We only select the first mode.\n",
    "    mean = dat.mean()\n",
    "    median = dat.median()\n",
    "\n",
    "    sns.histplot(dat, kde=True, ax=ax)  # Line for histogram and density line.\n",
    "    \n",
    "    # \".axvline()\" adds a vertical line to the plot, at the specified location.\n",
    "    ax.axvline(mean, color=\"r\", linestyle=\"--\", label=\"Mean\")\n",
    "    ax.axvline(median, color=\"g\", linestyle=\"-\", label=\"Median\")\n",
    "    ax.axvline(mode, color=\"b\", linestyle=\"-\", label=\"Mode\")\n",
    "    ax.legend(loc=\"upper right\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "53e83760",
   "metadata": {},
   "outputs": [],
   "source": [
    "f, axes = plt.subplots(2, 1, figsize=(15, 10))\n",
    "\n",
    "plot_with_mean_median_mode(df.Fare, axes[0])\n",
    "plot_with_mean_median_mode(df.Age , axes[1])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91db6b77",
   "metadata": {},
   "source": [
    "Of note here :\n",
    " * `axvline`: create a vertical line.\n",
    " * `label`: legend label associated with the element created by this function.\n",
    "   Common to a lot of matplotlib plotting functions.\n",
    " * `ax.legend()`: makes the legend appear.\n",
    " \n",
    "> **Customizing the legend position with the `loc` argument**\n",
    "> * The position of the legend can be manually set by passing the `loc` argument: `ax.legend(loc=\"upper right\")`.\n",
    "> * Possible values for `loc` are:\n",
    "    * `\"best\"` (the default), `\"upper left\"`, `\"upper right\"`, `\"lower left\"`, `\"lower right\"`,\n",
    "      `\"upper center\"`, `\"lower center\"`, `\"center left\"`, `\"center right\"`, `\"center\"`.\n",
    "    * A `(x,y)` tuple, giving the coordinates of the legend position.\n",
    "    * More options to manually specify elements in the `help()`.\n",
    "\n",
    "\n",
    "<br>\n",
    "<br>\n",
    "\n",
    "<div class=\"alert alert-block alert-success\">\n",
    "\n",
    "## Exercise 2.1 - histograms\n",
    "\n",
    "Exercises are located in the dedicated notebook `exercises_course1.ipynb`.\n",
    "\n",
    "</div>\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db2526e7",
   "metadata": {},
   "source": [
    "<br>\n",
    "<br>\n",
    "\n",
    "[Back to ToC](#toc)\n",
    "\n",
    "## 3. Accounting for categories in the data <a id='descr.3'></a>\n",
    "\n",
    "\n",
    "Here the categories would be the passenger's gender, but depending on data-sets they could be a genotype (Wild-Type versus KO), sex, experimental condition, ...\n",
    "\n",
    "Of course, we could manually compute the means and plot an histogram for each category in our data-set, but pandas and seaborn offer much more efficient routines for doing this.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2d8c5766",
   "metadata": {},
   "source": [
    "### 3.1 The `.groupby()` method <a id='descr.3.1'></a>\n",
    "\n",
    "Computing a summary statistics on a pandas DataFrame is done **using the `df.groupby(...)` method and applying some function** to the result of that grouping.\n",
    "\n",
    "**Example:** compute the median age per gender category. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4d66d97e",
   "metadata": {},
   "outputs": [],
   "source": [
    "grouped = df.groupby(\"Sex\")  # Grouping by gender.\n",
    "grouped[\"Age\"].median()      # Median age by gender."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "593f295a",
   "metadata": {},
   "source": [
    "<br>\n",
    "\n",
    "Of course here there are only two categories, but this gets very interesting when there is a lot of categories.  \n",
    "Remember our swiss census from 1880?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5ae31306",
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "df_census = pd.read_table(\"data/swiss_census_1880.csv\", sep=\",\")\n",
    "\n",
    "grouped = df_census.groupby(\"canton name\")\n",
    "grouped[\"Catholic\"].sum()                    # Number of catholics per canton in 1880."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c81e174",
   "metadata": {},
   "source": [
    "\n",
    "A common task is also to get one list per group, for instance when we want to do statistical testing.\n",
    "\n",
    "For this we combine the `groupby` with an `agg` (aggregation) to a `list`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d3f32acf",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "df_census.groupby(\"canton name\").Catholic.agg(list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b93c5171",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Example of usage when doing an ANOVA on the number of catholics between cantons\n",
    "from scipy import stats\n",
    "aggregatedCatholics = df_census.groupby(\"canton name\").Catholic.agg(list)\n",
    "\n",
    "Fstat, pval = stats.f_oneway(*aggregatedCatholics)\n",
    "# equivalent to stats.f_oneway( aggregated60fraction[0] , aggregated60fraction[1] , ... )\n",
    "\n",
    "print(\"1-way anova / F-test:\")\n",
    "print(\"F-stat :\",Fstat)\n",
    "print(\"p-value:\",pval)\n",
    "\n",
    "\n",
    "## otherwise we would have to write:\n",
    "# Fstat, pval = stats.f_oneway(df_census.Catholic[df_census.canton=='ZH'],\n",
    "#                              df_census.Catholic[df_census.canton=='LS'],\n",
    "#                              ...)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ac703ef",
   "metadata": {},
   "source": [
    "For convenience, the groupby Series can then be exported to a dictionary:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9f9227a2",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_census.groupby(\"canton name\").Catholic.agg(list).to_dict()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4917868e",
   "metadata": {},
   "source": [
    "<br>\n",
    "\n",
    "<div class=\"alert alert-block alert-success\">\n",
    " \n",
    "### Micro-Exercise 3\n",
    "    \n",
    "* Load the Titanic dataset with: `df = pd.read_csv(\"data/titanic.csv\")`.\n",
    "* Compute the mean fare (`Fare`) paid by passengers in each class (`Pclass`).\n",
    "* Compute the mean fare (`Fare`) for all combinations of passenger class (`Pclass`) and gender (`Sex`).\n",
    "\n",
    "<div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "05812c9c",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "34b61137",
   "metadata": {},
   "source": [
    "<br>\n",
    "\n",
    "[Back to ToC](#toc)\n",
    "\n",
    "### 3.2 The `hue` argument and `sns.catplot()` <a id='descr.3.2'></a>\n",
    "\n",
    "Computing these summary statistics is all good, but you can also visualize them using the **`seaborn`** argument **`hue`**:\n",
    "* **`hue`** is quite ubiquitous in seaborn functions, and lets you determine a **categorical column** by\n",
    "  which you want to split your representation.\n",
    "* Note how we switched our syntax too: we now use **column names**, and use the **`data=`** argument to\n",
    "  specify from which `DataFrame` the data comes.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3c7fe405",
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.displot(x=\"Age\", hue=\"Sex\", data=df, kind=\"kde\", height=7)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a6a070c2",
   "metadata": {},
   "source": [
    "<br>\n",
    "\n",
    "The default seaborn scheme is fairly nice, but maybe you want to change it:\n",
    "> Note: use the argument **`linewidth=`** to get wider/narrower lines.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f8f5d48d",
   "metadata": {},
   "outputs": [],
   "source": [
    "f, axes = plt.subplots(2, 2, figsize=(14, 7))\n",
    "\n",
    "# Default\n",
    "sns.kdeplot(x=\"Age\", hue=\"Sex\", linewidth=3, data=df , ax=axes[0][0])\n",
    "\n",
    "# Using an existing seaborn palette\n",
    "sns.set_palette(sns.color_palette(\"Set2\"))\n",
    "sns.kdeplot(x=\"Age\", hue=\"Sex\", data=df, linewidth=3, ax=axes[0][1])\n",
    "\n",
    "\n",
    "# Setting manually, with hex values\n",
    "sns.kdeplot(\n",
    "    x=\"Age\", hue=\"Sex\", data=df,\n",
    "    linewidth=3, ax=axes[1][0], \n",
    "    palette = [\"#FFCC04\",\"#1F8AB3\"]\n",
    ")\n",
    "\n",
    "# Setting semi-manually, using the xkcd palette https://xkcd.com/color/rgb\n",
    "sns.kdeplot(\n",
    "    x=\"Age\", hue=\"Sex\", data=df,\n",
    "    linewidth=3, ax=axes[1][1], \n",
    "    palette=[\"xkcd:lavender\", \"xkcd:dark mint\"]\n",
    ")\n",
    "\n",
    "plt.tight_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "863d763a",
   "metadata": {},
   "source": [
    "You can learn more on all the options there :\n",
    " * [matplotlib colors](https://matplotlib.org/stable/tutorials/colors/colors.html)\n",
    " * [seaborn colors](https://seaborn.pydata.org/tutorial/color_palettes.html)\n",
    " \n",
    "\n",
    "<br>\n",
    "\n",
    "Sometimes, the `displot` options don't really cut it, perhaps because you have many different categories, or maybe because you want to have more than one column to do the categorization.\n",
    "\n",
    "This is when **`sns.catplot()`**, another **figure-level** function with a lot a `kind`, saves the day!\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c682f9ab",
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.catplot(\n",
    "    x=\"Fare\", y=\"Pclass\",\n",
    "    orient=\"horizontal\",\n",
    "    data=df,\n",
    "    aspect=2.0, height=7\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b910242a",
   "metadata": {},
   "source": [
    "Like `sns.displot`, `sns.catplot` is a master function which is designed to create a full figure at once rather than be embedded in a larger figure. \n",
    "\n",
    "This is why its arguments are a bit different when it comes to setting the figure *height and width*, which is done using :\n",
    " * `height` : height of the figure (no trick there)\n",
    " * `aspect` : width/height ratio of the figure ( high aspect -> wider figure )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "46b308c9",
   "metadata": {},
   "source": [
    "The **`kind`** argument let's you control the overall look of the plot. I\n",
    "It can be:\n",
    "* 'strip' : this is the default, showing all data points. \n",
    "* **'box'** : the famous boxplot.\n",
    "* **'violin'** : an alternative to the boxplot using density lines.\n",
    "* **'bar'** : the (in)famous barplot.\n",
    "* 'swarm' : similar to 'strip' but with another way of arranging the points.\n",
    "* 'boxen' : some intermediary between a boxplot and a violin plot.\n",
    "* 'point' : alternative to the barplot where only the top point is shown.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9e0e0d16",
   "metadata": {},
   "outputs": [],
   "source": [
    "kinds = [\"box\", \"violin\", \"bar\", \"boxen\", \"strip\", \"point\"]  #,\"swarm\"] # swarm takes a long time to compute\n",
    "\n",
    "for i, k in enumerate(kinds):\n",
    "    sns.catplot(x=\"Fare\", y=\"Pclass\", orient=\"horizontal\", data=df, kind=k, aspect=5, height=2)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55f7c9c9",
   "metadata": {},
   "source": [
    "So here we can see:\n",
    " * **boxplot:** represent quartiles and a few other things.\n",
    " * **violinplot:** density line plot with an (optional) boxplot in the center.\n",
    " * **barplot:** just shows the mean + an error bar (by default 95% CI computed using bootstrapping).\n",
    " * **boxen:** the unholy child of a boxplot and a violinplot.\n",
    " * **strip:** the default, shows the points (super nice, except when you have a lot of points).\n",
    " * **point:** basically a barplot without bars and a line between categories.\n",
    "\n",
    "Given all this, one can reasonably ask the question : **which one is the *best* ?**\n",
    "\n",
    "While the answer is not clear about the best way to represent a distribution over one or multiple categories, we definitely have some [good answers regarding bad/misleading ways](https://stekhoven.shinyapps.io/barplotNonsense/) (hint, barplot does not shine).  \n",
    "\n",
    "<br>\n",
    "\n",
    "With `sns.catplot()`, the **`hue`** argument can also be used here to great effect:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4bfe4f95",
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.catplot(\n",
    "    x=\"Fare\", y=\"Pclass\",\n",
    "    hue=\"Sex\",\n",
    "    orient=\"horizontal\",\n",
    "    kind=\"box\",\n",
    "    data=df,\n",
    "    aspect=2.0, height=4\n",
    ")\n",
    "# It is also very nice with kind=\"point\"."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b1f986fa",
   "metadata": {},
   "source": [
    "<br>\n",
    "<br>\n",
    "\n",
    "<div class=\"alert alert-block alert-success\">\n",
    "\n",
    "## Exercise 2.2 - Representing categories\n",
    "\n",
    "Exercises are located in the dedicated notebook `exercises_course1.ipynb`.\n",
    "\n",
    "</div>\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ea7231f",
   "metadata": {},
   "source": [
    "<br>\n",
    "<br>\n",
    "<br>\n",
    "\n",
    "[Back to ToC](#toc)\n",
    "\n",
    "## 4. Representing the relationship between 2 numerical variables <a id=\"plot2\" ></a>\n",
    "-----------------------------------------------------------------------------------------------\n",
    "\n",
    "**`sns.scatterplot()`** lets us represent the relationship between two numerical variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2f75dcf0",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_table(\"data/titanic.csv\", sep=\",\")  # Just in case we need to re-read the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "008d3cfd",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Reset the seaborn default theme.\n",
    "sns.reset_orig()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a524f2ea",
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = plt.figure(figsize=(14, 6))\n",
    "ax = sns.scatterplot(x=\"Fare\", y=\"Age\", data=df)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "124c76d8",
   "metadata": {},
   "source": [
    "<br>\n",
    "\n",
    "OK, it is not so nice... let's set the x-axis to a log scale:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f370c609",
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = plt.figure(figsize=(14, 6))\n",
    "ax = sns.scatterplot(x=\"Fare\", y=\"Age\", data=df)\n",
    "ax.set(xscale=\"log\")      # Setting axis to log scale.\n",
    "ax.set_xlim((3, 1000))    # Manually setting the limit of the x axis.\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a4121e08",
   "metadata": {},
   "source": [
    "Of course, we can customize:\n",
    "> Note the argument **`style=`** used to associate a marker shape to a column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aa00594f",
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = plt.figure(figsize=(15, 10))\n",
    "ax = sns.scatterplot(\n",
    "    x=\"Fare\", y=\"Age\", \n",
    "    hue=\"Pclass\",\n",
    "    palette=[\"xkcd:tomato\", \"xkcd:teal\", \"xkcd:mustard\"],\n",
    "    style=\"Sex\",\n",
    "    data=df,\n",
    "    s=100\n",
    ")\n",
    "ax.set(xscale=\"log\")    # Setting axis to log scale.\n",
    "ax.set_xlim((3, 1000))  # Manually setting the limit of the x axis.\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f94a310",
   "metadata": {},
   "source": [
    "<br>\n",
    "\n",
    "An alternative with more colors and a single marker.\n",
    "This was adapted from [this stackoverflow question](https://stackoverflow.com/questions/62230507/multiple-columns-for-hue-parameter-in-seaborn-violinplot).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9ab57987",
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "fig = plt.figure(figsize=(15, 10))\n",
    "\n",
    "# We combine the 2 columns to later apply them as categories.\n",
    "CAT = df[[\"Pclass\", \"Sex\"]].apply(tuple, axis=1)\n",
    "\n",
    "ax = sns.scatterplot(\n",
    "    x=\"Fare\", y=\"Age\", \n",
    "    hue=CAT,\n",
    "    hue_order=sorted(CAT.unique()),           # Handle the order categories in legend.\n",
    "    palette=sns.color_palette(\"Paired\")[:6] , # Here the paired palette is great.\n",
    "    data=df, s=100\n",
    ")\n",
    "ax.set(xscale=\"log\")     # Setting axis to log scale.\n",
    "ax.set_xlim((3, 1000))   # Manually setting the limit of the x axis.\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e664e2d3",
   "metadata": {},
   "source": [
    "Sometimes a 2D density plot may give you a higher level view of your data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9366e3c4",
   "metadata": {},
   "outputs": [],
   "source": [
    "iris = sns.load_dataset(\"iris\")\n",
    "# Draw a contour plot to represent each bi-variate density\n",
    "sns.kdeplot(\n",
    "    data=iris,\n",
    "    x=\"sepal_width\",\n",
    "    y=\"sepal_length\",\n",
    "    hue=\"species\",\n",
    "    thresh=.1,\n",
    ")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "295e242b",
   "metadata": {},
   "source": [
    "In other cases you may prefer to represent lines:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cd8f3004",
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# Load an example dataset with long-form data\n",
    "fmri = sns.load_dataset(\"fmri\")\n",
    "\n",
    "# Plot the responses for different events and regions\n",
    "#   with 95% confidence interval, \n",
    "#    other options include (\"ci\", \"pi\", \"se\", or \"sd\") or something custom\n",
    "sns.lineplot(x=\"timepoint\", y=\"signal\",\n",
    "             hue=\"region\", style=\"event\",\n",
    "             errorbar = ('ci',95) , \n",
    "             data=fmri)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be1594d2",
   "metadata": {},
   "source": [
    "Finally, a little data exploration trick : **`sns.pairplot()`**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "428ca796",
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.pairplot(df_census.iloc[:, 3:10])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "759e97fa",
   "metadata": {},
   "source": [
    "<br>\n",
    "<br>\n",
    "<br>\n",
    "\n",
    "[Back to ToC](#toc)\n",
    "\n",
    "\n",
    "## 5. Writing plots to disk <a id='writing'></a>\n",
    "------------------------------------\n",
    "\n",
    "To save figures to disk, we rely on the **`savefig()`** method:\n",
    "* Different output formats can be specified by changing the output file name extension, \n",
    "  e.g. `.pdf`, `.svg`, `.jpg`, ...\n",
    "* For raster formats, the `dpi` argument can be used to define the image quality.\n",
    "* See `help(plt.savefig)` or [here](https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.savefig.html?highlight=savefig#matplotlib.pyplot.savefig) for more info."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "daf0d43c",
   "metadata": {},
   "outputs": [],
   "source": [
    "k = \"box\"\n",
    "\n",
    "# Make you plot as usual, redirecting it to a variable (my_plot).\n",
    "represented_variable, category = \"Fare\", \"Pclass\"\n",
    "my_plot = sns.catplot(\n",
    "    x=represented_variable,\n",
    "    y=category,\n",
    "    data=df,\n",
    "    kind=k, orient=\"h\",\n",
    "    height=10, aspect=2\n",
    ")\n",
    "\n",
    "# Save plot to disk, using the savefig() method:\n",
    "my_plot.savefig(\"output.png\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e452c0e2",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "da081fad",
   "metadata": {},
   "outputs": [],
   "source": [
    "# For multi-panel figures:\n",
    "f, axes = plt.subplots(2, 2, figsize=(7, 7))\n",
    "\n",
    "sns.histplot(df.Fare, kde=False, color=\"b\", ax=axes[0, 0]).set_title(\"automatic\")\n",
    "sns.histplot(df.Fare, bins=5   , kde=False, color=\"b\", ax=axes[0, 1]).set_title(\"5 bins\")\n",
    "sns.histplot(df.Fare, bins=10  , kde=False, color=\"b\", ax=axes[1, 0]).set_title(\"10 bins\")\n",
    "sns.histplot(df.Fare, bins=1000 , kde=False, color=\"b\", ax=axes[1, 1]).set_title(\"1000 bins\")\n",
    "plt.tight_layout()  # This makes the panels margins and setup more graceful.\n",
    "\n",
    "f.savefig(\"output_multipanel.pdf\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "28dd5f97",
   "metadata": {},
   "source": [
    "<br>\n",
    "<br>\n",
    "<br>\n",
    "\n",
    "[Back to ToC](#toc)\n",
    "\n",
    "<div class=\"alert alert-block alert-success\">\n",
    "\n",
    "## Exercise 2.3 - Free form exercise\n",
    "\n",
    "Exercises are located in the dedicated notebook `exercises_course1.ipynb`.\n",
    "\n",
    "</div>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f56aa895",
   "metadata": {},
   "source": [
    "<br>\n",
    "<br>\n",
    "<br>\n",
    "\n",
    "# Additional Material\n",
    "------------------------------\n",
    "\n",
    "## Annex 1 - seaborn themes + python graph gallery <a id=\"gallery\"></a>\n",
    "\n",
    "Seaborn and matplotlib offer nigh-endless possibilities when it comes to customizing graph elements.\n",
    "* A simple way to change the general feeling of a figure is to play with\n",
    "  [seaborn themes](https://seaborn.pydata.org/tutorial/aesthetics.html), or\n",
    "  [matplotlib styles](https://www.dunderdata.com/blog/view-all-available-matplotlib-styles).\n",
    "\n",
    "* In general, when composing a figure we often start from an existing examples.\n",
    "  Here are useful resources to do just that:\n",
    "  * [matplotlib gallery](https://matplotlib.org/stable/gallery/index.html)\n",
    "  * [seaborn gallery](https://seaborn.pydata.org/examples/index.html)\n",
    "  * [python-graph-gallery](https://www.python-graph-gallery.com/)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}