example_visualization_SFBA_medium_frequency_power_drain.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook demonstrates the result of the first round of data collection, collected in the San Francisco Bay Area by @shankari. The round had several shortcomings, some of which were addressed during the data collection and some of which were fixed before starting the second round of data collection."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import all the dependencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# for reading and validating data\n",
    "import emeval.input.spec_details as eisd\n",
    "import emeval.input.phone_view as eipv\n",
    "import emeval.input.eval_view as eiev"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# Visualization helpers\n",
    "import emeval.viz.phone_view as ezpv\n",
    "import emeval.viz.eval_view as ezev"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# For plots\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib notebook"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# For maps\n",
    "import branca.element as bre"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# For displaying dates\n",
    "import arrow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load and validate data\n",
    "\n",
    "The first issue to note is that we actually have two specs here. The first spec is the checked in `evaluation.spec.sample`, which defines calibration for both stationary and moving instances, and some evaluation trips. However, while starting with the calibration, we noticed some inconsistencies between the power curves. So in order to be more consistent, I defined a second, calibration-only spec `examples/calibration.only.json`, which essentially repeats the calibration experiments multiple times.\n",
    "\n",
    "After that, I returned to the first set of experiments for the moving calibration and the evaluation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "DATASTORE_URL = \"http://cardshark.cs.berkeley.edu\"\n",
    "AUTHOR_EMAIL = \"shankari@eecs.berkeley.edu\"\n",
    "sdmco1 = eisd.SpecDetails(DATASTORE_URL, AUTHOR_EMAIL, \"sfba_med_freq_calibration_only\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import importlib\n",
    "importlib.reload(ezev)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "pvmco1 = eipv.PhoneView(sdmco1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Issue #1: Multiple and missing transitions\n",
    "\n",
    "While exploring the data after the collection was done, there were still inconsistencies in the transitions pushed to the server - there were a bunch of duplicate transitions, and two of the phones were missing start transitions for the second trip."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# Commented out because this fails\n",
    "# pvt3.validate()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "pvmco1.validate()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "evmco1 = eiev.EvaluationView()\n",
    "evmco1.from_view_multiple_runs(pvmco1, \"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Now for the results (calibration, phone view)!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Battery drain over time (moving calibration)\n",
    "\n",
    "Again, the moving calibration runs were not very useful in terms of battery drain, since there were too few points to be useful. We actually have more points on android, but we have almost nothing for the iOS medium accuracy runs.\n",
    "\n",
    "Part of this is inherent in the definition of moving calibration, since it is unlikely that we will move for 10-15 hours at a time to collect the kind of data we have in the stationary case. And if our trip lasts for an hour, but we only read the battery level once an hour, we will end up with close to no data.\n",
    "\n",
    "But with some native code changes, I think we can do better wrt at least recording the battery reading at the trip start and end."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "(ifig, [android_ax, ios_ax]) = plt.subplots(ncols=1, nrows=2, figsize=(12,12))\n",
    "\n",
    "ezpv.plot_all_power_drain(ios_ax, pvmco1.map()[\"ios\"], \"calibration\", \"high\")\n",
    "ios_ax.legend(loc=\"center left\", bbox_to_anchor=(1, 0.5))\n",
    "ezpv.plot_all_power_drain(android_ax, pvmco1.map()[\"android\"], \"calibration\", \"high\")\n",
    "android_ax.legend(loc=\"center left\", bbox_to_anchor=(1, 0.5))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(ifig, [android_ax, ios_ax]) = plt.subplots(ncols=1, nrows=2, figsize=(12,12))\n",
    "\n",
    "ezpv.plot_all_power_drain(ios_ax, pvmco1.map()[\"ios\"], \"calibration\", \"medium\")\n",
    "ios_ax.legend(loc=\"center left\", bbox_to_anchor=(1, 0.5))\n",
    "ezpv.plot_all_power_drain(android_ax, pvmco1.map()[\"android\"], \"calibration\", \"medium\")\n",
    "android_ax.legend(loc=\"center left\", bbox_to_anchor=(1, 0.5))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(ifig, axes) = plt.subplots(ncols=0, nrows=0, figsize=(12,3))\n",
    "\n",
    "ezpv.plot_separate_power_drain(ifig, pvmco1.map()[\"ios\"], 4, \"calibration\", \"stationary\")\n",
    "# android_ax.legend(loc=\"center left\", bbox_to_anchor=(1, 0.5))\n",
    "(ifig, axes) = plt.subplots(ncols=0, nrows=0, figsize=(12,3))\n",
    "\n",
    "ezpv.plot_separate_power_drain(ifig, pvmco1.map()[\"android\"], 4, \"calibration\", \"stationary\")\n",
    "# android_ax.legend(loc=\"center left\", bbox_to_anchor=(1, 0.5))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(ifig, axes) = plt.subplots(ncols=0, nrows=0, figsize=(16,4))\n",
    "\n",
    "ezev.plot_separate_power_drain_multiple_runs(ifig, 2, evmco1.map(\"calibration\")[\"ios\"], \"\")\n",
    "android_ax.legend(loc=\"upper center\", bbox_to_anchor=(1, 0.5))\n",
    "(ifig, axes) = plt.subplots(ncols=0, nrows=0, figsize=(16,4))\n",
    "\n",
    "ezev.plot_separate_power_drain_multiple_runs(ifig, 2, evmco1.map(\"calibration\")[\"android\"], \"\")\n",
    "android_ax.legend(loc=\"upper center\", bbox_to_anchor=(1, 0.5))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "importlib.reload(ezev)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def display_transitions_ranges_values(ax, phone_map, range_key, trip_id_pattern, offset_hr=1):\n",
    "    for phone_label, phone_details in phone_map.items():\n",
    "        curr_calibration_ranges = phone_details[\"{}_ranges\".format(range_key)]\n",
    "        sel_calibration_ranges = [cr for cr in curr_calibration_ranges if trip_id_pattern in cr[\"trip_id\"]]\n",
    "        for r in sel_calibration_ranges:\n",
    "            battery_df = r[\"battery_df\"]\n",
    "            start_level = battery_df.battery_level_pct.iloc[0]\n",
    "            battery_df[\"battery_change_from_start\"] = battery_df.battery_level_pct - start_level\n",
    "            battery_df[\"hr_new\"] = battery_df.hr + offset_hr\n",
    "            ret_axes = battery_df.plot(x=\"hr_new\", y=\"battery_change_from_start\", ax=ax, label=phone_label+\"_\"+r[\"trip_id\"], ylim=(0, -100), sharey=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(ifig, [android_ax, ios_ax]) = plt.subplots(ncols=1, nrows=2, figsize=(6,12))\n",
    "\n",
    "display_transitions_ranges_values(ios_ax, pvmco1.map()[\"ios\"], \"calibration\", \"high_accuracy_stationary_0\", 0)\n",
    "display_transitions_ranges_values(ios_ax, pvmco1.map()[\"ios\"], \"calibration\", \"high_accuracy_stationary_1\", 0)\n",
    "display_transitions_ranges_values(ios_ax, pvmco1.map()[\"ios\"], \"calibration\", \"high_accuracy_stationary_2\", 0)\n",
    "# display_transitions_ranges_values(ios_ax, pvmco1.map()[\"ios\"], \"calibration\", \"medium_accuracy_stationary_0\", 0)\n",
    "# display_transitions_ranges_values(ios_ax, pvmco1.map()[\"ios\"], \"calibration\", \"medium_accuracy_stationary_1\", 0)\n",
    "# display_transitions_ranges_values(ios_ax, pvmco1.map()[\"ios\"], \"calibration\", \"medium_accuracy_stationary_2\", 0)\n",
    "ios_ax.legend(loc=\"center left\", bbox_to_anchor=(1, 0.5))\n",
    "display_transitions_ranges_values(android_ax, pvmco1.map()[\"android\"], \"calibration\", \"stationary\", 0)\n",
    "android_ax.legend(loc=\"center left\", bbox_to_anchor=(1, 0.5))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}