Reproducible research.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "# Reproducible Research\n",
    "Roberto Toro, Valparaíso 2016"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Workshop 3: Reproducibility, data sharing and open science"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "1. Reproducibility crisis\n",
    "2. Waste in research\n",
    "3. Data sharing requirements and strategies\n",
    "4. Open science: The concrete practice"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# 1. Reproducibility crisis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "![](Reproducible_research/why.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "![Have you ever failed to reproduce an experiment?](Reproducible_research/reproducibility-graphic-online3.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "![Factors contributing to irreproducible research](Reproducible_research/reproducibility-graphic-online4.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "![](Reproducible_research/causes.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Publication bias\n",
    "![](Reproducible_research/bias-a.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "![](Reproducible_research/bias-b.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Publication bias\n",
    "![](Reproducible_research/bias-c.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Funnel plot: A method to detect (extreme) publication bias\n",
    "![](Reproducible_research/funnel_shanks1.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## P-hacking\n",
    "![](Reproducible_research/edge-of-significance.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## P-Curve: A method to evaluate p-hacking\n",
    "![](Reproducible_research/p-hacking.png)\n",
    "http://datacolada.org/41"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Statistical Power\n",
    "![](Reproducible_research/power1.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Statistical Power\n",
    "![](Reproducible_research/power2.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### More:\n",
    "\n",
    "* http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020124\n",
    "* http://www.nature.com/news/1-500-scientists-lift-the-lid-on-reproducibility-1.19970\n",
    "* http://blogs.discovermagazine.com/neuroskeptic/2015/11/10/reproducibility-crisis-the-plot-thickens\n",
    "* http://datacolada.org/41\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# 2. Waste in research"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "![](Reproducible_research/waste1.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "![](Reproducible_research/waste2.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "![](Reproducible_research/equator.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Possible solutions\n",
    "![](Reproducible_research/strategies1.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Possible solutions\n",
    "![](Reproducible_research/strategies2.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### More\n",
    "http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1002165\n",
    "http://www.nature.com/news/irreproducible-biology-research-costs-put-at-28-billion-per-year-1.17711\n",
    "http://www.equator-network.org/\n",
    "http://www.acmedsci.ac.uk/policy/policy-projects/reproducibility-and-reliability-of-biomedical-research/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Data sharing requirements and strategies\n",
    "    - What is data sharing\n",
    "    - Who is sharing data\n",
    "        - NIH policy\n",
    "        - UK Biobank\n",
    "        - Precision medicine initiative\n",
    "        - Montreal Neurological Institute: First open science institute?\n",
    "        - Grass-roots initiatives: FCP, ADHD200, ABIDE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Data sharing\n",
    "https://en.wikipedia.org/wiki/Data_sharing\n",
    "\n",
    "The practice of making data used for scholarly research available to other investigators. Replication has a long history in science. The motto of The Royal Society is 'Nullius in verba', translated __\"Take no man's word for it\"__. Many funding agencies, institutions, and publication venues have policies regarding data sharing because transparency and openness are considered by many to be part of the scientific method.\n",
    "\n",
    "### F.A.I.R. Data sharing\n",
    "* Findable: Permanent repository\n",
    "* Accessible: Easy to retrieve\n",
    "* Interoperable: Use standard open data formats and ontologies\n",
    "* Re-usable: Clear license, indication of provenance\n",
    "\n",
    "https://www.force11.org/group/fairgroup/fairprinciples\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Open data requirements\n",
    "\n",
    "![](Reproducible_research/funding.png)\n",
    "\n",
    "http://whyopenresearch.org/funding"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Open data requirements\n",
    "\n",
    "![](Reproducible_research/funding2.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Open science: The concrete practice"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "![](Reproducible_research/open1.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Sign-in for a GitHub account\n",
    "* Make your jupyter notebooks executable using mybinder.org\n",
    "* Publish your code and make it citable with GitHub and Zenodo\n",
    "* Publish all outcomes of your research using biorxiv, zenodo, RIO journal, figshare\n",
    "\n",
    "### Git parable\n",
    "http://tom.preston-werner.com/2009/05/19/the-git-parable.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}