intro-to-biopython-blank.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to Biopython \n",
    "\n",
    "https://biopython.org/\n",
    "\n",
    "\n",
    "<img src=\"./images/Biopython_logo.png\" align=\"left\" alt=\"\" style=\"width: 200px;\"/>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Hardik I. Parikh, PhD  \n",
    "School of Medicine Research Computing  \n",
    "University of Virginia  \n",
    "10/18/2018  \n",
    "\n",
    "hiparikh@virginia.edu"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***\n",
    "## Outline\n",
    "\n",
    "Biopython is a set of freely available tools for biological computation written in Python. \n",
    "  \n",
    "This workshop assumes that you have mastered the basics of Python. It is aimed at introducing you to some of the core components of the Biopython library. We will learn some (of the many) modules available for dealing with biological data using practical examples.  \n",
    "\n",
    "We will only be scratching the surface! \n",
    "\n",
    "* Installation\n",
    "* Working with Sequences - `Bio.SeqIO`\n",
    "* Working with Multiple Sequence Alignments - `Bio.AlignIO`\n",
    "* Running Blast - `Bio.Blast`\n",
    "* Querying NCBI - `Bio.Entrez`\n",
    "\n",
    "\n",
    "Useful links: \n",
    "* [Tutorial and Cookbook](https://biopython.org/wiki/Documentation)\n",
    "* [API documentation](http://biopython.org/DIST/docs/api/)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***\n",
    "## Installation\n",
    "\n",
    "[Installation Guide](http://biopython.org/DIST/docs/install/Installation.html)  \n",
    "\n",
    "1. Using pip (on Terminal) \n",
    "```\n",
    "pip install biopython\n",
    "```  \n",
    "\n",
    "2. Anaconda Navigator \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***\n",
    "## Working with Sequences\n",
    "\n",
    "The `Bio.SeqIO` module provides a simple uniform interface to input and output various sequence file formats. \n",
    "\n",
    "In Biopython, sequences are held as `Seq` objects, which has two attributes: the sequence string and an associated alphabet. When additional information is available regarding a sequence, like an identifier or name, or description/annotation, then a `SeqRecord` object is used. \n",
    "\n",
    "Let's begin ..."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### `Seq` Object"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from Bio.Seq import Seq\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Add alphabet. Although optional, specify where possible.* "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from Bio import Alphabet\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Sequences - String Operations**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# iterate over each element \n",
    "    \n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# slicing; returns a `Seq` object\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get codons\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# changing case\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Sequences - Additional biological methods**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#mySeq.complement()\n",
    "#mySeq.reverse_complement()\n",
    "#mySeq.transcribe()\n",
    "#mySeq.translate()\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Calculate GC content\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Use GC function from SeqUtils\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# why alphabet is helpful\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### `SeqRecord` Object"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# additional attributes\n",
    "# .seq\n",
    "# .id\n",
    "# .name\n",
    "# .description\n",
    "# .letter_annotations\n",
    "# .annotations\n",
    "# .features\n",
    "# .dbxrefs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### `SeqIO` module"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### `SeqIO.parse()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from Bio import SeqIO\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Read sequence records from genbank file\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise:\n",
    "\n",
    "1. Read sequences from FASTA file `./data/sequence.fasta`, print following information for each read \n",
    "    - Sequence name \n",
    "    - Sequence length\n",
    "    - GC content "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Enter your code here\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Combine with other python packages for visualization "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# example to plot histogram of sequence lengths\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "seqFile = \"./data/16SMicrobial.fasta\"\n",
    "\n",
    "# seqLenList \n",
    "seqLenList = []\n",
    "for rec in SeqIO.parse(seqFile, \"fasta\"):\n",
    "    seqLenList.append(len(rec.seq))\n",
    "\n",
    "# plot histogram\n",
    "x = np.array(seqLenList)\n",
    "sns.distplot(x)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# SeqRecords from FASTQ file\n",
    "\n",
    "fastqFile = \"./data/sequence.fastq\"\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plot\n",
    "for rec in SeqIO.parse(fastqFile, \"fastq\"):\n",
    "    plt.plot(rec.letter_annotations[\"phred_quality\"])\n",
    "\n",
    "plt.ylabel(\"PHRED quality score\")\n",
    "plt.xlabel(\"Position\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### `SeqIO.write()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# files\n",
    "inFasta = \"./data/sequence.fasta\"\n",
    "outFasta = \"./data/sequence_prot.fasta\"\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### `SeqIO.convert()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***\n",
    "\n",
    "## Working with Alignment Files\n",
    "\n",
    "The `Bio.AlignIO` module provides a simple uniform interface to input and output various multiple sequence alignment file formats."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from Bio import AlignIO\n",
    "\n",
    "# read alignments\n",
    "sthFile = \"./data/PF09395.sth\"\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "### convert between formats\n",
    "\n",
    "# write to fasta \n",
    "\n",
    "\n",
    "\n",
    "# write to phylip format\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***\n",
    "\n",
    "## BLAST\n",
    "\n",
    "### Running WWW BLAST \n",
    "We can call the online version of BLAST using the function `qblast()` in the `Bio.Blast.NCBIWWW` module"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from Bio.Blast import NCBIWWW\n",
    "from Bio import SeqIO\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Running BLAST locally from python\n",
    "\n",
    "If you have BLAST+ executable installed locally, we can call it using `NcbiblastnCommandline` from `Bio.Blast.Applications`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from Bio.Blast.Applications import NcbiblastxCommandline\n",
    "\n",
    "# command line BLAST+ commands\n",
    "\n",
    "# create blast database from commandline (cannot do using Biopython)\n",
    "# makeblastdb -dbtype nucl -in ./data/16SMicrobial.fasta\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Parsing BLAST output\n",
    "\n",
    "The BLAST Record object contains all the information!!!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#xmlFile = \"./data/blastResults.xml\"\n",
    "#ifh = open(xmlFile)\n",
    "#blastRec = NCBIXML.parse(ifh)\n",
    "#for rec in blastRec:\n",
    "#    for aln in rec.alignments:\n",
    "#        for hsp in aln.hsps:\n",
    "#            print(dir(hsp))\n",
    "#ifh.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from Bio.Blast import NCBIXML\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***\n",
    "\n",
    "## Querying NCBI\n",
    "\n",
    "`Entrez` is a molecular biology database system that provides integrated access to NCBI's nucleotide and protein sequence data, gene-centered and genomic mapping information, 3D structure data, PubMed MEDLINE, and more.   \n",
    "<br>\n",
    "The `Bio.Entrez` module makes use of the Entrez Programming Utilities (also known as [EUtils](http://www.ncbi.nlm.nih.gov/entrez/utils/)), corresponding to individual Python functions. This module makes sure that the correct URL is used for the queries, and that not more than one request is made every three seconds, as required by NCBI.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from Bio import Entrez\n",
    "\n",
    "#Entrez.email = \"abc@example.com\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# What databases are available for me to query?\n",
    "# Entrez.einfo()\n",
    "#handle = Entrez.einfo()\n",
    "#record = Entrez.read(handle)\n",
    "#print(record['DbList'])\n",
    "#handle.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Query the NCBI databases\n",
    "# Entrez.esearch()\n",
    "# returns primary ID \n",
    "# vs\n",
    "# Entrez.efetch()\n",
    "# returns the record\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's fetch MH558576.1\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# If the rettype is one of the modes ompatible with SeqIO format, convert it to SeqIO object\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Write output to file\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Pubmed search\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}