docs/notebooks/1_introduction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Introduction\n",
    "\n",
    "Bayesian Interaction Primitives (referred to as BIP from this point forward) are a human-robot interaction framework which focuses on extracting a probabilistic model of interaction dynamics as presented in a set of example demonstrations.\n",
    "This section will provide a brief introduction as to what exactly this means as well as an intuitive description of how BIP goes about building its model (more in-depth mathematical details will be presented in Section 4).\n",
    "A quickstart on how to get up-and-running with the library can be found in Section 2 while a thorough tutorial can be found in Section 3.\n",
    "\n",
    "### What is human-robot interaction?\n",
    "\n",
    "Loosely speaking, human-robot interaction (HRI) is any sort of physical or non-physical interaction in which a human and a robot extend some form of mutual influence or action to each other.\n",
    "On it's own this is a massive field and can encompass interactions ranging from verbal communication to physical activity, and draws from fields such as natural language processing, computer vision, planning, control theory, and many more.\n",
    "Even the social sciences play an important role, as humans have certain expectations when interacting with others, and robots must be aware of and fulfill these expectations in order to result in comfortable interactions.\n",
    "In fact, colloquially speaking we consider something to be \"robotic\" precisely when it fails to fulfill these expectations.\n",
    "So as roboticists, our goal should be to produce human-robot interactions which are anything but \"robotic\"!\n",
    "\n",
    "![](media/examples_new.png)\n",
    "\n",
    "With Bayesian Interaction Primitives, we focus specifically on cooperative physical interactions, for example, shaking hands or jointly lifting a large object.\n",
    "Due to how we capture the dynamics of an interaction, BIP can actually be used to model *any* temporal multivariate observations which exhibit strong correlations, however, we will focus only on HRI scenarios in this documentation.\n",
    "\n",
    "### What do we mean by interaction dynamics?\n",
    "\n",
    "We define the dynamics of an interaction as how an interaction evolves over time, both in terms of the spatial relationship between interaction partners and the temporal relationship.\n",
    "For example, in a handshake interaction, the hands of each partner converge in physical space and clasp for an amount of time that is not explicitly communicated, rather these are inherent to the *dynamics* of an interaction and are understood by both partners.\n",
    "Modeling these dynamics allows us to reproduce the full interaction after only observing a part of it because we know the temporal and spatial relationship over time.\n",
    "Fundamentally, this is what allows us to perform inference with Bayesian Interaction Primitives.\n",
    "\n",
    "### How are the interaction dynamics modeled?\n",
    "\n",
    "The key question that we are answering with Bayesian Interaction Primitives is: how do we model these dynamics?\n",
    "Deriving an analytical model up front is extremely challenging given the inherent variability in human behavior.\n",
    "Instead, we take a data-driven approach known as *imitation learning* or *learning-from-demonstration*, in which we observe example demonstrations of the interaction in question in order to extract the interaction dynamics.\n",
    "This approach resembles how humans and animals learn new skills: through observation.\n",
    "\n",
    "Before diving into the math in Section 4, we will begin by giving a high-level overview of how the models are actually built.\n",
    "\n",
    "\n",
    "![](media/bip_overview.png)\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Training\n",
    "During training, we observe demonstrations of the interaction.\n",
    "These interactions can either be performed by two humans (in which case the correspondence problem must be solved at some point if robot control is desired) or between a human and a robot (known as kinesthetic teaching).\n",
    "Each interaction yields a sequence of observations over time for every measured degree of freedom (DoF).\n",
    "However, these sequences are difficult to work with directly, so we decompose them into a latent space composed of a linear combination of nonlinear basis functions, where the coefficients represent the latent state of an interaction.\n",
    "This allows us to construct a prior in the form of a joint probability distribution in the combined latent space of all DoFs, which represents the region where we believe a successful interaction lies.\n",
    "\n",
    "##### Testing\n",
    "During testing, we begin by observing a partial sequence of observations from a subset of the DoFs.\n",
    "At this point, there are two unknowns: 1) what specific latent state corresponds to the current interaction, and 2) how much of the interaction have we observed.\n",
    "These are related problems; the transformation between the latent space and observation space requires knowing how much of the interaction has occurred (the phase of the interaction).\n",
    "In order to integrate the observations then, we perform inference in which we simultaneously infer both the spatial state of the interaction (the latent state) and the temporal state (the phase).\n",
    "Inference involves updating our prior distribution based on these observations to produce a new posterior distribution from which we generate control trajectories for the robot.\n",
    "\n",
    "However, since both the prior and posterior distributions are joint probability distributions over all DoFs, we can also infer future states for any DoF in the model.\n",
    "For example, suppose the human partner provided demonstrations in which they wore pressure-sensitive shoes.\n",
    "If these shoes are no longer worn at test time we can still infer what the values should be given the observations that we do have.\n",
    "This provides a particularly powerful framework for dealing with sensor failures, be it from occlusion, noise, hardware failure, or otherwise.\n",
    "\n",
    "##### What is IntPrim?\n",
    "\n",
    "IntPrim is a Python library implementing Bayesian Interaction Primitives.\n",
    "It provides the following features out of the box:\n",
    "\n",
    "* Train a BIP model from demonstrations\n",
    "* Perform recursive inference with a BIP model and generate future states\n",
    "* Support for inference with ensemble Bayesian Interaction Primitives, Bayesian Interaction Primitives, Probabilistic Movement Primitives with DTW, and Particle Filter\n",
    "* Automatic basis space selection with support for Gaussian, Sigmoidal, and Polynomial functions\n",
    "* Automatic computation of the observation noise\n",
    "* Comprehensive interactive analysis tools\n",
    "\n",
    "The rest of this document will discuss these features and more."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [conda env:anaconda]",
   "language": "python",
   "name": "conda-env-anaconda-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}