generate_figure_inputs.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"toc\"></a>\n",
    "# GENERATE FIGURE INPUTS\n",
    "\n",
    "This notebook provides functions and scripts for generating input files for visualization with D3.\n",
    "\n",
    "---\n",
    "- [WORKSPACE VARIABLES](#workspace-variables)\n",
    "- **[DEFAULT](#default)**\n",
    "    - [default states](#default-default-states)\n",
    "    - [default singles](#default-default-singles)\n",
    "    - [nci 60](#default-nci60)\n",
    "    - [moon](#default-moon)\n",
    "    - [random distribution](#default-random-distribution)\n",
    "- **[MODULE COMPLEXITY](#module-complexity)**\n",
    "    - [temporal complexity](#module-complexity-temporal-complexity)\n",
    "    - [state distribution](#module-complexity-state-distribution)\n",
    "    - [module concentrations](#module-complexity-module-concentrations)\n",
    "- **[PARAMETER SENSITIVITY](#parameter-sensitivity)**\n",
    "    - [metric sensitivity](#parameter-sensitivity-metric-sensitivity)\n",
    "    - [temporal sensitivity](#parameter-sensitivity-temporal-sensitivity)\n",
    "- **[GROWTH CONTEXT](#growth-context)**\n",
    "    - [population fractions](#growth-context-population-fractions)\n",
    "    - [temporal context](#growth-context-temporal-context)\n",
    "    - [metric distribution](#growth-context-metric-distribution)\n",
    "    - [delta venn](#growth-context-delta-venn)\n",
    "    - [metric whisker](#growth-context-metric-whisker)\n",
    "- **[CELL COMPETITION](#cell-competition)**\n",
    "    - [competition fractions](#cell-competition-competition-fractions)\n",
    "    - [competition maps](#cell-competition-competition-maps)\n",
    "    - [spatial competition](#cell-competition-spatial-competition)\n",
    "    - [temporal competition](#cell-competition-temporal-competition)\n",
    "    - [modified distribution](#cell-competition-modified-distribution)\n",
    "    - [competition populations](#cell-competition-competition-populations)\n",
    "- **[POPULATION HETEROGENEITY](#population-heterogeneity)**\n",
    "    - [heterogeneity maps](#population-heterogeneity-heterogeneity-maps)\n",
    "    - [heterogeneity fractions](#population-heterogeneity-heterogeneity-fractions)\n",
    "    - [parameter distributions](#population-heterogeneity-parameter-distributions)\n",
    "    - [delta maps](#population-heterogeneity-delta-maps)\n",
    "    - [metric ladder](#population-heterogeneity-metric-ladder)\n",
    "    - [stats maps](#population-heterogeneity-stats-maps)\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each of the simulation sets has a corresponding script of the same name, which contains a class (of the same name) with relevant condition variables and methods to iterate through these condition.\n",
    "The `loop` method will be used the most often to iterate through all conditions, extract relevant information, and then compile the information into a single output file.\n",
    "The `loop` method works with both individual files for each condition (`.csv` or `.json`) or a `.tar.xz` compressed archive of the files for each condition.\n",
    "For a given figure, not all conditions may be used.\n",
    "\n",
    "All generated figure inputs are provided in the `analysis` directory.\n",
    "As much as possible, files are provided in compressed archive form, using the provided `compress.sh` script, taking three arguments: `[NAME OF SIMULATION SET] [EXTENSION NAME] [json OR csv]`.\n",
    "The corresponding arguments for each of the following figures is provided in the description.\n",
    "\n",
    "The `generate.py` file contains functions to generate figure inputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scripts.generate import *"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"workspace-variables\"></a>\n",
    "\n",
    "### WORKSPACE VARIABLES\n",
    "\n",
    "Set up workspace variables for analyzing simulations.\n",
    "\n",
    "- **`DATA_PATH`** is the path to data files (`.tar.xz` files of compressed simulation outputs)\n",
    "- **`RESULTS_PATH`** is the path to result files (`.pkl` files generated by parsing)\n",
    "- **`ANALYSIS_PATH`** is the path for analysis files (`.json` and `.csv` files, `.tar.xz` compressed archives)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "DATA_PATH = \"/path/to/data/files/\"\n",
    "RESULTS_PATH = \"/path/to/result/files/\"\n",
    "ANALYSIS_PATH = \"/path/to/analysis/files/\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"default\"></a>\n",
    "\n",
    "### DEFAULT\n",
    "\n",
    "Generate `DEFAULT` figure inputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scripts.DEFAULT import DEFAULT"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"default-default-states\"></a>\n",
    "\n",
    "**default states**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/default_states.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The default states figure plots the spatial distribution of cell states at different time points for default simulations.\n",
    "\n",
    "The function `make_default_states` extracts the cell states at each position of each grid hexagon.\n",
    "These files are then merged into a single file `DEFAULT.POSITIONS.csv` that is use as input for D3.\n",
    "Files created by the function `make_default_states` can be compressed with `compress.sh` using `DEFAULT .POSITIONS csv`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "timepoints = [2, 10, 16, 20, 30] \n",
    "DEFAULT.run(ANALYSIS_PATH, RESULTS_PATH, make_default_states, timepoints=timepoints, seeds=[0])\n",
    "DEFAULT.loop(ANALYSIS_PATH, merge_default_states, save_default_states, \".POSITIONS\", \\\n",
    "             timepoints=[\"010\", \"050\", \"080\", \"100\", \"150\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"default-default-singles\"></a>\n",
    "\n",
    "**default singles**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/default_singles.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The default singles figure plots timecourses of cell count or diamater over time for each simulation seed.\n",
    "\n",
    "The function `make_default_singles` extracts cell counts and diameters for each seed of `C` and `H` default simulations and compiles them into a finel file `DEFAULT.SINGLES.json`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "DEFAULT.call(ANALYSIS_PATH, RESULTS_PATH, make_default_singles)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"default-nci60\"></a>\n",
    "\n",
    "**NCI 60**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/nci60.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The NCI 60 figure compares simulated doubling time with experimental values from the NCI-60 Human Tumor Cell Lines Screen.\n",
    "\n",
    "The file `NCI60.json` contains experimental doubling times from 60 different human tumor cell lines.\n",
    "The function `make_doubling_time` calculates doubling time from simulated data, saved to the file `DEFAULTS.DOUBLING.json`.\n",
    "The function `make_exponential_fit` fits an exponetial to the simulated data and calculates the doubling time from the fit, saved to the file `DEFAULTS.EXPONENTIAL.json`.\n",
    "All three files are used to produce the final figure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "DEFAULT.call(ANALYSIS_PATH, RESULTS_PATH, make_doubling_time)\n",
    "DEFAULT.call(ANALYSIS_PATH, RESULTS_PATH, make_exponential_fit)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"default-moon\"></a>\n",
    "\n",
    "**moon**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/moon.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The moon figure compares simulated and experimental fits to colony diameter, average cell volume, and cell diameter based on the manuscript by Meyskens, Thomson, and Moon (1984).\n",
    "\n",
    "The function `make_moon_fit` extracts colony diameter, average cell volume, and cell diameter, saved to the file `DEFAULTS.MOON.csv`.\n",
    "The function `calculate_moon_fit` fits the equation presented in the manuscript to the simulated data.\n",
    "Note that this function does not produce an output file; the coefficients are used to draw fit lines in the `DECORATOR` function for the figure."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "DEFAULT.call(ANALYSIS_PATH, RESULTS_PATH, make_moon_fit)\n",
    "DEFAULT.call(ANALYSIS_PATH, RESULTS_PATH, calculate_moon_fit)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"default-random-distribution\"></a>\n",
    "\n",
    "**random distribution**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/random_distribution.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The random distribution figures shows distribution of cell states as a function of distance from the center of the colony for simulation using a random rule set.\n",
    "\n",
    "The `.DISTRIBUTION` files produced in the basic analysis step are merged into a single file (`DEFAULT.DISTRIBUTION.csv`) which is then used as inputs for D3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "DEFAULT.call(ANALYSIS_PATH, RESULTS_PATH, make_random_distribution)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"module-complexity\"></a>\n",
    "\n",
    "### MODULE COMPLEXITY\n",
    "\n",
    "Generate `MODULE_COMPLEXITY` figure inputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scripts.MODULE_COMPLEXITY import MODULE_COMPLEXITY"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"module-complexity-temporal-complexity\"></a>\n",
    "\n",
    "**temporal complexity**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/temporal_complexity.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The temporal complexity figures shows emergent metrics over time for different levels of module complexity.\n",
    "\n",
    "The `.METRICS` files produced in the basic analysis step are merged into a single file per metric (`MODULE_COMPLEXITY.METRICS.*.json`) which are then used as inputs for D3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "METRICS = [\"GROWTH\", \"SYMMETRY\", \"CYCLES\"]\n",
    "for metric in METRICS:\n",
    "    MODULE_COMPLEXITY.loop(ANALYSIS_PATH, merge_metrics, save_metrics, f\".METRICS.{metric}\", timepoints=[None])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"module-complexity-state-distribution\"></a>\n",
    "\n",
    "**state distribution**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/state_distribution.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The state distribution figure shows cell type distribution across the colony different levels of module complexity.\n",
    "\n",
    "The `.DISTRIBUTION` files produced in the basic analysis step are used as input.\n",
    "Make sure to EXTRACT THE ARCHIVE FILES from `MODULE_COMPLEXITY.DISTRIBUTION.tar.xz`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"module-complexity-module-concentrations\"></a>\n",
    "\n",
    "**module concentrations**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/module_concentrations.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The module concentrations figure shows concentrations of glucose, oxygen, and TGF$\\alpha$ for different module configurations: metabolism module only  (`.M`), signaling module only (`.S`), both modules (`.B`), and full rule set.\n",
    "\n",
    "The `.CONCENTRATIONS` files produced in the basic analysis step (which only analyzes the full rule set simulations) are merged into a single file (`MODULE_COMPLEXITY.CONCENTRATIONS.json`) which is then used as input for D3.\n",
    "The function `make_module_concentrations` extracts timecourse of concentrations from the non-full-rule simulations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "MODULE_COMPLEXITY.loop(ANALYSIS_PATH, merge_concentrations, save_concentrations, f\".CONCENTRATIONS\",\n",
    "                       timepoints=[\"010\", \"080\", \"150\"])\n",
    "MODULE_COMPLEXITY.call(ANALYSIS_PATH, DATA_PATH, make_module_concentrations)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"parameter-sensitivity\"></a>\n",
    "\n",
    "### PARAMETER SENSITIVITY\n",
    "\n",
    "Generate `PARAMETER_SENSITIVITY` figure inputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scripts.PARAMETER_SENSITIVITY import PARAMETER_SENSITIVITY"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"parameter-sensitivity-metric-sensitivity\"></a>\n",
    "\n",
    "**metric sensitivity**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/metric_sensitivity.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The metric sensitivity figure plots sensitivity of metrics as a function of change in parameter value.\n",
    "\n",
    "The `.METRICS` files produced in the basic analysis step are merged into a single file per metric (`PARAMETERS_SENSITIVITY.METRICS.*.json` ) which are then used as inputs for D3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "METRICS = [\"GROWTH\", \"SYMMETRY\", \"CYCLES\"]\n",
    "for metric in METRICS:\n",
    "    PARAMETER_SENSITIVITY.loop(ANALYSIS_PATH, merge_metrics, save_metrics, f\".METRICS.{metric}\", timepoints=[None])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"parameter-sensitivity-temporal-sensitivity\"></a>\n",
    "\n",
    "**temporal sensitivity**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/temporal_sensitivity.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The temporal sensitivity figures shows sensitivity of emergent metrics over time to different parameters.\n",
    "\n",
    "These figures use the same merged files (`*.METRICS.*.json`) as the **metric sensitivity** figures (see above)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"growth-context\"></a>\n",
    "\n",
    "### GROWTH CONTEXT\n",
    "\n",
    "Generate `GROWTH_CONTEXT` figure inputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scripts.GROWTH_CONTEXT import GROWTH_CONTEXT"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"growth-context-population-fractions\"></a>\n",
    "\n",
    "**population fractions**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/population_fractions.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The population fraction plot shows fraction of cells of in each population over time.\n",
    "\n",
    "The function `merge_fractions` uses the `.METRICS.COUNTS` and `.METRICS.POPS` files produced in the basic analysis step to calculate population fraction.\n",
    "Values across conditions are merged into a single file `GROWTH_CONTEXT.FRACTIONS.json` that is used as input for D3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "GROWTH_CONTEXT.loop(ANALYSIS_PATH, merge_fractions, save_fractions, f\".FRACTIONS\", timepoints=[None])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"growth-context-temporal-context\"></a>\n",
    "\n",
    "**temporal context**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/temporal_context.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The temporal context figures shows emergent metrics over time for populations in different contexts.\n",
    "\n",
    "The `.METRICS` files produced in the basic analysis step are merged into a single file per metric (`GROWTH_CONTEXT.METRICS.*.json`) which are then used as inputs for D3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "METRICS = [\"GROWTH\", \"SYMMETRY\", \"CYCLES\"]\n",
    "for metric in METRICS:\n",
    "    GROWTH_CONTEXT.loop(ANALYSIS_PATH, merge_metrics, save_metrics, f\".METRICS.{metric}\", timepoints=[None])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"growth-context-metric-distribution\"></a>\n",
    "\n",
    "**metric distribution**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/metric_distribution.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The metric distribution figure shows distribution of metric values for populations in different contexts.\n",
    "\n",
    "The `.SEEDS` files produced in the basic analysis step are merged into a single file per metric (`GROWTH_CONTEXT.SEEDS.*.json`) which are then used as inputs for D3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "METRICS = [\"GROWTH\", \"SYMMETRY\", \"CYCLES\"]\n",
    "for metric in METRICS:\n",
    "    GROWTH_CONTEXT.loop(ANALYSIS_PATH, merge_seeds, save_seeds, f\".SEEDS.{metric}\", timepoints=[None])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"growth-context-delta-venn\"></a>\n",
    "\n",
    "**delta venn**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/delta_venn.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The delta venn figure shows a venn diagram of change in metric values between contexts for all population combinations.\n",
    "\n",
    "These figures use the same merged files (`*.SEEDS.*.json`) as the **metric distribution** figures (see above)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"growth-context-metric-whisker\"></a>\n",
    "\n",
    "**metric whisker**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/metric_whisker.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The metric whisker figures shows emergent metrics for all possible population combinations for selected context.\n",
    "\n",
    "These figures use the same merged files (`*.METRICS.*.json`) as the **metric distribution** figures (see above)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"cell-competition\"></a>\n",
    "\n",
    "### CELL COMPETITION\n",
    "\n",
    "Generate `CELL_COMPETITION` figure inputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scripts.CELL_COMPETITION import CELL_COMPETITION"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"cell-competition-competition-fractions\"></a>\n",
    "\n",
    "**competition fractions**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/competition_fractions.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The competition fraction plot shows fraction of cells of in each population over time.\n",
    "\n",
    "The function `merge_fractions` uses the `.METRICS.COUNTS` and `.METRICS.POPS` files produced in the basic analysis step to calculate population fraction.\n",
    "Values across conditions are merged into a single file `CELL_COMPETITION.FRACTIONS.json` that is used as input for D3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "CELL_COMPETITION.loop(ANALYSIS_PATH, merge_fractions, save_fractions, f\".FRACTIONS\", timepoints=[None])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"cell-competition-competition-maps\"></a>\n",
    "\n",
    "**competition maps**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/competition_maps.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The competition maps plot shows metric values for all simulation conditions.\n",
    "\n",
    "The function `merge_maps` uses the `.METRICS` files produced in the basic analysis step and reformats into a list of values for each condition per metric (`CELL_COMPETITION.MAPS.*.csv`) which are then used as inputs for D3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "METRICS = [\"GROWTH\", \"SYMMETRY\", \"CYCLES\"]\n",
    "for metric in METRICS:\n",
    "    CELL_COMPETITION.loop(ANALYSIS_PATH, merge_maps, save_maps, f\".METRICS.{metric}\", timepoints=[14])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"cell-competition-spatial-competition\"></a>\n",
    "\n",
    "**spatial competition**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/spatial_competition.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The spatial competition shows relative distribution of cell populations for different conditions.\n",
    "\n",
    "The `.LOCATIONS` files produced in the basic analysis step are used as input.\n",
    "Make sure to EXTRACT THE ARCHIVE FILES from `CELL_COMPETITION.LOCATIONS.tar.xz`.\n",
    "Only the `*_050_050` conditions with extension `.LOCATIONS.140.csv` are needed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"cell-competition-temporal-competition\"></a>\n",
    "\n",
    "**temporal competition**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/temporal_competition.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The temporal context figures shows emergent metrics over time for different initial conditions.\n",
    "\n",
    "The `.METRICS` files produced in the basic analysis step are merged into a single file per metric (`CELL_COMPETITION.METRICS.*.json`) which are then used as inputs for D3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "METRICS = [\"GROWTH\", \"SYMMETRY\", \"CYCLES\"]\n",
    "for metric in METRICS:\n",
    "    CELL_COMPETITION.loop(ANALYSIS_PATH, merge_metrics, save_metrics, f\".METRICS.{metric}\", timepoints=[None])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"cell-competition-modified-distribution\"></a>\n",
    "\n",
    "**modified distribution**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/modified_distribution.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The modified distribution shows the cell distribution as a function of distance from the center of the basal and modified populations.\n",
    "\n",
    "The `.DISTRIBUTION` files produced in the basic analysis step are used as input.\n",
    "Make sure to EXTRACT THE ARCHIVE FILES from `CELL_COMPETITION.DISTRIBUTION.tar.xz`.\n",
    "Only the `*_050_050` conditions are needed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"cell-competition-competition-populations\"></a>\n",
    "\n",
    "**competition populations**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/competition_populations.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The competition populationsfigure plots the spatial distribution of cell populations for differen modified parameters.\n",
    "\n",
    "The function `make_competition_populations` extracts the cell populations at each position of each grid hexagon.\n",
    "These files are then merged into a single file `CELL_COMPETITION.POSITIONS.csv` that is use as input for D3.\n",
    "Files created by the function `make_competition_populations` can be compressed with `compress.sh` using `CELL_COMPETITION .POSITIONS csv`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "initializations = [('050','050')]\n",
    "CELL_COMPETITION.run(ANALYSIS_PATH, RESULTS_PATH, make_competition_populations, \\\n",
    "                     timepoints=[28], seeds=[0], initializations=initializations)\n",
    "CELL_COMPETITION.loop(ANALYSIS_PATH, merge_competition_populations, save_competition_populations, \\\n",
    "                      \".POSITIONS\", timepoints=[\"140\"], initializations=initializations)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"population-heterogeneity\"></a>\n",
    "\n",
    "### POPULATION HETEROGENEITY\n",
    "\n",
    "Generate `POPULATION_HETEROGENEITY` figure inputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scripts.POPULATION_HETEROGENEITY import POPULATION_HETEROGENEITY"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"population-heterogeneity-heterogeneity-maps\"></a>\n",
    "\n",
    "**heterogeneity maps**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/heterogeneity_maps.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The heterogeneity maps plot shows metric values for all simulation conditions.\n",
    "\n",
    "The function `merge_maps` uses the `.METRICS` files produced in the basic analysis step and reformats into a list of values for each condition per metric (`POPLATION_HETEROGENEITY.MAPS.*.csv`) which are then used as inputs for D3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "METRICS = [\"GROWTH\", \"SYMMETRY\", \"CYCLES\"]\n",
    "for metric in METRICS:\n",
    "    POPULATION_HETEROGENEITY.loop(ANALYSIS_PATH, merge_maps, save_maps, f\".METRICS.{metric}\", timepoints=[15])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"population-heterogeneity-heterogeneity-fractions\"></a>\n",
    "\n",
    "**heterogeneity fractions**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/heterogeneity_fractions.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The heterogeneity fraction plot shows fraction of cells of in each population over different heterogeneity values.\n",
    "\n",
    "The function `merge_fractions` uses the `.METRICS.COUNTS` and `.METRICS.POPS` files produced in the basic analysis step to calculate population fraction.\n",
    "Values across conditions are merged into a single file `POPULATION_HETEROGENEITY.FRACTIONS.json` that is used as input for D3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "POPULATION_HETEROGENEITY.loop(ANALYSIS_PATH, merge_fractions, save_fractions, f\".FRACTIONS\", timepoints=[None])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"population-heterogeneity-parameter-distributions\"></a>\n",
    "\n",
    "**parameter distributions**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/parameter_distributions.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The parameter distributions figure plots distribution of parameter values for selected time points.\n",
    "\n",
    "\n",
    "The function `make_parameter_distributions` extracts selected parameters from all cells and calculates statistics.\n",
    "Make sure to EXTRACT THE ARCHIVE FILES from `POPULATION_HETEROGENEITY.PARAMETERS.tar.xz`.\n",
    "Files created by the function `make_parameter_distributions` can be compressed with `compress.sh` using `POPULATION_HETEROGENEITY .PARAMETERS json`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "populations = [\"X\", \"A\", \"B\", \"C\"]\n",
    "POPULATION_HETEROGENEITY.load(ANALYSIS_PATH, DATA_PATH, make_parameter_distributions, \\\n",
    "    populations=populations, extension=\".PARAM\", timepoints=[\"010\", \"080\", \"150\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"population-heterogeneity-delta-maps\"></a>\n",
    "\n",
    "**delta maps**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/delta_maps.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The delta maps plot shows change in metric values between contexts.\n",
    "\n",
    "\n",
    "The `.SEEDS` files produced in the basic analysis step are merged into a single file per metric (`POPULATION_HETEROGENEITY.SEEDS.*.json`) which are then used as inputs for D3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "METRICS = [\"GROWTH\", \"SYMMETRY\", \"CYCLES\"]\n",
    "for metric in METRICS:\n",
    "    POPULATION_HETEROGENEITY.loop(ANALYSIS_PATH, merge_seeds, save_seeds, f\".SEEDS.{metric}\", timepoints=[None])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"population-heterogeneity-metric-ladder\"></a>\n",
    "\n",
    "**metric ladder**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/metric_ladder.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The metric ladder plot shows metric values for each simulation seed for selected populations.\n",
    "\n",
    "\n",
    "These figures use the same merged files (`*.SEEDS.*.json`) as the **delta maps** figures (see above)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"population-heterogeneity-stats-maps\"></a>\n",
    "\n",
    "**stats maps**\n",
    "<span style=\"float:right;\">\n",
    "    [go to figure](http://0.0.0.0:8000/figures/stats_maps.html) &#x2022;\n",
    "    [back to top](#toc)\n",
    "</span>\n",
    "\n",
    "The stats maps plot shows means and standard deviations of parameter values.\n",
    "\n",
    "The function `merge_stats_maps` uses the `.PARAMETERS` files produced from `make_parameter_distributions` (see above) reformats into a list of values for each condition (`POPLATION_HETEROGENEITY.STATS.csv`) which are then used as input for D3."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "populations = [\"X\", \"A\", \"B\", \"C\"]\n",
    "POPULATION_HETEROGENEITY.loop(ANALYSIS_PATH, merge_stats_maps, save_stats_maps, f\".PARAMETERS\", \\\n",
    "    populations=populations, timepoints=[15])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}