parse_simulation_outputs.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"toc\"></a>\n",
    "# PARSE SIMULATION OUTPUTS\n",
    "\n",
    "This notebook provides the functions and scripts for parsing simulation files (`.json`) into pickled numpy arrays (`.pkl`). \n",
    "\n",
    "---\n",
    "- [WORKSPACE VARIABLES](#workspace-variables)\n",
    "- **[SITE ARCHITECTURE](#site-architecture)**\n",
    "- **[ESTIMATED HEMODYNAMICS](#estimated-hemodynamics)**\n",
    "- **[EXACT HEMODYNAMICS](#exact-hemodynamics)**\n",
    "- **[VASCULAR DAMAGE](#vascular-damage)**\n",
    "- **[VASCULAR FUNCTION](#vascular-function)**\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The main parsing function (`parse_simulations`) iterates through each file in the data path and parses each simulation instance, extracting fields from the simulation setup, cells, and environment.\n",
    "\n",
    "The parsed arrays are organized as:\n",
    "\n",
    "```json\n",
    "{\n",
    "    \"setup\": {\n",
    "        \"radius\": R,\n",
    "        \"height\": H,\n",
    "        \"time\": [],\n",
    "        \"pops\": [],\n",
    "        \"types\": [],\n",
    "        \"coords\": []\n",
    "    },\n",
    "    \"agents\": (N seeds) x (T timepoints) x (H height) x (C coordinates) x (P positions),\n",
    "    \"environments\": {\n",
    "        \"glucose\": (N seeds) x (T timepoints) x (H height) x (R radius)\n",
    "        \"oxygen\": (N seeds) x (T timepoints) x (H height) x (R radius)\n",
    "        \"tgfa\": (N seeds) x (T timepoints) x (H height) x (R radius)\n",
    "    }\n",
    "}\n",
    "```\n",
    "\n",
    "where each entry in the agents array is a structured entry of the shape:\n",
    "\n",
    "```\n",
    "\"pop\"       int8    population code\n",
    "\"type\"      int8    cell type code\n",
    "\"volume\"    int16   cell volume (rounded)\n",
    "\"cycle\"     int16   average cell cycle length (rounded)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `parse.py` file contains general parsing functions.\n",
    "\n",
    "Parsing can take some time, so parsed `.pkl` files for all simulations are provided along with the raw simulation data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scripts.parse import *"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"workspace-variables\"></a>\n",
    "\n",
    "### WORKSPACE VARIABLES \n",
    "<span style=\"float:right;\">[back to top](#toc)</span>\n",
    "\n",
    "Set up workspace variables for parsing simulations.\n",
    "\n",
    "- **`DATA_PATH`** is the path to data files (`.tar.xz` files of compressed simulation outputs)\n",
    "- **`RESULT_PATH`** is the path for result files (`.pkl` files generated by parsing)\n",
    "- **`EXCLUDE`** is a list of seeds to exclude from parsing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "DATA_PATH = \"/path/to/data/files/\"\n",
    "RESULTS_PATH = \"/path/to/result/files/\"\n",
    "EXCLUDE = []"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"site-architecture\"></a>\n",
    "\n",
    "### SITE ARCHITECTURE\n",
    "<span style=\"float:right;\">[back to top](#toc)</span>\n",
    "\n",
    "Parse `SITE_ARCHITECTURE` simulations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "parse_simulations(\"SITE_ARCHITECTURE\", DATA_PATH, RESULTS_PATH, EXCLUDE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"estimated-hemodynamics\"></a>\n",
    "\n",
    "### ESTIMATED HEMODYNAMICS\n",
    "<span style=\"float:right;\">[back to top](#toc)</span>\n",
    "\n",
    "Parse `ESTIMATED_HEMODYNAMICS` simulations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "parse_simulations(\"ESTIMATED_HEMODYNAMICS\", DATA_PATH, RESULTS_PATH, EXCLUDE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"exact-hemodynamics\"></a>\n",
    "\n",
    "### EXACT HEMODYNAMICS\n",
    "<span style=\"float:right;\">[back to top](#toc)</span>\n",
    "\n",
    "Parse `EXACT_HEMODYNAMICS` simulations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "parse_simulations(\"EXACT_HEMODYNAMICS\", DATA_PATH, RESULTS_PATH, EXCLUDE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"vascular-damage\"></a>\n",
    "\n",
    "### VASCULAR DAMAGE\n",
    "<span style=\"float:right;\">[back to top](#toc)</span>\n",
    "\n",
    "Parse `VASCULAR_DAMAGE` simulations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "parse_simulations(\"VASCULAR_DAMAGE\", DATA_PATH, RESULTS_PATH, EXCLUDE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a class=\"anchor\" id=\"vascular-function\"></a>\n",
    "\n",
    "### VASCULAR FUNCTION\n",
    "<span style=\"float:right;\">[back to top](#toc)</span>\n",
    "\n",
    "Parse `VASCULAR_FUNCTION` simulations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "parse_simulations(\"VASCULAR_FUNCTION\", DATA_PATH, RESULTS_PATH, EXCLUDE)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}