01_Introduction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Introduction to Modern, Web-based Methodologies"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Learning objectives\n",
    "\n",
    "- What are modern, web-based image analysis methods?\n",
    "- Why are web-based methods relevant for large images and reproducibility?\n",
    "- How does open source software fit into the computational ecosystem? "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a href=\"https://colab.research.google.com/github/thewtex/modern-insights-from-microscopy-images/blob/master/01_Introduction.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Modern, web-based image analysis and visualization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What is modern, web-based imaging analysis and visualization?\n",
    "\n",
    "1. The user interface is **web browser**\n",
    "2. Defacto communication on the internet with the **HTTPS** protocol\n",
    "3. Computation can happen in the **cloud**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Important Technologies"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Evergreen browsers\n",
    "\n",
    "![Evergreen browsers](./images/evergreen-browsers.jpg)\n",
    "\n",
    "An **evergreen browser** releases frequently, striving for up-to-date and comprehensive support for *[web standards](https://en.wikipedia.org/wiki/Web_standards)*.\n",
    "\n",
    "Modern evergreen browsers include:\n",
    "\n",
    "- Google Chrome\n",
    "- Mozilla Firefox\n",
    "- Microsoft Edge\n",
    "- Opera\n",
    "- Apple Safari\n",
    "\n",
    "Note that Chrome, Edge, and Opera are all based on the same open source Chromium foundation. Safari often lags behind or is limited in standard support. Internet Explorer is no longer supported by Microsoft as of August 17, 2021, but lacks support for many Modern Web standard features."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As modern web standards evolve, how do you know what features are available to your potential users?\n",
    "\n",
    "<center><strong><a href=\"https://caniuse.com\">CanIUse.com</a></strong><center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Programming Languages"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Client-side programming languages\n",
    "\n",
    "![JavaScript](./images/js_js_js.png)\n",
    "\n",
    "Client-side programming languages are languages that run web browser page."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**JavaScript (JS)** is the *language of the web.* A working knowledge of JS is [a useful skill](https://www.youtube.com/watch?v=dFUlAQZB9Ng).\n",
    "\n",
    "![JS](./images/js.png)\n",
    "\n",
    "- JavaScript is the only language with ubiquitous support across modern web browsers.\n",
    "- Modern JavaScript runtimes are highly engineered and very performant.\n",
    "- JavaScript can also executed server side -- [*Node.js*](https://nodejs.org/en/) is the most popular server-side runtime.\n",
    "- JavaScript has one of the largest software package ecosystems: the [Node package manager, NPM, the npm registry, and the npm CLI](https://www.npmjs.com/).\n",
    "- The JavaScript language standard, [ECMAScript](https://en.wikipedia.org/wiki/ECMAScript), is a rapidly evolving, modern language. The most recent version is ES2020.\n",
    "- Most client-side code deployed on websites, whether written in JS or another language, is transpiled down to ECMAScript version 5 with Node.js.\n",
    "- The [Mozilla Developer Network (MDN)](https://developer.mozilla.org/en-US/docs/Web/JavaScript) is the best place to find JavaScript-related documentation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**TypeScript (TS)** is a superset of JavaScript that adds optional static type definitions.\n",
    "\n",
    "![TS](./images/ts.svg)\n",
    "\n",
    "- Typescript is a popular alternative to JavaScript for writing client-side code.\n",
    "- When Typescript is transpiled to JavaScript for deployment, static type checking occurs.\n",
    "- In addition to compilation error checking, explicit types in interfaces and other typed language advantages are available, e.g. IDE features.\n",
    "- Many language features that used to be unique to TypeScript, e.g. classes, have now been adopted into the JavaScript language standard."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Other languages can be compiled into **WebAssembly (Wasm)**, a portable compilation target enabling deployment on the web for client and server applications.\n",
    "\n",
    "![Wasm](./images/wasm.png)\n",
    "\n",
    "- WebAssembly is efficient and fast.\n",
    "- Wasm can be transmitted in a binary format.\n",
    "- Wasm supports hardware capabilities such as [atomics](https://courses.cs.washington.edu/courses/cse378/07au/lectures/L25-Atomic-Operations.pdf) and [SIMD](https://en.wikipedia.org/wiki/SIMD).\n",
    "- Wasm focuses on secure, memory-safe sandboxed execution in the browser.\n",
    "- Wasm runs on the same virtual machine as JavaScript, and it is debuggable in browsers.\n",
    "- Wasm is part of the open web platform, supported by all major browsers.\n",
    "- Wasm aims to be backwards compatible.\n",
    "- A JavaScript API is provided ito interface with WebAssembly.\n",
    "\n",
    "Performant image processing code can be written in the following languages, and compiled to Wasm:\n",
    "\n",
    "- C/C++ via [Emscripten](https://emscripten.org/)\n",
    "- Rust\n",
    "- Java"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**WASI**: WebAssembly beyond the browser!\n",
    "\n",
    "![WASI](https://wasi.dev/polyfill/WASI-small.png)\n",
    "\n",
    "[WASI (The WebAssembly System Interface)](https://wasi.dev/) brings WebAssembly beyond the browser by providing access to local filesystems, etc. while maintaining [a focus on *security* and *portability*](https://github.com/bytecodealliance/wasmtime/blob/main/docs/WASI-overview.md).\n",
    "\n",
    "![WASI architecture](https://github.com/bytecodealliance/wasmtime/raw/main/docs/wasi-software-architecture.png)\n",
    "\n",
    "There are [*many* WASI runtimes available](https://github.com/appcypher/awesome-wasm-runtimes) to run WASI WebAssembly:\n",
    "\n",
    "- Across hardware architectures, including even FPGA's\n",
    "- Interfaces with programming languages, Python, Java, C, C++, Rust, JavaScript, and more\n",
    "- Targets for execution on the command line, serverless edge environments, next-generation Web3 infrastructure, etc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " Other browser **web standards** *important for scientific imaging* include: \n",
    " \n",
    "![WebGL](./images/webgl.svg)\n",
    "\n",
    "- **[WebGL](https://www.khronos.org/webgl/)**: a *web standard for a low-level 3D graphics API based on OpenGL ES, exposed to ECMAScript via the HTML5 Canvas.*\n",
    "- **[WebGPU](https://gpuweb.github.io/gpuweb/)**: *an API that exposes the capabilities of GPU hardware for the Web.* The API is designed from the ground up to efficiently map to the Vulkan, Direct3D 12, and Metal native GPU APIs. WebGPU is not related to WebGL and does not explicitly target OpenGL ES. Includes [compute kernel support](https://web.dev/gpu-compute/) in addition to rendering shaders.\n",
    "- **Web Worker's**: Runs JavaScript in background threads separate from a web page's main thread. Useful for parallelism.\n",
    "- **ServiceWorker's**: another *worker* that acts as a proxy between the web browser page and the network. It can be used to *cache assets* to provide fast web apps with *offline* functionality."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notable scientific client-side language bridges!\n",
    "\n",
    "- [itk-wasm](https://wasm.itk.org/) (C++)\n",
    "- [PyScript](https://pyscript.net/) (Python)\n",
    "- [JupyterLite](https://jupyterlite.readthedocs.io/en/latest/) (Python and More)\n",
    "- [ImageJ.js](https://ij.imjoy.io/) (Java)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Server-side programming languages for web-based imaging include:\n",
    "\n",
    "- *Python*\n",
    "- Java\n",
    "- Rust\n",
    "- Julia\n",
    "- JavaScript\n",
    "- C++"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Web-based storage\n",
    "\n",
    "Discussed in the [data storage](./04_Data_Storage.ipynb) tutorial."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Cloud compute\n",
    "\n",
    "Discussed in the [distributed image processing](./05_Distributed_Processing.ipynb) tutorial."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Communication\n",
    "\n",
    "Communication web standard for scientific imaging include:\n",
    "\n",
    "- **REST**: *[Representational state transfer (REST)](https://en.wikipedia.org/wiki/Representational_state_transfer) is a software architectural style that defines a set of constraints to be used for creating Web services. HTTP-based RESTful APIs are defined with the following aspects: a base URI, such as http://api.example.com/, standard HTTP methods (e.g., GET, POST, PUT, and DELETE),\n",
    "and, a media type that defines state transition data element.*\n",
    "- **WebSockets** - a two-way communication channel that is more performant than HTTP requests. Used by Jupyter, Colab, etc.\n",
    "- **[IPFS](https://en.wikipedia.org/wiki/InterPlanetary_File_System)** - *The InterPlanetary File System (IPFS) is a protocol and peer-to-peer network for storing and sharing data in a distributed file system.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Web-based methods, large images, and reproducibility"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Why are web-based methods uniquely appropriate for working with extremely large images?**\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Natural approach to remote storage\n",
    "    - Helpful when data needs to be stored near the microscope or in the *cloud*\n",
    "    - **Warning**: however, data-compute locality is often critical for performance!\n",
    "- Partial image chunks can be fetched on demand for analysis or visualization\n",
    "- Compression is a natural and commonplace component"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Why do web-based methods uniquely support open science and reproducibility?**\n",
    "\n",
    "- Truely write once, run everywhere\n",
    "- Backwards and forwards compatibility\n",
    "- A standards-based ecosystem\n",
    "- Distributed compute and storage resources, which improves sustainability     "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "        <iframe\n",
       "            width=\"800\"\n",
       "            height=\"1000\"\n",
       "            src=\"https://insightsoftwareconsortium.github.io/ITKAnisotropicDiffusionLBR/\"\n",
       "            frameborder=\"0\"\n",
       "            allowfullscreen\n",
       "            \n",
       "        ></iframe>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.lib.display.IFrame at 0x7f8348128f70>"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# A C++ image denoising method compiled to JavaScript over five years ago.\n",
    "#\n",
    "# No maintenance required, hosted on free resources, and executed by the client.\n",
    "\n",
    "import IPython\n",
    "url = 'https://insightsoftwareconsortium.github.io/ITKAnisotropicDiffusionLBR/'\n",
    "width = 800\n",
    "height = 1000\n",
    "IPython.display.IFrame(url, width, height)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Modern methods and traditional open source imaging software\n",
    "\n",
    "**How to modern web-based methods extend and interface with traditional open source scientific imaging software?**\n",
    "\n",
    "\n",
    "- **ImJoy**\n",
    "    - [ImJoy](https://imjoy.io/docs/#/) is a plugin-powered hybrid computing platform for deploying deep learning applications such as advanced image analysis tools.\n",
    "    - JavaScript or Python-based plugins.\n",
    "    - Read [the paper](https://rdcu.be/bYbGO).\n",
    "- [ImageJ.js](https://ij.imjoy.io/) - ImageJ compiled to WebAssembly and exposed with ImJoy.  \n",
    "- [itk-wasm](https://wasm.itk.org) - itk-wasm combines ITK and WebAssembly to enable high-performance spatial analysis in a web browser, Node.js, and reproducible execution across programming languages and hardware architectures.\n",
    "- [pyodide](https://github.com/iodide-project/pyodide/): CPython scientific Python libraries, e.g. scikit-image, compiled to WebAssembly.\n",
    "- [PyScript](https://pyscript.net/) - PyScript is a framework that allows users to create rich Python applications in the browser using HTML’s interface. \n",
    "- [Jupyter](https://jupyter.org/): browser-based literate programming, combining interactive code, equations, and visualizations\n",
    "    - [JupyterLite](https://jupyterlite.readthedocs.io/en/latest/) - JupyterLite is a JupyterLab distribution that runs entirely in the browser built from the ground-up using JupyterLab components and extensions.\n",
    "    - [JupyterLab](https://jupyterlab.readthedocs.io/en/stable/): next generation browser interface, more JavaScript focus.\n",
    "    - [Colab](https://colab.research.google.com/notebooks/intro.ipynb#recent=true): alternative Jupyter notebook interface with GPGPU hardware backends for deep learning and Google integrations. \n",
    "    - [Voila](https://github.com/voila-dashboards/voila): quickly turn Jupyter notebooks into standalone web applications."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exercise: Learn about the web-browser development environment!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 1: JavaScript Hello World!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "console.log('Hello web world!')\n"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%%javascript\n",
    "console.log('Hello web world!')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After running the above cell, you just ran your first JavaScript *Hello World!* program -- congratulations!\n",
    "\n",
    "However, where is the output? We can find out by exploring the amazing integrated development environment bundled with modern web browsers.\n",
    "\n",
    "**To access Console in the brower's Developer Tools:** \n",
    "\n",
    "From the Chrome browser: Menu -> More tools -> Developer Tools (or CTRL + Shift + I)\n",
    "\n",
    "From the Firefox browser: Menu > Web Developer > Toggle Tools (or CTRL + Shift + I)\n",
    "\n",
    "- *Find the Hello World output*\n",
    "- *Use the console repl to:*\n",
    "   - *Print a new hello world message*\n",
    "   - *Create a [JavaScript Array](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array) with two elements, 3.0 and 8.0*\n",
    "   - *Create a [JavaScript Float32Array](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Float32Array) with two elements, 3.0 and 8.0* "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 2: The elements inspector\n",
    "\n",
    "The elements inspector tab is to the left of the Console tab.\n",
    "\n",
    "- *Use the elements inspector to select elements for inspection by:*\n",
    "  - *Clicking an element in the developers tools*\n",
    "  - *Click the \"Select an element\" button (to the left of the elements inspector tab) and clicking on an element in the page*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 3: The debugger\n",
    "\n",
    "Click on the *Sources* or *Debugger* tab to open up the debugger.\n",
    "\n",
    "Click on the *01_Introduction.ipynb* file. Scroll down to a `<script>` element.\n",
    "\n",
    "Click on a line number to set a breakpoint.\n",
    "\n",
    "Refresh the page. Use the debugger tools to step through the JavaScript code, and inspect the call stack and  variables."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Bonus Exercise\n",
    "\n",
    "Test the Network, Performance, and Memory developers tools."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}