Skip to content

Commit

Permalink
Ascent: Jupyter Replay Workflow (ECP-WarpX#1473)
Browse files Browse the repository at this point in the history 
Document an efficient workflow to quickly iterate on an
`ascent_actions.yaml` design.
  • Loading branch information
@ax3l
ax3l authored Oct 26, 2020
1 parent 1140e9f commit 51f5af9
Show file tree
Hide file tree
Showing 2 changed files with 333 additions and 3 deletions.
26 changes: 23 additions & 3 deletions Docs/source/visualization/ascent.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -286,10 +286,30 @@ Workflow
.. note::

This section is in-progress.
It will describe how to quickly create and iterate on ``ascent_actions.yaml`` files before running at scale.
TODOs: finalize acceptance testing; update 3D LWFA example

TODOs: finalize acceptance testing; update 3D LWFA example; share Jupyter notebook file
In the preparation of simulations, it is generally useful to run small, under-resolved versions of the planned simulation layout first.
Ascent replay is helpful in the setup of an in situ visualization pipeline during this process.
In the following, a Jupyter-based workflow is shown that can be used to quickly iterate on the design of a ``ascent_actions.yaml`` file, repeatedly rendering the same (small) data.

First, run a small simulation, e.g. on a local computer, and create conduit blueprint files (see above).
Second, copy the Jupyter Notebook file :download:`ascent_replay_warpx.ipynb <../../../Tools/Ascent/ascent_replay_warpx.ipynb>` into the simulation output directory.
Third, download and start a Docker container with a prepared Jupyter installation and Ascent Python bindings from the simulation output directory:

.. code-block:: bash
docker run -v$PWD:/home/user/ascent/install-debug/examples/ascentutorial/ascent_intro/notebooks/replay -p 8000:8000 -p 8888:8888 -p 9000:9000 -p 10000:10000 -t -i alpinedav/ascent-jupyter:latest
docker pull alpinedav/ascent-jupyter:latest
docker run -v$PWD:/home/user/ascent/install-debug/examples/ascent/tutorial/ascent_intro/notebooks/replay -p 8000:8000 -p 8888:8888 -p 9000:9000 -p 10000:10000 -t -i alpinedav/ascent-jupyter:latest
Now, access Jupyter Lab via: http://localhost:8888/lab (password: ``learn``).

Inside the Jupyter Lab is a ``replay/`` directory, which mounts the outer working directory.
You can now open ``ascent_replay_warpx.ipynb`` and execute all cells.
The last two cells are the replay action that can be quickly iterated: change ``replay_actions.yaml`` cell and execute both.

.. note::

* Keep an eye on the terminal, if a replay action is erroneous it will show up on the terminal that started the docker container.
(TODO: We might want to catch that inside python and print it in Jupyter instead.)
* If you remove a "key" from the replay action, you might see an error in the ``AscentViewer``.
Restart and execute all cells in that case.
310 changes: 310 additions & 0 deletions Tools/Ascent/ascent_replay_warpx.ipynb
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,310 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Replay Data\n",
"This notebook allows quick playing with Ascent actions on created \"conduit blueprint data\". See Ascent's [Extracts](https://ascent.readthedocs.io/en/latest/Actions/Extracts.html) docs for deeper details on Extracts."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import conduit\n",
"import conduit.blueprint\n",
"import conduit.relay\n",
"import ascent\n",
"\n",
"from IPython.display import Image\n",
"\n",
"import glob"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#print(conduit.about())"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# show existing conduit blueprint files\n",
"sim = \"./\"\n",
"glob.glob(sim + \"*root\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"# VisIt 2.13 or newer, when built with Conduit support, can visualize meshes from these files.\n",
"# Look at the Blueprint HDF5 extract with VisIt\n",
"#!visit -o conduit_blueprint.cycle_000400.root"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Prepare Data\n",
"Initialize Ascent and Load a Blueprint File"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# this step can take a while if the simulation used high resolution\n",
"data = conduit.Node()\n",
"conduit.relay.io.blueprint.read_mesh(data, sim + \"conduit_blueprint.cycle_000400.root\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# we publish this data now once to visualize it in multiple ways below\n",
"a = ascent.Ascent()\n",
"opts = conduit.Node()\n",
"opts[\"actions_file\"] = \"\" # work-around: ascent_actions.yaml file must not overwrite our replay action\n",
"#opts[\"exceptions\"] = \"forward\"\n",
"a.open(opts)\n",
"a.publish(data)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def rerender():\n",
" \"\"\"This is reads an updated Ascent Action and rerenders the image\"\"\"\n",
" actions = conduit.Node()\n",
" actions.load(\"replay_actions.yaml\")\n",
" a.execute(actions)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Relay Example\n",
"The next two cells are as follows:\n",
"1. modify the Ascent action (examples: https://ascent.readthedocs.io/en/latest/Actions/Examples.html)\n",
"2. rerender\n",
"\n",
"By only modifying and executing only those two cells one can quickly iterate over new visualizations."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%%writefile replay_actions.yaml\n",
"\n",
"# this block are data pipelines\n",
"# each entry in pipelines: executes a series of functions from top to bottom,\n",
"# results of prior functions can be used in later calls of the same pipeline\n",
"-\n",
" action: \"add_pipelines\"\n",
" pipelines:\n",
" slice_field:\n",
" f1:\n",
" type: \"slice\"\n",
" params:\n",
" topology: topo\n",
" point: {x: 0.0, y: 0.0, z: 0.0}\n",
" normal: {x: 1.0, y: 1.0, z: 0.0}\n",
" \n",
" sampled_particles:\n",
" f1:\n",
" type: histsampling\n",
" params:\n",
" field: particle_electrons_uz\n",
" bins: 64\n",
" sample_rate: 0.90\n",
" f2:\n",
" type: \"clip\"\n",
" params:\n",
" topology: particle_electrons # particle data\n",
" multi_plane:\n",
" point1: {x: 0.0, y: 0.0, z: 0.0}\n",
" normal1: {x: 1.0, y: 0.0, z: 0.0}\n",
" point2: {x: 0.0, y: 0.0, z: 0.0}\n",
" normal2: {x: 0.7, y: -0.7, z: 0.0}\n",
"\n",
" clipped_volume:\n",
" f0:\n",
" type: \"contour\"\n",
" params:\n",
" field: \"Ez\"\n",
" levels: 10\n",
" f1:\n",
" type: \"clip\"\n",
" params:\n",
" topology: topo # name of the amr mesh\n",
" multi_plane:\n",
" point1: {x: 0.0, y: 0.0, z: 0.0}\n",
" normal1: {x: 1.0, y: 0.0, z: 0.0}\n",
" point2: {x: 0.0, y: 0.0, z: 0.0}\n",
" normal2: {x: 0.3, y: -0.3, z: 0.0}\n",
"\n",
" mag_clipped_volume_E:\n",
" f0: \n",
" type: \"composite_vector\"\n",
" params: \n",
" field1: \"Ex\"\n",
" field2: \"Ey\"\n",
" field3: \"Ez\"\n",
" output_name: \"E_vec\"\n",
" f1: \n",
" type: \"vector_magnitude\"\n",
" params: \n",
" field: \"E_vec\"\n",
" output_name: \"E_mag\"\n",
" f2:\n",
" type: \"contour\"\n",
" params:\n",
" field: \"E_mag\"\n",
" levels: 4\n",
" f3:\n",
" type: \"clip\"\n",
" params:\n",
" topology: topo # name of the amr mesh\n",
" multi_plane:\n",
" point1: {x: 0.0, y: 0.0, z: 0.0}\n",
" normal1: {x: 1.0, y: 0.0, z: 0.0}\n",
" point2: {x: 0.0, y: 0.0, z: 0.0}\n",
" normal2: {x: 0.7, y: -0.7, z: 0.0}\n",
"\n",
"# A scene is describes the things to be rendered and how\n",
"# here one selects either data directly or data that goes into a data pipeline;\n",
"# then, the result gets represented with a visualization \"type\" and according\n",
"# parameters such as colorbars and color ranges (aka transfer functions)\n",
"# reference: https://ascent.readthedocs.io/en/latest/Actions/Scenes.html\n",
"# color tables: https://ascent.readthedocs.io/en/latest/Actions/VTKmColorTables.html\n",
"-\n",
" action: \"add_scenes\"\n",
" scenes:\n",
" scene1:\n",
" plots:\n",
"# p0:\n",
"# type: \"pseudocolor\"\n",
"# field: \"particle_electrons_uz\"\n",
"# pipeline: \"sampled_particles\"\n",
" p1:\n",
" type: \"pseudocolor\"\n",
" field: \"Ez\"\n",
" pipeline: \"clipped_volume\"\n",
"# color_table: \n",
"# name: \"plasma\"\n",
"# reverse: true\n",
"# control_points: \n",
"# - \n",
"# type: \"alpha\"\n",
"# position: 0.0\n",
"# alpha: 0.5\n",
"# - \n",
"# type: \"alpha\"\n",
"# position: 1.0\n",
"# alpha: 0.5\n",
"# p2:\n",
"# type: \"pseudocolor\"\n",
"# field: \"E_mag\"\n",
"# pipeline: \"mag_clipped_volume_E\"\n",
"# color_table: \n",
"# name: \"Black-Body Radiation\"\n",
"# reverse: true\n",
" #p3:\n",
" # type: \"pseudocolor\"\n",
" # field: \"Ez\"\n",
" # pipeline: \"slice_field\"\n",
" # min_value: -3.0e11\n",
" # max_value: 3.0e11\n",
" renders:\n",
" image1:\n",
" image_width: 512\n",
" image_height: 512\n",
" bg_color: [1.0, 1.0, 1.0]\n",
" fg_color: [0.0, 0.0, 0.0]\n",
" image_prefix: \"./replay_%06d\"\n",
" camera:\n",
" azimuth: -70\n",
" elevation: 30\n",
" zoom: 1.5"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"rerender()\n",
"ascent.jupyter.AscentViewer(a).show()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#!ls *png"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Image(filename=\"lwfa_Ex_e-uz_000400.png\")\n",
"# print(conduit.about())"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.3"
}
},
"nbformat": 4,
"nbformat_minor": 4
}

0 comments on commit 51f5af9

Please sign in to comment.