Merge pull request #86 from NREL/bsd_cal_tut

Add label to calibration images for normal-beta tutorial and rename calibration doc page
NREL · Jan 16, 2025 · 8ef48ca · 8ef48ca
commit 8ef48ca
Show file tree

Hide file tree

Showing 131 changed files with 8,665 additions and 0 deletions.
diff --git a/.buildinfo b/.buildinfo
@@ -0,0 +1,4 @@
+# Sphinx build info version 1
+# This file records the configuration used when building these files. When it is not found, a full rebuild will be done.
+config: 5323d8be09131c0fa917404abdc54afd
+tags: 645f666f9bcd5a90fca523b33c5a78b7
diff --git a/.nojekyll b/.nojekyll
diff --git a/_images/3dsparger.png b/_images/3dsparger.png
diff --git a/_images/Loss_surr.png b/_images/Loss_surr.png
diff --git a/_images/Loss_surr1.png b/_images/Loss_surr1.png
diff --git a/_images/Surr_cal_2.0_5.0_corner.png b/_images/Surr_cal_2.0_5.0_corner.png
diff --git a/_images/Surr_cal_2.0_5.0_prop.png b/_images/Surr_cal_2.0_5.0_prop.png
diff --git a/_images/Surr_cal_5.0_5.0_corner.png b/_images/Surr_cal_5.0_5.0_corner.png
diff --git a/_images/Surr_cal_5.0_5.0_prop.png b/_images/Surr_cal_5.0_5.0_prop.png
diff --git a/_images/Surr_cal_binary_corner.png b/_images/Surr_cal_binary_corner.png
diff --git a/_images/Surr_cal_binary_prop.png b/_images/Surr_cal_binary_prop.png
diff --git a/_images/Surr_cal_ternary_corner.png b/_images/Surr_cal_ternary_corner.png
diff --git a/_images/Surr_cal_ternary_prop.png b/_images/Surr_cal_ternary_prop.png
diff --git a/_images/Surr_opt_2.0_5.0_corner.png b/_images/Surr_opt_2.0_5.0_corner.png
diff --git a/_images/Surr_opt_2.0_5.0_prop.png b/_images/Surr_opt_2.0_5.0_prop.png
diff --git a/_images/Surr_opt_5.0_5.0_corner.png b/_images/Surr_opt_5.0_5.0_corner.png
diff --git a/_images/Surr_opt_5.0_5.0_prop.png b/_images/Surr_opt_5.0_5.0_prop.png
diff --git a/_images/Surr_opt_binary_corner.png b/_images/Surr_opt_binary_corner.png
diff --git a/_images/Surr_opt_binary_prop.png b/_images/Surr_opt_binary_prop.png
diff --git a/_images/Surr_opt_ternary_corner.png b/_images/Surr_opt_ternary_corner.png
diff --git a/_images/Surr_opt_ternary_prop.png b/_images/Surr_opt_ternary_prop.png
diff --git a/_images/True_cal_2.0_5.0_corner.png b/_images/True_cal_2.0_5.0_corner.png
diff --git a/_images/True_cal_2.0_5.0_prop.png b/_images/True_cal_2.0_5.0_prop.png
diff --git a/_images/True_cal_5.0_5.0_corner.png b/_images/True_cal_5.0_5.0_corner.png
diff --git a/_images/True_cal_5.0_5.0_prop.png b/_images/True_cal_5.0_5.0_prop.png
diff --git a/_images/True_opt_2.0_5.0_corner.png b/_images/True_opt_2.0_5.0_corner.png
diff --git a/_images/True_opt_2.0_5.0_prop.png b/_images/True_opt_2.0_5.0_prop.png
diff --git a/_images/True_opt_5.0_5.0_corner.png b/_images/True_opt_5.0_5.0_corner.png
diff --git a/_images/True_opt_5.0_5.0_prop.png b/_images/True_opt_5.0_5.0_prop.png
diff --git a/_images/cmap_bf_binary.png b/_images/cmap_bf_binary.png
diff --git a/_images/cmap_bf_ternary.png b/_images/cmap_bf_ternary.png
diff --git a/_images/cmap_cr_binary.png b/_images/cmap_cr_binary.png
diff --git a/_images/cmap_cr_ternary.png b/_images/cmap_cr_ternary.png
diff --git a/_images/co2g_cond_mean.png b/_images/co2g_cond_mean.png
diff --git a/_images/early_det.png b/_images/early_det.png
diff --git a/_images/early_uq.png b/_images/early_uq.png
diff --git a/_images/gh_cond_mean.png b/_images/gh_cond_mean.png
diff --git a/_images/loop_react.png b/_images/loop_react.png
diff --git a/_images/schematic.png b/_images/schematic.png
diff --git a/_images/schematicSpargerInlet.png b/_images/schematicSpargerInlet.png
diff --git a/_images/schematicSpargerWalls.png b/_images/schematicSpargerWalls.png
diff --git a/_images/simpleOutput.png b/_images/simpleOutput.png
diff --git a/_images/stirred_tank.png b/_images/stirred_tank.png
diff --git a/_images/succ_fail.png b/_images/succ_fail.png
diff --git a/_images/test_surr.png b/_images/test_surr.png
diff --git a/_sources/acknowledgments.rst.txt b/_sources/acknowledgments.rst.txt
@@ -0,0 +1,6 @@
+Acknowledgments
+=====
+
+This work was authored by the National Renewable Energy Laboratory (NREL), operated by Alliance for Sustainable Energy, LLC, for the U.S. Department of Energy (DOE) under Contract No. DE-AC36-08GO28308. This work was supported by funding from DOE Bioenergy Technologies Office (BETO) `CO2RUe consortium <https://www.energy.gov/eere/co2rue>`_. The research was performed using computational resources sponsored by the Department of Energy's Office of Energy Efficiency and Renewable Energy and located at the National Renewable Energy Laboratory. The views expressed in the article do not necessarily represent the views of the DOE or the U.S. Government. The U.S. Government retains and the publisher, by accepting the article for publication, acknowledges that the U.S. Government retains a nonexclusive, paid-up, irrevocable, worldwide license to publish or reproduce the published form of this work, or allow others to do so, for U.S. Government purposes.
+
+
diff --git a/_sources/bubbleColumn.rst.txt b/_sources/bubbleColumn.rst.txt
@@ -0,0 +1,14 @@
+Bubble column
+=====
+
+This tutorial demonstrates how to run a bubble column reactor case
+
+Running the tutorial
+------------
+
+To be completed ...
+
+
+
+
+
diff --git a/_sources/calibration_bsd.rst.txt b/_sources/calibration_bsd.rst.txt
@@ -0,0 +1,228 @@
+Parameter calibration for bubble size distribution (BSD)
+=====
+
+This tutorial demonstrates how to calibrate bubble dynamics models by scaling the breakup and coalescence rates to match an observed BSD.
+
+Running the tutorial
+------------
+We assume that bird is installed within an environment called ``bird`` (see either the :ref:`Installation section for developers<installation_dev>` or the :ref:`Installation section for users<installation_users>`)
+
+You will need to install additional dependencies
+
+.. code-block:: console
+
+   conda activate bird
+   BIRD_DIR = `python -c "import bird; print(bird.BIRD_DIR)"`
+   cd ${BIRD_DIR}/../
+   pip install .[calibration]
+
+The calibration tutorial is located at ``${BIRD_DIR}/../tutorial_cases/calibration/bsd``
+
+Simple modeling of bubble size distribution (BSD) dynamics
+------------
+
+In this example, the dynamics of the BSD are modeled for a frozen flow field, i.e. where the changes in BSD do not affect the fluid flow variables.
+Each bubble is modeled with a particle which can either be merged with another particle (coalescence) or be broken into multiple particles (breakup).
+
+At every timestep, a number of breakup and coalescence events are determined by the breakup and coalescence rates.
+
+In this example, breakup and coalescence rates are assumed independent of the bubble diameters. While simple, this implementation can result in a breakup or coalescence runaway in the case where breakup and coalescence rates are not balancing each other. In the case of breakup runaway, the number of particles grows monotonically, and in the base of coalescence runaway, the number of particles decreases monotonically. 
+
+For these runaway cases, the simulations are considered `failing` (see specific definition of failing in ``simulation.py``.
+
+At every event, we consider `N-ary` breakup and coalescence. If ``N=2``, then the system models a binary breakup and binary coalescence: at every coalescence event, only two bubbles coalesce; at every breakup event, only two daughter bubbles are created starting from one initial bubble.
+
+
+Calibration Objective
+------------
+
+For all the cases, the simulation initially contains 2000 bubbles all with diameter 1mm.
+
+Two target BSDs are considered:
+
+1. A BSD obtained with a ternary breakup and coalescence with breakup and coalescence rate 0.5 Hz.
+2. A BSD obtained with a binary breakup and coalescence with breakup rate 1.6 Hz and coalescence rate 2.0 Hz
+
+Generate the target data by running
+
+.. code-block:: console
+
+   cd make_dataset
+   bash gen_target.sh
+   cd ..
+
+This creates a folder ``data/`` with the file ``target_n_3_br_0.5_cr_0.5.npz`` and ``target_n_2_br_1.6_cr_2.0.npz``
+
+The numerical model is always a binary breakup and coalescence model with adjustable breakup and coalescence rate. To avoid breakup or coalescence runaway, we ensure that breakup rate is close to the coalescence rate. Specifically, the coalescence rate `Cr` varies in the interval `[0.02, 2]`. A breakup rate factor `Bf` varies in the interval [`0.8, 1.1]` and the breakup rate `Br = Bf Cr`. This ensures that the breakup rate and coalescence rate are close to one another, thereby avoiding breakup and coalescence runaway.
+
+Each simulation (target and from the numerical model) uses a timeste `dt = 0.01s` and the simulations are run for 150s to reach statistical stationarity and allow for sufficient averaging of the BSD. Since each forward simulation is expensive (~30s on a M1 Mac), a surrogate needs to be constructed.
+
+
+Building the surrogate
+------------
+
+The class ``Param_NN`` available in BiRD allows to create a parametric surrogate. Here, the input of the surrogate are the parameters ``beff_fact`` (corresponding to `Bf` as defined above) and ``ceff`` (corresponding to `Cr` as defined above), and the variable ``x`` that parameterize the PDF (the bubble diameter).
+
+Generate dataset
+
+.. code-block:: console
+   
+   cd make_dataset
+   bash gen_dataset.sh
+   cd ..
+
+This generates a file ``data/dataset.npz`` that contains the BSD of 400 simulations. This step may take time (~3h on a M1 Mac). We provide a ``dataset.npz`` that contains precomputed BSD.
+
+Before moving further, we can plot the data to understand how close it is to the target data and whether it is subject to coalescence and breakup runaway.
+
+.. code-block:: console
+   
+   cd data
+   python check_real.py -ter
+   cd ..
+
+The plot below shows the values of `Bf` and `Cr` simulated and whether they lead to runaway. Most of the simulations are successful, but very small and very large values of `Bf` lead to runaway. Those runs are labelled with a dummy PDF in order to instruct the surrogate model to avoid those regions
+
+
+.. container:: figures-succ-fail
+
+   .. figure:: ../assets/calibration/tutorial_bsd/succ_fail.png
+      :width: 50%
+      :align: center
+      :alt: Scatter plot of success and runaway (failed) simulations
+
+
+We can also look at how close the generated data is to the target data. For the ternary breakup and coalescence, there is clearly a discrepancy that is not resolvable by adjusting the coalescence and breakup rates.
+
+|figures-viz-pred-ter_bf| |figures-viz-pred-ter_cr|
+
+.. |figures-viz-pred-ter_bf| image:: ../assets/calibration/tutorial_bsd/cmap_bf_ternary.png
+   :width: 49.5%
+   :alt: Predicted data with the binary breakup and coalescence colored by Bf against the ternary target data
+
+.. |figures-viz-pred-ter_cr| image:: ../assets/calibration/tutorial_bsd/cmap_cr_ternary.png
+   :width: 49.5%
+   :alt: Predicted data with the binary breakup and coalescence colored by Cr against the ternary target data
+
+
+In the case of the binary breakup and coalescence target data, a low value of `Bf` and a high value of `Cr` should lead to a good agreement between the forward model and the target data.
+
+|figures-viz-pred-bin_bf| |figures-viz-pred-bin_cr|
+
+.. |figures-viz-pred-bin_bf| image:: ../assets/calibration/tutorial_bsd/cmap_bf_binary.png
+   :width: 49.5%
+   :alt: Predicted data with the binary breakup and coalescence colored by Bf against the binary target data
+
+.. |figures-viz-pred-bin_cr| image:: ../assets/calibration/tutorial_bsd/cmap_cr_binary.png
+   :width: 49.5%
+   :alt: Predicted data with the binary breakup and coalescence colored by Cr against the binary target data
+
+
+Train a neural net surrogate
+
+.. code-block:: console
+   
+   bash train_surrogate.sh
+
+This will generate a plot of the train and test loss history ``loss.png``. This will also generate ``Modeltmp`` which contains the weights of the trained model and ``Logtmp`` which contains the loss history (shown below).
+
+
+.. container:: figures-loss-bsdcal-surr
+
+   .. figure:: ../assets/calibration/tutorial_bsd/Loss_surr.png
+      :width: 50%
+      :align: center
+      :alt: Loss history
+
+
+Calibration with optimized likelihood uncertainty
+------------
+
+For the calibration, the objective PDF is noisy and subject to statistical uncertainty. We calibrate a likelihood uncertainty that contains both the noise and the missing physics estimates.
+
+Calibrate against the target data obtained with ternary breakup and coalescence 
+
+.. code-block:: console
+   
+   python tut_calibration.py -ter
+
+
+
+|figures-cal-tern-prop| |figures-cal-tern-corn|
+
+.. |figures-cal-tern-prop| image:: ../assets/calibration/tutorial_bsd/Surr_opt_ternary_prop.png
+   :width: 49.5%
+   :alt: Calibrated prediction with the surrogate forward model against ternary target data 
+
+.. |figures-cal-tern-corn| image:: ../assets/calibration/tutorial_bsd/Surr_opt_ternary_corner.png
+   :width: 49.5%
+   :alt: Parameter PDF obtained with the surrogate forward model with ternary target data 
+.. container:: figures-cal-tern
+
+
+Calibrate against the target data obtained with binary breakup and coalescence 
+
+.. code-block:: console
+   
+   python tut_calibration.py
+
+|figures-cal-bin-prop| |figures-cal-bin-corn|
+
+.. |figures-cal-bin-prop| image:: ../assets/calibration/tutorial_bsd/Surr_opt_binary_prop.png
+   :width: 49.5%
+   :alt: Calibrated prediction with the surrogate forward model against binary target data 
+
+.. |figures-cal-bin-corn| image:: ../assets/calibration/tutorial_bsd/Surr_opt_binary_corner.png
+   :width: 49.5%
+   :alt: Parameter PDF obtained with the surrogate forward model with binary target data 
+
+
+Clearly, the amount of missing physics vary depending on the observations and is significantly lower when calibrating against binary breakup and coalescence data.
+
+
+Calibration with calibrated likelihood uncertainty
+------------
+
+The same suite of tests can be done by calibrating the likelihood uncertainty in lieu of optimizing it. This has the advantage of rapid calibration since only one calibration is needed.
+
+Calibrate against the target data obtained with ternary breakup and coalescence
+
+.. code-block:: console
+
+   python tut_calibration.py -ter -cal_err
+
+|figures-cal-tern-calerr-prop| |figures-cal-tern-calerr-corn|
+
+.. |figures-cal-tern-calerr-prop| image:: ../assets/calibration/tutorial_bsd/Surr_cal_ternary_prop.png
+   :width: 49.5%
+   :alt: Calibrated prediction with the surrogate forward model against ternary target data 
+
+.. |figures-cal-tern-calerr-corn| image:: ../assets/calibration/tutorial_bsd/Surr_cal_ternary_corner.png
+   :width: 49.5%
+   :alt: Parameter PDF obtained with the surrogate forward model with ternary target data 
+
+
+
+Calibrate against the target data obtained with binary breakup and coalescence
+
+
+.. code-block:: console
+   
+   python tut_calibration.py -cal_err
+
+
+|figures-cal-bin-calerr-prop| |figures-cal-bin-calerr-corn|
+
+.. |figures-cal-bin-calerr-prop| image:: ../assets/calibration/tutorial_bsd/Surr_cal_binary_prop.png
+   :width: 49.5%
+   :alt: Calibrated prediction with the surrogate forward model against binary target data 
+
+.. |figures-cal-bin-calerr-corn| image:: ../assets/calibration/tutorial_bsd/Surr_cal_binary_corner.png
+   :width: 49.5%
+   :alt: Parameter PDF obtained with the surrogate forward model with binary target data 
+
+
+
+
+
+