From c3d6954dd6a479eb845854cc18b9ead14e4e6b8a Mon Sep 17 00:00:00 2001 From: mecrouch Date: Mon, 6 Jan 2025 15:44:20 -0500 Subject: [PATCH 1/2] Add info on using workflow templates --- docs/modeling/compare-models.md | 2 +- docs/workflows/index.md | 217 +++++++++++++++++++++++++++++++- 2 files changed, 216 insertions(+), 3 deletions(-) diff --git a/docs/modeling/compare-models.md b/docs/modeling/compare-models.md index be732c9..ffb90e4 100644 --- a/docs/modeling/compare-models.md +++ b/docs/modeling/compare-models.md @@ -53,7 +53,7 @@ The Compare model operator has a wizard view and a code notebook: The wizard view compares the selected models along the following lines. For the best results, make sure you've [enriched](review-and-enrich-model.md#enrich-model-metadata) each model before running the comparison. - [Goal-driven comparison overview](#create-a-goal-driven-comparison-overview) -- [Side-by-side model cards](#-get-side-by-side-model-card-comparisons) +- [Side-by-side model cards](#get-side-by-side-model-card-comparisons) - [Concept comparisons](#review-model-concept-comparisons) ### Create a goal-driven comparison overview diff --git a/docs/workflows/index.md b/docs/workflows/index.md index 8b3bbfc..1c8b6d3 100644 --- a/docs/workflows/index.md +++ b/docs/workflows/index.md @@ -8,9 +8,222 @@ A workflow is a visual canvas for building and running complex operations (calib ![Workflow graph with an SIR model and related dataset passed into a calibration operation](../img/workflows/workflow.png) -???+ list "Create a new workflow" +??? list "Create a new blank workflow" - * In the Resources panel, click :fontawesome-solid-plus:{ aria-hidden="true"} **New** in the Workflows section. + 1. In the Resources panel, click :fontawesome-solid-plus:{ aria-hidden="true"} **New** in the Workflows section. + 2. Enter a name for the workflow and click **Create**. + +??? list "Create a new workflow based on a template" + + 1. In the Resources panel, click :fontawesome-solid-plus:{ aria-hidden="true"} **New** in the Workflows section. + 2. Select the template for the type of workflow you want to create. + 3. Enter a name for the workflow, set the required inputs and outputs, and then click **Create**. + + ???+ note + + Before you can use a template, your project must contain the inputs (models, model configurations, intervention policies, or datasets) you want to use. + +## Create new workflows based on templates + +The following workflow templates streamline the process of building common modeling workflows. They provide preconfigured and linked resources and operators tailored to your objectives, such as analyzing uncertainty, forecasting potential outcomes, or comparing intervention strategies. + +???+ note + + - Before you can fill out a template, your project must contain the inputs (models, model configurations, intervention policies, or datasets) you want to use. + - Before you can see the results of a templated workflow, you must configure and run any Calibrate, Simulate, or Compare datasets operators it contains. + +??? list "Situational awareness" + + Use this template to determine what's likely to happen next. For example, you can: + + - Anticipate the arrival of a new variants. + - Evaluate the potential impact of growing vaccine hesitancy and declining Non-Pharmaceutical Interventions (NPIs). + + ### Fill out the Situational awareness template + + To use the Situational awareness template, select the following inputs and outputs: + +
+ + - :material-arrow-collapse-right:{ .lg .middle aria-hidden="true" } __Inputs__ + + --- + + - Model + - Model configuration + - Dataset + + - :material-arrow-expand-right:{ .lg .middle aria-hidden="true" } __Outputs__ + + --- + + - Metrics (model states) of interest + +
+ + ### Complete the Situational awareness workflow + + The new workflow first calibrates the model to historical data to obtain the best estimate of parameters for the present. Then it forecasts the model into the near future. To see the results, you first need to open the Calibrate operator and: + + 1. Map the model variables to the dataset columns. + 2. Run the calibration. + + This creates: + + - Charts comparing the selected model states before and after calibration with observations from the dataset. + - A new model calibrated to the dataset. + +??? list "Sensitivity analysis" + + Use this template to determine which model parameters introduce the most uncertainty in your outcomes of interest. For example, you can explore: + + - Unknown severity of new variant. + - Unknown speed of waning immunity. + + ### Fill out the Sensitivity analysis template + + To use the Situational awareness template, select the following inputs and outputs: + +
+ + - :material-arrow-collapse-right:{ .lg .middle aria-hidden="true" } __Inputs__ + + --- + + - Model + - Model configuration + - One or more uncertain parameters of interest and the ranges to explore + + - :material-arrow-expand-right:{ .lg .middle aria-hidden="true" } __Outputs__ + + --- + + - Metrics (model states) of interest + +
+ + ### Complete the Sensitivity analysis workflow + + The new workflow first configures the model with parameter distributions that reflect all the sources of uncertainty. Then it simulates the model into into near future. To see the results, you first need to open the Simulate operator, edit any settings, and run it. This creates: + + - A sensitivity analysis chart for each selected model state and pairwise comparison charts for each selected parameter. + - A simulation results dataset. + +??? list "Decision making" + + Use this template to determine the impact of different interventions. For example, you can find: + + - The impact of several combinations of vaccination and Non-Pharmaceutical Interventions (NPIs) levels. + - Whether it's better to implement an intervention in all locations, select locations, or not at all. + + ### Fill out the Decision making template + + To use the Decision making template, select the following inputs and outputs: + +
+ + - :material-arrow-collapse-right:{ .lg .middle aria-hidden="true" } __Inputs__ + + --- + + - Model + - Model configuration + - One or more intervention policies + + - :material-arrow-expand-right:{ .lg .middle aria-hidden="true" } __Outputs__ + + --- + + - Metrics (model states) of interest + +
+ + ### Complete the Decision making workflow + + The new workflow first runs simulations for the baseline (no intervention) and each intervention policy. It then compares the relative impact of each intervention policy to the baseline. To see the results, you first need to: + + 1. Open and run each Simulate operator. + 2. Open and run the Compare datasets operator. + + This creates a comparison of the simulated baseline and intervention policies. + +??? list "Horizon scanning" + + Use this template to determine how extreme scenarios impact the outcome of different interventions. For example, you can explore: + + - Potential emergence of a new variant. + - Rapidly waning immunity. + + ### Fill out the Horizon scanning template + + To use the Horizon scanning template, select the following inputs and outputs: + +
+ + - :material-arrow-collapse-right:{ .lg .middle aria-hidden="true" } __Inputs__ + + --- + + - Model + - One or more intervention policies + - Model configuration + - One or more uncertain parameters of interest and the ranges to explore + + - :material-arrow-expand-right:{ .lg .middle aria-hidden="true" } __Outputs__ + + --- + + - Metrics (model states) of interest + +
+ + ### Complete the Horizon scanning workflow + + The new workflow first configures the model to represent the extremes of uncertainty for some parameters. It then simulates into the near future with different intervention policies and compares the outcomes. To see the results, you first need to: + + 1. Open and run each Simulate operator + 2. Open and run the Compare datasets operator. + + This creates a comparison of the simulated extreme scenarios. + +??? list "Value of information" + + Use this template to determine how uncertainty impacts the outcomes of different interventions. For example, you can determine whether: + + - Uncertainty in severity changes the priority of which group to target for vaccination. + - Disease severity impacts the outcome of different social distancing policies. + + ### Fill out the Value of information template + + To use the Value of information template, select the following inputs and outputs: + +
+ + - :material-arrow-collapse-right:{ .lg .middle aria-hidden="true" } __Inputs__ + + --- + + - Model + - One or more intervention policies + - Model configuration + - One or more uncertain parameters of interest and the ranges to explore + + - :material-arrow-expand-right:{ .lg .middle aria-hidden="true" } __Outputs__ + + --- + + - Metrics (model states) of interest + +
+ + ### Complete the Value of information workflow + + The new workflow first configures the model with parameter distributions that reflect all the sources of uncertainty. It then simulates into the near future with different intervention policies. To see the results, you first need to: + + 1. Open and run each Simulate operator + 2. Open and run the Compare datasets operator. + + This creates a comparison of the uncertainty across the different interventions. ## Add resources and operators to a workflow From d291be53537cc22904916c0c9d0bfd291da923f0 Mon Sep 17 00:00:00 2001 From: mecrouch Date: Mon, 6 Jan 2025 16:00:11 -0500 Subject: [PATCH 2/2] Remove collapsed headings from TOC --- docs/workflows/index.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/docs/workflows/index.md b/docs/workflows/index.md index 1c8b6d3..41b1ca3 100644 --- a/docs/workflows/index.md +++ b/docs/workflows/index.md @@ -39,7 +39,7 @@ The following workflow templates streamline the process of building common model - Anticipate the arrival of a new variants. - Evaluate the potential impact of growing vaccine hesitancy and declining Non-Pharmaceutical Interventions (NPIs). - ### Fill out the Situational awareness template +

Fill out the Situational awareness template

To use the Situational awareness template, select the following inputs and outputs: @@ -61,7 +61,7 @@ The following workflow templates streamline the process of building common model - ### Complete the Situational awareness workflow +

Complete the Situational awareness workflow

The new workflow first calibrates the model to historical data to obtain the best estimate of parameters for the present. Then it forecasts the model into the near future. To see the results, you first need to open the Calibrate operator and: @@ -80,7 +80,7 @@ The following workflow templates streamline the process of building common model - Unknown severity of new variant. - Unknown speed of waning immunity. - ### Fill out the Sensitivity analysis template +

Fill out the Sensitivity analysis template

To use the Situational awareness template, select the following inputs and outputs: @@ -102,7 +102,7 @@ The following workflow templates streamline the process of building common model - ### Complete the Sensitivity analysis workflow +

Complete the Sensitivity analysis workflow

The new workflow first configures the model with parameter distributions that reflect all the sources of uncertainty. Then it simulates the model into into near future. To see the results, you first need to open the Simulate operator, edit any settings, and run it. This creates: @@ -116,7 +116,7 @@ The following workflow templates streamline the process of building common model - The impact of several combinations of vaccination and Non-Pharmaceutical Interventions (NPIs) levels. - Whether it's better to implement an intervention in all locations, select locations, or not at all. - ### Fill out the Decision making template +

Fill out the Decision making template

To use the Decision making template, select the following inputs and outputs: @@ -138,7 +138,7 @@ The following workflow templates streamline the process of building common model - ### Complete the Decision making workflow +

Complete the Decision making workflow

The new workflow first runs simulations for the baseline (no intervention) and each intervention policy. It then compares the relative impact of each intervention policy to the baseline. To see the results, you first need to: @@ -154,7 +154,7 @@ The following workflow templates streamline the process of building common model - Potential emergence of a new variant. - Rapidly waning immunity. - ### Fill out the Horizon scanning template +

Fill out the Horizon scanning template

To use the Horizon scanning template, select the following inputs and outputs: @@ -177,7 +177,7 @@ The following workflow templates streamline the process of building common model - ### Complete the Horizon scanning workflow +

Complete the Horizon scanning workflow

The new workflow first configures the model to represent the extremes of uncertainty for some parameters. It then simulates into the near future with different intervention policies and compares the outcomes. To see the results, you first need to: @@ -193,7 +193,7 @@ The following workflow templates streamline the process of building common model - Uncertainty in severity changes the priority of which group to target for vaccination. - Disease severity impacts the outcome of different social distancing policies. - ### Fill out the Value of information template +

Fill out the Value of information template

To use the Value of information template, select the following inputs and outputs: @@ -216,7 +216,7 @@ The following workflow templates streamline the process of building common model - ### Complete the Value of information workflow +

Complete the Value of information workflow

The new workflow first configures the model with parameter distributions that reflect all the sources of uncertainty. It then simulates into the near future with different intervention policies. To see the results, you first need to: