diff --git a/docs/models/configure-model.md b/docs/config-and-intervention/configure-model.md similarity index 55% rename from docs/models/configure-model.md rename to docs/config-and-intervention/configure-model.md index b867f83..75b92c7 100644 --- a/docs/models/configure-model.md +++ b/docs/config-and-intervention/configure-model.md @@ -1,13 +1,21 @@ --- -title: "Configure a model" +title: "Create and validate a model configuration" --- -# Configure a model +# Configure and validate a model -Before you can simulate a model, you need to configure it to set the initial values and parameters for the condition you want to test. +Before you can simulate a model, you need to configure it to set the initial values and parameters for the condition you want to test. You can use the Configure model operator to: + +- Select from previously saved configurations used in the current project. +- Automatically extract a configuration from an optional document or dataset. +- Manually enter values and account for uncertainty in any downstream simulations. ![Model configurations table showing the initial conditions and parameters for a default configuration](../img/models/configurations.png) +## Create a model configuration + +### Configure model operator + The Configure model operator takes a model as an input and outputs a model configuration. You can manually enter the variable and parameter values for the output configuration or automatically extract them from an optional document or dataset (such as a contact matrix or initial populations) input. ![](../img/models/model-configure-operator.png) @@ -30,26 +38,24 @@ The Configure model operator takes a model as an input and outputs a model confi -??? list "To configure a model" +???+ list "To configure a model" 1. Add the model to a workflow graph. 2. (Optional) Add a document or dataset from which you want to extract configuration values. - 2. Right-click anywhere on the workflow graph and select **Work with model** > **Configure model**. - 3. Connect the Model, Document, and Dataset operator outputs to the corresponding inputs on the Configure model operator. + 2. Right-click anywhere on the workflow graph and select **Config & Intervention** > **Configure model**. + 3. Connect the Model and any Document and Dataset operator outputs to the corresponding inputs on the Configure model operator. 4. On the Configure model operator, click **Open**. 5. Choose or edit the model configuration: === "Edit or create a model configuration" - 1. In the Context section, enter a name for the new configuration. - 2. Scroll down to the Initial variable values and Parameters sections. - 3. Edit any of the variable or parameter values and, if needed, describe the source that informed your changes. + 1. Scroll down to the Initials and Parameters sections and edit any of the values. If needed, describe the source that informed your changes. + To change the type, use the dropdown list to select one of the following options: - **Constant**: Enter a single value and (optionally) increase/decrease it by a certain percent. - - **Distribution**: Enter a minimum and maximum value. - - **Time varying**: Enter a comma-separated list of values at different time steps in the format *timestep:value* (e.g., *0:25, 1:26, 2:27*). - + To edit matrix values, click :fontawesome-solid-table-cells-large:{ aria-hidden="true" } **Matrix** or click :fontawesome-solid-greater-than:{ alt="Expand" title="Expand" } at the beginning of the row. - 4. Repeat step c for each initial condition or parameter you want to change. + - **Uniform**: Enter a minimum and maximum value. + + To edit matrix values, click :fontawesome-solid-greater-than:{ alt="Expand" title="Expand" } at the beginning of the row and then click :fontawesome-solid-table-cells-large:{ aria-hidden="true" } **Open matrix**. + 2. Repeat for each initial condition or parameter you want to change. + 3. To save the configuration for re-use, click **Save** to update the selected configuration or **Save as** to create a new configuration. === "Extract configurations from inputs" @@ -57,20 +63,27 @@ The Configure model operator takes a model as an input and outputs a model confi Extractions work best when the parameters in the inputs have descriptions and concepts. - 1. Click :fontawesome-solid-arrow-right-from-bracket:{ aria-hidden="true"} **Extract configurations from inputs** to attempt to extract variable and parameter values from any attached documents or datasets. + 1. Click **Extract from inputs** to attempt to extract variable and parameter values from any attached documents or datasets. 2. Edit any of the variable or parameter values as needed. === "Choose an existing model configuration" - 1. Review the Suggested configurations section. Click a configuration name to review its values. - 2. Click **Apply configuration values** to choose an existing configuration as the output for this operator. - - - 6. Click :material-play-outline:{ aria-hidden="true" } **Run**. + - Review the Configurations on the left. Click a configuration name to select it and review its values. + +### Use the model configuration output + +With the configure model output, you can: + +- [Edit a model](../modeling/edit-model.md) +- [Stratify a model](../modeling/stratify-model.md) +- [Validate a model configuration](#validate-a-model-configuration) +- [Simulate a model](../simulation/simulate-model.md) +- [Calibrate a model configuration](../simulation/calibrate-model.md) +- [Optimize an intervention policy](../modeling/edit-model.md) ## Validate a model configuration -You can use the Validate a model configuration operator to check if a population remains constant and identify non-negativity. You can also add more constraints to check additional conditions. +You can use the Validate configuration operator to check if a population remains constant and identify non-negativity. You can also add more constraints to check additional conditions.
![](../img/models/model-configuration-validate-operator.png) @@ -83,7 +96,8 @@ You can use the Validate a model configuration operator to check if a population --- - Model configuration + - Model configuration + - Dataset (optional) - :material-arrow-expand-right:{ .lg .middle aria-hidden="true" } __Outputs__ @@ -93,31 +107,29 @@ You can use the Validate a model configuration operator to check if a population -??? list "To validate a model configuration" +???+ list "To validate a model configuration" - 1. [Configure the model](#configure-a-model). + 1. [Configure the model](#create-a-model-configuration). ??? tip The input configuration should include some distribution on the parameters to allow for an exploration of the parameter space. - 2. Right-click anywhere on the workflow graph and select **Work with model** > **Validate configuration**. + 2. Right-click anywhere on the workflow graph and select **Config & Intervention** > **Validate configuration**. 3. Connect the Configure model output to the Validate configuration input. - 4. Click **Review checks**. 5. Edit the validation parameters: + - **Parameters of interest**: stricter checks are performed on selected parameters. - **Start** and **End time** - - **Number of steps** + - **Number of timepoints** - **Tolerance**: the lower the value, the more potential boxes will be split from each variable's range, introducing more sampling and more dynamics. - - **Parameters of interest**: stricter checks are performed on selected parameters. ??? tip Lowering the tolerance and checking parameters of interest can significantly increase the time it takes to run the Validate configuration operator. - 6. Click **Add another constraint** to specify a new constraint against which to test model configurations. Edit the constraint parameters. - - **Name of constant** - - **Constraint type** + 6. Click **Add new check** to specify a new constraint against which to test model configurations. Edit the constraint parameters. + - **Constraint type** and **name of constant** - **Target** - **Start** and **End time** - **Lower** and **Upper bound**: diff --git a/docs/config-and-intervention/create-intervention-policy.md b/docs/config-and-intervention/create-intervention-policy.md new file mode 100644 index 0000000..788030b --- /dev/null +++ b/docs/config-and-intervention/create-intervention-policy.md @@ -0,0 +1,61 @@ +--- +title: Create and optimize an intervention policy +--- + +# Create and optimize an intervention policy + +## Create an intervention policy + +More info coming soon. + +## Optimize an intervention policy + +When you optimize an intervention policy, you specify a set of constraints to find the optimal combination of parameters that match your objective. + +
![](../img/models/model-optimize-operator.png)
How it works: [PyCIEMSS](https://github.com/ciemss/pyciemss/blob/main/pyciemss/interfaces.py#L747) :octicons-link-external-24:{ alt="External link" title="External link" }
+ +
+ +- :material-arrow-collapse-right:{ .lg .middle aria-hidden="true" } __Inputs__ + + --- + + - Model configuration + - Intervention policy + +- :material-arrow-expand-right:{ .lg .middle aria-hidden="true" } __Outputs__ + + --- + + Optimized model configuration + +
+ +???+ list "To optimize an intervention policy" + + 1. [Configure the model](../config-and-intervention/configure-model.md). + 2. Right-click anywhere on the workflow graph and select **Simulation** > **Optimize intervention policy**. + 3. Connect the Configure model output to the Optimize input. + 4. Click **Edit**. + 5. Configure the optimization settings as needed: + + - **Start** and **End time**: Specify the simulation time range. + - **Number of samples to simulate model**: Enter the number of stochastic samples to draw from the model. + - **Solver method**: Choose whether to solve ordinary differential equations using [dopri5](https://en.wikipedia.org/wiki/Dormand-Prince_method) or [euler](https://en.wikipedia.org/wiki/Euler_method). + + 6. Specify your intervention policies: + + - **Parameter**: the parameter you want to optimize. + - **Initial guess**: your initial guess for the optimization. + - **Lower** and **Upper bound**: how much the parameter can vary. + - **Start time**: the start time of the model. + + To set additional interventions, click :octicons-plus-24:{ aria-hidden="true" } **Add more interventions**. + + 7. Add a constraint that limits the optimized configuration: + - Select the **target variables**. + - Set the **Acceptable risk of failure** and **Threshold**. + 8. Click :material-play-outline:{ aria-hidden="true" } **Run**. + 9. Save the the results: + - Enter a **Model config name** and click **Save as new model configuration**. + - Click :fontawesome-solid-floppy-disk:{ aria-hidden="true" } **Save as new dataset**. \ No newline at end of file diff --git a/docs/config-and-intervention/index.md b/docs/config-and-intervention/index.md new file mode 100644 index 0000000..a6e19db --- /dev/null +++ b/docs/config-and-intervention/index.md @@ -0,0 +1,10 @@ +--- +title: Configuration and intervention +--- + +# Configuration and intervention + +More info coming soon. + +- [Configure a model](configure-model.md) +- [Create and optimize an intervention policy](create-intervention-policy.md) \ No newline at end of file diff --git a/docs/datasets/index.md b/docs/datasets/index.md index 2853275..4944b45 100644 --- a/docs/datasets/index.md +++ b/docs/datasets/index.md @@ -1,20 +1,30 @@ --- -title: "Datasets" +title: "Working with data" --- -# Work with data +# Working with data -You can create workflows to calibrate or to configure models that use: +You can use uploaded datasets or simulation results to [configure](../config-and-intervention/configure-model.md) and [calibrate](../simulation/calibrate-model.md) models. If the data doesn't align with your intended analysis, you can [transform](transform-dataset.md) it by: -- Custom data from your uploaded from your computer. -- Data stored in a GitHub repository. -- Data from the Terarium database. +- Creating new variables +- Calculating summary statistics +- Filtering data +- Joining datasets -## Add a dataset to your project +The Transform data operator can also serve as a place to visually plot and compare multiple datasets or simulation results. -For information about uploading or finding datasets, see [Gather modeling resources](../modeling-resources/index.md). +!!! note -Once you add dataset to your project, you can open it to view more details or add it to a workflow for use in various scientific modeling processes. + For information about uploading datasets, see [Upload modeling resources](../upload-resources/index.md). + +## Dataset resource + +Once you add a dataset to your project, you can: + +- [Open it to explore the raw data](review-and-enrich-dataset.md). +- Drag it into a workflow from the Resources panel and connect it to transformation, configuration, and calibration operators. + +In a workflow, the Dataset resource lists the columns it contains. ![Resource node of a SEIRD dataset for UK compartments with a 5-row preview](../img/data/data-resource.png) @@ -34,118 +44,39 @@ Once you add dataset to your project, you can open it to view more details or ad -

To add a dataset to a workflow

- -- From the Resources panel, drag the dataset into a workflow graph. - -## Understand dataset descriptions - -Terarium relies on large language models (LLMs) to make sense of datasets and enable data transformations of: - -- Descriptions -- Columns - -![Metadata and column summaries for a SIDARTHE dataset](../img/data/data-details.png) - -

To view the raw data in a dataset

- -1. Click the dataset name in the Resources panel to open it in a new tab. -2. Click the **Data** tab. - -### Enrich dataset descriptions - -Terarium also has an enrichment service that can automatically populate dataset descriptions based on related publications you've identified. When you enrich a dataset, the service looks at your related publications to find and apply definitions for concepts. - -???+ tip - - Curating concepts improves structural comparison and alignment of models and data. - -

To enrich a dataset description

- -1. Open the dataset in a new tab. -2. Click **Enrich description**. -3. Perform one of the following actions: - - - To enrich descriptions without selecting a document, click **Generate descriptions**. - - To use a document, select a publication and click **Use document to generate descriptions**. - -## Transform a dataset +### What can I do with a dataset resource? -Terarium has an operator with a large language model interface, TGPT, to help you clean and transform data to suit your modeling and simulation processes. With TGPT, you can use plain language to explain the changes you want to make and save the transformed data as a new resource within your project. - -
![Transform dataset that takes a UK compartments dataset as an input and outputs a filtered version](../img/data/transform-operator.png)
How it works: [pandas](https://pandas.pydata.org/docs/user_guide/index.html#user-guide) :octicons-link-external-24:{ alt="External link" title="External link" }
+Hover over the output of the Dataset resource and click :octicons-plus-24:{ title="Link" aria-label="Link" } to use the dataset as an input to one of the following operators.
-- :material-arrow-collapse-right:{ .lg .middle aria-hidden="true" } __Inputs__ +- __Data__ --- - - Dataset - - Simulation output + - [Transform dataset](transform-dataset.md) + Guide an AI assistant to modify or visualize the dataset. + - [Subset dataset](subset-dataset.md) + Generate a subset of an Earth System Grid Federation (ESGF) dataset along geo-boundaries or temporal slices. + - [Transform gridded dataset](transform-gridded-dataset.md) + Guide an AI assistant to modify or visualize a gridded dataset. -- :material-arrow-expand-right:{ .lg .middle aria-hidden="true" } __Outputs__ +- __Configuration and intervention__ --- - Dataset - -
- -Each transformation you make builds off your earlier changes. TGPT can perform more than one command at a time. With it, you can: - -+ Perform math -+ Add or drop columns -+ Describe the dataset -+ Sort the data -+ Plot the data -+ Undo changes coming soon - -??? example "Example prompts" + - [Configure model](../config-and-intervention/configure-model.md) + Use the dataset to extract initial values and parameters for the condition you want to test. + - [Validate configuration](../config-and-intervention/configure-model.md#validate-a-model-configuration) + Use the dataset to validate a configuration - * Filter the data to just location = "US" - * Convert the date column to timestamps and plot the data - * Create a new census column that is a rolling sum of 'value' over the previous 10 days - * Add a new column that is the cumulative sum of the values - * Plot the data - * Rename column 'cases' to 'I', column 'hospitalizations' to 'H', and 'deaths' to 'E' +- __Simulation__ -

To use TGPT to transform a dataset

- -1. Drag the dataset into a workflow graph. -2. Right-click the graph and select **Work with dataset** > **Transform dataset**. -3. Click the output on the Dataset operator and then click the Dataset input on the Transform dataset operator. -4. Click **Edit**. -5. Use the text box at the bottom of the page to describe the transformation you want to make and then press ++enter++. - ![TGPT with a prompt and python code to filter convert a date column to timestamps](../img/data/transform.jpg) -6. Scroll to the data preview under your prompt to inspect the transformation. -7. To make changes to the transformation, edit the provided python code and click :material-play-outline:{ aria-hidden="true" } **Run again**. -8. Repeat steps 5 through 7 to make more transformations. -9. At any time, save the transformed data: - - Save the dataset as a new dataset in Terarium by clicking :material-content-save-outline:{ aria-hidden="true" } **Save as**, entering a unique name in the text box and clicking :octicons-check-24:{ title="Save" }. - - !!! tip - - Saved datasets appear in your Resources panel and the output of the Transform dataset operator. - - - Download the dataset to your computer by clicking :octicons-download-24:{ aria-hidden="true" } **Download**. - -### Understand how TGPT interprets your prompt - -To give you a sense whether it correctly interpreted your prompt, TGPT comments its code to explain what it's done. - - + --- -When the transformation is complete, TGPT shows: + - [Calibrate](../simulation/calibrate-model.md) + Use the dataset to improve the performance of a model by updating the value of configuration parameters. + - [Calibrate ensemble](../simulation/calibrate-ensemble.md) + Use the dataset to improve the performance of a model by updating the value of configuration parameters. -- A preview of the transformed data. -- Any python code used to complete the transformation. -- Any applicable error codes. -- Any requested visualizations. \ No newline at end of file + \ No newline at end of file diff --git a/docs/datasets/review-and-enrich-dataset.md b/docs/datasets/review-and-enrich-dataset.md new file mode 100644 index 0000000..11f364b --- /dev/null +++ b/docs/datasets/review-and-enrich-dataset.md @@ -0,0 +1,51 @@ +--- +title: Review and enrich a dataset +--- + +# Review and enrich a dataset + +Once you have [uploaded a dataset into your project](../upload-resources/index.md), you can open it to: + +- Explore and summarize its data and columns. +- Manually add metadata that explains the data in each column. +- Automatically enrich metadata using documents in your dataset or without additional context. + +## Review a dataset + +To get an understanding of your data, you can open a dataset and review its columns and a selection of its rows. A dataset resource previews up to 100 rows of data. + +??? list "To open a dataset" + + - Perform one of the following actions: + - In the Resources panel, click the name of the dataset. + - On a Dataset node in the workflow graph, click **Open**. + +??? list "To view the raw data in a dataset" + + - Click **Data** in the navigation list on the right. + +??? list "To download a dataset" + + - Next to the dataset name, click :fontawesome-solid-ellipsis-vertical:{ title="Menu" aria-label="Menu" } > :octicons-download-24:{ aria-hidden="true"} **Download**. + +## Enrich a dataset + +Terarium's enrichment service can automatically populate dataset descriptions and column metadata based on the documents in your project. When you enrich a dataset, the service looks at a selected document to find and apply definitions for concepts. + +???+ tip + + Curating concepts improves structural comparison and alignment of models and data. + +![Metadata and column summaries for a SIDARTHE dataset](../img/data/data-details.png) + +???+ list "To enrich dataset metadata" + + 1. Click **Enrich metadata**. + 2. Perform one of the following actions: + + - To enrich metadata without selecting a document, click **Generate information without context**. + - To use a document, select the document title. + + 3. Click **Enrich**. + 4. Review the updated description and column information. + 5. Click **Save**. \ No newline at end of file diff --git a/docs/datasets/subset-dataset.md b/docs/datasets/subset-dataset.md index 197004f..16132d9 100644 --- a/docs/datasets/subset-dataset.md +++ b/docs/datasets/subset-dataset.md @@ -24,22 +24,16 @@ The Subset dataset operator allows you to generate a subset of an Earth System G -

To generate a subset of an ESGF dataset

- -1. From the Resources panel, drag the Dataset resource into the workflow graph. - - ???+ note "To find ESGF datasets" - - On the Datasets tab of the :fontawesome-regular-compass:{ aria-hidden="true" } [Explorer](../modeling-resources/index.md#find-papers-models-and-datasets), change the **Source** to ESGF. - -2. Right-click anywhere on the workflow graph and select **Work with dataset** > **Transform dataset**. -3. Connect the output of the Dataset resource to the input of the Subset dataset operator. -4. Click **Open**. -5. Choose the subset of data you want to include: - - **Geo-boundaries**: Specify a pair of **Latitude** and **Longitude** coordinates that define a bounding box around an area of interest. - - **Temporal slide**: Specify the earliest (**From**) and latest (**To**) dates to include. -6. To sample the data from the subset: - 1. Select **Spatial** or **Time Skipping**. - 2. Enter the number of datapoints or time slices you want to include. For example, enter *5* to keep every fifth entry. -6. Click :material-play-outline:{ aria-hidden="true" } **Run**. -7. Click **Save as new dataset** to add the subset to your project resources. \ No newline at end of file +???+ list "To generate a subset of an ESGF dataset" + + 1. From the Resources panel, drag the Dataset into the workflow graph. + 2. Hover over the output of the Dataset resource and click :octicons-plus-24:{ title="Link" aria-label="Link" } > **Subset dataset**. + 3. On the Subset dataset operator, click **Open**. + 4. Choose the subset of data you want to include: + - **Geo-boundaries**: Specify a pair of **Latitude** and **Longitude** coordinates that define a bounding box around an area of interest. + - **Temporal slice**: Specify the earliest (**From**) and latest (**To**) dates to include. + 5. To sample the data from the subset: + 1. Select **Spatial** or **Time Skipping**. + 2. Enter the number of datapoints or time slices you want to include. For example, enter *5* to keep every fifth entry. + 6. Click :material-play-outline:{ aria-hidden="true" } **Run**. + 7. Click **Save as new dataset** to add the subset to your project resources. \ No newline at end of file diff --git a/docs/datasets/transform-dataset.md b/docs/datasets/transform-dataset.md new file mode 100644 index 0000000..98c6e49 --- /dev/null +++ b/docs/datasets/transform-dataset.md @@ -0,0 +1,116 @@ +--- +title: Transform a dataset +--- + +# Transform a dataset + +If a dataset doesn't align with your modeling goals, you can transform it by cleaning and modifying it or combining it with other datasets. Supported transformations include: + +
+ +- __Manipulation__ + + --- + + - Creating new variables. + - Filtering the data. + - Joining two or more datasets. + - Performing mathematical operations. + - Adding or dropping columns. + - Sorting the data. + - Handling missing values. + - Converting incidence data (e.g., daily new case counts) to prevalence data (current total counts at any given time). + +- __Visualization and summarization__ + + --- + + - Calculating summary statistics. + - Describing the dataset. + - Plotting the data. + - Answering specific questions about the data. + +
+ +The Transform dataset operator has an interactive AI assistant. You describe in plain language the changes you want to make, and the LLM-powered assistant automatically generates the code for you. + +!!! note + + The Transform dataset operator adapts to your level of coding experience. You can: + + - Work exclusively by prompting the assistant with plain language. + - Edit and rerun any of the assistant-generated code. + - Enter your own executable code to make custom transformations. + +## Transform dataset operator + +The Transform dataset operator takes one or more datasets or simulation results as inputs and outputs a transformed dataset for use in other operations. + +You can choose any step in your transformation process as the thumbnail preview on the node in the workflow graph. + +
![Transform dataset that takes a UK compartments dataset as an input and outputs a filtered version](../img/data/transform-operator.png)
How it works: [pandas](https://pandas.pydata.org/docs/user_guide/index.html#user-guide) :octicons-link-external-24:{ alt="External link" title="External link" }
+ +
+ +- :material-arrow-collapse-right:{ .lg .middle aria-hidden="true" } __Inputs__ + + --- + + - Dataset + - Simulation output + +- :material-arrow-expand-right:{ .lg .middle aria-hidden="true" } __Outputs__ + + --- + + Dataset + +
+ +# Work with the Transform dataset AI assistant + +Each transformation you make builds off your earlier changes. The Transform dataset operator can perform more than one command at a time. With it, you can: + +??? example "Example prompts" + + * Filter the data to just location = "US" + * Convert the date column to timestamps and plot the data + * Create a new census column that is a rolling sum of 'value' over the previous 10 days + * Add a new column that is the cumulative sum of the values + * Plot the data + * Rename column 'cases' to 'I', column 'hospitalizations' to 'H', and 'deaths' to 'E' + +???+ list "To transform a dataset" + + 1. Drag the dataset into a workflow graph. + 2. Hover over the output of the Dataset resource and click :octicons-plus-24:{ title="Link" aria-label="Link" } > **Transform dataset**. + 3. On the new Transform dataset operator, click **Edit**. + 4. Use the text box at the top of the page to describe the transformation you want to make and then click :octicons-paper-airplane-24:{ style="transform: rotate(-45deg);" title="" aira-label="" }. + ![A prompt and python code to filter convert a date column to timestamps](../img/data/transform.jpg) + 5. Scroll to the data preview under your prompt to inspect the transformation. + 6. To make changes to the transformation, edit the code and click :material-play-outline:{ title="Run again" }. + 7. Repeat steps 4 through 6 to make more transformations. + 8. At any time, save the transformed data as a new dataset in Terarium by clicking **Save for reuse**, entering a unique name in the text box and clicking **Save**. + + !!! tip + + Saved datasets appear in your Resources panel and the output of the Transform dataset operator. + + Download the dataset to your computer by clicking :octicons-download-24:{ aria-hidden="true" } **Download**. + +### Understand how the Transform dataset operator interprets your prompt + +To give you a sense whether it correctly interpreted your prompt, the Transform dataset operator: + +- Records its thoughts about your prompt (*I need to filter the dataset to only include rows with location equal to 'US'*). +- Shows how it intends to perform the transformation (*DatasetToolset.generate_python_code*). +- Comments its code to explain what it's done. + +You can show or hide these thoughts at any time by clicking **Show/Hide thoughts**. + +When the transformation is complete, the Transform dataset operator shows: + +- A preview of the transformed data. +- Any code used to complete the transformation. +- Any applicable error codes. +- Any requested visualizations. \ No newline at end of file diff --git a/docs/get-started/example-workflow.md b/docs/get-started/example-workflow.md new file mode 100644 index 0000000..400b551 --- /dev/null +++ b/docs/get-started/example-workflow.md @@ -0,0 +1,7 @@ +--- +title: Recreating, modifying, and simulating an SIR model +--- + +# Recreating, modifying, and simulating an SIR model + +More info coming soon. \ No newline at end of file diff --git a/docs/get-started/using-terarium.md b/docs/get-started/using-terarium.md index 07f4886..5bc7c5f 100644 --- a/docs/get-started/using-terarium.md +++ b/docs/get-started/using-terarium.md @@ -6,33 +6,20 @@ title: "Using Terarium" Terarium is a comprehensive platform that supports your scientific modeling processes from discovery to publication. With it, you can upload, search for, modify, and simulate scientific models and easily share and reproduce your results. -## Getting access to system - -To access Terarium, you need a [GitHub account](https://github.com/signup){ target="_blank" rel="noopener noreferrer" } :octicons-link-external-24:{ aria-hidden="true" }. - -1. Send your GitHub username to [support@terarium.ai](mailto:support@terarium.ai?subject=Terarium Phase 1 evaluation - access request). -2. When you receive approval, go to [https://app.terarium.ai/](https://app.terarium.ai/) and click sign in with :fontawesome-brands-github:{ aria-hidden="true"} **GitHub**. -3. If redirected to github.com, enter your account details and click **Sign in**. - ## Extracting knowledge -A project in Terarium acts as a repository for resources (models, datasets, documents, and code) related to your modeling goals. You can upload your own resources or search the Terarium databases for relevant papers. - -To upload resources - -1. Click :octicons-upload-16:{ aria-hidden="true" } **Upload resources** in the Resources panel. - ![](../img/get-started/upload.png) -2. Drag in your resources or click **open a file browser** to locate them. -3. Click **Upload**. +A project in Terarium acts as a repository for resources (models, datasets, and documents) related to your modeling goals. -To search for resources +???+ list "To upload resources" -1. Expand the dropdown list in the nav bar and select :fontawesome-regular-compass:{ aria-hidden="true" } **Explorer**. - ![](../img/get-started/explorer.png) -2. Enter keywords in the search bar and press ++enter++. -3. Choose the resources you want to review by clicking **Documents**, **Models**, or **Datasets**. -4. Use the Filters panel to understand and narrow down the results. -5. To add a resource to your project, click :octicons-plus-24:{ alt="Add to project" title="Add to project" } on the card in the results. + 1. Click **Upload** in the Resources panel. + ![](../img/get-started/upload.png) + 2. Drag in your resources or click **open a file browser** to locate them. + 3. Click **Upload**. + + !!! note + + When you upload a PDF, Terarium automatically scans the document and extracts any linear ordinary differential equations. Depending on the size of the document, this may take some time. Click :material-bell-outline:{ title="Notifications" aria-label="Notifications" } in the nav bar to check the progress of an extraction. ## Constructing scientific modeling workflows @@ -40,19 +27,20 @@ A workflow in Terarium is a graphical editor for building and executing complex ![](../img/get-started/workflow.png) -Graph nodes represent resources or operators that handle transformation and simulation. Link them together to extract code from project resources, iteratively modify them, and run sophisticated calibration, optimization, and simulation tasks. +Graph nodes represent resources or operators that handle transformation and simulation. Connect them to recreate models from documents, iteratively modify them, and run sophisticated calibration, optimization, and simulation tasks. -To create a workflow +???+ list "To create a workflow" -1. Click :octicons-plus-24:{ aria-hidden="true" } **New** in the Workflow section of the Resource panel. -2. Drag Models, Datasets, Documents, or Code into the workflow graph from the Resources panel. -3. Right-click anywhere on the workflow graph and choose from the library of operators. -4. Link your resources and operators by connecting outputs (![](../img/output.png){ aria-hidden="true" style="display: inline; margin-top: 0; margin-bottom: 0;" }) on the right side of a node and then clicking a corresponding input (![](../img/input.png){ aria-hidden="true" style="display: inline; margin-top: 0; margin-bottom: 0;" }) on the left side of the destination node. - - ???+ note - - - Labels on inputs show you the types of resources and operators each operator requires. - - Some operators don't display outputs until you edit their settings or run their associated task. Click **Open** or **Edit** on a node to edit settings and run tasks. + 1. Click :octicons-plus-24:{ aria-hidden="true" } **New** in the Workflows section of the Resource panel. + 2. Enter a name for the new workflow and click **Save**. + 2. Drag Models, Datasets, or Documents into the workflow graph from the Resources panel. + 3. Right-click anywhere on the workflow graph and choose from the library of operators. + 4. Connect your resources and operators by clicking ![](../img/output.png){ aria-hidden="true" class="node-icon" } on the right side of a node and then clicking a corresponding input ![](../img/input.png){ aria-hidden="true" class="node-icon" } on the left side of the destination node. + + ???+ note + + - Labels on inputs show you the types of resources and operators each operator requires. + - Some operators don't display outputs until you click **Open** or **Edit** and modify their settings or run their associated task. ## Modifying models and data @@ -60,98 +48,78 @@ Using Terarium's library of operators, you can recreate, reuse, and modify exist
-- __Add resource__ +- __Modeling__ --- - - [**Create model from code**](https://github.com/ml4ai/ASKEM-TA1-DockerVM?tab=readme-ov-file#code2amr){ target="_blank" rel="noopener noreferrer" } :octicons-link-external-24:{ alt="External link" title="External link" } - Build a model using selected lines within a code asset. - - - [**Create model from equations**](https://github.com/DARPA-ASKEM/model-service/blob/07ae21cae2d5465f9ac5b5bbbe6c7b28b7259f04/src/ModelService.jl#L54){ target="_blank" rel="noopener noreferrer" } :octicons-link-external-24:{ alt="External link" title="External link" } + - [**Create model from equations**](../modeling/create-model-from-equations.md) [:material-github:{ title="GitHub" aria-label="GitHub" class="md-annotation__index"}](https://github.com/DARPA-ASKEM/model-service/blob/07ae21cae2d5465f9ac5b5bbbe6c7b28b7259f04/src/ModelService.jl#L54){ target="_blank" rel="noopener noreferrer" } Build a model using LaTeX expressions or equations extracted from a paper. -- __Work with dataset__ + - [**Edit model**](../modeling/edit-model.md) [:material-github:{ title="GitHub" aria-label="GitHub" class="md-annotation__index" }](https://github.com/DARPA-ASKEM/beaker-kernel/blob/main/docs/contexts_mira_model_edit.md){ target="_blank" rel="noopener noreferrer" } + Modify model states and transitions using a graphical template editor or an AI assistant. - --- + - [**Stratify model**](../modeling/stratify-model.md) [:material-github:{ title="GitHub" aria-label="GitHub" class="md-annotation__index" }](https://github.com/gyorilab/mira/blob/main/notebooks/viz_strat_petri.ipynb){ target="_blank" rel="noopener noreferrer" } + Divide populations into subsets along characteristics such as age or location. - - [**Transform dataset**](https://pandas.pydata.org/docs/user_guide/index.html#user-guide){ target="_blank" rel="noopener noreferrer" } :octicons-link-external-24:{ alt="External link" title="External link" } - Modify a dataset by explaining your changes to an AI assistant. + - [**Compare models**](../modeling/compare-models.md) [:material-github:{ title="GitHub" aria-label="GitHub" class="md-annotation__index" }](https://github.com/gyorilab/mira/blob/7314765ab409ddc9647269ad2381055f1cd67706/notebooks/hackathon_2023.10/dkg_grounding_model_comparison.ipynb#L307){ target="_blank" rel="noopener noreferrer" } + Generate side-by-side summaries of two or more models or prompt an AI assistant to visually compare them. -- __Work with model__ +- __Data__ --- - - [**Edit model**](https://github.com/DARPA-ASKEM/beaker-kernel/blob/main/docs/contexts_mira_model_edit.md){ target="_blank" rel="noopener noreferrer" } :octicons-link-external-24:{ alt="External link" title="External link" } - Modify model states and transitions using a graphical template editor or an AI assistant. - -- __Work with multiple models__ + - [**Transform dataset**](../datasets/transform-dataset.md) [:octicons-book-24:{ title="User Guide" aria-label="User Guide" class="md-annotation__index" }](https://pandas.pydata.org/docs/user_guide/index.html#user-guide){ target="_blank" rel="noopener noreferrer" } + Modify a dataset by explaining your changes to an AI assistant. - --- + - [**Subset dataset**](../datasets/subset-dataset.md) + - - [**Compare models**](https://github.com/gyorilab/mira/blob/7314765ab409ddc9647269ad2381055f1cd67706/notebooks/hackathon_2023.10/dkg_grounding_model_comparison.ipynb#L307){ target="_blank" rel="noopener noreferrer" } :octicons-link-external-24:{ alt="External link" title="External link" } - Generate side-by-side summaries of two or more models or prompt an AI assistant to visually compare them. - - - [**Couple models**](https://algebraicjulia.github.io/Decapodes.jl/dev/overview/#Merging-Multiple-Physics){ target="_blank" rel="noopener noreferrer" } :octicons-link-external-24:{ alt="External link" title="External link" } - Combine two or more models. + - [**Transform gridded dataset**](../datasets/transform-gridded-dataset.md) +
-Some operators have a dual-view design, with a wizard for common settings and a notebook for direct coding. An AI assistant in the notebook helps you generate and refine code even without programming experience. - -To access resource or operator settings - -1. Click **Open** or **Edit** on the operator node. -2. Switch to the Wizard or Notebook view depending on your preference. +Some operators have a dual-view design, with a wizard for common settings and a notebook for direct coding. An AI assistant in the notebook helps you generate and refine code even if you don't have any programming experience. - ???+ note +???+ list "To access resource or operator settings" - Any changes you make in the Wizard view are automatically translated into code in the Notebook view. + 1. Click **Open** or **Edit** on the operator node. + 2. Switch to the Wizard or Notebook view depending on your preference. + + ???+ note + + Any changes you make in the Wizard view are automatically translated into code in the Notebook view. -## Simulating models +## Configuring and simulating models Terarium's library of operators also allows you to rapidly create of scenarios and interventions by linking operators to configure, validate, calibrate, and optimize models:
-- __Work with model__ +- __Config and intervention__ --- - - [**Configure model**](../models/configure-model.md) + - [**Configure model**](../config-and-intervention/configure-model.md) Edit variables and parameters or extract them from a reference resource. - - [**Validate configuration**](https://github.com/siftech/funman){ target="_blank" rel="noopener noreferrer" } :octicons-link-external-24:{ alt="External link" title="External link" } + - [**Validate configuration**](../config-and-intervention/configure-model.md#validate-a-model-configuration) [:material-github:{ title="GitHub" aria-label="GitHub" class="md-annotation__index" }](https://github.com/siftech/funman){ target="_blank" rel="noopener noreferrer" } Determine if a configuration generates valid outputs given a set of constraints. - - [**Stratify model**](https://github.com/gyorilab/mira/blob/main/notebooks/viz_strat_petri.ipynb){ target="_blank" rel="noopener noreferrer" } :octicons-link-external-24:{ alt="External link" title="External link" } - Divide populations into subsets along characteristics such as age or location. + - [**Create intervention policy**](../config-and-intervention/create-intervention-policy.md) -- __Work with multiple models__ +- __Simulation__ --- - - [**Simulate ensemble**](https://github.com/ciemss/pyciemss/blob/main/pyciemss/interfaces.py#L35){ target="_blank" rel="noopener noreferrer" } :octicons-link-external-24:{ alt="External link" title="External link" } - Run a deterministic simulation of multiple models or model configurations under specific conditions. - - [**Calibrate ensemble**](https://github.com/ciemss/pyciemss/blob/main/pyciemss/interfaces.py#L156){ target="_blank" rel="noopener noreferrer" } :octicons-link-external-24:{ alt="External link" title="External link" } - Determine or update the value of model parameters given a reference dataset of observations. - -- __Run model (deterministic)__ - - --- - - - [**Simulate with SciML**](https://github.com/DARPA-ASKEM/sciml-service/blob/main/src/operations.jl#L222){ target="_blank" rel="noopener noreferrer" } :octicons-link-external-24:{ alt="External link" title="External link" } - Run a deterministic simulation of a model under specific conditions. - - [**Calibrate with SciML**](https://github.com/DARPA-ASKEM/sciml-service/blob/main/src/operations.jl#L245){ target="_blank" rel="noopener noreferrer" } :octicons-link-external-24:{ alt="External link" title="External link" } - Determine or update the value of model parameters given a reference dataset of observations. - -- __Run model (probabilistic)__ - - --- - - - [**Simulate with PyCIEMSS**](https://github.com/ciemss/pyciemss/blob/main/pyciemss/interfaces.py#L323){ target="_blank" rel="noopener noreferrer" } :octicons-link-external-24:{ alt="External link" title="External link" } + - [**Simulate**](../simulation/simulate-model.md) [:material-github:{ title="GitHub" aria-label="GitHub" class="md-annotation__index" }](https://github.com/ciemss/pyciemss/blob/main/pyciemss/interfaces.py#L323){ target="_blank" rel="noopener noreferrer" } Run a probabilistic simulation of a model under specific conditions. - - [**Calibrate with PyCIEMSS**](https://github.com/ciemss/pyciemss/blob/main/pyciemss/interfaces.py#L529){ target="_blank" rel="noopener noreferrer" } :octicons-link-external-24:{ alt="External link" title="External link" } + - [**Calibrate**](../simulation/calibrate-model.md) [:material-github:{ title="GitHub" aria-label="GitHub" class="md-annotation__index" }](https://github.com/ciemss/pyciemss/blob/main/pyciemss/interfaces.py#L529){ target="_blank" rel="noopener noreferrer" } + Determine or update the value of model parameters given a reference dataset of observations. + - [**Optimize intervention policy**](../config-and-intervention/create-intervention-policy.md#optimize-an-intervention-policy) [:material-github:{ title="GitHub" aria-label="GitHub" class="md-annotation__index" }](https://github.com/ciemss/pyciemss/blob/main/pyciemss/interfaces.py#L747){ target="_blank" rel="noopener noreferrer" } + Determine the optimal values for variables that minimize or maximize an intervention given some constraints. + - [**Simulate ensemble**](../simulation/simulate-ensemble.md) [:material-github:{ title="GitHub" aria-label="GitHub" class="md-annotation__index" }](https://github.com/ciemss/pyciemss/blob/main/pyciemss/interfaces.py#L35){ target="_blank" rel="noopener noreferrer" } + Run a deterministic simulation of multiple models or model configurations under specific conditions. + - [**Calibrate ensemble**](../simulation/calibrate-ensemble.md) [:material-github:{ title="GitHub" aria-label="GitHub" class="md-annotation__index" }](https://github.com/ciemss/pyciemss/blob/main/pyciemss/interfaces.py#L156){ target="_blank" rel="noopener noreferrer" } Determine or update the value of model parameters given a reference dataset of observations. - - [**Optimize with PyCIEMSS**](https://github.com/ciemss/pyciemss/blob/main/pyciemss/interfaces.py#L747){ target="_blank" rel="noopener noreferrer" } :octicons-link-external-24:{ alt="External link" title="External link" } - Determine the optimal values for variables that minimize or maximize an intervention given some constraints.
diff --git a/docs/img/models/stratify-options.png b/docs/img/models/stratify-options.png new file mode 100644 index 0000000..848fa86 Binary files /dev/null and b/docs/img/models/stratify-options.png differ diff --git a/docs/img/terarium-home.jpg b/docs/img/terarium-home.jpg index fd6d597..c02b750 100644 Binary files a/docs/img/terarium-home.jpg and b/docs/img/terarium-home.jpg differ diff --git a/docs/index.md b/docs/index.md index ebeeaa6..6fd20d8 100644 --- a/docs/index.md +++ b/docs/index.md @@ -4,12 +4,12 @@ title: "About Terarium" # About Terarium -Terarium is a modeling and simulation workbench designed to help you assess and contribute to the scientific landscape. With Terarium, you can: +Terarium is a modeling and simulation workbench designed to help you assess and contribute to the scientific landscape. Regardless of your level of programming experience, Terarium allows you to:
-- :fontawesome-solid-magnifying-glass:{ .lg .middle } **Find models in academic literature** +- :fontawesome-solid-magnifying-glass:{ .lg .middle } **Extract models from academic literature** - :fontawesome-solid-sliders:{ .lg .middle } **Parameterize and calibrate them** -- :fontawesome-solid-gear:{ .lg .middle } **Simulate to a variety of scenarios** +- :fontawesome-solid-gear:{ .lg .middle } **Simulate a variety of scenarios** - :fontawesome-solid-chart-line:{ .lg .middle } **Analyze the results**
@@ -19,11 +19,14 @@ Need help? Check out these topics:
[Get started](get-started/using-terarium.md){ .md-button } -[Gather modeling resources](modeling-resources/index.md){ .md-button } +[Upload modeling resources](upload-resources/index.md){ .md-button } -[Manage workflows](workflows/index.md){ .md-button } -[Work with data](datasets/index.md){ .md-button } +[Build a workflow graph](workflows/index.md){ .md-button } +[Review and transform data](datasets/index.md){ .md-button } -[Work with models](models/index.md){ .md-button } +[Modeling](modeling/index.md){ .md-button } +[Configuration and intervention](config-and-intervention/index.md){ .md-button } + +[Simulation](simulation/index.md){ .md-button } [Integrated subsystems](subsystems/index.md){ .md-button }
\ No newline at end of file diff --git a/docs/modeling-resources/index.md b/docs/modeling-resources/index.md deleted file mode 100644 index 740aaf3..0000000 --- a/docs/modeling-resources/index.md +++ /dev/null @@ -1,175 +0,0 @@ ---- -title: "Gather modeling resources" ---- - -# Gather modeling resources - -With Terarium, you can upload, store and manage resources needed for your modeling and simulation workflows. You can upload resources from your computer or retrieve resources from online code repositories. Many resources are also available for search within the Terarium databases. - -??? info "Resource extractions" - - Terarium summarizes the following types of metadata and data extracted from the supported resources. - - === ":octicons-file-24:{ aria-hidden="true" } Documents" - - Each document has an associated PDF (from Unpaywall) and a set of extracted metadata and contents, which may include: - - * Metadata - - Journal - - Title - - Authors - - Publication year - - DOI - - Publisher - * Extractions - - Abstract - - Section summaries - - Figures - - Tables - - Equations - - References - - GitHub URLs - - === ":octicons-share-android-24:{ aria-hidden="true" } Models" - - - Description - - Diagram - - Equations - - Observables - - State variables - - Parameters - - Observables - - Transitions - - Other concepts - - Time - - Diagram - - === ":material-file-table-outline:{ aria-hidden="true" } Datasets" - - * Description - * Column information - -## Upload resources - -You can upload papers, models, and datasets in a variety of formats from your computer or from a GitHub repository. - -### Upload your own resources - -:octicons-upload-16:{ aria-hidden="true" } **Upload resources** in the Resources panel lets you add resources from your computer. - -![Upload dialog configured to upload and extract metadata from a scientific paper on COVID-19](../img/resources/upload.png) - -??? list "To upload resources" - - 1. In the Resource panel, click :octicons-upload-16:{ aria-hidden="true" } **Upload resources**. - 2. Drag your resource files into the Upload resources dialog or click **open a file browser** to navigate to the location of the files you want to add. - 3. Click **Upload**. - -### Import resources from a GitHub repository - -You can import the following resources directly from a GitHub repository: - -- Code files (PY, M, JS). -- Data files (CSV, JSON, TSV, XML, YAML / YML). -- Document files (PDF, TXT, MD) -- Unknown files (if you specify which resource type to import it as) - -![Import dialog showing the directory structure of the nytimes/covid-19-data GitHub repository](../img/resources/github.png) - -??? list "To import resources from a GitHub URL" - - 1. In the Resource panel, click :octicons-upload-16:{ aria-hidden="true" } **Upload resources**. - 2. Enter the URL of the repository in the format *https://github.com/nytimes/covid-19-data*. - 3. Click **Upload**. - 4. Select the code, data, and documents you want to import. - 5. To import an unknown file type, select it and choose whether you want to import it as code, data, or a document. - 6. Click **Import files to project**. - -??? list "To import resources from a GitHub URL referenced in a paper" - - 1. In the Resources panel, click the title of the paper to open it in a new tab. - 2. In the GitHub URLs section of the document extractions, click :material-cloud-download-outline:{ aria-hidden="true" } **Import**. - 3. Select the code, data, and documents you want to import. - 4. To import an unknown file type, select it and choose whether you want to import it as code, data, or a document. - 5. Click **Import files to project**. - - - -## Find papers, models, and datasets - -Using the Explorer view, you can search for the following types of resources to use in your modeling and simulation workflows: - -- :octicons-file-24:{ aria-hidden="true" } **Documents** from a continuously updated collection of published scientific papers related to the COVID-19 pandemic. -- :octicons-share-android-24:{ aria-hidden="true" } **Models** from the Terarium databases. -- :material-file-table-outline:{ aria-hidden="true" } **Datasets** from the Terarium databases. - -![Explorer showing document results for "covid" keyword search filtered by publication year](../img/resources/explorer.png) - -??? list "To open the Explorer" - - - Expand the dropdown list in the nav bar and select :fontawesome-regular-compass:{ aria-hidden="true" } **Explorer**. - -??? list "To search for resources" - - 1. Enter a keyword in the search bar and press ++enter++. - - ??? tip "DOI search" - - If you have a specific paper you want to add to your project, try searching for its DOI in the format *prefix/suffix* (for example, *10.3390/ijerph18179027*). - - 3. To choose the type of resources you want to review, click the **Documents**, **Models**, or **Datasets** button. - 4. Click :material-chevron-double-right:{ alt="Expand filters" title="Expand filters" } to review the filters, which break down the resources that match your search. Different filters are available for each type of resource. -
- -
Top five publication years. The bar and number shows how many of the matches were published in each year.
-
- 5. To view more details about a resource, click anywhere on its card in the search results. - -??? list "To find resources similar to an example" - - 1. In the search results, click and drag the card that represents the example resource into the Search by example dialog. - 2. Select any of the following options to choose how Terarium should determine similarity: - - **Similar content**: Use a similarity vector to find resources with similar contents. - - **Forward citations** coming soon - - **Backward citation** coming soon - - **Related resources**: Find resources based on shared references to assets such as papers, models, and datasets. - 3. Click **Search**. - -??? list "To browse all resources" - - 1. Click in the search bar, clear any search terms and press ++enter++. - 2. To choose the type of resources you want to review, click the **Documents**, **Models**, or **Datasets** button. - 3. Click :material-chevron-double-right:{ alt="Expand filters" title="Expand filters" } to review the filters, which break down all the resources in the Terarium databases. Different filters are available for each type of resource. - 4. To view more details about a resource, click anywhere on its card in the search results. - -??? list "To filter a search" - - - Click a value in the Filters. For example, if you only want to see the latest research, click the current year in the Publication year filter. -
- -
Added filters appear below the summary of the results. Click **x** any time to clear them.
-
- -??? list "To clear a filter" - - Perform one of the following actions: - - * Click the selected value in the Filters, or - * Click :octicons-x-24:{ title="Clear" } on the filter badge below the summary of the results. - -??? list "To add resources to your project" - - - Click :octicons-plus-24:{ title="Add to project" } on the upper right of the resource card and select a project. \ No newline at end of file diff --git a/docs/modeling/compare-models.md b/docs/modeling/compare-models.md new file mode 100644 index 0000000..875658a --- /dev/null +++ b/docs/modeling/compare-models.md @@ -0,0 +1,98 @@ +--- +title: Compare models +--- + +# Compare models + +You can compare two or models in Terarium to see: + +- A brief description of their similarities and differences. +- Side-by-side model card comparisons. +- An AI-assisted visualization of common and unique state variables and transitions across each model. + +
![](../img/models/model-compare-operator.png)
How it works: [MIRA](https://github.com/gyorilab/mira/blob/7314765ab409ddc9647269ad2381055f1cd67706/notebooks/hackathon_2023.10/dkg_grounding_model_comparison.ipynb#L307) :octicons-link-external-24:{ alt="External link" title="External link" }
+ +
+ +- :material-arrow-collapse-right:{ .lg .middle aria-hidden="true" } __Inputs__ + + --- + + Models + +- :material-arrow-expand-right:{ .lg .middle aria-hidden="true" } __Outputs__ + + --- + + N/A + +
+ +## Compare model cards + +Model cards briefly summarize key aspects of the models. Available aspects include: + +
+
+- Comparison of models + - Summary + - Structural Comparisons + - States + - Transitions + - Parameters + - Observables + - Metadata Comparisons + - Details + - Uses + - Biases, Risks, and Limitations + - Testing and Validation +- Model name +- Model diagram +- Summary +- Specs + - Domain + - Type + - Specification + - States + - Parameters +
+
+- Uses + - Direct use + - Out of scope use +- Bias risks limitations + - Biases + - Risks + - Limitations +- Testing + - Validation + - Metrics +- Getting started +- Glossary +- Authors +- Citation +- More information + - Funded by + - Links +
+
+ +???+ list "To compare models" + + 1. From the Resources panel, drag the models into a workflow graph. + 2. Right-click anywhere on the workflow graph and select **Modeling** > **Compare models**. + 3. Connect the outputs of the Model resources to the inputs of the Compare models operator. + 4. Click **Open**. + 5. Scroll down to review the side-by-side model cards. The overview at the top of the page may take a moment to load. + +## Compare model diagrams + +The AI assistant in the Compare models operator can visualize the similarities and differences of the attached models. It creates a comparison diagram for every pair of input models that color-codes the state variables and transitions that are unique to each model and common to both. + +???+ list "To visualize model similarities and differences" + + 1. In the Compare models operator details, click **Notebook**. + 2. Click in the prompt field and select **Compare the models and visualize and display them**. + 3. Click :material-play-outline:{ aria-hidden="true" } **Run**. + 4. Review the model diagrams in the Preview. + 5. To make changes, modify the selected prompt or the generated code and click :material-play-outline:{ aria-hidden="true" } **Run** again. \ No newline at end of file diff --git a/docs/modeling/create-model-from-equations.md b/docs/modeling/create-model-from-equations.md new file mode 100644 index 0000000..bf8919d --- /dev/null +++ b/docs/modeling/create-model-from-equations.md @@ -0,0 +1,117 @@ +--- +title: Create a model from equations +--- + +# Create a model from equations + +You can create models from equations that: + +- Have been automatically extracted from literature (a document). +- You enter in LaTeX format. +- You captured in a screenshot. + +!!! note + + - Terarium's equation extractor is not optimized to handle every single way to represent a model. Therefore, it's recommended that you to check any extracted equations before using them to create a model. + - When you modify equations extract from a document, your changes are only saved to the current Create model from equations operator. + +
+![](../img/models/model-equations-resource.png) +
How it works: [Model Service](https://github.com/DARPA-ASKEM/model-service/blob/07ae21cae2d5465f9ac5b5bbbe6c7b28b7259f04/src/ModelService.jl#L54) :octicons-link-external-24:{ alt="External link" title="External link" }
+
+ +
+ +- :material-arrow-collapse-right:{ .lg .middle aria-hidden="true" } __Inputs__ + + --- + + Document (optional) + +- :material-arrow-expand-right:{ .lg .middle aria-hidden="true" } __Outputs__ + + --- + + Model + +
+ +## Recommended LaTeX format + +
+ +- __Derivatives__ + + --- + + - Derivatives must be written in Leibniz notation (`\frac{d X}{d t}`). Convert any equations using Newton or Lagrange notation. + - Place first-order derivatives to the left of the equal sign. + +- __Mathematical notations__ + + --- + + Avoid using: + + - `(t)` to denote time. + - Superscripts and LaTeX superscripts `^`, particularly to denote sub-variables. + - Parentheses. + - Capital sigma (Σ) and pi (Π) notations for summation and product. + - Homoglyphs (characters that look similar but have different meanings). + +- __Subscripts__ + + --- + + Subscripts using LaTeX `_` are permitted, but ensure that all characters in the subscript are enclosed by curly brackets `{...}`. + +- __Variable names__ + + --- + + Avoid words or multi-character names for variables. Instead, use camel case to represent multi-word or multi-character names. + +- __ASCII characters__ + + --- + + Avoid using non-ASCII characters when possible. + +- __Symbolic notation for multiplication__ + + --- + + Do not use `\cdot` or `*` to indicate multiplication. Instead, use whitespace between variables. + +- __Symbolic notation for multiplication__ + + --- + + Avoid using symbolic notation for constants like `e` and `pi`. Instead, use their actual values (up to 3 decimal places). + +
+ +## Create a model from equations extracted from a document + +???+ list "To extract model equations from a document" + + 1. Add the Document to a workflow graph. + 3. Hover over the output of the Document resource and click :octicons-plus-24:{ title="Link" aria-label="Link" } > **Create model from equations**. + 4. Click **Edit** on the Create model from equations operator. + 5. To customize the extracted equations: + - Edit the LaTeX expressions. + - Clear a checkbox to remove an equation. + - Use the text box to add LaTeX equations to enter a new equation and click **Add**. + 6. Choose the **framework** for the new model. + 7. Click **Run**. + 8. Review the model preview and click **Save as new model**. + +## Create a model by uploading or entering equations + +???+ list "To create a model from manually entered equations" + + 1. Right-click anywhere on the workflow graph and select **Modeling** > **Create model from equations**. + 2. Click **Edit** on the Create model from equations operator. + 3. Use the text box to add LaTeX equations to enter a new equation and click **Add**. + 5. Repeat step 3 for each equation you want to add. + 6. Enter a name for the new model and click **Run**. \ No newline at end of file diff --git a/docs/modeling/edit-model.md b/docs/modeling/edit-model.md new file mode 100644 index 0000000..e5413d7 --- /dev/null +++ b/docs/modeling/edit-model.md @@ -0,0 +1,66 @@ +--- +title: "Edit a model" +--- + +# Edit a model + +Model editing lets you build on existing models. You can add, remove, or change state variables, transitions, parameters, rate laws, and observables. + +How you implement your edits is up to you. You can write your own code or prompt the integrated AI assistant to generate and execute code for you. + +
+![](../img/models/model-edit-operator.png) +
How it works: [MIRA Model Edit](https://darpa-askem.github.io/askem-beaker/contexts_mira_model_edit.html) :octicons-link-external-24:{ alt="External link" title="External link" }
+
+ +
+ +- :material-arrow-collapse-right:{ .lg .middle aria-hidden="true" } __Inputs__ + + --- + + Model + +- :material-arrow-expand-right:{ .lg .middle aria-hidden="true" } __Outputs__ + + --- + + Edited model + +
+ +## Edit a model with the AI assistant + +Using the Notebook view of the Edit model operator, you can steer an AI assistant to quickly create, modify, and refine a model from code. All generated code is directly editable, allowing you to make modifications at any time. + +???+ note + + Each time you submit a new prompt or command, the generated code is added below any existing code. Each time you run the code, the whole notebook is executed. + +???+ list "To edit a model using the AI assistant" + + 2. Click in the prompt field to see suggested actions. Select a suggestion or enter your own command using plain language. + 3. Click :material-play-outline:{ aria-hidden="true" } **Run**. + + ???+ tip + + To generate and preview model code but not run it, click :fontawesome-regular-paper-plane:{ alt="Generate code" title="Generate code"}. + + 4. Review the resulting model diagram in the preview on the right. + 5. To make changes, submit a new prompt that clarifies or builds on what you asked for or directly edit the generated code. + 6. Click :material-play-outline:{ aria-hidden="true" } **Run** again to update the model. + 7. To save an iteration of the model at any point, click **Save for re-use**, enter a unique name and click **Save**. + +## Edit a model with your own code + +More info coming soon. + +## Use the edit model operator output + +More info coming soon. + +- [Edit model](#) +- [Stratify model](stratify-model.md) +- [Compare models](compare-models.md) +- [Configure model](../config-and-intervention/configure-model.md) +- [Create intervention policy](../config-and-intervention/create-intervention-policy.md) diff --git a/docs/modeling/enrich-descriptions.md b/docs/modeling/enrich-descriptions.md new file mode 100644 index 0000000..8525841 --- /dev/null +++ b/docs/modeling/enrich-descriptions.md @@ -0,0 +1,13 @@ +--- +title: "Enrich model metadata" +--- + +# Enrich model metadata + +Terarium uses an enrichment service to automatically populate model metadata based on related publications in your project. When you enrich a model, the service looks at your related publications to find and apply definitions for concepts in the model. + +???+ list "To enrich a model description" + + 1. In the Resources panel, click the model open it. + 2. Click **Enrich metadata**. + 3. Select the resource you want to use or select **Generate information without context** and click **Enrich**. \ No newline at end of file diff --git a/docs/modeling/index.md b/docs/modeling/index.md new file mode 100644 index 0000000..5325c7d --- /dev/null +++ b/docs/modeling/index.md @@ -0,0 +1,105 @@ +--- +title: "Work with a model" +--- + +# Work with a model + +In Terarium, a model is an abstract representation that approximates the behavior of a system. You can build a chain of complex operations to edit, configure, stratify, calibrate, and simulate models. + +![Workflow graph with an SIR model and related dataset passed into calibration and simulation operations](../img/models/description.png) + +## Model resource + +A model resource represents a model you've uploaded or created in Terarium. In the workflow graph, you can display its diagram or equations. It takes no inputs and outputs the model for use in modeling, configuration, and intervention operators. + +![Model resource for a SIDARTHE model of COVID-19 with a diagram preview](../img/models/model-resource.png) + +
+ +- :material-arrow-collapse-right:{ .lg .middle aria-hidden="true" } __Inputs__ + + --- + + None + +- :material-arrow-expand-right:{ .lg .middle aria-hidden="true" } __Outputs__ + + --- + + Model + +
+ +## What can I do with a model resource? + +You can connect a model resource to the following operators: + +- [Edit model](edit-model.md): Add, remove, or change state variables, transitions, parameters, rate laws, and observables. +- [Stratify model](stratify-model.md): Divide populations into subsets along demographic characteristics such as age and location. +- [Compare models](compare-models.md): Compare side-by-side with other models to understand their similarities and differences. +- [Configure model](../config-and-intervention/configure-model.md): Set the initial values and parameters for the condition you want to test +- [Create intervention policy](../config-and-intervention/create-intervention-policy.md): Create static and dynamic interventions for "what-if" scenarios. + +## Add a model to a workflow + +!!! note + + For information about: + + - Uploading models, see [Upload modeling resources](../upload-resources/index.md). + - Creating models, see [Edit model](edit-model.md). + +Once you create a model or add it to your project, you can open it to view more details or add it to a workflow for use in scientific modeling processes. + +???+ list "To add a model to a workflow" + + - From the Resources panel, drag the model into a workflow graph. + +## Understand model representations + +Each model has a detailed view that summarizes the following extracted details: + +- Description +- Diagram +- Model equations +- State variables +- Parameters +- Observables +- Transitions +- Time +- Provenance +- Other concepts + +To support systematic curation of model structure and extracted metadata, you can view and [edit](edit-model.md) different representations of a model: + +- A diagram that summarizes the different states and transitions in the model. +- Equations that show the same. + +![Diagram and equations representing the structure of an SIR model](../img/models/diagram.png) + +???+ list "To view model details" + + - Click the model name in the Resources panel. + +## Manage models + +You can create, copy, and edit models for use in your workflows. + +??? list "To create a new model" + + 1. In the Resources panel, click :octicons-plus-24:{ aria-hidden="true" } **New** in the Model section. + 2. Enter a unique name for the model and click **Save**. + 3. From the Resources panel, drag the new model into a workflow. + 4. Hover over the Model output and click :octicons-plus-24:{ title="Link" aria-label="Link" } > **Edit model**. + 5. Click **Edit** on the Edit model operator. + 6. Use the AI assistant or code editor to construct a set of equations that describe the model. + +??? list "To rename a model" + + * Click :fontawesome-solid-ellipsis-vertical:{ title="Menu" } > :octicons-pencil-24:{ aria-hidden="true"} **Rename**, type a unique name for the model, and press ++enter++. + +??? list "To copy a model" + + 1. Add the Model operator to a workflow graph and connect it to an Edit model operator. + 2. Click **Edit** on the Edit model operator. + 3. Click **Save for re-use**, enter a name for the copy, and click **Save**. diff --git a/docs/models/stratify-model.md b/docs/modeling/stratify-model.md similarity index 55% rename from docs/models/stratify-model.md rename to docs/modeling/stratify-model.md index 90ed7a7..5eff1fb 100644 --- a/docs/models/stratify-model.md +++ b/docs/modeling/stratify-model.md @@ -4,7 +4,7 @@ title: "Stratify a model" # Stratify a model -By stratifying a model, you divide its populations into subsets along demographic characteristics such as age and location. +Stratifying a model divides its populations into subsets along demographic characteristics such as age and location. ![](../img/models/stratify.png) @@ -34,36 +34,37 @@ Terarium automatically updates both the graphical representation of the model an To create complex stratification schemes, chain several Stratify model operators together. To facilitate the interpretation of the generated matrices, use as many parameters as the number of stratification levels. -

To stratify a model

+???+ list "To stratify a model" -1. Add the model to a workflow graph. -2. Right-click anywhere on the workflow graph and select **Work with model** > **Stratify model**. -3. Connect the output of the Model resource to the input of the Stratify model operator. -4. On the Stratify model operator, click **Edit**. - - ??? tip - - Only stratify the parameters that differ for each strata. - -5. Enter a **Name** for the strata you want to add. -6. Use the **Select variables and parameters to stratify** dropdown. -7. Enter a comma-separated list of labels for each of the strata groups: - ``` - A1, A2 - ``` -8. Choose whether you want to: - - Create new transitions between strata + 1. Add the model to a workflow, hover over its output, and click :octicons-plus-24:{ title="Link" aria-label="Link" } > **Stratify model**. + 2. On the Stratify model operator, click **Edit**. + 3. Enter a **Name** for the strata you want to add. + 4. Use the **Select variables and parameters to stratify** dropdown. ??? tip - - - When stratifying by age, this option is usually off, as people do not change age groups. - - When stratifying by location, this option can be turned on if people are allowed to travel between locations. - - - Allow existing interactions to involve multiple strata - -9. Click :material-play-outline:{ aria-hidden="true" } **Stratify**. -10. Review the Preview of the stratified model. -11. Click **Save as new model**. + + Only stratify the parameters that differ for each strata. + + 5. Enter a comma-separated list of labels for each of the strata groups: + ``` + A1, A2 + ``` + 6. Choose whether you want to: + + - Create new transitions between strata + + ??? tip + + ![](../img/models/stratify-options.png) + + - When stratifying by age, this option is usually off, as people do not change age groups. + - When stratifying by location, this option can be turned on if people are allowed to travel between locations. + + - Allow existing interactions to involve multiple strata + + 7. Click **Stratify**. + 8. Review the preview of the stratified model. + 9. Click **Save for re-use**. ## Interpret a stratified model @@ -74,11 +75,11 @@ Terarium previews the a stratified model as: When reviewing the stratified model, you can get a sense of the initial variable and parameter values as a hierarchical list, but the results are easiest to understand in matrix form. -??? list "To review stratified model initials or parameters" +???+ list "To review stratified model initials or parameters" Perform one of the following actions: - - To view values as a hierarchical list, click :fontawesome-solid-angle-right:{ alt="Show strata" title="Show strata"} next a symbol name. - - To view values as a matrix, click :fontawesome-solid-table-cells-large:{ aria-hidden="true"} **Matrix**. + - To view values as a hierarchical list, click :fontawesome-solid-angle-right:{ alt="Show strata" title="Show strata"} next to a symbol name. + - To view values as a matrix, click :fontawesome-solid-table-cells-large:{ aria-hidden="true"} **Open matrix**. The block matrices in the model diagram should hint at the collapsed transitions. \ No newline at end of file diff --git a/docs/models/calibrate-model.md b/docs/models/calibrate-model.md deleted file mode 100644 index f0e50ca..0000000 --- a/docs/models/calibrate-model.md +++ /dev/null @@ -1,88 +0,0 @@ ---- -title: "Calibrate a model" ---- - -# Calibrate a model - -Calibrating a model lets you determine or update the value of model parameters given a reference dataset of observations. - -![Output ports (left) for a model and a dataset connected to the input ports of a calibrate operation](../img/workflows/nodes.png) - -Terarium supports two types of calibrations: deterministic (with SciML) and probabilistic (with PyCIEMSS). - -## Deterministic calibration - -Deterministic calibrations (SciML) take a model configuration and a dataset as an input. - -
![](../img/models/model-calibrate-sciml-operator.png)
How it works: [sciml-service](https://github.com/DARPA-ASKEM/sciml-service/blob/main/src/operations.jl#L245) :octicons-link-external-24:{ alt="External link" title="External link" }
- -
- -- :material-arrow-collapse-right:{ .lg .middle aria-hidden="true" } __Inputs__ - - --- - - - Model configuration - - Dataset - -- :material-arrow-expand-right:{ .lg .middle aria-hidden="true" } __Outputs__ - - --- - - Calibrated model configuration - -
- -

To calibrate a model

- -1. Add the model and reference dataset to a workflow graph. -2. Connect the Model operator output to a Configure model operator. See [Configure a model](configure-model.md) for information on selecting a model configuration. - - ??? tip - - At least one parameter in the configuration needs to be defined as a distribution, and there needs to be a timestamp column. - -3. Right-click anywhere on the workflow graph and select **Run model** > **Calibrate with SciML**. -4. Connect the outputs of the Configure model operator and Dataset resource to the inputs on the Calibrate with SciML operator. -5. Click **Edit** on the Calibrate with SciML operator. -6. In the Mapping section, map the model variables to the columns in the dataset. - - ??? tip - - If the model concepts are assigned to variables, click :octicons-plus-24:{ aria-hidden="true" } **Auto map** to speed the alignment process. - -7. Configure the calibration options as needed: - - **Chains**: Number of parallel chains to run during the calibration. - - **Iterations**: Number of steps each chain should take. - - **ODE method**: Method to solve ordinary differential equations. - - **Calibrate method**: Approach to calibration (*bayesian*, *local*, or *global*). -8. Click :material-play-outline:{ aria-hidden="true" } **Run**. - -## Probabilistic calibration - -Probabilistic calibrations (PyCIEMSS) take a model configuration and a dataset as an input. - -
![](../img/models/model-calibrate-pyciemss-operator.png)
How it works: [PyCIEMSS](https://github.com/ciemss/pyciemss/blob/main/pyciemss/interfaces.py#L529) :octicons-link-external-24:{ alt="External link" title="External link" }
- -
- -- :material-arrow-collapse-right:{ .lg .middle aria-hidden="true" } __Inputs__ - - --- - - - Model configuration - - Dataset - -- :material-arrow-expand-right:{ .lg .middle aria-hidden="true" } __Outputs__ - - --- - - Calibrated model configuration - -
- -## Intermediate results - -

To view calibration results

- -- Use the **Select variables to plot** dropdown to choose the variables you want to see, or click **Add Chart** to add another chart. \ No newline at end of file diff --git a/docs/models/edit-model.md b/docs/models/edit-model.md deleted file mode 100644 index e5fbeb5..0000000 --- a/docs/models/edit-model.md +++ /dev/null @@ -1,93 +0,0 @@ ---- -title: "Edit a model" ---- - -# Edit a model - -You can use Terarium to edit model resources in your project or even create new ones. The Edit model operator allows you to compose or modify a diagram of the model states and transitions or prompt an AI assistant to generate code that makes your requested changes. - -
-![](../img/models/model-edit-operator.png) -
How it works: [MIRA Model Edit](https://darpa-askem.github.io/askem-beaker/contexts_mira_model_edit.html) :octicons-link-external-24:{ alt="External link" title="External link" }
-
- -
- -- :material-arrow-collapse-right:{ .lg .middle aria-hidden="true" } __Inputs__ - - --- - - Model - -- :material-arrow-expand-right:{ .lg .middle aria-hidden="true" } __Outputs__ - - --- - - Edited model - -
- -## Edit a model diagram - -The Wizard view of the Edit model operator represents each model as a diagram where: - -- State variables are nodes. -- Dependencies are directed edges. - -![Diagram showing the connected states and transitions in a SIR model](../img/models/diagram-edit.png) - -For each model framework, the editor has a set of model templates that you can use to quickly edit or create different parts of the model. Available templates vary depending on the selected framework. Options include: - -
-
-- **Natural conversion**: subject converts into outcome at a constant rate. -- **Natural production**: outcome is generated at a constant rate. -- **Natural degradation**: subject is destroyed at a constant rate -- **Controlled conversion**: subject converts into outcome at a rate that depends on some controller variable -- **Controlled production**: outcome is produced at a rate that depends on some controller variable -- **Controlled degradation**: subject is destroyed at a rate that depends on some controller variable -
-
-- **Grouped controlled conversion**: subject converts into outcome at a rate that depends on several controller variables -- **Grouped controlled production**: subject is produced at a rate that depends on several controller variables -- **Grouped controlled degradation**: subject converts into outcome at a rate that depends on several controller variables -- **Natural replication**: subject is generated at a rate that depends on itself -- **Controlled replication**: subject generates at a rate that depends on itself and some controller variable
-
- -The model diagram is linked to the model equations. Changes you make to the diagram are automatically reflected in the equations. - -??? list "To begin editing a model diagram" - - 1. Add the Model to a workflow graph. - 2. Right-click anywhere on the workflow graph and select **Work with model** > **Edit model**. - 3. Connect the output port of the Model operator to the input port of the Edit model operator. - 4. Click **Edit** on the Edit model operator. - 5. Edit the model diagram: - - To add new states and transitions, drag model templates from the panel on the left into the diagram. - - To rename a state or transition, click the label of the node. - - To link states across templates, click an output port on a template and connect it to another and output port on another template. - 6. To view the edited model, click **Flattened**. - -## Edit a model with the AI assistant - -Using the Notebook view of the Edit model operator, you can steer an AI assistant to quickly create, modify, and refine a model from code. All generated code is directly editable, allowing you to make modifications at any time. - -???+ note - - Each time you submit a new prompt or command, the generated code is added below any existing code. Each time you run the code, the whole notebook is executed. - -??? list "To edit a model using the AI assistant" - - 1. In the Edit model operator details, click **Notebook**. - 2. Click :fontawesome-solid-angle-down:{ title="Suggested prompts" alt="Suggested prompts"" } in the prompt field to see suggested actions. Select a suggestion or enter your own command using plain language. - 3. Click :material-play-outline:{ aria-hidden="true" } **Run**. - - ???+ tip - - To generate model code but not run and preview it, click :fontawesome-regular-paper-plane:{ alt="Generate code" title="Generate code"}. - - 4. Review the resulting model diagram in the Preview. - 5. To make changes, submit a new prompt that clarifies or builds on what you asked for or directly edit the generated code. - 6. Click :material-play-outline:{ aria-hidden="true" } **Run** again to update the model. - 7. To save an iteration of the model at any point, enter a unique **name** and click **Save as new model**. diff --git a/docs/models/enrich-descriptions.md b/docs/models/enrich-descriptions.md deleted file mode 100644 index 65873ee..0000000 --- a/docs/models/enrich-descriptions.md +++ /dev/null @@ -1,13 +0,0 @@ ---- -title: "Enrich model descriptions" ---- - -# Enrich model descriptions - -Terarium uses an enrichment service to automatically populate model descriptions based on related publications in your project. When you enrich a model, the service looks at your related publications to find and apply definitions for concepts in the model. - -

To enrich a model description

- -1. In the Resources panel, click the model open it in a new tab. -2. In the Provenance section, click **Enrich description**. -3. Select the resource you want to use and click **Use Document to enrich description**. \ No newline at end of file diff --git a/docs/models/extract-code.md b/docs/models/extract-code.md deleted file mode 100644 index 3afb876..0000000 --- a/docs/models/extract-code.md +++ /dev/null @@ -1,73 +0,0 @@ ---- -title: "Create a model from code" ---- - -# Create a model from code - -In addition to the models you can find by using the Explorer, you can create models from code ([python](https://www.python.org/doc/) :octicons-link-external-24:{ aria-hidden="true" }, [r](https://www.r-project.org/) :octicons-link-external-24:{ aria-hidden="true" }, or [julia](https://julialang.org/) :octicons-link-external-24:{ aria-hidden="true" }) that you extract from a document or manually enter. - -![Python code for a SEIRD model on the Extract model page](../img/models/extract-code.png) - -Terarium considers all the provided code, including comments, to extract and define the underlying model. You can mark a code block as containing the core model dynamics by highlighting it before you begin the extraction. - -??? list "To create a code asset" - - 1. In the Resources panel, click :octicons-plus-24:{ aria-hidden="true" } **New** in the Code section. - 2. Use the dropdown to specify the type of code (julia, python, or r). - 3. Paste your code in the code editor. - 4. Highlight a part of the code that represents the core model dynamics and click :octicons-plus-24:{ aria-hidden="true" } **Add a code block**. - 5. Enter a name for the code block and click **Save**. - 6. Click **Apply changes**. - -## Extract a model from code - -To extract a model from a code asset or document, use the Create model from code operator. - -
-![](../img/models/model-code-resource.png) -
How it works: [Code2AMR](https://github.com/ml4ai/ASKEM-TA1-DockerVM?tab=readme-ov-file#code2amr) :octicons-link-external-24:{ alt="External link" title="External link" }
-
- -
- -- :material-arrow-collapse-right:{ .lg .middle aria-hidden="true" } __Inputs__ - - --- - - - Code asset (optional) - - Document (optional) - -- :material-arrow-expand-right:{ .lg .middle aria-hidden="true" } __Outputs__ - - --- - - Model - -
- -??? list "To extract a model from a code asset" - - 1. From the Resources panel, drag the Code asset into a workflow graph. - 2. Right-click anywhere on the workflow graph and select **Add resource** > **Create model from code**. - 3. Connect the output of the Code asset to the input of the Create model from code operator. - 4. On the Create model from code operator, click **Configure**. - 5. Scroll through the available code blocks and clear **Include in process** to deselect any you don't want to include in the model. - 6. Click **Run**. - -??? list "To extract a model from a document" - - 1. From the Resources panel, drag the Document into a workflow graph. - 2. Right-click anywhere on the workflow graph and select **Add resource** > **Create model from code**. - 3. Connect the output of the Document resource to the input of the Create model from code operator. - 4. On the Create model from code operator, click **Configure**. - 5. Scroll through the available code blocks and clear **Include in process** to deselect any you don't want to include in the model. - 6. Click **Run**. - -??? list "To enter code and create a model from it" - - 1. Right-click anywhere on the workflow graph and select **Add resource** > **Create model from code**. - 2. On the Create model from code operator, click **Configure**. - 3. Use the dropdown to specify the type of code (julia, python, or r). - 4. Paste your code in the code editor. - 5. Select **Include in process** - 6. Click **Run**. diff --git a/docs/models/index.md b/docs/models/index.md deleted file mode 100644 index 7297365..0000000 --- a/docs/models/index.md +++ /dev/null @@ -1,136 +0,0 @@ ---- -title: "Work with a model" ---- - -# Work with a model - -In Terarium, a model is an abstract representation that approximates the behavior of a system. In the workflow graph, you can build a chain of complex operations to configure, stratify, calibrate, and simulate models. - -![Workflow graph with an SIR model and related dataset passed into calibration and simulation operations](../img/models/description.png) - -## Add a model to your project - -For information about uploading or finding models, see [Gather modeling resources](../modeling-resources/index.md). - -Once you add model to your project, you can open it to view more details or add it to a workflow for use in various scientific modeling processes. - -![Model resource for a SIDARTHE model of COVID-19 with a diagram preview](../img/models/model-resource.png) - -
- -- :material-arrow-collapse-right:{ .lg .middle aria-hidden="true" } __Inputs__ - - --- - - None - -- :material-arrow-expand-right:{ .lg .middle aria-hidden="true" } __Outputs__ - - --- - - Model - -
- -

To add a model to a workflow

- -- From the Resources panel, drag the model into a workflow graph. - -## Create a model from an equation - -You can create models from equations that have been extracted from a paper or manually enter your own equations. - -
-![](../img/models/model-equations-resource.png) -
How it works: [Model Service](https://github.com/DARPA-ASKEM/model-service/blob/07ae21cae2d5465f9ac5b5bbbe6c7b28b7259f04/src/ModelService.jl#L54) :octicons-link-external-24:{ alt="External link" title="External link" }
-
- -
- -- :material-arrow-collapse-right:{ .lg .middle aria-hidden="true" } __Inputs__ - - --- - - Document (optional) - -- :material-arrow-expand-right:{ .lg .middle aria-hidden="true" } __Outputs__ - - --- - - Model - -
- -??? list "To extract model equations from a publication" - - 1. Add the Document to a workflow graph and click **Open**. - 2. Scroll through the extracted Equation images and select **Include in process** for each one you want to add to the model. - 3. Right-click anywhere on the workflow graph and select **Work with model** > **Create model from equations**. - 4. Connect the output of the Document resource to the input of the Create model from equations operator. - 5. Click **Edit** on the Create model from equations operator. - 6. To customize the selected equations: - - Edit the LaTeX expressions. - - Clear **Include in process** to remove an equation. - - Click :octicons-plus-24:{ aria-hidden="true" } **Add an equation** to enter a new equation. - 7. Choose the **framework** for the new model. - 8. Click **Run**. - 9. Review the model preview and click **Save as new model**. - -??? list "To create a model from manually entered equations" - - 1. Right-click anywhere on the workflow graph and select **Work with model** > **Create model from equations**. - 2. Click **Edit** on the Create model from equations operator. - 3. Click :octicons-plus-24:{ aria-hidden="true" } **Add an equation**. - 4. Enter an equation as a LaTeX expression. - 5. Repeat steps 3–4 for each equation you want to add. - 6. Enter a name for the new model and click **Run**. - -## Understand model representations - -Each model has a detailed view that summarizes the following extracted details: - -- Description -- Diagram -- Provenance -- Model equations -- Observables -- State variables -- Parameters -- Transitions -- Other concepts -- Time - -To support systematic curation of model structure and extracted metadata, you can view and [edit](edit-model.md) different representations of a model: - -- A diagram that summarizes the different states and transitions in the model. -- Equations that show the same. - -![Diagram and equations representing the structure of an SIR model](../img/models/diagram.png) - -

To view model details

- -- Click the model name in the Resources panel to open it in a new tab. - -## Manage models - -You can create, copy, and edit models for use in your workflows. - -??? list "To create a new model" - - 1. In the Resources panel, click :octicons-plus-24:{ aria-hidden="true" } **New** in the Model section. - 2. Enter a unique name for the model and click **Create model**. - 3. From the Resources panel, drag the new model into a workflow. - 4. Right-click anywhere on the workflow graph and select **Work with model** > **Edit model**. - 5. Click the output of the Model operator and then click the input of the Edit model operator to connect them. - 6. Click **Edit** on the Edit model operator. - 7. Use the Edit model Wizard or Notebook view to construct a set of equations that describe the model. - -??? list "To rename a model" - - * Click :fontawesome-solid-ellipsis-vertical:{ title="Menu" } > :octicons-pencil-24:{ aria-hidden="true"} **Rename**, type a unique name for the model, and press ++enter++. - -??? list "To make a copy of a model" - - 1. Add the Model operator to a workflow graph and connect it to an Edit model operator. - 2. Click **Edit** on the Edit model operator and then select the Notebook tab. - 3. Enter a name for the copy and click **Save as new model**. diff --git a/docs/models/optimize-model.md b/docs/models/optimize-model.md deleted file mode 100644 index dd89b2d..0000000 --- a/docs/models/optimize-model.md +++ /dev/null @@ -1,52 +0,0 @@ ---- -title: Optimize model ---- - -# Optimize model - -When you optimize a model, you specify a set of interventions and constraints to find the optimal combination of parameters that match your objective. - -
![](../img/models/model-optimize-operator.png)
How it works: [PyCIEMSS](https://github.com/ciemss/pyciemss/blob/main/pyciemss/interfaces.py#L747) :octicons-link-external-24:{ alt="External link" title="External link" }
- -
- -- :material-arrow-collapse-right:{ .lg .middle aria-hidden="true" } __Inputs__ - - --- - - Model configuration - -- :material-arrow-expand-right:{ .lg .middle aria-hidden="true" } __Outputs__ - - --- - - Optimized model configuration - -
- -To optimize a model - -1. [Configure the model](../models/configure-model.md). -2. Right-click anywhere on the workflow graph and select **Run model** > **Optimize with PyCIEMSS**. -3. Connect the Configure model output to the Optimize with PyCIEMSS input. -4. Click **Edit**. -5. Configure the optimization settings as needed: - - **Start** and **End time**: Specify the simulation time range. - - **Number of stochastic samples**: Enter the number of stochastic samples to draw from the model. - - **Solver method**: Choose whether to solve ordinary differential equations using [dopri5](https://en.wikipedia.org/wiki/Dormand-Prince_method) or [euler](https://en.wikipedia.org/wiki/Euler_method). - - **Minimized**: When true, the objective function will try to minimize the distance from your initial guess while respecting the constraint on the target variables with the acceptable risk of failure. -6. Specify your intervention policies: - - **Parameter**: the parameter you want to optimize. - - **Initial guess**: your initial guess for the optimization. - - **Lower** and **Upper bound**: how much the parameter can vary. - - **Start time**: the start time of the model. - - To set additional interventions, click :octicons-plus-24:{ aria-hidden="true" } **Add more interventions**. - -7. Add a constraint that limits the optimized configuration: - - Select the **target variables**. - - Set the **Acceptable risk of failure** and **Threshold**. -8. Click :material-play-outline:{ aria-hidden="true" } **Run**. -9. Save the the results: - - Enter a **Model config name** and click **Save as new model configuration**. - - Click :fontawesome-solid-floppy-disk:{ aria-hidden="true" } **Save as new dataset**. \ No newline at end of file diff --git a/docs/models/simulate-model.md b/docs/models/simulate-model.md deleted file mode 100644 index b10a518..0000000 --- a/docs/models/simulate-model.md +++ /dev/null @@ -1,128 +0,0 @@ ---- -title: "Simulate a model" ---- - -# Simulate a model - -Simulating a model lets you understand how the underlying system might behave under specific conditions. - -
- ![Workflow graph of two simulations (one deterministic and one probabilistic) of a SIDARTHE model of COVID-19 spread in Italy](../img/models/simulate.png) -
A deterministic simulation (center) showing various variables in a COVID-19 model (left) and a probabilistic simulation (right) showing the distribution of a single variable with uncertainty.
-
- -Terarium supports two types of simulations: deterministic (with SciML) and probabilistic (with PyCIEMSS). - -??? tip - - Simulating early with a simple model (one with a population of 1,000) can help you spot issues and fix them before you start using a more complex model. - -## Deterministic simulations - -Deterministic simulations (SciML) take a model configuration as an input and run a single simulation across an editable time range. - -
![](../img/models/simulate-sciml-operator.png)
How it works: [sciml-service](https://github.com/DARPA-ASKEM/sciml-service/blob/main/src/operations.jl#L222) :octicons-link-external-24:{ alt="External link" title="External link" }
- -
- -- :material-arrow-collapse-right:{ .lg .middle aria-hidden="true" } __Inputs__ - - --- - - Model configuration - -- :material-arrow-expand-right:{ .lg .middle aria-hidden="true" } __Outputs__ - - --- - - Simulation data - -
- -??? list "To run a deterministic simulation" - - 1. Add the model to a workflow graph. - 2. Connect the Model operator output to a Configure model operator. See [Configure a model](configure-model.md) for information on selecting a model configuration. - 3. Right-click anywhere on the workflow graph and select **Run model** > **Simulate with SciML**. - 4. Connect the Configure model operator output to the Simulate with SciML input. - 5. Click **Edit** on the Simulate with SciML operator. - 6. Configure the simulation options as needed: - - **Start time** and **End time**: Specify the simulation time range. - 8. Click :material-play-outline:{ aria-hidden="true" } **Run**. - 9. Review the output and click :fontawesome-solid-floppy-disk:{ aria-hidden="true" } **Save as new dataset** to add the results to your project resources. - -## Probabilistic simulations - -Probabilistic simulations (PyCIEMSS) take a model configuration and an optional model calibration as inputs and run an editable number of samples to account for uncertainty in results. - -
![](../img/models/simulate-pyciemss-operator.png)
How it works: [PyCIEMSS](https://github.com/ciemss/pyciemss/blob/main/pyciemss/interfaces.py#L323) :octicons-link-external-24:{ alt="External link" title="External link" }
- -
- -- :material-arrow-collapse-right:{ .lg .middle aria-hidden="true" } __Inputs__ - - --- - - - Model configuration - - Calibration (optional) - -- :material-arrow-expand-right:{ .lg .middle aria-hidden="true" } __Outputs__ - - --- - - Simulation data - -
- -??? list "To run a probabilistic simulation" - - 1. Add the model to a workflow graph. - 2. Connect the Model operator output to a Configure model operator. See [Configure a model](configure-model.md) for information on selecting a model configuration. - 3. Right-click anywhere on the workflow graph and select **Run model** > **Simulate with PyCIEMSS**. - 4. Connect the Configure model operator output to the Simulate with PyCIEMSS input. - 5. Click **Edit** on the Simulate with PyCIEMSS operator. - 6. Configure the simulation options as needed: - - **Start time** and **End time**: Specify the simulation time range. - - **Number of samples**: Enter the number of stochastic samples to generate. - - **Method**: Choose whether to solve ordinary differential equations using [dopri5](https://en.wikipedia.org/wiki/Dormand-Prince_method) or [euler](https://en.wikipedia.org/wiki/Euler_method). - - ??? tip - - Using a low number of samples and the dopri5 method can speed up your runtime for debugging purposes. - - 7. Click :material-play-outline:{ aria-hidden="true" } **Run**. - 8. Review the output and click :fontawesome-solid-floppy-disk:{ aria-hidden="true" } **Save as new dataset** to add the results to your project resources. - -## View and save simulation results - -Simulation results are summarized on each Simulate operator and in the operator details. You can choose which variables to plot. - -??? list "To view simulation results" - - 1. In the output section of the Simulate operator details, use the dropdown list to choose the variables you want to visualize. If you choose multiple variables, Terarium overlaps them on the same plot. - 2. To add an additional chart, click :octicons-plus-24:{ aria-hidden="true"} **Add chart** and repeat the previous step. - - ??? tip - - It's recommended to use multiple charts if the variables you want to display have very different magnitudes. - -??? list "To save simulation results as a new dataset" - - - On the Simulate pane, click :material-content-save-outline:{ aria-hidden="true" } **Save as new dataset**. - - \ No newline at end of file diff --git a/docs/multiple-models/compare-models.md b/docs/multiple-models/compare-models.md deleted file mode 100644 index 1e78922..0000000 --- a/docs/multiple-models/compare-models.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -title: Compare models ---- - -# Compare models - -You can compare two or models in Terarium to see: - -- A brief description of their similarities and differences. -- Side-by-side model card comparisons. -- An AI-assisted visualization of common and unique state variables and transitions across each model. - -
![](../img/models/model-compare-operator.png)
How it works: [MIRA](https://github.com/gyorilab/mira/blob/7314765ab409ddc9647269ad2381055f1cd67706/notebooks/hackathon_2023.10/dkg_grounding_model_comparison.ipynb#L307) :octicons-link-external-24:{ alt="External link" title="External link" }
- -
- -- :material-arrow-collapse-right:{ .lg .middle aria-hidden="true" } __Inputs__ - - --- - - Models - -- :material-arrow-expand-right:{ .lg .middle aria-hidden="true" } __Outputs__ - - --- - - N/A - -
- -## Compare model cards - -Model cards briefly summarize key aspects of the models. Available aspects include: - -
-
-- Model name -- Model details -- Uses -- Bias risks limitations -- Evaluation -- Technical specifications -
-
-- Glossary -- Model card authors -- How to get started with the model -- Citation -- More information -- Structural information -
-
- -

To compare models

- -1. From the Resources, drag the models into a workflow graph. -2. Right-click anywhere on the workflow graph and select **Work with multiple models** > **Compare models**. -3. Connect the outputs of the Model resources to the inputs of the Compare models operator. -4. Click **Open**. -5. Scroll down to review the side-by-side model cards. The overview at the top of the page may take a moment to load. - -## Compare model diagrams - -The AI assistant in the Compare models operator can visualize the similarities and differences of the attached models. It creates a comparison diagram for every pair of input models that color-codes the state variables and transitions that are unique to each model and common to both. - -

To compare models

- -1. In the Compare models operator details, click **Notebook**. -2. Click :fontawesome-solid-angle-down:{ title="Suggested prompts" alt="Suggested prompts"" } in the prompt field and select **Compare the models and visualize and display them**. -3. Click :material-play-outline:{ aria-hidden="true" } **Run**. -4. Review the model diagrams in the Preview. -5. To make changes, modify the selected prompt or the generated code and click :material-play-outline:{ aria-hidden="true" } **Run** again. \ No newline at end of file diff --git a/docs/multiple-models/couple-models.md b/docs/multiple-models/couple-models.md deleted file mode 100644 index ca5d1c1..0000000 --- a/docs/multiple-models/couple-models.md +++ /dev/null @@ -1,25 +0,0 @@ ---- -title: Couple models ---- - -# Couple models - -More info coming soon. - -
![](../img/models/model-couple-operator.png)
How it works: [Decapodes.jl](https://algebraicjulia.github.io/Decapodes.jl/dev/overview/#Merging-Multiple-Physics) :octicons-link-external-24:{ alt="External link" title="External link" }
- -
- -- :material-arrow-collapse-right:{ .lg .middle aria-hidden="true" } __Inputs__ - - --- - - Models - -- :material-arrow-expand-right:{ .lg .middle aria-hidden="true" } __Outputs__ - - --- - - Coupled model - -
\ No newline at end of file diff --git a/docs/multiple-models/simulate-ensemble.md b/docs/multiple-models/simulate-ensemble.md deleted file mode 100644 index 0b73038..0000000 --- a/docs/multiple-models/simulate-ensemble.md +++ /dev/null @@ -1,38 +0,0 @@ ---- -title: Simulate ensemble ---- - -# Simulate ensemble - -More info coming soon. - -
![](../img/models/model-simulate-ensemble-operator.png)
How it works: [PyCIEMSS](https://github.com/ciemss/pyciemss/blob/main/pyciemss/interfaces.py#L35) :octicons-link-external-24:{ alt="External link" title="External link" }
- -
- -- :material-arrow-collapse-right:{ .lg .middle aria-hidden="true" } __Inputs__ - - --- - - Model configurations - -- :material-arrow-expand-right:{ .lg .middle aria-hidden="true" } __Outputs__ - - --- - - Simulation data - -
- -To run an ensemble simulation - -1. [Configure two or more models](../models/configure-model.md). -2. Right-click anywhere on the workflow graph and select **Work with multiple models** > **Simulate ensemble**. -3. Connect the Configure model outputs to the Simulate model inputs. -4. Click **Edit**. -5. Enter custom weights for the model configurations or click **Set weights to be equal** to treat them as equally important. -6. Create a mapping between the state variables of the model configurations. -7. Set the time span: - - **Start** and **End Step** - - **Number of samples** -8. Click :material-play-outline:{ aria-hidden="true" } **Run**. \ No newline at end of file diff --git a/docs/notes/index.md b/docs/notes/index.md index 4941e0c..afeeeda 100644 --- a/docs/notes/index.md +++ b/docs/notes/index.md @@ -4,43 +4,33 @@ title: "Save insights as notes" # Save insights as notes -The Notes panel on the right side of the page lets you capture and save insights you've gained during your modeling processes. +You can capture and save insights you've gained during your modeling processes: -![Reference to scientific paper saved as an insight](../img/notes/note.png) - -To assist with publishing a paper, the Notes panel lets you organize insights into sections such as *Abstract*, *Intro*, *Methods*, *Results*, *Discussion*, or *References*. - -## Save an insight as a note - -Any notes you save apply to the current tab only. This means you can create unique notes for each resource (publications, documents, datasets, models, workflows) in your project. - -

To save an insight as a note

- -1. Click **<<** to expand the Notes panel. -2. Click **+ Add Note**. -3. Use the dropdown list to select which section you want to insert the note into. To add a general note, leave the section as *Unassigned*. -4. Type the contents of your insight in the text field and click **Save**. +- Directly on the operator where you discovered them. +- In the project overview. ## Add a note to a workflow operator -You can also add notes to operators in the workflow graph. +You can add notes to any operator in the workflow graph. -

To add a note to a workflow operator

+???+ list "To add a note to a workflow operator" -1. Perform one of the following actions: - - On the operator in the workflow graph, click :fontawesome-solid-ellipsis-vertical:{ title="Menu" alt="Menu" } > :octicons-pencil-24:{ aria-hidden="true"} **Add a note** - - In the operator details, click :octicons-pencil-24:{ aria-hidden="true"} **Add a note**. -2. Add your note and press ++enter++. + 1. Perform one of the following actions: + - On the operator in the workflow graph, click :fontawesome-solid-ellipsis-vertical:{ title="Menu" alt="Menu" } > :octicons-pencil-24:{ aria-hidden="true"} **Add a note** + - In the operator details, click :octicons-pencil-24:{ aria-hidden="true"} **Add a note**. + 2. Add your note and press ++enter++. ## Create a project overview The project overview page is another space for you to capture notes and summarize the findings and processes used in your modeling workflows. -???+ note "Saving workflows" +For example, to assist with publishing a paper, you could organize insights into sections such as *Abstract*, *Intro*, *Methods*, *Results*, *Discussion*, or *References*. + +???+ note "Saving your overview" Terarium automatically saves the state of your overview as you make changes. -

To edit the project overview

+???+ list "To edit the project overview" -1. In the Resources panel, click **Overview**. -2. Enter your project context in the text field. Use the style bar to format your text. \ No newline at end of file + 1. In the Resources panel, click **Overview**. + 2. Enter your insights in the text field. Use the style bar to format your text. \ No newline at end of file diff --git a/docs/multiple-models/calibrate-ensemble.md b/docs/simulation/calibrate-ensemble.md similarity index 100% rename from docs/multiple-models/calibrate-ensemble.md rename to docs/simulation/calibrate-ensemble.md diff --git a/docs/simulation/calibrate-model.md b/docs/simulation/calibrate-model.md new file mode 100644 index 0000000..1c13bca --- /dev/null +++ b/docs/simulation/calibrate-model.md @@ -0,0 +1,64 @@ +--- +title: "Calibrate a model" +--- + +# Calibrate a model + +Calibration lets you improve the performance of a model by updating the value of configuration parameters. You can calibrate a model with a reference dataset of observations and an optional intervention policy. + +![Output ports (left) for a model and a dataset connected to the input ports of a calibrate operation](../img/workflows/nodes.png) + +## Calibrate + +Calibrate takes a model configuration and a dataset as an input. + +
![](../img/models/model-calibrate-pyciemss-operator.png)
How it works: [PyCIEMSS](https://github.com/ciemss/pyciemss/blob/main/pyciemss/interfaces.py#L529) :octicons-link-external-24:{ alt="External link" title="External link" }
+ +
+ +- :material-arrow-collapse-right:{ .lg .middle aria-hidden="true" } __Inputs__ + + --- + + - Model configuration + - Dataset + - Interventions (optional) + +- :material-arrow-expand-right:{ .lg .middle aria-hidden="true" } __Outputs__ + + --- + + Calibrated model configuration + +
+ +???+ list "To calibrate a model" + + 1. Add the model and reference dataset to a workflow graph. + 2. Connect the Model operator output to a Configure model operator. See [Configure a model](../config-and-intervention/configure-model.md) for information on selecting a model configuration. + + ??? tip + + At least one parameter in the configuration needs to be defined as a distribution, and there needs to be a timestamp column. + + 3. Right-click anywhere on the workflow graph and select **Simulation** > **Calibrate**. + 4. Connect the outputs of the Configure model operator and Dataset resource to the inputs on the Calibrate operator. + 5. Click **Edit** on the Calibrate operator. + 6. In the Mapping section, map the model variables to the columns in the dataset. + + ??? tip + + If the model concepts are assigned to variables, click **Auto map** to speed the alignment process. + + 7. Configure the calibration options as needed: + - **Chains**: Number of parallel chains to run during the calibration. + - **Iterations**: Number of steps each chain should take. + - **ODE method**: Method to solve ordinary differential equations. + - **Calibrate method**: Approach to calibration (*bayesian*, *local*, or *global*). + 8. Click :material-play-outline:{ aria-hidden="true" } **Run**. + +## Intermediate results + +???+ list "To view calibration results" + + - Use the **Select variables to plot** dropdown to choose the variables you want to see, or click **Add Chart** to add another chart. \ No newline at end of file diff --git a/docs/simulation/index.md b/docs/simulation/index.md new file mode 100644 index 0000000..b0007f3 --- /dev/null +++ b/docs/simulation/index.md @@ -0,0 +1,7 @@ +--- +title: Simulation +--- + +# Simulation + +More info coming soon. \ No newline at end of file diff --git a/docs/simulation/simulate-ensemble.md b/docs/simulation/simulate-ensemble.md new file mode 100644 index 0000000..70d238c --- /dev/null +++ b/docs/simulation/simulate-ensemble.md @@ -0,0 +1,38 @@ +--- +title: Simulate ensemble +--- + +# Simulate ensemble + +More info coming soon. + +
![](../img/models/model-simulate-ensemble-operator.png)
How it works: [PyCIEMSS](https://github.com/ciemss/pyciemss/blob/main/pyciemss/interfaces.py#L35) :octicons-link-external-24:{ alt="External link" title="External link" }
+ +
+ +- :material-arrow-collapse-right:{ .lg .middle aria-hidden="true" } __Inputs__ + + --- + + Model configurations + +- :material-arrow-expand-right:{ .lg .middle aria-hidden="true" } __Outputs__ + + --- + + Simulation data + +
+ +???+ list "To run an ensemble simulation" + + 1. [Configure two or more models](../config-and-intervention/configure-model.md). + 2. Right-click anywhere on the workflow graph and select **Simulation** > **Simulate ensemble**. + 3. Connect the Configure model outputs to the Simulate ensemble inputs. + 4. Click **Edit**. + 5. Enter custom weights for the model configurations or click **Set weights to be equal** to treat them as equally important. + 6. Create a mapping between the state variables of the model configurations. + 7. Set the time span: + - **Start** and **End Step** + - **Number of samples** + 8. Click :material-play-outline:{ aria-hidden="true" } **Run**. \ No newline at end of file diff --git a/docs/simulation/simulate-model.md b/docs/simulation/simulate-model.md new file mode 100644 index 0000000..ab3ef86 --- /dev/null +++ b/docs/simulation/simulate-model.md @@ -0,0 +1,75 @@ +--- +title: "Simulate a model" +--- + +# Simulate a model + +Simulating a model lets you understand how the underlying system might behave under specific conditions. + +
+ ![Workflow graph of two simulations (one deterministic and one probabilistic) of a SIDARTHE model of COVID-19 spread in Italy](../img/models/simulate.png) +
A deterministic simulation (center) showing various variables in a COVID-19 model (left) and a probabilistic simulation (right) showing the distribution of a single variable with uncertainty.
+
+ +???+ tip + + Simulating early with a simple model (one with a population of 1,000) can help you spot issues and fix them before you start using a more complex model. + +## Simulate + +Simulations take a model configuration and an optional intervention as inputs and run an editable number of samples to account for uncertainty in results. + +
![](../img/models/simulate-pyciemss-operator.png)
How it works: [PyCIEMSS](https://github.com/ciemss/pyciemss/blob/main/pyciemss/interfaces.py#L323) :octicons-link-external-24:{ alt="External link" title="External link" }
+ +
+ +- :material-arrow-collapse-right:{ .lg .middle aria-hidden="true" } __Inputs__ + + --- + + - Model configuration + - Interventions (optional) + +- :material-arrow-expand-right:{ .lg .middle aria-hidden="true" } __Outputs__ + + --- + + Simulation data + +
+ +???+ list "To run a simulation" + + 1. Add the model to a workflow graph. + 2. Hover over the Model output and click :octicons-plus-24:{ title="Link" aria-label="Link" } > [**Configure**](../config-and-intervention/configure-model.md). + 3. Right-click anywhere on the workflow graph and select **Simulation** > **Simulate**. + 4. Connect the Configure model operator output to the Simulate input. + 5. Click **Edit** on the Simulate operator. + 6. Configure the simulation options as needed: + - **Start** and **End time**: Specify the simulation time range. + - **Number of samples**: Enter the number of stochastic samples to generate. + - **Method**: Choose whether to solve ordinary differential equations using [dopri5](https://en.wikipedia.org/wiki/Dormand-Prince_method) or [euler](https://en.wikipedia.org/wiki/Euler_method). + + ??? tip + + Using a low number of samples and the dopri5 method can speed up your runtime for debugging purposes. + + 7. Click :material-play-outline:{ aria-hidden="true" } **Run**. + 8. Review the output and click :fontawesome-solid-floppy-disk:{ aria-hidden="true" } **Save as new dataset** to add the results to your project resources. + +## View and save simulation results + +Simulation results are summarized on each Simulate operator and in the operator details. You can choose which variables to plot. + +??? list "To view simulation results" + + 1. In the output section of the Simulate operator details, use the dropdown list to choose the variables you want to visualize. If you choose multiple variables, Terarium overlaps them on the same plot. + 2. To add an additional chart, click :octicons-plus-24:{ aria-hidden="true"} **Add chart** and repeat the previous step. + + ??? tip + + It's recommended to use multiple charts if the variables you want to display have very different magnitudes. + +??? list "To save simulation results as a new dataset" + + - On the Simulate pane, click :material-content-save-outline:{ aria-hidden="true" } **Save as new dataset**. \ No newline at end of file diff --git a/docs/stylesheets/extra.css b/docs/stylesheets/extra.css index ed983b2..71ee42d 100644 --- a/docs/stylesheets/extra.css +++ b/docs/stylesheets/extra.css @@ -67,20 +67,28 @@ img.inline-img { } [data-md-color-scheme="terarium"] { + --md-accent-fg-color: #1b8073; --md-primary-fg-color: #fff; --md-primary-bg-color: #000; --md-primary-bg-color--light: #0000008a; --md-typeset-a-color: #1b8073;; --md-icon: #000; + --md-footer-bg-color: #f5f5f5; + --md-footer-bg-color--dark: #d6d6d6; + --md-footer-fg-color: #000000de; + --md-footer-fg-color--light: #000000de; + --md-footer-fg-color--lighter: #000000de; + --md-footer-typeset-color: ; } [data-md-color-scheme="slate"] { - --md-accent-fg-color: #AFBAFD; + --md-accent-fg-color: #1b8073; --md-primary-bg-color--light: #fff; + --md-primary-fg-color: #1b8073; } [data-md-color-scheme=slate][data-md-color-primary=indigo] { - --md-typeset-a-color: #AFBAFD; + --md-typeset-a-color: #1b8073; } .md-header { @@ -193,4 +201,15 @@ svg.feather { .gridcards p a { display: block; margin: 10px auto; +} + +div.tabbed-labels a { + text-decoration: none; +} + +img.node-icon { + display: inline; + margin-top: 0; + margin-bottom: 0; + font-size: 0.85em; } \ No newline at end of file diff --git a/docs/subsystems/modeling.md b/docs/subsystems/modeling.md index eee2592..391af71 100644 --- a/docs/subsystems/modeling.md +++ b/docs/subsystems/modeling.md @@ -72,21 +72,4 @@ More info coming soon. --- [MIRA | GitHub](https://github.com/gyorilab/mira/tree/main) :octicons-link-external-24:{ alt="External link" title="External link" } - - -## Couple models - -
- -- __Documentation__ - - --- - - [Decapodes.jl](https://algebraicjulia.github.io/Decapodes.jl/dev/overview/#Merging-Multiple-Physics) :octicons-link-external-24:{ alt="External link" title="External link" } - -- __Source code__ - - --- - - [Decapodes.jl | GitHub](https://github.com/AlgebraicJulia/Decapodes.jl) :octicons-link-external-24:{ alt="External link" title="External link" }
\ No newline at end of file diff --git a/docs/subsystems/simulation.md b/docs/subsystems/simulation.md index 238cd53..943cf15 100644 --- a/docs/subsystems/simulation.md +++ b/docs/subsystems/simulation.md @@ -6,41 +6,7 @@ title: "Simulation" More info coming soon. -## Simulate with SciML - -
- -- __Documentation__ - - --- - - [sciml-service](https://github.com/DARPA-ASKEM/sciml-service/blob/main/src/operations.jl#L222) :octicons-link-external-24:{ alt="External link" title="External link" } - -- __Source code__ - - --- - - [sciml-serivce | GitHub](https://github.com/DARPA-ASKEM/sciml-service/tree/main) :octicons-link-external-24:{ alt="External link" title="External link" } -
- -## Calibrate with SciML - -
- -- __Documentation__ - - --- - - [sciml-service](https://github.com/DARPA-ASKEM/sciml-service/blob/main/src/operations.jl#L245) :octicons-link-external-24:{ alt="External link" title="External link" } - -- __Source code__ - - --- - - [sciml-serivce | GitHub](https://github.com/DARPA-ASKEM/sciml-service/tree/main) :octicons-link-external-24:{ alt="External link" title="External link" } -
- -## Simulate with PyCIEMSS +## Simulate
@@ -57,7 +23,7 @@ More info coming soon. [PyCIEMSS | GitHub](https://github.com/ciemss/pyciemss/tree/main/pyciemss) :octicons-link-external-24:{ alt="External link" title="External link" }
-## Calibrate with PyCIEMSS +## Calibrate
@@ -74,7 +40,7 @@ More info coming soon. [PyCIEMSS | GitHub](https://github.com/ciemss/pyciemss/tree/main/pyciemss) :octicons-link-external-24:{ alt="External link" title="External link" }
-## Optimize with PyCIEMSS +## Optimize
diff --git a/docs/upload-resources/index.md b/docs/upload-resources/index.md new file mode 100644 index 0000000..1b7c4ef --- /dev/null +++ b/docs/upload-resources/index.md @@ -0,0 +1,82 @@ +--- +title: "Upload modeling resources" +--- + +# Upload modeling resources + +With Terarium, you can upload, store and manage resources needed for your modeling and simulation workflows. You can upload documents, models, and datasets from: + +- [Your computer](#upload-your-own-resources). +- [GitHub code repositories](#upload-resources-from-a-github-repository). + +You can also [copy resources from other projects](#get-resources-from-other-projects). + +Uploaded documents, models, and datasets appear in your project resources. You can transform and simulate them by dragging them into a workflow. + +!!! note + + Uploaded documents run through an [extraction process](#pdf-extraction) that, depending on the size of the PDF, make take some time. + +## Upload resources + +### Upload your own resources + +**Upload** in the Resources panel lets you add resources from your computer. + +![Upload dialog configured to upload and extract metadata from a scientific paper on COVID-19](../img/resources/upload.png) + +???+ list "To upload resources" + + 1. In the Resource panel, click **Upload**. + 2. Perform one of the following actions: + - Drag your files into the Upload resources dialog. + - Click **open a file browser** to navigate to the location of the files you want to add. + 3. Click **Upload**. + +### Upload resources from a GitHub repository + +You can import the following resources directly from a GitHub repository: + +- Data files (CSV, JSON, TSV, XML, YAML / YML). +- Document files (PDF, TXT, MD) +- Unknown files (if you specify which resource type to import it as) + +![Import dialog showing the directory structure of the nytimes/covid-19-data GitHub repository](../img/resources/github.png) + +???+ list "To import resources from a GitHub URL" + + 1. In the Resource panel, click **Upload**. + 2. Enter the URL of the repository in the format *https://github.com/nytimes/covid-19-data*. + 3. Click **Upload**. + 4. Select the data and documents you want to import. + 5. To import an unknown file type, select it and choose whether you want to import it as data or a document. + 6. Click **Import files** or **Import repo to project**. + +### PDF Extraction + +Most resources you upload are available for use right away. When you upload a PDF document however, Terarium begins extracting any linear ordinary differential equations it finds in the text. Depending on the size of the PDF, this process can take some time. + +!!! note + + The extractor isn't optimized to handle every way that equations can represent models. Before using any extracted equations to [create a model from equations](../modeling/create-model-from-equations.md), check and edit them if necessary. + +???+ list "To check the status of a PDF extraction" + + - Click :material-bell-outline:{ title="Notifications" aria-label="Notifications"}. + +## Get resources from other projects + +You can also get resources by copying them from other projects in Terarium. + +??? list "To get a model or dataset from another project" + + 1. Open the project that contains the model or dataset. + 2. Open the model or dataset by clicking its name in the Resources panel. + 3. Next to the model or dataset name, click :fontawesome-solid-ellipsis-vertical:{ title="Menu" aria-label="Menu" } > :octicons-plus-24:{ aria-hidden="true" } **Add to project** and select your project. + +??? list "To get a document from another project" + + 1. Open the project that contains the document. + 2. Open the document by clicking its name in the Resources panel. + 3. Click :fontawesome-solid-ellipsis:{ title="Menu" aria-label="Menu" } > :octicons-download-24:{ aria-hidden="true"} **Download this file** and save it to your computer. + 4. Open your project and [reupload the document](#upload-your-own-resources). \ No newline at end of file diff --git a/docs/workflows/index.md b/docs/workflows/index.md index 3014ad5..cc9b768 100644 --- a/docs/workflows/index.md +++ b/docs/workflows/index.md @@ -1,27 +1,26 @@ --- -title: "Workflows" +title: "Build a workflow graph" --- -# Manage workflows +# Build a workflow graph A workflow is a visual canvas that lets you to build and run complex operations (calibration, simulation, and stratification) on models and data. ![Workflow graph with an SIR model and related dataset passed into a calibration operation](../img/workflows/workflow.png) -??? list "To create a new workflow" +???+ list "To create a new workflow" - * In the Resources panel, click :fontawesome-solid-plus:{ aria-hidden="true"} **New** in the Workflow section. + * In the Resources panel, click :fontawesome-solid-plus:{ aria-hidden="true"} **New** in the Workflows section. ## Add resources and operators to a workflow -Workflows consist of models, datasets, documents, and code that you can feed into a series of operators that transform or simulate your resources. +Workflows consist of resources (models, datasets, and documents) that you can feed into a series of operators that transform or simulate them. -??? list "To add a resource to the workflow" +Each resource or operator is a "node" with a title, thumbnail preview, and a set of inputs and outputs. - Perform one of the following actions: +??? list "To add a resource to the workflow" - + Drag the model, dataset, document, or code in from the Resources panel. - + Right-click anywhere on the graph, select **Add resource** > **Model**, **Dataset**, **Document**, or **Code** and then select the resource from the dropdown list. + + Drag the model, dataset, or document in from the Resources panel. ??? list "To add an operator to the workflow" @@ -30,27 +29,31 @@ Workflows consist of models, datasets, documents, and code that you can feed int + Right-click anywhere on the graph and then select an operation from the menu. + Click :octicons-plus-24:{ aria-hidden="true"} **Add component** and then select the operation from the list. -### Link resources and operators +### Connect resources and operators -Resources and operators in your workflow graph have inputs and outputs that let you string them together to form complex model operations. +Resource and operator inputs and outputs let you string them together to form complex model operations. ![Outputs (left) for a model and a dataset connected to the input of a calibrate operation](../img/workflows/nodes.png) -??? list "To connect resources and operators" +??? list "To connect resources and operators already in the workflow" - - Click the output of one operator and then click the corresponding input on another operator. + - Click the output of one resource or operator and then click the corresponding input on another operator. ??? example - To configure a Calibrate operation to use a dataset, first click the output on the right side of the Dataset operator and then click the **Dataset** input on the left side of the Calibrate operator. +??? list "To connect resources and operators to a new operator" + + - Hover over the output of the resource or operator, then click :octicons-plus-24:{ title="Link" aria-label="Link" }, and then select an operator. + ??? list "To remove a connection between resources and operators" - Hover over the input or output and click :octicons-unlock-24:{ aria-hidden="true" } **Unlink**. ??? note "Operators with yellow headers" - An operator with a yellow header indicates that a resource or indicator that feeds into it has changed and the operator needs to be rerun. + An operator with a yellow header indicates that a resource or indicator that flows into it has changed and the operator needs to be rerun. ### Edit resource and operator details @@ -71,9 +74,9 @@ Resources and operators in the workflow graph summarize the data and inputs/outp To organize your workflow graph, you can move, rearrange, or remove any of the operators. -???+ note "Saving workflows" +???+ note "To save a workflow" - Terarium periodically autosaves the state of your workflow as you make changes. + Terarium automatically saves the state of your workflow as you make changes. ??? list "To rename a workflow" @@ -83,10 +86,6 @@ To organize your workflow graph, you can move, rearrange, or remove any of the o * Click the title of the operator and drag it to another location on the graph. -??? list "To adjust the layering of a workflow operator" - - * Click :fontawesome-solid-ellipsis-vertical:{ alt="Menu" title="Menu" } > :fontawesome-solid-arrow-up:{ aria-hidden="true" } **Bring to front** or :fontawesome-solid-arrow-down:{ aria-hidden="true" } **Send to back**. - ??? list "To remove a workflow operator" * Click :fontawesome-solid-ellipsis-vertical:{ alt="Menu" title="Menu" } > :fontawesome-regular-trash-can:{ aria-hidden="true" } **Remove**. \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index ec8c881..43268fc 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -1,4 +1,4 @@ -site_name: Terarium Help +site_name: Terarium help site_author: Uncharted Software Inc. copyright: Copyright © 2024 Uncharted Software Inc. @@ -22,13 +22,13 @@ theme: - media: "(prefers-color-scheme: light)" scheme: terarium toggle: - icon: material/brightness-7 + icon: material/weather-sunny name: Switch to dark mode # Palette toggle for dark mode - media: "(prefers-color-scheme: dark)" scheme: slate toggle: - icon: material/brightness-4 + icon: material/weather-night name: Switch to light mode favicon: img/terarium-icon.png font: false @@ -38,6 +38,11 @@ plugins: - search: {} - glightbox: {} - macros: {} + - redirects: + redirect_maps: + # 'old.md': 'new.md' + # 'old/file.md': 'new/file.md' + # 'some_file.md': 'http://external.url.com/foobar' - print-site: enumerate_headings: false add_cover_page: true @@ -79,27 +84,32 @@ nav: - index.md - Get started: - Using Terarium: get-started/using-terarium.md - - Gather modeling resources: modeling-resources/index.md - - Manage workflows: workflows/index.md - - Work with data: + - "Example project: Recreating, modifying, and simulating an SIR model": get-started/example-workflow.md + - Upload modeling resources: upload-resources/index.md + - Build a workflow graph: workflows/index.md + - Review and transform data: - datasets/index.md + - Review and enrich a dataset: datasets/review-and-enrich-dataset.md + - Transform dataset: datasets/transform-dataset.md - Subset dataset: datasets/subset-dataset.md - Transform gridded dataset: datasets/transform-gridded-dataset.md - - Work with a model: - - models/index.md - - Edit a model: models/edit-model.md - - Create a model from code: models/extract-code.md - - Enrich model descriptions: models/enrich-descriptions.md - - Stratify a model: models/stratify-model.md - - Configure a model: models/configure-model.md - - Simulate a model: models/simulate-model.md - - Calibrate a model: models/calibrate-model.md - - Optimize a model: models/optimize-model.md - - Work with multiple models: - - Compare models: multiple-models/compare-models.md - - Couple models: multiple-models/couple-models.md - - Simulate ensemble: multiple-models/simulate-ensemble.md - - Calibrate ensemble: multiple-models/calibrate-ensemble.md + - Modeling: + - modeling/index.md + - Create a model from equations: modeling/create-model-from-equations.md + - Enrich model metadata: modeling/enrich-descriptions.md + - Edit a model: modeling/edit-model.md + - Stratify a model: modeling/stratify-model.md + - Compare models: modeling/compare-models.md + - Configuration and intervention: + - config-and-intervention/index.md + - Configure a model: config-and-intervention/configure-model.md + - Create and optimize an intervention policy: config-and-intervention/create-intervention-policy.md + - Simulation: + - simulation/index.md + - Simulate a model: simulation/simulate-model.md + - Calibrate a model: simulation/calibrate-model.md + - Simulate ensemble: simulation/simulate-ensemble.md + - Calibrate ensemble: simulation/calibrate-ensemble.md - Save insights as notes: notes/index.md - Integrated subsystems: - subsystems/index.md diff --git a/requirements.txt b/requirements.txt index 70e0e04..b149b4b 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,6 +1,7 @@ # Requirements for mkdocs -mkdocs == 1.5.3 -mkdocs-material == 9.5.13 -mkdocs-glightbox == 0.3.7 +mkdocs == 1.6.1 +mkdocs-material == 9.5.40 +mkdocs-glightbox == 0.4.0 mkdocs-macros-plugin == 1.0.5 -mkdocs-print-site-plugin == 2.3.6 \ No newline at end of file +mkdocs-print-site-plugin == 2.5.0 +mkdocs-redirects == 1.2.1 \ No newline at end of file