Skip to content

Microdata comparison UI release 22 of April 2024
Compare
Choose a tag to compare
@amc1999 amc1999 released this 22 Apr 09:14
· 88 commits to master since this release
v1.17.2
79ce285

Microdata comparison UI release 22 of April 2024

20240422 : v1.17.2

This release provides UI for microdata run comparison (a.k.a. winners-losers) and multiple enhancements for openM++ language (extensions of Modgen language).

Main features of this release:

  • c++ core: new optional lifecycle attributes to probe events
  • c++ core: new "untransformed" tables
  • c++ core: new "snapshot" tables
  • c++ core: allow explicit declaration of table filter attribute
  • UI: microdata aggregation and run comparison
  • go: new fair scheduling of cloud CPU resources

Other changes:

  • c++ core: new example model TableProbe2
  • c++ core: protect model run-time code from Microsoft MPI bug
  • go: initial version of dbget utility to extract and transform model run results
  • go dbcopy: add output to TSV files
  • change oms UI back-end and MSBuild to correctly handle ini-files in UI
  • Linux and MacOS: fix issues with spaces in file path

New optional lifecycle attributes to probe events

A new option lifecycle_attributes creates 2 attributes and a supporting classification which can be used to probe model events. For more information, please see the wiki topic (containing worked examples with screenshots) at:
Lifecycle Attributes · openmpp/openmpp.github.io Wiki

New "untransformed" tables

In this release, if the " table" keyword is followed by the optional keyword "untransformed", the measures for that table are computed using no weights and no population scaling, and the run-level table is computed by assembling measures over subs (using the new measures_method=assemble), e.g. by summing counts rather than by averaging them over subs.

New "snapshot" tables

In this release, the "table" keyword can be followed by the optional table property "snapshot". Snapshot tables change the default increment from "delta" to "value_out", making it easier to specify a table with a zero-duration filter constructed using a trigger derived attribute.
For more information, see Entity Tables · openmpp/openmpp.github.io Wiki
Please note that wiki topic and related wiki topics are being actively edited.

Allow explicit declaration of table filter attribute

An entity table filter is an expression involving entity attributes.
OpenM++ creates identity attributes as needed to implement the table filters in a model.
They are shown in table and attribute topics in the Developer Edition of the model’s Symbol Reference.

A new syntax in this release allows a model developer to declare the table filter explicitly in the table declaration, e.g.:

table snapshot untransformed Person X01_Events //EN All Events
[ bool filter_X01 = trigger_changes(lifecycle_counter) ]
{
    lifecycle_event //EN Event
  * {
        unit //EN N
    } //EN Statistic
};

This can be a convenient way to explore a table filter directly, e.g. using microdata output.
It is equivalent to declaring the original filter expression as an identity attribute, then using that attribute instead of the expression as the table filter.
Table filters declared explicitly do not participate in de-duplication of table filters (sharing of identical filters) done by OpenM++.

New example model TableProbe2

This release includes a new example model TableProbe2, which is used in the wiki to illustrate entity table concepts and use.

TableProbe2 is not Modgen compatible.

UI: microdata aggregation and run comparison

Microdata UI provides new capabilities to aggregate and compare model runs microdata. For example, if there is a hypothetical tax model then user can compare different taxation policies to find an average percent of income difference by age group, province and gender.

Please visit our wiki for more information: UI: Aggregate and Compare Microdata

New fair scheduling of cloud CPU resources

Oms web-service back-end updated to fairly share cloud CPU resources between multiple users under heavy load. With previous version batch users (those who were submitting 100x model runs through R or Python scripts) were able to monopolize entire cluster CPU resources for multiple days. New version provides a fair share of CPUs to all users even under heavy load.

Initial version of dbget utility to extract and transform model run results

New dbget utility allow to extract and transform model run results from the database and save it into CSV files. For example it allows to compare run results microdata to find which age and gender groups are winning or losing from taxation policy changes:

dbget -db modelOne.sqlite -do microdata-compare
  -m TaxModel
  -dbget.Run Default
  -dbget.WithRuns "Updated Tax Policy"
  -dbget.Entity Person
  -dbget.GroupBy "AgeGroup, Gender"
  -dbget.Calc "OM_COUNT_IF( Income[base] - Income[variant] > param.Threshold[base] )"

This release include an initial version of dbget utility, functionality will be significantly extended in subsequent releases

dbcopy utility: add output to TSV files

Now it is possible to extract database content into TSV files.
For example to get .json model metadata files and TSV parameters, output tables, microdata files:

dbcopy -m modelOne -dbcopy.IntoTsv

To get get model metadata, parameters, output tables, microdata as TSV files:

dbcopy -m modelOne -dbcopy.To csv -dbcopy.IntoTsv

Download source code and binaries:

Download release documentation archive:

Download cluster version (MPI):

IMPORTANT:
Full version of OpenM++ source code always included into "Download" links above.
Please do NOT use "Source code (zip)" or "Source code (tar.gz)" archives from "Assets" links below.
It is auto-generated by GitHub release tools and contains only half of OpenM++ source code.

Assets 14
Loading