Skip to content

Linux use Docker

Jump to bottom
amc1999 edited this page Oct 11, 2022 · 6 revisions

Why Docker?

There are multiple cases when you want to use Docker containers for openM++ development:

  • build your models with latest version of openM++
  • build cluster-ready (and cloud-ready) version of your model without installing MPI on your host computer
  • do test run of your model in cluster environment without installing and configuring MPI cluster on multiple machines
  • build latest version of openM++ from source code without installing and configuring all necessary development tools

All above build and run tasks can be done without Docker and our wiki describes all steps necessary to achieve this. However in that case you will spend a few hours or even days with installing and configuring development and cluster environment. Use of Docker allow to skip unnecessary steps and focus on model development. Also because containers are isolated from host environment there is nothing (except of Docker itself) get installed on your host system and you keep it clean, no software versions conflicts.

To install Docker:

  • on Ubuntu do: sudo apt-get install docker
  • on Debian or MX Linux: su -c "apt-get install docker"
  • for RedHat 8 please follow RedHat 8: How to use Docker instructions.

Where to find openM++ Docker images

You can download openM++ images from Docker Hub:

If you want to use Ubuntu LTS (Ubuntu 20.04):

Documentation below contains Debian examples and it is also applicable to Ubuntu. Only difference is ubuntu Docker image name and sudo to run the docker command, for example:

sudo docker run openmpp/openmpp-run:ubuntu models/bin/MyModel

User name, user ID, group ID, home directory

Both containers openmpp/openmpp-run:debian and openmpp/openmpp-build:debian created with for user and group ompp, UID=1999, GID=1999, HOME=/home/ompp. To avoid permissions issues you may need to do one of:

  • create user ompp, UID=1999, group ompp, UID=1999 and login as that user
  • or use OMPP_* environment variables as in examples below to map your current login to container

For example, let assume you logged into your system as user:group = Me:MyGroup UID:GID = 1234:1234 and want to run model in your home directory: $HOME/models/bin/MyModel.

Simple attempt to run the model:

docker run openmpp/openmpp-run:debian models/bin/MyModel

will fail with error similar to: "models/bin/MyModel: No such file or directory" because container don't have an access to the files on your host system.

Let's bind your directory $HOME/models/bin/MyModel to the container default /home/ompp

docker run \
  -v $HOME/models/bin:/home/ompp \
  openmpp/openmpp-run:debian \
  ./MyModel

That will fail with error "Permission denied" because container default login user:group = ompp:ompp UID:GID = 1999:1999 don't have an access to your files user:group = Me:MyGroup UID:GID = 1234:1234.

You can create such login on your host system user:group = ompp:ompp UID:GID = 1999:1999 and use it to run the models

Or you can tell container to use your current user:group = Me:MyGroup UID:GID = 1234:1234 instead of default values:

docker run \
  -v $HOME/models/bin:/home/models \
  -e OMPP_USER=models -e OMPP_GROUP=models -e OMPP_UID=$UID -e OMPP_GID=`id -g` \
  openmpp/openmpp-run:debian \
  ./MyModel

How to use openmpp/openmpp-run:debian to run your models

To run openM++ model do:

docker run .... openmpp/openmpp-run:debian ./MyModel

For example, if your models are in $HOME/models/bin directory then:

docker run \
  -v $HOME/models/bin:/home/models \
  -e OMPP_USER=models -e OMPP_GROUP=models -e OMPP_UID=$UID -e OMPP_GID=`id -g` \
  openmpp/openmpp-run:debian \
  ./MyModel

docker run \
  -v $HOME/models/bin:/home/models \
  -e OMPP_USER=models -e OMPP_GROUP=models -e OMPP_UID=$UID -e OMPP_GID=`id -g` \
  openmpp/openmpp-run:debian \
  mpiexec -n 2 MyModel_mpi -OpenM.SubValues 16

also you can use -e OM_ROOT=/home/ompp to set environment variable for your model, if necessary.

To start shell inside of conatiner do:

docker run \
  -v $HOME:/home/${USER} \
  -e OMPP_USER=${USER} -e OMPP_GROUP=`id -gn` -e OMPP_UID=$UID -e OMPP_GID=`id -g` \
  -it openmpp/openmpp-run:debian
  bash

Following environment variables are used to map container user to your login:

OMPP_USER=ompp   # default: ompp, container user name and HOME
OMPP_GROUP=ompp  # default: ompp, container group name
OMPP_UID=1999    # default: 1999, container user ID
OMPP_GID=1999    # default: 1999, container group ID

How to use openmpp/openmpp-build:debian to build openM++ and models

To build latest version of openM++ from source code and rebuild your models do:

docker run ....options.... openmpp/openmpp-build:debian ./build-all

For example, if your build in $HOME/build directory then:

docker run \
  -v $HOME/build:/home/build \
  -e OMPP_USER=build -e OMPP_GROUP=build -e OMPP_UID=$UID -e OMPP_GID=`id -g` \
  openmpp/openmpp-build:debian \
  ./build-all

docker run \
  -v $HOME/build_mpi:/home/build_mpi \
  -e OMPP_USER=build_mpi -e OMPP_GROUP=build_mpi -e OMPP_UID=$UID -e OMPP_GID=`id -g` \
  -e OM_MSG_USE=MPI \
  openmpp/openmpp-build:debian \
  ./build-all

docker run ....user, group, home.... -e MODEL_DIRS=RiskPaths,IDMM      openmpp/openmpp-build:debian ./build-all
docker run ....user, group, home.... -e OM_BUILD_CONFIGS=RELEASE,DEBUG openmpp/openmpp-build:debian ./build-all
docker run ....user, group, home.... -e OM_MSG_USE=MPI                 openmpp/openmpp-build:debian ./build-all

Following environment variables used to control openM++ build:

OM_BUILD_CONFIGS=RELEASE,DEBUG # default: RELEASE,DEBUG for libraries and RELEASE for models
OM_MSG_USE=MPI                 # default: EMPTY
MODEL_DIRS=modelOne,NewCaseBased,NewTimeBased,NewCaseBased_bilingual,NewTimeBased_bilingual,IDMM,OzProj,OzProjGen,RiskPaths

Following environment variables are used to map container user to your login:

OMPP_USER=ompp   # default: ompp, container user name and HOME
OMPP_GROUP=ompp  # default: ompp, container group name
OMPP_UID=1999    # default: 1999, container user ID
OMPP_GID=1999    # default: 1999, container group ID

To build only openM++ libraries and omc compiler do:

docker run ....options.... openmpp/openmpp-build:debian ./build-openm

Environment variables to control build-openm: OM_BUILD_CONFIGS, OM_MSG_USE

To build only models do:

docker run ....options.... openmpp/openmpp-build:debian ./build-models

Environment variables to control build-models: OM_BUILD_CONFIGS, OM_MSG_USE, MODEL_DIRS

For example, if want to build your own model MyModel copy model code into $HOME/build/models/MyModel directory and do:

docker run ....user, group, home.... -e MODEL_DIRS=MyModel openmpp/openmpp-build:debian ./build-models
docker run ....user, group, home.... -e MODEL_DIRS=MyModel -e OM_BUILD_CONFIGS=RELEASE,DEBUG openmpp/openmpp-build:debian ./build-models

To build openM++ tools do any of:

docker run .... openmpp/openmpp-build:debian ./build-go   # Go oms web-service and dbcopy utility
docker run .... openmpp/openmpp-build:debian ./build-r    # openMpp R package
docker run .... openmpp/openmpp-build:debian ./build-ui   # openM++ UI

To create openmpp_redhat_YYYYMMDD.tar.gz deployment archive:

docker run .... openmpp/openmpp-build:debian ./build-tar-gz

Environment variables to control build-tar-gz: OM_MSG_USE, MODEL_DIRS

To customize build you can change any of build scripts inside of $HOME/build directory:

$HOME/build/build-all     # rebuild entire openM++ and create openmpp_redhat_YYYYMMDD.tar.gz archive
$HOME/build/build-openm   # rebuild entire openM++ runtime libraries and compiler
$HOME/build/build-models  # rebuild openM++ models specified by MODEL_DIRS
$HOME/build/build-go      # rebuild Go oms web-service and dbcopy utility
$HOME/build/build-r       # rebuild openMpp R package
$HOME/build/build-ui      # rebuild openM++ UI
$HOME/build/build-tar-gz  # create openmpp_redhat_YYYYMMDD.tar.gz archive

To start shell inside of container do:

docker run \
  -v $HOME:/home/${USER} \
  -e OMPP_USER=${USER} -e OMPP_GROUP=`id -gn` -e OMPP_UID=$UID -e OMPP_GID=`id -g` \
  -it openmpp/openmpp-build:debian \
  bash

How to use openmpp/openmpp-build:debian to update openM++ documentation

To build latest version of openM++ documentation do:

docker run ....options.... openmpp/openmpp-build:debian ./make-doc

For example, if your want to make a documenation in $HOME/build_doc directory then:

docker run \
  -v $HOME/build_doc:/home/build_doc \
  -e OMPP_USER=build_doc -e OMPP_GROUP=build_doc -e OMPP_UID=$UID -e OMPP_GID=`id -g` \
  openmpp/openmpp-build:debian \
  ./make-doc

Home

Getting Started

Model development in OpenM++

Using OpenM++

Model Development Topics

OpenM++ web-service: API and cloud setup

Using OpenM++ from Python and R

Docker

OpenM++ Development

OpenM++ Design, Roadmap and Status

OpenM++ web-service API

GET Model Metadata

GET Model Extras

GET Model Run results metadata

GET Model Workset metadata: set of input parameters

Read Parameters, Output Tables or Microdata values

GET Parameters, Output Tables or Microdata values

GET Parameters, Output Tables or Microdata as CSV

GET Modeling Task metadata and task run history

Update Model Profile: set of key-value options

Update Model Workset: set of input parameters

Update Model Runs

Update Modeling Tasks

Run Models: run models and monitor progress

Download model, model run results or input parameters

Upload model runs or worksets (input scenarios)

Download and upload user files

User: manage user settings

Model run jobs and service state

Administrative: manage web-service state

Clone this wiki locally