diff --git a/docs/developer/index.md b/docs/developer/index.md index 1f42881e3..86e9cd90f 100644 --- a/docs/developer/index.md +++ b/docs/developer/index.md @@ -8,20 +8,88 @@ and [Research paper](https://arxiv.org/abs/2305.07244). ## :computer: Development Environment -Ideally, developers should work on Ubuntu/Linux. Other operating systems -are not supported inherently and may require additional steps. +Ideally, developers should work on Ubuntu/Linux Operating System.There is +an ongoing effort +to bring support for Windows Operating System. But, the development and +software installation scripts are still heavily suited to Ubuntu/Linux -To start with, install the required software and git-hooks. +Please use the steps given here to install the required software packages. + +### Ubuntu/Linux ```bash bash script/env.sh bash script/configure-git-hooks.sh +bash script/docker.sh +``` + +:warning: The docker images are large and are likely to consume +about 5GB of bandwidth and 15GB of space. +You will have to download the docker images on a really good network. + +### Windows + +Two powershell installation scripts, namely `base.ps1` and `env.ps1` +are available to install the required +software packages. But errors might crop up due to missing +environment variables. The potential errors are: + +1. `npm is not recognized.........` in `base.ps1`. +1. `gem is not recognized.........` in `env.ps1` + +If you encounter these errors, +remember to include _node_ and _ruby_ installation locations in +**PATH** environment variable +(`Settings --> search for "system environment variables"` +`--> Advanced --> Environment Variables --> PATH`). + +The `base.ps1` and `env.ps1` scripts can be run again after setting +the correct **PATH** environment variable. + +#### Pre-install Nodejs and Ruby Software + +Another way to solve the **PATH** environment problem is to +install Nodejs and Ruby software packages before running the powershell +scripts. + +1. Install the latest stable version of NodeJS from the + [official NodeJS website](https://nodejs.org/en). +1. Install Ruby from + [official Ruby website](https://github.com/oneclick/rubyinstaller2/releases/download/RubyInstaller-3.1.2-1/rubyinstaller-devkit-3.1.2-1-x64.exe) + and follow all the defaults during the installation. + +#### Run Scripts + +Then, open powershell with **administrative** priviledges and run the +following commands in the given order: + +```bash +powershell -F script/base.ps1 +powershell -F script/env.ps1 +powershell -F script/configure-git-hooks.ps1 +powershell -F script/docker.ps1 ``` +:warning: The docker images are large and are likely to consume +about 5GB of bandwidth and 15GB of space. +You will have to download the docker images on a really good network. + +### git hooks + The git-hooks will ensure that your commits are formatted correctly and that the tests pass before you push the commits to remote repositories. +The project uses pre-commit for managing git hooks. Install git hooks using + +```bash +pre-commit install +``` + +The git hooks run during pre-commit (`git commit`) and +pre-push (`git push`) stages. No special flags and options are +needed to run these git hooks. + You can also run the git-hooks manually before committing or pushing by using the run commands below. The autoupdate command will set the revisions of the git repos used in the .pre-commit-config.yaml up to date. @@ -32,23 +100,12 @@ pre-commit run --hook-stage pre-push # runs test pre-commit autoupdate # update hooks to latests versions ``` -Be aware that the tests may take a long time to run. +Be aware that the some tests may take a long time to run. If you want to skip the tests or formatting, you can use the `--no-verify` flag on `git commit` or `git push`. Please use this option with care. -There is a script to download all the docker containers -used in the project. You can download them using - -```bash -bash script/docker.sh -``` - -:warning: The docker images are large and are likely to consume -about 5GB of bandwidth and 15GB of space. -You will have to download the docker images on a really good network. - ## :building_construction: Development Workflow To manage collaboration by multiple developers on the software, diff --git a/docs/redirect-page.html b/docs/redirect-page.html index 742a397bb..5309ef422 100644 --- a/docs/redirect-page.html +++ b/docs/redirect-page.html @@ -37,7 +37,7 @@ th, td { - padding: 20px; + padding: 10px; } diff --git a/docs/user/examples/examples.drawio b/docs/user/examples/examples.drawio index 7f35138b9..e70ba3475 100755 --- a/docs/user/examples/examples.drawio +++ b/docs/user/examples/examples.drawio @@ -1,6 +1,6 @@ - + - + @@ -1888,4 +1888,181 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/user/examples/index.md b/docs/user/examples/index.md index 5ca34a817..23e47b69a 100644 --- a/docs/user/examples/index.md +++ b/docs/user/examples/index.md @@ -35,5 +35,6 @@ to use the examples in the following order. 1. [Three Water Tanks with DT Manager Framework](./three-tank/README.md) 1. [Flex Cell Digital Twin with Two Industrial Robots](./flex-cell/README.md) 1. [Incubator](./incubator/README.md) +1. [Firefighters in Emergency Environments](./o5g/README.md) :material-download: [DTaaS examples](https://github.com/INTO-CPS-Association/DTaaS-examples) diff --git a/docs/user/examples/o5g/README.md b/docs/user/examples/o5g/README.md new file mode 100644 index 000000000..ac4cb0054 --- /dev/null +++ b/docs/user/examples/o5g/README.md @@ -0,0 +1,215 @@ +# Firefighter Mission in a Burning Building + +In an firefighter mission it is important to monitor the oxygen levels of +each firefighters Self Contained Breating Aparatus (SCBA) in context of +their mission. + +## Physical Twin Overview + +![Burning building](img/o5g_physical_twin_overview.png) + +> Image: Schematic overview of a firefighter mission. Note the mission +> commander on the lower left documenting the air supply pressure levels +> provided by radio communication from the firefighters inside and around +> the burning building. This image was created with the assistance of +> DALL·E. + +We assume the following scenario: + +* a set of firefighters work to extinguish a burning building +* they each use an SCBA with pressurised oxygen to breath +* a mission commander on the outside coordinates the efforts and + surveills the oxygen levels + +## Digital Twin Overview + +In this example a monitor is implemented, that calculates how much time +the firefighers have left, until they need to leave the building. +To that end, the inputs used are: + +* 3D-model of the building in which the mission takes place, +* pressure data of a firefighters SCBA and +* firefighters location inside of the building + +are used to estimate: + +* the shortest way out, +* how much time this will need and +* how much time is left until all oxygen from the SCBA is used up. + +The remaining mission time is monitored and the firefighter receive +a warning if it drops under a certain threshold. + +![Digital Twin Structure](img/o5g_digital_twin_architecture.png) + +This example is an implementation of the the paper +[_Digital Twin for Rescue Missions--a Case Study_](https://ceur-ws.org/Vol-3507/paper4.pdf) +by Leucker et al. + +### Quick Check + +Before runnnig this example please make sure the following files +are at the correct locations: + +```txt +/workspace/examples/ + data/o5g/input/ + runTessla.sh + sensorSimulation.py + telegraf.conf + + models/ + lab.ifc + makefmu.mos + PathOxygenEstimate.mo + + tools/ + graphToPath.py + ifc_to_graph + pathToTime.py + tessla-telegraf-connector/ + tessla-telegraf-connector/ + tessla.jar + specification.tessla (run-time specification) + + digital_twins/o5g/ + main.py + config + lifecycle/ (scripts) +``` + +### Digital Twin Configuration + +All configuration for this example is contained in `digital_twins/o5g/config`. + +To use the MQTT-Server, account information needs to be provided. +The topics are set to their default values, which allow the DT to +access the mock physical twins sensor metrics and to send back alerts. + +```ini +export O5G_MQTT_SERVER= +export O5G_MQTT_PORT= +export O5G_MQTT_USER= +export O5G_MQTT_PASS= + +export O5G_MQTT_TOPIC_SENSOR='vgiot/ue/metric' +export O5G_MQTT_TOPIC_AIR_PREDICTION='vgiot/dt/prediction' +export O5g_MQTT_TOPIC_ALERT='vgiot/dt/alerts' +``` + +This example uses InfuxDB as a data storage, which will need to be +configured to use your Access data. The following configuration +steps are needed: + +* Log into the InfluxDB Web UI +* Obtain **org** name (is below your _username_ in the sidebar) +* Create a data bucket if you don't have one already in + `Load Data -> Buckets` +* Create an API access token in `Load Data -> API Tokens`, + Copy and save this token somewhere immediately, you can not access it + later! + +```ini +export O5G_INFLUX_SERVER= +export O5G_INFLUX_PORT= +export O5G_INFLUX_TOKEN= +export O5G_INFLUX_ORG= +export O5G_INFLUX_BUCKET= +``` + +## Lifecycle Phases + +The lifecycles that are covered include: + +| Lifecycle Phase | Completed Tasks | +| --------- | ------- | +| Install | Installs Open Modelica, Rust, Telegraf and the required pip dependencies | +| Create | Create FMU from Open Modelica file | +| Execute | Execute the example in the background tmux terminal session | +| Terminate | Terminate the tmux terminal session running in the background | +| Clean | Delete the temporary files | + +## Run the example + +### Install + +Run the install script by executing + +```bash +lifecycle/install +``` + +This will install all the required dependencies from apt and pip, as well as +Open Modelica, Rust, Telegraf and the required pip dependencies +from their respective repos. + +#### Create + +Run the create script by executing + +```bash +lifecycle/create +``` + +This will compile the modelica model to an Functional Mockup Unit (FMU) +for the correct platform. + +#### Exceute + +To run the Digital Twin execute + +```bash +lifecycle/execute +``` + +This will start all the required components in a single tmux session +called `o5g` in the background. +To view the running Digital Twin attatch to this tmux session by executing + +```bash +tmux a -t o5g +``` + +To detatch press `Ctrl-b` followed by `d`. + +![Tmux session of live digital twin](img/o5g_running_screenshot.png) + +The _tmux_ session contains 4 components of the digital twin: + +| Panel location | Purpose | +|:---|:---| +| Top Left | Sensor simulator generating random location and O2-level data | +| Top Right | Main Digital Twin receives the sensor data and calculates an estimate of how many minutes of air remain | +| Bottom Left | Telegraf to convert between different message formats, also displays all messages between components | +| Bottom Right | TeSSLa monitor raises an alarm, if the remaining time is to low. | + +#### Examine the Results + +For additional mission awareness, we recommend utilising the Influx data +visualisation. We provide a dashboard configuration in the file +_influx-dashoard.json_. Log in to your Influx Server to import +(usually port 8086). A screenshot of the dashboard is given here. + +![Firefighter remaining mission time](img/influx-dashboard.png) + +The data gets stored in +`o5g->prediction->air-remaining->37ae3e4fb3ea->true->vgiot/dt/prediction` +variable of the InfluxDB. In addition to importing dashboard configuration +given above, it is possible to create your custom dashboards using +the stored data. + +#### Terminate + +To stop the all components and close the _tmux_ session execute + +```bash +lifecycle/terminate +``` + +#### Clean + +To remove temoporary files created during execution + +```bash +lifecycle/clean +``` diff --git a/docs/user/examples/o5g/img/influx-dashboard.png b/docs/user/examples/o5g/img/influx-dashboard.png new file mode 100755 index 000000000..0dcc34868 Binary files /dev/null and b/docs/user/examples/o5g/img/influx-dashboard.png differ diff --git a/docs/user/examples/o5g/img/o5g_digital_twin_architecture.png b/docs/user/examples/o5g/img/o5g_digital_twin_architecture.png new file mode 100644 index 000000000..184ddb72c Binary files /dev/null and b/docs/user/examples/o5g/img/o5g_digital_twin_architecture.png differ diff --git a/docs/user/examples/o5g/img/o5g_physical_twin_overview.png b/docs/user/examples/o5g/img/o5g_physical_twin_overview.png new file mode 100644 index 000000000..cd32454e4 Binary files /dev/null and b/docs/user/examples/o5g/img/o5g_physical_twin_overview.png differ diff --git a/docs/user/examples/o5g/img/o5g_running_screenshot.png b/docs/user/examples/o5g/img/o5g_running_screenshot.png new file mode 100644 index 000000000..7d214d004 Binary files /dev/null and b/docs/user/examples/o5g/img/o5g_running_screenshot.png differ diff --git a/mkdocs-github.yml b/mkdocs-github.yml index 24196ae4c..39d861793 100644 --- a/mkdocs-github.yml +++ b/mkdocs-github.yml @@ -61,6 +61,7 @@ nav: - Three Water Tanks: user/examples/three-tank/README.md - Flex Cell: user/examples/flex-cell/README.md - Incubator: user/examples/incubator/README.md + - Firefighters: user/examples/o5g/README.md - Codebase: https://github.com/INTO-CPS-Association/DTaaS-examples - FAQ: FAQ.md - Developer: diff --git a/mkdocs.yml b/mkdocs.yml index 91d811440..b0d62a489 100755 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -61,6 +61,7 @@ nav: - Three Water Tanks: user/examples/three-tank/README.md - Flex Cell: user/examples/flex-cell/README.md - Incubator: user/examples/incubator/README.md + - Firefighters: user/examples/o5g/README.md - Codebase: https://github.com/INTO-CPS-Association/DTaaS-examples - FAQ: FAQ.md - Developer: diff --git a/script/base.ps1 b/script/base.ps1 new file mode 100644 index 000000000..47e218752 --- /dev/null +++ b/script/base.ps1 @@ -0,0 +1,84 @@ +# Check if Chocolatey is installed, if not, install it +if (-not (Get-Command choco -ErrorAction SilentlyContinue)) { + # Check the current execution policy + $currentExecutionPolicy = Get-ExecutionPolicy + + # Set execution policy to Bypass temporarily + Set-ExecutionPolicy Bypass -Scope Process -Force + + # Download and install Chocolatey + [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072 + Invoke-Expression ((New-Object System.Net.WebClient).DownloadString('https://community.chocolatey.org/install.ps1')) + + # Restore the original execution policy + Set-ExecutionPolicy $currentExecutionPolicy -Scope Process -Force +} + +# Install Git if not already installed +if (-not (Get-Command git -ErrorAction SilentlyContinue)) { + choco install git -y +} + +# Update all installed packages +choco upgrade all -y + +# Check if docker-desktop is already installed +if (-not (Get-Command docker-desktop -ErrorAction SilentlyContinue)) { + # Install docker for containers and microservices + # https://docs.docker.com/engine/install/ubuntu/ + choco install -y docker-desktop + + # Check if docker group exists, if not, create it + $dockerGroup = 'docker' + if (-not (Get-LocalGroup -Name $dockerGroup -ErrorAction SilentlyContinue)) { + New-LocalGroup -Name $dockerGroup + } + + # Add user to docker group if not already a member + $user = $env:UserName + if (-not (Get-LocalGroupMember -Group $dockerGroup -Member $user)) { + Add-LocalGroupMember -Group $dockerGroup -Member $user + } + + # Make docker available to the user account + Write-Host "`n`n`nMake docker available to your user account....`n......." + Write-Host "Your user account is: $user" + Write-Host "If your user account is a member of the docker group, let the installation script continue." + Write-Host "Otherwise, exit this script and run the following command:`n" + Write-Host "Add-LocalGroupMember -Group docker -Member $user`n" + Write-Host "Log out and log in again. You can run this script again after login.`n" + + # Wait for user action + Write-Host "Press Ctl+C if you need to complete this task...." + Write-Host "Waiting for 60 seconds...." + Start-Sleep -Seconds 60 + + # Start Docker service + Start-Service -Name "com.docker.service" + + # Test Docker installation + docker run hello-world + + # Enable Docker service + Set-Service -Name "com.docker.service" -StartupType Automatic + + # Install docker-compose from https://docs.docker.com/compose/install/other/ + $url = "https://github.com/docker/compose/releases/download/v2.23.3/docker-compose-windows-x86_64.exe" + $outputPath = Join-Path $env:ProgramFiles "Docker\docker-compose.exe" + Invoke-WebRequest -Uri $url -OutFile $outputPath + # Ensure docker-compose is in the system path + [Environment]::SetEnvironmentVariable("Path", $env:Path + ";$outputPath", [EnvironmentVariableTarget]::Machine) +} + +# Check if Node.js is already installed +if (-not (Get-Command node -ErrorAction SilentlyContinue)) { + # Install Node.js environment if not already installed + choco install -y nodejs-lts +} + +# Install Yarn +choco install -y yarn + +# Install global npm packages +npm install -g serve +npm install -g pm2 \ No newline at end of file diff --git a/script/configure-git-hooks.ps1 b/script/configure-git-hooks.ps1 new file mode 100644 index 000000000..2516897f0 --- /dev/null +++ b/script/configure-git-hooks.ps1 @@ -0,0 +1,9 @@ +# Check if pip is available +if (-not (Get-Command pip -ErrorAction SilentlyContinue)) { + Write-Host "Error: pip is not installed. Please install pip before running this script." + exit 1 +} + +# Install pre-commit using pip +pip install pre-commit +pre-commit install \ No newline at end of file diff --git a/script/docker.ps1 b/script/docker.ps1 new file mode 100644 index 000000000..53f069b2e --- /dev/null +++ b/script/docker.ps1 @@ -0,0 +1,10 @@ +# get the required docker images +docker pull traefik:v2.10 +docker pull mltooling/ml-workspace-minimal:0.13.2 +docker pull grafana/grafana:10.1.4 +docker pull influxdb:2.7 +docker pull telegraf:1.28.2 +docker pull rabbitmq:3-management +docker pull eclipse-mosquitto:2 +docker pull mongo:7.0.3 +docker pull gitlab/gitlab-ce:16.4.1-ce.0 \ No newline at end of file diff --git a/script/env.ps1 b/script/env.ps1 new file mode 100644 index 000000000..f9b368438 --- /dev/null +++ b/script/env.ps1 @@ -0,0 +1,52 @@ +# Install base dependencies from the script/base.sh file + +# Install openssl for certificate generation +choco install -y openssl + +# Install playwright tool for integration tests on browsers +# Ensure npm is added to PATH +if (-not ([Environment]::GetEnvironmentVariable("Path", [EnvironmentVariableTarget]::Machine) -like "*npm*")) { + Write-Host "Adding npm to PATH..." + [Environment]::SetEnvironmentVariable("Path", "$env:Path;$(npm prefix -g)\node_modules\.bin", [EnvironmentVariableTarget]::Machine) +} + +npm install -g playwright + +# Installing required python packages +choco install -y python3 + +# Ensure Python scripts directory is added to PATH +if (-not ([Environment]::GetEnvironmentVariable("Path", [EnvironmentVariableTarget]::Machine) -like "*Python3*")) { + Write-Host "Adding Python scripts to PATH..." + [Environment]::SetEnvironmentVariable("Path", "$env:Path;$(Get-Command python3 | Select-Object -ExpandProperty Directory)", [EnvironmentVariableTarget]::Machine) +} + +# Install mkdocs +pip install mkdocs + +# Install Ruby: https://github.com/oneclick/rubyinstaller2/releases/download/RubyInstaller-3.1.2-1/rubyinstaller-devkit-3.1.2-1-x64.exe +choco install -y ruby + +# Ensure Ruby scripts directory is added to PATH +if (-not ([Environment]::GetEnvironmentVariable("Path", [EnvironmentVariableTarget]::Machine) -like "*Ruby*")) { + Write-Host "Adding Ruby scripts to PATH..." + [Environment]::SetEnvironmentVariable("Path", "$env:Path;$(Get-Command ruby | Select-Object -ExpandProperty Directory)", [EnvironmentVariableTarget]::Machine) +} + +# Install markdownlint +gem install mdl + +# Install mkdocs plugins +pip install mkdocs-material python-markdown-math mkdocs-open-in-new-tab mkdocs-with-pdf qrcode + +# Ensure Python scripts directory is added to PATH (for pip) +if (-not ([Environment]::GetEnvironmentVariable("Path", [EnvironmentVariableTarget]::Machine) -like "*Python3*")) { + Write-Host "Adding Python scripts to PATH (for pip)..." + [Environment]::SetEnvironmentVariable("Path", "$env:Path;$(Get-Command python3 | Select-Object -ExpandProperty Directory)", [EnvironmentVariableTarget]::Machine) +} + +# Install shellcheck +choco install -y shellcheck + +# Install madge for generating dependency graphs of typescript projects +npm install -g madge