Skip to content

Commit

Permalink
Merge pull request #127 from EcoExtreML/main_to_pc
Browse files Browse the repository at this point in the history 
Merge `main` into `python-compliant`
  • Loading branch information
@SarahAlidoost
SarahAlidoost authored Nov 15, 2022
2 parents e8cc76d + b7506a3 commit 3bdcd03
Show file tree
Hide file tree
Showing 5 changed files with 343 additions and 137 deletions.
175 changes: 38 additions & 137 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,142 +2,43 @@

Integrated code of SCOPE and STEMMUS.

## Run the model

You need the python package
[PyStemmusScope](https://pystemmusscope.readthedocs.io/en/latest/index.html).
See the relevant instructions for `Users` or `Developers` on how to run the
model.

### Users

As a user, you don't need to have a MATLAB license to run the STEMMUS-SCOPE
model. The workflow is executed using python and MATLAB Runtime on a Unix-like
system. Follow this
[instruction](https://pystemmusscope.readthedocs.io/en/latest/readme_link.html#users).

### Developers

If you want to contribute to the development of STEMMUS_SCOPE,
SCOPE is a radiative transfer and energy balance model, and STEMMUS model is a two-phase mass and heat transfer model. For more information about the coupling between these two models, please check [this reference](https://gmd.copernicus.org/articles/14/1379/2021/). Before running the model, you need to prepare input data and a configuration file. This can be done using the python package
[PyStemmusScope](https://pystemmusscope.readthedocs.io).

<img width="500" alt="Logo" src=./docs/assets/imgs/coupling_scheme.png>
(by Zeng & Su, 2021)

## Running STEMMUS_SCOPE

```mermaid
flowchart LR
subgraph Platform
direction RL
b[Snellius]
c[CRIB]
d[Your own machine]
end
A(Data)
Platform --> A
B(Config file)
A --> B
C{{Run model}}
B --> C
click b "https://github.com/EcoExtreML/STEMMUS_SCOPE/tree/main/docs/STEMMUS_SCOPE_on_Snellius.md" "Run STEMMUS_SCOPE on Snellius" _blank
click c "https://github.com/EcoExtreML/STEMMUS_SCOPE/tree/main/docs/STEMMUS_SCOPE_on_CRIB.md" "Run STEMMUS_SCOPE on CRIB" _blank
click d "https://github.com/EcoExtreML/STEMMUS_SCOPE/tree/main/docs/STEMMUS_SCOPE_on_local_device.md" "Run STEMMUS_SCOPE on your own machine" _blank
```
About how to run `STEMMUS_SCOPE` on Snellius, check [this document](./docs/STEMMUS_SCOPE_on_Snellius.md).

If you want to run `STEMMUS_SCOPE` on CRIB, check [this document](./docs/STEMMUS_SCOPE_on_CRIB.md).

If you want to run `STEMMUS_SCOPE` on your own machine, check [this document](./docs/STEMMUS_SCOPE_on_local_device.md).

## Contributing

If you want to contribute to the development of `STEMMUS_SCOPE`,
have a look at the [contribution guidelines](CONTRIBUTING.md).

### Dataflow of STEMMUS_SCOPE on CRIB:

[CRIB](https://crib.utwente.nl/) is the ITC Geospatial Computing Platform.

1. Data required by the model are in a folder named "input" under project
directory on CRIB. This folder includes:

- Plumber2_data: the forcing/driving data provided by PLUMBER2.
- SoilProperty: the soil texture data and soil hydraulic parameters.

Below the directory explanations are from [SCOPE
documentation](https://scope-model.readthedocs.io/en/latest/directories.html):

- directional: the observer’s zenith and azimuth angles.(only used for
multi-angle simulations (if the option ‘directional’ is switched on in
parameters).
- fluspect_parameters: absorption spectra of different leaf components are
provided, according to PROSPECT 3.1, as well as Fluspect input: standard
spectra for PSI and PSII.
- leafangles: example leaf inclination distribution data are provided.
- radiationdata: RTMo.m calculates spectra based on MODTRAN5 outputs (T-18
system).Note that in the input data (files as well as the spreadsheet),
the broadband input radiation may be provided. SCOPE linearly scales the
input spectra of the optical and the thermal domain in such a way that
the spectrally integrated input shortwave and long-wave radiation matches
with the measured values.
- soil_spectra: the soil spectrum is provided. Note that it is also possible
to simulate a soil reflectance spectrum with the BSM model. In that case,
the values for the BSM model parameters are taken from the input data, and
the archived spectra in this folder are not used.
- input_data.xlsx: the input to SCOPE model is provided. It
provides parameter inputs for PROSPECT, leaf_biochemical, fluorescence,
soil, canopy, aerodynamic, angles, photosynthetic temperature dependence
functional parameters, etc.

2. Config file: it is a text file that sets the paths **required** by the
model. For example, see `config_file_crib.txt` in this repository. This file
includes:

- SoilPropertyPath: a path to soil texture data and soil hydraulic
parameters.
- InputPath: this is the working/running directory of the model and should
include the data of `directional`, `fluspect_parameters`, `leafangles`,
`radiationdata`, `soil_spectra`, and `input_data.xlsx`.
- OutputPath: this is the base path to store outputs of the model. When the
model runs, it creates `sitename_timestamped` directories under this
path.
- ForcingPath: a path to the forcing/driving data.
- ForcingFileName: name of the forcing file in a netcdf format. Currently,
the model runs at the site scale. For example, if we put the
`FI-Hyy_1996-2014_FLUXNET2015_Met.nc` here, the model runs at the `FI-Hyy` site.
- NumberOfTimeSteps: total number of time steps which model runs for. It can be
`NA` or a number. Examples: `NumberOfTimeSteps=17520` runs the model for one year a
half-hour time step i.e. `365*24*2=17520`. `NumberOfTimeSteps=NA` runs the
model for all time steps in which the forcing data are available.

### Workflow of STEMMUS_SCOPE on CRIB:

1. The model reads the `ForcingFile` e.g. `FI-Hyy_1996-2014_FLUXNET2015_Met.nc`
from "ForcingPath" and extracts forcing variables in `.dat` format using
`filesread.m`. The `.dat` files are stored in the "InputPath" directory. In
addition, the model reads the site information i.e. the location and
vegetation parameters.
2. The model reads the soil parameters from "SoilPropertyPath" using
`soilpropertyread.m`.
3. Some constants are loaded using `Constant.m`.
4. The model runs step by step until the whole simulation period is completed
i.e till the last time step of the forcing data.
5. The results are saved as binary files temporarily. Then, the binary files are
converted to `.csv` files and stored in a `sitename_timestamped` output
directory under "OutputPath". A netcdf file is also created in the same
directory, see [csv_to_nc/README](./utils/csv_to_nc/README.md).

### Dataflow of STEMMUS_SCOPE on Snellius:

[Snellius](https://servicedesk.surfsara.nl/wiki/display/WIKI/Snellius) is the
Dutch National supercomputer hosted at SURF.

1. Data required by the model are in a folder named "data" in the project
directory `einf2480` on Snellius. This directory includes:

- forcing/plumber2_data: the forcing/driving data provided by PLUMBER2.
- model_parameters/soil_property: the soil texture data and soil hydraulic parameters.
- model_parameters/vegetation_property:
- directional
- fluspect_parameters
- leafangles
- radiationdata
- soil_spectra
- input_data.xlsx

For the explanation of the directories see
[Dataflow of STEMMUS_SCOPE on CRIB](#dataflow-of-stemmus_scope-on-crib).

2. Config file: it is a text file that sets the paths **required** by the model.
For example, see [config_file_snellius.txt](config_file_snellius.txt) in
this repository. This file includes:

- SoilPropertyPath: a path to soil texture data and soil hydraulic
parameters.
- InputPath: this is the working/running directory of the model and should
include the data of `directional`, `fluspect_parameters`, `leafangles`,
`radiationdata`, `soil_spectra`, and `input_data.xlsx`.
- OutputPath: this is the base path to store outputs of the model. When
the model runs, it creates `sitename_timestamped` directories under
this path.
- ForcingPath: a path to the forcing/driving data.
- ForcingFileName: name of the forcing file in a netcdf format. Currently,
the model runs at the site scale. For example, if we put the
`FI-Hyy_1996-2014_FLUXNET2015_Met.nc` here, the model runs at the `FI-Hyy` site.
- VegetationPropertyPath: path to required data except `Plumber2_data` and `SoilProperty`.
- NumberOfTimeSteps: total number of time steps in which model runs. It can be
`NA` or a number. Examples: `NumberOfTimeSteps=17520` runs the model for one year a
half-hour time step i.e. `365*24*2=17520`. `NumberOfTimeSteps=NA` runs the
model for all time steps in which the forcing data are available.

### Workflow of STEMMUS_SCOPE on Snellius:

This is the same as the workflow of STEMMUS_SCOPE on crib, see section
[Workflow of STEMMUS_SCOPE on CRIB](#workflow-of-stemmus_scope-on-crib).
## How to cite us
[![RSD](https://img.shields.io/badge/rsd-ecoextreml-00a3e3.svg)](https://research-software-directory.org/projects/ecoextreml)
<!-- [![DOI](https://zenodo.org/badge/DOI/<replace-with-created-DOI>.svg)](https://doi.org/<replace-with-created-DOI>) -->
123 changes: 123 additions & 0 deletions docs/STEMMUS_SCOPE_on_CRIB.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,123 @@
## STEMMUS_SCOPE on CRIB

[CRIB](https://crib.utwente.nl/) is the ITC Geospatial Computing Platform.

### Dataflow of STEMMUS_SCOPE on CRIB:

1. Data required by the model are in a folder named "input" under project
directory on CRIB. This folder includes:

- Plumber2_data: the forcing/driving data provided by PLUMBER2.
- SoilProperty: the soil texture data and soil hydraulic parameters.

Below the directory explanations are from [SCOPE
documentation](https://scope-model.readthedocs.io/en/latest/directories.html):

- directional: the observer’s zenith and azimuth angles.(only used for
multi-angle simulations (if the option ‘directional’ is switched on in
parameters).
- fluspect_parameters: absorption spectra of different leaf components are
provided, according to PROSPECT 3.1, as well as Fluspect input: standard
spectra for PSI and PSII.
- leafangles: example leaf inclination distribution data are provided.
- radiationdata: RTMo.m calculates spectra based on MODTRAN5 outputs (T-18
system).Note that in the input data (files as well as the spreadsheet),
the broadband input radiation may be provided. SCOPE linearly scales the
input spectra of the optical and the thermal domain in such a way that
the spectrally integrated input shortwave and long-wave radiation matches
with the measured values.
- soil_spectra: the soil spectrum is provided. Note that it is also possible
to simulate a soil reflectance spectrum with the BSM model. In that case,
the values for the BSM model parameters are taken from the input data, and
the archived spectra in this folder are not used.
- input_data.xlsx: the input to SCOPE model is provided. It
provides parameter inputs for PROSPECT, leaf_biochemical, fluorescence,
soil, canopy, aerodynamic, angles, photosynthetic temperature dependence
functional parameters, etc.

2. Config file: it is a text file that sets the paths **required** by the
model. For example, see [config_file_crib.txt](../config_file_crib.txt) in this repository. This file
includes:

- SoilPropertyPath: a path to soil texture data and soil hydraulic
parameters.
- InputPath: this is the working/running directory of the model and should
include the data of `directional`, `fluspect_parameters`, `leafangles`,
`radiationdata`, `soil_spectra`, and `input_data.xlsx`.
- OutputPath: this is the base path to store outputs of the model. When the
model runs, it creates `sitename_timestamped` directories under this
path.
- ForcingPath: a path to the forcing/driving data.
- ForcingFileName: name of the forcing file in a netcdf format. Currently,
the model runs at the site scale. For example, if we put the
`FI-Hyy_1996-2014_FLUXNET2015_Met.nc` here, the model runs at the `FI-Hyy`
site.
- DurationSize: total number of time steps in which model runs. It can be
`NA` or a number. Example `DurationSize=17520` runs the model for one year a
half-hour time step i.e. `365*24*2=17520`.

To edit the config file, open the file with a text editor and change the
paths. The variable names e.g. `SoilPropertyPath` should not be changed.
Also, note a `/` is required at the end of each line.

As explained above, the "InputPath" directory of the model is considered as
the working/running directory and should include some data required by the
model. As seen in the config file, the "InputPath" is now set as same as the
"input" folder. This means that the "input" folder is treated as the
model's working/running directory. However, it is possible to create a
different working/running directory and set the "InputPath" to it. Then,
you should copy the required data i.e. `directional`, `fluspect_parameters`,
`leafangles`, `radiationdata`, `soil_spectra`, and `input_data.xlsx` to the
working/running directory.

### Workflow of STEMMUS_SCOPE on CRIB:

1. The model reads the `ForcingFile` e.g. `FI-Hyy_1996-2014_FLUXNET2015_Met.nc`
from "ForcingPath" and extracts forcing variables in `.dat` format using
`filesread.m`. The `.dat` files are stored in the "InputPath" directory. In
addition, the model reads the site information i.e. the location and
vegetation parameters.
2. The model reads the soil parameters from "SoilPropertyPath" using
`soilpropertyread.m`.
3. Some constants are loaded using `Constant.m`.
4. The model runs step by step until the whole simulation period is completed
i.e till the last time step of the forcing data.
5. The results are saved as binary files temporarily. Then, the binary files are
converted to `.csv` files and stored in a `sitename_timestamped` output
directory under "OutputPath".

### How to run STEMMUS_SCOPE on CRIB:

1. Log in CRIB with your username and password and select a proper compute unit.
2. Download the source code from this repository or get it using `git clone` in
a terminal:

` git clone https://github.com/EcoExtreML/STEMMUS_SCOPE.git `

All the codes can be found in the folder `src` whereas the executable file in
the folder `exe`.

3. Check `config_file_crib.txt` and change the paths if needed, specifically
"InputPath" and "OutputPath".
4. Follow the instructions below:

#### Run using MATLAB that requires a license

If you want to use MATLAB desktop,

1. click on the `Remote Desktop` in the
Launcher. Click on the `Applications`. You will find the 'MATLAB' software under
the `Research`.
2. After clicking on 'MATLAB', it asks for your account information that is
connected to a MATLAB license.
3. Open the file `filesread.m` and set the
variable `CFG` to the path of the config file e.g. `CFG =
'/data/shared/EcoExtreML/STEMMUS_SCOPEv1.0.0/STEMMUS_SCOPE/config_file_crib.txt';`.
4. Then, run the main script `STEMMUS_SCOPE.m`.

As an alternative, you can run the
main script using MATLAB command line in a terminal:

```bash
matlab -nodisplay -nosplash -nodesktop -r "run('STEMMUS_SCOPE.m');exit;"
```
Loading

0 comments on commit 3bdcd03

Please sign in to comment.