Add tests

[CasparJungbacker]

@fjansson here's a basic prototype for performing simple tests using Pytest as we discussed. I've set up the test such that any of the cases in cases/ can be tested, as long as the case doesn't need a moduser.f90.

To perform tests locally:

1. (optional) make a new virtual environment:

python -m venv .venv
source .venv/bin/activate

2. install requirements (this will install Pytest and f90nml for patching namoptions):

python -m pip install -r requirements.txt

3. setup an environment variable to point to the dales executable:

export DALES=$(pwd)/build/src/dales4.4

4. run tests for a case in the cases directory:

pytest -rf --assert=plain --case bomex

If a test fails, the output of dales will be printed.

[fjansson]

Nice!

How about putting the instructions from here in a README.md in the tests/ directory?

For only two cores on CI, mpirun has a flag --oversubscribe to allow launching more MPI processes than there are cores. That could work.

[CasparJungbacker]

How about putting the instructions from here in a README.md in the tests/ directory?

Yes, will do that!

You also had some interesting domain configurations in mind, right?

[fjansson]

Some test suggestions

Domains

do the following cases with both:
kmax = 128 ! something divisible by nprocx, nprocy
kmax = 127 ! a prime

! itot not divisible by nprocy, jtot not divisible by nprocx
nprocx = 1
nprocy = 3
itot = 32
jtot = 48

! ncely square and divisible
nprocx = 2
nprocy = 2
itot = 32
jtot = 32

! odd size
nprocx = 1
nprocy = 1
itot = 31
jtot = 31

could repeat this test for different FFT choices (solver_id = ...)
FFTW (solver_id = 100), the default FFT, cuFFT
The default FFT probably fails on odd domain sizes.

Advection schemes

Try case 2 above for all advection schemes. Same scheme for all variables.
(Exception: kappa and hybrid(?) only work for central quantities, for those schemes do momentum advection with any other scheme).

Run once
Then do a warm start

[CasparJungbacker]

Domain and advection tests are in and they're still passing :-) Testing cuFFT is going to be difficult, since we don't have access to GPU-accelerated CI runners.

I was also thinking that rico may be a more interesting test case than bomex, because it also include some scalar fields. What do you think @fjansson?

[fjansson]

rico is good, but the one in cases/rico is annoying because it has the case-specific moduser which needs to be compiled in. In cases/benchmarks is a rico which works without moduser. We could do a botany case, since that uses radiation also.

[CasparJungbacker]

Do we want to test both RRTMG and RRTMGP for botany? If so, how can I change namoptions to use the new scheme?

[fjansson]

That would be nice. iradiation = 5 for RRTMGP, 4 for RRTMG.

The only other thing needed that I can think of is that RRTMGP needs other data files. They can probably be symlinks in the test case folder, pointing to the data files which are together with the source code.

I did this:

# make symlinks to data files in the DALES source code
# run this in the case directory
SRC=/path/to/dales/src
ln -s $SRC/RTE-RRTMGP/extensions/cloud_optics/rrtmgp-cloud-optics-coeffs-lw.nc ./
ln -s $SRC/RTE-RRTMGP/extensions/cloud_optics/rrtmgp-cloud-optics-coeffs-reordered-sw.nc ./
ln -s $SRC/RTE-RRTMGP/rrtmgp/data/rrtmgp-data-lw-g128-210809.nc ./
ln -s $SRC/RTE-RRTMGP/rrtmgp/data/rrtmgp-data-sw-g112-210809.nc ./
ln -s $SRC/RRTMG/RRTMG_LW/data/rrtmg_lw.nc ./
ln -s $SRC/RRTMG/RRTMG_SW/data/rrtmg_sw.nc ./

At some point I also had to do ulimit -s unlimited on my Ubuntu laptop before launching DALES with RRTMGP, presumably because it puts some large array on the stack. Not sure if this is still needed, after we changed to running RRTMG for one horizontal row at a time in stead of the whole field (which took too much memory).