Visualizing and processing output

Enabling output

To save cross sections or 3D fields the following settings are needed in the namelist file for Dales. Output is saved in netCDF format by default.

&NAMCROSSSECTION
lcross      = .true.
crossheight = 20,40,80
dtav        = 60
/

crossheights is a list of z-levels for which to save horizontal cross sections.

&NAMFIELDDUMP
lfielddump  = .true.
dtav        = 60
/

Space savings and enabling compression

The upcoming Dales 4.2 enables compressed netCDF files, by switching to the netCDF4 format. The space saving is great, especially for the ql and qr which contains many 0s. Compressed netCDFs have the following quirks:

• if the Dales is interrupted, the files are not readable.
• the files cannot be read while Dales is running.

To work around these:

• add periodic calls to NF90_SYNC(ncid) on the netCDF files
• export HDF5_USE_FILE_LOCKING=FALSE before trying to view incomplete files

To save space, consider turning off output of all fields which are not needed. Fielddump saves many fields by default. For cloud visualization, only ql and possibly qr are needed.

Dales feature wishlist

• add synchronization calls (added in 4.3, enabled with lsync = .true.)
• make it possible to select which fields are saved

Viewing the output

• ncview - oldfashioned but quick
• panoply
• paraview - 3D views possible

Merging output

When Dales is run in parallel with MPI, each worker creates its own set of netCDF files. The following methods can be used to merge the pieces into single large files.

dalesview/merge_grids.py

Convenient, possibly slow for large, compressed 3D data. Works with Python 2 or 3. For troubleshooting run without parallelizing (-j 1) because the threading hides error messages.

Dalesview on GitHub

Dalesview on Cartesius

Module set, tested May 2020.

module load pre2019
module load nco/intel/4.6.0
module load netcdf4-python/1.2.9-intel-2016b-Python-2.7.12

CDO

Fast but doesn't work with variables on different grids at once, e.g. velocity variables and temperature. A solution is to merge a single variable at a time.

Needs CDO version >= 1.8 to avoid problem in collgrid - cdo collgrid (Abort): Variable name 4 not found!

CDO on Cartesius

module load 2019
module load CDO/1.9.5-intel-2018b

NX=`ls surf_xy.x*y000.001.nc | wc -l` 
cdo -f nc4 -z zip_6 -r -O collgrid,$NX `ls fielddump.*.001.nc | sort -t . -k 3` 3d.nc

# can specify a single variable to merge:
cdo -f nc4 -z zip_6 -r -O collgrid,$NX,thlxy `ls crossxy.0001.*.nc | sort -t y -k 3` merged-crossxy-thl.nc
cdo -f nc4 -z zip_6 -r -O collgrid,$NX,twp `ls cape.*.nc | sort -t y -k 2` merged-cape-twp.nc

The first line determines the number of tiles in X. The sort commands sort the input files in the right order, so that consecutive tiles are adjacent in X.

One can add the flag -P to use n threads, which may be faster but sometimes leads to crashes in the NetCDF or HDF libraries.

Merging scripts bundled with Dales

dales/utils/

Change unlimited dimension of the netCDF, concatenate, change dimension. These scripts still merge only in one direction, they are from the time before 2D MPI parallelization. Probably they can be adapted.

Paraview

Don't merge - load all tiles in paraview. Note that paraview may show a gap between the tiles. This gap goes away if you switch to point rendering.