Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix bugs in inactive top cells feature #5

Conversation

cbegeman
Copy link
Collaborator

@cbegeman cbegeman commented Jul 7, 2021

This PR fixes 2 bugs that affect the inactive top cells feature (i.e., when minLevelCell > 1). One in ocn_diagnostic_solve_ssh and the other affects the sea ice salinity flux in ocn_surface_bulk_forcing.
No changes in the solution should occur when there are no inactive top cells.

This PR also includes a cleanup of Redi. The original clunky workaround was designed to avoid non-bfb changes in BGC tracers as discussed here, and this revision introduces those non-bfb changes as @maltrud determined that they are insignificant.

One commit f59b505 increases the generalizability of vertical coordinates by relaxing the assumption that restingThickness spans ssh=0 to bottomDepth. As such, it is tangentially related to inactive top cells.

@cbegeman cbegeman force-pushed the ocn/bugfix-inactive-top-cells branch from 4d42267 to 64e846f Compare July 20, 2021 22:24
@cbegeman cbegeman force-pushed the ocn/bugfix-inactive-top-cells branch 2 times, most recently from de0f78a to 2236e64 Compare May 2, 2022 22:31
@xylar xylar changed the base branch from master to alternate May 3, 2022 01:45
@xylar xylar changed the base branch from alternate to master May 3, 2022 01:45
@cbegeman cbegeman force-pushed the ocn/bugfix-inactive-top-cells branch from 2236e64 to 7c0f132 Compare May 11, 2022 20:24
Steven Brus added 24 commits September 8, 2022 10:17
 - Comment out Langmuir enhancement (lamult) and mixed layer depth
   (HML) variables, since they are not in the new wavewatch source
   code.

 - Modify the field output specification. The outputs are now grouped
   into 9 different groups. The flag array to turn them on/off is
   now 2D

 - Add field output flag variables for coupling variables (not used)

 - Update call to w3init

 - Increase size of odat variable to account for coupling variables
 - The NOTYPE variable wasn't being assigned and defaulted to 0.
   The loop that goes over output types to call the write routines
   was being skipped over.

 - Setting the NOTYPE variable to 7 in wav_comp_mct.F90 fixes the
   issue.
 - Add w3_clean command

 - Get rid of redundant environment variable specification

 - Switch NETCDF_CONFIG variable to intel
 - This is the cleanest way to include this file in the build
   process
 - Use WW3's w3_source script to pre-process the source code
   for the ww3_shel program only, instead of using w3_make to
   pre-process and compile all programs

 - w3_source creates a .tar.gz file that must be untared before
   calling the e3sm makefile
 - Only include ww3_shel files since those are now the only ones
   generated by buildlib

 - ww3_grid files are also included, since that program was built
   using w3_make to create the mod_def.ww3 file
 - Add argument to call to w3init in wav_comp_mct.F90

 - Update paths to ww3 repo in buildlib. Also handle setting up the
   wwatch3.env file (if it doesn't already exist) by calling w3_setup

 - Point to correct mod_def.ww3 and restart.ww3 files in buildnml and
   namelist_definition_ww3.xml
 - This file will get copied to WW3 repo and when  w3_setup is run,
   it will cause this to be used as the switch file for pre-processing
   the .ftn source code
 - This makes sure that the WW3 git working directory is clean (no
   untracked files) after building
 - Add fields in seq_flds_mod.F90

 - Set w2x indicies in ww3_cpl_indicies.F90

 - Set x2o indicies in mpaso)cpl_indices.F90

 - Pack variables in wav_comp_mod.F90

 - No modifications to MPAS-O source code yet
mark-petersen and others added 24 commits October 11, 2022 13:43
Fixes:
1) get_case_env: needs to load Config
2) bless_test_results: Never bless a failed test
3) PEM: fix potential invalid rootpe
4) fix for multi driver restarts

Changes:
1) cprnc: add a data type check
2) Updates config_batch.xml to facilitate queue specific flags e.g.
   -w docker would only apply when the queue name is long.
3) Updates workflow to build and push containers for PRs
4) Shorten test-job-id with --jenkins-id option
5) improve the pfs test by copying software_environment.txt

[BFB]
Double PEs to prevent walltime timeouts on Ascent for
- ERS.hcru_hcru.I20TRGSWCNPRDCTCBC.elm-erosion
- ERS.ne30pg2_r05_EC30to60E2r2.GPMPAS-JRA.mosart-rof_ocn_2way
    (on Crusher too)
- ERS.r05_r05.IELM.elm-V2_ELM_MOSART_features

[BFB]
Update to github.io in mpas license website

Until 2021 github supported the alternate web address github.com. In
2021 it changed exclusively to github.io. This PR updates the link to
the MPAS license on all MPAS headers. It only changes code comments,
not working code.

[BFB]
…SM-Project#5201)

Remove unneeded mesh declaration

Mesh variables are declared in mpas_ocn_mesh.F rather than individual
routines.

Fixes E3SM-Project#5200
[BFB]
add elm input files to the land-atm 1850 and 20tr compsets and make sure the simulations use correct namelist files.

[NBFB]
…oject#5231)

Update CIME submodule

... to 977c1f6cd9283d03e356c8370a32302be4991cf4

Fixes:
1) get_case_env: needs to load Config
2) bless_test_results: Never bless a failed test
3) PEM: fix potential invalid rootpe
4) fix for multi driver restarts

Changes:
1) cprnc: add a data type check
2) Updates config_batch.xml to facilitate queue specific flags e.g.
   -w docker would only apply when the queue name is long.
3) Updates workflow to build and push containers for PRs
4) Shorten test-job-id with --jenkins-id option
5) improve the pfs test by copying software_environment.txt

[BFB]
…t#5233)

Adding HOMMEXX_ENABLE_GPU to Hommexx_config in order for scream code to pick it up.

Scream builds of homme do not use config file in theta-l_kokkos.

[bfb]
Add WAVEWATCH III as new coupled wave component

This is a stealth feature PR that adds the WAVEWATCH III submodule, cime
configuration, namelist generation, build infrastracture, and coupling
interface.

Wave only, G case + waves, and B case + waves compsets are included
along with model resolutions that include the wave mesh intended for v1
and v2 LR configurations.

The initial coupled, active wave feedback is a parameterization for
Langmuir mixing.

[NML] for configurations with active ocean
[BFB]
…ct#5187)

Fix out-of-bounds index for vertical mixing with one active layer

The current implicit vertical mixing routine fails when a column has one
active layer, because of references to k+1 and k−1. This PR adds a check
on each edge, and if there is only one active layer, it applies the
implicit drag term but not the vertical mixing terms. When only a single
layer exists, implicit vertical mixing is not needed (or even valid),
but implicit drag is still desirable.

There are two cases where the model may have a single layer:
1. when the whole domain is single layer; and
2. when individual columns have only one active vertical cell, like near
   coastlines with wetting-and-drying.

Fixes E3SM-Project#4849
[BFB]
…#5196)

Use reduced g-point data when using RRTMGP in EAM and E3SM-MMF. This should half the cost of RRTMGP in E3SM and bring performance in line with RRTMG, which was already using a similar number of quadrature points. This will be non-BFB only for the tests that use RRTMGP (eam-rrtmgp/eam-rrtmgpxx testmods, and all of the MMF tests), but BFB for all other tests.

[non-BFB] for RRTMGP tests
…ject#5232)

Add zeroing of velocity to ocean init mode

Explicitly sets normal velocity to zero at the beginning of init mode.
Each test case can then overwrite the zeros with initial velocities, but
most do not. In the past the unassigned arrays wrote out zeros, but this
is not true now, as described in E3SM-Project#5230.

This code is used to create MPAS-Ocean initial conditions. It is not
compiled in the E3SM code base for simulations.

Fixes E3SM-Project#5230
[BFB]
@cbegeman cbegeman force-pushed the ocn/bugfix-inactive-top-cells branch 2 times, most recently from 38431f9 to 719a98f Compare October 26, 2022 22:04
@cbegeman
Copy link
Collaborator Author

This PR has been migrated and split:
E3SM-Project#5246
E3SM-Project#5254

@cbegeman cbegeman closed this Nov 22, 2022
mark-petersen pushed a commit that referenced this pull request May 31, 2023
cee/15.0.0 with GPU MPI buffers can crash in a system lib like this:

#4  0x00007fffe159e35b in (anonymous namespace)::do_free_with_callback(void*, void (*)(void*)) [clone .constprop.0] () from /opt/cray/pe/cce/15.0.0/cce/x86_64/lib/libtcmalloc_minimal.so.1
#5  0x00007fffe15a8f16 in tc_free () from /opt/cray/pe/cce/15.0.0/cce/x86_64/lib/libtcmalloc_minimal.so.1
#6  0x00007fffe99c2bcd in _dlerror_run () from /lib64/libdl.so.2
#7  0x00007fffe99c2481 in dlopen@@GLIBC_2.2.5 () from /lib64/libdl.so.2
#8  0x00007fffea7bce42 in _ad_cray_lock_init () from /opt/cray/pe/lib64/libmpi_cray.so.12
#9  0x00007fffed7eb37a in call_init.part () from /lib64/ld-linux-x86-64.so.2
#10 0x00007fffed7eb496 in _dl_init () from /lib64/ld-linux-x86-64.so.2
#11 0x00007fffed7dc58a in _dl_start_user () from /lib64/ld-linux-x86-64.so.2
#12 0x0000000000000001 in ?? ()
#13 0x00007fffffff42e7 in ?? ()
#14 0x0000000000000000 in ?? ()

Work around this by using cee/14.0.3.
xylar pushed a commit that referenced this pull request Nov 8, 2023
Update development branch to E3SM master
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.