Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update regression tests to run! #323

Open
wants to merge 11 commits into
base: development
Choose a base branch
from

Conversation

peverwhee
Copy link
Collaborator

@peverwhee peverwhee commented Nov 14, 2024

Tag name (required for release branches): sima0_00_001
Originator(s): peverwhee

Description (include the issue title, and the keyword ['closes', 'fixes', 'resolves'] followed by the issue number):
closes #229 (Need ability for ncdata_check routines to end model run)
addresses #230 (Need CIME smoke tests for FPHYStest compset and various physics suites)

Updates the regression tests to actually run and compare against previous baselines.

Describe any changes made to build system:

  • added new compset FCAM7 that exercises the cam7 physics suites
  • updated config_pes.xml for ne3 grid

Describe any changes made to the namelist:

  • updated namelist defaults for ne3 grid (though these are currently unused; using analytic_ic until the SE dycore is updated)

List any changes to the defaults for the input datasets (e.g. boundary datasets): none

List all files MOVED and why:
cime_config/testdefs/testmods_dirs/cam/outfrq_kessler_mpas_derecho_nooutput/user_nl_cam
cime_config/testdefs/testmods_dirs/cam/outfrq_kessler_mpas_derecho_nooutput/shell_commands
cime_config/testdefs/testmods_dirs/cam/outfrq_held_suarez_derecho_nooutput/shell_commands
cime_config/testdefs/testmods_dirs/cam/outfrq_held_suarez_derecho_nooutput/user_nl_cam
cime_config/testdefs/testmods_dirs/cam/outfrq_kessler_derecho_nooutput/shell_commands
cime_config/testdefs/testmods_dirs/cam/outfrq_kessler_derecho_nooutput/user_nl_cam
cime_config/testdefs/testmods_dirs/cam/outfrq_tj2016_derecho_nooutput/user_nl_cam
cime_config/testdefs/testmods_dirs/cam/outfrq_tj2016_derecho_nooutput/shell_commands

  • removed "nooutput" from directory name

cime_config/testdefs/testmods_dirs/cam/outfrq_kessler_se_cslam_derecho_nooutput/user_nl_cam
cime_config/testdefs/testmods_dirs/cam/outfrq_kessler_se_cslam_derecho_nooutput/shell_commands

  • moved to outfrq_se_sclam/*

List all files added and what they do:
A cime_config/config_archive.xml

  • brought over from CAM so tests will run

List all existing files that have been modified, and describe the changes:
(Helpful git command: git diff --name-status development...<your_branch_name>)
M cime_config/config_component.xml

  • added FCAM7 configuration

M cime_config/config_compset.xml

  • added FCAM7 compset

M cime_config/config_pes.xml

  • modified for ne3 grid

M cime_config/namelist_definition_cam.xml

  • added ne3 grid defaults

M cime_config/testdefs/testlist_cam.xml

  • added FHS94 and FTJ16 SE CSLAM tests
  • modified existing FKESSLER SE CSLAM test to use ne5

M src/dynamics/tests/namelist_definition_analy_ic.xml

  • use held-suarez initial conditions by default (not "none")

M test/system/archive_baseline.sh

  • update to add a symlink to the ${baseline_dir}/latest_${CAM_FC} directory for easier test_driver usage
  • add flag to turn off symlink addition

M test/system/test_driver.sh

  • use ${baseline_dir}/latest_${CAM_FC} by default
  • add flag to turn off default usage (no baseline comparison)

If there are new failures (compared to the test/existing-test-failures.txt file),
have them OK'd by the gatekeeper, note them here, and add them to the file.
If there are baseline differences, include the test and the reason for the
diff. What is the nature of the change? Roundoff?

derecho/intel/aux_sima:
SMS_Ln2.mpasa480_mpasa480.FKESSLER.derecho_intel.cam-outfrq_kessler_mpas_derecho

  • will fail until MPAS is fully integrated

derecho/gnu/aux_sima:
SMS_Ln2.mpasa480_mpasa480.FKESSLER.derecho_gnu.cam-outfrq_kessler_mpas_derecho

  • will fail until MPAS is fully integrated

If this changes climate describe any run(s) done to evaluate the new
climate in enough detail that it(they) could be reproduced: n/a

CAM-SIMA date used for the baseline comparison tests if different than latest: first baselines!

Copy link
Collaborator

@nusbaume nusbaume left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for getting the tests going! I do have some change requests (which is mostly me backpedaling from previous things I asked you to do, sorry!), but hopefully it won't be too hard to implement. Of course please let me know if that is not the case. Thanks again!

cime_config/config_component.xml Outdated Show resolved Hide resolved
cime_config/testdefs/testlist_cam.xml Outdated Show resolved Hide resolved
cime_config/testdefs/testlist_cam.xml Outdated Show resolved Hide resolved
cime_config/testdefs/testlist_cam.xml Outdated Show resolved Hide resolved
cime_config/testdefs/testlist_cam.xml Outdated Show resolved Hide resolved
src/dynamics/tests/namelist_definition_analy_ic.xml Outdated Show resolved Hide resolved
test/system/archive_baseline.sh Outdated Show resolved Hide resolved
test/system/archive_baseline.sh Outdated Show resolved Hide resolved
test/system/archive_baseline.sh Outdated Show resolved Hide resolved
test/system/archive_baseline.sh Outdated Show resolved Hide resolved
@peverwhee
Copy link
Collaborator Author

@nusbaume thanks for your review. I also added the NL option to endrun on ncdata_check failures (and added it to the kessler test, which is currently the only one that has very small diffs) - if you could take a look at that, that'd be great!

Copy link
Collaborator

@nusbaume nusbaume left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Almost there! Just two last requests.

src/data/write_init_files.py Outdated Show resolved Hide resolved
cime_config/testdefs/testlist_cam.xml Outdated Show resolved Hide resolved
Copy link
Collaborator

@nusbaume nusbaume left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great to me now. Thanks again!

@nusbaume
Copy link
Collaborator

Also forgot to mention that you can close issue #229 once this PR is merged. It might also be worth "addressing" issue #230 as well (we can close that one once we have all of snapshot checks working). Thanks!

Copy link
Collaborator

@mwaxmonsky mwaxmonsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great @peverwhee, thanks for getting this added!

One minor whitespace fix I think might be needed but nothing worth holding up this PR.

@@ -132,6 +134,9 @@ while [ "${1:0:1}" == "-" ]; do
--no-baseline ) no_baseline=false
;;

--no-bl-compare ) baseline_default=false
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
--no-bl-compare ) baseline_default=false
--no-bl-compare ) baseline_default=false

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

3 participants