Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Instantaneous CAM-SIMA history infrastructure #274

Merged
merged 97 commits into from
Oct 25, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
97 commits
Select commit Hold shift + click to select a range
8c066f0
Work on adding history
Jul 30, 2022
44f1660
Correct CIME paths for new CIME
Aug 2, 2022
ff48e4f
Start stripping out old stuff
Aug 2, 2022
56006e0
history test script and history unit test passing
Aug 7, 2022
ab8eb89
Added history external
Aug 7, 2022
478905b
Checkin to work on other branch
Feb 10, 2023
70dd483
Partial implementation of filename spec in new initialization scheme.
Mar 22, 2023
7f3bb0b
Improvements to reading configuration and associated tests
Mar 28, 2023
bbd2c4b
Resurrect Fortran history tests. Add ability to use real endrun for d…
Mar 31, 2023
ebb15c3
Merge remote-tracking branch 'ESCOMP/development' into history_court
Jun 26, 2023
e4fa834
Updating Externals_CAM.cfg to point to tj2016 atmospheric physics fork.
mwaxmonsky Mar 1, 2024
81224d5
Resolving merge conflicts with Externals_CAM.cfg
mwaxmonsky Mar 15, 2024
19e1586
Merging in updates from ccpp-framework and atmospheric_physcis. Upda…
mwaxmonsky Mar 15, 2024
680d5a8
Updates based on bug fixes and feedback.
mwaxmonsky Apr 3, 2024
fc35548
Updating to latest atmos_phys changes.
mwaxmonsky Apr 15, 2024
0eb4e8f
Merge branch 'development' into tj2016
mwaxmonsky Apr 25, 2024
b6e7ab4
merge forward; some namelist mods
May 6, 2024
dcb4a5c
Merge branch 'development' into tj2016
mwaxmonsky May 10, 2024
0b309d1
instantaneous file output working
May 28, 2024
05ddd39
move registry hist routines to new module; update precision logic
May 31, 2024
23bc2bb
merge to head of cam_development
May 31, 2024
1212435
add constituents handling for add and out fields
May 31, 2024
bc7eba2
code cleanup; better error handling; use new buffer interfaces
Jun 4, 2024
7baf682
code cleanup; fix for indexing on multiple files
Jun 5, 2024
c1acda8
fix and add hist_config tests
Jun 6, 2024
510ade6
add test for ic_names dictionary
Jun 6, 2024
0635898
more thoroughly separate physics history write from generate_registry
Jun 7, 2024
f76b038
Standard names update.
mwaxmonsky Jun 7, 2024
0f53092
Update to latest atmos_phys
mwaxmonsky Jun 7, 2024
7591a27
Update to match generated code from capgen.
mwaxmonsky Jun 7, 2024
52d9b06
create register history tests; update existing tests
Jun 8, 2024
abf329a
enable avg flags in registry; clear buffers
Jun 10, 2024
61963c8
remove fortran tests until we design a new framework
Jun 11, 2024
dc7fe59
merge to head of development
Jun 11, 2024
ae52006
fix mpi broadcast nl counts
Jun 12, 2024
222227a
Updating to latest atmos_phys.
mwaxmonsky Jun 14, 2024
138f410
Updating to latest atmos_phys.
mwaxmonsky Jun 17, 2024
c561a98
Updating to latest atmos_phys.
mwaxmonsky Jun 25, 2024
7eddc01
fix dimensions for parallel; move history to timestep_final
Jun 25, 2024
f1c8c30
merge to head of development
Jun 25, 2024
279a2ca
Fixing merge conflict with branch development
mwaxmonsky Jun 27, 2024
70c235c
Removing standard names csv file as no longer needed.
mwaxmonsky Jun 27, 2024
8a64782
Moving input names to standard names xml generator to tools directory.
mwaxmonsky Jun 27, 2024
dfd52b1
Updating to latest development ccpp-framework.
mwaxmonsky Jun 27, 2024
f63ebf2
Removing commented code and updating standard name.
mwaxmonsky Jun 27, 2024
4fd48f7
conditionally write instantaneous file
Jun 30, 2024
330509e
grid support clean-up
Jul 1, 2024
8319b96
remove pointers for iodesc to be passed to initdecomp
Jul 1, 2024
3ac1b72
Fixing pylint errors.
mwaxmonsky Jul 1, 2024
04fd2ba
fix for intel compiler; add "allocatable" for nl arrays
Jul 5, 2024
2c8c7c5
Merge remote-tracking branch 'ESCOMP/development' into HEAD
Jul 5, 2024
a1a5202
specify character len
Jul 5, 2024
8191b82
fix broadcasts for arrays of character arrays
Jul 5, 2024
1885961
fix frequency calculations; couple fixes for output
Jul 6, 2024
f7e9d1f
allow instantaneous and accumulated fields on same volume
Jul 8, 2024
6024452
Removing anomalous backslash in regex string by using raw string.
mwaxmonsky Jul 8, 2024
498dbbb
code cleanup
Jul 8, 2024
5a7c176
Adding missing docstrings.
mwaxmonsky Jul 9, 2024
2e4b42e
python clean-up
Jul 9, 2024
228fc52
python cleanup for test_hist_config
Jul 9, 2024
14dcf39
remove unnecessary python version check
Jul 9, 2024
d394b92
handle volume configured with no fields
Jul 9, 2024
d87e41f
fix hist_config test
Jul 9, 2024
1a58f15
fix write_nstep0 logic
Jul 9, 2024
67b0b21
add missing mpi_broadcast
Jul 10, 2024
0fe3378
Merge branch 'development' into tj2016
mwaxmonsky Jul 11, 2024
7e8562b
Addressing review comments.
mwaxmonsky Jul 12, 2024
dbdbbad
Addressing review comments and adding error checking to PIO calls.
mwaxmonsky Jul 15, 2024
0f673e2
Addressing review comments. Updating tphys variable name to better c…
mwaxmonsky Jul 15, 2024
277fbdf
Merge remote-tracking branch 'mwax/tj2016' into history_court
Jul 15, 2024
9bd89cb
use diagnostic schemes instead of python-generated history
Jul 22, 2024
512becc
bring to head of development
Jul 24, 2024
8ac3838
fix unit tests
Jul 24, 2024
48f0807
fix to enable no history configurations in nl
Aug 7, 2024
9525b75
remove pio dependency from modular history to enable unit testing
Aug 9, 2024
cf6aa62
merge to head of development
Aug 23, 2024
8322b49
address one wave of review comments
peverwhee Aug 29, 2024
ac7c8a2
Merge branch 'history_court' of https://github.com/peverwhee/CAM-SIMA…
peverwhee Aug 29, 2024
1f023c1
fix python unit tests
peverwhee Aug 29, 2024
0784bcf
fix another unit test
peverwhee Aug 29, 2024
68b73b5
address next chunk of reviewer comments; through cam_history_support
peverwhee Sep 5, 2024
39a66ed
next round of review comments; through cam_history.F90
peverwhee Sep 9, 2024
7db18c3
more cleanup of cam_hist_file
peverwhee Sep 12, 2024
a0533cc
put safe_endrun back
peverwhee Sep 13, 2024
e7a7fa1
append %f to user-specified filename template
peverwhee Sep 13, 2024
813f377
more python review comments; fix unit tests
peverwhee Sep 13, 2024
87a61f3
update regular expressions
peverwhee Sep 14, 2024
923089f
Merge remote-tracking branch 'ESCOMP/development' into history_court
Sep 16, 2024
ea14039
merge to head of cam development; update vertically integrated standa…
Sep 16, 2024
9120544
update standard names
Sep 16, 2024
3ea79b3
address review comments
Sep 25, 2024
b437934
fix unit tests
Sep 25, 2024
dd882d1
update atmospheric_physics hash; small clean-up to python
Oct 17, 2024
175b30b
merge to head of development
Oct 17, 2024
2f32b91
bring in formal history_output tag
Oct 22, 2024
cdf9222
fix num_samples bug
Oct 24, 2024
69ac862
merge to head of development
Oct 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,13 @@
url = https://github.com/NCAR/ccpp-framework
fxtag = 2024-07-19-dev
fxrequired = AlwaysRequired
fxDONOTUSEurl = https://github.com/NCAR/ccpp-framework
fxDONOTUSEurl = https://github.com/NCAR/ccpp-framework
[submodule "history"]
path = src/history/buffers
url = https://github.com/ESMCI/history_output
fxtag = history01_00
fxrequired = AlwaysRequired
fxDONOTUSEurl = https://github.com/ESMCI/history_output
[submodule "mpas"]
path = src/dynamics/mpas/dycore
url = https://github.com/MPAS-Dev/MPAS-Model.git
Expand All @@ -13,8 +19,8 @@
fxDONOTUSEurl = https://github.com/MPAS-Dev/MPAS-Model.git
[submodule "ncar-physics"]
path = src/physics/ncar_ccpp
url = https://github.com/ESCOMP/atmospheric_physics
fxtag = 67bb908e
url = https://github.com/ESCOMP/atmospheric_physics
fxtag = e95c172d7a5a0ebf054f420b08416228e211baa3
fxrequired = AlwaysRequired
fxDONOTUSEurl = https://github.com/ESCOMP/atmospheric_physics
[submodule "ccs_config"]
Expand Down
4 changes: 2 additions & 2 deletions cime_config/atm_in_paramgen.py
Original file line number Diff line number Diff line change
Expand Up @@ -1447,8 +1447,8 @@ def append_user_nl_file(self, user_nl_file):
#Notify loop to check the next line for a comma:
is_continue_line = False
#End if

else:
elif ('hist_' not in line_s[0]):
#Raise parsing error; ignore hist lines to be processed by hist_config.py
emsg = "Cannot parse the following line in '{}' :\n'{}'"
raise AtmInParamGenError(emsg.format(user_nl_file, line))
#End if ("=" sign check)
Expand Down
4 changes: 4 additions & 0 deletions cime_config/buildlib
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,10 @@ def _build_cam():
case.get_value("COMP_INTERFACE")),
os.path.join(atm_root, "src", "dynamics", "utils"),
os.path.join(atm_root, "src", "physics", "utils"),
os.path.join(atm_root, "src", "history"),
os.path.join(atm_root, "src", "history", "buffers", "src"),
os.path.join(atm_root, "src", "history", "buffers", "src", "hash"),
os.path.join(atm_root, "src", "history", "buffers", "src", "util"),
os.path.join(atm_root, "src", "utils")]
for path in phys_dirs:
if path not in paths:
Expand Down
17 changes: 16 additions & 1 deletion cime_config/buildnml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ CAM namelist creator
"""
import sys
import os
import re
import shutil
import logging
import glob
Expand All @@ -31,6 +32,8 @@ sys.path.append(_CIME_CONFIG_PATH)

# Import CAM's configure structure:
from cam_config import ConfigCAM
# HistoryConfig allows translation from user_nl_cam into Fortran namelists
from hist_config import HistoryConfig

#Import CAM's ParamGen class:
from atm_in_paramgen import AtmInParamGen
Expand All @@ -51,6 +54,7 @@ class CamBuildnmlError(ValueError):
# This simplifies the filename mangling for different cases.
def _create_ic_filename(inst_string, i_or_r,
run_refcase, run_refdate, run_reftod):
"""Simplify the filename mangling for different cases."""
return f"{run_refcase}.cam{inst_string}.{i_or_r}.{run_refdate}-{run_reftod}.nc"

##################
Expand Down Expand Up @@ -297,7 +301,8 @@ def buildnml(case, caseroot, compname):
# End if

# Determine location and name of "user_nl_cam" files:
user_nl_file = os.path.join(caseroot, "user_nl_cam" + inst_string)
user_nl_fname = "user_nl_cam" + inst_string
user_nl_file = os.path.join(caseroot, user_nl_fname)

# Check that file actually exists. If not then throw an error:
if not os.path.exists(user_nl_file):
Expand Down Expand Up @@ -362,6 +367,16 @@ def buildnml(case, caseroot, compname):
# Create CAM namelist using CIME's nmlgen routine:
pg_atm.write(namelist_file)

# Add history namelists to atm_in
hist_configs = HistoryConfig(filename=user_nl_file, logger=_LOGGER)
with open(namelist_file, 'a', encoding='utf-8') as nl_file:
hist_configs.output_class_namelist(nl_file)
for key in sorted(hist_configs.keys()):
hist_configs[key].output_config_namelist(nl_file, logger=_LOGGER)
# end for
# end with


###############################################################################
def _main_func():

Expand Down
Loading
Loading