Feature/mpas dycore #2451

dustinswales · 2024-10-07T14:41:08Z

Commit Queue Requirements:

Fill out all sections of this template.
All sub component pull requests have been reviewed by their code managers.
Run the full Intel+GNU RT suite (compared to current baselines) on either Hera/Derecho/Hercules
Commit 'test_changes.list' from previous step

Description:

This pull request contains changes to generalize atmospheric component of the UFS Weather Model:

The reorganization of the repository is to achieve the following goals:

Move dynamical core specific code into subdirectories.
Move common code (e.g. physics) into shareable, cross-dycore, higher-level directories. This is outlined in Figure 1 of the "Tiger Team" report.

Refactor the CMakeLists to build an atmospheric component with either an FV3
or MPAS dynamical core.
Add CMakeLists.txt to build MPAS core_atmosphere component. The CMake recipes from the "MPAS atmospheric core" were adapted to work in the UFS build system.
Modify CCPP physics CmakeLists.txt to build independent MPAS/FV3 couplings to the physics.
Create "stub" mpas infrastructure. This includes stubs for the MPAS atmospheric component driver and overlaying NUOPC cap.

With these changes, the MPAS dynamical core builds within UFS.
Next steps, build atmospheric driver for MPAS, which includes creating new typedefs for coupling between the CCPP and MPAS.

Commit Message:

* UFSWM - Modify build scripts to select which dynamical core to build for the UFS.
  * FV3 - Reorganize into dynamical core specific subdirectories. Modify build systems for multiple dynamical cores. Bring in

Priority:

Normal.

Git Tracking

None

Sub component Pull Requests:

FV3: NOAA-EMC/fv3atm#873
- MPAS-Model: ufs-community/MPAS-Model#1

UFSWM Blocking Dependencies:

None

Changes

No changes to baselines. All RTs passed on Hera using Intel.
New test to tests/rt.conf to build UFS with MPAS dynamical core.

Input data Changes:

None.

Library Changes/Upgrades:

No Updates

Testing Log:

…ture for CCPP. More CMake modifications

…r-model into feature/mpas-dycore-reorg

BrianCurtis-NOAA · 2024-10-07T15:03:18Z

@dustinswales It looks like you had a bunch of changes but a force push removed almost all of them. If you intend to work on this PR for a bit, please move it to a draft form until it's ready to be reviewed by the code managers.

dustinswales · 2024-10-07T15:05:17Z

@BrianCurtis-NOAA Sorry, I was doing some last minute cleanup. All is back now.

BrianCurtis-NOAA

I think that we will need to adjust rt.sh and the other .sh scripts in the tests directory to use MPAS.

I don't see any new tests files either, so you'll want to bring those in.

BrianCurtis-NOAA · 2024-10-07T15:16:01Z

driver/UFSDriver.F90

+#ifdef FRONT_MPAS
+!          if (trim(model) == "mpas") then
+!            call NUOPC_DriverAddComp(driver, trim(prefix), MPAS_SS, &
+!              info=info, petList=petList, comp=comp, rc=rc)
+!            if (ChkErr(rc,__LINE__,u_FILE_u)) return
+!            found_comp = .true.
+!          end if
+#endif


Is there a reason this is commented out currently?

It's just a placeholder for the timebeing.
The NUOPC pieces for MPAS will be added later, when the underlying atmospheric component work on MPAS is further along.

dustinswales · 2024-10-07T15:41:01Z

I think that we will need to adjust rt.sh and the other .sh scripts in the tests directory to use MPAS.

I modified default_vars.sh and added a new "MPAS build only" test to rt.conf. If there are other places that need modification, please let me know.

I don't see any new tests files either, so you'll want to bring those in.

There are no new test files at the moment.

climbfuji · 2024-10-09T03:05:55Z

FV3

This submodule pointer update seems to be wrong. If I click on the changes, I get taken to a update of 22 files all within the NCAR fv3atm fork - should this PR not update the code in the noaa-emc fv3atm? Maybe I am missing something (it's late).

NickSzapiro-NOAA · 2024-10-10T03:19:55Z

tests/default_vars.sh

@@ -499,6 +499,7 @@ export POSTXCONFIG=postxconfig-NT-gfs.txt
 export POSTXCONFIG_FH00=postxconfig-NT-gfs_FH00.txt

 export FV3=true
+export MPAS=true


I would suggest export MPAS=false here in export_fv3 so both aren't true. Making a separate export_mpas () is one option. If you want many of the same settings while turning on MPAS-A, it may be more concise to set:

export_fv3 export FV3=false export MPAS=true

in a regression test until there are defaults to set.

DusanJovic-NOAA · 2024-10-10T15:09:33Z

tests/rt.conf

@@ -433,3 +433,6 @@ RUN | datm_cdeps_control_cfsr                           | + hera hercules
 ### ATM-FBH test ###
 COMPILE | atm_fbh | intel | -DAPP=ATMF -DCCPP_SUITES=FV3_HRRR -D32BIT=ON | - wcoss2  noaacloud  acorn | fv3 |
 RUN | cpld_regional_atm_fbh                             | - wcoss2  noaacloud  acorn           | baseline |
+
+### UFS with MPAS dynamical core ###
+COMPILE | atm_mpas_dyn32 | intel | -DAPP=ATMMPAS -DCCPP_SUITES=FV3_GFS_v16,FV3_GFS_v17_p8_ugwpv1 -D32BIT=ON | | mpas |


I do not see RUN step that executes the ufs-model executable built with this COMPILE step. How do we even know the mpas model is compiled correctly. Do you plan to add at least one regression test for ufs with mpas core?

Okay, I see this question has been already asked in one of the previous comments, and the answer was no. I'm now confused, but I do not see what we get by making these changes in the ufs-weather-model at this time. We can discuss the corresponding PR in the fv3atm and if it's necessary merge it, but why updating ufs-wm without any tests is necessary

-D32BIT=ON is build flag used to build fv3 dycore using single-precision reals as a default. Is it used in MPAS core? I do not see it used in any mpas CMakeLists.txt. If it's not used, it should be removed from this COMPILE step.

NickSzapiro-NOAA · 2024-10-10T17:50:03Z

CMakeLists.txt

@@ -157,7 +159,7 @@ if(FMS)
  elseif (APP MATCHES "^(S2S|S2SA|S2SW|S2SWA|S2SWAL|ATM_DS2S|ATM_DS2S-PCICE|NG-GODAS|HAFS-MOM6|HAFS-MOM6W)$")
    add_library(fms ALIAS FMS::fms_r8)
  endif()
-  if(APP MATCHES "^(ATM|ATMAERO|ATMAQ|ATMWM|ATMW|ATML|ATMF|HAFS|HAFS-ALL)$")
+  if(APP MATCHES "^(ATM|ATMAERO|ATMAQ|ATMWM|ATMW|ATML|ATMF|ATMMPAS|HAFS|HAFS-ALL)$")


Remove if MPAS does not need FMS

dustinswales added 15 commits July 18, 2024 20:18

Create feature/mpas-dycore branch

b411c6c

Reorganize code for MPAS dycore. Isolate FV3 and MPAS pieces.

82dbe8c

Update submodules

4b3b5e3

Added more build step pieces, up until the MPAS dycore build

b01ced7

Some more changes. Almost building.

9e82655

MPAS building in UFSATM! Some more reorganization. Initial infrastruc…

f0c3a78

…ture for CCPP. More CMake modifications

Forgot to link to personal for of MPAS submodule

fe47b33

Address reviewers comments

f26d538

Address comments

4c2f900

Add MPAS to default_vars.sh

4952814

Move MPAS to ufs-community reporitory

d6418b4

Revert unneeded change

7d0e826

Synced FV3

ee33231

Updated FV3 submodule hash

55f4f66

Merge branch 'develop' of https://github.com/ufs-community/ufs-weathe…

e12268e

…r-model into feature/mpas-dycore-reorg

dustinswales force-pushed the feature/mpas-dycore branch from e1538f9 to b411c6c Compare October 7, 2024 14:52

Housekeeping

d08975f

BrianCurtis-NOAA reviewed Oct 7, 2024

View reviewed changes

climbfuji reviewed Oct 9, 2024

View reviewed changes

NickSzapiro-NOAA reviewed Oct 10, 2024

View reviewed changes

DusanJovic-NOAA reviewed Oct 10, 2024

View reviewed changes

NickSzapiro-NOAA reviewed Oct 10, 2024

View reviewed changes

dustinswales marked this pull request as draft November 1, 2024 15:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature/mpas dycore #2451

Feature/mpas dycore #2451

dustinswales commented Oct 7, 2024 •

edited

Loading

BrianCurtis-NOAA commented Oct 7, 2024

dustinswales commented Oct 7, 2024

BrianCurtis-NOAA left a comment

BrianCurtis-NOAA Oct 7, 2024

dustinswales Oct 7, 2024

dustinswales commented Oct 7, 2024 •

edited

Loading

climbfuji Oct 9, 2024

NickSzapiro-NOAA Oct 10, 2024

DusanJovic-NOAA Oct 10, 2024

DusanJovic-NOAA Oct 10, 2024

DusanJovic-NOAA Oct 10, 2024

NickSzapiro-NOAA Oct 10, 2024

Feature/mpas dycore #2451

Are you sure you want to change the base?

Feature/mpas dycore #2451

Conversation

dustinswales commented Oct 7, 2024 • edited Loading

Commit Queue Requirements:

Description:

Commit Message:

Priority:

Git Tracking

Sub component Pull Requests:

UFSWM Blocking Dependencies:

Changes

Input data Changes:

Library Changes/Upgrades:

Testing Log:

BrianCurtis-NOAA commented Oct 7, 2024

dustinswales commented Oct 7, 2024

BrianCurtis-NOAA left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dustinswales commented Oct 7, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dustinswales commented Oct 7, 2024 •

edited

Loading

dustinswales commented Oct 7, 2024 •

edited

Loading