Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Local w3emc module, removing sp and bacio dependency #468

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Local w3emc module, removing sp and bacio dependency #468

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 0 additions & 36 deletions .github/workflows/ci_build_scm_ubuntu_22.04.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,7 @@ jobs:
env:
NFHOME: /home/runner/netcdf-fortran
NFVERSION: v4.5.3
bacio_ROOT: /home/runner/bacio
sp_ROOT: /home/runner/NCEPLIBS-sp
w3emc_ROOT: /home/runner/myw3emc
SCM_ROOT: /home/runner/work/ccpp-scm/ccpp-scm
suites: SCM_GFS_v15p2,SCM_GFS_v16,SCM_GFS_v17_p8,SCM_HRRR,SCM_RRFS_v1beta,SCM_RAP,SCM_WoFS_v0,SCM_GFS_v15p2_ps,SCM_GFS_v16_ps,SCM_GFS_v17_p8_ps,SCM_HRRR_ps,SCM_RRFS_v1beta_ps,SCM_RAP_ps,SCM_WoFS_v0_ps

Expand Down Expand Up @@ -68,23 +66,6 @@ jobs:
echo "LD_LIBRARY_PATH=/home/runner/ompi-4.1.6/lib:$LD_LIBRARY_PATH" >> $GITHUB_ENV
echo "PATH=/home/runner/ompi-4.1.6/bin:$PATH" >> $GITHUB_ENV

- name: Cache bacio library v2.4.1
id: cache-bacio-fortran
uses: actions/cache@v3
with:
path: /home/runner/bacio
key: cache-bacio-fortran-${{matrix.fortran-compiler}}-key

- name: Install bacio library v2.4.1
if: steps.cache-bacio-fortran.outputs.cache-hit != 'true'
run: |
git clone --branch v2.4.1 https://github.com/NOAA-EMC/NCEPLIBS-bacio.git bacio
cd bacio && mkdir build && cd build
cmake -DCMAKE_INSTALL_PREFIX=${bacio_ROOT} ../
make -j2
make install
echo "bacio_DIR=/home/runner/bacio/lib/cmake/bacio" >> $GITHUB_ENV

- name: Cache SP-library v2.3.3
id: cache-sp-fortran
uses: actions/cache@v3
Expand All @@ -102,23 +83,6 @@ jobs:
make install
echo "sp_DIR=/home/runner/NCEPLIBS-sp/lib/cmake/sp" >> $GITHUB_ENV

- name: Cache w3emc library v2.9.2
id: cache-w3emc-fortran
uses: actions/cache@v3
with:
path: /home/runner/myw3emc
key: cache-w3emc-fortran-${{matrix.fortran-compiler}}-key

- name: Install w3emc library v2.9.2
if: steps.cache-w3emc-fortran.outputs.cache-hit != 'true'
run: |
git clone --branch v2.9.2 https://github.com/NOAA-EMC/NCEPLIBS-w3emc.git NCEPLIBS-w3emc
cd NCEPLIBS-w3emc && mkdir build && cd build
cmake -DCMAKE_INSTALL_PREFIX=${w3emc_ROOT} ../
make -j2
make install
echo "w3emc_DIR=/home/runner/myw3emc/lib/cmake/w3emc" >> $GITHUB_ENV

- name: Install NetCDF C library
run: |
sudo apt-get update
Expand Down
36 changes: 0 additions & 36 deletions .github/workflows/ci_build_scm_ubuntu_22.04_nvidia.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,7 @@ jobs:
# Environmental variables
env:
NETCDF: /home/runner/netcdf
bacio_ROOT: /home/runner/bacio
sp_ROOT: /home/runner/NCEPLIBS-sp
w3emc_ROOT: /home/runner/myw3emc
SCM_ROOT: /home/runner/work/ccpp-scm/ccpp-scm
zlib_ROOT: /home/runner/zlib
HDF5_ROOT: /home/runner/hdf5
Expand Down Expand Up @@ -185,23 +183,6 @@ jobs:
make
make install

- name: Cache bacio library v2.4.1
id: cache-bacio-fortran
uses: actions/cache@v3
with:
path: /home/runner/bacio
key: cache-bacio-fortran-${{matrix.fortran-compiler}}-key

- name: Install bacio library v2.4.1
if: steps.cache-bacio-fortran.outputs.cache-hit != 'true'
run: |
git clone --branch v2.4.1 https://github.com/NOAA-EMC/NCEPLIBS-bacio.git bacio
cd bacio && mkdir build && cd build
cmake -DCMAKE_INSTALL_PREFIX=${bacio_ROOT} ../
make -j2
make install
echo "bacio_DIR=/home/runner/bacio/lib/cmake/bacio" >> $GITHUB_ENV

- name: Cache SP-library v2.3.3
id: cache-sp-fortran
uses: actions/cache@v3
Expand All @@ -219,23 +200,6 @@ jobs:
make install
echo "sp_DIR=/home/runner/NCEPLIBS-sp/lib/cmake/sp" >> $GITHUB_ENV

- name: Cache w3emc library v2.9.2
id: cache-w3emc-fortran
uses: actions/cache@v3
with:
path: /home/runner/myw3emc
key: cache-w3emc-fortran-${{matrix.fortran-compiler}}-key

- name: Install w3emc library v2.9.2
if: steps.cache-w3emc-fortran.outputs.cache-hit != 'true'
run: |
git clone --branch v2.9.2 https://github.com/NOAA-EMC/NCEPLIBS-w3emc.git NCEPLIBS-w3emc
cd NCEPLIBS-w3emc && mkdir build && cd build
cmake -DCMAKE_INSTALL_PREFIX=${w3emc_ROOT} ../
make -j2
make install
echo "w3emc_DIR=/home/runner/myw3emc/lib/cmake/w3emc" >> $GITHUB_ENV

#######################################################################################
# Build and run SCM regression tests (ccpp-scm/test/rt_test_cases.py)
#######################################################################################
Expand Down
54 changes: 0 additions & 54 deletions .github/workflows/ci_run_scm_DEPHY.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,6 @@ jobs:
env:
NFHOME: /home/runner/netcdf-fortran
NFVERSION: v4.5.3
bacio_ROOT: /home/runner/bacio
sp_ROOT: /home/runner/NCEPLIBS-sp
w3emc_ROOT: /home/runner/myw3emc
SCM_ROOT: /home/runner/work/ccpp-scm/ccpp-scm
suites: SCM_GFS_v16,SCM_GFS_v16_ps

Expand Down Expand Up @@ -66,57 +63,6 @@ jobs:
echo "LD_LIBRARY_PATH=/home/runner/ompi-4.1.6/lib:$LD_LIBRARY_PATH" >> $GITHUB_ENV
echo "PATH=/home/runner/ompi-4.1.6/bin:$PATH" >> $GITHUB_ENV

- name: Cache bacio library v2.4.1
id: cache-bacio-fortran
uses: actions/cache@v3
with:
path: /home/runner/bacio
key: cache-bacio-fortran-${{matrix.fortran-compiler}}-key

- name: Install bacio library v2.4.1
if: steps.cache-bacio-fortran.outputs.cache-hit != 'true'
run: |
git clone --branch v2.4.1 https://github.com/NOAA-EMC/NCEPLIBS-bacio.git bacio
cd bacio && mkdir build && cd build
cmake -DCMAKE_INSTALL_PREFIX=${bacio_ROOT} ../
make -j2
make install
echo "bacio_DIR=/home/runner/bacio/lib/cmake/bacio" >> $GITHUB_ENV

- name: Cache SP-library v2.3.3
id: cache-sp-fortran
uses: actions/cache@v3
with:
path: /home/runner/NCEPLIBS-sp
key: cache-sp-fortran-${{matrix.fortran-compiler}}-key

- name: Install SP-library v2.3.3
if: steps.cache-sp-fortran.outputs.cache-hit != 'true'
run: |
git clone --branch v2.3.3 https://github.com/NOAA-EMC/NCEPLIBS-sp.git NCEPLIBS-sp
cd NCEPLIBS-sp && mkdir build && cd build
cmake -DCMAKE_INSTALL_PREFIX=${sp_ROOT} ../
make -j2
make install
echo "sp_DIR=/home/runner/NCEPLIBS-sp/lib/cmake/sp" >> $GITHUB_ENV

- name: Cache w3emc library v2.9.2
id: cache-w3emc-fortran
uses: actions/cache@v3
with:
path: /home/runner/myw3emc
key: cache-w3emc-fortran-${{matrix.fortran-compiler}}-key

- name: Install w3emc library v2.9.2
if: steps.cache-w3emc-fortran.outputs.cache-hit != 'true'
run: |
git clone --branch v2.9.2 https://github.com/NOAA-EMC/NCEPLIBS-w3emc.git NCEPLIBS-w3emc
cd NCEPLIBS-w3emc && mkdir build && cd build
cmake -DCMAKE_INSTALL_PREFIX=${w3emc_ROOT} ../
make -j2
make install
echo "w3emc_DIR=/home/runner/myw3emc/lib/cmake/w3emc" >> $GITHUB_ENV

- name: Install NetCDF C library
run: sudo apt-get install libnetcdf-dev

Expand Down
36 changes: 0 additions & 36 deletions .github/workflows/ci_run_scm_rts.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,7 @@ jobs:
env:
NFHOME: /home/runner/netcdf-fortran
NFVERSION: v4.5.3
bacio_ROOT: /home/runner/bacio
sp_ROOT: /home/runner/NCEPLIBS-sp
w3emc_ROOT: /home/runner/myw3emc
SCM_ROOT: /home/runner/work/ccpp-scm/ccpp-scm
suites: SCM_GFS_v15p2,SCM_GFS_v16,SCM_GFS_v17_p8,SCM_HRRR,SCM_RRFS_v1beta,SCM_RAP,SCM_WoFS_v0,SCM_RRFS_v1,SCM_GFS_v17_HR3,SCM_GFS_v17_HR3_RRTMGP
suites_ps: SCM_GFS_v15p2_ps,SCM_GFS_v16_ps,SCM_GFS_v17_p8_ps,SCM_HRRR_ps,SCM_RRFS_v1beta_ps,SCM_RAP_ps,SCM_WoFS_v0_ps,SCM_RRFS_v1_ps,SCM_GFS_v17_HR3_ps,SCM_GFS_v17_HR3_RRTMGP_ps
Expand Down Expand Up @@ -72,23 +70,6 @@ jobs:
echo "LD_LIBRARY_PATH=/home/runner/ompi-4.1.6/lib:$LD_LIBRARY_PATH" >> $GITHUB_ENV
echo "PATH=/home/runner/ompi-4.1.6/bin:$PATH" >> $GITHUB_ENV

- name: Cache bacio library v2.4.1
id: cache-bacio-fortran
uses: actions/cache@v3
with:
path: /home/runner/bacio
key: cache-bacio-fortran-${{matrix.fortran-compiler}}-key

- name: Install bacio library v2.4.1
if: steps.cache-bacio-fortran.outputs.cache-hit != 'true'
run: |
git clone --branch v2.4.1 https://github.com/NOAA-EMC/NCEPLIBS-bacio.git bacio
cd bacio && mkdir build && cd build
cmake -DCMAKE_INSTALL_PREFIX=${bacio_ROOT} ../
make -j2
make install
echo "bacio_DIR=/home/runner/bacio/lib/cmake/bacio" >> $GITHUB_ENV

- name: Cache SP-library v2.3.3
id: cache-sp-fortran
uses: actions/cache@v3
Expand All @@ -106,23 +87,6 @@ jobs:
make install
echo "sp_DIR=/home/runner/NCEPLIBS-sp/lib/cmake/sp" >> $GITHUB_ENV

- name: Cache w3emc library v2.9.2
id: cache-w3emc-fortran
uses: actions/cache@v3
with:
path: /home/runner/myw3emc
key: cache-w3emc-fortran-${{matrix.fortran-compiler}}-key

- name: Install w3emc library v2.9.2
if: steps.cache-w3emc-fortran.outputs.cache-hit != 'true'
run: |
git clone --branch v2.9.2 https://github.com/NOAA-EMC/NCEPLIBS-w3emc.git NCEPLIBS-w3emc
cd NCEPLIBS-w3emc && mkdir build && cd build
cmake -DCMAKE_INSTALL_PREFIX=${w3emc_ROOT} ../
make -j2
make install
echo "w3emc_DIR=/home/runner/myw3emc/lib/cmake/w3emc" >> $GITHUB_ENV

- name: Install NetCDF C library
run: sudo apt-get install libnetcdf-dev

Expand Down
1 change: 1 addition & 0 deletions ccpp/config/ccpp_prebuild_config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
# actual variable definition files
'ccpp/framework/src/ccpp_types.F90',
'ccpp/physics/physics/hooks/machine.F',
'ccpp/physics/physics/tools/w3emc.F90',
'ccpp/physics/physics/Radiation/RRTMG/radsw_param.f',
'ccpp/physics/physics/Radiation/RRTMG/radlw_param.f',
'ccpp/physics/physics/photochem/h2o_def.f',
Expand Down
2 changes: 1 addition & 1 deletion ccpp/physics
Open in desktop
Submodule physics updated 14 files
+4 −4 CMakeLists.txt
+1 −1 physics/GWD/cires_tauamf_data.F90
+5 −5 physics/GWD/cires_ugwpv1_module.F90
+3 −1 physics/Interstitials/UFS_SCM_NEPTUNE/GFS_phys_time_vary.fv3.F90
+2 −0 physics/Interstitials/UFS_SCM_NEPTUNE/GFS_phys_time_vary.scm.F90
+1 −1 physics/Interstitials/UFS_SCM_NEPTUNE/GFS_time_vary_pre.fv3.F90
+3 −1 physics/Interstitials/UFS_SCM_NEPTUNE/GFS_time_vary_pre.scm.F90
+2 −0 physics/Interstitials/UFS_SCM_NEPTUNE/iccninterp.F90
+397 −73 physics/Interstitials/UFS_SCM_NEPTUNE/sfcsub.F
+1 −0 physics/MP/Morrison_Gettelman/aerinterp.F90
+8 −8 physics/Radiation/radiation_astronomy.f
+1 −1 physics/SFC_Layer/UFS/module_nst_water_prop.f90
+1 −1 physics/photochem/h2ointerp.f90
+343 −0 physics/tools/w3emc.F90
4 changes: 2 additions & 2 deletions docker/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ RUN mkdir -p $NCEPLIBS_DIR/src && cd $NCEPLIBS_DIR/src \
&& mkdir NCEPLIBS-bacio/build && cd NCEPLIBS-bacio/build \
&& cmake -DCMAKE_INSTALL_PREFIX=$NCEPLIBS_DIR .. \
&& make VERBOSE=1 \
&& make install
&& make install

RUN cd $NCEPLIBS_DIR/src \
&& git clone -b v2.3.3 --recursive https://github.com/NOAA-EMC/NCEPLIBS-sp \
Expand Down Expand Up @@ -82,7 +82,7 @@ RUN cd /comsoftware \
&& make -j4

# The analysis scripts have options for using LaTeX when making figure labels.
# If you would like to install LaTeK, uncomment the section below.
# If you would like to install LaTeK, uncomment the section below.
# Note: This will increase the image size by 1 GB.
#USER root
#RUN yum -y update \
Expand Down
20 changes: 4 additions & 16 deletions scm/doc/TechGuide/chap_quick.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ System Requirements, Libraries, and Tools
The source code for the SCM and CCPP components is in the form of
programs written in FORTRAN 90 (with some required features from the
FORTRAN 2008 standard), and C. In addition, the model I/O
relies on the NetCDF libraries, as well as the NCEP libraries ``bacio``, ``sp`` and ``w3emc``.
relies on the NetCDF libraries.

Beyond the standard shell scripts, the build
system relies on use of the Python scripting language, along with cmake,
Expand All @@ -148,12 +148,6 @@ contains the following set of libraries needed for building the SCM:

- Netcdf-FORTRAN (v4.6.0)

- BACIO (v2.4.1) - Binary I/O Library

- SP (v2.3.3) - Spectral Transformation Library

- W3EMC (2.10.0) - GRIB decoder and encoder library

Instructions for installing spack-stack can be found in the `spack-stack documentation <https://spack-stack.readthedocs.io/en/latest/>`__.
Spack-stack is already installed and maintained on many HPC platforms, including NSF NCAR's Derecho, NOAA's Hera and
Jet, and MSU's Orion.
Expand Down Expand Up @@ -239,9 +233,9 @@ or ``linux_gnu`` modules.

If libraries were installed manually, users will need to set some environment variables
needed for specifying the location of the various prerequisites. Users will need to set variables for the
compilers (``CC``, ``CXX``, ``FC``), as well as the root directories for the library installs of NetCDF (``NetCDF_ROOT``),
``bacio`` (``bacio_ROOT``), ``sp`` (``sp_ROOT``), and ``w3emc`` (``w3emc_ROOT``). This is the procedure used in the
provided Dockerfile in ``ccpp-scm/docker/``, so users can reference that file for guidance on how to install this software
compilers (``CC``, ``CXX``, ``FC``), as well as the root directories for the library installs of NetCDF (``NetCDF_ROOT``).
This is the procedure used in the provided Dockerfile in ``ccpp-scm/docker/``,
so users can reference that file for guidance on how to install this software
and set these variables.


Expand Down Expand Up @@ -774,12 +768,6 @@ the Docker container contains all of this software built-in, you do not need to

- Python - 3.11.2

- NCEPLIBS BACIO - v2.4.1

- NCEPLIBS SP - v2.3.3

- NCEPLIBS W3EMC - v2.11.0

A Docker image containing the SCM, CCPP, and its software prerequisites
can be generated from the code in the software repository obtained by
following the instructions in :numref:`Section %s <obtaining_code>`,
Expand Down
4 changes: 2 additions & 2 deletions scm/doc/TechGuide/chap_repo.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,8 +76,8 @@ Continuous Integration
The CCPP SCM GitHub repository is set up with Continuous Integration (CI) testing for building the SCM and running some simple test cases. These tests are run automatically on code changes before they are merged, and so ensures that new changes to the SCM do not break basic capabilities. The latest set of tests use the following combinations of SCM prerequisites:

**Regression tests**
- GNU compilers 11.4.0, Python 3.9.12, netCDF-c 4.7.3, netCDF-FORTRAN 4.5.3, bacio 2.4.1, sp 2.3.3, and w3emc 2.9.2
- GNU compilers 11.4.0, Python 3.9.12, netCDF-c 4.7.3, and netCDF-FORTRAN 4.5.3

**Build tests**

All tests use the same versions of NCEP-supported libraries: bacio 2.4.1, sp 2.3.3, and w3emc 2.9.2. Detailed information on these tests can be found in the definition files for these tests, stored in the SCM repository under ``ccpp-scm/.github/workflows``.
Detailed information on these tests can be found in the definition files for these tests, stored in the SCM repository under ``ccpp-scm/.github/workflows``.
3 changes: 0 additions & 3 deletions scm/etc/modules/derecho_gnu.lua
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,6 @@ load("py-netcdf4/1.5.8")

load("netcdf-c/4.9.2")
load("netcdf-fortran/4.6.0")
load("bacio/2.4.1")
load("sp/2.3.3")
load("w3emc")

setenv("CMAKE_C_COMPILER","mpicc")
setenv("CMAKE_CXX_COMPILER","mpicxx")
Expand Down
3 changes: 0 additions & 3 deletions scm/etc/modules/derecho_intel.lua
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,6 @@ load("py-netcdf4/1.5.8")

load("netcdf-c/4.9.2")
load("netcdf-fortran/4.6.0")
load("bacio/2.4.1")
load("sp/2.3.3")
load("w3emc")

setenv("CMAKE_C_COMPILER","cc")
setenv("CMAKE_CXX_COMPILER","CC")
Expand Down
3 changes: 0 additions & 3 deletions scm/etc/modules/hera_gnu.lua
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,6 @@ load("py-netcdf4/1.5.8")

load("netcdf-c/4.9.2")
load("netcdf-fortran/4.6.0")
load("bacio/2.4.1")
load("sp/2.3.3")
load("w3emc")

setenv("CMAKE_C_COMPILER","mpicc")
setenv("CMAKE_CXX_COMPILER","mpicxx")
Expand Down
3 changes: 0 additions & 3 deletions scm/etc/modules/hera_intel.lua
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,6 @@ load("py-netcdf4/1.5.8")

load("netcdf-c/4.9.2")
load("netcdf-fortran/4.6.0")
load("bacio/2.4.1")
load("sp/2.3.3")
load("w3emc")

setenv("CMAKE_C_COMPILER","mpiicc")
setenv("CMAKE_CXX_COMPILER","mpiicpc")
Expand Down
3 changes: 0 additions & 3 deletions scm/etc/modules/jet_gnu.lua
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,6 @@ load("py-netcdf4/1.5.8")

load("netcdf-c/4.9.2")
load("netcdf-fortran/4.6.0")
load("bacio/2.4.1")
load("sp/2.3.3")
load("w3emc")

setenv("CMAKE_C_COMPILER","mpicc")
setenv("CMAKE_CXX_COMPILER","mpicxx")
Expand Down
3 changes: 0 additions & 3 deletions scm/etc/modules/jet_intel.lua
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,6 @@ load("py-netcdf4/1.5.8")

load("netcdf-c/4.9.2")
load("netcdf-fortran/4.6.0")
load("bacio/2.4.1")
load("sp/2.3.3")
load("w3emc")

setenv("CMAKE_C_COMPILER","mpiicc")
setenv("CMAKE_CXX_COMPILER","mpiicpc")
Expand Down
Loading
Loading