Benchmark with DaCe cpu and gpu backends #50

fmalatino · 2024-01-16T20:50:06Z

Purpose

Branch for benchmarking of PACE. It includes four new configuration files to be used in benchmarking with DaCe backends for use with cpus and gpus.

Code changes:

Addition of four configuration files: baroclinic_c384_cpu/gpu.yaml, baroclinic_c3072_cpu/gpu.yaml
Use of dace_inhibitor decorator in microphysics.py

Requirements changes:

In constraints.txt: dace updated to 0.15.1
In constraints.txt: gridtools-cpp updated to 2.3.1

Infrastructure changes:

N/A

Checklist

Before submitting this PR, please make sure:

You have followed the coding standards guidelines established at Code Review Checklist.
Docstrings and type hints are added to new and updated routines, as appropriate
All relevant documentation has been updated or added (e.g. README, CONTRIBUTING docs)
For each public change and fix in pace-util, HISTORY has been updated
Unit tests are added or updated for non-stencil code changes

FlorianDeconinck · 2024-01-17T01:32:23Z

The DaCe version has been updated? Did we run model validation on it?

fmalatino · 2024-01-17T15:18:29Z

The DaCe version has been updated? Did we run model validation on it?

I have to change it to your gcc_dies_on_dacecpu branch, will be in the coming push.

oelbert

Mostly fine, just a couple of questions to clear up

oelbert · 2024-01-23T21:09:31Z

.gitmodules

 	path = external/dace
-	url = https://github.com/spcl/dace.git
+	url = https://github.com/FlorianDeconinck/dace.git


I really do not love doing this, cannot wait to have it resolved

oelbert · 2024-01-23T21:12:09Z

driver/examples/configs/baroclinic_c384_cpu.yaml

@@ -72,7 +71,7 @@ dycore_config:
  nwat: 6
  p_fac: 0.1
  rf_cutoff: 800.
-  rf_fast: false
+  rf_fast: true


Has rf_fast=True been implemented?

It has not, but when run with it set to false, it throws an NotImplementedError when tau != 0. Should this instead be removed or tau == 0?

You could cherry pick over the not-implemented PR from Florian and remove the options from the yaml config.

I think I'd rather change tau than set something in the config that we haven't actually implemented in the model, at least for now. I'll create an issue for it as well though so we can implement a config that's internally consistent.

tests/main/driver/test_example_configs.py

Translate test: small improvements - Parametrize perturbation upon failure - Refactor error folder to be `pwd` based - Fix GPU translate unable to dump error `nc` - Fix mixmatch precision on translate test - Update README.md Test fix: - Orchestrate YPPM for translate purposes Misc: - Fix bad logger formatting on DaCeProgress

…FDL#40) (NOAA-GFDL#48) * [Feature] Guarding against unimplemented configuration (NOAA-GFDL#40) Guarding against unimplemented namelists options: - a2b_ord4 - d_sw - fv_dynamics - fv_subgridz - neg_adj3 - divergence damping - xppm - yppm Misc: - Fix `netcdf_monitor` not mkdir the directory - Add `as_dict` to the dycore state to dump the dycore more easily * Unused assert * Update fv3core/pace/fv3core/stencils/yppm.py Co-authored-by: Oliver Elbert <[email protected]> * Update fv3core/pace/fv3core/stencils/xppm.py Co-authored-by: Oliver Elbert <[email protected]> * Change NotImplemented to ValueError for n_sponge<3 * lint --------- Co-authored-by: Oliver Elbert <[email protected]>

oelbert

👍

* initial changes to read in ak bk * read ak/bk * add xfail * remove input dir * further changes to unit tests * finish up test * add history * commit uncommited files * fix test comment * add input to top * read in data * read in netcdf file in eta mod * remove txt file * test * modify test and fix generate.py * remove emacs backup file * driver tests pass * fix helper.py * fix fv3core tests * fix physics test * fix grid tests * nullcommconfig * cleanup input * remove driver input * remove top level input * fix circular import problems * modify eta_file readin for test_restart_serial * comment out 91 test * rm safety checks * revert diagnostics.py * restore driver.py * revert initialization.py * restore state.py * restore analytic_init.py * restore init_utils.py and analytic_init.py * restore c_sw.py * d2a2c_vect.py * restore fv3core/stensils * restore translate_fvdynamics * restore physics/stencils * restore stencils * remove circular dependency * use pytest parametrize * cleanup generation.py * fstrinngs * add eta_file to MetricTerm init * remove eta_file argument in new_from_metric_terms and geos_wrapper * use pytest parametrize for the xfail tests * use pytest parametrize for the xfail tests * fix geos_wrapper and grid * fix tests * fstring * add test comments * fix util/HISTORY.md * fix comments * remove __init__.py from tests/main/grid * add jupyter notebooks to generate eta files * generate ak,bk,ptop on metricterm init * fix tests * exploit np.all in eta mod * remove tests/main/grid/input * update ci * test * remove input * edit ci yaml * remove push --------- Co-authored-by: mlee03 <[email protected]>

* initial commit, need to adapt and run tests * revising scheme name * tests pass * update history * linting * changing typehints for physics schemes to enum instead of str * driver now works with physics config enum, tests pass * fixed tests * missed one

* Testing changes reflected across branches * Undoing changes made in build_gaea_c5.sh * Testing vscode functionality, by adding a change to external_grid branch * Testing vscode functionality, by adding a change to external_grid branch * Addition of from_generated method and calc_flag to util/pace/util/grid/generation.py * Added get_grid method for external grid data to driver/pace/driver/grid.py * Preliminary xarray netcdf read in method added to driver/pace/driver/grid.py * Updating util/pace/util/grid/generation.py from_generated method * Addition of external grid data read in methods for initialization of grid. Current method uses xarray to interact with netcdf tile files. Values for longitutde, latitude, number of points in x an y, grid edge distances read in. * driver/examples/configs/test_external_C12_1x1.yaml * Preliminary unit test for external grid data read in * Current state of unit tests as of 27 Nov 2023 * External grid method and unit tests added * Re-excluding external grid data yamls from test_example_configs.py * Update driver/pace/driver/grid.py Co-authored-by: Florian Deconinck <[email protected]> * Changed name of grid initializer function to match NetCDF dependency and class descriptor * Update util/pace/util/grid/generation.py Moved position of doc string for "from_external" MetricTerms class method Co-authored-by: Oliver Elbert <[email protected]> * Fixed indentation error in generation.py from suggestion in PR 42 * Removal of TODO comment in grid.py, changes to method of file accessing in test_analytic_init, test_external_grid_* * Changed grid data read-in unit tests to compare data directly from file to driver grid data generated from yaml * Change to reading in lon and lat, other metric terms calculated as needed * Removed read-in of dx, dy, and area. Changed unit tests to compare calculated area to 'ideal' surface area as given by selected constants type. * Update tests/mpi_54rank/test_external_grid_1x1.py Incorrect name of test in test_external_grid_1x1.py changed to match file name Co-authored-by: Oliver Elbert <[email protected]> * Added comparisons for read-in vs generated by driver lon, lat, dx, dy, and area data to unit tests * Added relative error calculations to unit tests for external grid data read-in * External grid data read in tests changed: relative errors printed by each rank and get_tile_number replacing get_tile_index * Removing commented out sections in test_external_grid_2x2.py Co-authored-by: Oliver Elbert <[email protected]> * Updated external grid data read-in to take configuration and input data locations from command line, updated test description, and added documentation on grid construction to external grid data configuration selection dataclass. * Updated documentation in grid.py * Updated external grid data read in unit test to use parametrize functionality of pytest * Ammended files to reference changes to PR 36 --------- Co-authored-by: Frank Malatino <[email protected]> Co-authored-by: Florian Deconinck <[email protected]> Co-authored-by: Oliver Elbert <[email protected]>

fmalatino added 2 commits January 12, 2024 19:16

Benchmark changes

f1d77b8

Added benchmark configurations

038caf4

fmalatino changed the title ~~Benchmark~~ Benchmark with DaCe cpu and gpu backends Jan 16, 2024

fmalatino requested a review from bensonr January 16, 2024 20:51

fmalatino and others added 3 commits January 16, 2024 15:52

Merge branch 'benchmark' into benchmark

3bd1772

Removed benchmark configs from configs to be tested in unit tests

3ea7ce3

Merge branch 'benchmark' of github.com:fmalatino/pace into benchmark

5d3539f

fmalatino added 3 commits January 17, 2024 10:27

Changed dace submodule to point to Florian fix

e7f2c39

Dace submodule points to gcc_dies_on_dacecpu branch

6f4d710

Set 'rf_fast' to true in baroclinic_c384_cpu/gpu.yaml

e69aaa1

bensonr approved these changes Jan 23, 2024

View reviewed changes

oelbert reviewed Jan 23, 2024

View reviewed changes

oelbert and others added 5 commits January 23, 2024 19:32

fix mpi4py version (NOAA-GFDL#51)

735e972

Re-try of updating dace submodule to track Florian fix branch

29a4727

Reverted gt4py submodule to match main checkout

101980d

oelbert approved these changes Jan 26, 2024

View reviewed changes

fmalatino and others added 6 commits January 26, 2024 15:24

Added benchmark README file

d0696f3

Merge branch 'main' into benchmark

208fd62

Merged main after merge of PR 42

35ef834

fmalatino merged commit cd1bd06 into NOAA-GFDL:benchmark Jan 30, 2024
2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Benchmark with DaCe cpu and gpu backends #50

Benchmark with DaCe cpu and gpu backends #50

fmalatino commented Jan 16, 2024 •

edited

Loading

FlorianDeconinck commented Jan 17, 2024

fmalatino commented Jan 17, 2024

oelbert left a comment

oelbert Jan 23, 2024

oelbert Jan 23, 2024

fmalatino Jan 24, 2024

bensonr Jan 24, 2024

oelbert Jan 26, 2024

oelbert left a comment

Benchmark with DaCe cpu and gpu backends #50

Benchmark with DaCe cpu and gpu backends #50

Conversation

fmalatino commented Jan 16, 2024 • edited Loading

Purpose

Code changes:

Requirements changes:

Infrastructure changes:

Checklist

FlorianDeconinck commented Jan 17, 2024

fmalatino commented Jan 17, 2024

oelbert left a comment

Choose a reason for hiding this comment

oelbert Jan 23, 2024

Choose a reason for hiding this comment

oelbert Jan 23, 2024

Choose a reason for hiding this comment

fmalatino Jan 24, 2024

Choose a reason for hiding this comment

bensonr Jan 24, 2024

Choose a reason for hiding this comment

oelbert Jan 26, 2024

Choose a reason for hiding this comment

oelbert left a comment

Choose a reason for hiding this comment

fmalatino commented Jan 16, 2024 •

edited

Loading