Merge branch 'maint/refactor-as-package' into datastore

.github/pull_request_template.md

@@ -0,0 +1,52 @@
+## Describe your changes
+
+< Summary of the changes.>
+
+< Please also include relevant motivation and context. >
+
+< List any dependencies that are required for this change. >
+
+## Issue Link
+
+< Link to the relevant issue or task. > (e.g. `closes #00` or `solves #00`)
+
+## Type of change
+
+- [ ] 🐛 Bug fix (non-breaking change that fixes an issue)
+- [ ] ✨ New feature (non-breaking change that adds functionality)
+- [ ] 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
+- [ ] 📖 Documentation (Addition or improvements to documentation)
+
+## Checklist before requesting a review
+
+- [ ] My branch is up-to-date with the target branch - if not update your fork with the changes from the target branch (use `pull` with `--rebase` option if possible).
+- [ ] I have performed a self-review of my code
+- [ ] For any new/modified functions/classes I have added docstrings that clearly describe its purpose, expected inputs and returned values
+- [ ] I have placed in-line comments to clarify the intent of any hard-to-understand passages of my code
+- [ ] I have updated the [README](README.MD) to cover introduced code changes
+- [ ] I have added tests that prove my fix is effective or that my feature works
+- [ ] I have given the PR a name that clearly describes the change, written in imperative form ([context](https://www.gitkraken.com/learn/git/best-practices/git-commit-message#using-imperative-verb-form)).
+- [ ] I have requested a reviewer and an assignee (assignee is responsible for merging)
+
+## Checklist for reviewers
+
+Each PR comes with its own improvements and flaws. The reviewer should check the following:
+- [ ] the code is readable
+- [ ] the code is well tested
+- [ ] the code is documented (including return types and parameters)
+- [ ] the code is easy to maintain
+
+## Author checklist after completed review
+
+- [ ] I have added a line to the CHANGELOG describing this change, in a section
+  reflecting type of change (add section where missing):
+  - *added*: when you have added new functionality
+  - *changed*: when default behaviour of the code has been changed
+  - *fixes*: when your contribution fixes a bug
+
+## Checklist for assignee
+
+- [ ] PR is up to date with the base branch
+- [ ] the tests pass
+- [ ] author has added an entry to the changelog (and designated the change as *added*, *changed* or *fixed*)
+- Once the PR is ready to be merged, squash commits and merge the PR.

.github/workflows/ci-pdm-install-and-test-gpu.yml

@@ -24,17 +24,18 @@ jobs:
 
       - name: Create venv
         run: |
+          pdm config venv.in_project False
+          pdm config venv.location /opt/dlami/nvme/venv
           pdm venv create --with-pip
-          pdm use --venv in-project
 
       - name: Install torch (GPU CUDA 12.1)
         run: |
-          python -m pip install torch  --index-url https://download.pytorch.org/whl/cu121
+          pdm run python -m pip install torch  --index-url https://download.pytorch.org/whl/cu121
 
       - name: Print and check torch version
         run: |
-          python -c "import torch; print(torch.__version__)"
-          python -c "import torch; assert not torch.__version__.endswith('+cpu')"
+          pdm run python -c "import torch; print(torch.__version__)"
+          pdm run python -c "import torch; assert not torch.__version__.endswith('+cpu')"
 
       - name: Install package (including dev dependencies)
         run: |

CHANGELOG.md

@@ -28,6 +28,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   [\#6](https://github.com/joeloskarsson/neural-lam/pull/6), [\#8](https://github.com/joeloskarsson/neural-lam/pull/8)
   @sadamov, @joeloskarsson
 
+- added github pull-request template to ease contribution and review process
+  [\#53](https://github.com/mllam/neural-lam/pull/53), @leifdenby
+
+- ci/cd setup for running both CPU and GPU-based testing both with pdm and pip based installs [\#37](https://github.com/mllam/neural-lam/pull/37), @khintz, @leifdenby
+
 ### Changed
 
   Optional multi-core/GPU support for statistics calculation in `create_parameter_weights.py`
@@ -85,6 +90,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   [\#52](https://github.com/mllam/neural-lam/pull/52)
   @joeloskarsson
 
+- Cap numpy version to < 2.0.0
+  [\#68](https://github.com/mllam/neural-lam/pull/68)
+  @joeloskarsson
+
 ## [v0.1.0](https://github.com/joeloskarsson/neural-lam/releases/tag/v0.1.0)
 
 First tagged release of `neural-lam`, matching Oskarsson et al 2023 publication

README.md

@@ -1,5 +1,6 @@
 ![Linting](https://github.com/mllam/neural-lam/actions/workflows/pre-commit.yml/badge.svg?branch=main)
-![Automatic tests](https://github.com/mllam/neural-lam/actions/workflows/run_tests.yml/badge.svg?branch=main)
+[![test (pdm install, gpu)](https://github.com/mllam/neural-lam/actions/workflows/ci-pdm-install-and-test-gpu.yml/badge.svg)](https://github.com/mllam/neural-lam/actions/workflows/ci-pdm-install-and-test-gpu.yml)
+[![test (pdm install, cpu)](https://github.com/mllam/neural-lam/actions/workflows/ci-pdm-install-and-test-cpu.yml/badge.svg)](https://github.com/mllam/neural-lam/actions/workflows/ci-pdm-install-and-test-cpu.yml)
 
 <p align="middle">
     <img src="figures/neural_lam_header.png" width="700">
@@ -58,27 +59,35 @@ Below follows instructions on how to use Neural-LAM to train and evaluate models
 
 ## Installation
 
-The dependencies in `neural-lam` is handled with [pdm](https://pdm.fming.dev/), but you can still install `neural-lam` directly with pip if you prefer. The benefits of using `pdm` are that [pyproject.toml](pyproject.toml) is automatically updated when you add/remove dependencies (with `pdm add <package_name>` or `pdm remove <package_name`).
+When installing `neural-lam` you have a choice of either installing with
+directly `pip` or using the `pdm` package manager.
+We recommend using `pdm` as it makes it easy to add/remove packages while
+keeping versions consistent (it automatically updates the `pyproject.toml`
+file), makes it easy to handle virtual environments and includes the
+development toolchain packages installation too.
 
-### Installing with pdm
+**regarding `torch` installation**: because `torch` creates different package
+variants for different CUDA versions and cpu-only support you will need to install
+`torch` separately if you don't want the most recent GPU variant that also
+expects the most recent version of CUDA on your system.
 
-### Installing with pip
+We cover all the installation options in our [github actions ci/cd
+setup](.github/workflows/) which you can use as a reference.
 
-Follow the steps below to create the necessary python environment.
+### Using `pdm`
 
-1. Install GEOS for your system. For example with `sudo apt-get install libgeos-dev`. This is necessary for the Cartopy requirement.
-2. Use python 3.9.
-3. Install version 2.0.1 of PyTorch. Follow instructions on the [PyTorch webpage](https://pytorch.org/get-started/previous-versions/) for how to set this up with GPU support on your system.
-4. Install required packages specified in `requirements.txt`.
-5. Install PyTorch Geometric version 2.2.0. This can be done by running
-```
-TORCH="2.0.1"
-CUDA="cu117"
+1. Clone this repository and navigate to the root directory.
+2. Install `pdm` if you don't have it installed on your system (either with `pip install pdm` or [following the install instructions](https://pdm-project.org/latest/#installation)). If you are happy using the latest version of `torch` with GPU support (expecting the latest version of CUDA is installed on your system) you can skip to step 5.
+3. Create a virtual environment for pdm to use with `pdm venv create --with-pip`.
+4. Install a specific version of `torch` with `pdm run python -m pip install torch --index-url https://download.pytorch.org/whl/cpu` for a CPU-only version or `pdm run python -m pip install torch --index-url https://download.pytorch.org/whl/cu111` for CUDA 11.1 support (you can find the correct URL for the variant you want on [PyTorch webpage](https://pytorch.org/get-started/locally/)).
+5. Install the dependencies with `pdm install`. If you will be developing `neural-lam` we recommend to install the development dependencies with `pdm install --dev`. By default `pdm` installs the `neural-lam` package in editable mode, so you can make changes to the code and see the effects immediately.
+
+### Using `pip`
+
+1. Clone this repository and navigate to the root directory. If you are happy using the latest version of `torch` with GPU support (expecting the latest version of CUDA is installed on your system) you can skip to step 3.
+2. Install a specific version of `torch` with `python -m pip install torch --index-url https://download.pytorch.org/whl/cpu` for a CPU-only version or `python -m pip install torch --index-url https://download.pytorch.org/whl/cu111` for CUDA 11.1 support (you can find the correct URL for the variant you want on [PyTorch webpage](https://pytorch.org/get-started/locally/)).
+3. Install the dependencies with `python -m pip install .`. If you will be developing `neural-lam` we recommend to install in editable mode with `python -m pip install -e .` so you can make changes to the code and see the effects immediately. The development dependencies to install are listed in `pyproject.toml`.
 
-pip install pyg-lib==0.2.0 torch-scatter==2.1.1 torch-sparse==0.6.17 torch-cluster==1.6.1\
-    torch-geometric==2.3.1 -f https://pytorch-geometric.com/whl/torch-${TORCH}+${CUDA}.html
-```
-You will have to adjust the `CUDA` variable to match the CUDA version on your system or to run on CPU. See the [installation webpage](https://pytorch-geometric.readthedocs.io/en/latest/install/installation.html) for more information.
 
 ## Data
 Datasets should be stored in a directory called `data`.