Skip to content

Commit

Permalink
Add release tests with separate workflow
Browse files Browse the repository at this point in the history
The added `cd` workflow for now only builds CADET and executes dedicated
tests that are too extensive for the CI but should be tested before
releasing a new version.
  • Loading branch information
jbreue16 committed Dec 4, 2024
1 parent ef257ea commit 8fc5a7b
Show file tree
Hide file tree
Showing 7 changed files with 70 additions and 8 deletions.
59 changes: 59 additions & 0 deletions .github/workflows/cd.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
name: CD tests

on:
workflow_dispatch:
inputs:
ref:
description: 'Commit hash, branch name, or tag to run the CD pipeline for'
required: false
default: 'HEAD'
type: string


jobs:
Ubuntu-latest:
runs-on: ubuntu-latest
strategy:
fail-fast: true
defaults:
run:
shell: bash -l {0}
env:
SRC_DIR: ${{ github.workspace }}/src
BUILD_DIR: ${{ github.workspace }}/build
INSTALL_PREFIX: ${{ github.workspace }}/install
BUILD_TYPE: Release
steps:
- uses: actions/checkout@v4
with:
submodules: true
path: src
ref: ${{ github.event.inputs.ref || github.ref }}
- name: Install Dependencies
run: |
sudo apt-get update
sudo apt -y install \
build-essential \
libhdf5-dev \
liblapack-dev \
libblas-dev \
libtbb-dev \
libsuperlu-dev \
libeigen3-dev;
- name: Build and Install
run: |
cmake -E make_directory "${BUILD_DIR}"
cmake -E make_directory "${INSTALL_PREFIX}"
cd "${BUILD_DIR}"
cmake -DCMAKE_INSTALL_PREFIX="${INSTALL_PREFIX}" -DCMAKE_BUILD_TYPE="${BUILD_TYPE}" "${SRC_DIR}" -DENABLE_TESTS=ON -DNUM_MAX_AD_DIRS=1320
make install -j$(nproc)
- name: Check if it runs
run: |
export LD_LIBRARY_PATH=${INSTALL_PREFIX}/lib:$LD_LIBRARY_PATH
${INSTALL_PREFIX}/bin/cadet-cli --version || true
${INSTALL_PREFIX}/bin/createLWE
${INSTALL_PREFIX}/bin/cadet-cli LWE.h5 || true
- name: Run tests
run: |
${BUILD_DIR}/test/testRunner [ReleaseCI]
2 changes: 1 addition & 1 deletion doc/developer_guide/release_new_version.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Release checklist

- The release tests contain extensive testing that is not included in our CI, such as EOC tests.
Running these tests might take a while and this should be done on the server.
- Some tests are implemented in CADET-Core, and can be run with the [ReleaseCI] flag.
- Some tests are implemented in CADET-Core with a `ReleaseCI` flag, which should be run by executing the ci workflow.
- More tests are implemented in Python, the code can be found in `CADET-Verification <https://github.com/cadet/CADET-Verification>`_
Compare the results with the previous run.
The release process can only be continued if the results are reasonable.
Expand Down
2 changes: 1 addition & 1 deletion doc/developer_guide/testing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ This way, we make sure that ongoing CADET-Core development doesnt break the mode
**4. Add EOC tests to CADET-Core (optional):**
These tests should be part of the paper publication which introduces the new model implemented in CADET-Core and can also be included in the CADET-Core tests.
Verifying the experimental order of convergence (EOC) is widely considered the most rigorous and best scientific practice in model and method validation, which is why we recommend including the EOC tables in your publication.
The convergence tests should not be added to the standard CI but only be rerun on release, i.e. by adding the [releaseCI] flag.
The convergence tests should not be added to the standard CI but only be rerun on release, i.e. by adding the [ReleaseCI] flag.
Details on how to compute EOC tables can be found elsewhere, please also refer to the already implemented EOC tests in CADET-Verification.

Manufactured solution
Expand Down
5 changes: 3 additions & 2 deletions test/GeneralRateModel.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,8 @@ TEST_CASE("GRM numerical Benchmark with parameter sensitivities for SMA LWE case
cadet::test::column::testReferenceBenchmark(modelFilePath, refFilePath, "000", absTol, relTol, disc, true);
}

TEST_CASE("GRM numerical EOC Benchmark with parameter sensitivities for linear case", "[GRM],[FV],[releaseCI],[EOC],[EOC_GRM_FV]")
// Note that more extensive EOC tests are now part of CADET-Verification
TEST_CASE("GRM numerical EOC Benchmark with parameter sensitivities for linear case", "[GRM],[FV],[EOC],[EOC_GRM_FV]")
{
const std::string& modelFilePath = std::string("/data/model_GRM_dynLin_1comp_benchmark1.json");
const std::string& refFilePath = std::string("/data/ref_GRM_dynLin_1comp_sensbenchmark1_FV_Z1024parZ128.h5");
Expand All @@ -93,7 +94,7 @@ TEST_CASE("GRM numerical EOC Benchmark with parameter sensitivities for linear c
cadet::test::column::testEOCReferenceBenchmark(modelFilePath, refFilePath, convFilePath, "001", absTol, relTol, 3, disc, true);
}

TEST_CASE("GRM numerical EOC Benchmark with parameter sensitivities for SMA LWE case", "[GRM],[FV],[releaseCI],[EOC],[EOC_GRM_FV]")
TEST_CASE("GRM numerical EOC Benchmark with parameter sensitivities for SMA LWE case", "[GRM],[FV],[EOC],[EOC_GRM_FV]")
{
const std::string& modelFilePath = std::string("/data/model_GRM_reqSMA_4comp_benchmark1.json");
const std::string& refFilePath = std::string("/data/ref_GRM_reqSMA_4comp_sensbenchmark1_FV_Z512parZ64.h5");
Expand Down
4 changes: 2 additions & 2 deletions test/LumpedRateModelWithPores.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ TEST_CASE("LRMP numerical Benchmark with parameter sensitivities for SMA LWE cas
cadet::test::column::testReferenceBenchmark(modelFilePath, refFilePath, "000", absTol, relTol, disc, true);
}

TEST_CASE("LRMP numerical EOC Benchmark with parameter sensitivities for linear case", "[releaseCI],[EOC],[EOC_LRMP_FV]")
TEST_CASE("LRMP numerical EOC Benchmark with parameter sensitivities for linear case", "[LRMP],[FV],[EOC],[EOC_LRMP_FV]")
{
const std::string& modelFilePath = std::string("/data/model_LRMP_dynLin_1comp_benchmark1.json");
const std::string& refFilePath = std::string("/data/ref_LRMP_dynLin_1comp_sensbenchmark1_FV_Z32768.h5");
Expand All @@ -93,7 +93,7 @@ TEST_CASE("LRMP numerical EOC Benchmark with parameter sensitivities for linear
cadet::test::column::testEOCReferenceBenchmark(modelFilePath, refFilePath, convFilePath, "001", absTol, relTol, 4, disc, true);
}

TEST_CASE("LRMP numerical EOC Benchmark with parameter sensitivities for SMA LWE case", "[releaseCI],[EOC],[EOC_LRMP_FV]")
TEST_CASE("LRMP numerical EOC Benchmark with parameter sensitivities for SMA LWE case", "[LRMP],[FV],[EOC],[EOC_LRMP_FV]")
{
const std::string& modelFilePath = std::string("/data/model_LRMP_reqSMA_4comp_benchmark1.json");
const std::string& refFilePath = std::string("/data/ref_LRMP_reqSMA_4comp_sensbenchmark1_FV_Z2048.h5");
Expand Down
4 changes: 2 additions & 2 deletions test/LumpedRateModelWithoutPores.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ TEST_CASE("LRM numerical Benchmark with parameter sensitivities for SMA LWE case
cadet::test::column::testReferenceBenchmark(modelFilePath, refFilePath, "000", absTol, relTol, disc, true);
}

TEST_CASE("LRM numerical EOC Benchmark with parameter sensitivities for linear case", "[releaseCI],[EOC],[EOC_LRM_FV]")
TEST_CASE("LRM numerical EOC Benchmark with parameter sensitivities for linear case", "[LRM],[FV],[EOC],[EOC_LRM_FV]")
{
const std::string& modelFilePath = std::string("/data/model_LRM_dynLin_1comp_benchmark1.json");
const std::string& refFilePath = std::string("/data/ref_LRM_dynLin_1comp_sensbenchmark1_FV_Z131072.h5");
Expand All @@ -92,7 +92,7 @@ TEST_CASE("LRM numerical EOC Benchmark with parameter sensitivities for linear c
cadet::test::column::testEOCReferenceBenchmark(modelFilePath, refFilePath, convFilePath, "001", absTol, relTol, 4, disc, true);
}

TEST_CASE("LRM numerical EOC Benchmark with parameter sensitivities for SMA LWE case", "[releaseCI],[EOC],[EOC_LRM_FV]")
TEST_CASE("LRM numerical EOC Benchmark with parameter sensitivities for SMA LWE case", "[LRM],[FV],[EOC],[EOC_LRM_FV]")
{
const std::string& modelFilePath = std::string("/data/model_LRM_reqSMA_4comp_benchmark1.json");
const std::string& refFilePath = std::string("/data/ref_LRM_reqSMA_4comp_sensbenchmark1_FV_Z4096.h5");
Expand Down
2 changes: 2 additions & 0 deletions test/UnitOperationTests.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,8 @@ namespace unitoperation
cadet::IUnitOperation* const unitAD = createAndConfigureUnit(jpp, *mb);
unitAD->useAnalyticJacobian(false);

REQUIRE(unitAD->requiredADdirs() <= cadet::ad::getMaxDirections()); // this explicit check is required in this test but is automatically performed in simulations

cadet::active* adRes = new cadet::active[unitAD->numDofs()];
cadet::active* adY = new cadet::active[unitAD->numDofs()];

Expand Down

0 comments on commit 8fc5a7b

Please sign in to comment.