From 027d8069d88854dcfcb27d4db2eae6cb4b4bc319 Mon Sep 17 00:00:00 2001 From: civsiv Date: Tue, 19 Mar 2024 16:32:01 +0000 Subject: [PATCH] feat: add workflows/README.md to elaborate on GitHub workflows (#210) * feat: add workflows/README.md to elaborate on GitHub workflows * feat: fix CI by upgrading Node version (#220) * move most of the detail into the workflows * Update .github/workflows/README.md Co-authored-by: Luke Winship * Update .github/workflows/README.md Co-authored-by: Luke Winship * Update .github/workflows/README.md Co-authored-by: Luke Winship * Update .github/workflows/README.md Co-authored-by: Luke Winship * review comments * fix conflict --------- Co-authored-by: Luke Winship --- .github/workflows/README.md | 69 ++++++++++++++++++ .github/workflows/openactive-test-suite.yml | 79 ++++++++++++++++++++- 2 files changed, 147 insertions(+), 1 deletion(-) create mode 100644 .github/workflows/README.md diff --git a/.github/workflows/README.md b/.github/workflows/README.md new file mode 100644 index 00000000..d7fd3702 --- /dev/null +++ b/.github/workflows/README.md @@ -0,0 +1,69 @@ +# CI Workflows +This project contains two GitHub Action workflows that are performed following various triggers. For more information on Github Actions see [here](https://docs.github.com/en/actions). + +## openactive-test-suite.yml + + +This workflow defines the following jobs: +- **test-server** +- **test-fake-database** +- **core** +- **framework** +- **deploy-reference-implementation** +- **publish-server** +- **publish-fake-database** + +### test-server job +This job builds and runs the `OpenActive.Server.NET.Tests` project i.e. it runs unit tests against `OpenActive.Server.NET`. + +### test-fake-database job +This job builds and runs the `Fakes/OpenActive.FakeDatabase.NET.Tests` project i.e. it runs unit tests against `Fakes/OpenActive.FakeDatabase.NET`. + +### core job +This job uses Test Suite to run a matrix of tests against the `OpenActive.Server.NET` project. + +### framework job +This job is very similar to the above but builds `OpenActive.Server.NET` using .NET Framework and runs Test Suite against it. + + +### deploy-reference-implementation job +This job deploys the `master` branch of Reference Implementation Booking Server ([see here](../../Examples/BookingSystem.AspNetCore/README.md)) and Identity Server Reference Implementation ([see here](../../Examples/BookingSystem.AspNetCore.IdentityServer/README.md)) to two Azure webapps + + +### publish-server job +This job publishes the OpenActive.Server.NET project to Nuget. It then creates a release in the GitHub repository. + + +### publish-fake-database job +Very similar to the above `publish-server` job, this job publishes the Fake Database project within OpenActive.Server.NET to Nuget. + +## create-dependencies-pr.yaml + +This workflow creates a pull request to OpenActive.Server.NET's OpenActive (OA) dependencies if there has been a change in one of those projects. This therefore keeps Server.NET up-to-date with the rest of the OA ecosystem. + +This workflow can be triggered in two ways: + +- **workflow_dispatch**: This allows you to manually trigger the workflow through the GitHub Actions UI. + +- **repository_dispatch**: This allows the workflow to be triggered when a custom repository dispatch event is sent. It listens for dispatch events from two OA repos: `OpenActive.NET` and `OpenActive.DatasetSite.NET`. For more information on repository dispatch events [see here](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#repository_dispatch) + +This workflow defines one job: **generate**. + +### generate job +The main steps of this job are as follows: +- **Update OpenActive.NET to latest version in OpenActive.Server.NET**: This step updates the Server.NET solution with the latest OpenActive.NET library. + +- **Update OpenActive.DatasetSite.NET to latest version in OpenActive.Server.NET**: Similar to the previous step, this updates Server.NET with the latest OpenActive.DatasetSite.NET. + +- **Update OpenActive.NET to latest version in OpenActive.FakeDatabase.NET**: Similar to the previous step, this updates FakeDatabase.NET with the latest OpenActive.NET. + +- **Create Pull Request**: This step uses the peter-evans/create-pull-request action to create a pull request. It performs several actions which are relatively self explanatory: + + - Sets up the access token needed for making changes to the repository. + - Specifies the path to the repository containing the changes. + - Defines a commit message, committer, author, branch name, and other PR-related details. + - Provides a title and body for the pull request, explaining the purpose of the update. + - Adds labels to the pull request (`automated pr`). + - Marks the pull request as not a draft. + +- **Check outputs** prints information about the created pull request, including the pull request number and URL, to the console. diff --git a/.github/workflows/openactive-test-suite.yml b/.github/workflows/openactive-test-suite.yml index 891d1bc2..e74348af 100644 --- a/.github/workflows/openactive-test-suite.yml +++ b/.github/workflows/openactive-test-suite.yml @@ -1,4 +1,5 @@ name: Ref Impl +# This workflow is triggered when there is a push or a pull request to the master branch. on: push: branches: [ master ] @@ -9,56 +10,92 @@ jobs: test-server: runs-on: ubuntu-latest steps: + # Checks out this project ie OpenActive.Server.NET - name: Checkout uses: actions/checkout@v2 + + # Uses the actions/setup-dotnet action to set up the .NET Core runtime environment with version 3.1.419 - name: Setup .NET Core 3.1.419 uses: actions/setup-dotnet@v1 with: dotnet-version: 3.1.419 + + # Runs the `dotnet build` command to build the "OpenActive.Server.NET.Tests" project and its dependencies. It uses + # the `--configuration Release` flag to build in release mode, which results in optimized code generation - name: Build OpenActive.Server.NET.Tests and dependencies run: dotnet build ./OpenActive.Server.NET.Tests/OpenActive.Server.NET.Tests.csproj --configuration Release + + # Runs the tests for the "OpenActive.Server.NET.Tests" project using the `dotnet test` command. It runs the unit + # tests and uses the `--configuration Release` flag for release mode, `--no-build` to skip rebuilding the project + # since it was built in the previous step, and `--verbosity normal` for the level of detail in the test output - name: Run OpenActive.Server.NET.Tests run: dotnet test ./OpenActive.Server.NET.Tests/OpenActive.Server.NET.Tests.csproj --configuration Release --no-build --verbosity normal test-fake-database: runs-on: ubuntu-latest steps: + # Checks out this project ie OpenActive.Server.NET - name: Checkout uses: actions/checkout@v2 + + # Uses the actions/setup-dotnet action to set up the .NET Core runtime environment with version 3.1.419 - name: Setup .NET Core 3.1.419 uses: actions/setup-dotnet@v1 with: dotnet-version: 3.1.419 + + # Runs the `dotnet build` command to build the "OpenActive.FakeDatabase.NET.Tests" project and its dependencies. + # It uses the `--configuration Release` flag to build in release mode, which results in optimized code generation - name: Build OpenActive.FakeDatabase.NET.Tests run: dotnet build ./Fakes/OpenActive.FakeDatabase.NET.Tests/OpenActive.FakeDatabase.NET.Tests.csproj --configuration Release + + # Runs the tests for the "OpenActive.FakeDatabase.NET.Tests" project using the `dotnet test` command. It runs the + # unit tests and uses the `--configuration Release` flag for release mode, `--no-build` to skip rebuilding the + # project since it was built in the previous step, and `--verbosity normal` for the level of detail in the test + # output - name: Run OpenActive.FakeDatabase.NET.Tests run: dotnet test ./Fakes/OpenActive.FakeDatabase.NET.Tests/OpenActive.FakeDatabase.NET.Tests.csproj --configuration Release --no-build --verbosity normal core: + # Specifies that this job depends on the successful completion of two other jobs: "test-server" and "test-fake-database" needs: - test-server - test-fake-database runs-on: ubuntu-latest + + # Defines a matrix strategy for running tests with different combinations of parameters, allowing for parallel test + # runs with various configurations. For more information about these parameters, see `.github/README.md` in Test + # Suite strategy: fail-fast: false matrix: mode: ['random', 'controlled'] profile: ['all-features', 'single-seller', 'no-payment-reconciliation', 'no-auth', 'no-tax-calculation', 'prepayment-always-required', 'facilityuse-has-slots'] steps: + + # checks out the `OpenActive.Server.NET` project's code repository and places it in a directory named "server" - name: Checkout OpenActive.Server.NET uses: actions/checkout@v2 with: path: server + + # checks if the workflow is triggered by a branch starting with `coverage/`. If so, it sets an output variable named + # `mirror_ref` with the branch name, allowing later steps to use it - name: Use matching coverage/* branch ${{ github.head_ref }} in OpenActive Test Suite if: ${{ startsWith(github.head_ref, 'coverage/') }} id: refs run: echo "::set-output name=mirror_ref::${{ github.head_ref }}" + + # checks out Test Suite and places it in a directory named "tests." The branch used for checkout is determined by + # the `mirror_ref` set in the previous step - name: Checkout OpenActive Test Suite ${{ steps.refs.outputs.mirror_ref }} uses: actions/checkout@v2 with: repository: openactive/openactive-test-suite ref: ${{ steps.refs.outputs.mirror_ref }} path: tests + + # sets up the .NET Core SDK version 3.1.419 in the job's environment - name: Setup .NET Core SDK 3.1.419 uses: actions/setup-dotnet@v1 with: @@ -67,26 +104,44 @@ jobs: uses: actions/setup-node@v1 with: node-version: 18.17.1 + + # runs `dotnet restore` to install the dependencies for the "OpenActive.Server.NET" project. It is conditional and + # depends on the "profile" value not being 'no-auth' or 'single-seller' - name: Install OpenActive.Server.NET dependencies if: ${{ matrix.profile != 'no-auth' && matrix.profile != 'single-seller' }} run: dotnet restore ./server/ + + # builds `BookingSystem.AspNetCore.IdentityServer` if the `profile` is not `no-auth` or `single-seller` as it is not + # needed for these profiles - name: Build .NET Core Authentication Authority Reference Implementation if: ${{ matrix.profile != 'no-auth' && matrix.profile != 'single-seller' }} run: dotnet build ./server/Examples/BookingSystem.AspNetCore.IdentityServer/BookingSystem.AspNetCore.IdentityServer.csproj --configuration Release --no-restore + + # starts IdentityServer if the profiles are relevant (see above) - name: Start .NET Core Authentication Authority Reference Implementation if: ${{ matrix.profile != 'no-auth' && matrix.profile != 'single-seller' }} run: | dotnet run --no-launch-profile --project ./server/Examples/BookingSystem.AspNetCore.IdentityServer/BookingSystem.AspNetCore.IdentityServer.csproj --configuration Release --no-build & + + # builds Reference Implementation booking server - name: Build .NET Core Booking Server Reference Implementation run: dotnet build ./server/Examples/BookingSystem.AspNetCore/BookingSystem.AspNetCore.csproj --configuration Release ${{ matrix.profile != 'no-auth' && matrix.profile != 'single-seller' && '--no-restore' || '' }} + + # starts Reference Implementation booking server - name: Start .NET Core Booking Server Reference Implementation run: | dotnet run --no-launch-profile --project ./server/Examples/BookingSystem.AspNetCore/BookingSystem.AspNetCore.csproj --configuration Release --no-build & env: ASPNETCORE_ENVIRONMENT: ${{ matrix.profile }} + + # runs `npm install` to install the JavaScript dependencies for Test Suite, which is located in the "tests" + # directory - name: Install OpenActive Test Suite run: npm install working-directory: tests + + # runs the OpenActive integration tests using `npm start` and specifies various environment variables based on the + # `mode` and `profile` matrix variables specified above - name: Run OpenActive Integration Tests in ${{ matrix.mode }} mode run: npm start env: @@ -96,12 +151,18 @@ jobs: NODE_ENV: .example.${{ matrix.profile }} NODE_APP_INSTANCE: ci working-directory: tests + + # uploads the test output as an artifact, which can be used for reference or debugging later - name: Upload test output for ${{ matrix.mode }} mode as artifact uses: actions/upload-artifact@v2 if: ${{ success() || failure() }} with: name: core.${{ matrix.mode }}.${{ matrix.profile }} path: ./tests/output/ + + # deploys a conformance certificate to Azure Blob Storage, but it is conditional and only runs when the GitHub + # branch is `master` and the `profile` is 'all-features' and the `mode` is 'controlled'. For more information on + # Coformance Certificates, see `packages/openactive-integration-tests/test/certification/README.md` in Test Suite - name: Deploy conformance certificate to Azure Blob Storage (master branch for 'all-features' profile in controlled mode only) uses: bacongobbler/azure-blob-storage-upload@v1.2.0 if: ${{ github.ref == 'refs/heads/master' }} @@ -111,11 +172,15 @@ jobs: connection_string: ${{ secrets.CONFORMANCE_CERTIFICATE_BLOB_STORAGE_CONNECTION_STRING }} sync: false + # This job is very similar to the `core` job explained above. However because it runs on .NET Framework as opposed to + # .NET Core, there are some differences. Due to the similarity, only the differences are outlined below. framework: needs: - test-server - test-fake-database + # runs on an Windows 2019 virtual machine, not an Ubuntu machine runs-on: windows-2019 + strategy: fail-fast: false matrix: @@ -140,6 +205,9 @@ jobs: uses: actions/setup-node@v1 with: node-version: 18.17.1 + + # Building and deploying Framework is done using `msbuild` as opposed to `dotnet build`/`dotnet run`. It is started + # using IIS Express - name: Setup MSBuild path uses: microsoft/setup-msbuild@v1.0.2 - name: Setup NuGet @@ -172,6 +240,8 @@ jobs: name: framework.${{ matrix.mode }}.${{ matrix.profile }} path: ./tests/output/ + # This job uses many of the steps and setup used above. Because of this, only important or different parts have been + # highlighted. deploy-reference-implementation: # Master branch only if: ${{ github.ref == 'refs/heads/master' }} @@ -197,7 +267,9 @@ jobs: run: dotnet build ./Examples/BookingSystem.AspNetCore.IdentityServer/BookingSystem.AspNetCore.IdentityServer.csproj --configuration Release --no-restore - name: Publish OpenActive.Server.NET Authentication Authority Reference Implementation run: dotnet publish ./Examples/BookingSystem.AspNetCore.IdentityServer/BookingSystem.AspNetCore.IdentityServer.csproj --configuration Release --no-build --output './web-app-package/BookingSystem.AspNetCore.IdentityServer' - # Deploy to Azure Web apps + # Deploy to Azure Web apps + # deploys the `master` branch of Reference Implementation Booking Server and Identity + # Server Reference Implementation to two Azure webapps - name: 'Deploy Booking Server Reference Implementation using publish profile credentials' uses: azure/webapps-deploy@v2 with: @@ -211,6 +283,8 @@ jobs: publish-profile: ${{ secrets.AZURE_WEBAPP_AUTH_PUBLISH_PROFILE }} # Define secret variable in repository settings as per action documentation package: './web-app-package/BookingSystem.AspNetCore.IdentityServer' + # This job uses many of the steps and setup used above. Because of this, only important or different parts have been + # highlighted. publish-server: # Master branch only if: ${{ github.ref == 'refs/heads/master' }} @@ -240,6 +314,9 @@ jobs: run: dotnet test ./OpenActive.Server.NET.Tests/OpenActive.Server.NET.Tests.csproj --configuration Release --no-restore --verbosity normal - name: Pack run: dotnet pack ./OpenActive.Server.NET/OpenActive.Server.NET.csproj --configuration Release --include-source --no-build -p:IncludeSymbols=true -p:SymbolPackageFormat=snupkg + + # publishes the `master` branch of OpenActive.Server.NET to Nuget. It then creates a release in the GitHub + # repository - name: Push to Nuget if: "! contains(toJSON(github.event.commits.*.message), '[no-release]')" run: dotnet nuget push "./OpenActive.Server.NET/**/*.nupkg" -k ${{secrets.NUGET_API_KEY}} --skip-duplicate -s https://api.nuget.org/v3/index.json