feat: passthrough router

fix: adapt passthrough router fix: docs and hash chore: rm unused test Release v0.11.3 (#1886) * Switch to full wasm execution * feat: use correct encoded size. * fix: lint * refactor: reduce dev session duration from 6h to 2min * chore: update centrifuge weights * chore: update dev weights * chore: update altair weights * chore: update frame_system weights * fmt * fix: re-enable frame_system benches * chore: bump spec version * fmt: revert using latest nightly * v0.11.3-rc1: revert checkmetadata ext * feat: `CheckMetadataHash` extension (#1865) * set dependency versions * upgrade libs/* * upgrade pallet-anchors * upgrade pallet-fees * upgrade pallet-bridge * migrate simple pallets * migrate pallet-order-book * migrated collator-allowlist & swaps * upgrade rewards * upgraded pallet-mux * upgrade transfer-allowlist * fix hold reason in restricted tokens * simplify set_balance removing the holding part * upgrade restricted-xtokens * upgrade some pallets * upgrade pallet-ethereum-transaction * upgrade pallet-loans * upgrade pool-system * upgrade pool-fees * upgrade pool-registry * upgrade liquidity-pools stuffs * avoid duplicated polkadot-sdk repos * minor fixes * upgraded runtime-common * CfgLocation to RestrictedTransferLocation * restricted tokens with NativeIndex for native fungibles * rename dependency * upgraded development-runtime * fix partially benchmarks * fix benchmarks * overpass xcmp-queue integrity tests * minor coments * upgrade altair & centrifuge * remove some benchmarking pallets that are not needed * fix runtime upgrades * upgrade integration-test proc * upgrade integration-tests framework * upgraded all tests except liquidity pools * 99% upgraded liquidity-pools tests * fix lookup * cargo fmt * taplo fmt * using nightly cargo in CI * restore set_balance as it was * allow nightly support in CI * use restricted_tokens again to fetch from investement portfolio * Install rust-src for docs * fix tests * remove unused restricted tokens * fix block rewards * fix WrongWitness for some tests in IT * fix liquidity-pools * minor fixes * fix clippy * remove unneeded tests * feat: Update client to Polkadot v1.7.2 (#1844) * wip: update client * chore: update crate versions * chore: update anchor rpc api * chore: remove rewards, pools rpc * chore: compile with development runtime * fix: client for all runtimes * fix: build spec * feat: update relay docker images * fix: apply deprecation of export genesis state to scripts * fmt: taplo * refactor: use xcm v4 sugar * fix: revert tmp change in local para run * refactor: simplify xcm v4 locations in chain spec * cargo fmt * fix clippy * feat: Polkadot v1.7.2 migrations (#1849) * feat: add hold reason migration * feat: centrifuge migrations * feat: altair migrations * feat: dev + demo migrations * fix: clippy * fix: build * fmt: fix using nightly * last William iteration review * increase passed blocks * use rococo instead of polkadot-test-runtime * fix tests * remove line * dirty fix to fix Hrmp test issue * use default weights for treasury * remove getrandom unused dep * upgrade to last polkadot-sdk version * feat: `CheckMetadataHash` extension * fix it (#1866) * fmt: taplo * refactor: signed ext order * fix: signed ext order for ITs * IT: more support for router tests (#1885) * move routers to its own module * remove outdated markers * for all runtimes * remove previous tests * minor fixes * v0.11.2 rc * panic if event is not found in the expected blocks (#1880) * fix: ci * remove generic module (#1887) * revert check metadata hash disable * fix: disable metadata hash check for integration tests --------- Co-authored-by: lemunozm <[email protected]> --------- Co-authored-by: Bastian Köcher <[email protected]> Co-authored-by: Frederik Gartenmeister <[email protected]> Co-authored-by: lemunozm <[email protected]> Refactor: Decoupling routers from gateway (#1891) * decoupling routers from gateway. Use Vec<u8> * gateway adapted * fix imports * move test msg to cfg-traits * fix imports feat: Open Gov for Development and Altair Runtimes (#1828) * chore: add OpenGov pallets to root toml * feat: add OpenGov tracks to runtime common * feat: add technical committee migration tc: fix clippy * refactor: move origins from primitives to common common: fix typo common: fix test after HalfOfCouncil refactor * feat: add OpenGov to Development fix: complete Development OpenGov development: finalize opengov development: add proxy calls development: overwrite missing origins dev: fix treasury spender dev: remove crowdloan pallets dev: taplo fmt * feat: add OpenGov to Altair fix: complete Altair OpenGov fix: add missing technical fellowship weights Altair altair: finalize opengov altair: rename track tests altair: gather imports altair: remove crowdloan pallets altair: taplo fmt * feat: add OpenGov to Centrifuge chain fix: centrifuge toml chore: minor centrifuge chain improvements refactor: Centrifuge OpenGov centrifuge: finalize opengov centrifuge: fix cargo tml cfg: fix clippy test * Revert "feat: add OpenGov to Centrifuge chain" This reverts commit 4d78c10. * fix: clippy * tests: fix CouncilCollective import path * fix: technical committee migrations * altair: configure tc migration (incomplete) * dev: configure tc migration * chore: cleanup altair dep diff * chore: remove unused getrandom crate from runtimes * chore: add missing Altair TC members * refactor: move to_ppm and to_percent to utils * Merge remote-tracking branch 'origin/main' into feat/open-gov-2 * fix: issues after rebasing * fmt * fix: imports * feat: v0.12.0 for altair RU Altair: release v0.12.0 (#1896) * fix: altair chain spec * chore: update gov2 weights * chore: update dev OpenGov weights * fix: weight declaration Fix RustDocs deployment & codecov patch checks (#1870) * Fix deploy docs * Make patch checks also informational (not fail) LP Message with custom data format for serialization/deserialization (#1889) * test compiling * fix Domain serialization * fix issues * some reorganizations * using bincode 2 with non-std support * use custom trait * tests passing for lp * remove ENCODED_MSG constant * fix runtimes compilation * cargo fmt & taplo fmt * cargo clippy * minor change * add custom serializer passing all tests * no_std support * minor renames * cargo fmt * cargo clippy * minor comment change * cargo fmt extra Succeed when patch codecov fails (#1900) Liquidity pools: Add UTs to for `update_token_price()` (#1890) * add token price UTs * update IT * fix tests compilation remove some unused code investment-related (#1902) Release v0.13.0 (#1898) * chore: update deps to enable metadata hash check * chore: update srtool + add on-chain-release-build opts * chore: bump spec versions + cleanup migrations * fmt: taplo * ci: fix srtool fmt * ci: apply proper CLI * attempt2 to enable on-chain-release-build * change srtool build for chevdor/strool-actions action * desperate attempts to get it working * override workdir * Try to set permissions wide open before running docker * add RUST_BACKTRACE * revert to our GHA manual process & add enhancements * fix package name * Output information about srtool * Revert "desperate attempts to get it working" This reverts commit a882fd9. * fix: some scripts * fix bad colon on echo command * Fix missing colon and remove docker publish release * upload wasm to release * fix issue with gchr tags * more semicolon issues * unique name for delete untagged * fix delete_untagged * move delete_untagged under workflows * move delete untagged to manual runs * review bash syntax for wasm build * recover cache and limit sanity check build time --------- Co-authored-by: Guillermo Perez <[email protected]> cargo update (#1904) add uts (#1905) IT: Support `#ignore = reason` for `test_runtimes` macro (#1908) * support #ignore = reason for test_runtimes macro * fix clippy ci: disable checks for drafts (#1913) LP: Unitary testing for add_currency (#1912) * port add-currency tests into uts * remove LiquidityPoolsWrappedToken struct * remove unused code * fix compilation issues * fix clippy * fix clippy feat: adatpt to latest version LP: Unitary testing for all pending extrinsics (#1916) * allow and disallow currency tests * schedule_upgrade cancel_upgrade and update_tranche_token_metadata Feat: Adapt proxy settings (#1922) * feat: adapt proxy settings * fix: match pattern Small CI improvements (#1924) LP: Unitary testing for inbound messages (#1927) * tests inbound messages * some cleanings * some adjustements fix: array size fix: clippy + fmt fix: update blake hash fix: compilation Refactor: Transform `TrancheCurrency` type in a tuple (#1926) * remove TrancheCurrency type * fix benchmarks * fix clippy fix: std import KeccakHasher
centrifuge · Jul 25, 2024 · 6183257 · 6183257
1 parent e5c8926
commit 6183257
Show file tree

Hide file tree

Showing 239 changed files with 11,287 additions and 7,211 deletions.
diff --git a/.github/workflows/build-docker.yml b/.github/workflows/build-docker.yml
@@ -86,6 +86,7 @@ jobs:
       - name: Build and push centrifugeio/centrifuge-chain
         uses: docker/build-push-action@2cdde995de11925a030ce8070c3d77a52ffcf1c0 #v5.3.0
         with:
+          provenance: false # Solves untagged containers getting into the GH registry https://github.com/docker/build-push-action/issues/894
           context: .
           file: ./docker/centrifuge-chain/Dockerfile
           build-args: |
@@ -111,14 +112,14 @@ jobs:
           short-description: ${{ github.event.repository.description }}
           enable-url-completion: true
 
-      - name: Update GitHub release
-        if: github.event_name == 'release' && github.event.action == 'released' && matrix.target == 'release'
-        uses: softprops/action-gh-release@v1
-        with:
-          append_body: true
-          body: |
-            **Docker tags (${{ env.NOW }}):**
-               ${{ steps.meta.outputs.tags }}
+      # - name: Update GitHub release
+      #   if: github.event_name == 'release' && github.event.action == 'released' && matrix.target == 'release'
+      #   uses: softprops/action-gh-release@v1
+      #   with:
+      #     append_body: true
+      #     body: |
+      #       **Docker tags (${{ env.NOW }}):**
+      #          ${{ steps.meta.outputs.tags }}
 
       - if: failure()
         name: Check available space after build failed

diff --git a/.github/workflows/build-wasm.yml b/.github/workflows/build-wasm.yml
@@ -18,14 +18,16 @@ jobs:
     # Benchmark (with cache hit)
     # ubuntu-latest 50 min
     # ubuntu-latest-4-cores 34 min
-    runs-on:  ubuntu-latest-4-cores
+    # As this is not a critical PR feedback item, we can run it on free ubuntu-latest
+    runs-on: ubuntu-latest
     strategy:
       matrix:
-        target: [build-runtime]
-        package: [altair-runtime, centrifuge-runtime, development-runtime]
-    name: ${{ matrix.target }}-${{ matrix.package }}
+        chain: [ altair, centrifuge, development ]
+    name: build-runtime-${{ matrix.chain }}
+    env:
+      SRTOOL_IMAGE: "paritytech/srtool:1.77.0-0.15.0"
     steps:
-    # PREP
+      # PREP
       - name: Check out code
         uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b #4.1.4
 
@@ -42,22 +44,13 @@ jobs:
             ./target/srtool
           cache-directories: ./runtime
 
-    # BUILD
-      - name: Run srtool # https://docs.substrate.io/reference/command-line-tools/srtool/
-        run: >
-          docker run --rm  --user root --platform=linux/amd64
-          -e PACKAGE=${{ matrix.package }} -v /home/runner/.cargo:/cargo-home  
-          -v ${{ github.workspace }}:/build paritytech/srtool:1.75.0-0.14.0
-          /srtool/build --app
-
-      # Alternative way of running SRTool that allows for "script-like" execution,
-      # extremely useful to debug:
-
+      # Use this to debug what's going on inside the srtool container
       # - name: Run Docker SRTool
       #   uses: addnab/docker-run-action@4f65fabd2431ebc8d299f8e5a018d79a769ae185 #v3
       #   with:
       #     image: paritytech/srtool:${{ matrix.rust_version }}
-      #     options: --user root -v /home/runner/.cargo:/cargo-home -v ${{ github.workspace }}:/build -e BUILD_OPTS=${{ env.BUILD_OPTS || ''}} -e PACKAGE=${{ matrix.package }}
+      #     options: --user root -v /home/runner/.cargo:/cargo-home -v ${{ github.workspace }}:/build -e BUILD_OPTS=${{ env.BUILD_OPTS || ''}} -e PACKAGE=${{ matrix.chain }}-runtime
+      #     options: --user root -v /home/runner/.cargo:/cargo-home -v ${{ github.workspace }}:/build -e BUILD_OPTS=${{ env.BUILD_OPTS || ''}} -e PACKAGE=${{ matrix.chain }}-runtime
       #     run: |
       #       echo "---- Env VARS ----"
       #       echo "BUILD_OPT=${BUILD_OPT}"
@@ -66,13 +59,47 @@ jobs:
       #       ls -la /cargo-home/
       #       du -sh /cargo-home/*
       #       echo "---- RUNNING BUILD ----"
-      #       /srtool/build --save
-
-      - name: Upload srtool report
-        uses: actions/upload-artifact@26f96dfa697d77e81fd5907df203aa23a56210a8 #v4.3.0
-        with:
-          name: srtool-wasm-report.txt
-          path: ./${{ matrix.package }}-srtool-wasm-report.json
+      #       /srtool/build --app --json -cM   
+
+
+      # Build and setup env vars:
+      - name: Run srtool # https://docs.substrate.io/reference/command-line-tools/srtool/
+        id: srtool_build
+        shell: bash
+        run: |
+          ##"RUNNING SRTOOL"
+          echo ::group::Docker run srtool build ${{ inputs.chain }}
+          CMD="docker run --rm  --user root --platform=linux/amd64
+          -e PROFILE=release -e PACKAGE=${{ matrix.chain }}-runtime -e BUILD_OPTS="--features=on-chain-release-build" 
+          -v /home/runner/.cargo:/cargo-home -v ${{ github.workspace }}:/build 
+          ${{ env.SRTOOL_IMAGE }} /srtool/build --app --json -cM"
+          echo ::debug::build::docker_run $CMD
+          echo ::endgroup
+          # here we keep streaming the progress and fetch the last line for the json result
+          stdbuf -oL $CMD | {
+            while IFS= read -r line
+            do
+              echo ║ $line
+              JSON="$line"
+            done
+            echo "json=$JSON" >> $GITHUB_OUTPUT
+  
+            WASM=`echo $JSON | jq -r .runtimes.compact.wasm`
+            echo "wasm=$WASM" >> $GITHUB_OUTPUT
+
+            Z_WASM=`echo $JSON | jq -r .runtimes.compressed.wasm`
+            echo "wasm_compressed=$Z_WASM" >> $GITHUB_OUTPUT
+          }
+
+      - name: Summary
+        run: |
+          ## JSON output
+          echo "::group::JSON output"
+          PRETTY_JSON=$(echo '${{ steps.srtool_build.outputs.json }}' | jq .)
+          echo '${{ steps.srtool_build.outputs.json }}' | jq . > "${{ matrix.chain }}-srtool-digest.json"
+          echo "JSON Output:"
+          cat "${{ matrix.chain }}-srtool-digest.json"
+          echo "::endgroup"
 
       - id: 'auth'
         if: github.event_name != 'pull_request'
@@ -89,8 +116,34 @@ jobs:
         if: ${{ steps.auth.outcome == 'success' }}
         shell: bash
         run: |
-          runtime_name=$(echo "${{ matrix.package }}" | sed -e "s/-runtime$//" )
-          filename=$(echo "${{ matrix.package }}" | sed -e 's/-/_/g' )
+          ## Publish WASM and JSON summary
+          echo '${{ steps.srtool_build.outputs.json }}' | jq . > "${{ matrix.chain }}-srtool-digest.json"
+          gsutil cp \
+            "${{ steps.srtool_build.wasm_compressed }}" \
+            "gs://centrifuge-wasm-repo/${{ matrix.chain }}/${{ matrix.chain }}-$(git rev-parse --short HEAD).wasm"
+          ## Publish WASM and JSON summary
+          echo '${{ steps.srtool_build.outputs.json }}' | jq . > "${{ matrix.chain }}-srtool-digest.json"
           gsutil cp \
-            ./runtime/${runtime_name}/target/srtool/release/wbuild/${{ matrix.package }}/${filename}.compact.compressed.wasm \
-            gs://centrifuge-wasm-repo/${runtime_name}/${runtime_name}-$(git rev-parse --short HEAD).wasm
+            "${{ steps.srtool_build.wasm_compressed }}" \
+            "gs://centrifuge-wasm-repo/${{ matrix.chain }}/${{ matrix.chain }}-$(git rev-parse --short HEAD).wasm"
+          gsutil cp \
+            "${{ matrix.chain }}-srtool-digest.json" \
+            "gs://centrifuge-wasm-repo/${{ matrix.chain }}/${{ matrix.chain }}-$(git rev-parse --short HEAD)-srtool-digest.json"
+
+      - name: Upload WASM to release tag ${{ github.ref_name }}
+        uses: ncipollo/release-action@a2e71bdd4e7dab70ca26a852f29600c98b33153e # @v1
+        if: github.event_name == 'release'
+        # https://github.com/ncipollo/release-action
+        with:
+          tag: ${{ github.ref_name }}
+          commit: ${{ github.sha }}
+          artifacts: "${{ steps.srtool_build.wasm_compressed }}, ${{ matrix.chain }}-srtool-digest.json"
+          replacesArtifacts: true
+          allowUpdates: true
+          artifactErrorsFailBuild: true
+          omitBodyDuringUpdate: true
+          omitDraftDuringUpdate: true
+          omitNameDuringUpdate: true
+          omitPrereleaseDuringUpdate: true
+          # updateOnlyUnreleased: true # When allowUpdates is enabled, this will fail the action if the release it is updating is not a draft or a prerelease.
+
diff --git a/.github/workflows/codecov.yml b/.github/workflows/codecov.yml
@@ -2,6 +2,10 @@ on:
   push:
     branches: [main]
   pull_request:
+    types: [opened, synchronize, reopened, ready_for_review]
+  pull_request_review:
+    types: [submitted]
+  workflow_dispatch:
 name: Code coverage
 concurrency:
   group: 'codecov-${{ github.event.pull_request.head.label || github.head_ref || github.ref }}'
@@ -11,8 +15,13 @@ permissions:
     contents: read
 jobs:
   codecov:
-    runs-on: ubuntu-latest-8-cores
-    if: github.event.pull_request.draft == false || github.ref == 'refs/heads/main'
+    runs-on: ubuntu-latest-4-cores
+    if: >
+      github.event.pull_request.draft == false ||
+      github.ref == 'refs/heads/main' ||
+      (github.event_name == 'pull_request_review' && github.event.review.state == 'approved') ||
+      github.event.action == 'ready_for_review' ||
+      github.event_name == 'workflow_dispatch'
     steps:
       - uses: actions/checkout@master
 

diff --git a/.github/workflows/delete_untagged.yml b/.github/workflows/delete_untagged.yml
@@ -0,0 +1,45 @@
+name: Delete untagged
+on:
+  workflow_dispatch:
+jobs:
+  delete_untagged:
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      packages: write
+      # id-token: write
+    steps:
+      - name: Delete untagged ghcr
+        uses: Chizkiyahu/delete-untagged-ghcr-action@v3
+        with:
+          # Personal access token (PAT) used to fetch the repository. The PAT is configured
+          # with the local git config, which enables your scripts to run authenticated git
+          # commands. The post-job step removes the PAT.
+          # needs delete:packages permissions
+          # required: true
+          # [Learn more about creating and using encrypted secrets](https://help.github.com/en/actions/automating-your-workflow-with-github-actions/creating-and-using-encrypted-secrets)
+          token: ${{ secrets.GITHUB_TOKEN }}
+          # Repository name or  name with owner
+          # Delete only from repository name
+          # Default: ${{ github.repository }}
+          repository: 'centrifuge-chain'
+          # 'The repository owner name'
+          # Default: ${{ github.repository_owner }}
+          # repository_owner: ''
+          # 'The package names'
+          # Delete only from comma separated package names
+          # required: false
+          package_name: 'centrifuge-chain'
+          # Delete only package versions without tag
+          # required: false
+          # Default: true
+          # choices: true, false
+          untagged_only: true
+          # Except untagged multiplatform packages from deletion
+          # only for untagged_only=true
+          # needs docker installed
+          except_untagged_multiplatform: false
+          # the owner type
+          # required: true
+          # choices: org, user
+          owner_type: 'org'
diff --git a/.github/workflows/deploy-docs.yml b/.github/workflows/deploy-docs.yml
@@ -57,12 +57,18 @@ jobs:
             echo "::warning title=Invalid file permissions automatically fixed::$line"
           done
 
-      - name: upload Docs files
-        uses: actions/upload-artifact@26f96dfa697d77e81fd5907df203aa23a56210a8 #v4.3.0
+      # - name: upload Docs files
+      #   uses: actions/upload-artifact@26f96dfa697d77e81fd5907df203aa23a56210a8 #v4.3.0
+      #   with:
+      #     name: github-pages
+      #     path: ./target/doc
+      #     retention-days: 1
+      # https://github.com/actions/deploy-pages/issues/179
+
+      - name: Upload Docs
+        uses: actions/upload-pages-artifact@v3
         with:
-          name: github-pages
-          path: ./target/doc
-          retention-days: 1
+            path: ./target/doc
 
       - name: Deploy Docs
         # if: github.ref == 'refs/heads/main'

diff --git a/.github/workflows/sanity-checks.yml b/.github/workflows/sanity-checks.yml
@@ -1,5 +1,6 @@
 on:
   pull_request:
+    types: [opened, synchronize, reopened, ready_for_review]    
 name: Sanity checks
 concurrency:
   group: 'tests-${{ github.event.pull_request.head.label || github.head_ref || github.ref }}'
@@ -9,6 +10,8 @@ permissions:
   contents: read
 jobs:
   test-n-lint:
+    if: ${{ !github.event.pull_request.draft }}
+    timeout-minutes: 90
     name: ${{ matrix.target }}
     runs-on: ubuntu-latest-8-cores
     strategy:
@@ -40,11 +43,13 @@ jobs:
           RUSTC_WRAPPER: "sccache"
 
   benchmark-check:
+    if: ${{ !github.event.pull_request.draft }}
+    # timeout-minutes: 90 # <- Not needed as ubunt-latest is free of charge
     name: bench-check-${{ matrix.runtime }}
     runs-on: ubuntu-latest #-4-cores
     strategy:
       matrix:
-        runtime: [ altair, centrifuge ]
+        runtime: [ centrifuge ]
     steps:
       - name: Check out code
         uses: actions/checkout@0ad4b8fadaa221de15dcec353f45205ec38ea70b #4.1.4