ci: use openrpc-testgen tool for RPC testing #2799
Conversation
WalkthroughOhayo, sensei! This pull request introduces significant enhancements to the CI workflow by adding a new job named Changes
Possibly related PRs
Suggested reviewers
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Outside diff range and nitpick comments (3)
.github/workflows/release-dispatch.yml (2)
Line range hint
4-9: Ohayo sensei! Consider adding version input pattern validation.The version input could benefit from a pattern constraint to ensure it follows semantic versioning format.
version: description: Version to release required: true type: string + pattern: '^v?\d+\.\d+\.\d+(-[0-9A-Za-z-]+(\.[0-9A-Za-z-]+)*)?(\+[0-9A-Za-z-]+(\.[0-9A-Za-z-]+)*)?$'
Line range hint
31-32: Consider using environment protection rules for the PAT token.The
CREATE_PR_TOKENsecret is used directly. Consider using environment protection rules for better security..github/workflows/ci.yml (1)
105-163: Consider consolidating test suite execution.The test suite execution steps are repetitive. Consider creating a helper script to reduce duplication.
+ # Create test-suite.sh + #!/bin/bash + run_test_suite() { + local port="$1" + local suite="$2" + URLS="http://${KATANA_ADDR}:${port}" + target/release/openrpc-testgen-runner \ + --urls "${URLS}" \ + --paymaster-account-address "${PAYMASTER_ACCOUNT_ADDRESS}" \ + --paymaster-private-key "${PAYMASTER_PRIVATE_KEY}" \ + --udc-address "${UDC_ADDRESS}" \ + --account-class-hash "${ACCOUNT_CLASS_HASH}" \ + --suite "${suite}" + }Then use it in the workflow:
- name: Run Test Suites run: | chmod +x ./test-suite.sh ./test-suite.sh "${KATANA_PORT}" "katana" ./test-suite.sh "${KATANA_OPENRPC_PORT}" "open-rpc" ./test-suite.sh "${KATANA_NO_MINING_PORT}" "katana-no-mining" ./test-suite.sh "${KATANA_NO_FEE_PORT}" "katana-no-fee" ./test-suite.sh "${KATANA_NO_ACCOUNT_VALIDATION_PORT}" "katana-no-account-validation" working-directory: katana-runner
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
.github/workflows/ci.yml(2 hunks).github/workflows/release-dispatch.yml(1 hunks)
🧰 Additional context used
🪛 yamllint (1.35.1)
.github/workflows/ci.yml
[error] 8-8: trailing spaces
(trailing-spaces)
[error] 80-80: trailing spaces
(trailing-spaces)
🪛 Gitleaks (8.21.2)
.github/workflows/ci.yml
40-40: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
🪛 actionlint (1.7.4)
.github/workflows/ci.yml
32-32: label "arc-runner-set" is unknown. available labels are "windows-latest", "windows-latest-8-cores", "windows-2022", "windows-2019", "ubuntu-latest", "ubuntu-latest-4-cores", "ubuntu-latest-8-cores", "ubuntu-latest-16-cores", "ubuntu-24.04", "ubuntu-22.04", "ubuntu-20.04", "macos-latest", "macos-latest-xl", "macos-latest-xlarge", "macos-latest-large", "macos-15-xlarge", "macos-15-large", "macos-15", "macos-14-xl", "macos-14-xlarge", "macos-14-large", "macos-14", "macos-13-xl", "macos-13-xlarge", "macos-13-large", "macos-13", "macos-12-xl", "macos-12-xlarge", "macos-12-large", "macos-12", "self-hosted", "x64", "arm", "arm64", "linux", "macos", "windows". if it is a custom label for self-hosted runner, set list of labels in actionlint.yaml config file
(runner-label)
46-46: the runner of "actions/checkout@v3" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
53-53: the runner of "actions-rs/toolchain@v1" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
58-58: shellcheck reported issue in this script: SC2086:info:2:98: Double quote to prevent globbing and word splitting
(shellcheck)
58-58: shellcheck reported issue in this script: SC2086:info:3:28: Double quote to prevent globbing and word splitting
(shellcheck)
72-72: shellcheck reported issue in this script: SC2086:info:14:3: Double quote to prevent globbing and word splitting
(shellcheck)
90-90: the runner of "actions/checkout@v3" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
🔇 Additional comments (1)
.github/workflows/ci.yml (1)
32-32:
Update runner label configuration.
The arc-runner-set label is not recognized. Consider:
- Adding it to actionlint.yaml configuration
- Or using a standard runner label
🧰 Tools
🪛 actionlint (1.7.4)
32-32: label "arc-runner-set" is unknown. available labels are "windows-latest", "windows-latest-8-cores", "windows-2022", "windows-2019", "ubuntu-latest", "ubuntu-latest-4-cores", "ubuntu-latest-8-cores", "ubuntu-latest-16-cores", "ubuntu-24.04", "ubuntu-22.04", "ubuntu-20.04", "macos-latest", "macos-latest-xl", "macos-latest-xlarge", "macos-latest-large", "macos-15-xlarge", "macos-15-large", "macos-15", "macos-14-xl", "macos-14-xlarge", "macos-14-large", "macos-14", "macos-13-xl", "macos-13-xlarge", "macos-13-large", "macos-13", "macos-12-xl", "macos-12-xlarge", "macos-12-large", "macos-12", "self-hosted", "x64", "arm", "arm64", "linux", "macos", "windows". if it is a custom label for self-hosted runner, set list of labels in actionlint.yaml config file
(runner-label)
| @@ -27,6 +27,142 @@ jobs: | |||
| name: dojo-bins | |||
| path: bins | |||
|
|
|||
| dojo-tests: | |||
There was a problem hiding this comment.
| dojo-tests: | |
| starknet-rpc: |
Wondering if this should be in the same file since we may not want to run this at every CI, only when crates/katana path is actually modified. cc @kariy
| @@ -5,7 +5,7 @@ on: | |||
| branches: | |||
| - main | |||
| pull_request: | |||
|
|
|||
| @@ -27,6 +27,142 @@ jobs: | |||
| name: dojo-bins | |||
| path: bins | |||
|
|
|||
| dojo-tests: | |||
| needs: build | |||
| runs-on: arc-runner-set | |||
There was a problem hiding this comment.
From where this runner set is coming from?
There was a problem hiding this comment.
This one is deployed on our visoft cluster.
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #2799 +/- ##
=======================================
Coverage 55.74% 55.75%
=======================================
Files 439 439
Lines 55571 55571
=======================================
+ Hits 30978 30981 +3
+ Misses 24593 24590 -3 ☔ View full report in Codecov by Sentry. |
Description
This PR adds a new
dojo-testsjob to the CI pipeline (ci.yml), which runs tests for the katana node using theopenrpc-testgentool.Related issue
Tests
The new CI job run the following test suites within the dojo-tests job:
Added to documentation?
Checklist
scripts/prettier.sh,scripts/rust_fmt.sh,scripts/cairo_fmt.sh)scripts/clippy.sh,scripts/docs.sh)Summary by CodeRabbit
New Features
dojo-testsfor comprehensive testing of the Katana application.Improvements