-
Notifications
You must be signed in to change notification settings - Fork 134
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Move info about testing from README to testing.md (#616)
* Update README.md * move testing info to testing.md
- Loading branch information
Showing
2 changed files
with
108 additions
and
103 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,107 @@ | ||
| # Testing | ||
|
|
||
| To increase confidence in the correctness of the Interchain Security code we consider various testing approaches. | ||
|
|
||
| ## Unit Tests | ||
|
|
||
| Unit tests are useful for simple standalone functionality, and CRUD operations. Unit tests should use golang's standard testing package, and be defined in files formatted as ```<file being tested>_test.go``` in the same directory as the file being tested, following standard conventions. | ||
|
|
||
| [Mocked external keepers](./testutil/keeper/mocks.go) (implemented with [gomock](https://github.com/golang/mock)) are available for testing code that briefly interacts with external modules, but still only a single function/method relevant to ccv, and a single chain. Ie. do not use mocked external keepers to test the integration of the ccv module with external modules, or integration between consumer and provider. | ||
|
|
||
| ## End to End (e2e) Tests | ||
|
|
||
| [e2e-tests](./tests/e2e/) utilize the [IBC Testing Package](https://github.com/cosmos/ibc-go/tree/main/testing), and test functionality that is wider in scope than a unit test, but still able to be validated in-memory. Ie. code where advancing blocks would be useful, simulated handshakes, simulated packet relays, etc. | ||
|
|
||
| To run e2e tests against your own consumer/provider implementations, use [instance_test.go](./tests/e2e/instance_test.go) as an example. All you'll need to do is make sure your applications implement the necessary interfaces defined in [interfaces.go](./testutil/e2e/interfaces.go), pattern match [specific_setup.go](./testutil/ibc_testing/specific_setup.go), then pass in the appropriate types and parameters to the suite, as is done in `instance_test.go` for the dummy provider/consumer implementations. | ||
|
|
||
| ## Differential Tests (WIP) | ||
|
|
||
| Similar to e2e tests, but they compare the system state to an expected state generated from a model implementation. | ||
|
|
||
| ## Integration Tests | ||
|
|
||
| [Integration tests](./tests/integration/) run true consumer and provider chain binaries within a docker container and are relevant to the highest level of functionality. Integration tests use queries/transactions invoked from CLI to drive and validate the code. | ||
|
|
||
| ## Running Tests | ||
| Tests can be run using `make`: | ||
|
|
||
| ```bash | ||
| # run unit, e2e, diff, and integration tests | ||
| make test | ||
|
|
||
| # run unit and e2e tests - prefer this for local development | ||
| make test-short | ||
|
|
||
| # run difference tests | ||
| make test-diff | ||
|
|
||
| # run integration tests | ||
| make test-integration | ||
|
|
||
| # equivalent to make test with caching disabled | ||
| make test-no-cache | ||
| ``` | ||
|
|
||
| Alternatively you can run tests using `go test`: | ||
| ```bash | ||
| # run all unit, e2e, and diff tests using go | ||
| go test ./... | ||
| # run all unit, e2e, and diff tests with verbose output | ||
| go test -v ./.. | ||
| # run all unit, e2e, and diff tests with coverage stats | ||
| go test -coverpkg=./x/... -coverprofile=coverage.out ./... | ||
| # run a single unit test | ||
| go test -run <unit-test-name> path/to/package | ||
| # example: run a single unit test | ||
| go test -run TestSlashAcks ./x/ccv/provider/keeper | ||
| # run a single e2e test | ||
| go test -run <test-suite-name>/<test-name> ./... | ||
| # example: run a single e2e test | ||
| go test -run TestProviderTestSuite/TestPacketRoundtrip ./... | ||
| # run all integration tests | ||
| go run ./tests/integration/... | ||
| # run all integration tests with a local cosmos sdk | ||
| go run ./tests/integration/... --local-sdk-path "/Users/bob/Documents/cosmos-sdk/" | ||
| # run golang native fuzz tests (https://go.dev/doc/tutorial/fuzz) | ||
| go test -fuzz=<regex-to-match-test-name> | ||
| # run verbose integration tests | ||
| go run ./tests/integration/... --local-sdk-path "/Users/bob/Documents/cosmos-sdk/" --verbose | ||
| ``` | ||
|
|
||
| ## Linters and Static Analysis | ||
|
|
||
| Several analyzers are used on the code including [CodeQL](https://codeql.github.com/), [SonarCloud](https://sonarcloud.io/), [golangci-lint](https://golangci-lint.run/) and [gosec](https://github.com/securego/gosec). Some of these are run on github when committing to PRs ect, but some tools are also applicable locally, and are built into golang. | ||
|
|
||
| ```bash | ||
| # gofmt to format and simplify code (https://pkg.go.dev/cmd/gofmt) | ||
| gofmt -w -s -e . | ||
| # go vet to search for suspicious code (https://pkg.go.dev/cmd/vet) | ||
| go vet ./... | ||
| ``` | ||
|
|
||
| Some useful tools are included in the repository using [pre-commit](https://pre-commit.com/hooks.html). pre-commit lets you run developer tools either on every git commit, or manually with `pre-commit run --all-files`. See the [config](./.pre-commit-config.yaml) for details. In this repo the hooks are not installed to git, as that can be cumbersome, but it is still possible to benefit from them. | ||
|
|
||
| ```bash | ||
| ## Prerequisites | ||
|
|
||
| # pre-commit | ||
| brew install pre-commit | ||
| # goimports (https://pkg.go.dev/golang.org/x/tools/cmd/goimports) | ||
| go install golang.org/x/tools/cmd/goimports@latest | ||
| # gocyclo (https://github.com/fzipp/gocyclo) | ||
| go install github.com/fzipp/gocyclo/cmd/gocyclo@latest | ||
| # go-critic https://github.com/go-critic/go-critic | ||
| go install github.com/go-critic/go-critic/cmd/gocritic@latest | ||
|
|
||
| ## Run the tools | ||
|
|
||
| pre-commit run --all-files | ||
| ``` | ||
|
|
||
| ## Debugging | ||
|
|
||
| If using VSCode, see [vscode-go/wiki/debugging](https://github.com/golang/vscode-go/wiki/debugging) to debug unit tests or go binaries. | ||
|
|
||
| ## More | ||
|
|
||
| More instructions will be added soon, in time for the testnet. |