You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We should add a small page to the documentation on how to port legacy tests and also how to deal with expected coverage failures from the CI runs.
Expected coverage decrements usually come from the distinct approaches to test case writing that we have in eest versus the tests written in ethereum/tests:
EEST favors dynamic contract creation for each different test vector, whereas ethereum/tests favors creating a single static contract with jumps to different test vectors depending on the transaction input data.
For this reason, there will be normally some lost coverage on JUMP opcodes and also jumpdest analysis because we don't rely on those features (unless of course that is the purpose of the test).
And then explain how to navigate the resulting coverage artifacts, visualize them and also how to bring the obtained information into the PR discussion in order to justify merging the PR with the failing coverage CI.
Also highlight that there could be cases where the coverage contains legitimate drops in coverage along with the expected drops, so we should be aware of this and pay attention.
The text was updated successfully, but these errors were encountered:
marioevz
changed the title
Create documentation guide on how to deal with coverage errors on legacy tests porting PRs
Create documentation guide on how to deal with coverage errors on PRs porting legacy tests
Jan 29, 2025
We should add a small page to the documentation on how to port legacy tests and also how to deal with expected coverage failures from the CI runs.
Expected coverage decrements usually come from the distinct approaches to test case writing that we have in eest versus the tests written in ethereum/tests:
EEST favors dynamic contract creation for each different test vector, whereas ethereum/tests favors creating a single static contract with jumps to different test vectors depending on the transaction input data.
For this reason, there will be normally some lost coverage on
JUMP
opcodes and also jumpdest analysis because we don't rely on those features (unless of course that is the purpose of the test).We could link to an example failing action in the documentation such as: https://github.com/ethereum/execution-spec-tests/actions/runs/13037332959/job/36370897481
And then explain how to navigate the resulting coverage artifacts, visualize them and also how to bring the obtained information into the PR discussion in order to justify merging the PR with the failing coverage CI.
Also highlight that there could be cases where the coverage contains legitimate drops in coverage along with the expected drops, so we should be aware of this and pay attention.
The text was updated successfully, but these errors were encountered: