Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: e2e initialization #1486

Merged
merged 19 commits into from
Jan 24, 2024
Merged

test: e2e initialization #1486

merged 19 commits into from
Jan 24, 2024

Conversation

bermuell
Copy link
Contributor

@bermuell bermuell commented Dec 5, 2023

Description

Closes: #1133

Infra work on e2e tests to support testruns against different versions of provider and consumer
Proposed solution allows to define provider version and consumer version to be build and runs local test cases
on the constructed target.
Note: dependeing on the selected versions the tests might fail due to required changes as transformations and not supported checks. A separate PR will be opened targeting compatiblity tests.


Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

  • Included the correct type prefix in the PR title
  • Targeted the correct branch (see PR Targeting)
  • Provided a link to the relevant issue or specification
  • Reviewed "Files changed" and left comments if necessary
  • Confirmed all CI checks have passed

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

I have...

  • Confirmed the correct type prefix in the PR title
  • Confirmed all author checklist items have been addressed
  • Confirmed that this PR does not change production code

@github-actions github-actions bot added C:Testing Assigned automatically by the PR labeler C:Build Assigned automatically by the PR labeler labels Dec 5, 2023
Makefile Outdated Show resolved Hide resolved
@bermuell bermuell requested a review from MSalopek December 5, 2023 18:08
@bermuell bermuell force-pushed the bernd/1133-e2e-compatiblity-tests branch from e441d2c to 9724657 Compare January 10, 2024 12:02
@bermuell bermuell requested a review from p-offtermatt January 10, 2024 12:08
tests/e2e/actions.go Dismissed Show dismissed Hide dismissed
tests/e2e/actions.go Dismissed Show dismissed Hide dismissed
tests/e2e/main.go Fixed Show fixed Hide fixed
tests/e2e/test_driver.go Dismissed Show dismissed Hide dismissed
tests/e2e/actions.go Outdated Show resolved Hide resolved
tests/e2e/compatibility_tests.go Outdated Show resolved Hide resolved
tests/e2e/main.go Outdated Show resolved Hide resolved
@bermuell bermuell marked this pull request as ready for review January 10, 2024 13:00
@bermuell bermuell requested a review from a team as a code owner January 10, 2024 13:00
@bermuell bermuell changed the title test: e2e compatiblity testing test: e2e initialization Jan 11, 2024
Copy link
Contributor

@p-offtermatt p-offtermatt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work, I really like particularly the way the test target abstracts away so much of the underlying logic.

Some comments, none too major, mostly about adding a bit of documentation.
Could you also please update https://github.com/cosmos/interchain-security/blob/bernd/1133-e2e-compatiblity-tests/tests/e2e/README.md#L14 to match the changed setup? Nothing major, but at least adjust file names

tests/e2e/actions.go Show resolved Hide resolved
tests/e2e/actions.go Outdated Show resolved Hide resolved
tests/e2e/builder.go Outdated Show resolved Hide resolved
tests/e2e/builder.go Outdated Show resolved Hide resolved
tests/e2e/main.go Outdated Show resolved Hide resolved
tests/e2e/main.go Outdated Show resolved Hide resolved
tests/e2e/main.go Show resolved Hide resolved
tests/e2e/main.go Show resolved Hide resolved
tests/e2e/test_driver.go Show resolved Hide resolved
tests/e2e/test_target.go Show resolved Hide resolved
tests/e2e/main.go Dismissed Show dismissed Hide dismissed
@p-offtermatt
Copy link
Contributor

Hey, there's one more Gosec error, could you please fix it? :) https://github.com/cosmos/interchain-security/actions/runs/7489943947/job/20387679246?pr=1486#step:4:536

@bermuell
Copy link
Contributor Author

Hey, there's one more Gosec error, could you please fix it? :) https://github.com/cosmos/interchain-security/actions/runs/7489943947/job/20387679246?pr=1486#step:4:536
yes, just pushed the fix + improvement on worktree creation (in case the revision is already checked out) and docker image creation to deal with issues realted to caching.

tests/e2e/main.go Outdated Show resolved Hide resolved
tests/e2e/main.go Outdated Show resolved Hide resolved
tests/e2e/main.go Outdated Show resolved Hide resolved
tests/e2e/main.go Outdated Show resolved Hide resolved
tests/e2e/test_target.go Outdated Show resolved Hide resolved
Dockerfile.combined Outdated Show resolved Hide resolved
tests/e2e/main.go Outdated Show resolved Hide resolved
@sainoe
Copy link
Contributor

sainoe commented Jan 17, 2024

Thanks for this work! It's clear and well-integrated into the current infrastructure.
See my comments.

Copy link
Contributor

@MSalopek MSalopek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

I would like to have an option to use pre-built docker images that can just be pulled from a registry. That way we could avoid extraneous build steps an using git worktree.

The released versions don't change so it's safe to push docker images (eg v3.3.0 will never change, all changes will exist in subsequent versions such as v3.3.1, v3.4.0, v4.x.y etc)

@bermuell bermuell added this pull request to the merge queue Jan 24, 2024
Merged via the queue into main with commit 317cd29 Jan 24, 2024
20 checks passed
@bermuell bermuell deleted the bernd/1133-e2e-compatiblity-tests branch January 24, 2024 10:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C:Build Assigned automatically by the PR labeler C:Testing Assigned automatically by the PR labeler
Projects
None yet
Development

Successfully merging this pull request may close these issues.

update e2e initialization
4 participants