build end-to-end tests and live tests by default #6469
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
In a workspace without these changes: In one with these changes: |
|
The |
|
This also needs updated: omicron/dev-tools/xtask/src/check_workspace_deps.rs Lines 120 to 136 in c3c5f84 |
However, I tried running the live-tests archive and it doesn't run the live tests. The archive was built with: but when I run this in the usual way, it uses the default profile and omits those tests: If I try to pick a specific test, it (reasonably) says you can't use I thought this was a bug in the archive stuff but actually I can't run the live tests in the usual way either: Rereading https://nexte.st/docs/running/#running-a-subset-of-tests-by-default, it looks like maybe I need to be using (I expected that failure -- I'm just trying to exercise the filtering behavior here.) So then I tried to use So then I tried passing it to So I guess the change needed here is to the instructions that get printed out by |
|
From 8b77172, I've retested like this:
|
iliana
approved these changes
Aug 28, 2024
sunshowers
reviewed
Aug 28, 2024
.config/nextest.toml
Outdated
| @@ -35,6 +35,9 @@ clickhouse-cluster = { max-threads = 1 } | |||
| # behaviors that conflict with each other. They need to be run serially. | |||
| live-tests = { max-threads = 1 } | |||
|
|
|||
| [profile.default] | |||
| default-set = 'all() - package(omicron-live-tests) - package(end-to-end-tests)' | |||
There was a problem hiding this comment.
Id recommend also adding a live-tests profile with exactly 'package(omicron-live-tests)' as the default set, so folks can run this with -P live-tests.
dev-tools/xtask/src/live_tests.rs
Outdated
| @@ -134,8 +134,11 @@ pub fn run_cmd(_args: Args) -> Result<()> { | |||
| // TMPDIR=/var/tmp puts stuff on disk, cached as needed, rather than the | |||
| // default /tmp which requires that stuff be in-memory. That can lead to | |||
| // great sadness if the tests wind up writing a lot of data. | |||
| // | |||
| // --bound=all is required to cause nextest to *not* ignore these tests, | |||
There was a problem hiding this comment.
This shouldn't be an issue with the profile. The bound argument was needed because the UI gets pretty confusing if you try to make it more magical. But it's better to just use a profile per use case I think.
|
Alright, with f9bb4eb:
|
|
nextest 0.9.77 renames a few of these fields. I'll be taking another lap shortly (when the release binaries are available). |
|
6038bf9 brings this to nextest 0.9.77. With that commit:
|
|
Confirmed that helios-deploy ran the expected end-to-end-tests. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Thanks to recent nextest work by @sunshowers, we can add end-to-end-tests and live-tests to the workspace default members (so that they get built with
cargo check, for example) without running their tests by default. These had previously been excluded from the default members because their tests don't work in dev environments.