Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move doctest out to a dedicated job #8396

Closed
wants to merge 4 commits into from

Conversation

elliottt
Copy link
Member

@elliottt elliottt commented Apr 17, 2024

Move doctests out of individual tests by shadowing rustdoc when running ./ci/run-tests.sh, and explicitly run doctests only in their own dedicated job on an ubuntu-latest runner.

@elliottt elliottt force-pushed the trevor/nextest branch 6 times, most recently from 97eb4a9 to c4e4439 Compare April 17, 2024 18:50
@alexcrichton
Copy link
Member

Comparing this with another run sampled historically I don't think there's any speedup? That matches my gut on this given the low core count of actions runners and how all our tests are internally parallel too.

@elliottt
Copy link
Member Author

elliottt commented Apr 17, 2024

Comparing this with another run sampled historically I don't think there's any speedup? That matches my gut on this given the low core count of actions runners and how all our tests are internally parallel too.

I'm seeing 15 vs 18 minutes for the Linux x64 run, though that's with the cached build of cargo-nextest.

@elliottt
Copy link
Member Author

Comparing this with another run sampled historically I don't think there's any speedup? That matches my gut on this given the low core count of actions runners and how all our tests are internally parallel too.

I'm seeing 15 vs 18 minutes for the Linux x64 run, though that's with the cached build of cargo-nextest.

Never mind, @alexcrichton pointed out that this is due to not running the lldb tests on branches :)

@elliottt
Copy link
Member Author

Given that this isn't really speeding up testing, I don't think this is worth pursuing. If we decide to start partitioning tests onto different runners in the future it would be worth picking this back up, but without that the benefit just isn't there.

@elliottt elliottt closed this Apr 17, 2024
@elliottt elliottt reopened this Apr 17, 2024
@elliottt elliottt changed the title Swich CI to test with nextest Move doctest out to a dedicated job Apr 17, 2024
@elliottt elliottt force-pushed the trevor/nextest branch 2 times, most recently from 4fbe153 to 2e73894 Compare April 17, 2024 22:36
@elliottt
Copy link
Member Author

OK, I've also concluded that it's not worth merging just the change to move the doctests out to a separate job, as it would only speed up CI on branches by about 2 minutes, and would have no effect on total merge time on main. It has the additional downside of ignoring any doctest = false settings in any Cargo.toml file, as cargo test --doc unconditionally runs doctests.

@elliottt elliottt closed this Apr 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants