Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add compile_fail_utils, compile fail tests, and benchmarks to Cargo workspace #16770

Closed
wants to merge 7 commits into from

Conversation

BD103
Copy link
Member

@BD103 BD103 commented Dec 11, 2024

Objective

  • compile_fail_utils, benches, and all of our compile_fail sub-crates are outside of the Cargo workspace, so they are not checked by default nor caught by CI. As a result, these crates have several compile errors and warnings that were never discovered.

Solution

  • Add compile_fail_utils, benches, and all compile_fail crates to the Cargo workspace.
  • Fix several compile errors and warnings, and run rustfmt.
  • Remove custom release profile from benchmarks.
    • This may cause a regression in benchmark statistics, since we no longer set lto = true.
    • I can re-add this back as a custom profile available to all crates within the workspace, but I'm unsure its worth the hassle.
  • Do not run cargo test --benches in CI.
    • This would have caused all of the benchmarks to be run for every pull request, which doesn't seem desired.

Testing

  • cargo check --workspace --all-targets
  • cd tools/compile_fail_utils && cargo test --test example

@alice-i-cecile alice-i-cecile added A-Build-System Related to build systems or continuous integration C-Code-Quality A section of code that is hard to understand or change labels Dec 11, 2024
@BD103 BD103 added C-Bug An unexpected or incorrect behavior D-Trivial Nice and easy! A great choice to get started with Bevy C-Testing A change that impacts how we test Bevy or how users test their apps S-Needs-Review Needs reviewer attention (from anyone!) to move forward A-Build-System Related to build systems or continuous integration and removed A-Build-System Related to build systems or continuous integration C-Code-Quality A section of code that is hard to understand or change labels Dec 11, 2024
Copy link
Member

@alice-i-cecile alice-i-cecile left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good fix, but can we add something to CI to ensure that this is automatically checked?

I had to remove the benchmarks' custom release profile, which enabled link-time optimizations. This _may_ cause benchmarks to perform slightly worse, but I haven't tested it yet.
@BD103
Copy link
Member Author

BD103 commented Dec 12, 2024

Good fix, but can we add something to CI to ensure that this is automatically checked?

Yup! The simplest fix I could find was to add all of the compile_fail tests to the Cargo workspace. (Which works, surprisingly, so I'm unsure why we didn't do this in the past!)

@BenjaminBrienen
Copy link
Contributor

BenjaminBrienen commented Dec 12, 2024

I thought that we don't include the compile_fail tests in the workspace because it doesn't work with crater. @alice-i-cecile am I hallucinating that? 😆

@alice-i-cecile alice-i-cecile added D-Straightforward Simple bug fixes and API improvements, docs, test and examples S-Waiting-on-Author The author needs to make changes or address concerns before this can be merged and removed D-Trivial Nice and easy! A great choice to get started with Bevy S-Needs-Review Needs reviewer attention (from anyone!) to move forward labels Dec 12, 2024
@alice-i-cecile
Copy link
Member

I don't remember that! And not seeing any issues on https://github.com/rust-lang/crater. Are you sure you haven't been replaced by an LLM?

Anyways @BD103 CI is now mad at you, so kicking this back to you until it's placated.

@BenjaminBrienen
Copy link
Contributor

@BD103
Copy link
Member Author

BD103 commented Dec 12, 2024

https://github.com/bevyengine/bevy/blob/main/tools%2Fcompile_fail_utils%2FREADME.md

I wonder why it says that! As far as I can tell, Crater just runs cargo build && cargo test. The only time Crater may fail is if it manually runs rustc on any .rs file it finds, which wouldn't make sense... @cart, do you have anything to add to this?

Either way, let me fix the final few Clippy lints!

@BD103 BD103 changed the title Fix errors in compile_fail_utils Add compile_fail_utils, compile fail tests, and benchmarks to Cargo workspace Dec 12, 2024
@BD103
Copy link
Member Author

BD103 commented Dec 12, 2024

Turns out CI was running cargo test --benches, which would cause all of our benchmarks to be run for every PR. I turned that off in 2d763ab, since that seems undesired.

@BenjaminBrienen
Copy link
Contributor

For the lifetime elisions, disable that clippy lint to be in line with the workspace.

@BD103 BD103 force-pushed the compile-fail-utils branch from 677682e to 8a13e12 Compare December 12, 2024 21:55
@BD103
Copy link
Member Author

BD103 commented Dec 12, 2024

While there's a bit more I could do to improve the code quality of the benchmarks, I'm going to save it for a separate PR.

@BD103
Copy link
Member Author

BD103 commented Dec 12, 2024

I'm going to take another shot at this tomorrow. For some reason I'm unable to reproduce CI locally...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-Build-System Related to build systems or continuous integration C-Bug An unexpected or incorrect behavior C-Testing A change that impacts how we test Bevy or how users test their apps D-Straightforward Simple bug fixes and API improvements, docs, test and examples S-Waiting-on-Author The author needs to make changes or address concerns before this can be merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants