-
-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make benchmark setup consistent #16733
Conversation
See rust-lang/rust#133942, there's not much point in using `black_box()` on a unit type, especially since `map.insert()` has side-effects and will be executed.
Now the distinguishing factor will be their module path, removing the need to rename the group itself if it switches modules.
Modules first, imports second
There's nothing else within the module folder, so it doesn't need to be a folder!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me! Definitely a cleaner structure.
harness = false | ||
|
||
[[bench]] | ||
name = "reflect_function" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was previously running function-reflection-specific benches like:
cargo bench --bench reflect_function --all-features
Does this change mean I won't be able to run specific sets of benches anymore?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, but only temporarily! Just like unit tests, you can filter what subset of benchmarks you want to run:
# Will run `typed/function`, `typed/closure`, `typed/closure_mut`, and more.
cargo bench --bench reflect -- typed
My plan with #16647 is to rename all benchmarks to include their module paths, so you'll be able to run:
# This is equivalent to `cargo bench --bench reflect_function`, using the new system.
cargo bench --bench reflect -- bevy_reflect::function
This PR was a precursor, since I didn't want to have too many changes at once.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I want to second this, because when I still was putting the stress test , it was very annoying to test just the bench while not testing everything else. Can you still run specific benchmarks (or groups of them) with this reorganization?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good!
@BD103 once this is merge conflict free I'll press the button. Thanks for this: the benchmarks folder has been in a state of disarray for way too long. |
This comment was marked as resolved.
This comment was marked as resolved.
be39a7d
to
df26ba2
Compare
@alice-i-cecile should be good to go now! |
Objective
bevy-bencher
to the official benchmarks.Solution
Tip
I recommend reviewing this PR commit-by-commit, instead of all at once!
In 5d26f56 I reorganized how benches were registered. Now this is one
[[bench]]
per Bevy crate. In each crate benchmark folder, there is amain.rs
that callscriterion_main!
. I also disabled automatic benchmark discovery, which isn't necessarily required, but may clear up confusion with our custom setup. I also fixed a few errors that were causing the benchmarks to fail to compile.In afc8d33 I ran
rustfmt
on all of the benchmarks.In d6cdf96 I fixed all of the Clippy warnings.
In ee94d48 I fixed some of the benchmarks' usage of
black_box()
. I ended up opening rust-lang/rust#133942 due to this, which should help prevent this in the future.In cbe1688 I renamed all of the ECS benchmark groups to be called
benches
, to be consistent with the other crate benchmarks.In e701c21 and 8815bb7 I re-ordered some imports and module definitions, and uplifted
fragmentation/mod.rs
tofragementation.rs
.Finally, in b0065e0 I organized
Cargo.toml
and bumped Criterion to v0.5.Testing
cd benches && cargo clippy --benches
cd benches && cargo fmt --all