Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

iai benchmarks on PRs? #25

Open
danielparks opened this issue Jan 12, 2023 · 3 comments
Open

iai benchmarks on PRs? #25

danielparks opened this issue Jan 12, 2023 · 3 comments
Labels
enhancement New feature or request

Comments

@danielparks
Copy link
Owner

Maybe it would be good to run iai benchmarks on each PR? I’m not quite sure how we would track and report results — maybe it would add a commit to the PR before it’s merged?
@danielparks danielparks added the enhancement New feature or request label Jan 12, 2023
@danielparks danielparks mentioned this issue Jan 12, 2023
Merged
4 tasks
danielparks added a commit that referenced this issue Jan 12, 2023
@danielparks
First pass at cycle counting with iai
873c127 
[iai] “benchmarks” functions by counting instructions, cache hits, and
memory accesses. It also estimates cycles.

It uses [`valgrind`], which officially doesn’t run on macOS (though
there is a [port]), and `setarch`, a Linux-only command they use to
disable address space layout randomization (ASLR). See [iai #25] for
more information about running on macOS.

This includes a Dockerfile and a docker.sh script to provide an
environment to run `valgrind`.

Note that iai hasn’t been updated since February 2021. Some folks are
working on [calliper], but it doesn’t seem to be ready yet.

[iai]: https://crates.io/crates/iai
[`valgrind`]: https://valgrind.org
[port]: https://github.com/LouisBrunner/valgrind-macos
[iai #25]: bheisler/iai#25
[calliper]: https://crates.io/crates/calliper
@epompeii
Copy link

Hey @danielparks, I've been working a continuous benchmarking tool to do just that, Bencher: https://github.com/bencherdev/bencher
Adding iai support is currently in the works: bencherdev/bencher#98
Once that has merged, I would be more than happy to help add Bencher to htmlize.

@danielparks
Copy link
Owner Author

Interesting, thanks! I’ll keep an eye on that PR.

@epompeii
Copy link

@danielparks just to follow up here, iai support has been added to Bencher!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@danielparks @epompeii