Skip to content

Commit

Permalink
clarify contributing docs
Browse files Browse the repository at this point in the history
  • Loading branch information
MarquessV committed Jun 12, 2024
1 parent 00c88d0 commit aadb847
Showing 1 changed file with 5 additions and 3 deletions.
8 changes: 5 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -287,9 +287,11 @@ Note that benchmark results are unique to your machine. They can't be directly c
results on another machine unless it's a machine with identical specifications running in a similar
environment. To track performance over time in a controlled way, we use _continuous benchmarking_.
When a PR is opened, CI will run the benchmarks and compare the results to the most recent results
on the `master` branch. Since CI always uses the same image and workflow, the results are fair
to compare. When opening or reviewing a PR, you should evaluate the results and ensure there are no
unexpected regressions.
on the `master` branch. Since CI always uses the same image and workflow, the results should be
reasonably consistent. That said, the runners could share resources or do something else unexpected
that impacts the benchmarks. If you get unexpected results, you may want to re-run the benchmark
to see if the results are consistent. When opening or reviewing a PR, you should evaluate the results
and ensure there are no unexpected regressions.

Continuous benchmarking is implemented with
[bencher](https://bencher.dev/docs/tutorial/quick-start/). See their documentation for more
Expand Down

0 comments on commit aadb847

Please sign in to comment.