fix: enable arbitrary benchmark aggregations

[gdalle]

DIT source

• Replace aggregation(benchmark) with aggregation(getfield.(benchmark.samples, :time)) so that it works beyond the aggregators that Chairmarks extends. For instance, one can now include uncertainty in the measurements, like so:

julia> using DifferentiationInterface, DifferentiationInterfaceTest, Measurements, Statistics

julia> import ForwardDiff

julia> scen = Scenario{:gradient, :out}(sum, rand(1000));

julia> backends = [AutoForwardDiff(; chunksize=1), AutoForwardDiff(; chunksize=10)];

julia> data = benchmark_differentiation(backends, [scen]; benchmark_aggregation=x -> mean(x) ± std(x))
Test Summary:                   | Pass  Total  Time
Testing benchmarks              |    4      4  4.6s
  AutoForwardDiff(chunksize=1)  |    2      2  2.3s
  AutoForwardDiff(chunksize=10) |    2      2  2.3s
4×12 DataFrame
 Row │ backend                        scenario                           operator            prepared  calls  samples  evals  time             allocs     bytes       gc_fraction    compile_fract ⋯
     │ AutoForw…                      Scenario…                          Symbol              Bool      Int64  Int64    Int64  Measurem…        Measurem…  Measurem…   Measurem…      Measurem…     ⋯
─────┼──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1 │ AutoForwardDiff(chunksize=1)   Scenario{:gradient,:out} sum : V…  value_and_gradient      true   1000      651      1  0.00146±0.00046    4.0±0.0  8096.0±0.0  0.0014±0.035            0.0± ⋯
   2 │ AutoForwardDiff(chunksize=1)   Scenario{:gradient,:out} sum : V…  gradient                true   1000      694      1  0.00137±9.2e-5     3.0±0.0  8064.0±0.0  0.0±0.0                 0.0±
   3 │ AutoForwardDiff(chunksize=10)  Scenario{:gradient,:out} sum : V…  value_and_gradient      true    100     4212      1  0.00022±0.00016    4.0±0.0  8096.0±0.0  0.00046±0.021           0.0±
   4 │ AutoForwardDiff(chunksize=10)  Scenario{:gradient,:out} sum : V…  gradient                true    100     4150      1  0.000227±5.1e-5    3.0±0.0  8064.0±0.0  0.00043±0.019           0.0±
                          

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.53%. Comparing base (6b55559) to head (ee7f668).
Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #639      +/-   ##
==========================================
- Coverage   97.53%   97.53%   -0.01%     
==========================================
  Files         111      111              
  Lines        5551     5550       -1     
==========================================
- Hits         5414     5413       -1     
  Misses        137      137              

Flag	Coverage Δ
DI	98.65% <ø> (ø)
DIT	95.35% <ø> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

---

🚨 Try these New Features:

• Flaky Tests Detection - Detect and resolve failed and flaky tests