WIP: Check for performance improvements with C++

[IndrajeetPatil]

Related to #558 and #759

Not converting this to draft because I want touchstone to continuously monitor benchmarking.

[codecov-commenter]

Codecov Report

Merging #997 (d26bbdd) into main (38f0a3b) will increase coverage by 0.02%.
The diff coverage is 93.33%.

@@            Coverage Diff             @@
##             main     #997      +/-   ##
==========================================
+ Coverage   90.27%   90.30%   +0.02%     
==========================================
  Files          47       49       +2     
  Lines        2683     2691       +8     
==========================================
+ Hits         2422     2430       +8     
  Misses        261      261              

Impacted Files	Coverage Δ
R/addins.R	14.28% <0.00%> (ø)
R/utils.R	85.00% <ø> (-3.47%)	⬇️
src/ensure_last_n_empty.cpp	100.00% <100.00%> (ø)
src/line_col_names.cpp	100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[lorenzwalthert]

Oh, looking forward to this one. Also, can we exclude .cpp files from the wordlist hook? See how other hooks use the exclude: key in the .pre-commit-config.yaml

[github-actions]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if b4afa55 is merged into main:

• ❗🐌cache_applying: 24.6ms -> 25.9ms [+4.95%, +6.22%]
•   :ballot_box_with_check:cache_recording: 1.13s -> 1.13s [-0.05%, +0.79%]
•   :ballot_box_with_check:without_cache: 2.98s -> 2.99s [-0.19%, +0.37%]

Further explanation regarding interpretation and methodology can be found in the documentation.

[lorenzwalthert]

@IndrajeetPatil you will always get a statistically significant result if you increase iterations. Bu the question is is it also significant in terms of domain. And I think a significant change for cache_recording is ~10% improvement. So the idea was to choose the iterations with 30 to make the statistical and domain significance coincide (roughly).

[github-actions]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 454756c is merged into main:

• ❗🐌cache_applying: 26.6ms -> 28.6ms [+6.36%, +8.53%]
•   :ballot_box_with_check:cache_recording: 1.19s -> 1.19s [-0.48%, +0.28%]
•   :ballot_box_with_check:without_cache: 3.13s -> 3.12s [-0.55%, +0.1%]

Further explanation regarding interpretation and methodology can be found in the documentation.

[github-actions]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if fb191f7 is merged into main:

• ❗🐌cache_applying: 27ms -> 28.8ms [+5.88%, +7.5%]
•   :ballot_box_with_check:cache_recording: 1.26s -> 1.26s [-0.73%, +1.17%]
•   :ballot_box_with_check:without_cache: 3.3s -> 3.3s [-0.86%, +0.73%]

Further explanation regarding interpretation and methodology can be found in the documentation.

[IndrajeetPatil]

At this point, I don't think this is the way to go for improving performance. We should explore all possibilities from R itself, and then move to this option.

[lorenzwalthert]

👍