Better CI and CD

[brainstorm]

1. Fix caching mechanism on GHA, it currently takes too long and sometimes even times out: https://ectobit.com/blog/speed-up-github-actions-rust-pipelines/
2. Implement continuous benchmarking: https://github.com/benchmark-action/github-action-benchmark

[brainstorm]

3. Also add CD to the mix with https://github.com/googleapis/release-please, or just copying the skeleton that Johannes uses for rust-htslib: https://github.com/rust-bio/rust-htslib/blob/master/.github/workflows/release-please.yml ... OR use a more rust-idiomatic alternative: https://www.marcoieni.com/2022/06/release-plz-release-rust-packages-from-ci/

Might be interesting to compare against the reference Go impl: https://rhysd-hatenablog-com.translate.goog/entry/2019/11/11/131505?_x_tr_sl=auto&_x_tr_tl=en&_x_tr_hl=sv&_x_tr_pto=wapp

Thanks @victorskl for the heads up on this, we definitely need proper release engineering on this project now that it's going to prod (https://crates.io, docker builds, binaries for different arches, doc generation, etc...).

[brainstorm]

As discussed with @mmalenic, we will experiment with https://www.conventionalcommits.org/en/v1.0.0/ and suitable integrations so that we can easily generate changelogs in releases.

[brainstorm]

More goodies on the benchmarking side:

https://www.tweag.io/blog/2022-03-03-criterion-rs/
https://github.com/BurntSushi/critcmp

I'm looking at some system that could keep historical benchmarking data in order to detect regressions and performance improvements, automatically (if something else than benchmark-action came up in the last 5 months).

For now this kind of output is more than enough:

https://benchmark-action.github.io/github-action-benchmark/dev/bench/

[mmalenic]

It seems like benchmark-action still has the best features. This action does store previous data which you can visualise in graphs such as the one you linked.

[brainstorm]

It seems like benchmark-action still has the best features. This action does store previous data which you can visualise in graphs such as the one you linked.

Indeed, basing my current iteration off of it, just wanted to see what's out there, this action still seems to be the best so far.

[mmalenic]

As discussed with @mmalenic, we will experiment with https://www.conventionalcommits.org/en/v1.0.0/ and suitable integrations so that we can easily generate changelogs in releases.

I'm happy to be pretty flexible about these. I notice that some projects tend to avoid the predefined types, and instead just make the type the scope of the change. E.g. tokio-rs.

I'm not sure how types like feat interact with a larger PR that might be working towards one feature, but should be split up into multiple commits. Should each commit have feat in front of it?

[brainstorm]

As discussed with @mmalenic, we will experiment with https://www.conventionalcommits.org/en/v1.0.0/ and suitable integrations so that we can easily generate changelogs in releases.

I'm happy to be pretty flexible about these. I notice that some projects tend to avoid the predefined types, and instead just make the type the scope of the change. E.g. tokio-rs.

I'm not sure how types like feat interact with a larger PR that might be working towards one feature, but should be split up into multiple commits. Should each commit have feat in front of it?

I still haven't research the GHA for conventional commits... I would base our way of writing the commits on the features available on those, i.e:

https://github.com/marketplace/actions/changelog-from-conventional-commits

There's actually quite a few of them in the marketplace we might want to review:

https://github.com/marketplace?type=actions&query=conventional+commits

I'd like to prioritize automation over our own conventions first. Pretty much like git and nvie workflows: stick to third party conventions first to avoid cognitive burden and then adapt to our needs if that shirt doesn't fit us?

[mmalenic]

Yeah agreed, we should just do what others do I think.

[brainstorm]

Benchmarks now available on those two locations:

https://umccr.github.io/htsget-rs/dev/bench/
https://umccr.github.io/htsget-rs/dev/bench/native/criterion/reports/

A few things to polish w.r.t history keeping, displaying results, routes, add 3rd party projects for comparison, etc... but at least it's up and running now.

Also PRs upstream need to be reviewed and merged, so using my own fork for now:

benchmark-action/github-action-benchmark#145
benchmark-action/github-action-benchmark#146

Will move on to CD with release-plz, stash, review&merge.

[brainstorm]

This is awesome: https://www.jibbow.com/posts/criterion-flamegraphs/