A0-3958: Refactor extender

[timorleph]

This mainly separates it into the logic itself and the async service that executes it. Due to this I was able to add a bit more unit tests.

In preparation for adding the performance reports, since they will need to order everything in the DAG, even if slightly behind, and that will require changing this code.

Also fixed some small pieces of documentation here and there.

[github-actions]

Please make sure the following happened

• Appropriate tests created
• Infrastructure updated accordingly
• Updated existing documentation
• New documentation created
• Version bumped if breaking changes

[fixxxedpoint]

Great work!

[DamianStraszak]

Looks much more healthy than before.

However, I would strongly suggest running random tests against the previous implementation to make sure it matches the behavior. It's very easy to make a small mistake here that's hard to catch by manual inspection.

For such tests, the simplest approach would be perhaps to adapt

AlephBFT/consensus/src/testing/dag.rs

Line 276 in 039c752

async fn ordering_random_dag_consistency_under_permutations() {

-- this generates a random dag (including forker cases) and pushes two different permutations of the units through the whole abft pipeline, and then verifies the batches are the same in both cases.
In the linked unit case, this runs just 4 iterations -- that's to not make unit tests too long. But what I used to do back in the days was to run this for millions of iterations (leave it for ~24h or so) after sensitive changes. This can catch a vast majority of bugs that can occur in practice. In particular -- letting it run on devnet, or even testnet, for lots of time, would not be as effective.

The way this could be adapted is, instead of running on two permutations, somehow run on both versions of Extender, old and new. Such a test would be run just once, no need to add it as a permanent "unit test".

[timorleph]

Oh bloody hell, the comments I made yesterday did not save 'cause they are arbitrarily part of a review, what the heck. >.<

[fixxxedpoint]

Great work!