Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Offload block processor notifications #4763

Conversation

pwojcikdev
Copy link
Contributor

This offloads block processors notifications to be processed on a background thread, so that block processor can continue checking new blocks without waiting for other components to do their work. Checking new blocks requires a write-transaction, while handling batch_processed event is done with read-only transaction, both can happen in parallel.

The changes here are based on top of #4762 which is a necessary prerequisite.

@qwahzi qwahzi added this to the V28 milestone Oct 22, 2024
@gr0vity-dev-bot
Copy link

gr0vity-dev-bot commented Oct 22, 2024

Test Results for Commit bac144f

Pull Request 4763: Results
Overall Status:

Test Case Results

  • 5n4pr_conf_10k_bintree: PASS (Duration: 111s)
  • 5n4pr_conf_10k_change: PASS (Duration: 184s)
  • 5n4pr_conf_change_dependant: PASS (Duration: 149s)
  • 5n4pr_conf_change_independant: PASS (Duration: 153s)
  • 5n4pr_conf_send_dependant: PASS (Duration: 134s)
  • 5n4pr_conf_send_independant: PASS (Duration: 135s)
  • 5n4pr_rocks_10k_bintree: PASS (Duration: 108s)
  • 5n4pr_rocks_10k_change: PASS (Duration: 254s)

Last updated: 2024-10-28 09:33:34 UTC

@pwojcikdev pwojcikdev force-pushed the offload-block-processor-notifications branch 2 times, most recently from 2a1b379 to 8fb8dc2 Compare October 23, 2024 10:34
@qwahzi
Copy link
Collaborator

qwahzi commented Oct 23, 2024

Related tests from @gr0vity-dev, showing the reduction in delays for "block processed" to "publish sent" under load:

image

image

@pwojcikdev pwojcikdev force-pushed the offload-block-processor-notifications branch from 8fb8dc2 to 1a2e6ed Compare October 27, 2024 10:06
clemahieu
clemahieu previously approved these changes Oct 27, 2024
@@ -255,6 +255,7 @@ nano::container_info nano::confirming_set::container_info () const

nano::container_info info;
info.put ("set", set);
info.add ("notification_workers", notification_workers.container_info ());
info.put ("notifications", workers.queued_tasks ());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this already added with workers.container_info() below?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be redundant, right

@pwojcikdev pwojcikdev dismissed clemahieu’s stale review October 27, 2024 11:25

The merge-base changed after approval.

@pwojcikdev pwojcikdev force-pushed the offload-block-processor-notifications branch from 5e206ec to bac144f Compare October 28, 2024 08:09
@pwojcikdev pwojcikdev merged commit 104b74d into nanocurrency:develop Oct 28, 2024
27 of 28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Merged / V28.0
Development

Successfully merging this pull request may close these issues.

4 participants