Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decouple block and blobs publishing\import #8728

Merged
merged 9 commits into from
Nov 4, 2024

Conversation

tbenr
Copy link
Contributor

@tbenr tbenr commented Oct 15, 2024

fixes #8682

PR Description

Documentation

  • I thought about documentation and added the doc-change-required label to this PR if updates are required.

Changelog

  • I thought about adding a changelog entry, and added one if I deemed necessary.

}

// when broadcast validation is enabled, we need to wait for the validation to complete before
// publishing the block (and blob sidecars)

final SafeFuture<BlockImportAndBroadcastValidationResults>
blockImportAndBroadcastValidationResults =
importBlockAndBlobSidecars(
block, blobSidecars, broadcastValidationLevel, blockPublishingPerformance);
importBlock(block, broadcastValidationLevel, blockPublishingPerformance);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel it requires more refactoring. Isn't block import always going to fail, because blob sidecars are not imported.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed the way I push data in the tracker. Now i put them separated and the import should wait until the tracker completes.
This is what I have in mind, need to test if it actually behaves this way:)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is actually working.

@tbenr tbenr changed the title Detach block and blobs publishing\import Decouple block and blobs publishing\import Oct 16, 2024

blockImportAndBroadcastValidationResults
.thenCompose(BlockImportAndBroadcastValidationResults::broadcastValidationResult)
.thenAccept(
broadcastValidationResult -> {
if (broadcastValidationResult == BroadcastValidationResult.SUCCESS) {
publishBlockAndBlobSidecars(block, blobSidecars, blockPublishingPerformance);
publishBlock(block, blockPublishingPerformance)
.always(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, why do we wait for block to finish publishing again and not just send both across async?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's a strategy that has been discussed in several places. The reason are essentially help solo stakers with low up bandwidth to reduce latency in block propagation. With the adoption of engine_getBlobs blobs can be retrieved once the block is known, thus it is good to give priority to the block over the blobs.

I'll put this under a feature flag.

@tbenr tbenr added blocked by another PR/issue 💔 This issue or pull request is blocked by another and removed blocked by another PR/issue 💔 This issue or pull request is blocked by another labels Oct 24, 2024
@tbenr tbenr force-pushed the detach-block-and-blobs-publishing branch from 6a1c0a6 to d5218e6 Compare October 29, 2024 10:47
@tbenr tbenr marked this pull request as ready for review October 31, 2024 12:37
@tbenr tbenr force-pushed the detach-block-and-blobs-publishing branch from ad0ff80 to 5e2d991 Compare October 31, 2024 15:01
Copy link
Contributor

@StefanBratanov StefanBratanov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tbenr tbenr enabled auto-merge (squash) November 4, 2024 11:34
@tbenr tbenr merged commit 4266035 into Consensys:master Nov 4, 2024
17 checks passed
mehdi-aouadi added a commit that referenced this pull request Nov 4, 2024
* Decouple block and blobs publishing\import (#8728)

* use correct schema definitions when creation full beacon block (#8806)

* use correct schema definitions when creation full beacon block

* use forkChoiceUpdatedV4 with PayloadAttributesV4 (#8752) (#8799)

---------

Co-authored-by: Enrico Del Fante <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Publish block before constructing blob sidecars
2 participants