From ed70ee16b535da0d4aae88bc4cb83178f8fec208 Mon Sep 17 00:00:00 2001 From: Felix Lange Date: Mon, 13 Jan 2025 14:42:44 +0100 Subject: [PATCH 1/9] EIP-7642: add earliestBlock and update receipts encoding --- EIPS/eip-7642.md | 74 ++++++++++++++++++++++++++---------------------- 1 file changed, 40 insertions(+), 34 deletions(-) diff --git a/EIPS/eip-7642.md b/EIPS/eip-7642.md index 2a8835e5d92689..94d5e5c1540ee6 100644 --- a/EIPS/eip-7642.md +++ b/EIPS/eip-7642.md @@ -1,7 +1,7 @@ --- eip: 7642 -title: eth/69 - Drop pre-merge fields -description: Drop unnecessary fields after the merge +title: eth/69 - Protocol version for history expiry and simplifying receipts +description: Adds history serving window and removes bloom filter in receipt author: Marius van der Wijden (@MariusVanDerWijden), Felix Lange discussions-to: https://ethereum-magicians.org/t/eth-70-drop-pre-merge-fields-from-eth-protocol/19005 status: Stagnant @@ -13,21 +13,31 @@ requires: 5793 ## Abstract -After the merge the `td` field in the networking protocol became obsolete. -This EIP modifies the networking messages such that this field is not sent anymore. -Additionally we propose to remove the `Bloom` field from the receipts networking messages. +This EIP modifies the 'eth' p2p protocol to announce the historical block range served by +the node. We also simplify the handshake to remove total difficulty information, which +isn't used anymore after the merge. Additionally we propose to remove the `Bloom` field +from receipts transfered over the protocol. ## Motivation -We recently discovered that none of the clients store the `Bloom` field of the receipts as it can be recomputed on demand. -However the networking spec requires the `Bloom` field to be sent over the network. -Thus a syncing node will ask for the Bloom filters for all receipts. -The serving node will regenerate roughly 530GB of bloom filters (2.3B txs * 256 byte). -These 530GBs are send over the network to the syncing peer, the syncing peer will verify them and not store them either. -This adds an additional 530GB of unnecessary bandwidth to every sync. +### Removing Bloom in Receipts -Additionally we propose to remove the field that was deprecated by the merge, namely -the `TD` field in the `Status` message. +We recently discovered that none of the clients store the `Bloom` field of the receipts as +it can be recomputed on demand. However the networking spec requires the `Bloom` field to +be sent over the network. Thus a syncing node will ask for the Bloom filters for all +receipts. The serving node will regenerate roughly 530GB of bloom filters (2.3B txs * 256 +byte). These 530GBs are send over the network to the syncing peer, the syncing peer will +verify them and not store them either. This adds an additional 530GB of unnecessary +bandwidth to every sync. + +### earliestBlock in Status message + +In the history expiry working group, it was decided that clients may drop pre-merge +history from their storage after April 1, 2025. For clients that want to sync history +through the 'eth' protocol, it is essential to know whether a peer still serves old +history. A similar idea was proposed in +[EIP-7542](https://eips.ethereum.org/EIPS/eip-7542) but was later withdrawn because a +political decision on history expiry had not been reached at the time. ## Specification @@ -35,47 +45,43 @@ Modify the `Status (0x00)` message as follows: - (eth/68): `[version: P, networkid: P, td: P, blockhash: B_32, genesis: B_32, forkid]` -- (eth/69): `[version: P, networkid: P, blockhash: B_32, genesis: B_32, forkid]` +- (eth/69): `[version: P, networkid: P, blockhash: B_32, genesis: B_32, forkid, earliestBlock: P]` Modify the encoding for receipts in the `Receipts (0x10)` message as follows: -- (eth/68): `receipt = {legacy-receipt, typed-receipt}` with `typed-receipt = tx-type || receipt-data` and +- (eth/68): `receipt = {legacy-receipt, typed-receipt}` with ``` -legacy-receipt = [ - post-state-or-status: {B_32, {0, 1}}, - cumulative-gas: P, - bloom: B_256, - logs: [log₁, log₂, ...] -] -``` +typed-receipt = tx-type || legacy-receipt -- (eth/69): `receipt = {legacy-receipt, typed-receipt}` with `typed-receipt = tx-type || receipt-data` and - -``` legacy-receipt = [ post-state-or-status: {B_32, {0, 1}}, cumulative-gas: P, + bloom: B_256, logs: [log₁, log₂, ...] ] ``` -We omit the bloom filter from both the legacy and typed receipts. -Receiving nodes will be able to recompute the bloom filter based on the logs. +- (eth/69): `receipt = [tx-type, post-state-or-status, cumulative-gas, logs]` ## Rationale -After the merge, the `TD` field of the `Status` message became meaningless since the difficulty of post-merge blocks are 0. -It could in theory be used to distinguish synced with unsynced nodes, -but the same thing can be accomplished with the forkid as well. -It is not used in the go-ethereum codebase in any way. +After the merge, the `TD` field of the `Status` message became meaningless since the +difficulty of post-merge blocks are 0. It could in theory be used to distinguish synced +with unsynced nodes, but the same thing can be accomplished with the forkid as well. -Removing the bloom filters from the `Receipt` message reduces the cpu load of serving nodes as well as the bandwidth significantly. The receiving nodes will need to recompute the bloom filter. The recomputation is not very CPU intensive. -The bandwidth gains amount to roughly 530GiB per syncing node or (at least) 95GiB snappy compressed. +Removing the bloom filters from the `Receipt` message reduces the CPU load of serving +nodes as well as the bandwidth significantly. The receiving nodes will need to recompute +the bloom filter in order to fully verify the receipt hash. The recomputation is not very +CPU intensive. The bandwidth gains amount to roughly 530GiB per syncing node or (at least) +95GiB snappy compressed. ## Backwards Compatibility -This EIP changes the eth protocol and requires rolling out a new version, `eth/69`. Supporting multiple versions of a wire protocol is possible. Rolling out a new version does not break older clients immediately, since they can keep using protocol version `eth/68`. +This EIP changes the eth protocol and requires rolling out a new version, `eth/69`. +Supporting multiple versions of a wire protocol is possible. Rolling out a new version +does not break older clients immediately, since they can keep using protocol version +`eth/68`. This EIP does not change consensus rules of the EVM and does not require a hard fork. From e7462cbde1363aab39978ffa2508fb849ba8677c Mon Sep 17 00:00:00 2001 From: Felix Lange Date: Mon, 13 Jan 2025 14:52:46 +0100 Subject: [PATCH 2/9] EIP-7642: update title --- EIPS/eip-7642.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EIPS/eip-7642.md b/EIPS/eip-7642.md index 94d5e5c1540ee6..1790bc9e41bbf8 100644 --- a/EIPS/eip-7642.md +++ b/EIPS/eip-7642.md @@ -1,6 +1,6 @@ --- eip: 7642 -title: eth/69 - Protocol version for history expiry and simplifying receipts +title: eth/69 - history expiry and simpler receipts description: Adds history serving window and removes bloom filter in receipt author: Marius van der Wijden (@MariusVanDerWijden), Felix Lange discussions-to: https://ethereum-magicians.org/t/eth-70-drop-pre-merge-fields-from-eth-protocol/19005 From 8a4b8e215055951c9a8c490bdd395653d11c340a Mon Sep 17 00:00:00 2001 From: Felix Lange Date: Mon, 13 Jan 2025 14:53:21 +0100 Subject: [PATCH 3/9] EIP-7642: update link --- EIPS/eip-7642.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/EIPS/eip-7642.md b/EIPS/eip-7642.md index 1790bc9e41bbf8..db69ca98f7030b 100644 --- a/EIPS/eip-7642.md +++ b/EIPS/eip-7642.md @@ -35,9 +35,8 @@ bandwidth to every sync. In the history expiry working group, it was decided that clients may drop pre-merge history from their storage after April 1, 2025. For clients that want to sync history through the 'eth' protocol, it is essential to know whether a peer still serves old -history. A similar idea was proposed in -[EIP-7542](https://eips.ethereum.org/EIPS/eip-7542) but was later withdrawn because a -political decision on history expiry had not been reached at the time. +history. A similar idea was proposed in [EIP-7542](./eip-7542.md) but was later withdrawn +because a political decision on history expiry had not been reached at the time. ## Specification From 70d7c2549d81a82447a3bc7d0bd863d2915bd1c0 Mon Sep 17 00:00:00 2001 From: Felix Lange Date: Mon, 13 Jan 2025 19:31:04 +0100 Subject: [PATCH 4/9] EIP-7642: add rationale for earliestBlock --- EIPS/eip-7642.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/EIPS/eip-7642.md b/EIPS/eip-7642.md index db69ca98f7030b..6fe6800ecced1d 100644 --- a/EIPS/eip-7642.md +++ b/EIPS/eip-7642.md @@ -65,10 +65,25 @@ legacy-receipt = [ ## Rationale +### Status changes + After the merge, the `TD` field of the `Status` message became meaningless since the difficulty of post-merge blocks are 0. It could in theory be used to distinguish synced with unsynced nodes, but the same thing can be accomplished with the forkid as well. +The new `earliestBlock` field is technically not required for history expiry, but there +are a couple reasons why adding it can help: + +- It improves peer finding for clients that still want to sync history from p2p after the + agreed-upon removal of pre merge history has taken place. Without `earliestBlock`, the + client would have to perform a request for history to check if the earlier range exists, + and assume that a failed request means it's not there. +- The new field can be used for census in a specialized crawler. We will be able to see + how many users/nodes enable history, and in which implementation. +- It prepares us for a future where the history expiry window is dynamic. + +### Receipts changes + Removing the bloom filters from the `Receipt` message reduces the CPU load of serving nodes as well as the bandwidth significantly. The receiving nodes will need to recompute the bloom filter in order to fully verify the receipt hash. The recomputation is not very From b9ab3518d552bab272ac70f7c04fe75a77d809d4 Mon Sep 17 00:00:00 2001 From: Felix Lange Date: Tue, 14 Jan 2025 10:49:02 +0100 Subject: [PATCH 5/9] EIP-7642: update deadline --- EIPS/eip-7642.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EIPS/eip-7642.md b/EIPS/eip-7642.md index 6fe6800ecced1d..1595cb99a0a6a5 100644 --- a/EIPS/eip-7642.md +++ b/EIPS/eip-7642.md @@ -33,7 +33,7 @@ bandwidth to every sync. ### earliestBlock in Status message In the history expiry working group, it was decided that clients may drop pre-merge -history from their storage after April 1, 2025. For clients that want to sync history +history from their storage after May 1, 2025. For clients that want to sync history through the 'eth' protocol, it is essential to know whether a peer still serves old history. A similar idea was proposed in [EIP-7542](./eip-7542.md) but was later withdrawn because a political decision on history expiry had not been reached at the time. From 4dd19d5c82f65e217969cb42bbe1757e8ab21cc5 Mon Sep 17 00:00:00 2001 From: Felix Lange Date: Tue, 14 Jan 2025 10:59:08 +0100 Subject: [PATCH 6/9] EIP-7642: add rationale for receipt encoding --- EIPS/eip-7642.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/EIPS/eip-7642.md b/EIPS/eip-7642.md index 1595cb99a0a6a5..98461e58f120e3 100644 --- a/EIPS/eip-7642.md +++ b/EIPS/eip-7642.md @@ -90,6 +90,14 @@ the bloom filter in order to fully verify the receipt hash. The recomputation is CPU intensive. The bandwidth gains amount to roughly 530GiB per syncing node or (at least) 95GiB snappy compressed. +In Ethereum consensus, the encoding of receipts differs between legacy transactions and +typed transactions. Typed transaction receipts are technically 'opaque' and have the data +wrapped in a byte array. However, all receipt types ultimately contain the same four +fields. Since the network protocol now introduces a deviation from the encoding used by +consensus by removing the bloom filter, there is no need to replicate the weird and +expensive encoding used by consensus. The proposed receipt encoding is just a flat list of +the required data fields. + ## Backwards Compatibility This EIP changes the eth protocol and requires rolling out a new version, `eth/69`. From 75925a8f1746ea4019c2c1f5eca5e225deaab27d Mon Sep 17 00:00:00 2001 From: Felix Lange Date: Tue, 14 Jan 2025 11:05:11 +0100 Subject: [PATCH 7/9] EIP-7642: rephrase --- EIPS/eip-7642.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/EIPS/eip-7642.md b/EIPS/eip-7642.md index 98461e58f120e3..d8d55abb19e3b8 100644 --- a/EIPS/eip-7642.md +++ b/EIPS/eip-7642.md @@ -93,10 +93,10 @@ CPU intensive. The bandwidth gains amount to roughly 530GiB per syncing node or In Ethereum consensus, the encoding of receipts differs between legacy transactions and typed transactions. Typed transaction receipts are technically 'opaque' and have the data wrapped in a byte array. However, all receipt types ultimately contain the same four -fields. Since the network protocol now introduces a deviation from the encoding used by -consensus by removing the bloom filter, there is no need to replicate the weird and -expensive encoding used by consensus. The proposed receipt encoding is just a flat list of -the required data fields. +fields. With the removal of the bloom filter, the networking protocol now deviates from +the encoding used by consensus, and there is no need to replicate the weird and expensive +encoding used there. The proposed receipt encoding is just a flat list of the required +data fields. ## Backwards Compatibility From 6ca875f8f0291bcece69e2188da7104ba31a53f0 Mon Sep 17 00:00:00 2001 From: Felix Lange Date: Tue, 14 Jan 2025 17:22:44 +0100 Subject: [PATCH 8/9] EIP-7642: remove 'technically' --- EIPS/eip-7642.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/EIPS/eip-7642.md b/EIPS/eip-7642.md index d8d55abb19e3b8..5954c747ae1153 100644 --- a/EIPS/eip-7642.md +++ b/EIPS/eip-7642.md @@ -91,12 +91,11 @@ CPU intensive. The bandwidth gains amount to roughly 530GiB per syncing node or 95GiB snappy compressed. In Ethereum consensus, the encoding of receipts differs between legacy transactions and -typed transactions. Typed transaction receipts are technically 'opaque' and have the data -wrapped in a byte array. However, all receipt types ultimately contain the same four -fields. With the removal of the bloom filter, the networking protocol now deviates from -the encoding used by consensus, and there is no need to replicate the weird and expensive -encoding used there. The proposed receipt encoding is just a flat list of the required -data fields. +typed transactions. Typed transaction receipts are 'opaque' and have the data wrapped in a +byte array. However, all receipt types ultimately contain the same four fields. With the +removal of the bloom filter, the networking protocol now deviates from the encoding used +by consensus, and there is no need to replicate the weird and expensive encoding used +there. The proposed receipt encoding is just a flat list of the required data fields. ## Backwards Compatibility From fe08623157f6e983b601ad865e087f0e0975c96e Mon Sep 17 00:00:00 2001 From: Felix Lange Date: Tue, 14 Jan 2025 17:24:26 +0100 Subject: [PATCH 9/9] EIP-7642: fixup encoding --- EIPS/eip-7642.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EIPS/eip-7642.md b/EIPS/eip-7642.md index 5954c747ae1153..e34f1d37c7cda5 100644 --- a/EIPS/eip-7642.md +++ b/EIPS/eip-7642.md @@ -51,7 +51,7 @@ Modify the encoding for receipts in the `Receipts (0x10)` message as follows: - (eth/68): `receipt = {legacy-receipt, typed-receipt}` with ``` -typed-receipt = tx-type || legacy-receipt +typed-receipt = tx-type || rlp(legacy-receipt) legacy-receipt = [ post-state-or-status: {B_32, {0, 1}},