From f3a128af8a218c81e92c27c99ac8432aeda87687 Mon Sep 17 00:00:00 2001 From: ian Date: Thu, 3 Mar 2022 15:27:45 +0800 Subject: [PATCH 1/2] rfc(0004): update the network messages. Use the real messages used in the protocol. --- .../0004-ckb-block-sync.md | 60 +++++++++---------- .../0004-ckb-block-sync.zh.md | 27 ++++----- 2 files changed, 41 insertions(+), 46 deletions(-) diff --git a/rfcs/0004-ckb-block-sync/0004-ckb-block-sync.md b/rfcs/0004-ckb-block-sync/0004-ckb-block-sync.md index 982dc4d1..52a2526e 100644 --- a/rfcs/0004-ckb-block-sync/0004-ckb-block-sync.md +++ b/rfcs/0004-ckb-block-sync/0004-ckb-block-sync.md @@ -13,7 +13,7 @@ Glossary of Terms - Chain: a list of blocks starting with genesis block and consisted of successive blocks. - Best Chain: a chain with the most accumulated PoW, and starting with a common genesis block which nodes agree with the consensus. - Best Header Chain: a chain with the most PoW and consisted only of blocks in the status of Connected, Downloaded or Accepted. Please refer to block status part for more details. -- Tip: the latest block of a chain and Tip can be used to determine a specific chain. +- Tip: the latest block of a chain and Tip can be used to determine a specific chain. - Best Chain Tip: the tip of Best Chain. ## Abstract @@ -22,9 +22,9 @@ Block synchronization **must** be performed in stages with [Bitcoin Headers Firs 1. Connecting Header: Get block header, and validate format and PoW. 2. Downloading Block: Get and validate the complete block. Transactions in ancestor blocks are not required. -3. Accepting Block: Validate the block in the context of the chain. +3. Accepting Block: Validate the block in the context of the chain. -The purpose of stage execution is trying to preclude most of the attacks with the least cost. For example, in the first step, header connecting only accounts for 5% workload while there would be 95% possibility to say the block is valid. +The purpose of stage execution is trying to preclude most of the attacks with the least cost. For example, in the first step, header connecting only accounts for 5% workload while there would be 95% possibility to say the block is valid. According to the execution stages, there are 5 statuses of blocks: @@ -38,25 +38,25 @@ Block status is propagated from the previous block to the later ones. Using the Initially, Genesis block is in status Accepted and the rest is in status Unknown. -Below figures are used to indicate blocks in different status later on. +Below figures are used to indicate blocks in different status later on. ![](images/block-status.jpg "Block Status") -Genesis block of the nodes synchronizing **must be** the same, and all blocks can be constructed as a tree with the genesis block being the root. Blocks will be removed if they cannot connect to the root eventually. +Genesis block of the nodes synchronizing **must be** the same, and all blocks can be constructed as a tree with the genesis block being the root. Blocks will be removed if they cannot connect to the root eventually. -Every participating node forms its local status tree where the chain consisting of Accepted blocks with the most PoW is considered as Best Chain. The chain that consists of blocks in the status of connected, downloaded or accepted with the most PoW is Best Header Chain. +Every participating node forms its local status tree where the chain consisting of Accepted blocks with the most PoW is considered as Best Chain. The chain that consists of blocks in the status of connected, downloaded or accepted with the most PoW is Best Header Chain. -The graph below is an example of Status Tree formed by Alice and blocks signed with name Alice is this node's current Best Chain Tip. +The graph below is an example of Status Tree formed by Alice and blocks signed with name Alice is this node's current Best Chain Tip. ![](images/status-tree.jpg "Status Tree by Alice") - + ## Connecting Header Headers first synchronization helps to validate PoW with the least cost. Since it costs the same work to construct PoW whether the included transactions are valid or not, attackers may use other more efficient ways. It means it's highly possible to regard the whole block as valid when the PoW is valid. This is why headers first synchronization would avoid resource-wasting on invalid blocks. Because of the low cost, Headers synchronization can be processed in parallel with all peers and construct a highly reliable global graph. In this way, block downloading can be scheduled in the most efficient way to avoid wasting resource on lower PoW branch. -The goal of connecting header is demonstrated using the following example. When Alice connects to Bob, Alice asks Bob to send all block headers in Bob's Best Chain but not in Alice's **Best Header Chain** and then validate them to decide the blocks status are either Connected or Invalid. +The goal of connecting header is demonstrated using the following example. When Alice connects to Bob, Alice asks Bob to send all block headers in Bob's Best Chain but not in Alice's **Best Header Chain** and then validate them to decide the blocks status are either Connected or Invalid. When Alice connects header, keeping Best Header Chain Tip updated could help to decrease numbers of receiving headers already existed. @@ -64,11 +64,11 @@ When Alice connects header, keeping Best Header Chain Tip updated could help to The graph above instructs the process of connecting headers. After a round of connecting headers, nodes are supposed to keep up-to-date using new block notification. -Take Alice and Bob above as an example, firstly Alice samples blocks from her Best Header Chain and sent the hashes to Bob. The basic principle of sampling is that later blocks are more possible to be selected than early blocks. For example, choose latest 10 blocks from the chain, then sample other blocks backward with 2's exponential increased intervals, a.k.a, 2, 4, 8, and etc. The list of hashes of the sampled blocks is called a Locator. In the following figure, the undimmed blocks are sampled. The genesis block should be always in the Locator. +Take Alice and Bob above as an example, firstly Alice samples blocks from her Best Header Chain and sent the hashes to Bob. The basic principle of sampling is that later blocks are more possible to be selected than early blocks. For example, choose latest 10 blocks from the chain, then sample other blocks backward with 2's exponential increased intervals, a.k.a, 2, 4, 8, and etc. The list of hashes of the sampled blocks is called a Locator. In the following figure, the undimmed blocks are sampled. The genesis block should be always in the Locator. ![](images/locator.jpg) -Bob can get the latest common block between these two chains according to Locator and his own Best Chain. Because the genesis block is identical, there must be such kind of block. Bob will send all block headers from the common block to his Best Chain Tip to Alice. +Bob can get the latest common block between these two chains according to Locator and his own Best Chain. Because the genesis block is identical, there must be such kind of block. Bob will send all block headers from the common block to his Best Chain Tip to Alice. ![](images/connect-header-conditions.jpg) @@ -82,11 +82,11 @@ If there are too many blocks to send, pagination is required. Bob sends the firs Alice could observe Bob's present Best Chain Tip, which is the last block received during each round of synchronization. If Alice's Best Header Chain Tip is exactly Bob's Best Chain Tip, Alice couldn't observe Bob's present Best Chain because Bob has no block headers to send. Therefore, it should start building from the parent block of Best Header Chain Tip when sending the first request in each round. -In the following cases, a new round of connection block header synchronization must be performed. +In the following cases, a new round of connection block header synchronization must be performed. - Received a new block notification from the others, but the parent block status of the new block is Unknown. -The following exceptions may occur when connecting a block header: +The following exceptions may occur when connecting a block header: - Alice observed that Bob's Best Chain Tip has not been updated for a long time, or its timestamp is old. In this case, Bob does not provide valuable data. When the number of connections reaches a limit, Bob could be disconnected first. - Alice observed that the status of Bob's Best Chain Tip is Invalid. This can be found in any page without waiting for the end of a round of Connect Head. There, Bob is on an invalid branch, Alice can stop synchronizing with Bob and add Bob to the blacklist. @@ -95,10 +95,10 @@ The following exceptions may occur when connecting a block header: Upon receiving the block header message, the format should be verified first. - The blocks in the message are continuous. -- The status of all blocks and the parent block of the first block are not Invalid in the local Status Tree. +- The status of all blocks and the parent block of the first block are not Invalid in the local Status Tree. - The status of the parent block of the first block is not Unknown in the local Status Tree, which means Orphan Block will not be processed in synchronizing. -In this stage, verification includes checking if block header satisfies the consensus rules and if Pow is valid or not. Since Orphan Blocks are not processed, difficulty adjustment can be verified as well. +In this stage, verification includes checking if block header satisfies the consensus rules and if Pow is valid or not. Since Orphan Blocks are not processed, difficulty adjustment can be verified as well. ![](images/connect-header-status.jpg) @@ -165,7 +165,7 @@ How to send the announcement is determined by connection negotiated parameters a When receiving a new block announcement, there may be a situation the parent block's status is Unknown, also called Orphan Block. If so, a new round of Connecting Header is required immediately. When a Compact Block is received, and its parent block is the local Best Chain Tip, then the full block may be recovered from the transaction pool. If the recovery succeeds, the work of these three stages can be compacted into one. Otherwise, it falls back to a header-only announcement. -## Synchronization Status +## Synchronization Status ### Configuration - `GENESIS_HASH`: hash of genesis block @@ -186,35 +186,33 @@ Each connection peer should store: Only related message and fields are listed here. See completed definition and documentation in the reference implementation. -The message passing is completely asynchronous. For example, sending `getheaders` does not block other requests. Also, there is no need to guarantee the order relationship between the requests and the responses. For example, node A sends `getheaders` and `getdata` to B, and B can replies `block` firstly, and then `headers` to A. +The message passing is completely asynchronous. For example, sending `GetHeaders` does not block other requests. Also, there is no need to guarantee the order relationship between the requests and the responses. For example, node A sends `GetHeaders` and `GetBlocks` to B, and B can replies `SendBlock` firstly, and then `SendHeaders` to A. -Compact Block [^1] messages `cmpctblock` and `getblocktxn` will be described in related Compact Block documentation. +Compact Block [^1] messages will be described in related Compact Block documentation. -### getheaders +### GetHeaders -It is used to request a block header from a peer in stage Connecting Header. The first-page request, and subsequent pages request can share the same getheaders message format. The difference between them is that the first page requests generate a Locator from the parent block of the local Best Header Chain Tip, and the subsequent page request generates the Locator using the last block in the last received page. +It is used to request a block header from a peer in stage Connecting Header. The first-page request, and subsequent pages request can share the same GetHeaders message format. The difference between them is that the first page requests generate a Locator from the parent block of the local Best Header Chain Tip, and the subsequent page request generates the Locator using the last block in the last received page. -- `locator`: Sampled hashes of the already known blocks +- `hash_stop`: tells peer to early return when building `SendHeaders` response. +- `block_locator_hashes`: Sampled hashes of the already known blocks -### headers +### SendHeaders -It is used to reply `getheaders` and announce new blocks. There is no difference in processing logic, but if an Orphan Block is founded when the number of block headers is less than `MAX_BLOCKS_TO_ANNOUNCE`, a new round of Connecting Header is required. If the number of block `headers` received equals is equal to `MAX_HEADERS_RESULTS`, it indicates that there are more blocks to request. +It is used to reply `GetHeaders`. It returns a headers list containing the headers of blocks starting right after the last common hash via the Locator, up to `hash_stop` or `MAX_BLOCKS_TO_ANNOUNCE` blocks, whichever comes first. - `headers`：block headers list -### getdata +### GetBlocks It is used in Downloading Block stage. -- `inventory`: object lists for download, with following fields in each list element: - - `type`: type of the object, only "block" here - - `hash`: hash of the object as identity +- `block_hashes`: list of block hashes to download. -### block +### SendBlock -It is used to reply block downloading request of `getdata` +It is used to reply block downloading request of `GetBlocks` -- `header` block header -- `transactions` transaction list +- `block`: the requested block content [^1]: Compact Block is a technique for compressing and transferring complete blocks. It is based on the fact that when a new block is propagated, the transactions should already be in the pool of other nodes. Under this circumstances, Compact Block only contains the list of transaction txid list and complete transactions which are predicated unknown to the peers. The receiver can recover the complete block using the transaction pool. Please refer to [Block and Compact Block Structure](../0020-ckb-consensus-protocol/0020-ckb-consensus-protocol.md#block-and-compact-block-structure) and related Bitcoin [BIP](https://github.com/bitcoin/bips/blob/master/bip-0152.mediawiki) for details. diff --git a/rfcs/0004-ckb-block-sync/0004-ckb-block-sync.zh.md b/rfcs/0004-ckb-block-sync/0004-ckb-block-sync.zh.md index 6862b464..89819859 100644 --- a/rfcs/0004-ckb-block-sync/0004-ckb-block-sync.zh.md +++ b/rfcs/0004-ckb-block-sync/0004-ckb-block-sync.zh.md @@ -189,37 +189,34 @@ Bob 根据 Locator 和自己的 Best Chain 可以找出两条链的最后一个 具体消息定义见参考实现，这里只列出同步涉及到的消息和必要的一些字段和描述。 -消息的发送是完全异步的，比如发送 `getheaders` 并不需要等待对方回复 `headers` 再发送其它请求，也不需要保证请求和回复的顺序关系，比如节点 A 发送了 `getheaders` 和 `getdata` 给 B，B 可以先发送 `block`，然后再发送 `headers` 给 A。 +消息的发送是完全异步的，比如发送 `GetHeaders` 并不需要等待对方回复 `SendHeaders` 再发送其它请求，也不需要保证请求和回复的顺序关系，比如节点 A 发送了 `GetHeaders` 和 `GetBlocks` 给 B，B 可以先发送 `SendBlock`，然后再发送 `SendHeaders` 给 A。 -Compact Block [^1] 需要使用到的消息 `cmpctblock` 和 `getblocktxn` 会在 Compact Block 相关文档中说明。 +Compact Block [^1] 需要使用到的消息会在 Compact Block 相关文档中说明。 -### getheaders +### GetHeaders 用于连接块头时向邻居节点请求块头。请求第一页，和收到后续页使用相同的 getheaders 消息，区别是第一页是给本地的 Best Header Chain Tip 的父块生成 Locator，而后续页是使用上一页的最后一个块生成 Locator。 -- `locator`: 对 Chain 上块采样，得到的哈希列表 +- `hash_stop`: 通知对端构建 `SendHeaders` 时如果处理到指定 hash 的区块应该提前返回。 +- `block_locator_hashes`: 对 Chain 上块采样，得到的哈希列表。 -### headers +### SendHeaders -用于回复 `getheaders` 和通知新块，处理逻辑没有区别，只是当块头数量小于 `MAX_BLOCKS_TO_ANNOUNCE` 时如果发现有 Orphan Block，因为可能是新块通知，所以需要做一次连接块同步。收到 `headers` 如果块头数量等于 `MAX_HEADERS_RESULTS` 表示还有更多的块需要请求。 +用于回复 `GetHeaders`。返回块头列表。从 Locator 获得的最后一个共同块开始到 `hash_stop` 或者数量达到 `MAX_BLOCKS_TO_ANNOUNCE`。两个条件满足任意一个就必须停止添加并返回结果。 - `headers`：块头列表 -### getdata +### GetBlocks 用于下载块阶段 -- `inventory`: 要下载对象列表，每个成员包含字段 - - `type`: 下载对象的类型，这里只用到块 - - `hash`: 使用对象哈希做标识符 +- `block_hashes`: 要下载的区块哈希列表 -### block +### SendBlock -回复 `getdata` 的块下载请求 - -- `header` 块头 -- `transactions` 交易列表 +回复 `GetBlocks` 的块下载请求 +- `block`: 请求的块的完整内容 [^1]: Compact Block 是种压缩传输完整块的技术。它基于在传播新块时，其中的交易应该都已经在对方节点的交易池中。这时只需要包含 交易 txid 列表，和预测对方可能没有的交易的完整信息，接收方就能基于交易池恢复出完整的交易。详细请查阅 [Block and Compact Block Structure](../0020-ckb-consensus-protocol/0020-ckb-consensus-protocol.md#block-and-compact-block-structure) 和 Bitcoin 相关 [BIP](https://github.com/bitcoin/bips/blob/master/bip-0152.mediawiki)。 From 30448d4f77c0ceef4baf96a4978498e2aa5959aa Mon Sep 17 00:00:00 2001 From: ian Date: Wed, 3 Apr 2024 13:50:02 +0000 Subject: [PATCH 2/2] rfc(0004): mark rfc0004 as Informational and Final --- README.md | 2 +- rfcs/0004-ckb-block-sync/0004-ckb-block-sync.md | 6 ++++-- rfcs/0004-ckb-block-sync/0004-ckb-block-sync.zh.md | 5 +++-- 3 files changed, 8 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 67d8b9f2..37bec883 100644 --- a/README.md +++ b/README.md @@ -58,7 +58,7 @@ The maintainers of RFCs and the community will review the PR, and you should upd | [1](rfcs/0001-positioning) | [The Nervos Network Positioning Paper](rfcs/0001-positioning/0001-positioning.md) | The Nervos Team | Informational | Final | | [2](rfcs/0002-ckb) | [Nervos CKB: A Common Knowledge Base for Crypto-Economy](rfcs/0002-ckb/0002-ckb.md) | Jan Xie | Informational | Final | | [3](rfcs/0003-ckb-vm) | [CKB-VM](rfcs/0003-ckb-vm/0003-ckb-vm.md) | Xuejie Xiao | Informational | Final | -| [4](rfcs/0004-ckb-block-sync) | [CKB Block Synchronization Protocol](rfcs/0004-ckb-block-sync/0004-ckb-block-sync.md) | Ian Yang | Standards Track | Proposal | +| [4](rfcs/0004-ckb-block-sync) | [CKB Block Synchronization Protocol](rfcs/0004-ckb-block-sync/0004-ckb-block-sync.md) | Ian Yang | Informational | Final | | [5](rfcs/0005-priviledged-mode) | [Privileged architecture support for CKB VM](rfcs/0005-priviledged-mode/0005-priviledged-mode.md) | Xuejie Xiao | Informational | Withdrawn | | [6](rfcs/0006-merkle-tree) | [Merkle Tree for Static Data](rfcs/0006-merkle-tree/0006-merkle-tree.md) | Ke Wang | Standards Track | Active | | [7](rfcs/0007-scoring-system-and-network-security) | [P2P Scoring System And Network Security](rfcs/0007-scoring-system-and-network-security/0007-scoring-system-and-network-security.md) | Jinyang Jiang | Standards Track | Withdrawn | diff --git a/rfcs/0004-ckb-block-sync/0004-ckb-block-sync.md b/rfcs/0004-ckb-block-sync/0004-ckb-block-sync.md index 52a2526e..323ad74f 100644 --- a/rfcs/0004-ckb-block-sync/0004-ckb-block-sync.md +++ b/rfcs/0004-ckb-block-sync/0004-ckb-block-sync.md @@ -1,8 +1,9 @@ --- Number: "0004" -Category: Standards Track -Status: Proposal +Category: Information +Status: Final Author: Ian Yang <@doitian> +Organization: Nervos Foundation Created: 2018-07-25 --- @@ -15,6 +16,7 @@ Glossary of Terms - Best Header Chain: a chain with the most PoW and consisted only of blocks in the status of Connected, Downloaded or Accepted. Please refer to block status part for more details. - Tip: the latest block of a chain and Tip can be used to determine a specific chain. - Best Chain Tip: the tip of Best Chain. +- Chain: a list of blocks starting with genesis block and consisted of successive blocks. ## Abstract diff --git a/rfcs/0004-ckb-block-sync/0004-ckb-block-sync.zh.md b/rfcs/0004-ckb-block-sync/0004-ckb-block-sync.zh.md index 89819859..de69750b 100644 --- a/rfcs/0004-ckb-block-sync/0004-ckb-block-sync.zh.md +++ b/rfcs/0004-ckb-block-sync/0004-ckb-block-sync.zh.md @@ -1,8 +1,9 @@ --- Number: "0004" -Category: Standards Track -Status: Proposal +Category: Information +Status: Final Author: Ian Yang <@doitian> +Organization: Nervos Foundation Created: 2018-07-25 ---