diff --git a/BEPs/BEP-396.md b/BEPs/BEP-396.md new file mode 100644 index 00000000..e7c20fb0 --- /dev/null +++ b/BEPs/BEP-396.md @@ -0,0 +1,261 @@ +
+  BEP: 396
+  Title: Accelerate Block Execution by TxDAG
+  Status: Draft
+  Type: Standards
+  Created: 2024-06-17
+  Author:  galaio (@galaio), zoro (@unclezoro), zzzckck (@zzzckck), sunny2022da (@sunny2022da), owen (@owen-reorg)
+  Description: Generate proposal block's tx dependency DAG to accelerate block execution when syncing
+  Discussions: https://forum.bnbchain.org/t/bep-396-accelerate-block-execution-by-txdag/2869
+
+ +# BEP-396: Accelerate Block Execution by TxDAG + +## 1. Summary + +More and more blockchains are aware of the importance of parallel execution for performance improvement. Common parallel EVM processor will optimistically execute transactions in parallel. When conflicts are found, they can only be re-executed to obtain the correct results, which wastes a lot of computing resources. And the worst case may cause performance degradation even worse than sequential execution. + +This BEP introduces a transaction dependency graph, `TxDAG`, which provides accurate dependencies in the form of a directed acyclic graph (DAG) for parallelization instead of optimistically estimated dependencies when executing blocks, will greatly improve the efficiency of parallel execution by eliminating the re-execution caused by TX conflict. + +## 2. Abstract + +This BEP introduces the transaction dependency graph, `TxDAG`, which is used to accelerate the validation of blockchain blocks. This structure has strict format requirements to ensure the correctness of dependencies and is compatible with multiple-chain execution processors. + +## 3. Status + +Draft + +## 4. Motivation + +As the scale of transactions and state data grows, performance improvement is the key to providing a good user experience for blockchain. One of the current main optimization directions is to use an optimistic transaction parallel EVM processor, which optimistically assumes that transactions have no dependencies and executes them in parallel, then continues to retry execution after verification fails until all transactions are verified. + +However, these additional retries waste a lot of computing resources. The read-write set generated by transaction execution can easily build a dependency graph between transactions, called `TxDAG`. This data can be used in the validating block phase to accelerate block execution and further improve the overall performance of the blockchain. + +## 5. Specification + +In BSC, the block interval is 3s, that is, the more transactions are executed within 3s, the better the chain performance and the greater the throughput. + +The two parts that take the most time in the 3s are the validating block and the mining block. Validators often validate the correctness of the current block and continue to mine the next block based on the latest state. + +To improve the performance of block execution, transactions need to be executed in parallel as much as possible to make full use of computing and IO resources. Imagine that if the dependencies of transactions are known, then the parallel execution path can be directly determined, but the dependencies need to be obtained by the read-write set after the transaction pre-execution (static analysis is not within the scope of this BEP). + +So two different TX execution strategies can be identified: + +![](./assets/BEP-396/pic1.png) + +In the mining phase, TXs of blocks are executed in parallel or serial to generate `TxDAG`. In the validating phase, `TxDAG` is used to further improve the parallel execution efficiency. + +### 5.1 Tx Dependency + +Assume that the execution of each transaction is atomic, and the input and output of the execution generate a state read-write set. In concurrent processing, read-only access to the resources is safe, but write operations require exclusive resources. + +At the same time, all transactions in the blockchain have a preset order, and transactions with lower TxIndex have higher execution priority. + +Therefore, the dependency constraint goes to, if transaction Txi writes `StateA` first, then transaction Txj reads `StateA` and j>i is satisfied. That is to say, Txj depends on Txi, Txj must be executed after Txi. + +If a transaction reads and writes the same value at a memory location, the location can be considered read-only. + +### 5.2 TxDAG Structure + +The structure of `TxDAG` is defined as follows. + +```go +// TxDAGType Used to extend TxDAG and customize a new DAG structure +const ( + EmptyTxDAGType byte = iota + PlainTxDAGType +) + +type TxDAG interface { + // Type return TxDAG type + Type() byte + + // TxDep query TxDeps from TxDAG + TxDep(int) *TxDep + + // TxCount return tx count + TxCount() int +} + + +func EncodeTxDAG(dag TxDAG) ([]byte, error) { + if dag == nil { + return nil, errors.New("input nil TxDAG") + } + var buf bytes.Buffer + buf.WriteByte(dag.Type()) + if err := rlp.Encode(&buf, dag.Inner()); err != nil { + return nil, err + } + return buf.Bytes(), nil +} + +func DecodeTxDAG(enc []byte) (TxDAG, error) { + if len(enc) <= 1 { + return nil, errors.New("too short TxDAG bytes") + } + + switch enc[0] { + case EmptyTxDAGType: + return NewEmptyTxDAG(), nil + case PlainTxDAGType: + dag := new(PlainTxDAG) + if err := rlp.DecodeBytes(enc[1:], dag); err != nil { + return nil, err + } + return dag, nil + default: + return nil, errors.New("unsupported TxDAG bytes") + } +} + +const ( + // NonDependentRelFlag indicates that the txs described is non-dependent + // and is used to reduce storage when there are a large number of dependencies. + NonDependentRelFlag uint8 = 0x01 + // ExcludedTxFlag indicates that the tx is excluded from TxDAG, user should execute them in sequence. + // These excluded transactions should be consecutive in the head or tail. + ExcludedTxFlag uint8 = 0x02 + TxDepFlagMask = NonDependentRelFlag | ExcludedTxFlag +) + +// TxDep store the current tx dependency relation with other txs +type TxDep struct { + TxIndexes []uint64 + // Flags may has multi flag meaning, ref NonDependentRelFlag, ExcludedTxFlag. + Flags *uint8 `rlp:"optional"` +} + +// EmptyTxDAG indicate that execute txs in sequence +// It means no transactions or need timely distribute transaction fees +// it only keep partial serial execution when tx cannot delay the distribution or just execute txs in sequence +type EmptyTxDAG struct { +} + +func (d *EmptyTxDAG) Type() byte { + return EmptyTxDAGType +} + +func (d *EmptyTxDAG) TxDep(int) *TxDep { + return nil +} + +func (d *EmptyTxDAG) TxCount() int { + return 0 +} + +// PlainTxDAG indicate how to use the dependency of txs, and delay the distribution of GasFee +type PlainTxDAG struct { + // Tx Dependency List, the list index is equal to TxIndex + TxDeps []TxDep +} + +func (d *PlainTxDAG) Type() byte { + return PlainTxDAGType +} + +func (d *PlainTxDAG) TxDep(i int) *TxDep { + return &d.TxDeps[i] +} + +func (d *PlainTxDAG) TxCount() int { + return len(d.TxDeps) +} +``` + +This BEP can use `PlainTxDAGType` to extend more types. + +There is also a potential `TxDAG` "compression algorithm" to further optimize the size. For the DAG describing the dependencies, some are redundant because it only needs to ensure that the final execution path is consistent. + +![](./assets/BEP-396/pic2.png) + +Specific rules: If Txj depends on Txi, j>i, then Txj can ignore the same dependencies as Txi, and the execution path of Txj after the update remains the same as before. + +This can keep the size of `TxDAG` as small as possible and avoid generating too large `TxDAG` and wasting network bandwidth. + +### 5.3 TxDAG Validation + +Since there is a preset order of transactions in the block, Txi can't depend on Txj (i