From bdd6099bc2ea1525ad0df09af6b6408e60e7bbf5 Mon Sep 17 00:00:00 2001 From: Fangyu Gai Date: Tue, 5 Nov 2024 23:04:38 +0800 Subject: [PATCH 1/6] init spec --- docs/commit-pub-rand.md | 98 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 98 insertions(+) create mode 100644 docs/commit-pub-rand.md diff --git a/docs/commit-pub-rand.md b/docs/commit-pub-rand.md new file mode 100644 index 00000000..28e1c425 --- /dev/null +++ b/docs/commit-pub-rand.md @@ -0,0 +1,98 @@ +# Public Randomness Commit Specification + +## Overview + +The finality provider periodically commits public randomness to the consumer +chain to be used for future block finalization. This document specifies the +process of committing public randomness. + +## Commit Process + +A public randomness commit is composed by a merkle root of a list of public, +along with the start height and the number of randomness contained in the merkle +tree. +Public randomness is an essential component of finality. The finality provider +must commit randomness before it can send finality votes, ensuring the randomness +is available when needed. + +To achieve this, randomness must be committed well in advance. +The finality provider runs a loop to check whether it needs to make a new commit +periodically. In particualar, the following statement is checked: + +```go +if lastCommittedHeight < currentHeight + uint64(MinRandHeightGap) +``` + +If the statement is true, a new commit should be made to ensure sufficient +randomness is available for future blocks. + +## Determining MinRandHeightGap + +The value of `MinRandHeightGap` must account for the BTC-timestamping protocol, +which activates randomness for a specific height after the committed epoch is +BTC-timestamped. Here's an example: + +- Consumer chain receives a commit with: + - Start height: 100 + - Number of randomness values: 1000 + - Current epoch: 10 +- This means randomness for heights [100, 1099] becomes available after epoch 10 + is finalized + +The BTC-timestamping protocol requires: + +- 100 BTC blocks for epoch finalization +- ≈ 1000 minutes (17 hours) at 10-minute average block time +- With consumer chain blocks every 10 seconds, this equals approximately 6,000 + blocks + +Therefore, + +- `MinRandHeightGap` should be > 6,000 to ensure randomness is always available +- Recommended production value: > 10,000 to provide additional safety margin + +## Determining commit start height + +The general rule of determing the start height of a commit is to the heights +of the randomness are consecutive. In particular, + +1. For first-time commit: + - `startHeight = baseHeight + 1` + - where `baseHeight` is a future height which is estimated based on the + BTC-timestamping time. +2. For subsequent commit: + - `startHeight = lastCommittedHeight + 1` + - Note that the finality provider might have very long down time. In this + case, we can consider it as the same case as the first-time commit + +Note that `startHeight` should not be higher than `finalityActivationHeight`, +a parameter defined in Babylon. Therefore, + +```go +startHeight = max(startHeight, finalityActivationHeight) +``` + +Also note that consecutiveness is not enforced by the consumer chain but it +is required that different commits should not have overlaps. + +## Determining the number of randomness + +The number of randomness is specified in the config `NumPubRand`. A general +strategy is that the value should be as large as possible. This is because each +commit to the consumer chain costs gas. + +However, in real life, this stategy might not always gain due to the following +reasons: + +- a finality provider might not have voting power for every block. Randomness + for those heights is a waste. +- generating more randomness leads to a larger merkle proof size which will be + used for sending finality votes. +- generating randomness and saving the merkle proofs require time. + +Additionally, given that the end height of a commit equals to +`startHeight + NumPubRand - 1`, we should ensure that the condition +`lastCommittedHeight > currentHeight + uint64(MinRandHeightGap)` can hold for +a long period of time to avoid frequent commit of randomness. +In real life, the value of `NumPubRand` should be much larger than +`MinRandHeightGap`, e.g., `NumPubRand = 2 * MinRandHeightGap`. From bb1137a0632dfa9860184d0c28ad36bab22f6dde Mon Sep 17 00:00:00 2001 From: Fangyu Gai Date: Wed, 6 Nov 2024 15:56:31 +0800 Subject: [PATCH 2/6] minor --- docs/commit-pub-rand.md | 85 ++++++++++++++++++++++++----------------- 1 file changed, 49 insertions(+), 36 deletions(-) diff --git a/docs/commit-pub-rand.md b/docs/commit-pub-rand.md index 28e1c425..9e5414a9 100644 --- a/docs/commit-pub-rand.md +++ b/docs/commit-pub-rand.md @@ -6,33 +6,47 @@ The finality provider periodically commits public randomness to the consumer chain to be used for future block finalization. This document specifies the process of committing public randomness. +## Public Randomness Commit + +A public randomness commit is essentially a list of public +randomness, each committed to a specific height. In particular, it consists of: + +- a merkle root containing a list of public randomness values, +- a start height, indicating from which height the randomness starts, and +- the number of randomness contained in the merkle tree. + ## Commit Process -A public randomness commit is composed by a merkle root of a list of public, -along with the start height and the number of randomness contained in the merkle -tree. -Public randomness is an essential component of finality. The finality provider -must commit randomness before it can send finality votes, ensuring the randomness -is available when needed. +Public randomness is an essential component of finality. It should be +committed before finality votes can be sent. Otherwise, the finality provider +looses voting power for this height. -To achieve this, randomness must be committed well in advance. -The finality provider runs a loop to check whether it needs to make a new commit -periodically. In particualar, the following statement is checked: +To this end, when a finality provider is started, it runs a loop to periodically +check whether it needs to make a new commit. In particualar, +the following statement is checked: ```go if lastCommittedHeight < currentHeight + uint64(MinRandHeightGap) ``` +where: + +- `lastCommittedHeight` is the end height (`startHeight + numRand - 1`) +from the latest public randomness commit recorded on the consumer chain +- `currentHeight` is the current height of the consumer chain +- `MinRandHeightGap` is a configuration value, which measures when to make a + new commit + If the statement is true, a new commit should be made to ensure sufficient randomness is available for future blocks. -## Determining MinRandHeightGap +### Determining MinRandHeightGap -The value of `MinRandHeightGap` must account for the BTC-timestamping protocol, -which activates randomness for a specific height after the committed epoch is -BTC-timestamped. Here's an example: +The value of `MinRandHeightGap` must account for BTC-timestamping +delays, which is needed to activate the randomness for a specific height +after the committed epoch is BTC-timestamped. Here's an example: -- Consumer chain receives a commit with: +- The consumer chain receives a commit with: - Start height: 100 - Number of randomness values: 1000 - Current epoch: 10 @@ -51,44 +65,43 @@ Therefore, - `MinRandHeightGap` should be > 6,000 to ensure randomness is always available - Recommended production value: > 10,000 to provide additional safety margin -## Determining commit start height +### Determining Start Height -The general rule of determing the start height of a commit is to the heights -of the randomness are consecutive. In particular, +To determine the start height of a commit: 1. For first-time commit: - - `startHeight = baseHeight + 1` + - `startHeight = baseHeight + 1`, - where `baseHeight` is a future height which is estimated based on the - BTC-timestamping time. + BTC-timestamping delays. 2. For subsequent commit: - - `startHeight = lastCommittedHeight + 1` - - Note that the finality provider might have very long down time. In this - case, we can consider it as the same case as the first-time commit + - `startHeight = lastCommittedHeight + 1`, + - where `lastCommittedHeight` is obtained from the consumer chain. -Note that `startHeight` should not be higher than `finalityActivationHeight`, -a parameter defined in Babylon. Therefore, +The `baseHeight` can be specified via configuration or CLI options. -```go -startHeight = max(startHeight, finalityActivationHeight) -``` +**Important Notes:** -Also note that consecutiveness is not enforced by the consumer chain but it -is required that different commits should not have overlaps. +- After long downtime, treat as first-time commit by specifying `baseHeight`. +- Consecutiveness across commits is not enforced by the system but + different commits must not overlap. +- `startHeight` should not be higher than `finalityActivationHeight`, +a parameter defined in Babylon. Therefore, +`startHeight = max(startHeight, finalityActivationHeight)`. -## Determining the number of randomness +### Determining the Number of Randomness -The number of randomness is specified in the config `NumPubRand`. A general -strategy is that the value should be as large as possible. This is because each -commit to the consumer chain costs gas. +The number of randomness contained in a commit is specified in the config +`NumPubRand`. A general strategy is that the value should be as large +as possible. This is because each commit to the consumer chain costs gas. However, in real life, this stategy might not always gain due to the following reasons: -- a finality provider might not have voting power for every block. Randomness +- A finality provider might not have voting power for every block. Randomness for those heights is a waste. -- generating more randomness leads to a larger merkle proof size which will be +- Generating more randomness leads to a larger merkle proof size which will be used for sending finality votes. -- generating randomness and saving the merkle proofs require time. +- Generating randomness and saving the merkle proofs require time. Additionally, given that the end height of a commit equals to `startHeight + NumPubRand - 1`, we should ensure that the condition From b9859777ade53293a3fed39b15a93dff771cb8b4 Mon Sep 17 00:00:00 2001 From: Fangyu Gai Date: Wed, 6 Nov 2024 20:30:36 +0800 Subject: [PATCH 3/6] changelog --- CHANGELOG.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1b230bb5..ebfda584 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -37,6 +37,10 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) ## Unreleased +### Documentation + +* [#117](https://github.com/babylonlabs-io/finality-provider/pull/117) Spec of commit public randomness + ## v0.10.0 ### Improvements From e68a16bb661adbef72580f1810dc88fc1855def3 Mon Sep 17 00:00:00 2001 From: Fangyu Gai Date: Thu, 7 Nov 2024 13:52:08 +0800 Subject: [PATCH 4/6] add generating commit --- docs/commit-pub-rand.md | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/docs/commit-pub-rand.md b/docs/commit-pub-rand.md index 9e5414a9..8ae49d56 100644 --- a/docs/commit-pub-rand.md +++ b/docs/commit-pub-rand.md @@ -6,7 +6,9 @@ The finality provider periodically commits public randomness to the consumer chain to be used for future block finalization. This document specifies the process of committing public randomness. -## Public Randomness Commit +## Commit Process + +### Generating a Commit A public randomness commit is essentially a list of public randomness, each committed to a specific height. In particular, it consists of: @@ -15,7 +17,18 @@ randomness, each committed to a specific height. In particular, it consists of: - a start height, indicating from which height the randomness starts, and - the number of randomness contained in the merkle tree. -## Commit Process +To generate a new commit, following steps are needed: + +- Generate a list of randomness. This requires a RPC call to the EOTS manager + (eotsd) to generate a list of public randomness, each corresponding to a + specific height according to the start height and the number of randomness in + the request. +- Construct the merkle tree based on the list of randomness and save the merkle + proof of each randomness to the database indexed by height. +- Construct the commit message based on the root of the merkle tree. +- Sign the commit message and sent a transaction to Babylon. + +### Timing to Commit Public randomness is an essential component of finality. It should be committed before finality votes can be sent. Otherwise, the finality provider From a73f7457cf1fd2f845935c7ed80b6351b2900209 Mon Sep 17 00:00:00 2001 From: Fangyu Gai Date: Fri, 8 Nov 2024 20:05:32 +0800 Subject: [PATCH 5/6] add links to libraries --- docs/commit-pub-rand.md | 24 ++++++++++++++++-------- 1 file changed, 16 insertions(+), 8 deletions(-) diff --git a/docs/commit-pub-rand.md b/docs/commit-pub-rand.md index 8ae49d56..0ff1b4e9 100644 --- a/docs/commit-pub-rand.md +++ b/docs/commit-pub-rand.md @@ -10,8 +10,10 @@ process of committing public randomness. ### Generating a Commit -A public randomness commit is essentially a list of public -randomness, each committed to a specific height. In particular, it consists of: +A randomness pair is essentially a pair of `32-byte` points over `secp256k1`. +A public randomness commit is a list of public +randomness, each committed to a specific height. In particular, a commit +consists of: - a merkle root containing a list of public randomness values, - a start height, indicating from which height the randomness starts, and @@ -19,14 +21,20 @@ randomness, each committed to a specific height. In particular, it consists of: To generate a new commit, following steps are needed: -- Generate a list of randomness. This requires a RPC call to the EOTS manager +- Generate a list of randomness. This requires an RPC call to the EOTS manager (eotsd) to generate a list of public randomness, each corresponding to a specific height according to the start height and the number of randomness in - the request. -- Construct the merkle tree based on the list of randomness and save the merkle - proof of each randomness to the database indexed by height. -- Construct the commit message based on the root of the merkle tree. -- Sign the commit message and sent a transaction to Babylon. + the request. The details of the generator can be found in [eotsmanager/randgenerator](../eotsmanager/randgenerator/randgenerator.go). +- Construct the merkle tree based on the list of randomness using the CometBFT's [merkle](https://github.com/cometbft/cometbft/tree/main/crypto/merkle) + library. +- Save each randomness along with the merkle + [proof](https://github.com/cometbft/cometbft/blob/978b84614992cb009b2e37500b6b3a598665a535/crypto/merkle/proof.go#L53) + to the database indexed by height. +- Send a [Schnorr](https://github.com/btcsuite/btcd/blob/684d64ad74fed203fb846c032f2b55b3e3c36734/btcec/schnorr/signature.go#L391) + signature request to the EOTS manager over the hash of the commit + (concatenated by the start height, number of randomness, and the merkle root). +- Build the commit message ([MsgCommitPubRandList](https://github.com/babylonlabs-io/babylon/blob/aa99e2eb093e06cb9a28a58f373e8fa5f2494383/proto/babylon/finality/v1/tx.proto#L29)) + and send a transaction to Babylon. ### Timing to Commit From ac29934f95e449415ba71dfbb3f8d8303750c16e Mon Sep 17 00:00:00 2001 From: Fangyu Gai Date: Wed, 13 Nov 2024 15:43:29 +0800 Subject: [PATCH 6/6] make steps more general --- docs/commit-pub-rand.md | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/docs/commit-pub-rand.md b/docs/commit-pub-rand.md index 0ff1b4e9..81b66a6e 100644 --- a/docs/commit-pub-rand.md +++ b/docs/commit-pub-rand.md @@ -21,19 +21,18 @@ consists of: To generate a new commit, following steps are needed: -- Generate a list of randomness. This requires an RPC call to the EOTS manager +1. Generate a list of randomness. This requires an RPC call to the EOTS manager (eotsd) to generate a list of public randomness, each corresponding to a specific height according to the start height and the number of randomness in - the request. The details of the generator can be found in [eotsmanager/randgenerator](../eotsmanager/randgenerator/randgenerator.go). -- Construct the merkle tree based on the list of randomness using the CometBFT's [merkle](https://github.com/cometbft/cometbft/tree/main/crypto/merkle) - library. -- Save each randomness along with the merkle - [proof](https://github.com/cometbft/cometbft/blob/978b84614992cb009b2e37500b6b3a598665a535/crypto/merkle/proof.go#L53) - to the database indexed by height. -- Send a [Schnorr](https://github.com/btcsuite/btcd/blob/684d64ad74fed203fb846c032f2b55b3e3c36734/btcec/schnorr/signature.go#L391) + the request. Randomness generation is required to be deterministic. +2. Construct the merkle tree based on the list of randomness using the CometBFT's [merkle](https://github.com/cometbft/cometbft/tree/main/crypto/merkle) + library. The merkle root will be used in the commit, while each randomness + number and their merkle proofs will be used for finality vote submission + in the future. +3. Send a [Schnorr](https://github.com/btcsuite/btcd/blob/684d64ad74fed203fb846c032f2b55b3e3c36734/btcec/schnorr/signature.go#L391) signature request to the EOTS manager over the hash of the commit (concatenated by the start height, number of randomness, and the merkle root). -- Build the commit message ([MsgCommitPubRandList](https://github.com/babylonlabs-io/babylon/blob/aa99e2eb093e06cb9a28a58f373e8fa5f2494383/proto/babylon/finality/v1/tx.proto#L29)) +4. Build the commit message ([MsgCommitPubRandList](https://github.com/babylonlabs-io/babylon/blob/aa99e2eb093e06cb9a28a58f373e8fa5f2494383/proto/babylon/finality/v1/tx.proto#L29)) and send a transaction to Babylon. ### Timing to Commit