From 3c90fc85e2159c1867337d87b9f29b6c6e947a3c Mon Sep 17 00:00:00 2001 From: insumity Date: Mon, 22 Jan 2024 13:11:43 +0100 Subject: [PATCH 01/26] added Partial Set Security ADR --- .../docs/adrs/adr-015-partial-set-security.md | 285 ++++++++++++++++++ docs/docs/adrs/intro.md | 1 + 2 files changed, 286 insertions(+) create mode 100644 docs/docs/adrs/adr-015-partial-set-security.md diff --git a/docs/docs/adrs/adr-015-partial-set-security.md b/docs/docs/adrs/adr-015-partial-set-security.md new file mode 100644 index 0000000000..472675bcd7 --- /dev/null +++ b/docs/docs/adrs/adr-015-partial-set-security.md @@ -0,0 +1,285 @@ +--- +sidebar_position: 16 +title: Partial Set Security +--- +# ADR 015: Partial Set Security + +## Changelog +* 2024-01-22: Proposed, first draft of ADR. + +## Status + +Proposed + +## Context +Currently, in _Replicated Security_, the **whole** validator set of the Cosmos Hub _provider chain_ is used to secure consumer chains. There are at least three concerns with this approach. First, a big chunk of validators might be forced to validate consumer chains they are not interested in securing. Second, it is costly for small validators to secure additional chains. This concern is only partially addressed through [soft opt-out](https://github.com/cosmos/interchain-security/blob/main/docs/docs/adrs/adr-009-soft-opt-out.md) where the bottom (based on voting power) 5% validator can opt out from validating consumer chains. Third and for the above reasons, it is challenging for a new consumer chain to join Replicated Security. + +As a solution, we present _Partial Set Security_ (PSS). As the name suggests, PSS allows for only a subset of the validator set of Cosmos to secure a consumer chain. In what follows we propose the exact steps we need to take to implement PSS. This is a first iteration of Partial Set Security, and therefore we present the most minimal solution that achieves PSS possible. + +## Decision +In Replicated Security, 95% (see soft opt-out) of the top Cosmos Hub validators have to secure a consumer chain. In Partial Set Security (PSS) we introduce a parameter `N` that is associated with a consumer chain and require that the top `N%` of the Cosmos Hub validators have to secure this consumer chain. Additionally, we allow validators that do not belong to the top `N%` of the Cosmos Hub validators to dynamically opt in if they want to validate on the consumer chain or not. +For example, if a consumer chain has `N = 95%`, then it ultimately receives the same security it receives today by Replicated Security. On the other hand, if a consumer chain has `N = 0`%, then no validator is forced to validate the chain but validators can opt in to do so instead. +In what follows, we call a consumer chain _Top N_ if it has joined as Top N chain with `N > 0` and _Opt In_ chain otherwise. An Opt In consumer chain is secured only by the validators that have opted in to secure that chain. + +We intend to implement PSS using a feature branch off [v3.3.0 interchain security](https://github.com/cosmos/interchain-security/tree/v3.3.0). + +### How do consumer chains join? +As a [simplification](https://forum.cosmos.network/t/pss-permissionless-vs-premissioned-lite-opt-in-consumer-chains/12984/17), a consumer chain can only join Partial Set Security through a governance proposal and not in a permissionless way. + +Consumer chains join Partial Set Security the same way chains now join Replicated Security (e.g., see [Stride](https://www.mintscan.io/cosmos/proposals/799)). We extend [`ConsumerAdditionProposal`](https://github.com/cosmos/interchain-security/blob/v3.3.0/proto/interchain_security/ccv/provider/v1/provider.proto#L27) with 2 optional fields: +`bool is_top_N`: flag that captures whether the consumer chain joins under the Top N case. +`int8 top_N`: value that captures the percentage of the Top N% validators that should validate this consumer chain. This is only set if `is_top_N` is `true`. + +Note that `top_N` can be between `[34, 95]`. `top_N` cannot be less than 34 (~⅓) because in the case of `Top N` we want to give the ability to the top 33% validators to veto a proposal so that they are not forced by the remaining validators to secure a consumer chain they do not want to secure. `top_N` can be up to 95% to capture the current case Replicated Security where we allow the bottom 5% of validators to opt out. There is no reason for `top_N` to be higher than 95%. Smaller chains that belong in the bottom 5% validators can choose to opt in if they want to validate. + +So, the fields can take the following values: +| `is_top_N` | `top_N`| +| :--- | :---| +| `true` | `[34, 95]` | +| `false` | not set | + + +If a proposal has those arguments wrongly set, it should get rejected in [ValidateBasic](https://github.com/cosmos/interchain-security/blob/v3.3.0/x/ccv/provider/types/proposal.go#L86). + +Note that we could easily distinguish whether a chain is _Top N_ or _Opt In_ by only having the single field `top_N` and if `top_N == 0`, then we’re in the _Opt In_ case, although for clarity, we introduce both fields. + +In a future version of PSS, we intend to introduce a way to modify the parameters of a consumer chain, e.g, a chain that is _Opt In_ to become _Top N—, etc. + +#### State & Query +We have to augment the provider module’s state to keep track of the `is_top_N` and `top_N` values for each consumer chain. They key to store this information would be: + +``` +ChainIdWithLenKey(IsTopNBytePrefix, chainID) +``` +reusing [`ChainIdWithLenKey`](https://github.com/cosmos/interchain-security/blob/v3.3.0/x/ccv/provider/types/keys.go#L418). + +Then in the [keeper](https://github.com/cosmos/interchain-security/blob/v3.3.0/x/ccv/provider/keeper/keeper.go) we introduce methods as follows: +``` +func (k Keeper) SetIsTopN(ctx sdk.Context, chainID string, isTopN bool) error +func (k Keeper) IsTopN(ctx sdk.Context, chainID string) bool + +// returns the N if Top N chain, otherwise an error +func (k Keeper) GetTopN(ctx sdk.Context, chainID string) (uint8, error) +``` + +In a future version of PSS, we intend to extend the `gaiad query provider list-consumer-chains` query to also return information on whether a consumer chain is an Opt In or a Top N chain and with what N. This way, block explorers like [Mintscan](https://www.mintscan.io/) would show for a chain “This chain is secured by N% of Cosmos Hub.” + +### How do validators opt in? +A validator can opt in by sending a new type of message that we introduce in [tx.proto](https://github.com/cosmos/interchain-security/blob/v3.3.0/proto/interchain_security/ccv/provider/v1/tx.proto#L1). +``` +message MsgOptIn { +// the chain id of the consumer chain to opt in to +string chain_id = 1; +// the provider address of the validator on Cosmos Hub +string provider_addr = 2; +} +``` +Note that in a Top N consumer chain, the top N% Cosmos Hub validators are forced by default to validate the consumer chain. Nevertheless, validators in the bottom 100 - N% can opt in to validate as well. +Additionally, note that due to undelegations, etc. a validator that is in the top N% validators might fall down to the 100 - N% in which case this validator is not forced to validate anymore. Therefore, a validator that might move out of the top N% might want to send a `MsgOptIn` to keep validating even if this happens. + +Note that a validator can send a `MsgOptIn` message even if the consumer chain is not yet running. To do this we reuse the [`IsConsumerProposedOrRegistered`](https://github.com/cosmos/interchain-security/blob/v3.3.0/x/ccv/provider/keeper/key_assignment.go#L644). If the `chainID` does not exist, the `MsgOptIn` should fail, as well as if the provider address does not exist. + +Additionally, a validator that opts in can then use `MsgAssignConsumerKey` to change the consumer key on the consumer chain as is currently done in Replicated Security. + +#### State & Query +We store the list of validators that have opted in under the key: +``` +ChainIdAndConsAddrKey(OptedInBytePrefix, chainID) +``` + +And for simplicity for each validator we store a boolean on whether this validator has opted in on this chain or not. +``` +ChainIdAndConsAddrKey(ValidatorOptedInBytePrefix, chainID, addr) +``` + +We augment the state of the provider to keep track for each chain all the validators that have opted in and if a specific validator has opted in. +``` +// returns all the validators that have opted in on chain `chainID` +func (k Keeper) GetOptedInValidators(ctx sdk.Context, chainID string) []Validators + +func (k Keeper) IsValidatorOptedIn(ctx sdk.Context, chainID string, val Validator) bool +``` + +We introduce a query to retrieve the validators that are opted in. The query would operate as follows. +``` +gaiad query provider optedInValidators $chainId +``` + +#### When do validators opt in? +We could technically opt in validators automatically if they voted Yes during the `ConsumerAdditionProposal`. We do not follow this approach and all validators that opt in should send a `MsgOptIn` message. This is because, opting in validators automatically would obfuscate the semantics of proposals. We can envision cases where a validator votes `Yes` for a proposal but might not want to validate or is not ready to validate the consumer chain. Note that if a validator opts in on a consumer chain, the validator cannot opt out immediately as is described later on. Also, because we provide a `MsgOptIn` message for validators that opt in after a `ConsumerAdditionProposal` has passed, it is reasonable to enforce the same interface for all validators that opt in. + +### How do validators opt out? +Validators that have opted in on a chain can opt out as well. The opt out is not instantaneous and the validator only opts out after a tunable delay of `X` weeks (e.g., 5 weeks). We introduce the delay in order to inform consumer chains about a certain upcoming validator set change in case they want to use this information to take action (e.g., all validators in an opt in chain decide to opt out). + +A validator has send this message to opt out: +``` +message MsgInitiateOptOut { + // the chain id of the consumer chain to opt out from + string chain_id = 1; + // the provider address of the validator on Cosmos Hub + string provider_addr = 2; +} +``` +Note the `Initiate` in the name of the message that points to the fact that opting out is not instantaneous. The above should only be called when a consumer chain is actually running. We want to prevent cases where you opt in and then opt out before the chain has even started running. + +Another approach for opting out could be the following. A validator just opts out by stopping validating on the consumer chain. If we notice this, then we could opt out the validator if the validator is not in the Top N case. We do not follow this approach because it would slow consumer chains (opted out validators would still become proposers in consensus rounds) and additionally it would make it harder for the consumer chain to know which validators are currently securing it – Is a validator just down or did it fully stop validating it? + +Another benefit of introducing a time delay on when the actual opt out takes place is that we prevent a validator from joining and then rejoining multiple times in short bursts. This way we simplify the reward distribution because we know that when we distribute the rewards on the provider chain that a validator that is currently there has actually contributed for some time and deserves rewards. + +Finally, a validator could technically opt out automatically by having all the tokens undelegated or redelegated. This is not an issue per se because: i) the validator remains opted in if it joins later on (during the time delay it takes for the opt out to take place), and ii) unbonding tokens need 21 days to unbond and be reused and similarly the number of redelegations is bounded so an adversary with multiple tokens cannot be joining and leaving Opt In chains ad infinitum. + +#### State & Query +We store information for each the list of validators that have initiated opt out and update the list accordingly at the beginning of each block. +``` +ChainIdAndConsAddrKey(OptedOutBytePrefix, chainID) +``` + +And for simplicity for each validator we store a boolean on whether this validator has opted out on this chain or not. This also includes information on the exact date the opt out was initiated and when it is going to complete. + +``` +ChainIdAndConsAddrKey(ValidatorOptedOutBytePrefix, chainID, addr.ToSdkConsAddr()) +``` + +We augment the state of the provider to keep track for each chain all the validators that have opted in. +``` +// returns all the validators that have opted out on chain `chainID` +func (k Keeper) GetOptedOutValidators(ctx sdk.Context, chainID string) []validators +``` + +We also update the state of the opted-in validators when a validator has opted out. We do this check in every `BeginBlock`. + +We introduce a query to retrieve the validators that are about to be opted out. The query would operate as follows. +``` +gaiad query provider optedOutValidators $chainId +``` + +### When does a consumer chain start? +A Top N consumer chain starts at the specified date (`spawn_time`) if the [`ConsumerAdditionProposal`](https://github.com/cosmos/interchain-security/blob/v3.3.0/proto/interchain_security/ccv/provider/v1/provider.proto#L27) has passed. An Opt In consumer chain starts if at least one validator has opted in. We check this in [BeginBlockInit](https://github.com/cosmos/interchain-security/blob/v3.3.0/x/ccv/provider/keeper/proposal.go#L372): +``` +func (k Keeper) BeginBlockInit(ctx sdk.Context) { + propsToExecute := k.GetConsumerAdditionPropsToExecute(ctx) + + for _, prop := range propsToExecute { + chainId := prop.ChainId + if !k.IsTopN(ctx, chainId) && len(k.GetOptedInValidators(ctx, chainId)) == 0 { + // drop the proposal + ctx.Logger().Info("could not start chain because noone has opted in") + continue + } + ... +``` + +### How do we send the partial validator sets to the consumer chains? +To send the validator set to a consumer chain we first have to generate the validator updates with a function similar to this one: + +``` +func (k Keeper) GetPartialSetValidators(ctx types.Context, chainId string valUpdates []abci.ValidatorUpdate) (newUpdates []abci.ValidatorUpdate) + + powerSum := sdk.ZeroDec() + totalPower := k.ComputeTotalPower(ctx) + topNThreshold := k.GetTopN(ctx, chainId) / 100 + for _, val := range valUpdates { + powerSum = powerSum.Add(sdk.NewDecFromInt(sdk.NewInt(val.Power))) + // if powerSum / totalPower > topNThreshold + if k.IsTopN(ctx, chainId) && powerSum.Quo(totalPower).GT(topNThreshold) { + // is Top N validator + updates = append(updates, abci.ValidatorUpdate{ + PubKey: val.PubKey, + Power: val.Power, + }) + } else { + if k.IsValidatorOptedIn(ctx, chainId, val.PubKey) { + updates = append(updates, abci.ValidatorUpdate{ + PubKey: val.PubKey, + Power: val.Power, + }) + } else { + // remove this validator + updates = append(updates, abci.ValidatorUpdate{ + PubKey: val.PubKey, + Power: 0, + }) + } + } + + return updates +} +``` +And then we change `QueueVSCPackets` to: + +``` +// QueueVSCPackets queues latest validator updates for every registered consumer chain +func (k Keeper) QueueVSCPackets(ctx sdk.Context) { + valUpdateID := k.GetValidatorSetUpdateId(ctx) // current valset update ID + // Get the validator updates from the staking module. + // Note: GetValidatorUpdates panics if the updates provided by the x/staking module + // of cosmos-sdk is invalid. + valUpdates := k.stakingKeeper.GetValidatorUpdates(ctx) + + for _, chain := range k.GetAllConsumerChains(ctx) { + // Apply the key assignment to the validator updates. + valUpdates := k.GetPartialSetValidators(ctx, chain.ChainId, valUpdates) + valUpdates := k.MustApplyKeyAssignmentToValUpdates(ctx, chain.ChainId, valUpdates) + ... +``` + +Note that we use `0` to remove validators that are not opted in as described [here](https://docs.cometbft.com/v0.37/spec/abci/abci++_methods#endblock) and as is done for the [assignment of consumer keys](https://github.com/cosmos/interchain-security/blob/v3.3.0/x/ccv/provider/keeper/key_assignment.go#L525). + +### How do we distribute rewards? +Currently, rewards are distributed as follows: The consumer [periodically sends rewards](https://github.com/cosmos/interchain-security/blob/v3.3.0/x/ccv/consumer/keeper/distribution.go#L148) on the provider `ConsumerRewardsPool` address. The provider then [transfers those rewards to the fee collector address](https://github.com/cosmos/interchain-security/blob/v3.3.0/x/ccv/provider/keeper/distribution.go#L77) and those transferred rewards are distributed to validators and delegators. + +We could use something like `AllocateTokenRewards` but still needs a lot of work because we need to deduce tokens beforehand and so on … Probably doesn’t have to be exact science .. since we already do not distribute exactly. + +### Misbehaviour + +#### Fraud votes +For fraud votes, we add a new type `FraudVoteProposal` similar to [EquivocationProposal](https://github.com/cosmos/interchain-security/pull/703). The time to detect incorrect execution evidence is 7 days (the same time as that of detecting a light client attack on a client with a `trustingPeriod` of 14 days); We have 7 days to detect an incorrect execution and then 14 days for the proposal to be accepted. + +##### When are fraud votes eligible? +Because fraud votes allow the slashing of a validator through a governance proposal, we are cautious on when a fraud vote is eligible and try to restrict their eligibility as much as possible to avoid an adversary proposing a bogus fraud vote. For this, we only consider validators that have opted in in an Opt In chain. Fraud votes are also **only** intended to be used for **malicious** behaviour, for example, in a scenario similar to what [happened to Neutron](https://blog.neutron.org/neutron-halt-post-mortem-927dbe4540c8), validators should vote against slashing those validators. + +#### Message +``` +message FraudVoteProposal { + string title = 1; + string description = 2; + + // validator that performed incorrect execution (consensus address on consumer chain) + Validator validator = 3; + + // validator's vote contains BlockId on what was signed + Vote vote = 4; + + // header for which the hash led to the blockId in the vote + Header header = 5; +} +``` + +The message includes the `vote` that was signed by the validator. Validators that vote on a fraud vote can look at the `BlockID` contained in a `vote` and the `appHash`, `lastResultHash`, etc. in the `header` (that led to this `BlockID`) to conclude whether the to-be-voted validator has performed incorrect execution or not. + +#### Double signing +We do not change the way slashing for double signing and light client attacks functions. If a validator misbehaves on a consumer, then we slash that validator on the provider. + +#### Downtime +We do not change the way downtime jailing functions. If a validator is down on a consumer chain for an adequate amount of time, we jail this validator on Cosmos Hub. + +## Consequences + +### Positive +Easier for new consumer chains to consume Cosmos Hub security because proposals are more likely to pass if not everyone is forced to validate. +Small validators are not forced to validate chains anymore if they do not want to. + +### Negative +Fraud votes introduce one more way to slash validators. +Depending on whether a consumer chain is Top N or Opt In, a consumer chain might not be as secure as with Replicated Security. + + +## References + +> Are there any relevant PR comments, issues that led up to this, or articles referenced for why we made the given design choice? If so link them here! + +[PSS: Permissionless vs premissioned-lite opt-in consumer chains](https://forum.cosmos.network/t/pss-permissionless-vs-premissioned-lite-opt-in-consumer-chains/12984) +[CHIPs discussion phase: Partial Set Security (updated)](https://forum.cosmos.network/t/chips-discussion-phase-partial-set-security-updated/11775) +[PSS: Exclusive vs Inclusive Top-N](https://forum.cosmos.network/t/pss-exclusive-vs-inclusive-top-n/13058) +[Initial PSS ADR and notes #1518](https://github.com/cosmos/interchain-security/pull/1518) +[Replicated vs. Mesh Security](https://informal.systems/blog/replicated-vs-mesh-security) diff --git a/docs/docs/adrs/intro.md b/docs/docs/adrs/intro.md index c837cbd0b2..0ba2b5096e 100644 --- a/docs/docs/adrs/intro.md +++ b/docs/docs/adrs/intro.md @@ -45,6 +45,7 @@ To suggest an ADR, please make use of the [ADR template](./adr-template.md) prov - [ADR 011: Improving testing and increasing confidence](./adr-011-improving-test-confidence.md) - [ADR 014: Epochs](./adr-014-epochs.md) +- [ADR 015: Partial Set Security](./adr-015-partial-set-security.md) ### Rejected From a86272d0a1b09a9a9083c81cc1ec917624fb1cf6 Mon Sep 17 00:00:00 2001 From: insumity Date: Mon, 22 Jan 2024 16:30:32 +0100 Subject: [PATCH 02/26] fix indentation --- .../docs/adrs/adr-015-partial-set-security.md | 20 +++++++++---------- 1 file changed, 9 insertions(+), 11 deletions(-) diff --git a/docs/docs/adrs/adr-015-partial-set-security.md b/docs/docs/adrs/adr-015-partial-set-security.md index 472675bcd7..69286bfdc2 100644 --- a/docs/docs/adrs/adr-015-partial-set-security.md +++ b/docs/docs/adrs/adr-015-partial-set-security.md @@ -266,20 +266,18 @@ We do not change the way downtime jailing functions. If a validator is down on a ## Consequences ### Positive -Easier for new consumer chains to consume Cosmos Hub security because proposals are more likely to pass if not everyone is forced to validate. -Small validators are not forced to validate chains anymore if they do not want to. +- Easier for new consumer chains to consume Cosmos Hub security because proposals are more likely to pass if not everyone is forced to validate. +- Small validators are not forced to validate chains anymore if they do not want to. ### Negative -Fraud votes introduce one more way to slash validators. -Depending on whether a consumer chain is Top N or Opt In, a consumer chain might not be as secure as with Replicated Security. +- Fraud votes introduce one more way to slash validators. +- Depending on whether a consumer chain is Top N or Opt In, a consumer chain might not be as secure as with Replicated Security. ## References -> Are there any relevant PR comments, issues that led up to this, or articles referenced for why we made the given design choice? If so link them here! - -[PSS: Permissionless vs premissioned-lite opt-in consumer chains](https://forum.cosmos.network/t/pss-permissionless-vs-premissioned-lite-opt-in-consumer-chains/12984) -[CHIPs discussion phase: Partial Set Security (updated)](https://forum.cosmos.network/t/chips-discussion-phase-partial-set-security-updated/11775) -[PSS: Exclusive vs Inclusive Top-N](https://forum.cosmos.network/t/pss-exclusive-vs-inclusive-top-n/13058) -[Initial PSS ADR and notes #1518](https://github.com/cosmos/interchain-security/pull/1518) -[Replicated vs. Mesh Security](https://informal.systems/blog/replicated-vs-mesh-security) +- [PSS: Permissionless vs premissioned-lite opt-in consumer chains](https://forum.cosmos.network/t/pss-permissionless-vs-premissioned-lite-opt-in-consumer-chains/12984) +- [CHIPs discussion phase: Partial Set Security (updated)](https://forum.cosmos.network/t/chips-discussion-phase-partial-set-security-updated/11775) +- [PSS: Exclusive vs Inclusive Top-N](https://forum.cosmos.network/t/pss-exclusive-vs-inclusive-top-n/13058) +- [Initial PSS ADR and notes #1518](https://github.com/cosmos/interchain-security/pull/1518) +- [Replicated vs. Mesh Security](https://informal.systems/blog/replicated-vs-mesh-security) From 4a418c81a54e348403ddf04a95efc4512d509dcb Mon Sep 17 00:00:00 2001 From: insumity Date: Mon, 22 Jan 2024 17:13:57 +0100 Subject: [PATCH 03/26] add on AllocateTokens --- docs/docs/adrs/adr-015-partial-set-security.md | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/docs/docs/adrs/adr-015-partial-set-security.md b/docs/docs/adrs/adr-015-partial-set-security.md index 69286bfdc2..f77e9c20c3 100644 --- a/docs/docs/adrs/adr-015-partial-set-security.md +++ b/docs/docs/adrs/adr-015-partial-set-security.md @@ -228,7 +228,17 @@ Note that we use `0` to remove validators that are not opted in as described [he ### How do we distribute rewards? Currently, rewards are distributed as follows: The consumer [periodically sends rewards](https://github.com/cosmos/interchain-security/blob/v3.3.0/x/ccv/consumer/keeper/distribution.go#L148) on the provider `ConsumerRewardsPool` address. The provider then [transfers those rewards to the fee collector address](https://github.com/cosmos/interchain-security/blob/v3.3.0/x/ccv/provider/keeper/distribution.go#L77) and those transferred rewards are distributed to validators and delegators. -We could use something like `AllocateTokenRewards` but still needs a lot of work because we need to deduce tokens beforehand and so on … Probably doesn’t have to be exact science .. since we already do not distribute exactly. +In PSS, we distribute rewards only to validators that actually validated the consumer chain. To do this we generate a slice with bogus `VoteInfo` that contains the opted in (or Top N) validators and call [AllocateTokens](https://github.com/cosmos/cosmos-sdk/blob/v0.47.7/x/distribution/keeper/allocation.go#L14). + +``` +totalPower := 0 +var []abci.VoteInfo votes +for _, val := range validators { + votes = append(votes, []abci.VoteInfo{{Validator: val, SignedLastBlock: true}}) +} + +k.AllocateTokens(ctx, totalPower, votes) +``` ### Misbehaviour From 3bdb04b02366584ce499402b2238a71a43e485c8 Mon Sep 17 00:00:00 2001 From: insumity Date: Tue, 23 Jan 2024 17:19:25 +0100 Subject: [PATCH 04/26] fixed MsgOptOut to not have a delay and added different way to opt in --- .../docs/adrs/adr-015-partial-set-security.md | 116 ++++++++++-------- 1 file changed, 63 insertions(+), 53 deletions(-) diff --git a/docs/docs/adrs/adr-015-partial-set-security.md b/docs/docs/adrs/adr-015-partial-set-security.md index f77e9c20c3..40f075734d 100644 --- a/docs/docs/adrs/adr-015-partial-set-security.md +++ b/docs/docs/adrs/adr-015-partial-set-security.md @@ -14,7 +14,7 @@ Proposed ## Context Currently, in _Replicated Security_, the **whole** validator set of the Cosmos Hub _provider chain_ is used to secure consumer chains. There are at least three concerns with this approach. First, a big chunk of validators might be forced to validate consumer chains they are not interested in securing. Second, it is costly for small validators to secure additional chains. This concern is only partially addressed through [soft opt-out](https://github.com/cosmos/interchain-security/blob/main/docs/docs/adrs/adr-009-soft-opt-out.md) where the bottom (based on voting power) 5% validator can opt out from validating consumer chains. Third and for the above reasons, it is challenging for a new consumer chain to join Replicated Security. -As a solution, we present _Partial Set Security_ (PSS). As the name suggests, PSS allows for only a subset of the validator set of Cosmos to secure a consumer chain. In what follows we propose the exact steps we need to take to implement PSS. This is a first iteration of Partial Set Security, and therefore we present the most minimal solution that achieves PSS possible. +As a solution, we present _Partial Set Security_ (PSS). As the name suggests, PSS allows for only a subset of the validator set of the Cosmos Hub to secure a consumer chain. In what follows we propose the exact steps we need to take to implement PSS. This is a first iteration of Partial Set Security, and therefore we present the most minimal solution that achieves PSS possible. ## Decision In Replicated Security, 95% (see soft opt-out) of the top Cosmos Hub validators have to secure a consumer chain. In Partial Set Security (PSS) we introduce a parameter `N` that is associated with a consumer chain and require that the top `N%` of the Cosmos Hub validators have to secure this consumer chain. Additionally, we allow validators that do not belong to the top `N%` of the Cosmos Hub validators to dynamically opt in if they want to validate on the consumer chain or not. @@ -30,12 +30,16 @@ Consumer chains join Partial Set Security the same way chains now join Replicate `bool is_top_N`: flag that captures whether the consumer chain joins under the Top N case. `int8 top_N`: value that captures the percentage of the Top N% validators that should validate this consumer chain. This is only set if `is_top_N` is `true`. -Note that `top_N` can be between `[34, 95]`. `top_N` cannot be less than 34 (~⅓) because in the case of `Top N` we want to give the ability to the top 33% validators to veto a proposal so that they are not forced by the remaining validators to secure a consumer chain they do not want to secure. `top_N` can be up to 95% to capture the current case Replicated Security where we allow the bottom 5% of validators to opt out. There is no reason for `top_N` to be higher than 95%. Smaller chains that belong in the bottom 5% validators can choose to opt in if they want to validate. +Note that `top_N` resides between `[50, 95]`. +Assuming that at most `1/3` of validators could be malicious, by having 50% as the minimum value for `top_N` we guarantee that we cannot have a successful incorrect-execution attack on a Top N consumer chain. This is because, a Top N consumer chain with `N >= 50%` would have at least `1/3` correct validators that would be able to censor any incorrect-execution attack. +Additionally, by having a `N >= 50%` and hence `N > 33%` we provide the ability for validators to `Veto` a `ConsumerAdditionProposal` if they do not desire to validate a consumer chain. + +`top_N` can be up to 95% to capture the current case of Replicated Security where we allow the bottom 5% of validators to opt out. There is no reason for `top_N` to be higher than 95%. Smaller chains that belong in the bottom 5% validators can choose to opt in if they want to validate. So, the fields can take the following values: | `is_top_N` | `top_N`| | :--- | :---| -| `true` | `[34, 95]` | +| `true` | `[50, 95]` | | `false` | not set | @@ -43,10 +47,10 @@ If a proposal has those arguments wrongly set, it should get rejected in [Valida Note that we could easily distinguish whether a chain is _Top N_ or _Opt In_ by only having the single field `top_N` and if `top_N == 0`, then we’re in the _Opt In_ case, although for clarity, we introduce both fields. -In a future version of PSS, we intend to introduce a way to modify the parameters of a consumer chain, e.g, a chain that is _Opt In_ to become _Top N—, etc. +In a future version of PSS, we intend to introduce a way to modify the parameters of a consumer chain, e.g, a chain that is _Opt In_ to become _Top N_, etc. #### State & Query -We have to augment the provider module’s state to keep track of the `is_top_N` and `top_N` values for each consumer chain. They key to store this information would be: +We augment the provider module’s state to keep track of the `is_top_N` and `top_N` values for each consumer chain. They key to store this information would be: ``` ChainIdWithLenKey(IsTopNBytePrefix, chainID) @@ -68,10 +72,10 @@ In a future version of PSS, we intend to extend the `gaiad query provider list-c A validator can opt in by sending a new type of message that we introduce in [tx.proto](https://github.com/cosmos/interchain-security/blob/v3.3.0/proto/interchain_security/ccv/provider/v1/tx.proto#L1). ``` message MsgOptIn { -// the chain id of the consumer chain to opt in to -string chain_id = 1; -// the provider address of the validator on Cosmos Hub -string provider_addr = 2; + // the chain id of the consumer chain to opt in to + string chain_id = 1; + // the provider address of the validator on Cosmos Hub + string provider_addr = 2; } ``` Note that in a Top N consumer chain, the top N% Cosmos Hub validators are forced by default to validate the consumer chain. Nevertheless, validators in the bottom 100 - N% can opt in to validate as well. @@ -84,12 +88,12 @@ Additionally, a validator that opts in can then use `MsgAssignConsumerKey` to ch #### State & Query We store the list of validators that have opted in under the key: ``` -ChainIdAndConsAddrKey(OptedInBytePrefix, chainID) +ChainIdWithLenKey(OptedInByteKey, chainID) ``` And for simplicity for each validator we store a boolean on whether this validator has opted in on this chain or not. ``` -ChainIdAndConsAddrKey(ValidatorOptedInBytePrefix, chainID, addr) +ChainIdAndConsAddrKey(OptedInByteKey, chainID, addr) ``` We augment the state of the provider to keep track for each chain all the validators that have opted in and if a specific validator has opted in. @@ -106,52 +110,31 @@ gaiad query provider optedInValidators $chainId ``` #### When do validators opt in? -We could technically opt in validators automatically if they voted Yes during the `ConsumerAdditionProposal`. We do not follow this approach and all validators that opt in should send a `MsgOptIn` message. This is because, opting in validators automatically would obfuscate the semantics of proposals. We can envision cases where a validator votes `Yes` for a proposal but might not want to validate or is not ready to validate the consumer chain. Note that if a validator opts in on a consumer chain, the validator cannot opt out immediately as is described later on. Also, because we provide a `MsgOptIn` message for validators that opt in after a `ConsumerAdditionProposal` has passed, it is reasonable to enforce the same interface for all validators that opt in. +Validators can not only opt in by sending a `MsgOptIn` message, but can also opt in automatically if they voted `Yes` +during the `ConsumerAdditionProposal` that introduced a consumer chain. This simplifies validators operations because +they do not have to send an additional message to opt in. + +Because the `Tally` method [deletes the votes](https://github.com/cosmos/cosmos-sdk/blob/v0.47.7/x/gov/keeper/tally.go#L71) after reading them, we cannot check the votes of the validators after the votes have been tallied. To circumvent this, we introduce +a hook for [`AfterProposalVote`](https://github.com/cosmos/cosmos-sdk/blob/v0.47.7/x/gov/keeper/vote.go#L35) and keep track +of all the votes cast by a validator. If a validator votes more than once, we only consider the latest vote. +Finally, we only consider a validator has opted in if it casts a 100% `Yes` vote in case of a [weighted vote](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-037-gov-split-vote.md). + ### How do validators opt out? -Validators that have opted in on a chain can opt out as well. The opt out is not instantaneous and the validator only opts out after a tunable delay of `X` weeks (e.g., 5 weeks). We introduce the delay in order to inform consumer chains about a certain upcoming validator set change in case they want to use this information to take action (e.g., all validators in an opt in chain decide to opt out). +Validators that have opted in on a chain can opt out as well by sending the following message: -A validator has send this message to opt out: ``` -message MsgInitiateOptOut { +message MsgOptOut { // the chain id of the consumer chain to opt out from string chain_id = 1; // the provider address of the validator on Cosmos Hub string provider_addr = 2; } ``` -Note the `Initiate` in the name of the message that points to the fact that opting out is not instantaneous. The above should only be called when a consumer chain is actually running. We want to prevent cases where you opt in and then opt out before the chain has even started running. - -Another approach for opting out could be the following. A validator just opts out by stopping validating on the consumer chain. If we notice this, then we could opt out the validator if the validator is not in the Top N case. We do not follow this approach because it would slow consumer chains (opted out validators would still become proposers in consensus rounds) and additionally it would make it harder for the consumer chain to know which validators are currently securing it – Is a validator just down or did it fully stop validating it? - -Another benefit of introducing a time delay on when the actual opt out takes place is that we prevent a validator from joining and then rejoining multiple times in short bursts. This way we simplify the reward distribution because we know that when we distribute the rewards on the provider chain that a validator that is currently there has actually contributed for some time and deserves rewards. - -Finally, a validator could technically opt out automatically by having all the tokens undelegated or redelegated. This is not an issue per se because: i) the validator remains opted in if it joins later on (during the time delay it takes for the opt out to take place), and ii) unbonding tokens need 21 days to unbond and be reused and similarly the number of redelegations is bounded so an adversary with multiple tokens cannot be joining and leaving Opt In chains ad infinitum. +Validators can only opt out after a consumer chain has started and hence the above message returns an error if the chain with `chain_id` is not running. #### State & Query -We store information for each the list of validators that have initiated opt out and update the list accordingly at the beginning of each block. -``` -ChainIdAndConsAddrKey(OptedOutBytePrefix, chainID) -``` - -And for simplicity for each validator we store a boolean on whether this validator has opted out on this chain or not. This also includes information on the exact date the opt out was initiated and when it is going to complete. - -``` -ChainIdAndConsAddrKey(ValidatorOptedOutBytePrefix, chainID, addr.ToSdkConsAddr()) -``` - -We augment the state of the provider to keep track for each chain all the validators that have opted in. -``` -// returns all the validators that have opted out on chain `chainID` -func (k Keeper) GetOptedOutValidators(ctx sdk.Context, chainID string) []validators -``` - -We also update the state of the opted-in validators when a validator has opted out. We do this check in every `BeginBlock`. - -We introduce a query to retrieve the validators that are about to be opted out. The query would operate as follows. -``` -gaiad query provider optedOutValidators $chainId -``` +We also update the state of the opted-in validators when a validator has opted out. ### When does a consumer chain start? A Top N consumer chain starts at the specified date (`spawn_time`) if the [`ConsumerAdditionProposal`](https://github.com/cosmos/interchain-security/blob/v3.3.0/proto/interchain_security/ccv/provider/v1/provider.proto#L27) has passed. An Opt In consumer chain starts if at least one validator has opted in. We check this in [BeginBlockInit](https://github.com/cosmos/interchain-security/blob/v3.3.0/x/ccv/provider/keeper/proposal.go#L372): @@ -226,18 +209,44 @@ func (k Keeper) QueueVSCPackets(ctx sdk.Context) { Note that we use `0` to remove validators that are not opted in as described [here](https://docs.cometbft.com/v0.37/spec/abci/abci++_methods#endblock) and as is done for the [assignment of consumer keys](https://github.com/cosmos/interchain-security/blob/v3.3.0/x/ccv/provider/keeper/key_assignment.go#L525). ### How do we distribute rewards? -Currently, rewards are distributed as follows: The consumer [periodically sends rewards](https://github.com/cosmos/interchain-security/blob/v3.3.0/x/ccv/consumer/keeper/distribution.go#L148) on the provider `ConsumerRewardsPool` address. The provider then [transfers those rewards to the fee collector address](https://github.com/cosmos/interchain-security/blob/v3.3.0/x/ccv/provider/keeper/distribution.go#L77) and those transferred rewards are distributed to validators and delegators. +Currently, rewards are distributed as follows: The consumer [periodically sends rewards](https://github.com/cosmos/interchain-security/blob/v3.3.0/x/ccv/consumer/keeper/distribution.go#L148) on the provider `ConsumerRewardsPool` address. +The provider then [transfers those rewards to the fee collector address](https://github.com/cosmos/interchain-security/blob/v3.3.0/x/ccv/provider/keeper/distribution.go#L77) and those transferred rewards are distributed to validators and delegators. -In PSS, we distribute rewards only to validators that actually validated the consumer chain. To do this we generate a slice with bogus `VoteInfo` that contains the opted in (or Top N) validators and call [AllocateTokens](https://github.com/cosmos/cosmos-sdk/blob/v0.47.7/x/distribution/keeper/allocation.go#L14). +In PSS, we distribute rewards only to validators that actually validate the consumer chain. To do this, we have a pool associated with each consumer chain and consumers IBC transfer the rewards to this pool. +Then, because we do not use the canonical fee pool and hence rewards are not automatically distributed, we will create a new and modified version of [AllocateTokens](https://github.com/cosmos/cosmos-sdk/blob/v0.47.7/x/distribution/keeper/allocation.go#L14) as follows: ``` -totalPower := 0 -var []abci.VoteInfo votes -for _, val := range validators { - votes = append(votes, []abci.VoteInfo{{Validator: val, SignedLastBlock: true}}) +func (k Keeper) AllocateTokensFromPool(ctx sdk.Context, poolAddress string, totalPreviousPower int64, bondedVotes []abci.VoteInfo) { + rewardsCollectedInt := k.bankKeeper.GetAllBalances(ctx, poolAddress) + // rest in the same that `AllocateTokens` operates + ... +``` + +In [TransferRewardsToFeeCollector](https://github.com/cosmos/interchain-security/blob/v3.3.0/x/ccv/provider/keeper/distribution.go#L61) we then do: + +``` +for _, chain := range consumerChains { + ... + totalPower := 0 + var []abci.VoteInfo votes + for _, val := range (validators that opted in or Top N) { + votes = append(votes, []abci.VoteInfo{{Validator: val, SignedLastBlock: true}}) + } + + k.AllocateTokens(ctx, pool associated with this chain, totalPower, votes) + ... } +``` -k.AllocateTokens(ctx, totalPower, votes) +instead of calling: +``` +// 4. Transfer the balance to the fee collector address +err := k.bankKeeper.SendCoinsFromModuleToModule( + ctx, + types.ConsumerRewardsPool, + k.feeCollectorName, + sdk.NewCoins(balance), +) ``` ### Misbehaviour @@ -246,7 +255,8 @@ k.AllocateTokens(ctx, totalPower, votes) For fraud votes, we add a new type `FraudVoteProposal` similar to [EquivocationProposal](https://github.com/cosmos/interchain-security/pull/703). The time to detect incorrect execution evidence is 7 days (the same time as that of detecting a light client attack on a client with a `trustingPeriod` of 14 days); We have 7 days to detect an incorrect execution and then 14 days for the proposal to be accepted. ##### When are fraud votes eligible? -Because fraud votes allow the slashing of a validator through a governance proposal, we are cautious on when a fraud vote is eligible and try to restrict their eligibility as much as possible to avoid an adversary proposing a bogus fraud vote. For this, we only consider validators that have opted in in an Opt In chain. Fraud votes are also **only** intended to be used for **malicious** behaviour, for example, in a scenario similar to what [happened to Neutron](https://blog.neutron.org/neutron-halt-post-mortem-927dbe4540c8), validators should vote against slashing those validators. +Because fraud votes allow the slashing of a validator through a governance proposal, we are cautious on when a fraud vote is eligible and try to restrict their eligibility as much as possible to avoid an adversary proposing a bogus fraud vote. +As described earlier, because in Top N chains we consider that `N > 50%` we only consider validators that have opted in an Opt In chain. Fraud votes are also **only** intended to be used for **malicious** behaviour, for example, in a scenario similar to what [happened to Neutron](https://blog.neutron.org/neutron-halt-post-mortem-927dbe4540c8), validators should vote against slashing those validators. #### Message ``` From bedb15a2fe48273d5b534a46d100ea91f4de3227 Mon Sep 17 00:00:00 2001 From: insumity Date: Tue, 23 Jan 2024 17:41:10 +0100 Subject: [PATCH 05/26] added some clarifications --- docs/docs/adrs/adr-015-partial-set-security.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/docs/docs/adrs/adr-015-partial-set-security.md b/docs/docs/adrs/adr-015-partial-set-security.md index 40f075734d..8f51dae1de 100644 --- a/docs/docs/adrs/adr-015-partial-set-security.md +++ b/docs/docs/adrs/adr-015-partial-set-security.md @@ -86,7 +86,7 @@ Note that a validator can send a `MsgOptIn` message even if the consumer chain i Additionally, a validator that opts in can then use `MsgAssignConsumerKey` to change the consumer key on the consumer chain as is currently done in Replicated Security. #### State & Query -We store the list of validators that have opted in under the key: +We store the list of validators that have opted in and at the block height they opted it in under the key: ``` ChainIdWithLenKey(OptedInByteKey, chainID) ``` @@ -230,7 +230,9 @@ for _, chain := range consumerChains { totalPower := 0 var []abci.VoteInfo votes for _, val := range (validators that opted in or Top N) { - votes = append(votes, []abci.VoteInfo{{Validator: val, SignedLastBlock: true}}) + if val has opted in for more than X blocks { + votes = append(votes, []abci.VoteInfo{{Validator: val, SignedLastBlock: true}}) + } } k.AllocateTokens(ctx, pool associated with this chain, totalPower, votes) @@ -249,6 +251,8 @@ err := k.bankKeeper.SendCoinsFromModuleToModule( ) ``` +Note that we would only distribute rewards to validators that are opted in but are opted in for some time (e.g., 10000 blocks) to avoid cases where validators opt in and out in short intervals just in order to receive rewards. + ### Misbehaviour #### Fraud votes From 0a354aefdf65d387871c7095f2e5bb57ed039f77 Mon Sep 17 00:00:00 2001 From: insumity Date: Tue, 23 Jan 2024 17:57:31 +0100 Subject: [PATCH 06/26] added note on downtime --- docs/docs/adrs/adr-015-partial-set-security.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/docs/adrs/adr-015-partial-set-security.md b/docs/docs/adrs/adr-015-partial-set-security.md index 8f51dae1de..e4d3dd74df 100644 --- a/docs/docs/adrs/adr-015-partial-set-security.md +++ b/docs/docs/adrs/adr-015-partial-set-security.md @@ -134,7 +134,8 @@ message MsgOptOut { Validators can only opt out after a consumer chain has started and hence the above message returns an error if the chain with `chain_id` is not running. #### State & Query -We also update the state of the opted-in validators when a validator has opted out. +We also update the state of the opted-in validators when a validator has opted out by removing it from there. +Nevertheless, we have to keep track of all the validators that have opted in during the last `X` (to be defined) blocks to be able to jail validators for downtime. ### When does a consumer chain start? A Top N consumer chain starts at the specified date (`spawn_time`) if the [`ConsumerAdditionProposal`](https://github.com/cosmos/interchain-security/blob/v3.3.0/proto/interchain_security/ccv/provider/v1/provider.proto#L27) has passed. An Opt In consumer chain starts if at least one validator has opted in. We check this in [BeginBlockInit](https://github.com/cosmos/interchain-security/blob/v3.3.0/x/ccv/provider/keeper/proposal.go#L372): @@ -285,7 +286,7 @@ The message includes the `vote` that was signed by the validator. Validators tha We do not change the way slashing for double signing and light client attacks functions. If a validator misbehaves on a consumer, then we slash that validator on the provider. #### Downtime -We do not change the way downtime jailing functions. If a validator is down on a consumer chain for an adequate amount of time, we jail this validator on Cosmos Hub. +We do not change the way downtime jailing functions. If a validator is down on a consumer chain for an adequate amount of time, we jail this validator on Cosmos Hub only if the validator was opted in in the recent past. ## Consequences From f3f6d019a3f65d9acf42a83a32bc8fc90ca4a084 Mon Sep 17 00:00:00 2001 From: insumity Date: Thu, 25 Jan 2024 09:54:41 +0100 Subject: [PATCH 07/26] Update docs/docs/adrs/adr-015-partial-set-security.md Co-authored-by: Simon Noetzlin --- docs/docs/adrs/adr-015-partial-set-security.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/adrs/adr-015-partial-set-security.md b/docs/docs/adrs/adr-015-partial-set-security.md index e4d3dd74df..867548b780 100644 --- a/docs/docs/adrs/adr-015-partial-set-security.md +++ b/docs/docs/adrs/adr-015-partial-set-security.md @@ -50,7 +50,7 @@ Note that we could easily distinguish whether a chain is _Top N_ or _Opt In_ by In a future version of PSS, we intend to introduce a way to modify the parameters of a consumer chain, e.g, a chain that is _Opt In_ to become _Top N_, etc. #### State & Query -We augment the provider module’s state to keep track of the `is_top_N` and `top_N` values for each consumer chain. They key to store this information would be: +We augment the provider module’s state to keep track of the `is_top_N` and `top_N` values for each consumer chain. The key to store this information would be: ``` ChainIdWithLenKey(IsTopNBytePrefix, chainID) From 66122adddc67ff1f6995c373d75ad1a7b0dc9063 Mon Sep 17 00:00:00 2001 From: insumity Date: Thu, 25 Jan 2024 09:55:24 +0100 Subject: [PATCH 08/26] Update docs/docs/adrs/adr-015-partial-set-security.md Co-authored-by: Simon Noetzlin --- docs/docs/adrs/adr-015-partial-set-security.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/adrs/adr-015-partial-set-security.md b/docs/docs/adrs/adr-015-partial-set-security.md index 867548b780..041ff1803a 100644 --- a/docs/docs/adrs/adr-015-partial-set-security.md +++ b/docs/docs/adrs/adr-015-partial-set-security.md @@ -66,7 +66,7 @@ func (k Keeper) IsTopN(ctx sdk.Context, chainID string) bool func (k Keeper) GetTopN(ctx sdk.Context, chainID string) (uint8, error) ``` -In a future version of PSS, we intend to extend the `gaiad query provider list-consumer-chains` query to also return information on whether a consumer chain is an Opt In or a Top N chain and with what N. This way, block explorers like [Mintscan](https://www.mintscan.io/) would show for a chain “This chain is secured by N% of Cosmos Hub.” +In a future version of PSS, we intend to extend the `interchain-security-pd query provider list-consumer-chains` query to also return information on whether a consumer chain is an Opt In or a Top N chain and with what N. This way, block explorers like [Mintscan](https://www.mintscan.io/) would show for a chain “This chain is secured by N% of Cosmos Hub.” ### How do validators opt in? A validator can opt in by sending a new type of message that we introduce in [tx.proto](https://github.com/cosmos/interchain-security/blob/v3.3.0/proto/interchain_security/ccv/provider/v1/tx.proto#L1). From 84e2a8f15c8aa518abbd191803bf7a9a3000808e Mon Sep 17 00:00:00 2001 From: insumity Date: Fri, 26 Jan 2024 11:31:51 +0100 Subject: [PATCH 09/26] Update docs/docs/adrs/adr-015-partial-set-security.md Co-authored-by: Simon Noetzlin --- docs/docs/adrs/adr-015-partial-set-security.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/adrs/adr-015-partial-set-security.md b/docs/docs/adrs/adr-015-partial-set-security.md index 041ff1803a..f357031005 100644 --- a/docs/docs/adrs/adr-015-partial-set-security.md +++ b/docs/docs/adrs/adr-015-partial-set-security.md @@ -157,7 +157,7 @@ func (k Keeper) BeginBlockInit(ctx sdk.Context) { To send the validator set to a consumer chain we first have to generate the validator updates with a function similar to this one: ``` -func (k Keeper) GetPartialSetValidators(ctx types.Context, chainId string valUpdates []abci.ValidatorUpdate) (newUpdates []abci.ValidatorUpdate) +func (k Keeper) GetPartialSetValidators(ctx types.Context, chainId string, valUpdates []abci.ValidatorUpdate) (newUpdates []abci.ValidatorUpdate) powerSum := sdk.ZeroDec() totalPower := k.ComputeTotalPower(ctx) From 9989e54bfabf7e78d34c31608fe4b27a6717f7a4 Mon Sep 17 00:00:00 2001 From: insumity Date: Mon, 29 Jan 2024 14:42:35 +0100 Subject: [PATCH 10/26] Update docs/docs/adrs/adr-015-partial-set-security.md Co-authored-by: Marius Poke --- docs/docs/adrs/adr-015-partial-set-security.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/docs/adrs/adr-015-partial-set-security.md b/docs/docs/adrs/adr-015-partial-set-security.md index f357031005..e56a5de4b7 100644 --- a/docs/docs/adrs/adr-015-partial-set-security.md +++ b/docs/docs/adrs/adr-015-partial-set-security.md @@ -14,7 +14,8 @@ Proposed ## Context Currently, in _Replicated Security_, the **whole** validator set of the Cosmos Hub _provider chain_ is used to secure consumer chains. There are at least three concerns with this approach. First, a big chunk of validators might be forced to validate consumer chains they are not interested in securing. Second, it is costly for small validators to secure additional chains. This concern is only partially addressed through [soft opt-out](https://github.com/cosmos/interchain-security/blob/main/docs/docs/adrs/adr-009-soft-opt-out.md) where the bottom (based on voting power) 5% validator can opt out from validating consumer chains. Third and for the above reasons, it is challenging for a new consumer chain to join Replicated Security. -As a solution, we present _Partial Set Security_ (PSS). As the name suggests, PSS allows for only a subset of the validator set of the Cosmos Hub to secure a consumer chain. In what follows we propose the exact steps we need to take to implement PSS. This is a first iteration of Partial Set Security, and therefore we present the most minimal solution that achieves PSS possible. +As a solution, we present _Partial Set Security_ (PSS). As the name suggests, PSS allows for every consumer chain to be secured by only a subset of the provider validator set. +In what follows we propose the exact steps we need to take to implement PSS. This is a first iteration of Partial Set Security, and therefore we present the most minimal solution that make PSS possible. ## Decision In Replicated Security, 95% (see soft opt-out) of the top Cosmos Hub validators have to secure a consumer chain. In Partial Set Security (PSS) we introduce a parameter `N` that is associated with a consumer chain and require that the top `N%` of the Cosmos Hub validators have to secure this consumer chain. Additionally, we allow validators that do not belong to the top `N%` of the Cosmos Hub validators to dynamically opt in if they want to validate on the consumer chain or not. From 99182df943fc86a55353d6d1352f314648bd7d59 Mon Sep 17 00:00:00 2001 From: insumity Date: Mon, 29 Jan 2024 14:43:03 +0100 Subject: [PATCH 11/26] Update docs/docs/adrs/adr-015-partial-set-security.md Co-authored-by: Marius Poke --- docs/docs/adrs/adr-015-partial-set-security.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/docs/adrs/adr-015-partial-set-security.md b/docs/docs/adrs/adr-015-partial-set-security.md index e56a5de4b7..33f7814382 100644 --- a/docs/docs/adrs/adr-015-partial-set-security.md +++ b/docs/docs/adrs/adr-015-partial-set-security.md @@ -18,7 +18,9 @@ As a solution, we present _Partial Set Security_ (PSS). As the name suggests, PS In what follows we propose the exact steps we need to take to implement PSS. This is a first iteration of Partial Set Security, and therefore we present the most minimal solution that make PSS possible. ## Decision -In Replicated Security, 95% (see soft opt-out) of the top Cosmos Hub validators have to secure a consumer chain. In Partial Set Security (PSS) we introduce a parameter `N` that is associated with a consumer chain and require that the top `N%` of the Cosmos Hub validators have to secure this consumer chain. Additionally, we allow validators that do not belong to the top `N%` of the Cosmos Hub validators to dynamically opt in if they want to validate on the consumer chain or not. +In Replicated Security, all the provider validators have to secure every consumer chain (with the exception of those validators allowed to opt out through the [soft opt-out](https://github.com/cosmos/interchain-security/blob/main/docs/docs/adrs/adr-009-soft-opt-out.md) feature). +In Partial Set Security (PSS), we introduce a parameter `N` for each consumer chain and require that the validators in top `N%` of the provider's voting power have to secure the consumer chain. +Additionally, we allow the validators outside of the top `N%` to dynamically opt in if they want to validate on the consumer chain. For example, if a consumer chain has `N = 95%`, then it ultimately receives the same security it receives today by Replicated Security. On the other hand, if a consumer chain has `N = 0`%, then no validator is forced to validate the chain but validators can opt in to do so instead. In what follows, we call a consumer chain _Top N_ if it has joined as Top N chain with `N > 0` and _Opt In_ chain otherwise. An Opt In consumer chain is secured only by the validators that have opted in to secure that chain. From fcf8f6b06cb3fc4649b89ce19b9c3473d3dc6a48 Mon Sep 17 00:00:00 2001 From: insumity Date: Mon, 29 Jan 2024 14:43:34 +0100 Subject: [PATCH 12/26] Update docs/docs/adrs/adr-015-partial-set-security.md Co-authored-by: Marius Poke --- docs/docs/adrs/adr-015-partial-set-security.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/docs/adrs/adr-015-partial-set-security.md b/docs/docs/adrs/adr-015-partial-set-security.md index 33f7814382..19ba1f01bc 100644 --- a/docs/docs/adrs/adr-015-partial-set-security.md +++ b/docs/docs/adrs/adr-015-partial-set-security.md @@ -22,7 +22,8 @@ In Replicated Security, all the provider validators have to secure every consume In Partial Set Security (PSS), we introduce a parameter `N` for each consumer chain and require that the validators in top `N%` of the provider's voting power have to secure the consumer chain. Additionally, we allow the validators outside of the top `N%` to dynamically opt in if they want to validate on the consumer chain. For example, if a consumer chain has `N = 95%`, then it ultimately receives the same security it receives today by Replicated Security. On the other hand, if a consumer chain has `N = 0`%, then no validator is forced to validate the chain but validators can opt in to do so instead. -In what follows, we call a consumer chain _Top N_ if it has joined as Top N chain with `N > 0` and _Opt In_ chain otherwise. An Opt In consumer chain is secured only by the validators that have opted in to secure that chain. + +For the remainder of this ADR, we call a consumer chain _Top N_ if it has joined as a Top N chain with `N > 0` and _Opt In_ chain otherwise. An Opt In consumer chain is secured only by the validators that have opted in to secure that chain. We intend to implement PSS using a feature branch off [v3.3.0 interchain security](https://github.com/cosmos/interchain-security/tree/v3.3.0). From 53113c126b99d0ab6063f052a2846f502ff14ba5 Mon Sep 17 00:00:00 2001 From: insumity Date: Mon, 29 Jan 2024 14:45:49 +0100 Subject: [PATCH 13/26] Update docs/docs/adrs/adr-015-partial-set-security.md Co-authored-by: Marius Poke --- docs/docs/adrs/adr-015-partial-set-security.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/docs/adrs/adr-015-partial-set-security.md b/docs/docs/adrs/adr-015-partial-set-security.md index 19ba1f01bc..e8ae3ce836 100644 --- a/docs/docs/adrs/adr-015-partial-set-security.md +++ b/docs/docs/adrs/adr-015-partial-set-security.md @@ -35,7 +35,8 @@ Consumer chains join Partial Set Security the same way chains now join Replicate `int8 top_N`: value that captures the percentage of the Top N% validators that should validate this consumer chain. This is only set if `is_top_N` is `true`. Note that `top_N` resides between `[50, 95]`. -Assuming that at most `1/3` of validators could be malicious, by having 50% as the minimum value for `top_N` we guarantee that we cannot have a successful incorrect-execution attack on a Top N consumer chain. This is because, a Top N consumer chain with `N >= 50%` would have at least `1/3` correct validators that would be able to censor any incorrect-execution attack. +Assuming that at most `1/3` of provider validators could be malicious, by having `50%` as the minimum value for `top_N` we guarantee that we cannot have a successful invalid-execution attack on a Top N consumer chain. +This is because, a Top N consumer chain with `N >= 50%` would have at least `1/3` honest validators, which is sufficient to stop any invalid-execution attack. Additionally, by having a `N >= 50%` and hence `N > 33%` we provide the ability for validators to `Veto` a `ConsumerAdditionProposal` if they do not desire to validate a consumer chain. `top_N` can be up to 95% to capture the current case of Replicated Security where we allow the bottom 5% of validators to opt out. There is no reason for `top_N` to be higher than 95%. Smaller chains that belong in the bottom 5% validators can choose to opt in if they want to validate. From 3bdfa2cb8b262cc243e89d9046026b7085840184 Mon Sep 17 00:00:00 2001 From: insumity Date: Mon, 29 Jan 2024 14:46:35 +0100 Subject: [PATCH 14/26] Update docs/docs/adrs/adr-015-partial-set-security.md Co-authored-by: Marius Poke --- docs/docs/adrs/adr-015-partial-set-security.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/adrs/adr-015-partial-set-security.md b/docs/docs/adrs/adr-015-partial-set-security.md index e8ae3ce836..23f080fbce 100644 --- a/docs/docs/adrs/adr-015-partial-set-security.md +++ b/docs/docs/adrs/adr-015-partial-set-security.md @@ -144,7 +144,7 @@ Nevertheless, we have to keep track of all the validators that have opted in dur ### When does a consumer chain start? A Top N consumer chain starts at the specified date (`spawn_time`) if the [`ConsumerAdditionProposal`](https://github.com/cosmos/interchain-security/blob/v3.3.0/proto/interchain_security/ccv/provider/v1/provider.proto#L27) has passed. An Opt In consumer chain starts if at least one validator has opted in. We check this in [BeginBlockInit](https://github.com/cosmos/interchain-security/blob/v3.3.0/x/ccv/provider/keeper/proposal.go#L372): -``` +```golang func (k Keeper) BeginBlockInit(ctx sdk.Context) { propsToExecute := k.GetConsumerAdditionPropsToExecute(ctx) From 0f9bd8bf5c7d8ca0fd32f34aba3669ad69447c52 Mon Sep 17 00:00:00 2001 From: insumity Date: Mon, 29 Jan 2024 15:01:52 +0100 Subject: [PATCH 15/26] Update docs/docs/adrs/adr-015-partial-set-security.md Co-authored-by: Marius Poke --- docs/docs/adrs/adr-015-partial-set-security.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/adrs/adr-015-partial-set-security.md b/docs/docs/adrs/adr-015-partial-set-security.md index 23f080fbce..e7763b1369 100644 --- a/docs/docs/adrs/adr-015-partial-set-security.md +++ b/docs/docs/adrs/adr-015-partial-set-security.md @@ -37,7 +37,7 @@ Consumer chains join Partial Set Security the same way chains now join Replicate Note that `top_N` resides between `[50, 95]`. Assuming that at most `1/3` of provider validators could be malicious, by having `50%` as the minimum value for `top_N` we guarantee that we cannot have a successful invalid-execution attack on a Top N consumer chain. This is because, a Top N consumer chain with `N >= 50%` would have at least `1/3` honest validators, which is sufficient to stop any invalid-execution attack. -Additionally, by having a `N >= 50%` and hence `N > 33%` we provide the ability for validators to `Veto` a `ConsumerAdditionProposal` if they do not desire to validate a consumer chain. +Additionally, by having `N >= 50%` (and hence `N > 33%`) we enable the top N validators to `Veto` any `ConsumerAdditionProposal` for consumer chains they do not want to validate. `top_N` can be up to 95% to capture the current case of Replicated Security where we allow the bottom 5% of validators to opt out. There is no reason for `top_N` to be higher than 95%. Smaller chains that belong in the bottom 5% validators can choose to opt in if they want to validate. From a6c705b9e54dd90fbf8b8f253609787ca6bc7bee Mon Sep 17 00:00:00 2001 From: insumity Date: Mon, 29 Jan 2024 16:31:46 +0100 Subject: [PATCH 16/26] updated based on comments --- .../docs/adrs/adr-015-partial-set-security.md | 140 +++++++++--------- 1 file changed, 73 insertions(+), 67 deletions(-) diff --git a/docs/docs/adrs/adr-015-partial-set-security.md b/docs/docs/adrs/adr-015-partial-set-security.md index e7763b1369..51cea4506e 100644 --- a/docs/docs/adrs/adr-015-partial-set-security.md +++ b/docs/docs/adrs/adr-015-partial-set-security.md @@ -12,106 +12,110 @@ title: Partial Set Security Proposed ## Context -Currently, in _Replicated Security_, the **whole** validator set of the Cosmos Hub _provider chain_ is used to secure consumer chains. There are at least three concerns with this approach. First, a big chunk of validators might be forced to validate consumer chains they are not interested in securing. Second, it is costly for small validators to secure additional chains. This concern is only partially addressed through [soft opt-out](https://github.com/cosmos/interchain-security/blob/main/docs/docs/adrs/adr-009-soft-opt-out.md) where the bottom (based on voting power) 5% validator can opt out from validating consumer chains. Third and for the above reasons, it is challenging for a new consumer chain to join Replicated Security. +Currently, in _Replicated Security_, the entire validator set of the provider chain is used to secure consumer chains. There are at least three concerns with this approach. +First, a large number of validators might be forced to validate consumer chains they are not interested in securing. +Second, it is costly for small validators to secure additional chains. This concern is only partially addressed through [soft opt-out](https://github.com/cosmos/interchain-security/blob/main/docs/docs/adrs/adr-009-soft-opt-out.md) that allows small validators to opt out from validating consumer chains. +Third and for the above reasons, it is challenging for a new consumer chain to join Replicated Security. As a solution, we present _Partial Set Security_ (PSS). As the name suggests, PSS allows for every consumer chain to be secured by only a subset of the provider validator set. -In what follows we propose the exact steps we need to take to implement PSS. This is a first iteration of Partial Set Security, and therefore we present the most minimal solution that make PSS possible. +In what follows we propose the exact steps we need to take to implement PSS. This is a first iteration of PSS, and therefore we present the most minimal solution that make PSS possible. ## Decision In Replicated Security, all the provider validators have to secure every consumer chain (with the exception of those validators allowed to opt out through the [soft opt-out](https://github.com/cosmos/interchain-security/blob/main/docs/docs/adrs/adr-009-soft-opt-out.md) feature). -In Partial Set Security (PSS), we introduce a parameter `N` for each consumer chain and require that the validators in top `N%` of the provider's voting power have to secure the consumer chain. +In PSS, we introduce a parameter `N` for each consumer chain and require that the validators in top `N%` of the provider's voting power have to secure the consumer chain. Additionally, we allow the validators outside of the top `N%` to dynamically opt in if they want to validate on the consumer chain. -For example, if a consumer chain has `N = 95%`, then it ultimately receives the same security it receives today by Replicated Security. On the other hand, if a consumer chain has `N = 0`%, then no validator is forced to validate the chain but validators can opt in to do so instead. +For example, if a consumer chain has `N = 95%`, then it ultimately receives the same security it receives today with Replicated Security (with a default [SoftOptOutThreshold](https://github.com/cosmos/interchain-security/blob/main/docs/docs/adrs/adr-009-soft-opt-out.md) of 5%). +On the other hand, if a consumer chain has `N = 0%`, then no validator is forced to validate the chain, but validators can opt in to do so instead. For the remainder of this ADR, we call a consumer chain _Top N_ if it has joined as a Top N chain with `N > 0` and _Opt In_ chain otherwise. An Opt In consumer chain is secured only by the validators that have opted in to secure that chain. -We intend to implement PSS using a feature branch off [v3.3.0 interchain security](https://github.com/cosmos/interchain-security/tree/v3.3.0). +We intend to implement PSS using a feature branch off [v4.0.0 interchain security](https://github.com/cosmos/interchain-security/tree/v4.0.0). ### How do consumer chains join? -As a [simplification](https://forum.cosmos.network/t/pss-permissionless-vs-premissioned-lite-opt-in-consumer-chains/12984/17), a consumer chain can only join Partial Set Security through a governance proposal and not in a permissionless way. +As a simplification and to avoid [chain id squatting](https://forum.cosmos.network/t/pss-permissionless-vs-premissioned-lite-opt-in-consumer-chains/12984/17), a consumer chain can only join PSS through a governance proposal and not in a permissionless way. -Consumer chains join Partial Set Security the same way chains now join Replicated Security (e.g., see [Stride](https://www.mintscan.io/cosmos/proposals/799)). We extend [`ConsumerAdditionProposal`](https://github.com/cosmos/interchain-security/blob/v3.3.0/proto/interchain_security/ccv/provider/v1/provider.proto#L27) with 2 optional fields: -`bool is_top_N`: flag that captures whether the consumer chain joins under the Top N case. -`int8 top_N`: value that captures the percentage of the Top N% validators that should validate this consumer chain. This is only set if `is_top_N` is `true`. +Consumer chains join PSS the same way chains now join Replicated Security (e.g., see [Stride](https://www.mintscan.io/cosmos/proposals/799)). We extend [`ConsumerAdditionProposal`](https://github.com/cosmos/interchain-security/blob/v4.0.0/proto/interchain_security/ccv/provider/v1/provider.proto#L27) with one optional field: -Note that `top_N` resides between `[50, 95]`. -Assuming that at most `1/3` of provider validators could be malicious, by having `50%` as the minimum value for `top_N` we guarantee that we cannot have a successful invalid-execution attack on a Top N consumer chain. +`string top_N_fraction`: Corresponds to the percentage of validators that join under the Top N case. +For example, `0.53` corresponds to a Top 53% chain, meaning that the top `53%` provider validators would validate the proposed consumer chain. +If the `string` is not set, the consumer chain corresponds to an Opt In chain. +`top_N_fraction` resides between `[0.5, 0.95]`. +Assuming that at most `1/3` of provider validators could be malicious, by having `0.5` (`50%`) as the minimum value for `top_N_fraction` we guarantee that we cannot have a successful invalid-execution attack on a Top N consumer chain. This is because, a Top N consumer chain with `N >= 50%` would have at least `1/3` honest validators, which is sufficient to stop any invalid-execution attack. Additionally, by having `N >= 50%` (and hence `N > 33%`) we enable the top N validators to `Veto` any `ConsumerAdditionProposal` for consumer chains they do not want to validate. -`top_N` can be up to 95% to capture the current case of Replicated Security where we allow the bottom 5% of validators to opt out. There is no reason for `top_N` to be higher than 95%. Smaller chains that belong in the bottom 5% validators can choose to opt in if they want to validate. +`top_N_fraction` can be up to `0.95` (`95%`) to capture the current case of Replicated Security where we allow the bottom `5%` of validators to opt out. There is no reason for `top_N_fraction` to be higher than `95%`. Smaller chains that belong in the bottom `5%` validators can choose to opt in if they want to validate. -So, the fields can take the following values: -| `is_top_N` | `top_N`| -| :--- | :---| -| `true` | `[50, 95]` | -| `false` | not set | +If a proposal has those arguments wrongly set, it should get rejected in [ValidateBasic](https://github.com/cosmos/interchain-security/blob/v4.0.0/x/ccv/provider/types/proposal.go#L86). -If a proposal has those arguments wrongly set, it should get rejected in [ValidateBasic](https://github.com/cosmos/interchain-security/blob/v3.3.0/x/ccv/provider/types/proposal.go#L86). - -Note that we could easily distinguish whether a chain is _Top N_ or _Opt In_ by only having the single field `top_N` and if `top_N == 0`, then we’re in the _Opt In_ case, although for clarity, we introduce both fields. +In the code, we distinguish whether a chain is _Top N_ or _Opt In_ by checking whether `top_N_fraction` is set or not. In a future version of PSS, we intend to introduce a way to modify the parameters of a consumer chain, e.g, a chain that is _Opt In_ to become _Top N_, etc. #### State & Query -We augment the provider module’s state to keep track of the `is_top_N` and `top_N` values for each consumer chain. The key to store this information would be: +We augment the provider module’s state to keep track of the `top_N_fraction` value for each consumer chain. The key to store this information would be: ``` -ChainIdWithLenKey(IsTopNBytePrefix, chainID) +topNFractionBytePrefix | len(chainID) | chainID ``` -reusing [`ChainIdWithLenKey`](https://github.com/cosmos/interchain-security/blob/v3.3.0/x/ccv/provider/types/keys.go#L418). +To create the above key, we can use [`ChainIdWithLenKey`](https://github.com/cosmos/interchain-security/blob/v4.0.0/x/ccv/provider/types/keys.go#L418). -Then in the [keeper](https://github.com/cosmos/interchain-security/blob/v3.3.0/x/ccv/provider/keeper/keeper.go) we introduce methods as follows: -``` -func (k Keeper) SetIsTopN(ctx sdk.Context, chainID string, isTopN bool) error +Then in the [keeper](https://github.com/cosmos/interchain-security/blob/v4.0.0/x/ccv/provider/keeper/keeper.go) we introduce methods as follows: +```golang +func (k Keeper) SetTopN(ctx sdk.Context, chainID string, topNFraction string) func (k Keeper) IsTopN(ctx sdk.Context, chainID string) bool +func (k Keeper) IsOptIn(ctx sdk.Context, chainID string) bool // returns the N if Top N chain, otherwise an error func (k Keeper) GetTopN(ctx sdk.Context, chainID string) (uint8, error) ``` -In a future version of PSS, we intend to extend the `interchain-security-pd query provider list-consumer-chains` query to also return information on whether a consumer chain is an Opt In or a Top N chain and with what N. This way, block explorers like [Mintscan](https://www.mintscan.io/) would show for a chain “This chain is secured by N% of Cosmos Hub.” +We also extend the `interchain-security-pd query provider list-consumer-chains` query to return information on whether a consumer chain is an Opt In or a Top N chain and with what N. This way, block explorers like [Mintscan](https://www.mintscan.io/) could show for a chain “This chain is secured by N% of the provider chain.” ### How do validators opt in? -A validator can opt in by sending a new type of message that we introduce in [tx.proto](https://github.com/cosmos/interchain-security/blob/v3.3.0/proto/interchain_security/ccv/provider/v1/tx.proto#L1). -``` +A validator can opt in by sending a new type of message that we introduce in [tx.proto](https://github.com/cosmos/interchain-security/blob/v4.0.0/proto/interchain_security/ccv/provider/v1/tx.proto#L1). +```protobuf message MsgOptIn { // the chain id of the consumer chain to opt in to string chain_id = 1; - // the provider address of the validator on Cosmos Hub + // the provider address of the validator string provider_addr = 2; } ``` -Note that in a Top N consumer chain, the top N% Cosmos Hub validators are forced by default to validate the consumer chain. Nevertheless, validators in the bottom 100 - N% can opt in to validate as well. -Additionally, note that due to undelegations, etc. a validator that is in the top N% validators might fall down to the 100 - N% in which case this validator is not forced to validate anymore. Therefore, a validator that might move out of the top N% might want to send a `MsgOptIn` to keep validating even if this happens. +Note that in a Top N consumer chain, the top `N%` provider validators have to validate the consumer chain. +Nevertheless, validators in the bottom `(100 - N)%` can opt in to validate as well. +Provider validators that belong to the top `N%` validators are immediately opted in to validate a Top N consumer chain. +This means that if a validator `V` belongs to the top `N%` validators but later falls (e.g., due to undelegations) to the bottom `(100 - N)%`, `V` would still be considered opted in and has to validate unless `V` sends a `MsgOptOut` message (see below). +By automatically opting in validators when they enter the top `N%` of the validators and by forcing top `N%` validators to explicitly opt out in case they fall to the `(100 - N)%` bottom validators we simplify our implementation because i) we only have to find all the opted in validators to retrieve the validators that have to validate a consumer chain, and ii) we simplify reward distribution by preventing cases where we opted out a validator before it receives its rewards. -Note that a validator can send a `MsgOptIn` message even if the consumer chain is not yet running. To do this we reuse the [`IsConsumerProposedOrRegistered`](https://github.com/cosmos/interchain-security/blob/v3.3.0/x/ccv/provider/keeper/key_assignment.go#L644). If the `chainID` does not exist, the `MsgOptIn` should fail, as well as if the provider address does not exist. +Note that a validator can send a `MsgOptIn` message even if the consumer chain is not yet running. To do this we reuse the [`IsConsumerProposedOrRegistered`](https://github.com/cosmos/interchain-security/blob/v4.0.0/x/ccv/provider/keeper/key_assignment.go#L644). If the `chainID` does not exist, the `MsgOptIn` should fail, as well as if the provider address does not exist. Additionally, a validator that opts in can then use `MsgAssignConsumerKey` to change the consumer key on the consumer chain as is currently done in Replicated Security. #### State & Query -We store the list of validators that have opted in and at the block height they opted it in under the key: -``` -ChainIdWithLenKey(OptedInByteKey, chainID) -``` - -And for simplicity for each validator we store a boolean on whether this validator has opted in on this chain or not. +For each validator, we store on whether the validator has opted in and at the block height they opted in under the key: ``` -ChainIdAndConsAddrKey(OptedInByteKey, chainID, addr) +optedInBytePrefix | len(chainID) | chainID | addr ``` +By using a prefix iterator on `optedInBytePrefix | len(chainID) | chainID ` we retrieve all the opted in validators. -We augment the state of the provider to keep track for each chain all the validators that have opted in and if a specific validator has opted in. -``` +We introduce the following `Keeper` methods. +```golang // returns all the validators that have opted in on chain `chainID` func (k Keeper) GetOptedInValidators(ctx sdk.Context, chainID string) []Validators func (k Keeper) IsValidatorOptedIn(ctx sdk.Context, chainID string, val Validator) bool ``` -We introduce a query to retrieve the validators that are opted in. The query would operate as follows. +We introduce a query to retrieve the validators that are opted in and hence the validators that need to validate the consumer chain: +```bash +interchain-security-pd query provider optedInValidators $chainID ``` -gaiad query provider optedInValidators $chainId + +We also introduce a query that given a validator's address returns all the chains this validator has to validate: +```bash +interchain-security-pd query provider hasToValidate providerAddr ``` #### When do validators opt in? @@ -128,11 +132,11 @@ Finally, we only consider a validator has opted in if it casts a 100% `Yes` vote ### How do validators opt out? Validators that have opted in on a chain can opt out as well by sending the following message: -``` +```protobuf message MsgOptOut { // the chain id of the consumer chain to opt out from string chain_id = 1; - // the provider address of the validator on Cosmos Hub + // the provider address of the validator string provider_addr = 2; } ``` @@ -140,19 +144,21 @@ Validators can only opt out after a consumer chain has started and hence the abo #### State & Query We also update the state of the opted-in validators when a validator has opted out by removing it from there. -Nevertheless, we have to keep track of all the validators that have opted in during the last `X` (to be defined) blocks to be able to jail validators for downtime. + +Note that we only punish validators for downtime that were opted in in a consumer chain. For this, we keep historical info of all the validators that have opted in during the last `X` (to be defined) blocks. This way we can jail validators for downtime knowing that indeed the validators have opted in at some point in the past. +Otherwise, we can think of a scenario where a validator `V` is down for a period of time, but before `V` gets punished for downtime, validator `V` opts out and then we do not know whether `V` should be punished or not. ### When does a consumer chain start? -A Top N consumer chain starts at the specified date (`spawn_time`) if the [`ConsumerAdditionProposal`](https://github.com/cosmos/interchain-security/blob/v3.3.0/proto/interchain_security/ccv/provider/v1/provider.proto#L27) has passed. An Opt In consumer chain starts if at least one validator has opted in. We check this in [BeginBlockInit](https://github.com/cosmos/interchain-security/blob/v3.3.0/x/ccv/provider/keeper/proposal.go#L372): +A Top N consumer chain starts at the specified date (`spawn_time`) if the [`ConsumerAdditionProposal`](https://github.com/cosmos/interchain-security/blob/v4.0.0/proto/interchain_security/ccv/provider/v1/provider.proto#L27) has passed. An Opt In consumer chain starts if at least one validator has opted in. We check this in [BeginBlockInit](https://github.com/cosmos/interchain-security/blob/v4.0.0/x/ccv/provider/keeper/proposal.go#L357): ```golang func (k Keeper) BeginBlockInit(ctx sdk.Context) { propsToExecute := k.GetConsumerAdditionPropsToExecute(ctx) for _, prop := range propsToExecute { - chainId := prop.ChainId - if !k.IsTopN(ctx, chainId) && len(k.GetOptedInValidators(ctx, chainId)) == 0 { + chainID := prop.ChainId + if !k.IsTopN(ctx, chainID) && len(k.GetOptedInValidators(ctx, chainID)) == 0 { // drop the proposal - ctx.Logger().Info("could not start chain because noone has opted in") + ctx.Logger().Info("could not start chain because no validator has opted in") continue } ... @@ -161,23 +167,23 @@ func (k Keeper) BeginBlockInit(ctx sdk.Context) { ### How do we send the partial validator sets to the consumer chains? To send the validator set to a consumer chain we first have to generate the validator updates with a function similar to this one: -``` -func (k Keeper) GetPartialSetValidators(ctx types.Context, chainId string, valUpdates []abci.ValidatorUpdate) (newUpdates []abci.ValidatorUpdate) +```golang +func (k Keeper) GetPartialSetValidators(ctx types.Context, chainID string, valUpdates []abci.ValidatorUpdate) (newUpdates []abci.ValidatorUpdate) powerSum := sdk.ZeroDec() totalPower := k.ComputeTotalPower(ctx) - topNThreshold := k.GetTopN(ctx, chainId) / 100 + topNThreshold := k.GetTopN(ctx, chainID) / 100 for _, val := range valUpdates { powerSum = powerSum.Add(sdk.NewDecFromInt(sdk.NewInt(val.Power))) // if powerSum / totalPower > topNThreshold - if k.IsTopN(ctx, chainId) && powerSum.Quo(totalPower).GT(topNThreshold) { + if k.IsTopN(ctx, chainID) && powerSum.Quo(totalPower).GT(topNThreshold) { // is Top N validator updates = append(updates, abci.ValidatorUpdate{ PubKey: val.PubKey, Power: val.Power, }) } else { - if k.IsValidatorOptedIn(ctx, chainId, val.PubKey) { + if k.IsValidatorOptedIn(ctx, chainID, val.PubKey) { updates = append(updates, abci.ValidatorUpdate{ PubKey: val.PubKey, Power: val.Power, @@ -196,7 +202,7 @@ func (k Keeper) GetPartialSetValidators(ctx types.Context, chainId string, valUp ``` And then we change `QueueVSCPackets` to: -``` +```golang // QueueVSCPackets queues latest validator updates for every registered consumer chain func (k Keeper) QueueVSCPackets(ctx sdk.Context) { valUpdateID := k.GetValidatorSetUpdateId(ctx) // current valset update ID @@ -212,25 +218,25 @@ func (k Keeper) QueueVSCPackets(ctx sdk.Context) { ... ``` -Note that we use `0` to remove validators that are not opted in as described [here](https://docs.cometbft.com/v0.37/spec/abci/abci++_methods#endblock) and as is done for the [assignment of consumer keys](https://github.com/cosmos/interchain-security/blob/v3.3.0/x/ccv/provider/keeper/key_assignment.go#L525). +Note that we use `0` to remove validators that are not opted in as described [here](https://docs.cometbft.com/v0.37/spec/abci/abci++_methods#endblock) and as is done for the [assignment of consumer keys](https://github.com/cosmos/interchain-security/blob/v4.0.0/x/ccv/provider/keeper/key_assignment.go#L525). ### How do we distribute rewards? -Currently, rewards are distributed as follows: The consumer [periodically sends rewards](https://github.com/cosmos/interchain-security/blob/v3.3.0/x/ccv/consumer/keeper/distribution.go#L148) on the provider `ConsumerRewardsPool` address. -The provider then [transfers those rewards to the fee collector address](https://github.com/cosmos/interchain-security/blob/v3.3.0/x/ccv/provider/keeper/distribution.go#L77) and those transferred rewards are distributed to validators and delegators. +Currently, rewards are distributed as follows: The consumer [periodically sends rewards](https://github.com/cosmos/interchain-security/blob/v4.0.0/x/ccv/consumer/keeper/distribution.go#L148) on the provider `ConsumerRewardsPool` address. +The provider then [transfers those rewards to the fee collector address](https://github.com/cosmos/interchain-security/blob/v4.0.0/x/ccv/provider/keeper/distribution.go#L77) and those transferred rewards are distributed to validators and delegators. In PSS, we distribute rewards only to validators that actually validate the consumer chain. To do this, we have a pool associated with each consumer chain and consumers IBC transfer the rewards to this pool. Then, because we do not use the canonical fee pool and hence rewards are not automatically distributed, we will create a new and modified version of [AllocateTokens](https://github.com/cosmos/cosmos-sdk/blob/v0.47.7/x/distribution/keeper/allocation.go#L14) as follows: -``` +```golang func (k Keeper) AllocateTokensFromPool(ctx sdk.Context, poolAddress string, totalPreviousPower int64, bondedVotes []abci.VoteInfo) { rewardsCollectedInt := k.bankKeeper.GetAllBalances(ctx, poolAddress) // rest in the same that `AllocateTokens` operates ... ``` -In [TransferRewardsToFeeCollector](https://github.com/cosmos/interchain-security/blob/v3.3.0/x/ccv/provider/keeper/distribution.go#L61) we then do: +In [TransferRewardsToFeeCollector](https://github.com/cosmos/interchain-security/blob/v4.0.0/x/ccv/provider/keeper/distribution.go#L61) we then do: -``` +```golang for _, chain := range consumerChains { ... totalPower := 0 @@ -247,7 +253,7 @@ for _, chain := range consumerChains { ``` instead of calling: -``` +```golang // 4. Transfer the balance to the fee collector address err := k.bankKeeper.SendCoinsFromModuleToModule( ctx, @@ -269,7 +275,7 @@ Because fraud votes allow the slashing of a validator through a governance propo As described earlier, because in Top N chains we consider that `N > 50%` we only consider validators that have opted in an Opt In chain. Fraud votes are also **only** intended to be used for **malicious** behaviour, for example, in a scenario similar to what [happened to Neutron](https://blog.neutron.org/neutron-halt-post-mortem-927dbe4540c8), validators should vote against slashing those validators. #### Message -``` +```protobuf message FraudVoteProposal { string title = 1; string description = 2; @@ -291,12 +297,12 @@ The message includes the `vote` that was signed by the validator. Validators tha We do not change the way slashing for double signing and light client attacks functions. If a validator misbehaves on a consumer, then we slash that validator on the provider. #### Downtime -We do not change the way downtime jailing functions. If a validator is down on a consumer chain for an adequate amount of time, we jail this validator on Cosmos Hub only if the validator was opted in in the recent past. +We do not change the way downtime jailing functions. If a validator is down on a consumer chain for an adequate amount of time, we jail this validator on the provider only if the validator was opted in in the recent past. ## Consequences ### Positive -- Easier for new consumer chains to consume Cosmos Hub security because proposals are more likely to pass if not everyone is forced to validate. +- Easier for new consumer chains to consume the provider's chain security because proposals are more likely to pass if not everyone is forced to validate. - Small validators are not forced to validate chains anymore if they do not want to. ### Negative From 9ffa1e2cf3f777e61559ad21336cd9836b16e748 Mon Sep 17 00:00:00 2001 From: insumity Date: Mon, 29 Jan 2024 18:45:49 +0100 Subject: [PATCH 17/26] remove fraud votes --- .../docs/adrs/adr-015-partial-set-security.md | 26 +++---------------- 1 file changed, 3 insertions(+), 23 deletions(-) diff --git a/docs/docs/adrs/adr-015-partial-set-security.md b/docs/docs/adrs/adr-015-partial-set-security.md index 51cea4506e..36d568a3d3 100644 --- a/docs/docs/adrs/adr-015-partial-set-security.md +++ b/docs/docs/adrs/adr-015-partial-set-security.md @@ -268,30 +268,10 @@ Note that we would only distribute rewards to validators that are opted in but a ### Misbehaviour #### Fraud votes -For fraud votes, we add a new type `FraudVoteProposal` similar to [EquivocationProposal](https://github.com/cosmos/interchain-security/pull/703). The time to detect incorrect execution evidence is 7 days (the same time as that of detecting a light client attack on a client with a `trustingPeriod` of 14 days); We have 7 days to detect an incorrect execution and then 14 days for the proposal to be accepted. +In an Opt In chain, a set of validators might attempt to perform an invalid-execution attack. To deter such potential attacks, PSS allows for the use of fraud votes. +A _fraud vote_ is a governance proposal that enables the slashing of validators that performed an invalid-execution attack. +Due to their inherent complexity, we intend to introduce fraud votes in a different ADR and at a future iteration of PSS. -##### When are fraud votes eligible? -Because fraud votes allow the slashing of a validator through a governance proposal, we are cautious on when a fraud vote is eligible and try to restrict their eligibility as much as possible to avoid an adversary proposing a bogus fraud vote. -As described earlier, because in Top N chains we consider that `N > 50%` we only consider validators that have opted in an Opt In chain. Fraud votes are also **only** intended to be used for **malicious** behaviour, for example, in a scenario similar to what [happened to Neutron](https://blog.neutron.org/neutron-halt-post-mortem-927dbe4540c8), validators should vote against slashing those validators. - -#### Message -```protobuf -message FraudVoteProposal { - string title = 1; - string description = 2; - - // validator that performed incorrect execution (consensus address on consumer chain) - Validator validator = 3; - - // validator's vote contains BlockId on what was signed - Vote vote = 4; - - // header for which the hash led to the blockId in the vote - Header header = 5; -} -``` - -The message includes the `vote` that was signed by the validator. Validators that vote on a fraud vote can look at the `BlockID` contained in a `vote` and the `appHash`, `lastResultHash`, etc. in the `header` (that led to this `BlockID`) to conclude whether the to-be-voted validator has performed incorrect execution or not. #### Double signing We do not change the way slashing for double signing and light client attacks functions. If a validator misbehaves on a consumer, then we slash that validator on the provider. From 45c796d8ce8a543aa4042fa0b434455397903264 Mon Sep 17 00:00:00 2001 From: insumity Date: Mon, 29 Jan 2024 18:50:21 +0100 Subject: [PATCH 18/26] smaller updates --- docs/docs/adrs/adr-015-partial-set-security.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/docs/adrs/adr-015-partial-set-security.md b/docs/docs/adrs/adr-015-partial-set-security.md index 36d568a3d3..2ae3600833 100644 --- a/docs/docs/adrs/adr-015-partial-set-security.md +++ b/docs/docs/adrs/adr-015-partial-set-security.md @@ -140,7 +140,7 @@ message MsgOptOut { string provider_addr = 2; } ``` -Validators can only opt out after a consumer chain has started and hence the above message returns an error if the chain with `chain_id` is not running. +Validators can only opt out after a consumer chain has started and hence the above message returns an error if the chain with `chain_id` is not running. Additionally, a validator that belongs to the top N% validators cannot opt out from a Top N and hence a `MsgOptOut` would error in such a case. #### State & Query We also update the state of the opted-in validators when a validator has opted out by removing it from there. @@ -286,7 +286,6 @@ We do not change the way downtime jailing functions. If a validator is down on a - Small validators are not forced to validate chains anymore if they do not want to. ### Negative -- Fraud votes introduce one more way to slash validators. - Depending on whether a consumer chain is Top N or Opt In, a consumer chain might not be as secure as with Replicated Security. From 0b56b0f1d9030f008975a3311313757d8861ef99 Mon Sep 17 00:00:00 2001 From: insumity Date: Mon, 29 Jan 2024 18:57:50 +0100 Subject: [PATCH 19/26] add consumer key in the opt in message --- docs/docs/adrs/adr-015-partial-set-security.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/docs/adrs/adr-015-partial-set-security.md b/docs/docs/adrs/adr-015-partial-set-security.md index 2ae3600833..c26a77bc76 100644 --- a/docs/docs/adrs/adr-015-partial-set-security.md +++ b/docs/docs/adrs/adr-015-partial-set-security.md @@ -81,6 +81,8 @@ message MsgOptIn { string chain_id = 1; // the provider address of the validator string provider_addr = 2; + // (optional) the consensus public key to use on the consumer + optional string consumer_key = 3; } ``` Note that in a Top N consumer chain, the top `N%` provider validators have to validate the consumer chain. @@ -91,7 +93,7 @@ By automatically opting in validators when they enter the top `N%` of the valida Note that a validator can send a `MsgOptIn` message even if the consumer chain is not yet running. To do this we reuse the [`IsConsumerProposedOrRegistered`](https://github.com/cosmos/interchain-security/blob/v4.0.0/x/ccv/provider/keeper/key_assignment.go#L644). If the `chainID` does not exist, the `MsgOptIn` should fail, as well as if the provider address does not exist. -Additionally, a validator that opts in can then use `MsgAssignConsumerKey` to change the consumer key on the consumer chain as is currently done in Replicated Security. +Optionally, a validator that opts in can provide a `consumer_key` so that it assigns a different consumer key (from the provider) to the consumer chain. Naturally, a validator that does not assign a key when opting in can always send a `MsgAssignConsumerKey` to change the consumer key on the consumer chain at a later point in time, as is done in Replicated Security. #### State & Query For each validator, we store on whether the validator has opted in and at the block height they opted in under the key: From 4ccb9252d85b7e43b39b5ce5f4e2045df561211f Mon Sep 17 00:00:00 2001 From: insumity Date: Tue, 30 Jan 2024 11:24:12 +0100 Subject: [PATCH 20/26] remove partial set computation and reward pseudocode --- .../docs/adrs/adr-015-partial-set-security.md | 203 ++++++------------ 1 file changed, 63 insertions(+), 140 deletions(-) diff --git a/docs/docs/adrs/adr-015-partial-set-security.md b/docs/docs/adrs/adr-015-partial-set-security.md index c26a77bc76..a2fe345be6 100644 --- a/docs/docs/adrs/adr-015-partial-set-security.md +++ b/docs/docs/adrs/adr-015-partial-set-security.md @@ -34,24 +34,26 @@ We intend to implement PSS using a feature branch off [v4.0.0 interchain securit ### How do consumer chains join? As a simplification and to avoid [chain id squatting](https://forum.cosmos.network/t/pss-permissionless-vs-premissioned-lite-opt-in-consumer-chains/12984/17), a consumer chain can only join PSS through a governance proposal and not in a permissionless way. -Consumer chains join PSS the same way chains now join Replicated Security (e.g., see [Stride](https://www.mintscan.io/cosmos/proposals/799)). We extend [`ConsumerAdditionProposal`](https://github.com/cosmos/interchain-security/blob/v4.0.0/proto/interchain_security/ccv/provider/v1/provider.proto#L27) with one optional field: +Consumer chains join PSS the same way chains now join Replicated Security, namely through a `ConsumerAdditionProposal` proposal. +We extend [`ConsumerAdditionProposal`](https://github.com/cosmos/interchain-security/blob/v4.0.0/proto/interchain_security/ccv/provider/v1/provider.proto#L27) with one optional field: `string top_N_fraction`: Corresponds to the percentage of validators that join under the Top N case. -For example, `0.53` corresponds to a Top 53% chain, meaning that the top `53%` provider validators would validate the proposed consumer chain. -If the `string` is not set, the consumer chain corresponds to an Opt In chain. -`top_N_fraction` resides between `[0.5, 0.95]`. -Assuming that at most `1/3` of provider validators could be malicious, by having `0.5` (`50%`) as the minimum value for `top_N_fraction` we guarantee that we cannot have a successful invalid-execution attack on a Top N consumer chain. -This is because, a Top N consumer chain with `N >= 50%` would have at least `1/3` honest validators, which is sufficient to stop any invalid-execution attack. -Additionally, by having `N >= 50%` (and hence `N > 33%`) we enable the top N validators to `Veto` any `ConsumerAdditionProposal` for consumer chains they do not want to validate. +For example, `0.53` corresponds to a Top 53% chain, meaning that the top `53%` provider validators have to validate the proposed consumer chain. +`top_N_fraction` can be `0` or include any value in `[0.5, 0.95]`. A chain can join with `top_N_fraction == 0` as an Opt In, or with `top_N_fraction ∈ [0.5, 0.95]` as a Top N chain. -`top_N_fraction` can be up to `0.95` (`95%`) to capture the current case of Replicated Security where we allow the bottom `5%` of validators to opt out. There is no reason for `top_N_fraction` to be higher than `95%`. Smaller chains that belong in the bottom `5%` validators can choose to opt in if they want to validate. +In case of a Top N chain, we restrict the possible values of `top_N_fraction` from `(0, 1]` to `[0.5, 0.95]`. +By having `top_N_fraction >= 0.5` we can guarantee that we cannot have a successful invalid-execution attack, assuming that at most `1/3` of provider validators can be malicious. +This is because, a Top N chain with `N >= 50%` would have at least `1/3` honest validators, which is sufficient to stop invalid-execution attacks. +Additionally, by having `N >= 50%` (and hence `N > (VetoThreshold = 33.4%)`) we enable the top N validators to `Veto` any `ConsumerAdditionProposal` for consumer chains they do not want to validate. +`top_N_fraction` can be up to `0.95` (`95%`) to capture how Replicated Security is currently used, where we allow the bottom `5%` of validators to soft opt out. +Validators that belong in the bottom `5%` of validators can choose to opt in if they want to validate. If a proposal has those arguments wrongly set, it should get rejected in [ValidateBasic](https://github.com/cosmos/interchain-security/blob/v4.0.0/x/ccv/provider/types/proposal.go#L86). -In the code, we distinguish whether a chain is _Top N_ or _Opt In_ by checking whether `top_N_fraction` is set or not. +In the code, we distinguish whether a chain is _Top N_ or _Opt In_ by checking whether `top_N_fraction` is zero or not. -In a future version of PSS, we intend to introduce a way to modify the parameters of a consumer chain, e.g, a chain that is _Opt In_ to become _Top N_, etc. +In a future version of PSS, we intend to introduce a `ConsumerModificationProposal` so that we can modify the parameters of a consumer chain, e.g, a chain that is _Opt In_ to become _Top N_, etc. #### State & Query We augment the provider module’s state to keep track of the `top_N_fraction` value for each consumer chain. The key to store this information would be: @@ -63,40 +65,41 @@ To create the above key, we can use [`ChainIdWithLenKey`](https://github.com/cos Then in the [keeper](https://github.com/cosmos/interchain-security/blob/v4.0.0/x/ccv/provider/keeper/keeper.go) we introduce methods as follows: ```golang -func (k Keeper) SetTopN(ctx sdk.Context, chainID string, topNFraction string) +func (k Keeper) SetTopNFraction(ctx sdk.Context, chainID string, topNFraction string) func (k Keeper) IsTopN(ctx sdk.Context, chainID string) bool func (k Keeper) IsOptIn(ctx sdk.Context, chainID string) bool // returns the N if Top N chain, otherwise an error -func (k Keeper) GetTopN(ctx sdk.Context, chainID string) (uint8, error) +func (k Keeper) GetTopNFraction(ctx sdk.Context, chainID string) (Dec, error) ``` -We also extend the `interchain-security-pd query provider list-consumer-chains` query to return information on whether a consumer chain is an Opt In or a Top N chain and with what N. This way, block explorers like [Mintscan](https://www.mintscan.io/) could show for a chain “This chain is secured by N% of the provider chain.” +We also extend the `interchain-security-pd query provider list-consumer-chains` query to return information on whether a consumer chain is an Opt In or a Top N chain and with what fraction of N. This way, block explorers can present informative messages such as "This chain is secured by N% of the provider chain" for consumer chains. ### How do validators opt in? A validator can opt in by sending a new type of message that we introduce in [tx.proto](https://github.com/cosmos/interchain-security/blob/v4.0.0/proto/interchain_security/ccv/provider/v1/tx.proto#L1). ```protobuf message MsgOptIn { // the chain id of the consumer chain to opt in to - string chain_id = 1; + string chainID = 1; // the provider address of the validator - string provider_addr = 2; + string providerAddr = 2; // (optional) the consensus public key to use on the consumer - optional string consumer_key = 3; + optional string consumerKey = 3; } ``` Note that in a Top N consumer chain, the top `N%` provider validators have to validate the consumer chain. Nevertheless, validators in the bottom `(100 - N)%` can opt in to validate as well. -Provider validators that belong to the top `N%` validators are immediately opted in to validate a Top N consumer chain. -This means that if a validator `V` belongs to the top `N%` validators but later falls (e.g., due to undelegations) to the bottom `(100 - N)%`, `V` would still be considered opted in and has to validate unless `V` sends a `MsgOptOut` message (see below). -By automatically opting in validators when they enter the top `N%` of the validators and by forcing top `N%` validators to explicitly opt out in case they fall to the `(100 - N)%` bottom validators we simplify our implementation because i) we only have to find all the opted in validators to retrieve the validators that have to validate a consumer chain, and ii) we simplify reward distribution by preventing cases where we opted out a validator before it receives its rewards. +Provider validators that belong or enter the top `N%` validators are _automatically_ opted in to validate a Top N consumer chain. +This means that if a validator `V` belongs to the top `N%` validators but later falls (e.g., due to undelegations) to the bottom `(100 - N)%`, `V` is still considered opted in and has to validate unless `V` sends a `MsgOptOut` message (see below). +By automatically opting in validators when they enter the top `N%` validators and by forcing top `N%` validators to explicitly opt out in case they fall to the `(100 - N)%` bottom validators we simplify the design of PSS. Note that a validator can send a `MsgOptIn` message even if the consumer chain is not yet running. To do this we reuse the [`IsConsumerProposedOrRegistered`](https://github.com/cosmos/interchain-security/blob/v4.0.0/x/ccv/provider/keeper/key_assignment.go#L644). If the `chainID` does not exist, the `MsgOptIn` should fail, as well as if the provider address does not exist. -Optionally, a validator that opts in can provide a `consumer_key` so that it assigns a different consumer key (from the provider) to the consumer chain. Naturally, a validator that does not assign a key when opting in can always send a `MsgAssignConsumerKey` to change the consumer key on the consumer chain at a later point in time, as is done in Replicated Security. +Optionally, a validator that opts in can provide a `consumerKey` so that it assigns a different consumer key (from the provider) to the consumer chain. +Naturally, a validator can always change the consumer key on a consumer chain by sending a `MsgAssignConsumerKey` message at a later point in time, as is done in Replicated Security. #### State & Query -For each validator, we store on whether the validator has opted in and at the block height they opted in under the key: +For each validator, we store on whether the validator has opted in and at which block height the validator opted in under the key: ``` optedInBytePrefix | len(chainID) | chainID | addr ``` @@ -110,48 +113,51 @@ func (k Keeper) GetOptedInValidators(ctx sdk.Context, chainID string) []Validato func (k Keeper) IsValidatorOptedIn(ctx sdk.Context, chainID string, val Validator) bool ``` -We introduce a query to retrieve the validators that are opted in and hence the validators that need to validate the consumer chain: +We introduce the following two queries: ```bash interchain-security-pd query provider optedInValidators $chainID +interchain-security-pd query provider hasToValidate $providerAddr ``` - -We also introduce a query that given a validator's address returns all the chains this validator has to validate: -```bash -interchain-security-pd query provider hasToValidate providerAddr -``` +One query to retrieve the validators that are opted in and hence the validators that need to validate the consumer chain and one query that given a validator's address returns all the chains this validator has to validate. #### When do validators opt in? -Validators can not only opt in by sending a `MsgOptIn` message, but can also opt in automatically if they voted `Yes` -during the `ConsumerAdditionProposal` that introduced a consumer chain. This simplifies validators operations because -they do not have to send an additional message to opt in. +As described earlier, validators can manually opt in by sending a `MsgOptIn` message. +Additionally, in a Top N chain, a validator is automatically opted in when it moves from the bottom `(100 - N)%` to the top `N%` validators. + +Lastly, validators can also opt in if they vote `Yes` during the `ConsumerAdditionProposal` that introduces a consumer chain. +This simplifies validators operations because they do not have to send an additional message to opt in. -Because the `Tally` method [deletes the votes](https://github.com/cosmos/cosmos-sdk/blob/v0.47.7/x/gov/keeper/tally.go#L71) after reading them, we cannot check the votes of the validators after the votes have been tallied. To circumvent this, we introduce -a hook for [`AfterProposalVote`](https://github.com/cosmos/cosmos-sdk/blob/v0.47.7/x/gov/keeper/vote.go#L35) and keep track -of all the votes cast by a validator. If a validator votes more than once, we only consider the latest vote. +Because the `Tally` method [deletes the votes](https://github.com/cosmos/cosmos-sdk/blob/v0.47.7/x/gov/keeper/tally.go#L71) after reading them, we cannot check the votes of the validators after the votes have been tallied. +To circumvent this, we introduce a hook for [`AfterProposalVote`](https://github.com/cosmos/cosmos-sdk/blob/v0.47.7/x/gov/keeper/vote.go#L35) and keep track of all the votes cast by a validator. +If a validator casts more than one vote, we only consider the latest vote. Finally, we only consider a validator has opted in if it casts a 100% `Yes` vote in case of a [weighted vote](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-037-gov-split-vote.md). ### How do validators opt out? -Validators that have opted in on a chain can opt out as well by sending the following message: +Validators that have opted in on a chain can opt out by sending the following message: ```protobuf message MsgOptOut { // the chain id of the consumer chain to opt out from - string chain_id = 1; + string chainID = 1; // the provider address of the validator - string provider_addr = 2; + string providerAddr = 2; } ``` -Validators can only opt out after a consumer chain has started and hence the above message returns an error if the chain with `chain_id` is not running. Additionally, a validator that belongs to the top N% validators cannot opt out from a Top N and hence a `MsgOptOut` would error in such a case. +Validators can only opt out after a consumer chain has started and hence the above message returns an error if the chain with `chainID` is not running. +Additionally, a validator that belongs to the top `N%` validators cannot opt out from a Top N chain and hence a `MsgOptOut` would error in such a case. #### State & Query -We also update the state of the opted-in validators when a validator has opted out by removing it from there. +We also update the state of the opted-in validators when a validator has opted out by removing the opted-out validator. -Note that we only punish validators for downtime that were opted in in a consumer chain. For this, we keep historical info of all the validators that have opted in during the last `X` (to be defined) blocks. This way we can jail validators for downtime knowing that indeed the validators have opted in at some point in the past. -Otherwise, we can think of a scenario where a validator `V` is down for a period of time, but before `V` gets punished for downtime, validator `V` opts out and then we do not know whether `V` should be punished or not. +Note that we only punish validators for downtime that were opted in on a consumer chain. +For this, we keep historical info of all the validators that have opted in during the last `X` (to be defined) blocks. +This way we can jail validators for downtime knowing that indeed the validators have opted in at some point in the past. +Otherwise, we can think of a scenario where a validator `V` is down for a period of time, but before `V` gets punished for downtime, validator `V` opts out, and then we do not know whether `V` should be punished or not. ### When does a consumer chain start? -A Top N consumer chain starts at the specified date (`spawn_time`) if the [`ConsumerAdditionProposal`](https://github.com/cosmos/interchain-security/blob/v4.0.0/proto/interchain_security/ccv/provider/v1/provider.proto#L27) has passed. An Opt In consumer chain starts if at least one validator has opted in. We check this in [BeginBlockInit](https://github.com/cosmos/interchain-security/blob/v4.0.0/x/ccv/provider/keeper/proposal.go#L357): +A Top N consumer chain always starts at the specified date (`spawn_time`) if the [`ConsumerAdditionProposal`](https://github.com/cosmos/interchain-security/blob/v4.0.0/proto/interchain_security/ccv/provider/v1/provider.proto#L27) has passed. +An Opt In consumer chain only starts if at least one validator has opted in. We check this in [BeginBlockInit](https://github.com/cosmos/interchain-security/blob/v4.0.0/x/ccv/provider/keeper/proposal.go#L357): ```golang func (k Keeper) BeginBlockInit(ctx sdk.Context) { propsToExecute := k.GetConsumerAdditionPropsToExecute(ctx) @@ -167,105 +173,21 @@ func (k Keeper) BeginBlockInit(ctx sdk.Context) { ``` ### How do we send the partial validator sets to the consumer chains? -To send the validator set to a consumer chain we first have to generate the validator updates with a function similar to this one: - -```golang -func (k Keeper) GetPartialSetValidators(ctx types.Context, chainID string, valUpdates []abci.ValidatorUpdate) (newUpdates []abci.ValidatorUpdate) - - powerSum := sdk.ZeroDec() - totalPower := k.ComputeTotalPower(ctx) - topNThreshold := k.GetTopN(ctx, chainID) / 100 - for _, val := range valUpdates { - powerSum = powerSum.Add(sdk.NewDecFromInt(sdk.NewInt(val.Power))) - // if powerSum / totalPower > topNThreshold - if k.IsTopN(ctx, chainID) && powerSum.Quo(totalPower).GT(topNThreshold) { - // is Top N validator - updates = append(updates, abci.ValidatorUpdate{ - PubKey: val.PubKey, - Power: val.Power, - }) - } else { - if k.IsValidatorOptedIn(ctx, chainID, val.PubKey) { - updates = append(updates, abci.ValidatorUpdate{ - PubKey: val.PubKey, - Power: val.Power, - }) - } else { - // remove this validator - updates = append(updates, abci.ValidatorUpdate{ - PubKey: val.PubKey, - Power: 0, - }) - } - } - - return updates -} -``` -And then we change `QueueVSCPackets` to: - -```golang -// QueueVSCPackets queues latest validator updates for every registered consumer chain -func (k Keeper) QueueVSCPackets(ctx sdk.Context) { - valUpdateID := k.GetValidatorSetUpdateId(ctx) // current valset update ID - // Get the validator updates from the staking module. - // Note: GetValidatorUpdates panics if the updates provided by the x/staking module - // of cosmos-sdk is invalid. - valUpdates := k.stakingKeeper.GetValidatorUpdates(ctx) - - for _, chain := range k.GetAllConsumerChains(ctx) { - // Apply the key assignment to the validator updates. - valUpdates := k.GetPartialSetValidators(ctx, chain.ChainId, valUpdates) - valUpdates := k.MustApplyKeyAssignmentToValUpdates(ctx, chain.ChainId, valUpdates) - ... -``` - -Note that we use `0` to remove validators that are not opted in as described [here](https://docs.cometbft.com/v0.37/spec/abci/abci++_methods#endblock) and as is done for the [assignment of consumer keys](https://github.com/cosmos/interchain-security/blob/v4.0.0/x/ccv/provider/keeper/key_assignment.go#L525). +A consumer chain should only be validated by opted in validators. +We introduce logic to do this when we [queue](https://github.com/cosmos/interchain-security/blob/v4.0.0/x/ccv/provider/keeper/relay.go#L213) the `VSCPacket`s. +The logic behind this, is not as straightforward as it seems because CometBFT does not receive the validator set that has to validate a chain, but rather a delta of [validator updates](https://docs.cometbft.com/v0.37/spec/abci/abci++_methods#validatorupdate). +For example, to remove an opted-out validator from a consumer chain, we have to send a validator update with a `power` of `0`, similarly to what is done in the [assignment of consumer keys](https://github.com/cosmos/interchain-security/blob/v4.0.0/x/ccv/provider/keeper/key_assignment.go#L525). +We intend to update this ADR at a later stage on how exactly we intend to implement this logic. ### How do we distribute rewards? Currently, rewards are distributed as follows: The consumer [periodically sends rewards](https://github.com/cosmos/interchain-security/blob/v4.0.0/x/ccv/consumer/keeper/distribution.go#L148) on the provider `ConsumerRewardsPool` address. The provider then [transfers those rewards to the fee collector address](https://github.com/cosmos/interchain-security/blob/v4.0.0/x/ccv/provider/keeper/distribution.go#L77) and those transferred rewards are distributed to validators and delegators. -In PSS, we distribute rewards only to validators that actually validate the consumer chain. To do this, we have a pool associated with each consumer chain and consumers IBC transfer the rewards to this pool. -Then, because we do not use the canonical fee pool and hence rewards are not automatically distributed, we will create a new and modified version of [AllocateTokens](https://github.com/cosmos/cosmos-sdk/blob/v0.47.7/x/distribution/keeper/allocation.go#L14) as follows: +In PSS, we distribute rewards only to validators that actually validate the consumer chain. +To do this, we have a pool associated with each consumer chain and consumers IBC transfer the rewards to this pool. +We then extract the rewards from each consumer pool and distribute them to the opted in validators. -```golang -func (k Keeper) AllocateTokensFromPool(ctx sdk.Context, poolAddress string, totalPreviousPower int64, bondedVotes []abci.VoteInfo) { - rewardsCollectedInt := k.bankKeeper.GetAllBalances(ctx, poolAddress) - // rest in the same that `AllocateTokens` operates - ... -``` - -In [TransferRewardsToFeeCollector](https://github.com/cosmos/interchain-security/blob/v4.0.0/x/ccv/provider/keeper/distribution.go#L61) we then do: - -```golang -for _, chain := range consumerChains { - ... - totalPower := 0 - var []abci.VoteInfo votes - for _, val := range (validators that opted in or Top N) { - if val has opted in for more than X blocks { - votes = append(votes, []abci.VoteInfo{{Validator: val, SignedLastBlock: true}}) - } - } - - k.AllocateTokens(ctx, pool associated with this chain, totalPower, votes) - ... -} -``` - -instead of calling: -```golang -// 4. Transfer the balance to the fee collector address -err := k.bankKeeper.SendCoinsFromModuleToModule( - ctx, - types.ConsumerRewardsPool, - k.feeCollectorName, - sdk.NewCoins(balance), -) -``` - -Note that we would only distribute rewards to validators that are opted in but are opted in for some time (e.g., 10000 blocks) to avoid cases where validators opt in and out in short intervals just in order to receive rewards. +Note that we only distribute rewards to validators that have been opted in for some time (e.g., 10000 blocks) to avoid cases where validators opt in just to receive rewards and then opt out immediately afterward. ### Misbehaviour @@ -276,20 +198,21 @@ Due to their inherent complexity, we intend to introduce fraud votes in a differ #### Double signing -We do not change the way slashing for double signing and light client attacks functions. If a validator misbehaves on a consumer, then we slash that validator on the provider. +We do not change the way slashing for double signing and light client attacks functions. +If a validator misbehaves on a consumer, then we slash that validator on the provider. #### Downtime -We do not change the way downtime jailing functions. If a validator is down on a consumer chain for an adequate amount of time, we jail this validator on the provider only if the validator was opted in in the recent past. +We do not change the way downtime jailing functions. +If a validator is down on a consumer chain for an adequate amount of time, we jail this validator on the provider but only if the validator was opted in on this consumer chain in the recent past. ## Consequences ### Positive -- Easier for new consumer chains to consume the provider's chain security because proposals are more likely to pass if not everyone is forced to validate. -- Small validators are not forced to validate chains anymore if they do not want to. +- Easier for new consumer chains to consume the provider's chain economic security because proposals are more likely to pass if not everyone is forced to validate. +- Smaller validators are not forced to validate chains anymore if they do not want to. ### Negative -- Depending on whether a consumer chain is Top N or Opt In, a consumer chain might not be as secure as with Replicated Security. - +- A consumer chain does not receive the same economic security as with Replicated Security, unless it is a Top N chain with `N = 95%`. ## References From 5ac5b122446e8b09344f2fb0c889eada6e4a4116 Mon Sep 17 00:00:00 2001 From: insumity Date: Tue, 30 Jan 2024 15:55:41 +0100 Subject: [PATCH 21/26] changed top_N_fraction to int --- docs/docs/adrs/adr-015-partial-set-security.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/docs/adrs/adr-015-partial-set-security.md b/docs/docs/adrs/adr-015-partial-set-security.md index a2fe345be6..9e863763ae 100644 --- a/docs/docs/adrs/adr-015-partial-set-security.md +++ b/docs/docs/adrs/adr-015-partial-set-security.md @@ -37,16 +37,16 @@ As a simplification and to avoid [chain id squatting](https://forum.cosmos.netwo Consumer chains join PSS the same way chains now join Replicated Security, namely through a `ConsumerAdditionProposal` proposal. We extend [`ConsumerAdditionProposal`](https://github.com/cosmos/interchain-security/blob/v4.0.0/proto/interchain_security/ccv/provider/v1/provider.proto#L27) with one optional field: -`string top_N_fraction`: Corresponds to the percentage of validators that join under the Top N case. -For example, `0.53` corresponds to a Top 53% chain, meaning that the top `53%` provider validators have to validate the proposed consumer chain. -`top_N_fraction` can be `0` or include any value in `[0.5, 0.95]`. A chain can join with `top_N_fraction == 0` as an Opt In, or with `top_N_fraction ∈ [0.5, 0.95]` as a Top N chain. +`uint32 top_N_fraction`: Corresponds to the percentage of validators that join under the Top N case. +For example, `53` corresponds to a Top 53% chain, meaning that the top `53%` provider validators have to validate the proposed consumer chain. +`top_N_fraction` can be `0` or include any value in `[50, 95]`. A chain can join with `top_N_fraction == 0` as an Opt In, or with `top_N_fraction ∈ [0.5, 0.95]` as a Top N chain. -In case of a Top N chain, we restrict the possible values of `top_N_fraction` from `(0, 1]` to `[0.5, 0.95]`. -By having `top_N_fraction >= 0.5` we can guarantee that we cannot have a successful invalid-execution attack, assuming that at most `1/3` of provider validators can be malicious. +In case of a Top N chain, we restrict the possible values of `top_N_fraction` from `(0, 100]` to `[50, 95]`. +By having `top_N_fraction >= 50` we can guarantee that we cannot have a successful invalid-execution attack, assuming that at most `1/3` of provider validators can be malicious. This is because, a Top N chain with `N >= 50%` would have at least `1/3` honest validators, which is sufficient to stop invalid-execution attacks. Additionally, by having `N >= 50%` (and hence `N > (VetoThreshold = 33.4%)`) we enable the top N validators to `Veto` any `ConsumerAdditionProposal` for consumer chains they do not want to validate. -`top_N_fraction` can be up to `0.95` (`95%`) to capture how Replicated Security is currently used, where we allow the bottom `5%` of validators to soft opt out. +`top_N_fraction` can be up to `95` (`95%`) to capture how Replicated Security is currently used, where we allow the bottom `5%` of validators to soft opt out. Validators that belong in the bottom `5%` of validators can choose to opt in if they want to validate. If a proposal has those arguments wrongly set, it should get rejected in [ValidateBasic](https://github.com/cosmos/interchain-security/blob/v4.0.0/x/ccv/provider/types/proposal.go#L86). From 537da61725e373f67d00618ff253ebb9ff4dcf11 Mon Sep 17 00:00:00 2001 From: insumity Date: Tue, 30 Jan 2024 15:56:44 +0100 Subject: [PATCH 22/26] nit changes --- docs/docs/adrs/adr-015-partial-set-security.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/adrs/adr-015-partial-set-security.md b/docs/docs/adrs/adr-015-partial-set-security.md index 9e863763ae..3b0d2a4664 100644 --- a/docs/docs/adrs/adr-015-partial-set-security.md +++ b/docs/docs/adrs/adr-015-partial-set-security.md @@ -39,7 +39,7 @@ We extend [`ConsumerAdditionProposal`](https://github.com/cosmos/interchain-secu `uint32 top_N_fraction`: Corresponds to the percentage of validators that join under the Top N case. For example, `53` corresponds to a Top 53% chain, meaning that the top `53%` provider validators have to validate the proposed consumer chain. -`top_N_fraction` can be `0` or include any value in `[50, 95]`. A chain can join with `top_N_fraction == 0` as an Opt In, or with `top_N_fraction ∈ [0.5, 0.95]` as a Top N chain. +`top_N_fraction` can be `0` or include any value in `[50, 95]`. A chain can join with `top_N_fraction == 0` as an Opt In, or with `top_N_fraction ∈ [50, 95]` as a Top N chain. In case of a Top N chain, we restrict the possible values of `top_N_fraction` from `(0, 100]` to `[50, 95]`. By having `top_N_fraction >= 50` we can guarantee that we cannot have a successful invalid-execution attack, assuming that at most `1/3` of provider validators can be malicious. From 01a709778acdebcdd92e1aec994d29b30c079d66 Mon Sep 17 00:00:00 2001 From: insumity Date: Tue, 30 Jan 2024 15:58:56 +0100 Subject: [PATCH 23/26] remove fraction --- docs/docs/adrs/adr-015-partial-set-security.md | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/docs/docs/adrs/adr-015-partial-set-security.md b/docs/docs/adrs/adr-015-partial-set-security.md index 3b0d2a4664..4639300442 100644 --- a/docs/docs/adrs/adr-015-partial-set-security.md +++ b/docs/docs/adrs/adr-015-partial-set-security.md @@ -37,26 +37,26 @@ As a simplification and to avoid [chain id squatting](https://forum.cosmos.netwo Consumer chains join PSS the same way chains now join Replicated Security, namely through a `ConsumerAdditionProposal` proposal. We extend [`ConsumerAdditionProposal`](https://github.com/cosmos/interchain-security/blob/v4.0.0/proto/interchain_security/ccv/provider/v1/provider.proto#L27) with one optional field: -`uint32 top_N_fraction`: Corresponds to the percentage of validators that join under the Top N case. +`uint32 top_N`: Corresponds to the percentage of validators that join under the Top N case. For example, `53` corresponds to a Top 53% chain, meaning that the top `53%` provider validators have to validate the proposed consumer chain. -`top_N_fraction` can be `0` or include any value in `[50, 95]`. A chain can join with `top_N_fraction == 0` as an Opt In, or with `top_N_fraction ∈ [50, 95]` as a Top N chain. +`top_N` can be `0` or include any value in `[50, 95]`. A chain can join with `top_N == 0` as an Opt In, or with `top_N ∈ [50, 95]` as a Top N chain. -In case of a Top N chain, we restrict the possible values of `top_N_fraction` from `(0, 100]` to `[50, 95]`. -By having `top_N_fraction >= 50` we can guarantee that we cannot have a successful invalid-execution attack, assuming that at most `1/3` of provider validators can be malicious. +In case of a Top N chain, we restrict the possible values of `top_N` from `(0, 100]` to `[50, 95]`. +By having `top_N >= 50` we can guarantee that we cannot have a successful invalid-execution attack, assuming that at most `1/3` of provider validators can be malicious. This is because, a Top N chain with `N >= 50%` would have at least `1/3` honest validators, which is sufficient to stop invalid-execution attacks. Additionally, by having `N >= 50%` (and hence `N > (VetoThreshold = 33.4%)`) we enable the top N validators to `Veto` any `ConsumerAdditionProposal` for consumer chains they do not want to validate. -`top_N_fraction` can be up to `95` (`95%`) to capture how Replicated Security is currently used, where we allow the bottom `5%` of validators to soft opt out. +`top_N` can be up to `95` (`95%`) to capture how Replicated Security is currently used, where we allow the bottom `5%` of validators to soft opt out. Validators that belong in the bottom `5%` of validators can choose to opt in if they want to validate. If a proposal has those arguments wrongly set, it should get rejected in [ValidateBasic](https://github.com/cosmos/interchain-security/blob/v4.0.0/x/ccv/provider/types/proposal.go#L86). -In the code, we distinguish whether a chain is _Top N_ or _Opt In_ by checking whether `top_N_fraction` is zero or not. +In the code, we distinguish whether a chain is _Top N_ or _Opt In_ by checking whether `top_N` is zero or not. In a future version of PSS, we intend to introduce a `ConsumerModificationProposal` so that we can modify the parameters of a consumer chain, e.g, a chain that is _Opt In_ to become _Top N_, etc. #### State & Query -We augment the provider module’s state to keep track of the `top_N_fraction` value for each consumer chain. The key to store this information would be: +We augment the provider module’s state to keep track of the `top_N` value for each consumer chain. The key to store this information would be: ``` topNFractionBytePrefix | len(chainID) | chainID @@ -73,7 +73,8 @@ func (k Keeper) IsOptIn(ctx sdk.Context, chainID string) bool func (k Keeper) GetTopNFraction(ctx sdk.Context, chainID string) (Dec, error) ``` -We also extend the `interchain-security-pd query provider list-consumer-chains` query to return information on whether a consumer chain is an Opt In or a Top N chain and with what fraction of N. This way, block explorers can present informative messages such as "This chain is secured by N% of the provider chain" for consumer chains. +We also extend the `interchain-security-pd query provider list-consumer-chains` query to return information on whether a consumer chain is an Opt In or a Top N chain and with what N. +This way, block explorers can present informative messages such as "This chain is secured by N% of the provider chain" for consumer chains. ### How do validators opt in? A validator can opt in by sending a new type of message that we introduce in [tx.proto](https://github.com/cosmos/interchain-security/blob/v4.0.0/proto/interchain_security/ccv/provider/v1/tx.proto#L1). From 3dea3ffadffb9154b77d054ff632825769b51ada Mon Sep 17 00:00:00 2001 From: insumity Date: Wed, 31 Jan 2024 09:55:59 +0100 Subject: [PATCH 24/26] Update docs/docs/adrs/adr-015-partial-set-security.md Co-authored-by: Marius Poke --- docs/docs/adrs/adr-015-partial-set-security.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/adrs/adr-015-partial-set-security.md b/docs/docs/adrs/adr-015-partial-set-security.md index 4639300442..f28a23eb45 100644 --- a/docs/docs/adrs/adr-015-partial-set-security.md +++ b/docs/docs/adrs/adr-015-partial-set-security.md @@ -49,7 +49,7 @@ Additionally, by having `N >= 50%` (and hence `N > (VetoThreshold = 33.4%)`) we `top_N` can be up to `95` (`95%`) to capture how Replicated Security is currently used, where we allow the bottom `5%` of validators to soft opt out. Validators that belong in the bottom `5%` of validators can choose to opt in if they want to validate. -If a proposal has those arguments wrongly set, it should get rejected in [ValidateBasic](https://github.com/cosmos/interchain-security/blob/v4.0.0/x/ccv/provider/types/proposal.go#L86). +If a proposal has the `top_N` argument wrongly set, it should get rejected in [ValidateBasic](https://github.com/cosmos/interchain-security/blob/v4.0.0/x/ccv/provider/types/proposal.go#L86). In the code, we distinguish whether a chain is _Top N_ or _Opt In_ by checking whether `top_N` is zero or not. From c1bd9b8dab95426b289f260db559bc1817d19eff Mon Sep 17 00:00:00 2001 From: insumity Date: Wed, 31 Jan 2024 09:56:18 +0100 Subject: [PATCH 25/26] Update docs/docs/adrs/adr-015-partial-set-security.md Co-authored-by: Marius Poke --- docs/docs/adrs/adr-015-partial-set-security.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docs/adrs/adr-015-partial-set-security.md b/docs/docs/adrs/adr-015-partial-set-security.md index f28a23eb45..54691614a9 100644 --- a/docs/docs/adrs/adr-015-partial-set-security.md +++ b/docs/docs/adrs/adr-015-partial-set-security.md @@ -151,7 +151,7 @@ Additionally, a validator that belongs to the top `N%` validators cannot opt out #### State & Query We also update the state of the opted-in validators when a validator has opted out by removing the opted-out validator. -Note that we only punish validators for downtime that were opted in on a consumer chain. +Note that only opted-in validators can be punished for downtime on a consumer chain. For this, we keep historical info of all the validators that have opted in during the last `X` (to be defined) blocks. This way we can jail validators for downtime knowing that indeed the validators have opted in at some point in the past. Otherwise, we can think of a scenario where a validator `V` is down for a period of time, but before `V` gets punished for downtime, validator `V` opts out, and then we do not know whether `V` should be punished or not. From 2e5dd4ca2eb864e013c602651c755c98ecacf548 Mon Sep 17 00:00:00 2001 From: insumity Date: Wed, 31 Jan 2024 11:45:18 +0100 Subject: [PATCH 26/26] applied some changes based on comments --- docs/docs/adrs/adr-015-partial-set-security.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/docs/adrs/adr-015-partial-set-security.md b/docs/docs/adrs/adr-015-partial-set-security.md index 54691614a9..e15a888849 100644 --- a/docs/docs/adrs/adr-015-partial-set-security.md +++ b/docs/docs/adrs/adr-015-partial-set-security.md @@ -39,16 +39,13 @@ We extend [`ConsumerAdditionProposal`](https://github.com/cosmos/interchain-secu `uint32 top_N`: Corresponds to the percentage of validators that join under the Top N case. For example, `53` corresponds to a Top 53% chain, meaning that the top `53%` provider validators have to validate the proposed consumer chain. -`top_N` can be `0` or include any value in `[50, 95]`. A chain can join with `top_N == 0` as an Opt In, or with `top_N ∈ [50, 95]` as a Top N chain. +`top_N` can be `0` or include any value in `[50, 100]`. A chain can join with `top_N == 0` as an Opt In, or with `top_N ∈ [50, 100]` as a Top N chain. -In case of a Top N chain, we restrict the possible values of `top_N` from `(0, 100]` to `[50, 95]`. +In case of a Top N chain, we restrict the possible values of `top_N` from `(0, 100]` to `[50, 100]`. By having `top_N >= 50` we can guarantee that we cannot have a successful invalid-execution attack, assuming that at most `1/3` of provider validators can be malicious. This is because, a Top N chain with `N >= 50%` would have at least `1/3` honest validators, which is sufficient to stop invalid-execution attacks. Additionally, by having `N >= 50%` (and hence `N > (VetoThreshold = 33.4%)`) we enable the top N validators to `Veto` any `ConsumerAdditionProposal` for consumer chains they do not want to validate. -`top_N` can be up to `95` (`95%`) to capture how Replicated Security is currently used, where we allow the bottom `5%` of validators to soft opt out. -Validators that belong in the bottom `5%` of validators can choose to opt in if they want to validate. - If a proposal has the `top_N` argument wrongly set, it should get rejected in [ValidateBasic](https://github.com/cosmos/interchain-security/blob/v4.0.0/x/ccv/provider/types/proposal.go#L86). In the code, we distinguish whether a chain is _Top N_ or _Opt In_ by checking whether `top_N` is zero or not. @@ -100,10 +97,12 @@ Optionally, a validator that opts in can provide a `consumerKey` so that it assi Naturally, a validator can always change the consumer key on a consumer chain by sending a `MsgAssignConsumerKey` message at a later point in time, as is done in Replicated Security. #### State & Query -For each validator, we store on whether the validator has opted in and at which block height the validator opted in under the key: +For each validator, we store a pair `(blockHeight, isOptedIn)` that contains the block height the validator opted in and whether the validator is currently opted in or not, under the key: + ``` optedInBytePrefix | len(chainID) | chainID | addr ``` + By using a prefix iterator on `optedInBytePrefix | len(chainID) | chainID ` we retrieve all the opted in validators. We introduce the following `Keeper` methods. @@ -152,7 +151,7 @@ Additionally, a validator that belongs to the top `N%` validators cannot opt out We also update the state of the opted-in validators when a validator has opted out by removing the opted-out validator. Note that only opted-in validators can be punished for downtime on a consumer chain. -For this, we keep historical info of all the validators that have opted in during the last `X` (to be defined) blocks. +For this, we use historical info of all the validators that have opted in; We can examine the `blockHeight` stored under the key `optedInBytePrefix | len(chainID) | chainID | addr` to see if a validator was opted in. This way we can jail validators for downtime knowing that indeed the validators have opted in at some point in the past. Otherwise, we can think of a scenario where a validator `V` is down for a period of time, but before `V` gets punished for downtime, validator `V` opts out, and then we do not know whether `V` should be punished or not. @@ -211,9 +210,10 @@ If a validator is down on a consumer chain for an adequate amount of time, we ja ### Positive - Easier for new consumer chains to consume the provider's chain economic security because proposals are more likely to pass if not everyone is forced to validate. - Smaller validators are not forced to validate chains anymore if they do not want to. +- We can deprecate the soft opt-out implementation. ### Negative -- A consumer chain does not receive the same economic security as with Replicated Security, unless it is a Top N chain with `N = 95%`. +- A consumer chain does not receive the same economic security as with Replicated Security (assuming a value of `SoftOptOutThreshold` is `5%`), unless it is a Top N chain with `N >= 95%`. ## References