From 653139af16dad904821fe55bcc5a20aba0719c63 Mon Sep 17 00:00:00 2001 From: insumity Date: Tue, 12 Mar 2024 10:47:35 +0100 Subject: [PATCH 1/5] feat!: introduce epochs (#1691) * docs: modify epochs ADR to capture latest design (#1668) * modified ADR to capture the epoch design * feat!: introduce epochs (#1660) * cleanup ./changelog entries * rebase * fix!: Validation of SlashAcks fails due to marshaling to Bech32 (backport #1570) (#1577) fix!: Validation of SlashAcks fails due to marshaling to Bech32 (#1570) * add different Bech32Prefix for consumer and provider * separate app encoding and params * remove ConsumerValPubKey from ValidatorConfig * update addresses in tests * make SlashAcks consistent across chains * add comments for clarity * Regenerate traces * Fix argument order * set bech32prefix for provider to cosmos * add changelog entries * add consumer-double-downtime e2e test * update nightly-e2e workflow * fix typo * add consumer-double-downtime to testConfigs * remove changes on provider * skip invalid SlashAcks * seal the config * clear the outstanding downtime flag for new vals * add info on upgrading to v4.0.0 * fix upgrade handler * fix changeover e2e test * Update tests/e2e/config.go Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> * Update tests/e2e/config.go Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> * add AccountPrefix to ChainConfig * fix docstrings * update AccountAddressPrefix in app.go * fix consumer-misb e2e test --------- Co-authored-by: Philip Offtermatt Co-authored-by: Simon Noetzlin Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> (cherry picked from commit 86046926502f7b0ba795bebcdd1fdc97ac776573) Co-authored-by: Marius Poke * docs: update changelog for v4.0.0 (#1578) update changelog * docs: prepare for v4.0.0 (#1581) * unclog build * update release notes * update release date * added proto declaration * temp commit * temp commit * more changes * first commit * add param and fix tests * reduce epoch size for e2e * clean up * mbt fix * fix diff bug * cleaning up * cleaning up * cleaning up * cleaning up * cleaning up * cleaning up * added more tests * more fixes * nit fixes * cleaning up * increase downtime by one block * fix logs * took into account Marius' comments * tiny fixes * Update x/ccv/provider/keeper/params.go Co-authored-by: Simon Noetzlin * use Bech32 addresses as keys for maps * refactor nextBlocks(epoch) to nextEpoch * fixed comment * Remove new block creation during consumer chain setup * Revert "Remove new block creation during consumer chain setup" This reverts commit 85a52b74c1998dfebadfedbf287c9c9547cfec78. * added simple param test * added upper bound and addressed a comment * Add another edge case for diffing * used smarted solution (based on Philip's comment) for diffing validators * refactor!: remove key-assignment replacements (#1672) * initial commit * removed KeyAssignmentReplacementsKey * refactor: simplify key-assignment logic (#1684) * fixed typo: depreciated to deprecated --------- Co-authored-by: Marius Poke * add the epoch param in the docs --------- Co-authored-by: mpoke Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Simon Noetzlin Co-authored-by: Philip Offtermatt * test: Add epochs to MBT (#1676) * cleanup ./changelog entries * rebase * fix!: Validation of SlashAcks fails due to marshaling to Bech32 (backport #1570) (#1577) fix!: Validation of SlashAcks fails due to marshaling to Bech32 (#1570) * add different Bech32Prefix for consumer and provider * separate app encoding and params * remove ConsumerValPubKey from ValidatorConfig * update addresses in tests * make SlashAcks consistent across chains * add comments for clarity * Regenerate traces * Fix argument order * set bech32prefix for provider to cosmos * add changelog entries * add consumer-double-downtime e2e test * update nightly-e2e workflow * fix typo * add consumer-double-downtime to testConfigs * remove changes on provider * skip invalid SlashAcks * seal the config * clear the outstanding downtime flag for new vals * add info on upgrading to v4.0.0 * fix upgrade handler * fix changeover e2e test * Update tests/e2e/config.go Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> * Update tests/e2e/config.go Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> * add AccountPrefix to ChainConfig * fix docstrings * update AccountAddressPrefix in app.go * fix consumer-misb e2e test --------- Co-authored-by: Philip Offtermatt Co-authored-by: Simon Noetzlin Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> (cherry picked from commit 86046926502f7b0ba795bebcdd1fdc97ac776573) Co-authored-by: Marius Poke * docs: update changelog for v4.0.0 (#1578) update changelog * docs: prepare for v4.0.0 (#1581) * unclog build * update release notes * update release date * added proto declaration * temp commit * temp commit * more changes * first commit * add param and fix tests * reduce epoch size for e2e * clean up * mbt fix * fix diff bug * cleaning up * cleaning up * cleaning up * cleaning up * cleaning up * cleaning up * added more tests * more fixes * nit fixes * cleaning up * increase downtime by one block * fix logs * took into account Marius' comments * tiny fixes * Update x/ccv/provider/keeper/params.go Co-authored-by: Simon Noetzlin * use Bech32 addresses as keys for maps * refactor nextBlocks(epoch) to nextEpoch * Start adding epochs * Adjust tests for epochs * Use invariant script instead of handwriting Makefile * Fix key assignment valset invariant * Add better run_invariants script * Start adding epochs from trace into driver * Remove new block creation during consumer chain setup * Adjust model for epochs * Take into account comments * Revert changes to actions.go * Revert changes to x/ * Remove unused listMul * Advance time by epochLength instead of 1 second * Indent condition and clarify EndProviderEpoch --------- Co-authored-by: mpoke Co-authored-by: insumity Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Simon Noetzlin * added changelogs * rebase and fix compatibility test * Update docs/docs/adrs/adr-014-epochs.md Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> * Update docs/docs/adrs/adr-014-epochs.md Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> * nit change in test * removed blocks per epoch upper limit --------- Co-authored-by: mpoke Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Simon Noetzlin Co-authored-by: Philip Offtermatt Co-authored-by: Philip Offtermatt <57488781+p-offtermatt@users.noreply.github.com> --- .../provider/1516-introduce-epochs.md | 3 + .../provider/1516-introduce-epochs.md | 3 + Makefile | 2 + docs/docs/adrs/adr-001-key-assignment.md | 116 +--- docs/docs/adrs/adr-014-epochs.md | 77 +++ docs/docs/introduction/params.md | 12 + .../ccv/provider/v1/provider.proto | 14 + tests/e2e/actions.go | 14 +- tests/e2e/config.go | 131 ++++- tests/integration/common.go | 11 +- tests/integration/distribution.go | 6 +- tests/integration/expired_client.go | 18 +- tests/integration/key_assignment.go | 18 +- tests/integration/setup.go | 6 + tests/integration/slashing.go | 4 +- tests/integration/soft_opt_out.go | 6 +- tests/integration/unbonding.go | 11 +- tests/integration/valset_update.go | 4 +- tests/mbt/driver/core.go | 4 + tests/mbt/driver/mbt_test.go | 57 +- tests/mbt/driver/model_viewer.go | 4 + tests/mbt/driver/setup.go | 36 +- tests/mbt/model/ccv.qnt | 103 +++- tests/mbt/model/ccv_model.qnt | 316 ++++++++++- tests/mbt/model/ccv_test.qnt | 9 +- tests/mbt/model/ccv_utils.qnt | 463 +++++++++++++++ tests/mbt/run_invariants.sh | 38 +- testutil/keeper/unit_test_helpers.go | 1 - x/ccv/consumer/types/keys.go | 4 +- x/ccv/provider/keeper/grpc_query_test.go | 1 - x/ccv/provider/keeper/key_assignment.go | 244 +------- x/ccv/provider/keeper/key_assignment_test.go | 105 +--- x/ccv/provider/keeper/params.go | 8 + x/ccv/provider/keeper/params_test.go | 1 + x/ccv/provider/keeper/proposal.go | 12 +- x/ccv/provider/keeper/proposal_test.go | 1 + x/ccv/provider/keeper/relay.go | 29 +- x/ccv/provider/keeper/relay_test.go | 60 +- x/ccv/provider/keeper/validator_set_update.go | 178 ++++++ .../keeper/validator_set_update_test.go | 355 ++++++++++++ x/ccv/provider/types/genesis_test.go | 26 +- x/ccv/provider/types/keys.go | 19 +- x/ccv/provider/types/keys_test.go | 2 +- x/ccv/provider/types/params.go | 12 + x/ccv/provider/types/params_test.go | 24 +- x/ccv/provider/types/provider.pb.go | 526 ++++++++++++++---- 46 files changed, 2420 insertions(+), 674 deletions(-) create mode 100644 .changelog/unreleased/features/provider/1516-introduce-epochs.md create mode 100644 .changelog/unreleased/state-breaking/provider/1516-introduce-epochs.md create mode 100644 docs/docs/adrs/adr-014-epochs.md create mode 100644 tests/mbt/model/ccv_utils.qnt create mode 100644 x/ccv/provider/keeper/validator_set_update.go create mode 100644 x/ccv/provider/keeper/validator_set_update_test.go diff --git a/.changelog/unreleased/features/provider/1516-introduce-epochs.md b/.changelog/unreleased/features/provider/1516-introduce-epochs.md new file mode 100644 index 0000000000..1ebce4000b --- /dev/null +++ b/.changelog/unreleased/features/provider/1516-introduce-epochs.md @@ -0,0 +1,3 @@ +- Introduce epochs (i.e., send a VSCPacket every X blocks instead of in every + block) so that we reduce the cost of relaying IBC packets needed for ICS. + ([\#1516](https://github.com/cosmos/interchain-security/pull/1516)) \ No newline at end of file diff --git a/.changelog/unreleased/state-breaking/provider/1516-introduce-epochs.md b/.changelog/unreleased/state-breaking/provider/1516-introduce-epochs.md new file mode 100644 index 0000000000..1ebce4000b --- /dev/null +++ b/.changelog/unreleased/state-breaking/provider/1516-introduce-epochs.md @@ -0,0 +1,3 @@ +- Introduce epochs (i.e., send a VSCPacket every X blocks instead of in every + block) so that we reduce the cost of relaying IBC packets needed for ICS. + ([\#1516](https://github.com/cosmos/interchain-security/pull/1516)) \ No newline at end of file diff --git a/Makefile b/Makefile index 350c66f8af..1b0ddec7f9 100644 --- a/Makefile +++ b/Makefile @@ -132,6 +132,8 @@ verify-models: quint test tests/mbt/model/ccv_test.qnt;\ quint test tests/mbt/model/ccv_model.qnt;\ quint run --invariant "all{ValidatorUpdatesArePropagatedInv,ValidatorSetHasExistedInv,SameVscPacketsInv,MatureOnTimeInv,EventuallyMatureOnProviderInv}" tests/mbt/model/ccv_model.qnt --max-steps 200 --max-samples 200 + cd tests/mbt/model;\ + ../run_invariants.sh diff --git a/docs/docs/adrs/adr-001-key-assignment.md b/docs/docs/adrs/adr-001-key-assignment.md index 874321db0c..36dbdfdb09 100644 --- a/docs/docs/adrs/adr-001-key-assignment.md +++ b/docs/docs/adrs/adr-001-key-assignment.md @@ -7,6 +7,7 @@ title: Key Assignment ## Changelog * 2022-12-01: Initial Draft +* 2024-03-01: Updated to take into account they key-assigment-replacement deprecation. ## Status @@ -30,10 +31,6 @@ ConsumerValidatorsBytePrefix | len(chainID) | chainID | providerConsAddress -> c ```golang ValidatorsByConsumerAddrBytePrefix | len(chainID) | chainID | consumerConsAddress -> providerConsAddress ``` -- `KeyAssignmentReplacements` - Stores the key assignments that need to be replaced in the current block. Needed to apply the key assignments received in a block to the validator updates sent to the consumer chains. -```golang -KeyAssignmentReplacementsBytePrefix | len(chainID) | chainID | providerConsAddress -> abci.ValidatorUpdate{PubKey: oldConsumerKey, Power: currentPower}, -``` - `ConsumerAddrsToPrune` - Stores the mapping from VSC ids to consumer validators addresses. Needed for pruning `ValidatorByConsumerAddr`. ```golang ConsumerAddrsToPruneBytePrefix | len(chainID) | chainID | vscID -> []consumerConsAddresses @@ -67,20 +64,6 @@ if _, consumerRegistered := GetConsumerClientId(chainID); consumerRegistered { oldConsumerAddr := utils.TMCryptoPublicKeyToConsAddr(oldConsumerKey) vscID := GetValidatorSetUpdateId() AppendConsumerAddrsToPrune(chainID, vscID, oldConsumerAddr) - } else { - // the validator had no key assigned on this consumer chain - oldConsumerKey := validator.TmConsPublicKey() - } - - // check whether the validator is valid, i.e., its power is positive - if currentPower := stakingKeeper.GetLastValidatorPower(providerAddr); currentPower > 0 { - // to enable multiple calls of AssignConsumerKey in the same block by the same validator - // the key assignment replacement should not be overwritten - if _, found := GetKeyAssignmentReplacement(chainID, providerConsAddr); !found { - // store old key and power for modifying the valset update in EndBlock - oldKeyAssignment := abci.ValidatorUpdate{PubKey: oldConsumerKey, Power: currentPower} - SetKeyAssignmentReplacement(chainID, providerConsAddr, oldKeyAssignment) - } } } else { // if the consumer chain is not registered, then remove the previous reverse mapping @@ -129,89 +112,24 @@ func (k Keeper) MakeConsumerGenesis(chainID string) (gen consumertypes.GenesisSt } ``` -On `EndBlock` while queueing `VSCPacket`s to send to registered consumer chains: +Note that key assignment works hand-in-hand with [epochs](https://github.com/cosmos/interchain-security/blob/main/docs/docs/adrs/adr-014-epochs.md). +For each consumer chain, we store the consumer validator set that is currently (i.e., in this epoch) validating the consumer chain. +Specifically, for each validator in the set we store among others, the public key that it is using on the consumer chain during the current (i.e., ongoing) epoch. +At the end of every epoch, if there were validator set changes on the provider, then for every consumer chain, we construct a `VSCPacket` +with all the validator updates and add it to the list of `PendingVSCPacket`s. We compute the validator updates needed by a consumer chain by +comparing the stored list of consumer validators with the current bonded validators on the provider, with something similar to this: ```golang -func QueueVSCPackets() { - valUpdateID := GetValidatorSetUpdateId() - // get the validator updates from the staking module - valUpdates := stakingKeeper.GetValidatorUpdates() - - IterateConsumerChains(func(chainID, clientID string) (stop bool) { - // apply the key assignment to the validator updates - valUpdates := ApplyKeyAssignmentToValUpdates(chainID, valUpdates) - // .. - }) - // ... -} - -func ApplyKeyAssignmentToValUpdates( - chainID string, - valUpdates []abci.ValidatorUpdate, -) (newUpdates []abci.ValidatorUpdate) { - for _, valUpdate := range valUpdates { - providerAddr := utils.TMCryptoPublicKeyToConsAddr(valUpdate.PubKey) - - // if a key assignment replacement is found, then - // remove the valupdate with the old consumer key - // and create two new valupdates - prevConsumerKey, _, found := GetKeyAssignmentReplacement(chainID, providerAddr) - if found { - // set the old consumer key's power to 0 - newUpdates = append(newUpdates, abci.ValidatorUpdate{ - PubKey: prevConsumerKey, - Power: 0, - }) - // set the new consumer key's power to the power in the update - newConsumerKey := GetValidatorConsumerPubKey(chainID, providerAddr) - newUpdates = append(newUpdates, abci.ValidatorUpdate{ - PubKey: newConsumerKey, - Power: valUpdate.Power, - }) - // delete key assignment replacement - DeleteKeyAssignmentReplacement(chainID, providerAddr) - } else { - // there is no key assignment replacement; - // check if the validator's key is assigned - consumerKey, found := k.GetValidatorConsumerPubKey(ctx, chainID, providerAddr) - if found { - // replace the update containing the provider key - // with an update containing the consumer key - newUpdates = append(newUpdates, abci.ValidatorUpdate{ - PubKey: consumerKey, - Power: valUpdate.Power, - }) - } else { - // keep the same update - newUpdates = append(newUpdates, valUpdate) - } - } - } - - // iterate over the remaining key assignment replacements - IterateKeyAssignmentReplacements(chainID, func( - pAddr sdk.ConsAddress, - prevCKey tmprotocrypto.PublicKey, - power int64, - ) (stop bool) { - // set the old consumer key's power to 0 - newUpdates = append(newUpdates, abci.ValidatorUpdate{ - PubKey: prevCKey, - Power: 0, - }) - // set the new consumer key's power to the power in key assignment replacement - newConsumerKey := GetValidatorConsumerPubKey(chainID, pAddr) - newUpdates = append(newUpdates, abci.ValidatorUpdate{ - PubKey: newConsumerKey, - Power: power, - }) - return false - }) - - // remove all the key assignment replacements - - return newUpdates -} +// get the valset that has been validating the consumer chain during this epoch +currentValidators := GetConsumerValSet(consumerChain) +// generate the validator updates needed to be sent through a `VSCPacket` by comparing the current validators +// in the epoch with the latest bonded validators +valUpdates := DiffValidators(currentValidators, stakingmodule.GetBondedValidators()) +// update the current validators set for the upcoming epoch to be the latest bonded validators instead +SetConsumerValSet(stakingmodule.GetBondedValidators()) ``` +where `DiffValidators` internally checks if the consumer public key for a validator has changed since the last +epoch and if so generates a validator update. This way, a validator can change its consumer public key for a consumer +chain an arbitrary amount of times and only the last set consumer public key would be taken into account. On receiving a `SlashPacket` from a consumer chain with id `chainID` for a infraction of a validator `data.Validator`: ```golang diff --git a/docs/docs/adrs/adr-014-epochs.md b/docs/docs/adrs/adr-014-epochs.md new file mode 100644 index 0000000000..fc669e9b36 --- /dev/null +++ b/docs/docs/adrs/adr-014-epochs.md @@ -0,0 +1,77 @@ +--- +sidebar_position: 15 +title: Epochs +--- +# ADR 014: Epochs + +## Changelog +* 2024-01-05: Proposed, first draft of ADR. +* 2024-02-29: Updated so that it describes the implementation where we store the whole consumer validator set. + +## Status + +Proposed + +## Context + +In every block that the provider valset changes, a `VSCPacket` must be sent to every consumer and a corresponding `VSCMaturedPacket` sent back. +Given that the validator powers may change very often on the provider chain (e.g., the Cosmos Hub), this approach results in a large workload for the relayers. +Although the validator powers may change very often, these changes are usually small and have an insignificant impact on the chain's security. +In other words, the valset on the consumers can be slightly outdated without affecting security. +As a matter of fact, this already happens due to relaying delays. + +As a solution, this ADR introduces the concept of _epochs_. +An epoch consists of multiple blocks. +The provider sends `VSCPacket`s once per epoch. +A `VSCPacket` contains all the validator updates that are needed by a consumer chain. + +## Decision + +The implementation of epochs requires the following changes: + +- For each consumer chain, we store the consumer validator set that is currently (i.e., in this epoch) validating the + consumer chain. For each validator in the set we store i) its voting power, and ii) the public key that it is + using on the consumer chain during the current (i.e., ongoing) epoch. + The initial consumer validator set for a chain is set during the creation of the consumer genesis. +- We introduce the `BlocksPerEpoch` param that sets the number of blocks in an epoch. By default, `BlocksPerEpoch` is + set to be 600 which corresponds to 1 hour, assuming 6 seconds per block. This param can be changed through + a _governance proposal_. In the provider `EndBlock` we check `BlockHeight() % BlocksPerEpoch() == 0` + to decide when an epoch has ended. +- At the end of every epoch, if there were validator set changes on the provider, then for every consumer chain, we + construct a `VSCPacket` with all the validator updates and add it to the list of `PendingVSCPackets`. We compute the + validator updates needed by a consumer chain by comparing the stored list of consumer validators with the current + bonded validators on the provider, with something similar to this: +```go +// get the valset that has been validating the consumer chain during this epoch +currentValidators := GetConsumerValSet(consumerChain) +// generate the validator updates needed to be sent through a `VSCPacket` by comparing the current validators +// in the epoch with the latest bonded validators +valUpdates := DiffValidators(currentValidators, stakingmodule.GetBondedValidators()) +// update the current validators set for the upcoming epoch to be the latest bonded validators instead +SetConsumerValSet(stakingmodule.GetBondedValidators()) +``` +Note that a validator can change its consumer public key for a specific consumer chain an arbitrary amount of times during +a block and during an epoch. Then, when we generate the validator updates in `DiffValidators`, we have to check whether +the current consumer public key (retrieved by calling `GetValidatorConsumerPubKey`) is different from the consumer public +key the validator was using in the current epoch. + +## Consequences + +### Positive + +- Reduce the cost of relaying. +- Reduce the amount of IBC packets needed for ICS. +- Simplifies [key-assignment code](https://github.com/cosmos/interchain-security/blob/main/docs/docs/adrs/adr-001-key-assignment.md) because + we only need to check if the `consumer_public_key` has been modified since the last epoch to generate an update. + +### Negative + +- Increase the delay in the propagation of validator set changes (but for reasonable epoch lengths on the order of ~hours or less, this is unlikely to be significant). + +### Neutral + +N/A + +## References + +* [EPIC](https://github.com/cosmos/interchain-security/issues/1087) diff --git a/docs/docs/introduction/params.md b/docs/docs/introduction/params.md index 5a9e8462c7..8d917b3070 100644 --- a/docs/docs/introduction/params.md +++ b/docs/docs/introduction/params.md @@ -149,3 +149,15 @@ This param would allow provider binaries to panic deterministically in the event `RetryDelayPeriod` exists on the consumer for **ICS versions >= v3.2.0** (introduced by the implementation of [ADR-008](../adrs/adr-008-throttle-retries.md)) and is the period at which the consumer retries to send a `SlashPacket` that was rejected by the provider. + +## Epoch Parameters + +### BlocksPerEpoch +`BlocksPerEpoch` exists on the provider for **ICS versions >= 3.3.0** (introduced by the implementation of [ADR-014](../adrs/adr-014-epochs.md)) +and corresponds to the number of blocks that constitute an epoch. This param is set to 600 by default. Assuming we need 6 seconds to +commit a block, the duration of an epoch corresponds to 1 hour. This means that a `VSCPacket` would be sent to a consumer +chain once at the end of every epoch, so once every 600 blocks. This parameter can be adjusted via a governance proposal, +however careful consideration is needed so that `BlocksPerEpoch` is not too large. A large `BlocksPerEpoch` could lead to a delay +of `VSCPacket`s and hence potentially lead to [unbonding pausing](https://informal.systems/blog/learning-to-live-with-unbonding-pausing). +For setting `BlocksPerEpoch`, we also need to consider potential slow chain upgrades that could delay the sending of a +`VSCPacket`, as well as potential increases in the time it takes to commit a block (e.g., from 6 seconds to 30 seconds). \ No newline at end of file diff --git a/proto/interchain_security/ccv/provider/v1/provider.proto b/proto/interchain_security/ccv/provider/v1/provider.proto index f9bdf0a53f..4da89022e8 100644 --- a/proto/interchain_security/ccv/provider/v1/provider.proto +++ b/proto/interchain_security/ccv/provider/v1/provider.proto @@ -188,6 +188,9 @@ message Params { // The fee required to be paid to add a reward denom cosmos.base.v1beta1.Coin consumer_reward_denom_registration_fee = 9 [ (gogoproto.nullable) = false ]; + + // The number of blocks that comprise an epoch. + int64 blocks_per_epoch = 10; } // SlashAcks contains cons addresses of consumer chain validators @@ -295,3 +298,14 @@ message ConsumerAddrsToPrune { uint64 vsc_id = 2; AddressList consumer_addrs = 3; } + +// ConsumerValidator is used to facilitate epoch-based transitions. It contains relevant info for +// a validator that is expected to validate on a consumer chain during an epoch. +message ConsumerValidator { + // validator's consensus address on the provider chain + bytes provider_cons_addr = 1; + // voting power the validator has during this epoch + int64 power = 2; + // public key the validator uses on the consumer chain during this epoch + tendermint.crypto.PublicKey consumer_public_key = 3; +} \ No newline at end of file diff --git a/tests/e2e/actions.go b/tests/e2e/actions.go index 84c3020b91..32ab3f714a 100644 --- a/tests/e2e/actions.go +++ b/tests/e2e/actions.go @@ -1256,6 +1256,12 @@ func (tr TestConfig) relayPacketsGorelayer( action RelayPacketsAction, verbose bool, ) { + // Because `.app_state.provider.params.blocks_per_epoch` is set to 3 in the E2E tests, we wait 3 blocks + // before relaying the packets to guarantee that at least one epoch passes and hence any `VSCPacket`s get + // queued and are subsequently relayed. + tr.waitBlocks(action.ChainA, 3, 90*time.Second) + tr.waitBlocks(action.ChainB, 3, 90*time.Second) + pathName := tr.GetPathNameForGorelayer(action.ChainA, action.ChainB) // rly transact relay-packets [path-name] --channel [channel-id] @@ -1280,6 +1286,12 @@ func (tr TestConfig) relayPacketsHermes( action RelayPacketsAction, verbose bool, ) { + // Because `.app_state.provider.params.blocks_per_epoch` is set to 3 in the E2E tests, we wait 3 blocks + // before relaying the packets to guarantee that at least one epoch passes and hence any `VSCPacket`s get + // queued and are subsequently relayed. + tr.waitBlocks(action.ChainA, 3, 90*time.Second) + tr.waitBlocks(action.ChainB, 3, 90*time.Second) + // hermes clear packets ibc0 transfer channel-13 //#nosec G204 -- Bypass linter warning for spawning subprocess with cmd arguments. cmd := exec.Command("docker", "exec", tr.containerConfig.InstanceName, "hermes", "clear", "packets", @@ -1591,7 +1603,7 @@ func (tr TestConfig) invokeDowntimeSlash(action DowntimeSlashAction, verbose boo // Bring validator down tr.setValidatorDowntime(action.Chain, action.Validator, true, verbose) // Wait appropriate amount of blocks for validator to be slashed - tr.waitBlocks(action.Chain, 10, 3*time.Minute) + tr.waitBlocks(action.Chain, 11, 3*time.Minute) // Bring validator back up tr.setValidatorDowntime(action.Chain, action.Validator, false, verbose) } diff --git a/tests/e2e/config.go b/tests/e2e/config.go index c2079af0c3..677a68e748 100644 --- a/tests/e2e/config.go +++ b/tests/e2e/config.go @@ -4,6 +4,8 @@ import ( "fmt" "strconv" "time" + + "golang.org/x/mod/semver" ) var ( @@ -236,7 +238,8 @@ func SlashThrottleTestConfig() TestConfig { ".app_state.slashing.params.downtime_jail_duration = \"60s\" | " + ".app_state.slashing.params.slash_fraction_downtime = \"0.010000000000000000\" | " + ".app_state.provider.params.slash_meter_replenish_fraction = \"0.10\" | " + - ".app_state.provider.params.slash_meter_replenish_period = \"20s\"", + ".app_state.provider.params.slash_meter_replenish_period = \"20s\" | " + + ".app_state.provider.params.blocks_per_epoch = 3", }, ChainID("consu"): { ChainId: ChainID("consu"), @@ -245,7 +248,7 @@ func SlashThrottleTestConfig() TestConfig { IpPrefix: "7.7.8", VotingWaitTime: 20, GenesisChanges: ".app_state.gov.params.voting_period = \"20s\" | " + - ".app_state.slashing.params.signed_blocks_window = \"15\" | " + + ".app_state.slashing.params.signed_blocks_window = \"20\" | " + ".app_state.slashing.params.min_signed_per_window = \"0.500000000000000000\" | " + ".app_state.slashing.params.downtime_jail_duration = \"60s\" | " + ".app_state.slashing.params.slash_fraction_downtime = \"0.010000000000000000\" | " + @@ -259,6 +262,105 @@ func SlashThrottleTestConfig() TestConfig { return tr } +// CompatibilityTestConfig returns a test configuration for a given version of a consumer and provider +func CompatibilityTestConfig(providerVersion, consumerVersion string) TestConfig { + // Base configuration is the default + testCfg := DefaultTestConfig() + + // get version dependent validator configs + testCfg.validatorConfigs = getValidatorConfigFromVersion(providerVersion, consumerVersion) + + var providerConfig, consumerConfig ChainConfig + if !semver.IsValid(consumerVersion) { + fmt.Println("Using default provider chain config") + consumerConfig = testCfg.chainConfigs[ChainID("consu")] + } else if semver.Compare(consumerVersion, "v3.0.0") < 0 { + fmt.Println("Using consumer chain config for v2.0.0") + consumerConfig = ChainConfig{ + ChainId: ChainID("consu"), + AccountPrefix: "cosmos", + BinaryName: "interchain-security-cd", + IpPrefix: "7.7.8", + VotingWaitTime: 20, + GenesisChanges: ".app_state.gov.voting_params.voting_period = \"20s\" | " + + ".app_state.slashing.params.signed_blocks_window = \"15\" | " + + ".app_state.slashing.params.min_signed_per_window = \"0.500000000000000000\" | " + + ".app_state.slashing.params.downtime_jail_duration = \"2s\" | " + + ".app_state.slashing.params.slash_fraction_downtime = \"0.010000000000000000\"", + } + } else if semver.Compare(consumerVersion, "v4.0.0") < 0 { + fmt.Println("Using consumer chain config for v3.x.x") + consumerConfig = ChainConfig{ + ChainId: ChainID("consu"), + AccountPrefix: "cosmos", + BinaryName: "interchain-security-cd", + IpPrefix: "7.7.8", + VotingWaitTime: 20, + GenesisChanges: ".app_state.gov.params.voting_period = \"20s\" | " + + ".app_state.slashing.params.signed_blocks_window = \"15\" | " + + ".app_state.slashing.params.min_signed_per_window = \"0.500000000000000000\" | " + + ".app_state.slashing.params.downtime_jail_duration = \"60s\" | " + + ".app_state.slashing.params.slash_fraction_downtime = \"0.010000000000000000\"", + } + } else { + fmt.Println("Using default consumer chain config") + consumerConfig = testCfg.chainConfigs[ChainID("consu")] + } + + // Get the provider chain config for a specific version + if !semver.IsValid(providerVersion) { + fmt.Println("Using default provider chain config") + providerConfig = testCfg.chainConfigs[ChainID("provi")] + } else if semver.Compare(providerVersion, "v3.0.0") < 0 { + fmt.Println("Using provider chain config for v2.x.x") + providerConfig = ChainConfig{ + ChainId: ChainID("provi"), + AccountPrefix: "cosmos", + BinaryName: "interchain-security-pd", + IpPrefix: "7.7.7", + VotingWaitTime: 20, + GenesisChanges: ".app_state.gov.voting_params.voting_period = \"20s\" | " + + // Custom slashing parameters for testing validator downtime functionality + // See https://docs.cosmos.network/main/modules/slashing/04_begin_block.html#uptime-tracking + ".app_state.slashing.params.signed_blocks_window = \"10\" | " + + ".app_state.slashing.params.min_signed_per_window = \"0.500000000000000000\" | " + + ".app_state.slashing.params.downtime_jail_duration = \"2s\" | " + + ".app_state.slashing.params.slash_fraction_downtime = \"0.010000000000000000\" | " + + ".app_state.provider.params.slash_meter_replenish_fraction = \"1.0\" | " + // This disables slash packet throttling + ".app_state.provider.params.slash_meter_replenish_period = \"3s\"", + } + } else if semver.Compare(providerVersion, "v4.0.0") <= 0 { + fmt.Println("Using provider chain config for v3.x.x") + providerConfig = ChainConfig{ + ChainId: ChainID("provi"), + AccountPrefix: "cosmos", + BinaryName: "interchain-security-pd", + IpPrefix: "7.7.7", + VotingWaitTime: 20, + GenesisChanges: ".app_state.gov.params.voting_period = \"20s\" | " + + // Custom slashing parameters for testing validator downtime functionality + // See https://docs.cosmos.network/main/modules/slashing/04_begin_block.html#uptime-tracking + ".app_state.slashing.params.signed_blocks_window = \"10\" | " + + ".app_state.slashing.params.min_signed_per_window = \"0.500000000000000000\" | " + + ".app_state.slashing.params.downtime_jail_duration = \"60s\" | " + + ".app_state.slashing.params.slash_fraction_downtime = \"0.010000000000000000\" | " + + ".app_state.provider.params.slash_meter_replenish_fraction = \"1.0\" | " + // This disables slash packet throttling + ".app_state.provider.params.slash_meter_replenish_period = \"3s\"", + } + } else { + fmt.Println("Using default provider chain config") + providerConfig = testCfg.chainConfigs[ChainID("provi")] + } + + testCfg.chainConfigs[ChainID("consu")] = consumerConfig + testCfg.chainConfigs[ChainID("provi")] = providerConfig + testCfg.name = string(CompatibilityTestCfg) + testCfg.containerConfig.InstanceName = fmt.Sprintf("%s_%s-%s", + testCfg.containerConfig.InstanceName, + consumerVersion, providerVersion) + return testCfg +} + func DefaultTestConfig() TestConfig { tr := TestConfig{ name: "default", @@ -284,7 +386,8 @@ func DefaultTestConfig() TestConfig { ".app_state.slashing.params.downtime_jail_duration = \"60s\" | " + ".app_state.slashing.params.slash_fraction_downtime = \"0.010000000000000000\" | " + ".app_state.provider.params.slash_meter_replenish_fraction = \"1.0\" | " + // This disables slash packet throttling - ".app_state.provider.params.slash_meter_replenish_period = \"3s\"", + ".app_state.provider.params.slash_meter_replenish_period = \"3s\" | " + + ".app_state.provider.params.blocks_per_epoch = 3", }, ChainID("consu"): { ChainId: ChainID("consu"), @@ -293,7 +396,7 @@ func DefaultTestConfig() TestConfig { IpPrefix: "7.7.8", VotingWaitTime: 20, GenesisChanges: ".app_state.gov.params.voting_period = \"20s\" | " + - ".app_state.slashing.params.signed_blocks_window = \"15\" | " + + ".app_state.slashing.params.signed_blocks_window = \"20\" | " + ".app_state.slashing.params.min_signed_per_window = \"0.500000000000000000\" | " + ".app_state.slashing.params.downtime_jail_duration = \"60s\" | " + ".app_state.slashing.params.slash_fraction_downtime = \"0.010000000000000000\"", @@ -343,7 +446,8 @@ func DemocracyTestConfig(allowReward bool) TestConfig { ".app_state.slashing.params.min_signed_per_window = \"0.500000000000000000\" | " + ".app_state.slashing.params.downtime_jail_duration = \"60s\" | " + ".app_state.slashing.params.slash_fraction_downtime = \"0.010000000000000000\" | " + - ".app_state.provider.params.slash_meter_replenish_fraction = \"1.0\"", // This disables slash packet throttling + ".app_state.provider.params.slash_meter_replenish_fraction = \"1.0\" | " + // This disables slash packet throttling + ".app_state.provider.params.blocks_per_epoch = 3", }, ChainID("democ"): { ChainId: ChainID("democ"), @@ -385,7 +489,8 @@ func MultiConsumerTestConfig() TestConfig { ".app_state.slashing.params.min_signed_per_window = \"0.500000000000000000\" | " + ".app_state.slashing.params.downtime_jail_duration = \"60s\" | " + ".app_state.slashing.params.slash_fraction_downtime = \"0.010000000000000000\" | " + - ".app_state.provider.params.slash_meter_replenish_fraction = \"1.0\"", // This disables slash packet throttling + ".app_state.provider.params.slash_meter_replenish_fraction = \"1.0\" | " + // This disables slash packet throttling + ".app_state.provider.params.blocks_per_epoch = 3", }, ChainID("consu"): { ChainId: ChainID("consu"), @@ -394,7 +499,7 @@ func MultiConsumerTestConfig() TestConfig { IpPrefix: "7.7.8", VotingWaitTime: 20, GenesisChanges: ".app_state.gov.params.voting_period = \"20s\" | " + - ".app_state.slashing.params.signed_blocks_window = \"10\" | " + + ".app_state.slashing.params.signed_blocks_window = \"20\" | " + ".app_state.slashing.params.min_signed_per_window = \"0.500000000000000000\" | " + ".app_state.slashing.params.downtime_jail_duration = \"60s\" | " + ".app_state.slashing.params.slash_fraction_downtime = \"0.010000000000000000\"", @@ -406,7 +511,7 @@ func MultiConsumerTestConfig() TestConfig { IpPrefix: "7.7.9", VotingWaitTime: 20, GenesisChanges: ".app_state.gov.params.voting_period = \"20s\" | " + - ".app_state.slashing.params.signed_blocks_window = \"10\" | " + + ".app_state.slashing.params.signed_blocks_window = \"20\" | " + ".app_state.slashing.params.min_signed_per_window = \"0.500000000000000000\" | " + ".app_state.slashing.params.downtime_jail_duration = \"60s\" | " + ".app_state.slashing.params.slash_fraction_downtime = \"0.010000000000000000\"", @@ -444,7 +549,8 @@ func ChangeoverTestConfig() TestConfig { ".app_state.slashing.params.downtime_jail_duration = \"60s\" | " + ".app_state.slashing.params.slash_fraction_downtime = \"0.010000000000000000\" | " + ".app_state.provider.params.slash_meter_replenish_fraction = \"1.0\" | " + // This disables slash packet throttling - ".app_state.provider.params.slash_meter_replenish_period = \"3s\"", + ".app_state.provider.params.slash_meter_replenish_period = \"3s\" | " + + ".app_state.provider.params.blocks_per_epoch = 3", }, ChainID("sover"): { ChainId: ChainID("sover"), @@ -454,7 +560,7 @@ func ChangeoverTestConfig() TestConfig { IpPrefix: "7.7.8", VotingWaitTime: 20, GenesisChanges: ".app_state.gov.params.voting_period = \"20s\" | " + - ".app_state.slashing.params.signed_blocks_window = \"15\" | " + + ".app_state.slashing.params.signed_blocks_window = \"20\" | " + ".app_state.slashing.params.min_signed_per_window = \"0.500000000000000000\" | " + ".app_state.slashing.params.downtime_jail_duration = \"60s\" | " + ".app_state.slashing.params.slash_fraction_downtime = \"0.010000000000000000\" | " + @@ -544,7 +650,8 @@ func ConsumerMisbehaviourTestConfig() TestConfig { ".app_state.slashing.params.downtime_jail_duration = \"60s\" | " + ".app_state.slashing.params.slash_fraction_downtime = \"0.010000000000000000\" | " + ".app_state.provider.params.slash_meter_replenish_fraction = \"1.0\" | " + // This disables slash packet throttling - ".app_state.provider.params.slash_meter_replenish_period = \"3s\"", + ".app_state.provider.params.slash_meter_replenish_period = \"3s\" | " + + ".app_state.provider.params.blocks_per_epoch = 3", }, ChainID("consu"): { ChainId: ChainID("consu"), @@ -553,7 +660,7 @@ func ConsumerMisbehaviourTestConfig() TestConfig { IpPrefix: "7.7.8", VotingWaitTime: 20, GenesisChanges: ".app_state.gov.params.voting_period = \"20s\" | " + - ".app_state.slashing.params.signed_blocks_window = \"15\" | " + + ".app_state.slashing.params.signed_blocks_window = \"20\" | " + ".app_state.slashing.params.min_signed_per_window = \"0.500000000000000000\" | " + ".app_state.slashing.params.downtime_jail_duration = \"60s\" | " + ".app_state.slashing.params.slash_fraction_downtime = \"0.010000000000000000\"", diff --git a/tests/integration/common.go b/tests/integration/common.go index a4ff9e254a..18b657ae56 100644 --- a/tests/integration/common.go +++ b/tests/integration/common.go @@ -127,7 +127,7 @@ func delegateAndRedelegate(s *CCVTestSuite, delAddr sdk.AccAddress, srcValTokensAfter := s.getVal(s.providerCtx(), srcValAddr).GetBondedTokens() s.Require().Equal(srcValTokensAfter.Sub(srcValTokensBefore), amount) - s.providerChain.NextBlock() + s.nextEpoch() dstValTokensBefore := s.getVal(s.providerCtx(), dstValAddr).GetBondedTokens() @@ -625,3 +625,12 @@ func (s *CCVTestSuite) mustGetStakingValFromTmVal(tmVal tmtypes.Validator) (stak s.Require().True(found) return stakingVal } + +// nextEpoch moves `chain` forward by an epoch +func (s *CCVTestSuite) nextEpoch() { + blocksPerEpoch := s.providerApp.GetProviderKeeper().GetParams(s.providerCtx()).BlocksPerEpoch + + for i := int64(0); i < blocksPerEpoch; i++ { + s.providerChain.NextBlock() + } +} diff --git a/tests/integration/distribution.go b/tests/integration/distribution.go index 25cbcb3132..1ea5bfcd00 100644 --- a/tests/integration/distribution.go +++ b/tests/integration/distribution.go @@ -23,7 +23,7 @@ func (s *CCVTestSuite) TestRewardsDistribution() { bondAmt := sdk.NewInt(10000000) delAddr := s.providerChain.SenderAccount.GetAddress() delegate(s, delAddr, bondAmt) - s.providerChain.NextBlock() + s.nextEpoch() // register a consumer reward denom params := s.consumerApp.GetConsumerKeeper().GetConsumerParams(s.consumerCtx()) @@ -124,7 +124,7 @@ func (s *CCVTestSuite) TestSendRewardsRetries() { bondAmt := sdk.NewInt(10000000) delAddr := s.providerChain.SenderAccount.GetAddress() delegate(s, delAddr, bondAmt) - s.providerChain.NextBlock() + s.nextEpoch() // Register denom on consumer chain params := s.consumerApp.GetConsumerKeeper().GetConsumerParams(s.consumerCtx()) @@ -253,7 +253,7 @@ func (s *CCVTestSuite) TestEndBlockRD() { bondAmt := sdk.NewInt(10000000) delAddr := s.providerChain.SenderAccount.GetAddress() delegate(s, delAddr, bondAmt) - s.providerChain.NextBlock() + s.nextEpoch() if tc.denomRegistered { params := s.consumerApp.GetConsumerKeeper().GetConsumerParams(s.consumerCtx()) diff --git a/tests/integration/expired_client.go b/tests/integration/expired_client.go index 1981e85828..a46df32d8e 100644 --- a/tests/integration/expired_client.go +++ b/tests/integration/expired_client.go @@ -33,7 +33,7 @@ func (s *CCVTestSuite) TestVSCPacketSendExpiredClient() { delegate(s, delAddr, bondAmt) // try to send CCV packet to consumer - s.providerChain.NextBlock() + s.nextEpoch() // check that the packet was added to the list of pending VSC packets packets := providerKeeper.GetPendingVSCPackets(s.providerCtx(), s.consumerChain.ChainID) @@ -41,7 +41,7 @@ func (s *CCVTestSuite) TestVSCPacketSendExpiredClient() { s.Require().Equal(1, len(packets), "unexpected number of pending VSC packets") // try again to send CCV packet to consumer - s.providerChain.NextBlock() + s.nextEpoch() // check that the packet is still in the list of pending VSC packets packets = providerKeeper.GetPendingVSCPackets(s.providerCtx(), s.consumerChain.ChainID) @@ -52,7 +52,7 @@ func (s *CCVTestSuite) TestVSCPacketSendExpiredClient() { delegate(s, delAddr, bondAmt) // try again to send CCV packets to consumer - s.providerChain.NextBlock() + s.nextEpoch() // check that the packets are still in the list of pending VSC packets packets = providerKeeper.GetPendingVSCPackets(s.providerCtx(), s.consumerChain.ChainID) @@ -62,8 +62,8 @@ func (s *CCVTestSuite) TestVSCPacketSendExpiredClient() { // upgrade expired client to the consumer upgradeExpiredClient(s, Consumer) - // go to next block - s.providerChain.NextBlock() + // go to next epoch + s.nextEpoch() // check that the packets are not in the list of pending VSC packets packets = providerKeeper.GetPendingVSCPackets(s.providerCtx(), s.consumerChain.ChainID) @@ -73,7 +73,7 @@ func (s *CCVTestSuite) TestVSCPacketSendExpiredClient() { // - bond more tokens on provider to change validator powers delegate(s, delAddr, bondAmt) // - send CCV packet to consumer - s.providerChain.NextBlock() + s.nextEpoch() // - relay all VSC packet from provider to consumer relayAllCommittedPackets(s, s.providerChain, s.path, ccv.ProviderPortID, s.path.EndpointB.ChannelID, 3) // - increment time so that the unbonding period ends on the consumer @@ -96,13 +96,13 @@ func (s *CCVTestSuite) TestConsumerPacketSendExpiredClient() { delegate(s, delAddr, bondAmt) // send CCV packet to consumer - s.providerChain.NextBlock() + s.nextEpoch() // bond more tokens on provider to change validator powers delegate(s, delAddr, bondAmt) // send CCV packets to consumer - s.providerChain.NextBlock() + s.nextEpoch() // check that the packets are not in the list of pending VSC packets providerPackets := providerKeeper.GetPendingVSCPackets(s.providerCtx(), s.consumerChain.ChainID) @@ -172,7 +172,7 @@ func (s *CCVTestSuite) TestConsumerPacketSendExpiredClient() { // - bond more tokens on provider to change validator powers delegate(s, delAddr, bondAmt) // - send CCV packet to consumer - s.providerChain.NextBlock() + s.nextEpoch() // - relay 1 VSC packet from provider to consumer relayAllCommittedPackets(s, s.providerChain, s.path, ccv.ProviderPortID, s.path.EndpointB.ChannelID, 1) // - increment time so that the unbonding period ends on the provider diff --git a/tests/integration/key_assignment.go b/tests/integration/key_assignment.go index 20e746ae63..ab6cb63146 100644 --- a/tests/integration/key_assignment.go +++ b/tests/integration/key_assignment.go @@ -30,7 +30,7 @@ func (s *CCVTestSuite) TestKeyAssignment() { } // check that a VSCPacket is queued - s.providerChain.NextBlock() + s.nextEpoch() pendingPackets := pk.GetPendingVSCPackets(s.providerCtx(), s.consumerChain.ChainID) s.Require().Len(pendingPackets, 1) @@ -51,7 +51,7 @@ func (s *CCVTestSuite) TestKeyAssignment() { if err != nil { return err } - s.providerChain.NextBlock() + s.nextEpoch() return nil }, false, 2, @@ -73,7 +73,7 @@ func (s *CCVTestSuite) TestKeyAssignment() { delAddr := s.providerChain.SenderAccount.GetAddress() delegate(s, delAddr, bondAmt) - s.providerChain.NextBlock() + s.nextEpoch() return nil }, false, 2, @@ -95,7 +95,7 @@ func (s *CCVTestSuite) TestKeyAssignment() { if err != nil { return err } - s.providerChain.NextBlock() + s.nextEpoch() return nil }, true, 2, @@ -118,7 +118,7 @@ func (s *CCVTestSuite) TestKeyAssignment() { if err != nil { return err } - s.providerChain.NextBlock() + s.nextEpoch() return nil }, false, 2, @@ -134,14 +134,14 @@ func (s *CCVTestSuite) TestKeyAssignment() { if err != nil { return err } - s.providerChain.NextBlock() + s.nextEpoch() // same key assignment err = pk.AssignConsumerKey(s.providerCtx(), s.consumerChain.ChainID, validator, consumerKey) if err != nil { return err } - s.providerChain.NextBlock() + s.nextEpoch() return nil }, true, 2, @@ -157,7 +157,7 @@ func (s *CCVTestSuite) TestKeyAssignment() { if err != nil { return err } - s.providerChain.NextBlock() + s.nextEpoch() // same key assignment validator, consumerKey = generateNewConsumerKey(s, 0) @@ -165,7 +165,7 @@ func (s *CCVTestSuite) TestKeyAssignment() { if err != nil { return err } - s.providerChain.NextBlock() + s.nextEpoch() return nil }, false, 3, diff --git a/tests/integration/setup.go b/tests/integration/setup.go index e401324c82..03c63d5502 100644 --- a/tests/integration/setup.go +++ b/tests/integration/setup.go @@ -129,6 +129,12 @@ func (suite *CCVTestSuite) SetupTest() { suite.registerPacketSniffer(suite.providerChain) providerKeeper := suite.providerApp.GetProviderKeeper() + // set `BlocksPerEpoch` to 10: a reasonable small value greater than 1 that prevents waiting for too + // many blocks and slowing down the integration tests + params := providerKeeper.GetParams(suite.providerCtx()) + params.BlocksPerEpoch = 10 + providerKeeper.SetParams(suite.providerCtx(), params) + // re-assign all validator keys for the first consumer chain providerKeeper.SetPendingConsumerAdditionProp(suite.providerCtx(), &types.ConsumerAdditionProposal{ ChainId: icstestingutils.FirstConsumerChainID, diff --git a/tests/integration/slashing.go b/tests/integration/slashing.go index 2339538292..e7f585f756 100644 --- a/tests/integration/slashing.go +++ b/tests/integration/slashing.go @@ -107,8 +107,10 @@ func (s *CCVTestSuite) TestRelayAndApplyDowntimePacket() { s.Require().True(found) } + s.nextEpoch() + // Confirm the valset update Id was incremented twice on provider, - // since two endblockers have passed. + // since an epoch has passed. s.Require().Equal(valsetUpdateIdN+2, providerKeeper.GetValidatorSetUpdateId(s.providerCtx())) diff --git a/tests/integration/soft_opt_out.go b/tests/integration/soft_opt_out.go index a5ee566a4b..a9508118bd 100644 --- a/tests/integration/soft_opt_out.go +++ b/tests/integration/soft_opt_out.go @@ -73,7 +73,7 @@ func (suite *CCVTestSuite) TestSoftOptOut() { bondAmt := sdk.NewInt(100).Mul(sdk.DefaultPowerReduction) delegateByIdx(suite, delAddr, bondAmt, valIdx) - suite.providerChain.NextBlock() + suite.nextEpoch() // Relay 1 VSC packet from provider to consumer relayAllCommittedPackets(suite, suite.providerChain, suite.path, ccv.ProviderPortID, suite.path.EndpointB.ChannelID, 1) @@ -112,7 +112,7 @@ func (suite *CCVTestSuite) TestSoftOptOut() { bondAmt := sdk.NewInt(100).Mul(sdk.DefaultPowerReduction) delegateByIdx(suite, delAddr, bondAmt, valIdx) - suite.providerChain.NextBlock() + suite.nextEpoch() // Relay 1 VSC packet from provider to consumer relayAllCommittedPackets(suite, suite.providerChain, suite.path, ccv.ProviderPortID, suite.path.EndpointB.ChannelID, 1) @@ -149,6 +149,8 @@ func (suite *CCVTestSuite) TestSoftOptOut() { validatorPowers := []int64{1000, 500, 50, 10} suite.setupValidatorPowers(validatorPowers) + suite.nextEpoch() + // Relay 1 VSC packet from provider to consumer relayAllCommittedPackets(suite, suite.providerChain, suite.path, ccv.ProviderPortID, suite.path.EndpointB.ChannelID, 1) diff --git a/tests/integration/unbonding.go b/tests/integration/unbonding.go index 00f48871c2..7f87516444 100644 --- a/tests/integration/unbonding.go +++ b/tests/integration/unbonding.go @@ -231,8 +231,7 @@ func (s *CCVTestSuite) TestUndelegationDuringInit() { // update init timeout timestamp tc.updateInitTimeoutTimestamp(&providerKeeper, providerUnbondingPeriod) - // call NextBlock on the provider (which increments the height) - s.providerChain.NextBlock() + s.nextEpoch() // check that the VSC packet is stored in state as pending pendingVSCs := providerKeeper.GetPendingVSCPackets(s.providerCtx(), s.consumerChain.ChainID) @@ -241,8 +240,7 @@ func (s *CCVTestSuite) TestUndelegationDuringInit() { // delegate again to create another VSC packet delegate(s, delAddr, bondAmt) - // call NextBlock on the provider (which increments the height) - s.providerChain.NextBlock() + s.nextEpoch() // check that the VSC packet is stored in state as pending pendingVSCs = providerKeeper.GetPendingVSCPackets(s.providerCtx(), s.consumerChain.ChainID) @@ -266,6 +264,7 @@ func (s *CCVTestSuite) TestUndelegationDuringInit() { // complete CCV channel setup s.SetupCCVChannel(s.path) + s.nextEpoch() // relay VSC packets from provider to consumer relayAllCommittedPackets(s, s.providerChain, s.path, ccv.ProviderPortID, s.path.EndpointB.ChannelID, 2) @@ -429,8 +428,8 @@ func (s *CCVTestSuite) TestRedelegationProviderFirst() { // Check that CCV unbonding op was created from AfterUnbondingInitiated hook checkCCVUnbondingOp(s, s.providerCtx(), s.consumerChain.ChainID, valsetUpdateID, true) - // Call NextBlock on the provider (which increments the height) - s.providerChain.NextBlock() + // move forward by an epoch to be able to relay VSC packets + s.nextEpoch() // Relay 2 VSC packets from provider to consumer (original delegation, and redelegation) relayAllCommittedPackets(s, s.providerChain, s.path, diff --git a/tests/integration/valset_update.go b/tests/integration/valset_update.go index dedcce2b86..eb0560a35e 100644 --- a/tests/integration/valset_update.go +++ b/tests/integration/valset_update.go @@ -23,8 +23,8 @@ func (s *CCVTestSuite) TestPacketRoundtrip() { delAddr := s.providerChain.SenderAccount.GetAddress() delegate(s, delAddr, bondAmt) - // Send CCV packet to consumer - s.providerChain.NextBlock() + // Send CCV packet to consumer at the end of the epoch + s.nextEpoch() // Relay 1 VSC packet from provider to consumer relayAllCommittedPackets(s, s.providerChain, s.path, ccv.ProviderPortID, s.path.EndpointB.ChannelID, 1) diff --git a/tests/mbt/driver/core.go b/tests/mbt/driver/core.go index f9f1e12e05..b49e98766a 100644 --- a/tests/mbt/driver/core.go +++ b/tests/mbt/driver/core.go @@ -75,6 +75,10 @@ func (s *Driver) providerChain() *ibctesting.TestChain { return s.chain("provider") } +func (s *Driver) providerHeight() int64 { + return s.providerChain().CurrentHeader.Height +} + func (s *Driver) providerCtx() sdk.Context { return s.providerChain().GetContext() } diff --git a/tests/mbt/driver/mbt_test.go b/tests/mbt/driver/mbt_test.go index 78f9e7910f..01faa9d089 100644 --- a/tests/mbt/driver/mbt_test.go +++ b/tests/mbt/driver/mbt_test.go @@ -151,10 +151,24 @@ func RunItfTrace(t *testing.T, path string) { nodes[i] = addressMap[valName] } + // very hacky: the system produces a lot of extra blocks, e.g. when setting up consumer chains, when updating clients, etc. + // to be able to compare the model to the system, we make the blocks per epoch a very large number (such that an epoch never ends naturally in the system while running the trace) + // When an epoch in the model ends (which we can detect by the height modulo the epoch length), we produce many, many blocks in the system, such that an epoch actually ends. + blocksPerEpoch := int64(200) + modelBlocksPerEpoch := params["BlocksPerEpoch"].Value.(int64) + driver := newDriver(t, nodes, valNames) driver.DriverStats = &stats driver.setupProvider(modelParams, valSet, signers, nodes, valNames) + providerParams := driver.providerKeeper().GetParams(driver.providerCtx()) + providerParams.BlocksPerEpoch = blocksPerEpoch + driver.providerKeeper().SetParams(driver.providerCtx(), providerParams) + + // begin enough blocks to end the first epoch + for i := int64(1); i < blocksPerEpoch; i++ { + driver.endAndBeginBlock("provider", 1*time.Nanosecond) + } // remember the time offsets to be able to compare times to the model // this is necessary because the system needs to do many steps to initialize the chains, @@ -166,8 +180,16 @@ func RunItfTrace(t *testing.T, path string) { t.Log("Reading the trace...") for index, state := range trace.States { + t.Log("Height modulo epoch length:", driver.providerChain().CurrentHeader.Height%blocksPerEpoch) + t.Log("Model height modulo epoch length:", ProviderHeight(state.VarValues["currentState"].Value.(itf.MapExprType))%modelBlocksPerEpoch) t.Logf("Reading state %v of trace %v", index, path) + // store the height of the provider state before each step. + // The height should only pass an epoch when it passes an epoch in the model, too, + // otherwise there is an error, and blocksPerEpoch needs to be increased. + // See the comment for blocksPerEpoch above. + heightBeforeStep := driver.providerHeight() + trace := state.VarValues["trace"].Value.(itf.ListExprType) // lastAction will get the last action that was executed so far along the model trace, // i.e. the action we should perform before checking model vs actual system equivalence @@ -205,13 +227,34 @@ func RunItfTrace(t *testing.T, path string) { stats.numStartedChains += len(consumersToStart) stats.numStops += len(consumersToStop) - // we need 2 blocks, because for a packet sent at height H, the receiving chain + // get the block height in the model + modelHeight := ProviderHeight(currentModelState) + + if modelHeight%modelBlocksPerEpoch == 0 { + // in the model, an epoch ends, so we need to produce blocks in the system to get the actual height + // to end an epoch with the first of the two subsequent calls to endAndBeginBlock below + actualHeight := driver.providerHeight() + + heightInEpoch := actualHeight % blocksPerEpoch + + // produce blocks until the next block ends the epoch + for i := heightInEpoch; i < blocksPerEpoch; i++ { + driver.endAndBeginBlock("provider", 1*time.Nanosecond) + } + } + + // we need at least 2 blocks, because for a packet sent at height H, the receiving chain // needs a header of height H+1 to accept the packet - // so we do one time advancement with a very small increment, - // and then increment the rest of the time + // so, we do two blocks, one with a very small increment, + // and then another to increment the rest of the time runningConsumersBefore := driver.runningConsumers() + driver.endAndBeginBlock("provider", 1*time.Nanosecond) + for _, consumer := range driver.runningConsumers() { + UpdateProviderClientOnConsumer(t, driver, consumer.ChainId) + } driver.endAndBeginBlock("provider", time.Duration(timeAdvancement)*time.Second-1*time.Nanosecond) + runningConsumersAfter := driver.runningConsumers() // the consumers that were running before but not after must have timed out @@ -378,6 +421,14 @@ func RunItfTrace(t *testing.T, path string) { stats.EnterStats(driver) + // should not have ended an epoch, unless we also ended an epoch in the model + heightAfterStep := driver.providerHeight() + + if heightBeforeStep/blocksPerEpoch != heightAfterStep/blocksPerEpoch { + // we changed epoch during this step, so ensure that the model also changed epochs + require.True(t, ProviderHeight(state.VarValues["currentState"].Value.(itf.MapExprType))%modelBlocksPerEpoch == 0, "Height in model did not change epoch, but did in system. increase blocksPerEpoch in the system") + } + t.Logf("State %v of trace %v is ok!", index, path) } t.Log("🟢 Trace is ok!") diff --git a/tests/mbt/driver/model_viewer.go b/tests/mbt/driver/model_viewer.go index ed090b2b86..f1f786c4f0 100644 --- a/tests/mbt/driver/model_viewer.go +++ b/tests/mbt/driver/model_viewer.go @@ -40,6 +40,10 @@ func RunningTime(curStateExpr itf.MapExprType, chain string) int64 { return ChainState(curStateExpr, chain)["runningTimestamp"].Value.(int64) } +func ProviderHeight(curStateExpr itf.MapExprType) int64 { + return ProviderState(curStateExpr)["chainState"].Value.(itf.MapExprType)["currentBlockHeight"].Value.(int64) +} + // PacketQueue returns the queued packets between sender and receiver. // Either sender or receiver need to be the provider. func PacketQueue(curStateExpr itf.MapExprType, sender, receiver string) itf.ListExprType { diff --git a/tests/mbt/driver/setup.go b/tests/mbt/driver/setup.go index 83fa6e0669..c0020a4095 100644 --- a/tests/mbt/driver/setup.go +++ b/tests/mbt/driver/setup.go @@ -356,6 +356,26 @@ func (s *Driver) ConfigureNewPath(consumerChain, providerChain *ibctesting.TestC NewChain: consumerGenesis.NewChain, } + var stakingValidators []stakingtypes.Validator + + // set up the current consumer validators by utilizing the initial validator set + for _, val := range consumerGenesisForProvider.Provider.InitialValSet { + pubKey := val.PubKey + consAddr, err := ccvtypes.TMCryptoPublicKeyToConsAddr(pubKey) + if err != nil { + continue + } + + v, found := s.providerStakingKeeper().GetValidatorByConsAddr(s.providerCtx(), consAddr) + if !found { + continue + } + stakingValidators = append(stakingValidators, v) + } + + nextValidators := s.providerKeeper().ComputeNextEpochConsumerValSet(s.providerCtx(), string(consumerChainId), stakingValidators) + s.providerKeeper().SetConsumerValSet(s.providerCtx(), string(consumerChainId), nextValidators) + err = s.providerKeeper().SetConsumerGenesis( providerChain.GetContext(), string(consumerChainId), @@ -377,22 +397,6 @@ func (s *Driver) ConfigureNewPath(consumerChain, providerChain *ibctesting.TestC // their channel, and are ready for anything to happen. s.consumerKeeper(consumerChainId).SetProviderChannel(s.ctx(consumerChainId), consumerEndPoint.ChannelID) - // Commit a block on both chains, giving us two committed headers from - // the same time and height. This is the starting point for all our - // data driven testing. - lastConsumerHeader, _ := simibc.EndBlock(consumerChain, func() {}) - lastProviderHeader, _ := simibc.EndBlock(providerChain, func() {}) - - // Get ready to update clients. - simibc.BeginBlock(providerChain, 5) - simibc.BeginBlock(consumerChain, 5) - - // Update clients to the latest header. - err = simibc.UpdateReceiverClient(consumerEndPoint, providerEndPoint, lastConsumerHeader, false) - require.NoError(s.t, err, "Error updating client on consumer for chain %v", consumerChain.ChainID) - err = simibc.UpdateReceiverClient(providerEndPoint, consumerEndPoint, lastProviderHeader, false) - require.NoError(s.t, err, "Error updating client on provider for chain %v", consumerChain.ChainID) - // path is ready to go return path } diff --git a/tests/mbt/model/ccv.qnt b/tests/mbt/model/ccv.qnt index 6d9450b6f0..51def4c01e 100644 --- a/tests/mbt/model/ccv.qnt +++ b/tests/mbt/model/ccv.qnt @@ -52,6 +52,8 @@ module ccv_types { // the running timestamp of the current block (that will be put on chain when the block is ended) runningTimestamp: Time, + + currentBlockHeight: int, } // utility function: returns a chain state that is initialized minimally. @@ -61,6 +63,7 @@ module ccv_types { currentValidatorSet: Map(), lastTimestamp: -1, // last timestamp -1 means that in the model, there was no block committed on chain yet. runningTimestamp: 0, + currentBlockHeight: 0 } // Defines the current state of the provider chain. Essentially, all information here is stored by the provider on-chain (or could be derived purely by information that is on-chain). @@ -80,11 +83,8 @@ module ccv_types { // Stores VscPackets which have been sent but where the provider has *not received a response yet*. sentVscPacketsToConsumer: Chain -> List[VscPacket], - // stores whether, in this block, the validator set has changed. - // this is needed because the validator set might be considered to have changed, even though - // it is still technically identical at our level of abstraction, e.g. a validator power change on the provider - // might leave the validator set the same because a delegation and undelegation cancel each other out. - providerValidatorSetChangedInThisBlock: bool, + // stores for which consumer chains, in this epoch, the validator set is considered to have changed and we thus need to send a VscPacket to the consumer chains. + consumersWithPowerChangesInThisEpoch: Set[Chain], // stores, for each consumer chain, its current status - // not consumer, running, or stopped @@ -93,6 +93,28 @@ module ccv_types { // a monotonic strictly increasing and positive ID that is used // to uniquely identify the Vscs sent to the consumer chains. runningVscId: int, + + // For every consumer chain, stores the consumer address assigned by each validator. + validatorToConsumerAddr: Chain -> (Node -> ConsumerAddr), + + // For every consumer chain, holds the provider validator for each assigned consumer address. + // Note that this is *not* precisely the reverse of validatorToConsumerAddr, + // because when a validator changes their consumer addr, + // the old one stays in this map until pruned. + consumerAddrToValidator: Chain -> (ConsumerAddr -> Node), + + // For every consumer chain, stores whether the key assignment for the consumer chain has changed in this block. + consumersWithAddrAssignmentChangesInThisEpoch: Set[Chain], + + // the history of validator sets on the provider, but with the key assignments applied. + // This is needed to check invariants about the validator set when key assignments are in play. + keyAssignedValSetHistory: Chain -> VotingPowerHistory, + + // Stores the mapping from VSC ids to consumer validators addresses. Needed for pruning consumerAddrToValidator. + consumerAddrsToPrune: Chain -> VscId -> List[ConsumerAddr], + + // For every sent VSCPacket, stores the key assignments that were applied to send it. + keyAssignmentsForVSCPackets: VscId -> (Chain -> (Node -> ConsumerAddr)) } // utility function: returns a provider state that is initialized minimally. @@ -102,9 +124,15 @@ module ccv_types { outstandingPacketsToConsumer: Map(), receivedMaturations: Set(), sentVscPacketsToConsumer: Map(), - providerValidatorSetChangedInThisBlock: false, + consumersWithPowerChangesInThisEpoch: Set(), consumerStatus: Map(), runningVscId: 0, + validatorToConsumerAddr: Map(), + keyAssignedValSetHistory: Map(), + consumerAddrToValidator: Map(), + consumerAddrsToPrune: Map(), + keyAssignmentsForVSCPackets: Map(), + consumersWithAddrAssignmentChangesInThisEpoch: Set() } @@ -236,6 +264,10 @@ module ccv { // they expire and the channel will be closed. const TrustingPeriodPerChain: Chain -> int + // The number of blocks in an epoch. + // VscPackets are only sent to consumer chains at the end of every epoch. + const BlocksPerEpoch: int + // =================== // PROTOCOL LOGIC contains the meat of the protocol // functions here roughly correspond to API calls that can be triggered from external sources @@ -259,7 +291,7 @@ module ccv { } else { // set the validator set changed flag val newProviderState = currentState.providerState.with( - "providerValidatorSetChangedInThisBlock", true + "consumersWithPowerChangesInThisEpoch", getRunningConsumers(currentState.providerState) ) pure val tmpState = currentState.with( "providerState", newProviderState @@ -407,14 +439,16 @@ module ccv { // send vsc packets val providerStateAfterSending = - if (currentProviderState.providerValidatorSetChangedInThisBlock and - // important: check this on the provider state after the consumer advancement, not on the current state. - getRunningConsumers(providerStateAfterTimeAdvancement).size() > 0) { - // need to use the old timestamp because this happens during EndBlock - providerStateAfterTimeAdvancement.sendVscPackets(currentProviderState.chainState.runningTimestamp) - } else { - providerStateAfterTimeAdvancement - } + // if currentBlockHeight is a multiple of BlocksPerEpoch, send VscPackets + if (providerStateAfterTimeAdvancement.chainState.currentBlockHeight % BlocksPerEpoch == 0) { + providerStateAfterTimeAdvancement.sendVscPackets( + currentProviderState.chainState.runningTimestamp, + CcvTimeout.get(PROVIDER_CHAIN) + ) + } else { + // otherwise, just do a noop + providerStateAfterTimeAdvancement + } // start/stop chains @@ -427,8 +461,6 @@ module ccv { val err = res._2 val providerStateAfterConsumerAdvancement = providerStateAfterSending.with( "consumerStatus", newConsumerStatus - ).with( - "providerValidatorSetChangedInThisBlock", false ) if (err != "") { @@ -553,6 +585,43 @@ module ccv { } else { currentConsumerStatusMap.get(chain) } + // this key can be assigned + + // get the old assigned key + pure val consKeyByVal = currentState.providerState.validatorToConsumerAddr.getOrElse(consumer, Map()) + pure val p = if (consKeyByVal.keys().contains(providerNode)) { + // providerNode had previously assigned a consumer key + (consKeyByVal.get(providerNode), true) + } else { + (providerNode, false) + } + // the consumer address that was previously associated with the node + pure val oldConsAddr = p._1 + // whether the old address was explicitly assigned, or the default key + pure val prevAssigned = p._2 + + // set the old address for pruning, if it was assigned + pure val tmpState = if (prevAssigned) { + AppendConsumerAddrToPrune(currentState, oldConsAddr, consumer) + } else { + currentState + } + + // check whether the validator has positive power + pure val provValSet = currentState.providerState.chainState.currentValidatorSet + pure val provValPower = if (provValSet.keys().contains(providerNode)) provValSet.get(providerNode) else 0 + pure val consumersWithAddrAssignmentChangesInThisEpoch = + if (provValPower > 0) { + // if the consumer has positive power, the relevant key assignment for the consumer changed + currentState.providerState.consumersWithAddrAssignmentChangesInThisEpoch.union(Set(consumer)) + } else { + // otherwise, the consumer doesn't need to know about the change, so no change + currentState.providerState.consumersWithAddrAssignmentChangesInThisEpoch + } + pure val tmpStateAfterKeyAssignmentReplacement = tmpState.with( + "providerState", tmpState.providerState.with( + "consumersWithAddrAssignmentChangesInThisEpoch", consumersWithAddrAssignmentChangesInThisEpoch + ) ) (newConsumerStatusMap, "") } diff --git a/tests/mbt/model/ccv_model.qnt b/tests/mbt/model/ccv_model.qnt index fba4ddea6c..3665837c2a 100644 --- a/tests/mbt/model/ccv_model.qnt +++ b/tests/mbt/model/ccv_model.qnt @@ -12,6 +12,7 @@ module ccv_model { pure val unbondingPeriods = chains.mapBy(chain => defUnbondingPeriod) pure val trustingPeriods = chains.mapBy(chain => defUnbondingPeriod - 1 * Hour) pure val ccvTimeouts = chains.mapBy(chain => 3 * Week) + pure val epochLength = 3 pure val nodes = Set("node1", "node2", "node3", "node4", "node5", "node6", "node7", "node8", "node9", "node10") pure val InitialValidatorSet = nodes.mapBy(node => 100) @@ -21,7 +22,8 @@ module ccv_model { CcvTimeout = ccvTimeouts, UnbondingPeriodPerChain = unbondingPeriods, ConsumerChains = consumerChains, - TrustingPeriodPerChain = trustingPeriods + TrustingPeriodPerChain = trustingPeriods, + BlocksPerEpoch = epochLength ).* from "./ccv" type Parameters = { @@ -32,6 +34,7 @@ module ccv_model { ConsumerChains: Set[Chain], Nodes: Set[Node], InitialValidatorSet: Node -> int, + BlocksPerEpoch: int, } // The params variable is never actually changed, and @@ -120,6 +123,8 @@ module ccv_model { Nodes: nodes, InitialValidatorSet: InitialValidatorSet, TrustingPeriodPerChain: TrustingPeriodPerChain, + ConsumerAddresses: consumerAddresses, + BlocksPerEpoch: epochLength, } } @@ -225,6 +230,25 @@ module ccv_model { stepCommon } + // Runs epochLength many blocks on the provider. + // The first block will start all consumers in consumersToStart and stop all consumers in consumersToStop, + // and advance time by timeAdvancement - ((epochLength-1) * Seconds). + // The rest of the blocks will not start or stop any consumers, and will advance time by 1 second each. + // As a a `Run`, it is only used in tests, not during simulation or verification. + run EndProviderEpoch( + timeAdvancement: Time, + consumersToStart: Set[Chain], + consumersToStop: Set[Chain] + ): bool = + epochLength.reps( + i => + if (i == 0) { + EndAndBeginBlockForProvider(timeAdvancement-((epochLength-1)*Second), consumersToStart, consumersToStop) + } else { + EndAndBeginBlockForProvider(1 * Second, Set(), Set()) + } + ) + // ================== // UTILITY FUNCTIONS // ================== @@ -296,16 +320,19 @@ module ccv_model { ) ) + + val CurrentBlockEndsEpoch = currentState.providerState.chainState.currentBlockHeight % epochLength == 0 + // Any update in the power of a validator on the provider // MUST be present in a ValidatorSetChangePacket that is sent to all registered consumer chains val ValUpdatePrecondition = trace[trace.length()-1].kind == "EndAndBeginBlockForProvider" val ValidatorUpdatesArePropagatedInv = - // when the provider has just entered a validator set into a block... - ValUpdatePrecondition and currentState.providerState.providerValidatorSetChangedInThisBlock + // when the an epoch ends and the provider has just entered a validator set into a block... + ValUpdatePrecondition and CurrentBlockEndsEpoch implies val providerValSetInCurBlock = providerValidatorHistory.head() - // ... for each consumer that is running then ... - runningConsumers.forall( + // ... for each consumer for which we need to send a vsc packet ... + currentState.providerState.consumersWithPowerChangesInThisEpoch.forall( // ...the validator set is in a sent packet... consumer => currentState.providerState.sentVscPacketsToConsumer.get(consumer).toSet().exists( packet => packet.validatorSet == providerValSetInCurBlock @@ -494,9 +521,10 @@ module ccv_model { assert(currentState.providerState.chainState.votingPowerHistory == List(InitialValidatorSet.put("node1", 150), InitialValidatorSet)), // change voting power on provider again VotingPowerChange("node1", 50).then( - // end another block - EndAndBeginBlockForProvider(1 * Second, Set(), Set()) - ).then( + // end the epoch + EndProviderEpoch(epochLength * Second, Set(), Set()) + ) + .then( // deliver packet to consumer1 DeliverVscPacket("consumer1") ) @@ -565,7 +593,7 @@ module ccv_model { VotingPowerChange("node1", 50) ).then( // send packet to consumer1 and consumer2 - EndAndBeginBlockForProvider(1 * Second, Set(), Set()) + EndProviderEpoch(epochLength * Second, Set(), Set()) ).then( // deliver the packets DeliverVscPacket("consumer1") @@ -580,7 +608,7 @@ module ccv_model { VotingPowerChange("node2", 50) ).then( // send packets - EndAndBeginBlockForProvider(1 * Second, Set(), Set()) + EndProviderEpoch(epochLength * Second, Set(), Set()) ).then( //deliver to consumer1 DeliverVscPacket("consumer1") @@ -613,7 +641,7 @@ module ccv_model { ) .then( // send packets - EndAndBeginBlockForProvider(1 * Second, Set(), Set()) + EndProviderEpoch(epochLength * Second, Set(), Set()) ) .then( // advance time on provider by VscTimeout + 1 Second @@ -632,4 +660,270 @@ module ccv_model { VotingPowerChange("node1", 50) // action needs to be there but does not matter what it is } ) + + + // ===== KEY ASSIGNMENT ======= + action stepKeyAssignment = + any { + step, + nondetKeyAssignment, + } + + action nondetKeyAssignment = + all { + runningConsumers.size() > 0, + nondet node = oneOf(nodes) + nondet consumerAddr = oneOf(consumerAddresses) + nondet consumer = oneOf(runningConsumers) + KeyAssignment(consumer, node, consumerAddr), + } + + action KeyAssignment( + chain: Chain, + validator: Node, + consumerAddr: ConsumerAddr + ): bool = + val result = assignConsumerKey(currentState, chain, validator, consumerAddr) + all { + hasError(result) == false, + currentState' = result.newState, + trace' = trace.append( + {...emptyAction, + kind: "KeyAssignment", + consumerChain: chain, + validator: validator, + consumerAddr: consumerAddr + } + ), + params' = params, + } + + // invariants for key assignment - some invariants are in addition, some need to be adjusted from the original model + + // Every validator set on any consumer chain MUST either be or have been + // a validator set on the provider chain, under the key assignment at the time. + val providerKeyAssignedValSetHistory = currentState.providerState.keyAssignedValSetHistory + + val ValidatorSetHasExistedKeyAssignmentInv = + runningConsumers.forall(chain => // for every running consumer + currentState.consumerStates.get(chain).chainState.votingPowerHistory.toSet().forall( + // for every validator set the consumer ever had + validatorSet => providerKeyAssignedValSetHistory.getOrElse(chain, List()).toSet().exists( + // that validator set needs to also have existed on the provider + provValSet => removeZeroPowers(provValSet) == removeZeroPowers(validatorSet) + ) + ) + ) + + // Any update in the power of a validator on the provider + // MUST be present in a ValidatorSetChangePacket that is sent to all registered consumer chains, + // and the key assignment of each validator should be applied in that VSCPacket. + val ValidatorUpdatesArePropagatedKeyAssignmentInv = + // when the provider has just entered a validator set into a block... + ValUpdatePrecondition and CurrentBlockEndsEpoch + implies + val providerValSetInCurBlock = providerValidatorHistory.head() + // ... for each consumer that is running then ... + currentState.providerState.consumersWithPowerChangesInThisEpoch.forall( + // ...the validator set under key assignment is in a sent packet... + val providerState = currentState.providerState + consumer => providerState.sentVscPacketsToConsumer.get(consumer).toSet().exists( + packet => + packet.validatorSet == + applyKeyAssignmentToValSet(providerState, consumer, providerValSetInCurBlock) + ) + ) + + // Every consumer chain receives the same sequence of + // ValidatorSetChangePackets in the same order. + // NOTE: since not all consumer chains are running all the time, + // we need a slightly weaker invariant: + // For consumer chains c1, c2, if both c1 and c2 received a packet p1 sent at t1 and a packet p2 sent at t2, + // then both have received ALL packets that were sent between t1 and t2. + val SameVscPacketsKeyAssignmentInv = + runningConsumers.forall( + consumer1 => runningConsumers.forall( + consumer2 => { + val packets1 = currentState.consumerStates.get(consumer1).receivedVscPackets + val packets2 = currentState.consumerStates.get(consumer2).receivedVscPackets + val commonPackets = packets1.toSet().intersect(packets2.toSet()) + if (commonPackets.size() == 0) { + true // they don't share any packets, so nothing to check + } else { + val newestCommonPacket = newest(commonPackets) + val oldestCommonPacket = oldest(commonPackets) + // get all packets sent between the oldest and newest common packet + val packetsBetween1 = packets1.select( + packet => packet.sendingTime >= oldestCommonPacket.sendingTime and packet.sendingTime <= newestCommonPacket.sendingTime + ) + val packetsBetween2 = packets2.select( + packet => packet.sendingTime >= oldestCommonPacket.sendingTime and packet.sendingTime <= newestCommonPacket.sendingTime + ) + + // revert key assignments + val packetsBetween1noKeyAssignment = packetsBetween1.foldl( + List(), + (acc, packet) => + acc.concat(List({...packet, + validatorSet: revertKeyAssignment( + currentState.providerState.keyAssignmentsForVSCPackets + .getOrElse(packet.id, Map()) + .getOrElse(consumer1, Map()), + packet.validatorSet) + })) + ) + + val packetsBetween2noKeyAssignment = packetsBetween2.foldl( + List(), (acc, packet) => + acc.concat(List({...packet, + validatorSet: revertKeyAssignment( + currentState.providerState.keyAssignmentsForVSCPackets + .getOrElse(packet.id, Map()) + .getOrElse(consumer2, Map()), + packet.validatorSet) + })) + ) + // check that the packets between the common packets are equal + // when key assignment is reversed + packetsBetween1noKeyAssignment == packetsBetween2noKeyAssignment + } + } + ) + ) + + + // Rules for key assignment: + val KeyAssignmentRulesInv = + NoProviderReuse and NoDuplicationOnSameConsumer + + // validator A cannot assign consumer key K to consumer chain X if there is already a validator B (B!=A) using K on the provider + val NoProviderReuse = + consumerChains.forall( + consumer => + val valConsPk = currentState.providerState.validatorToConsumerAddr.getOrElse(consumer, Map()) + valConsPk.keys().forall( + node => + val consAddr = valConsPk.get(node) + // either the key is the nodes key itself (B == A) + consAddr == node or + // or the consAddr must not be a validator on the provider + not(currentState.providerState.chainState.currentValidatorSet.keys().contains(consAddr)) + ) + ) + + // validator A cannot assign consumer key K to consumer chain X if there is already a validator B using K on X + val NoDuplicationOnSameConsumer = + consumerChains.forall( + consumer => + val valConsPk = currentState.providerState.validatorToConsumerAddr.getOrElse(consumer, Map()) + valConsPk.keys().forall( + node => + val consAddr = valConsPk.get(node) + // no other node may use consAddr + not(valConsPk.keys().exists( + otherNode => otherNode != node and valConsPk.get(otherNode) == consAddr + )) + ) + ) + + // sanity checks + val CanAssignConsumerKey = + not(consumerChains.exists( + consumer => + currentState.providerState.consumerAddrToValidator.getOrElse(consumer, Map()).keys().size() > 0 + )) + + val CanHaveConsumerAddresses = + not(consumerChains.exists( + consumer => + currentState.consumerStates.get(consumer).chainState.currentValidatorSet.keys().exists( + addr => addr.in(consumerAddresses) + ) + )) + + // == tests for key assignment == + run KeyAssignmentTest = + init + .then( + // start all consumer chains + EndAndBeginBlockForProvider(1 * Second, consumerChains, Set()) + ) + .then( + // node 1 assigns a key on consumer1 + KeyAssignment("consumer1", "node1", "consAddr1") + ) + .then( + // end and begin block to make sure the key assignment is processed and the packet is sent + EndProviderEpoch(epochLength * Second, Set(), Set()) + ) + .then( + // receive the packet on the consumer + DeliverVscPacket("consumer1") + ) + .then( + // end and begin block to make sure the packet is processed + EndAndBeginBlockForConsumer("consumer1", 1 * Second) + ) + .then( + all { + // the key should be present in the valset on the consumer, and the node itself should not + assert(currentState.consumerStates.get("consumer1").chainState.currentValidatorSet.getOrElse("node1", 0) == 0), + assert(currentState.consumerStates.get("consumer1").chainState.currentValidatorSet.get("consAddr1") == 100), + // try some key assignments that should fail/succeed without committing to state + val res = assignConsumerKey(currentState, "consumer1", "node1", "consAddr1") + // fail - key already assigned (even if it is the same node) + assert(hasError(res)), + val res2 = assignConsumerKey(currentState, "consumer1", "node2", "consAddr1") + // fail - key assigned to other node + assert(hasError(res2)), + val res3 = assignConsumerKey(currentState, "consumer2", "node2", "consAddr1") + // ok - may reuse the key on a different consumer + assert(not(hasError(res3))), + val res4 = assignConsumerKey(currentState, "consumer1", "node2", "node1") + // fail - may not reuse a provider key of a different val + assert(hasError(res4)), + val res5 = assignConsumerKey(currentState, "consumer1", "node1", "consAddr2") + // ok - assigning unused key to node + assert(not(hasError(res5))), + val res6 = assignConsumerKey(currentState, "consumer1", "node1", "node1") + // ok - going back to original key + assert(not(hasError(res6))), + // mature the vsc packet on the consumer + EndAndBeginBlockForConsumer("consumer1", unbondingPeriods.get("consumer1") + 1 * Hour) + } + ) + .then( + // End a block to send the maturation + EndAndBeginBlockForConsumer("consumer1", 1 * Second) + ) + .then( + // deliver the vsc matured packet to the provider + DeliverVscMaturedPacket("consumer1") + ) + .then( + // the old key should have been pruned + all { + // check that pruning has been performed nicely + assert(currentState.providerState.consumerAddrToValidator.get("consumer1").get("consAddr1") == "node1"), + assert(currentState.providerState.consumerAddrsToPrune.get("consumer1").getOrElse(0, List()).length() == 0), + // action does not matter + VotingPowerChange("node1", 50) + } + ) + + run KeyAssignmentInvTest = + init.then( + KeyAssignment("consumer3", "node3", "consAddr6") + ).then( + VotingPowerChange("node1", 50) + ) + .then( + EndAndBeginBlockForProvider(1 * Second, Set("consumer1", "consumer2"), Set()) + ).then( + all { + ValidatorSetHasExistedKeyAssignmentInv, + // action doesn't matter + VotingPowerChange("node1", 50) + } + ) } \ No newline at end of file diff --git a/tests/mbt/model/ccv_test.qnt b/tests/mbt/model/ccv_test.qnt index 4dae28ec03..9c21d8e541 100644 --- a/tests/mbt/model/ccv_test.qnt +++ b/tests/mbt/model/ccv_test.qnt @@ -11,8 +11,9 @@ module ccv_test { pure val unbondingPeriods = chains.mapBy(chain => 2 * Week) pure val ccvTimeouts = chains.mapBy(chain => 3 * Week) pure val trustingPeriods = chains.mapBy(chain => 2 * Week - 1 * Hour) + pure val epochLength = 3 - import ccv(VscTimeout = 5 * Week, CcvTimeout = ccvTimeouts, UnbondingPeriodPerChain = unbondingPeriods, ConsumerChains = consumerChains, TrustingPeriodPerChain=trustingPeriods).* from "./ccv" + import ccv(VscTimeout = 5 * Week, CcvTimeout = ccvTimeouts, UnbondingPeriodPerChain = unbondingPeriods, ConsumerChains = consumerChains, TrustingPeriodPerChain=trustingPeriods, BlocksPerEpoch=epochLength).* from "./ccv" val votingPowerTestInitState = GetEmptyProtocolState.with( @@ -24,6 +25,10 @@ module ccv_test { "validator2", 5 ) ) + ).with( + "consumerStatus", Map( + "consumer" -> RUNNING + ) ) ) @@ -77,7 +82,7 @@ module ccv_test { ) // still should set the flag not(finalResult.hasError()) and - finalResult.newState.providerState.providerValidatorSetChangedInThisBlock + finalResult.newState.providerState.consumersWithPowerChangesInThisEpoch.contains("consumer") } diff --git a/tests/mbt/model/ccv_utils.qnt b/tests/mbt/model/ccv_utils.qnt new file mode 100644 index 0000000000..476ef923dc --- /dev/null +++ b/tests/mbt/model/ccv_utils.qnt @@ -0,0 +1,463 @@ +// This file contains utility functions for ccv.qnt +// that are not part of the API/core logic of ccv, but still relevant +// for the functional logic of the protocol. +module ccv_utils { + import ccv_types.* from "./ccv" + import extraSpells.* from "./libraries/extraSpells" + import Time.* from "./libraries/Time" + + + // Takes the current provider state and validator set and returns + // the validator set under the current key assignments for the given consumer, as stored in the provider state. + pure def applyKeyAssignmentToValSet( + providerState: ProviderState, + consumer: Chain, + valSet: ValidatorSet + ): ValidatorSet = { + // map each validator to a tuple of (consumer address, voting power) + valSet.keys().map( + (node) => + pure val power = valSet.get(node) + // check if the validator has a key assigned + pure val validatorToConsumerAddr = providerState.validatorToConsumerAddr.getOrElse(consumer, Map()) + if (validatorToConsumerAddr.keys().contains(node)) { + // the validator has a key assigned + pure val consAddr = validatorToConsumerAddr.get(node) + (consAddr, power) + } else { + // the validator has no key assigned + // use the default key + (node, power) + } + ).fold( // fold the (addr,pow) tuples into a map addr -> pow + Map(), + (acc, pair) => acc.put(pair._1, pair._2) + ) + } + + // Takes a validator set, to which a key assignment has been applied, and reverts it, + // i.e. returns the original validator set. + // This also filters out validators that are assigned 0 power. + pure def revertKeyAssignment( + keyAssignment: Node -> ConsumerAddr, + valSetWithAssignment: ValidatorSet + ): ValidatorSet = { + // get an assignment from consumer addr to nodes + pure val reverseAssignment = keyAssignment.keys().map( + (consAddr) => + (keyAssignment.get(consAddr), consAddr) + ).fold( + Map(), + (acc, pair) => acc.put(pair._1, pair._2) + ) + + // for each node in the valset, reverse its key assignment + valSetWithAssignment.keys().map( + (addr) => + pure val power = valSetWithAssignment.get(addr) + // if the addr has a key assigned, use that. otherwise, the addr doesn't have a key assigned, + // and therefore *is* the key that should be used. + pure val consAddr = reverseAssignment.getOrElse(addr, addr) + (consAddr, power) + ).filter( + pair => pair._2 > 0 + ).fold( + Map(), + (acc, pair) => acc.put(pair._1, pair._2) + ) + } + + // Appends the key assignment for the given oldConsAddr on the consumer by a validator + // to be pruned when a VscMaturedPacket for the current runningVscId is received from the consumer. + pure def AppendConsumerAddrToPrune(currentState: ProtocolState, oldConsAddr: ConsumerAddr, consumer: Chain): ProtocolState = { + pure val vscId = currentState.providerState.runningVscId + pure val consumerAddrsToPrune = currentState.providerState.consumerAddrsToPrune.getOrElse(consumer, Map()) + pure val prevConsAddrs = consumerAddrsToPrune.getOrElse(oldConsAddr, Map()).getOrElse(vscId, []) + + pure val newConsAddrsToPrune = consumerAddrsToPrune.put(vscId, prevConsAddrs.append(oldConsAddr)) + + currentState.with( + "providerState", + currentState.providerState.with( + "consumerAddrsToPrune", + currentState.providerState.consumerAddrsToPrune.put(consumer, newConsAddrsToPrune) + ) + ) + } + + // Returns the new ConsumerStatusMap according to the consumers to stop + // and the consumers to time out. + // If a consumer is both stopped and timed out, it will be timed out. + // The second return is an error string: If it is not equal to "", + // it contains an error message, and the first return should be ignored. + pure def stopConsumers( + currentConsumerStatusMap: Chain -> str, + consumersToStop: Set[Chain], + consumersToTimeout: Set[Chain]): (Chain -> str, str) = { + val runningConsumers = currentConsumerStatusMap.keys().filter( + chain => currentConsumerStatusMap.get(chain) == RUNNING + ) + // all consumers to stop must be running right now, else we have an error + if (consumersToStop.exclude(runningConsumers).size() > 0) { + (currentConsumerStatusMap, "Cannot stop a consumer that is not running") + } else { + val newConsumerStatusMap = currentConsumerStatusMap.keys().mapBy( + (chain) => + if (consumersToTimeout.contains(chain)) { + TIMEDOUT + } else if (consumersToStop.contains(chain)) { + STOPPED + } else { + currentConsumerStatusMap.get(chain) + } + ) + (newConsumerStatusMap, "") + } + } + + // Returns the new ConsumerStatusMap according to the consumers to start. + // The second return is an error string: If it is not equal to "", + // it contains an error message, and the first return should be ignored. + pure def startConsumers( + currentConsumerStatusMap: Chain -> str, + consumersToStart: Set[Chain]): (Chain -> str, str) = { + val nonConsumers = currentConsumerStatusMap.keys().filter( + chain => currentConsumerStatusMap.get(chain) == NOT_CONSUMER + ) + // all consumers to start must be nonConsumers right now, otherwise we have an error + if (consumersToStart.exclude(nonConsumers).size() > 0) { + (currentConsumerStatusMap, "cannot start a consumer that is stopped or already a consumer") + } else { + val newConsumerStatusMap = currentConsumerStatusMap.keys().mapBy( + (chain) => + if (consumersToStart.contains(chain)) { + RUNNING + } else { + currentConsumerStatusMap.get(chain) + } + ) + (newConsumerStatusMap, "") + } + } + + pure def StartStopConsumers( + currentConsumerStatusMap: Chain -> str, + consumersToStart: Set[Chain], + consumersToStop: Set[Chain], + consumersToTimeout: Set[Chain] + ): (Chain -> str, str) = { + // check if any consumer is both started and stopped + if (consumersToStart.intersect(consumersToStop).size() > 0) { + (currentConsumerStatusMap, "Cannot start and stop a consumer at the same time") + } else { + val res1 = currentConsumerStatusMap.startConsumers(consumersToStart) + val newConsumerStatus = res1._1 + val err1 = res1._2 + val res2 = newConsumerStatus.stopConsumers(consumersToStop, consumersToTimeout) + val err2 = res2._2 + if (err1 != "") { + (currentConsumerStatusMap, err1) + } else if (err2 != "") { + (currentConsumerStatusMap, err2) + } else { + (res2._1, "") + } + } + } + + + // Takes the currentValidatorSet and puts it as the newest set of the voting history + pure def enterCurValSetIntoBlock(chainState: ChainState): ChainState = { + chainState.with( + "votingPowerHistory", chainState.votingPowerHistory.prepend( + chainState.currentValidatorSet + ) + ) + } + + // Advances the timestamp in the chainState by timeAdvancement + pure def advanceTime(chainState: ChainState, timeAdvancement: Time): ChainState = + { + ...chainState, + lastTimestamp: chainState.runningTimestamp, + runningTimestamp: chainState.runningTimestamp + timeAdvancement, + } + + pure def incrementBlockHeight(chainState: ChainState): ChainState = + { + chainState.with( + "currentBlockHeight", chainState.currentBlockHeight + 1 + ) + } + + // common logic to update the chain state, used by both provider and consumers. + pure def endAndBeginBlockShared(chainState: ChainState, timeAdvancement: Time): ChainState = { + chainState.enterCurValSetIntoBlock().advanceTime(timeAdvancement).incrementBlockHeight() + } + + // returns the providerState with the following modifications: + // * sends VscPackets to all running consumers, using the provided timestamp as sending time + // * increments the runningVscId + // This should only be called when the provider chain is ending a block. + // If no vsc packets need to be sent, this will be a noop. + // the ccv timeout should be the ccv timeout for the provider chain. + pure def sendVscPackets( + providerState: ProviderState, + sendingTimestamp: Time, + ccvTimeout: Time): ProviderState = { + val newSentPacketsPerConsumer = providerState.getConsumers().mapBy( // compute, for each consumer, a list of new packets to be sent + (consumer) => + // if validator set changed or the key assignments for this chain changed, and the consumer is running, send a packet + if ((providerState.consumersWithPowerChangesInThisEpoch.contains(consumer) or + providerState.consumersWithAddrAssignmentChangesInThisEpoch.contains(consumer)) + and + isRunningConsumer(consumer, providerState)) { + // send a packet, i.e. use a list with one element (the packet to be sent) + List({ + id: providerState.runningVscId, + // apply key assignment to the current validator set + validatorSet: providerState.applyKeyAssignmentToValSet( + consumer, + providerState.chainState.currentValidatorSet + ), + sendingTime: sendingTimestamp, + timeoutTime: sendingTimestamp + ccvTimeout + }) + } else { + // no packet to be sent, so empty list + List() + } + ) + val newOutstandingPacketsToConsumer = providerState.getConsumers().mapBy( + (consumer) => + providerState.outstandingPacketsToConsumer.getOrElse(consumer, List()).concat( + newSentPacketsPerConsumer.get(consumer) + ) + ) + val newSentVscPackets = providerState.getConsumers().mapBy( + (consumer) => + providerState.sentVscPacketsToConsumer.getOrElse(consumer, List()).concat( + newSentPacketsPerConsumer.get(consumer) + ) + ) + { + ...providerState, + outstandingPacketsToConsumer: newOutstandingPacketsToConsumer, + sentVscPacketsToConsumer: newSentVscPackets, + runningVscId: providerState.runningVscId + 1, + // we ended the block and processed that the valset or key assignments changed, + // so reset the flags + consumersWithPowerChangesInThisEpoch: Set(), + consumersWithAddrAssignmentChangesInThisEpoch: Set(), + // remember the key assignments that were applied to send the packets + keyAssignmentsForVSCPackets: providerState.keyAssignmentsForVSCPackets.put( + providerState.runningVscId, + providerState.validatorToConsumerAddr + ) + } + } + + // receives a given packet (sent by the provider) on the consumer. The arguments are the consumer chain that is receiving the packet, and the packet itself, + // as well as the unbonding period for the consumer chain. + // To receive a packet, modify the running validator set (not the one entered into the block yet, + // but the candidate that would be put into the block if it ended now) + // and store the maturation time for the packet. + pure def recvPacketOnConsumer( + currentState: ProtocolState, + receiver: Chain, + packet: VscPacket, + receiverUnbondingPeriod: Time): Result = { + if(not(isRunningConsumer(receiver, currentState.providerState))) { + Err("Receiver is not currently a consumer - must have 'running' status!") + } else { + // update the running validator set, but not the history yet, + // as that only happens when the next block is started + val currentConsumerState: ConsumerState = currentState.consumerStates.get(receiver) + val newConsumerState: ConsumerState = + { + ...currentConsumerState, + chainState: currentConsumerState.chainState.with( + "currentValidatorSet", packet.validatorSet + ), + maturationTimes: currentConsumerState.maturationTimes.append( + ( + packet, + currentConsumerState.chainState.runningTimestamp + receiverUnbondingPeriod + ) + ), + receivedVscPackets: currentConsumerState.receivedVscPackets.prepend(packet) + } + val newConsumerStates = currentState.consumerStates.set(receiver, newConsumerState) + val newState = currentState.with( + "consumerStates", newConsumerStates + ) + Ok(newState) + } + } + + // receives a given packet on the provider. The arguments are the consumer chain that sent the packet, and the packet itself. + // To receive a packet, add it to the list of received maturations. + pure def recvPacketOnProvider(currentState: ProtocolState, sender: Chain, packet: VscMaturedPacket): Result = { + if (not(isRunningConsumer(sender, currentState.providerState))) { + Err("Sender is not currently a consumer - must have 'running' status!") + } else if (currentState.providerState.sentVscPacketsToConsumer.get(sender).head().id != packet.id) { + // the packet is not the oldest sentVscPacket, something went wrong + Err("Received maturation is not for the oldest sentVscPacket") + } else { + val currentReceivedMaturations = currentState.providerState.receivedMaturations + val newReceivedMaturations = currentReceivedMaturations.union(Set(packet)) + val newProviderState = currentState.providerState.with( + "receivedMaturations", newReceivedMaturations + ) + // prune the sentVscPacket + val newSentVscPacket = currentState.providerState.sentVscPacketsToConsumer.get(sender).tail() + val newState = currentState.with( + "providerState", + {...newProviderState, + sentVscPacketsToConsumer: currentState.providerState.sentVscPacketsToConsumer.set(sender, newSentVscPacket) + } + ) + + // prune assigned keys that are no longer needed + + // get the keys that should be pruned + pure val consumerAddrsToPrune = + currentState.providerState.consumerAddrsToPrune + .getOrElse(sender, Map()) + .getOrElse(packet.id, List()).toSet() + + // actually do the pruning + pure val senderValByConsAddr = + // for the sender chain, get the consAddrToValidator mapping + newState.providerState.consumerAddrToValidator.getOrElse(sender, Map()) + pure val newSenderValByConsAddr = + senderValByConsAddr.keys().filter( + // filter out the assigned keys that should be pruned + consAddr => not(consumerAddrsToPrune.contains(consAddr)) + ).mapBy( + // rebuild the map with the pruned keys removed + consAddr => senderValByConsAddr.get(consAddr) + ) + + // update the provider state + pure val newProviderState2 = newState.providerState.with( + // update the consumerAddrToValidator map to remove the pruned keys + "consumerAddrToValidator", + newState.providerState.consumerAddrToValidator.put(sender, newSenderValByConsAddr) + ).with( + // delete the consumer addresses to prune, since they are pruned now + "consumerAddrsToPrune", + newState.providerState.consumerAddrsToPrune.put( + sender, + newState.providerState.consumerAddrsToPrune.getOrElse(sender, Map()).put( + packet.id, + List() + ) + ) + ) + + // update the state + pure val newState2 = newState.with( + "providerState", newProviderState2 + ) + + Ok(newState2) + } + } + + // removes the oldest outstanding packet from the consumer. on-chain, this would happen when the packet is acknowledged. + // only the oldest packet can be removed, since we model ordered channels. + pure def removeOutstandingPacketFromConsumer(currentState: ProtocolState, sender: Chain): ProtocolState = { + val currentOutstandingPackets = currentState.consumerStates.get(sender).outstandingPacketsToProvider + val newOutstandingPackets = currentOutstandingPackets.tail() + val newConsumerState = currentState.consumerStates.get(sender).with( + "outstandingPacketsToProvider", newOutstandingPackets + ) + val newConsumerStates = currentState.consumerStates.set(sender, newConsumerState) + val newState = currentState.with( + "consumerStates", newConsumerStates + ) + newState + } + + // removes the oldest outstanding packet (to the given consumer) from the provider. + // on-chain, this would happen when the packet is acknowledged. + // only the oldest packet can be removed, since we model ordered channels. + pure def removeOutstandingPacketFromProvider(currentState: ProtocolState, receiver: Chain): ProtocolState = { + val currentOutstandingPackets = currentState.providerState.outstandingPacketsToConsumer.get(receiver) + val newOutstandingPackets = currentOutstandingPackets.tail() + val newProviderState = currentState.providerState.with( + "outstandingPacketsToConsumer", + currentState.providerState.outstandingPacketsToConsumer.set(receiver, newOutstandingPackets) + ) + val newState = currentState.with( + "providerState", newProviderState + ) + newState + } + + // Returns a ProtocolState where the current validator set on the provider is set to + // newValidatorSet. + pure def setProviderValidatorSet(currentState: ProtocolState, newValidatorSet: ValidatorSet): ProtocolState = { + pure val newChainState = currentState.providerState.chainState.with( + "currentValidatorSet", newValidatorSet + ) + currentState.with( + "providerState", + currentState.providerState.with( + "chainState", newChainState + ) + ) + } + + // Returns true if the given chain is currently a running consumer, false otherwise. + pure def isRunningConsumer(chain: Chain, providerState: ProviderState): bool = { + val status = providerState.consumerStatus.get(chain) + status == RUNNING + } + + // Returns the set of all consumer chains. + pure def getConsumers(providerState: ProviderState): Set[Chain] = providerState.consumerStatus.keys() + + // Returns the set of all consumer chains that currently have the status RUNNING. + pure def getRunningConsumers(providerState: ProviderState): Set[Chain] = { + providerState.consumerStatus.keys().filter( + chain => providerState.consumerStatus.get(chain) == RUNNING + ) + } + + // Returns the set of all consumer chains that currently have the status NOT_CONSUMER. + pure def getNonConsumers(providerState: ProviderState): Set[Chain] = { + providerState.consumerStatus.keys().filter( + chain => providerState.consumerStatus.get(chain) == NOT_CONSUMER + ) + } + + // Returns whether the consumer has timed out due to the vscTimeout, and an error message. + // If the second return is not equal to "", the first return should be ignored. + // If it is equal to "", the first return will be true if the consumer has timed out and should be dropped, + // or false otherwise. + pure def TimeoutDueToVscTimeout(currentState: ProtocolState, consumer: Chain, vscTimeout: Time): (bool, str) = + // check for errors: the consumer is not running + if (not(isRunningConsumer(consumer, currentState.providerState))) { + (false, "Consumer is not currently a consumer - must have 'running' status!") + } else { + val providerState = currentState.providerState + val consumerState: ConsumerState = currentState.consumerStates.get(consumer) + + // has a packet been sent on the provider more than vscTimeout ago, but we have not received an answer since then? + val sentVscPacketsToConsumer = providerState.sentVscPacketsToConsumer.getOrElse(consumer, List()) + if(sentVscPacketsToConsumer.length() > 0) { + val oldestSentVscPacket = sentVscPacketsToConsumer.head() // if length is 0, this is undefined, but we check for this before we use it + if(oldestSentVscPacket.sendingTime + vscTimeout < providerState.chainState.runningTimestamp) { + (true, "") + } else { + // no timeout yet, it has not been vscTimeout since that packet was sent + (false, "") + } + } else { + // no packet has been sent yet, so no timeout + (false, "") + } + } +} \ No newline at end of file diff --git a/tests/mbt/run_invariants.sh b/tests/mbt/run_invariants.sh index 613664aeae..2d0b33bed9 100755 --- a/tests/mbt/run_invariants.sh +++ b/tests/mbt/run_invariants.sh @@ -1,5 +1,41 @@ #!/bin/bash +# to stop on any errors +set -e + quint test ccv_model.qnt quint test ccv_test.qnt -quint run --invariant "all{ValidatorUpdatesArePropagatedInv,ValidatorSetHasExistedInv,SameVscPacketsInv,MatureOnTimeInv,EventuallyMatureOnProviderInv}" ccv_model.qnt --max-steps 200 --max-samples 200 \ No newline at end of file +quint run --invariant "all{ValidatorUpdatesArePropagatedInv,ValidatorSetHasExistedInv,SameVscPacketsInv,MatureOnTimeInv,EventuallyMatureOnProviderInv}" ccv_model.qnt --max-steps 200 --max-samples 200 +quint run --invariant "all{ValidatorUpdatesArePropagatedKeyAssignmentInv,ValidatorSetHasExistedKeyAssignmentInv,SameVscPacketsKeyAssignmentInv,MatureOnTimeInv,EventuallyMatureOnProviderInv,KeyAssignmentRulesInv}" ccv_model.qnt --step stepKeyAssignment --max-steps 200 --max-samples 200 + + +# do not stop on errors anymore, so we can give better output if we error +set +e + +run_invariant() { + local invariant=$1 + local step=$2 + local match=$3 + + if [[ -z "$step" ]]; then + quint run --invariant $invariant ccv_model.qnt | grep -q $match + else + quint run --invariant $invariant --step $step ccv_model.qnt | grep -q $match + fi + + if [[ $? -eq 0 ]]; then + echo "sanity check $invariant ok" + else + echo "sanity check $invariant not ok" + exit 1 + fi +} + +run_invariant "CanRunConsumer" "" '[violation]' +run_invariant "CanStopConsumer" "" '[violation]' +run_invariant "CanTimeoutConsumer" "" '[violation]' +run_invariant "CanSendVscPackets" "" '[violation]' +run_invariant "CanSendVscMaturedPackets" "" '[violation]' +run_invariant "CanAssignConsumerKey" "stepKeyAssignment" '[violation]' +run_invariant "CanHaveConsumerAddresses" "stepKeyAssignment" '[violation]' +run_invariant "CanReceiveMaturations" "stepKeyAssignment" '[violation]' diff --git a/testutil/keeper/unit_test_helpers.go b/testutil/keeper/unit_test_helpers.go index dc901712ff..ff3df99763 100644 --- a/testutil/keeper/unit_test_helpers.go +++ b/testutil/keeper/unit_test_helpers.go @@ -252,7 +252,6 @@ func TestProviderStateIsCleanedAfterConsumerChainIsStopped(t *testing.T, ctx sdk // test key assignment state is cleaned require.Empty(t, providerKeeper.GetAllValidatorConsumerPubKeys(ctx, &expectedChainID)) require.Empty(t, providerKeeper.GetAllValidatorsByConsumerAddr(ctx, &expectedChainID)) - require.Empty(t, providerKeeper.GetAllKeyAssignmentReplacements(ctx, expectedChainID)) require.Empty(t, providerKeeper.GetAllConsumerAddrsToPrune(ctx, expectedChainID)) } diff --git a/x/ccv/consumer/types/keys.go b/x/ccv/consumer/types/keys.go index 0292ca84c5..20163f5ed9 100644 --- a/x/ccv/consumer/types/keys.go +++ b/x/ccv/consumer/types/keys.go @@ -51,7 +51,7 @@ const ( // received over CCV channel but not yet flushed over ABCI PendingChangesByteKey - // NOTE: This prefix is depreciated, but left in place to avoid consumer state migrations + // NOTE: This prefix is deprecated, but left in place to avoid consumer state migrations // [DEPRECATED] PendingDataPacketsByteKey @@ -61,7 +61,7 @@ const ( // InitialValSetByteKey is the byte to store the initial validator set for a consumer InitialValSetByteKey - // NOTE: This prefix is depreciated, but left in place to avoid consumer state migrations + // NOTE: This prefix is deprecated, but left in place to avoid consumer state migrations // [DEPRECATED] LastStandaloneHeightByteKey diff --git a/x/ccv/provider/keeper/grpc_query_test.go b/x/ccv/provider/keeper/grpc_query_test.go index dfe0a73895..e3273d1fb5 100644 --- a/x/ccv/provider/keeper/grpc_query_test.go +++ b/x/ccv/provider/keeper/grpc_query_test.go @@ -23,7 +23,6 @@ func TestQueryAllPairsValConAddrByConsumerChainID(t *testing.T) { defer ctrl.Finish() pk.SetValidatorConsumerPubKey(ctx, chainID, providerAddr, consumerKey) - pk.SetKeyAssignmentReplacement(ctx, chainID, providerAddr, consumerKey, 100) consumerPubKey, found := pk.GetValidatorConsumerPubKey(ctx, chainID, providerAddr) require.True(t, found, "consumer pubkey not found") diff --git a/x/ccv/provider/keeper/key_assignment.go b/x/ccv/provider/keeper/key_assignment.go index c54d922f0f..89dbcfda4e 100644 --- a/x/ccv/provider/keeper/key_assignment.go +++ b/x/ccv/provider/keeper/key_assignment.go @@ -8,7 +8,6 @@ import ( sdk "github.com/cosmos/cosmos-sdk/types" stakingtypes "github.com/cosmos/cosmos-sdk/x/staking/types" - abci "github.com/cometbft/cometbft/abci/types" tmprotocrypto "github.com/cometbft/cometbft/proto/tendermint/crypto" "github.com/cosmos/interchain-security/v4/x/ccv/provider/types" @@ -184,99 +183,6 @@ func (k Keeper) DeleteValidatorByConsumerAddr(ctx sdk.Context, chainID string, c store.Delete(types.ValidatorsByConsumerAddrKey(chainID, consumerAddr)) } -// GetKeyAssignmentReplacement returns the previous assigned consumer key and the current power -// for a provider validator for which a key assignment was received in this block. Both are -// needed to update the validator's power on the consumer chain at the end of the current block. -func (k Keeper) GetKeyAssignmentReplacement( - ctx sdk.Context, - chainID string, - providerAddr types.ProviderConsAddress, -) (prevCKey tmprotocrypto.PublicKey, power int64, found bool) { - var pubKeyAndPower abci.ValidatorUpdate - store := ctx.KVStore(k.storeKey) - bz := store.Get(types.KeyAssignmentReplacementsKey(chainID, providerAddr)) - if bz == nil { - return pubKeyAndPower.PubKey, pubKeyAndPower.Power, false - } - - err := pubKeyAndPower.Unmarshal(bz) - if err != nil { - // An error here would indicate something is very wrong, - // the public key and power are assumed to be correctly serialized in SetKeyAssignmentReplacement. - panic(fmt.Sprintf("failed to unmarshal public key and power: %v", err)) - } - return pubKeyAndPower.PubKey, pubKeyAndPower.Power, true -} - -// SetKeyAssignmentReplacement sets the previous assigned consumer key and the current power -// for a provider validator for which a key assignment was received in this block. Both are -// needed to update the validator's power on the consumer chain at the end of the current block. -func (k Keeper) SetKeyAssignmentReplacement( - ctx sdk.Context, - chainID string, - providerAddr types.ProviderConsAddress, - prevCKey tmprotocrypto.PublicKey, - power int64, -) { - store := ctx.KVStore(k.storeKey) - pubKeyAndPower := abci.ValidatorUpdate{PubKey: prevCKey, Power: power} - bz, err := pubKeyAndPower.Marshal() - if err != nil { - // An error here would indicate something is very wrong, - // prevCKey is obtained from GetValidatorConsumerPubKey (called from AssignConsumerKey), - // and power is obtained from GetLastValidatorPower (called from AssignConsumerKey). - // Both of which are assumed to return valid values. - panic(fmt.Sprintf("failed to marshal public key and power: %v", err)) - } - store.Set(types.KeyAssignmentReplacementsKey(chainID, providerAddr), bz) -} - -// GetAllKeyAssignmentReplacements gets all pairs of previous assigned consumer keys -// and current powers for all provider validator for which key assignments were received in this block. -// -// Note that the pairs are stored under keys with the following format: -// KeyAssignmentReplacementsBytePrefix | len(chainID) | chainID | providerAddress -// Thus, the iteration is in ascending order of providerAddresses. -func (k Keeper) GetAllKeyAssignmentReplacements(ctx sdk.Context, chainID string) (replacements []types.KeyAssignmentReplacement) { - store := ctx.KVStore(k.storeKey) - iteratorPrefix := types.ChainIdWithLenKey(types.KeyAssignmentReplacementsBytePrefix, chainID) - iterator := sdk.KVStorePrefixIterator(store, iteratorPrefix) - defer iterator.Close() - for ; iterator.Valid(); iterator.Next() { - // TODO: store chainID and provider cons address in value bytes, marshaled as protobuf type - _, providerAddrTmp, err := types.ParseChainIdAndConsAddrKey(types.KeyAssignmentReplacementsBytePrefix, iterator.Key()) - if err != nil { - // An error here would indicate something is very wrong, - // store keys are assumed to be correctly serialized in SetKeyAssignmentReplacement. - panic(err) - } - providerAddr := types.NewProviderConsAddress(providerAddrTmp) - var pubKeyAndPower abci.ValidatorUpdate - err = pubKeyAndPower.Unmarshal(iterator.Value()) - if err != nil { - // An error here would indicate something is very wrong, - // the public key and power are assumed to be correctly serialized in SetKeyAssignmentReplacement. - panic(fmt.Sprintf("failed to unmarshal public key and power: %v", err)) - } - - replacements = append(replacements, types.KeyAssignmentReplacement{ - ProviderAddr: providerAddr.ToSdkConsAddr(), - PrevCKey: &pubKeyAndPower.PubKey, - Power: pubKeyAndPower.Power, - }) - } - - return replacements -} - -// DeleteKeyAssignmentReplacement deletes the previous assigned consumer key and the current power -// for a provider validator for which a key assignment was received in this block. Both are -// needed to update the validator's power on the consumer chain at the end of the current block. -func (k Keeper) DeleteKeyAssignmentReplacement(ctx sdk.Context, chainID string, providerAddr types.ProviderConsAddress) { - store := ctx.KVStore(k.storeKey) - store.Delete(types.KeyAssignmentReplacementsKey(chainID, providerAddr)) -} - // AppendConsumerAddrsToPrune appends a consumer validator address to the list of consumer addresses // that can be pruned once the VSCMaturedPacket with vscID is received. // @@ -425,20 +331,20 @@ func (k Keeper) AssignConsumerKey( ) } - // check whether the consumer chain is already registered, - // i.e., a client to the consumer was already created - if _, consumerRegistered := k.GetConsumerClientId(ctx, chainID); consumerRegistered { - // get the previous key assigned for this validator on this consumer chain - oldConsumerKey, found := k.GetValidatorConsumerPubKey(ctx, chainID, providerAddr) - if found { - // mark this old consumer key as prunable once the VSCMaturedPacket + // get the previous key assigned for this validator on this consumer chain + if oldConsumerKey, found := k.GetValidatorConsumerPubKey(ctx, chainID, providerAddr); found { + oldConsumerAddrTmp, err := ccvtypes.TMCryptoPublicKeyToConsAddr(oldConsumerKey) + if err != nil { + return err + } + oldConsumerAddr := types.NewConsumerConsAddress(oldConsumerAddrTmp) + + // check whether the consumer chain is already registered, + // i.e., a client to the consumer was already created + if _, consumerRegistered := k.GetConsumerClientId(ctx, chainID); consumerRegistered { + // mark the old consumer address as prunable once the VSCMaturedPacket // for the current VSC ID is received; // note: this state is removed on receiving the VSCMaturedPacket - oldConsumerAddrTmp, err := ccvtypes.TMCryptoPublicKeyToConsAddr(oldConsumerKey) - if err != nil { - return err - } - oldConsumerAddr := types.NewConsumerConsAddress(oldConsumerAddrTmp) k.AppendConsumerAddrsToPrune( ctx, chainID, @@ -446,42 +352,8 @@ func (k Keeper) AssignConsumerKey( oldConsumerAddr, ) } else { - // the validator had no key assigned on this consumer chain - providerKey, err := validator.TmConsPublicKey() - if err != nil { - return err - } - oldConsumerKey = providerKey - } - - // check whether the validator is valid, i.e., its power is positive - power := k.stakingKeeper.GetLastValidatorPower(ctx, validator.GetOperator()) - if 0 < power { - // to enable multiple calls of AssignConsumerKey in the same block by the same validator - - // the key assignment replacement should not be overwritten - if _, _, found := k.GetKeyAssignmentReplacement(ctx, chainID, providerAddr); !found { - // store old key and current power for modifying the valset update in EndBlock; - // note: this state is deleted at the end of the block - k.SetKeyAssignmentReplacement( - ctx, - chainID, - providerAddr, - oldConsumerKey, - power, - ) - } - } - } else { - // if the consumer chain is not registered, then remove the mapping - // from the old consumer address to the provider address (if any) - // get the previous key assigned for this validator on this consumer chain - if oldConsumerKey, found := k.GetValidatorConsumerPubKey(ctx, chainID, providerAddr); found { - oldConsumerAddrTmp, err := ccvtypes.TMCryptoPublicKeyToConsAddr(oldConsumerKey) - if err != nil { - return err - } - oldConsumerAddr := types.NewConsumerConsAddress(oldConsumerAddrTmp) + // if the consumer chain is not registered, then remove the mapping + // from the old consumer address to the provider address k.DeleteValidatorByConsumerAddr(ctx, chainID, oldConsumerAddr) } } @@ -499,88 +371,6 @@ func (k Keeper) AssignConsumerKey( return nil } -// MustApplyKeyAssignmentToValUpdates applies the key assignment to the validator updates -// received from the staking module. -// The method panics if the key-assignment state is corrupted. -func (k Keeper) MustApplyKeyAssignmentToValUpdates( - ctx sdk.Context, - chainID string, - valUpdates []abci.ValidatorUpdate, -) (newUpdates []abci.ValidatorUpdate) { - for _, valUpdate := range valUpdates { - providerAddrTmp, err := ccvtypes.TMCryptoPublicKeyToConsAddr(valUpdate.PubKey) - if err != nil { - panic(fmt.Errorf("cannot get provider address from pub key: %s", err.Error())) - } - providerAddr := types.NewProviderConsAddress(providerAddrTmp) - - // If a key assignment replacement is found, we remove the valupdate with the old consumer key, - // create two new valupdates, - // - setting the old consumer key's power to 0 - // - and setting the new consumer key's power to the power in the update - prevConsumerKey, _, found := k.GetKeyAssignmentReplacement(ctx, chainID, providerAddr) - if found { - newUpdates = append(newUpdates, abci.ValidatorUpdate{ - PubKey: prevConsumerKey, - Power: 0, - }) - - newConsumerKey, found := k.GetValidatorConsumerPubKey(ctx, chainID, providerAddr) - if !found { - // This should never happen as for every KeyAssignmentReplacement there should - // be a ValidatorConsumerPubKey that was stored when AssignConsumerKey() was called. - panic(fmt.Errorf("consumer key not found for provider addr %s stored in KeyAssignmentReplacement", providerAddr)) - } - newUpdates = append(newUpdates, abci.ValidatorUpdate{ - PubKey: newConsumerKey, - Power: valUpdate.Power, - }) - k.DeleteKeyAssignmentReplacement(ctx, chainID, providerAddr) - } else { - // If a key assignment replacement is not found, we check if the validator's key is assigned. - // If it is, we replace the update containing the provider key with an update containing - // the consumer key. - // Note that this will always be the branch taken when creating the genesis state - // of a newly registered consumer chain. - consumerKey, found := k.GetValidatorConsumerPubKey(ctx, chainID, providerAddr) - if found { - newUpdates = append(newUpdates, abci.ValidatorUpdate{ - PubKey: consumerKey, - Power: valUpdate.Power, - }) - } else { - // keep the same update - newUpdates = append(newUpdates, valUpdate) - } - } - } - - // For any key assignment replacements that did not have a corresponding validator update already, - // set the old consumer key's power to 0 and the new consumer key's power to the - // power in the pending key assignment. - for _, replacement := range k.GetAllKeyAssignmentReplacements(ctx, chainID) { - providerAddr := types.NewProviderConsAddress(replacement.ProviderAddr) - k.DeleteKeyAssignmentReplacement(ctx, chainID, providerAddr) - newUpdates = append(newUpdates, abci.ValidatorUpdate{ - PubKey: *replacement.PrevCKey, - Power: 0, - }) - - newConsumerKey, found := k.GetValidatorConsumerPubKey(ctx, chainID, providerAddr) - if !found { - // This should never happen as for every KeyAssignmentReplacement there should - // be a ValidatorConsumerPubKey that was stored when AssignConsumerKey() was called. - panic(fmt.Errorf("consumer key not found for provider addr %s stored in KeyAssignmentReplacement", replacement.ProviderAddr)) - } - newUpdates = append(newUpdates, abci.ValidatorUpdate{ - PubKey: newConsumerKey, - Power: replacement.Power, - }) - } - - return newUpdates -} - // GetProviderAddrFromConsumerAddr returns the consensus address of a validator with // consAddr set as the consensus address on a consumer chain func (k Keeper) GetProviderAddrFromConsumerAddr( @@ -627,12 +417,6 @@ func (k Keeper) DeleteKeyAssignments(ctx sdk.Context, chainID string) { k.DeleteValidatorByConsumerAddr(ctx, chainID, consumerAddr) } - // delete KeyAssignmentReplacements - for _, keyAssignmentReplacement := range k.GetAllKeyAssignmentReplacements(ctx, chainID) { - providerAddr := types.NewProviderConsAddress(keyAssignmentReplacement.ProviderAddr) - k.DeleteKeyAssignmentReplacement(ctx, chainID, providerAddr) - } - // delete ValidatorConsumerPubKey for _, consumerAddrsToPrune := range k.GetAllConsumerAddrsToPrune(ctx, chainID) { k.DeleteConsumerAddrsToPrune(ctx, chainID, consumerAddrsToPrune.VscId) diff --git a/x/ccv/provider/keeper/key_assignment_test.go b/x/ccv/provider/keeper/key_assignment_test.go index 4fab08c981..7cb222a3be 100644 --- a/x/ccv/provider/keeper/key_assignment_test.go +++ b/x/ccv/provider/keeper/key_assignment_test.go @@ -2,6 +2,7 @@ package keeper_test import ( "bytes" + codectypes "github.com/cosmos/cosmos-sdk/codec/types" "math/rand" "sort" "testing" @@ -184,67 +185,6 @@ func TestGetAllValidatorsByConsumerAddr(t *testing.T) { require.Len(t, result, len(testAssignments)) } -func TestKeyAssignmentReplacementCRUD(t *testing.T) { - chainID := consumer - providerAddr := types.NewProviderConsAddress([]byte("providerAddr")) - expCPubKey := cryptotestutil.NewCryptoIdentityFromIntSeed(1).TMProtoCryptoPublicKey() - var expPower int64 = 100 - - keeper, ctx, ctrl, _ := testkeeper.GetProviderKeeperAndCtx(t, testkeeper.NewInMemKeeperParams(t)) - defer ctrl.Finish() - - keeper.SetKeyAssignmentReplacement(ctx, chainID, providerAddr, expCPubKey, expPower) - - cPubKey, power, found := keeper.GetKeyAssignmentReplacement(ctx, chainID, providerAddr) - require.True(t, found, "key assignment replacement not found") - require.Equal(t, expCPubKey, cPubKey, "previous consumer key not matching") - require.Equal(t, expPower, power, "power not matching") - - keeper.DeleteKeyAssignmentReplacement(ctx, chainID, providerAddr) - _, _, found = keeper.GetKeyAssignmentReplacement(ctx, chainID, providerAddr) - require.False(t, found, "key assignment replacement found") -} - -func TestGetAllKeyAssignmentReplacements(t *testing.T) { - pk, ctx, ctrl, _ := testkeeper.GetProviderKeeperAndCtx(t, testkeeper.NewInMemKeeperParams(t)) - defer ctrl.Finish() - - chainID := "consumer-1" - - seed := time.Now().UnixNano() - rng := rand.New(rand.NewSource(seed)) - - numAssignments := 10 - testAssignments := []types.KeyAssignmentReplacement{} - for i := 0; i < numAssignments; i++ { - consumerKey := cryptotestutil.NewCryptoIdentityFromIntSeed(i).TMProtoCryptoPublicKey() - providerAddr := cryptotestutil.NewCryptoIdentityFromIntSeed(numAssignments + i).ProviderConsAddress() - testAssignments = append(testAssignments, - types.KeyAssignmentReplacement{ - ProviderAddr: providerAddr.ToSdkConsAddr(), - PrevCKey: &consumerKey, - Power: rng.Int63(), - }, - ) - } - expectedGetAllOrder := testAssignments - // sorting by KeyAssignmentReplacement.ProviderAddr - sort.Slice(expectedGetAllOrder, func(i, j int) bool { - return bytes.Compare(expectedGetAllOrder[i].ProviderAddr, expectedGetAllOrder[j].ProviderAddr) == -1 - }) - - firstTestAssignmentProviderAddr := types.NewProviderConsAddress(testAssignments[0].ProviderAddr) - pk.SetKeyAssignmentReplacement(ctx, "consumer-2", firstTestAssignmentProviderAddr, *testAssignments[0].PrevCKey, testAssignments[0].Power) - for _, assignment := range testAssignments { - providerAddr := types.NewProviderConsAddress(assignment.ProviderAddr) - pk.SetKeyAssignmentReplacement(ctx, chainID, providerAddr, *assignment.PrevCKey, assignment.Power) - } - - result := pk.GetAllKeyAssignmentReplacements(ctx, chainID) - require.Len(t, result, len(testAssignments)) - require.Equal(t, expectedGetAllOrder, result) -} - func TestConsumerAddrsToPruneCRUD(t *testing.T) { chainID := consumer consumerAddr := types.NewConsumerConsAddress([]byte("consumerAddr1")) @@ -420,9 +360,6 @@ func TestAssignConsensusKeyForConsumerChain(t *testing.T) { mocks.MockStakingKeeper.EXPECT().GetValidatorByConsAddr(ctx, consumerIdentities[0].SDKValConsAddress(), ).Return(stakingtypes.Validator{}, false), - mocks.MockStakingKeeper.EXPECT().GetLastValidatorPower( - ctx, providerIdentities[0].SDKValOpAddress(), - ).Return(int64(0)), ) }, doActions: func(ctx sdk.Context, k providerkeeper.Keeper) { @@ -445,15 +382,9 @@ func TestAssignConsensusKeyForConsumerChain(t *testing.T) { mocks.MockStakingKeeper.EXPECT().GetValidatorByConsAddr(ctx, consumerIdentities[0].SDKValConsAddress(), ).Return(stakingtypes.Validator{}, false), - mocks.MockStakingKeeper.EXPECT().GetLastValidatorPower( - ctx, providerIdentities[0].SDKValOpAddress(), - ).Return(int64(0)), mocks.MockStakingKeeper.EXPECT().GetValidatorByConsAddr(ctx, consumerIdentities[1].SDKValConsAddress(), ).Return(stakingtypes.Validator{}, false), - mocks.MockStakingKeeper.EXPECT().GetLastValidatorPower( - ctx, providerIdentities[0].SDKValOpAddress(), - ).Return(int64(0)), ) }, doActions: func(ctx sdk.Context, k providerkeeper.Keeper) { @@ -481,9 +412,6 @@ func TestAssignConsensusKeyForConsumerChain(t *testing.T) { mocks.MockStakingKeeper.EXPECT().GetValidatorByConsAddr(ctx, consumerIdentities[0].SDKValConsAddress(), ).Return(stakingtypes.Validator{}, false), - mocks.MockStakingKeeper.EXPECT().GetLastValidatorPower( - ctx, providerIdentities[0].SDKValOpAddress(), - ).Return(int64(0)), mocks.MockStakingKeeper.EXPECT().GetValidatorByConsAddr(ctx, consumerIdentities[0].SDKValConsAddress(), ).Return(stakingtypes.Validator{}, false), @@ -692,7 +620,6 @@ func (vs *ValSet) apply(updates []abci.ValidatorUpdate) { // note: an insertion index should always be found for _, u := range updates { for i, id := range vs.identities { // n2 looping but n is tiny - // cons := sdk.ConsAddress(utils.GetChangePubKeyAddress(u)) cons, _ := ccvtypes.TMCryptoPublicKeyToConsAddr(u.PubKey) if id.SDKValConsAddress().Equals(cons) { vs.power[i] = u.Power @@ -828,7 +755,21 @@ func TestSimulatedAssignmentsAndUpdateApplication(t *testing.T) { // and increment the provider vscid. applyUpdatesAndIncrementVSCID := func(updates []abci.ValidatorUpdate) { providerValset.apply(updates) - updates = k.MustApplyKeyAssignmentToValUpdates(ctx, CHAINID, updates) + + var bondedValidators []stakingtypes.Validator + for _, v := range providerValset.identities { + pkAny, _ := codectypes.NewAnyWithValue(v.ConsensusSDKPubKey()) + + bondedValidators = append(bondedValidators, stakingtypes.Validator{ + OperatorAddress: v.SDKValOpAddress().String(), + ConsensusPubkey: pkAny, + }) + } + + nextValidators := k.ComputeNextEpochConsumerValSet(ctx, CHAINID, bondedValidators) + updates = providerkeeper.DiffValidators(k.GetConsumerValSet(ctx, CHAINID), nextValidators) + k.SetConsumerValSet(ctx, CHAINID, nextValidators) + consumerValset.apply(updates) // Simulate the VSCID update in EndBlock k.IncrementValidatorSetUpdateId(ctx) @@ -844,10 +785,13 @@ func TestSimulatedAssignmentsAndUpdateApplication(t *testing.T) { // The consumer chain has not yet been registered // Apply some randomly generated key assignments - applyAssignments(getAssignments()) + assignments := getAssignments() + applyAssignments(assignments) // And generate a random provider valset which, in the real system, will // be put into the consumer genesis. - applyUpdatesAndIncrementVSCID(getStakingUpdates()) + stakingUpdates := getStakingUpdates() + + applyUpdatesAndIncrementVSCID(stakingUpdates) // Register the consumer chain k.SetConsumerClientId(ctx, CHAINID, "") @@ -861,9 +805,12 @@ func TestSimulatedAssignmentsAndUpdateApplication(t *testing.T) { // and a random set of validator power updates for block := 0; block < NUM_BLOCKS_PER_EXECUTION; block++ { + stakingUpdates = getStakingUpdates() + assignments = getAssignments() + // Generate and apply assignments and power updates - applyAssignments(getAssignments()) - applyUpdatesAndIncrementVSCID(getStakingUpdates()) + applyAssignments(assignments) + applyUpdatesAndIncrementVSCID(stakingUpdates) // Randomly fast forward the greatest pruned VSCID. This simulates // delivery of maturity packets from the consumer chain. diff --git a/x/ccv/provider/keeper/params.go b/x/ccv/provider/keeper/params.go index 209d0f0ddb..f74baf656e 100644 --- a/x/ccv/provider/keeper/params.go +++ b/x/ccv/provider/keeper/params.go @@ -78,6 +78,13 @@ func (k Keeper) GetConsumerRewardDenomRegistrationFee(ctx sdk.Context) sdk.Coin return c } +// GetBlocksPerEpoch returns the number of blocks that constitute an epoch +func (k Keeper) GetBlocksPerEpoch(ctx sdk.Context) int64 { + var b int64 + k.paramSpace.Get(ctx, types.KeyBlocksPerEpoch, &b) + return b +} + // GetParams returns the paramset for the provider module func (k Keeper) GetParams(ctx sdk.Context) types.Params { return types.NewParams( @@ -89,6 +96,7 @@ func (k Keeper) GetParams(ctx sdk.Context) types.Params { k.GetSlashMeterReplenishPeriod(ctx), k.GetSlashMeterReplenishFraction(ctx), k.GetConsumerRewardDenomRegistrationFee(ctx), + k.GetBlocksPerEpoch(ctx), ) } diff --git a/x/ccv/provider/keeper/params_test.go b/x/ccv/provider/keeper/params_test.go index a941523e87..88175431c0 100644 --- a/x/ccv/provider/keeper/params_test.go +++ b/x/ccv/provider/keeper/params_test.go @@ -48,6 +48,7 @@ func TestParams(t *testing.T) { Denom: "stake", Amount: sdk.NewInt(10000000), }, + 600, ) providerKeeper.SetParams(ctx, newParams) params = providerKeeper.GetParams(ctx) diff --git a/x/ccv/provider/keeper/proposal.go b/x/ccv/provider/keeper/proposal.go index 89d71ff344..b0e2845502 100644 --- a/x/ccv/provider/keeper/proposal.go +++ b/x/ccv/provider/keeper/proposal.go @@ -253,6 +253,8 @@ func (k Keeper) MakeConsumerGenesis( return false }) + var bondedValidators []stakingtypes.Validator + initialUpdates := []abci.ValidatorUpdate{} for _, p := range lastPowers { addr, err := sdk.ValAddressFromBech32(p.Address) @@ -274,10 +276,16 @@ func (k Keeper) MakeConsumerGenesis( PubKey: tmProtoPk, Power: p.Power, }) + + // gather all the bonded validators in order to construct the consumer validator set for consumer chain `chainID` + bondedValidators = append(bondedValidators, val) } - // Apply key assignments to the initial valset. - initialUpdatesWithConsumerKeys := k.MustApplyKeyAssignmentToValUpdates(ctx, chainID, initialUpdates) + nextValidators := k.ComputeNextEpochConsumerValSet(ctx, chainID, bondedValidators) + k.SetConsumerValSet(ctx, chainID, nextValidators) + + // get the initial updates with the latest set consumer public keys + initialUpdatesWithConsumerKeys := DiffValidators([]types.ConsumerValidator{}, nextValidators) // Get a hash of the consumer validator set from the update with applied consumer assigned keys updatesAsValSet, err := tmtypes.PB2TM.ValidatorUpdates(initialUpdatesWithConsumerKeys) diff --git a/x/ccv/provider/keeper/proposal_test.go b/x/ccv/provider/keeper/proposal_test.go index e78823899a..fc1c7a4344 100644 --- a/x/ccv/provider/keeper/proposal_test.go +++ b/x/ccv/provider/keeper/proposal_test.go @@ -775,6 +775,7 @@ func TestMakeConsumerGenesis(t *testing.T) { Denom: "stake", Amount: sdk.NewInt(1000000), }, + BlocksPerEpoch: 600, } providerKeeper.SetParams(ctx, moduleParams) defer ctrl.Finish() diff --git a/x/ccv/provider/keeper/relay.go b/x/ccv/provider/keeper/relay.go index 59fec69534..939f6d3995 100644 --- a/x/ccv/provider/keeper/relay.go +++ b/x/ccv/provider/keeper/relay.go @@ -148,13 +148,17 @@ func (k Keeper) EndBlockVSU(ctx sdk.Context) { // notify the staking module to complete all matured unbonding ops k.completeMaturedUnbondingOps(ctx) - // collect validator updates - k.QueueVSCPackets(ctx) + if ctx.BlockHeight()%k.GetBlocksPerEpoch(ctx) == 0 { + // only queue and send VSCPackets at the boundaries of an epoch - // try sending VSC packets to all registered consumer chains; - // if the CCV channel is not established for a consumer chain, - // the updates will remain queued until the channel is established - k.SendVSCPackets(ctx) + // collect validator updates + k.QueueVSCPackets(ctx) + + // try sending VSC packets to all registered consumer chains; + // if the CCV channel is not established for a consumer chain, + // the updates will remain queued until the channel is established + k.SendVSCPackets(ctx) + } } // SendVSCPackets iterates over all registered consumers and sends pending @@ -212,14 +216,15 @@ func (k Keeper) SendVSCPacketsToChain(ctx sdk.Context, chainID, channelID string // QueueVSCPackets queues latest validator updates for every registered consumer chain func (k Keeper) QueueVSCPackets(ctx sdk.Context) { valUpdateID := k.GetValidatorSetUpdateId(ctx) // current valset update ID - // Get the validator updates from the staking module. - // Note: GetValidatorUpdates panics if the updates provided by the x/staking module - // of cosmos-sdk is invalid. - stakingValUpdates := k.stakingKeeper.GetValidatorUpdates(ctx) + + // get the bonded validators from the staking module + bondedValidators := k.stakingKeeper.GetLastValidators(ctx) for _, chain := range k.GetAllConsumerChains(ctx) { - // Apply the key assignment to the validator updates. - valUpdates := k.MustApplyKeyAssignmentToValUpdates(ctx, chain.ChainId, stakingValUpdates) + currentValidators := k.GetConsumerValSet(ctx, chain.ChainId) + nextValidators := k.ComputeNextEpochConsumerValSet(ctx, chain.ChainId, bondedValidators) + valUpdates := DiffValidators(currentValidators, nextValidators) + k.SetConsumerValSet(ctx, chain.ChainId, nextValidators) // check whether there are changes in the validator set; // note that this also entails unbonding operations diff --git a/x/ccv/provider/keeper/relay_test.go b/x/ccv/provider/keeper/relay_test.go index 02df262d53..b7e89b3dc2 100644 --- a/x/ccv/provider/keeper/relay_test.go +++ b/x/ccv/provider/keeper/relay_test.go @@ -1,23 +1,24 @@ package keeper_test import ( + cryptocodec "github.com/cosmos/cosmos-sdk/crypto/codec" + ibctesting "github.com/cosmos/ibc-go/v7/testing" "strings" "testing" clienttypes "github.com/cosmos/ibc-go/v7/modules/core/02-client/types" channeltypes "github.com/cosmos/ibc-go/v7/modules/core/04-channel/types" - ibctesting "github.com/cosmos/ibc-go/v7/testing" "github.com/golang/mock/gomock" "github.com/stretchr/testify/require" "cosmossdk.io/math" - cryptocodec "github.com/cosmos/cosmos-sdk/crypto/codec" sdk "github.com/cosmos/cosmos-sdk/types" stakingtypes "github.com/cosmos/cosmos-sdk/x/staking/types" abci "github.com/cometbft/cometbft/abci/types" + "github.com/cosmos/interchain-security/v4/testutil/crypto" cryptotestutil "github.com/cosmos/interchain-security/v4/testutil/crypto" testkeeper "github.com/cosmos/interchain-security/v4/testutil/keeper" "github.com/cosmos/interchain-security/v4/x/ccv/provider/keeper" @@ -66,16 +67,7 @@ func TestQueueVSCPackets(t *testing.T) { ctrl := gomock.NewController(t) defer ctrl.Finish() mocks := testkeeper.NewMockedKeepers(ctrl) - mockStakingKeeper := mocks.MockStakingKeeper - - mockUpdates := []abci.ValidatorUpdate{} - if len(tc.packets) != 0 { - mockUpdates = tc.packets[0].ValidatorUpdates - } - - gomock.InOrder( - mockStakingKeeper.EXPECT().GetValidatorUpdates(gomock.Eq(ctx)).Return(mockUpdates), - ) + mocks.MockStakingKeeper.EXPECT().GetLastValidators(ctx).Times(1) pk := testkeeper.NewInMemProviderKeeper(keeperParams, mocks) // no-op if tc.packets is empty @@ -661,3 +653,47 @@ func TestOnAcknowledgementPacketWithAckError(t *testing.T) { testkeeper.TestProviderStateIsCleanedAfterConsumerChainIsStopped(t, ctx, providerKeeper, "chainID", "channelID") require.NoError(t, err) } + +// TestEndBlockVSU tests that during `EndBlockVSU`, we only queue VSC packets at the boundaries of an epoch +func TestEndBlockVSU(t *testing.T) { + providerKeeper, ctx, ctrl, mocks := testkeeper.GetProviderKeeperAndCtx(t, testkeeper.NewInMemKeeperParams(t)) + defer ctrl.Finish() + + // 10 blocks constitute an epoch + params := providertypes.DefaultParams() + params.BlocksPerEpoch = 10 + providerKeeper.SetParams(ctx, params) + + // create 4 sample lastValidators + var lastValidators []stakingtypes.Validator + for i := 0; i < 4; i++ { + lastValidators = append(lastValidators, crypto.NewCryptoIdentityFromIntSeed(i).SDKStakingValidator()) + } + mocks.MockStakingKeeper.EXPECT().GetLastValidators(gomock.Any()).Return(lastValidators).AnyTimes() + mocks.MockStakingKeeper.EXPECT().GetLastValidatorPower(gomock.Any(), gomock.Any()).Return(int64(2)).AnyTimes() + + // set a sample client for a consumer chain so that `GetAllConsumerChains` in `QueueVSCPackets` iterates at least once + providerKeeper.SetConsumerClientId(ctx, "chainID", "clientID") + + // with block height of 1 we do not expect any queueing of VSC packets + ctx = ctx.WithBlockHeight(1) + providerKeeper.EndBlockVSU(ctx) + require.Equal(t, 0, len(providerKeeper.GetPendingVSCPackets(ctx, "chainID"))) + + // with block height of 5 we do not expect any queueing of VSC packets + ctx = ctx.WithBlockHeight(5) + providerKeeper.EndBlockVSU(ctx) + require.Equal(t, 0, len(providerKeeper.GetPendingVSCPackets(ctx, "chainID"))) + + // with block height of 10 we expect the queueing of one VSC packet + ctx = ctx.WithBlockHeight(10) + providerKeeper.EndBlockVSU(ctx) + require.Equal(t, 1, len(providerKeeper.GetPendingVSCPackets(ctx, "chainID"))) + + // With block height of 15 we expect no additional queueing of a VSC packet. + // Note that the pending VSC packet is still there because `SendVSCPackets` does not send the packet. We + // need to mock channels, etc. for this to work, and it's out of scope for this test. + ctx = ctx.WithBlockHeight(15) + providerKeeper.EndBlockVSU(ctx) + require.Equal(t, 1, len(providerKeeper.GetPendingVSCPackets(ctx, "chainID"))) +} diff --git a/x/ccv/provider/keeper/validator_set_update.go b/x/ccv/provider/keeper/validator_set_update.go new file mode 100644 index 0000000000..71238d210d --- /dev/null +++ b/x/ccv/provider/keeper/validator_set_update.go @@ -0,0 +1,178 @@ +package keeper + +import ( + "fmt" + abci "github.com/cometbft/cometbft/abci/types" + sdk "github.com/cosmos/cosmos-sdk/types" + stakingtypes "github.com/cosmos/cosmos-sdk/x/staking/types" + "github.com/cosmos/interchain-security/v4/x/ccv/provider/types" +) + +// SetConsumerValidator sets provided consumer `validator` on the consumer chain with `chainID` +func (k Keeper) SetConsumerValidator( + ctx sdk.Context, + chainID string, + validator types.ConsumerValidator, +) { + store := ctx.KVStore(k.storeKey) + bz, err := validator.Marshal() + if err != nil { + panic(fmt.Errorf("failed to marshal ConsumerValidator: %w", err)) + } + + store.Set(types.ConsumerValidatorKey(chainID, validator.ProviderConsAddr), bz) +} + +// DeleteConsumerValidator removes consumer validator with `providerAddr` address +func (k Keeper) DeleteConsumerValidator( + ctx sdk.Context, + chainID string, + providerConsAddr types.ProviderConsAddress, +) { + store := ctx.KVStore(k.storeKey) + store.Delete(types.ConsumerValidatorKey(chainID, providerConsAddr.ToSdkConsAddr())) +} + +// DeleteConsumerValSet deletes all the stored consumer validators for chain `chainID` +func (k Keeper) DeleteConsumerValSet( + ctx sdk.Context, + chainID string) { + store := ctx.KVStore(k.storeKey) + key := types.ChainIdWithLenKey(types.ConsumerValidatorBytePrefix, chainID) + iterator := sdk.KVStorePrefixIterator(store, key) + + var keysToDel [][]byte + defer iterator.Close() + for ; iterator.Valid(); iterator.Next() { + keysToDel = append(keysToDel, iterator.Key()) + } + for _, delKey := range keysToDel { + store.Delete(delKey) + } +} + +// IsConsumerValidator returns `true` if the consumer validator with `providerAddr` exists for chain `chainID` +// and `false` otherwise +func (k Keeper) IsConsumerValidator( + ctx sdk.Context, + chainID string, + providerAddr types.ProviderConsAddress, +) bool { + store := ctx.KVStore(k.storeKey) + return store.Get(types.ConsumerValidatorKey(chainID, providerAddr.ToSdkConsAddr())) != nil +} + +// GetConsumerValSet returns all the consumer validators for chain `chainID` +func (k Keeper) GetConsumerValSet( + ctx sdk.Context, + chainID string) (validators []types.ConsumerValidator) { + store := ctx.KVStore(k.storeKey) + key := types.ChainIdWithLenKey(types.ConsumerValidatorBytePrefix, chainID) + iterator := sdk.KVStorePrefixIterator(store, key) + defer iterator.Close() + + for ; iterator.Valid(); iterator.Next() { + iterator.Value() + var validator types.ConsumerValidator + if err := validator.Unmarshal(iterator.Value()); err != nil { + panic(fmt.Errorf("failed to unmarshal ConsumerValidator: %w", err)) + } + validators = append(validators, validator) + } + + return validators +} + +// ComputeNextEpochConsumerValSet returns the next validator set that is responsible for validating consumer +// chain `chainID`, based on the bonded validators. +func (k Keeper) ComputeNextEpochConsumerValSet( + ctx sdk.Context, + chainID string, + bondedValidators []stakingtypes.Validator, +) []types.ConsumerValidator { + var nextValidators []types.ConsumerValidator + for _, val := range bondedValidators { + // get next voting power and the next consumer public key + nextPower := k.stakingKeeper.GetLastValidatorPower(ctx, val.GetOperator()) + consAddr, err := val.GetConsAddr() + if err != nil { + // this should never happen but is recoverable if we exclude this validator from the `nextValidators` + k.Logger(ctx).Error("could not get consensus address of validator", + "validator", val.GetOperator().String(), + "error", err) + continue + } + nextConsumerPublicKey, foundConsumerPublicKey := k.GetValidatorConsumerPubKey(ctx, chainID, types.NewProviderConsAddress(consAddr)) + if !foundConsumerPublicKey { + // if no consumer key assigned then use the validator's key itself + k.Logger(ctx).Info("could not retrieve public key for validator on consumer chain because"+ + " the validator did not assign a new consumer key", + "validator", val.GetOperator().String(), + "chainID", chainID) + nextConsumerPublicKey, err = val.TmConsPublicKey() + if err != nil { + // this should never happen and might not be recoverable because without the public key + // we cannot generate a validator update + panic(fmt.Errorf("could not retrieve validator's (%+v) public key: %w", val, err)) + } + } + + nextValidator := types.ConsumerValidator{ + ProviderConsAddr: consAddr, + Power: nextPower, + ConsumerPublicKey: &nextConsumerPublicKey, + } + nextValidators = append(nextValidators, nextValidator) + } + + return nextValidators +} + +// DiffValidators compares the current and the next epoch's consumer validators and returns the `ValidatorUpdate` diff +// needed by CometBFT to update the validator set on a chain. +func DiffValidators( + currentValidators []types.ConsumerValidator, + nextValidators []types.ConsumerValidator) []abci.ValidatorUpdate { + var updates []abci.ValidatorUpdate + + isCurrentValidator := make(map[string]types.ConsumerValidator) + for _, val := range currentValidators { + isCurrentValidator[val.ConsumerPublicKey.String()] = val + } + + isNextValidator := make(map[string]types.ConsumerValidator) + for _, val := range nextValidators { + isNextValidator[val.ConsumerPublicKey.String()] = val + } + + for _, currentVal := range currentValidators { + if nextVal, found := isNextValidator[currentVal.ConsumerPublicKey.String()]; !found { + // this consumer public key does not appear in the next validators and hence we remove the validator + // with that consumer public key by creating an update with 0 power + updates = append(updates, abci.ValidatorUpdate{PubKey: *currentVal.ConsumerPublicKey, Power: 0}) + } else if currentVal.Power != nextVal.Power { + // validator did not modify its consumer public key but has changed its voting power, so we + // have to create an update with the new power + updates = append(updates, abci.ValidatorUpdate{PubKey: *nextVal.ConsumerPublicKey, Power: nextVal.Power}) + } + // else no update is needed because neither the consumer public key changed, nor the power of the validator + } + + for _, nextVal := range nextValidators { + if _, found := isCurrentValidator[nextVal.ConsumerPublicKey.String()]; !found { + // this consumer public key does not exist in the current validators and hence we introduce this validator + updates = append(updates, abci.ValidatorUpdate{PubKey: *nextVal.ConsumerPublicKey, Power: nextVal.Power}) + } + } + + return updates +} + +// SetConsumerValSet resets the current consumer validators with the `nextValidators` computed by +// `ComputeNextEpochConsumerValSet` and hence this method should only be called after `ComputeNextEpochConsumerValSet` has completed. +func (k Keeper) SetConsumerValSet(ctx sdk.Context, chainID string, nextValidators []types.ConsumerValidator) { + k.DeleteConsumerValSet(ctx, chainID) + for _, val := range nextValidators { + k.SetConsumerValidator(ctx, chainID, val) + } +} diff --git a/x/ccv/provider/keeper/validator_set_update_test.go b/x/ccv/provider/keeper/validator_set_update_test.go new file mode 100644 index 0000000000..8505158816 --- /dev/null +++ b/x/ccv/provider/keeper/validator_set_update_test.go @@ -0,0 +1,355 @@ +package keeper_test + +import ( + "bytes" + "sort" + "testing" + + abci "github.com/cometbft/cometbft/abci/types" + "github.com/cometbft/cometbft/crypto/ed25519" + "github.com/cometbft/cometbft/proto/tendermint/crypto" + codectypes "github.com/cosmos/cosmos-sdk/codec/types" + cryptocodec "github.com/cosmos/cosmos-sdk/crypto/codec" + sdk "github.com/cosmos/cosmos-sdk/types" + stakingtypes "github.com/cosmos/cosmos-sdk/x/staking/types" + cryptotestutil "github.com/cosmos/interchain-security/v4/testutil/crypto" + testkeeper "github.com/cosmos/interchain-security/v4/testutil/keeper" + "github.com/cosmos/interchain-security/v4/x/ccv/provider/keeper" + "github.com/cosmos/interchain-security/v4/x/ccv/provider/types" + "github.com/stretchr/testify/require" +) + +// TestConsumerValidator tests the `SetConsumerValidator`, `IsConsumerValidator`, and `DeleteConsumerValidator` methods +func TestConsumerValidator(t *testing.T) { + providerKeeper, ctx, ctrl, _ := testkeeper.GetProviderKeeperAndCtx(t, testkeeper.NewInMemKeeperParams(t)) + defer ctrl.Finish() + + validator := types.ConsumerValidator{ + ProviderConsAddr: []byte("providerConsAddr"), + Power: 2, + ConsumerPublicKey: &crypto.PublicKey{}, + } + + require.False(t, providerKeeper.IsConsumerValidator(ctx, "chainID", types.NewProviderConsAddress(validator.ProviderConsAddr))) + providerKeeper.SetConsumerValidator(ctx, "chainID", validator) + require.True(t, providerKeeper.IsConsumerValidator(ctx, "chainID", types.NewProviderConsAddress(validator.ProviderConsAddr))) + providerKeeper.DeleteConsumerValidator(ctx, "chainID", types.NewProviderConsAddress(validator.ProviderConsAddr)) + require.False(t, providerKeeper.IsConsumerValidator(ctx, "chainID", types.NewProviderConsAddress(validator.ProviderConsAddr))) +} + +func TestGetConsumerValSet(t *testing.T) { + providerKeeper, ctx, ctrl, _ := testkeeper.GetProviderKeeperAndCtx(t, testkeeper.NewInMemKeeperParams(t)) + defer ctrl.Finish() + + // create 3 validators and set them as current validators + expectedValidators := []types.ConsumerValidator{ + { + ProviderConsAddr: []byte("providerConsAddr1"), + Power: 1, + ConsumerPublicKey: &crypto.PublicKey{ + Sum: &crypto.PublicKey_Ed25519{ + Ed25519: []byte{1}, + }, + }, + }, + { + ProviderConsAddr: []byte("providerConsAddr2"), + Power: 2, + ConsumerPublicKey: &crypto.PublicKey{ + Sum: &crypto.PublicKey_Ed25519{ + Ed25519: []byte{2}, + }, + }, + }, + { + ProviderConsAddr: []byte("providerConsAddr3"), + Power: 3, + ConsumerPublicKey: &crypto.PublicKey{ + Sum: &crypto.PublicKey_Ed25519{ + Ed25519: []byte{3}, + }, + }, + }, + } + + for _, expectedValidator := range expectedValidators { + providerKeeper.SetConsumerValidator(ctx, "chainID", + types.ConsumerValidator{ + ProviderConsAddr: expectedValidator.ProviderConsAddr, + Power: expectedValidator.Power, + ConsumerPublicKey: expectedValidator.ConsumerPublicKey, + }) + } + + actualValidators := providerKeeper.GetConsumerValSet(ctx, "chainID") + + // sort validators first to be able to compare + sortValidators := func(validators []types.ConsumerValidator) { + sort.Slice(validators, func(i int, j int) bool { + return bytes.Compare(validators[i].ProviderConsAddr, validators[j].ProviderConsAddr) < 0 + }) + } + sortValidators(expectedValidators) + sortValidators(actualValidators) + require.Equal(t, expectedValidators, actualValidators) +} + +// createConsumerValidator is a helper function to create a consumer validator with the given `power`. It uses `index` as +// the `ProviderConsAddr` of the validator, and the `seed` to generate the consumer public key. Returns the validator +// and its consumer public key. +func createConsumerValidator(index int, power int64, seed int) (types.ConsumerValidator, crypto.PublicKey) { + publicKey := cryptotestutil.NewCryptoIdentityFromIntSeed(seed).TMProtoCryptoPublicKey() + + return types.ConsumerValidator{ + ProviderConsAddr: []byte{byte(index)}, + Power: power, + ConsumerPublicKey: &publicKey, + }, publicKey +} + +func TestComputeNextEpochConsumerValSet(t *testing.T) { + providerKeeper, ctx, ctrl, mocks := testkeeper.GetProviderKeeperAndCtx(t, testkeeper.NewInMemKeeperParams(t)) + defer ctrl.Finish() + + chainID := "chainID" + + // helper function to generate a validator with the given power and with a provider address based on index + createStakingValidator := func(ctx sdk.Context, mocks testkeeper.MockedKeepers, index int, power int64) stakingtypes.Validator { + providerConsPubKey := ed25519.GenPrivKeyFromSecret([]byte{byte(index)}).PubKey() + consAddr := sdk.ConsAddress(providerConsPubKey.Address()) + providerAddr := types.NewProviderConsAddress(consAddr) + pk, _ := cryptocodec.FromTmPubKeyInterface(providerConsPubKey) + pkAny, _ := codectypes.NewAnyWithValue(pk) + + var providerValidatorAddr sdk.ValAddress + providerValidatorAddr = providerAddr.Address.Bytes() + + mocks.MockStakingKeeper.EXPECT(). + GetLastValidatorPower(ctx, providerValidatorAddr).Return(power).AnyTimes() + + return stakingtypes.Validator{ + OperatorAddress: providerValidatorAddr.String(), + ConsensusPubkey: pkAny, + } + } + + // no consumer validators returned if we have no bonded validators + require.Empty(t, providerKeeper.ComputeNextEpochConsumerValSet(ctx, chainID, []stakingtypes.Validator{})) + + var expectedValidators []types.ConsumerValidator + + // create a staking validator A that has not set a consumer public key + valA := createStakingValidator(ctx, mocks, 1, 1) + // because validator A has no consumer key set, the `ConsumerPublicKey` we expect is the key on the provider chain + valAConsAddr, _ := valA.GetConsAddr() + valAPublicKey, _ := valA.TmConsPublicKey() + expectedValidators = append(expectedValidators, types.ConsumerValidator{ + ProviderConsAddr: types.NewProviderConsAddress(valAConsAddr).Address.Bytes(), + Power: 1, + ConsumerPublicKey: &valAPublicKey, + }) + + // create a staking validator B that has set a consumer public key + valB := createStakingValidator(ctx, mocks, 2, 2) + // validator B has set a consumer key, the `ConsumerPublicKey` we expect is the key set by `SetValidatorConsumerPubKey` + valBConsumerKey := cryptotestutil.NewCryptoIdentityFromIntSeed(1).TMProtoCryptoPublicKey() + valBConsAddr, _ := valB.GetConsAddr() + providerKeeper.SetValidatorConsumerPubKey(ctx, chainID, types.NewProviderConsAddress(valBConsAddr), valBConsumerKey) + expectedValidators = append(expectedValidators, types.ConsumerValidator{ + ProviderConsAddr: types.NewProviderConsAddress(valBConsAddr).Address.Bytes(), + Power: 2, + ConsumerPublicKey: &valBConsumerKey, + }) + + bondedValidators := []stakingtypes.Validator{valA, valB} + actualValidators := providerKeeper.ComputeNextEpochConsumerValSet(ctx, "chainID", bondedValidators) + require.Equal(t, expectedValidators, actualValidators) +} + +func TestDiff(t *testing.T) { + // In what follows we create 6 validators: A, B, C, D, E, and F where currentValidators = {A, B, C, D, E} + // and nextValidators = {B, C, D, E, F}. For the validators {B, C, D, E} in the intersection we have: + // - validator B has no power or consumer key change + // - validator C has changed its power + // - validator D has no power change but has changed its consumer key + // - validator E has both changed its power and its consumer key + + var expectedUpdates []abci.ValidatorUpdate + + // validator A only exists in `currentValidators` and hence an update with 0 power would be generated + // to remove this validator + currentA, currentPublicKeyA := createConsumerValidator(1, 1, 1) + expectedUpdates = append(expectedUpdates, abci.ValidatorUpdate{PubKey: currentPublicKeyA, Power: 0}) + + // validator B exists in both `currentValidators` and `nextValidators` but it did not change its + // power or consumer public key and hence no validator update is generated + currentB, _ := createConsumerValidator(2, 1, 2) + nextB, _ := createConsumerValidator(2, 1, 2) + + // validator C exists in both `currentValidators` and `nextValidators` and it changes its power, so + // a validator update is generated with the new power + currentC, currentPublicKeyC := createConsumerValidator(3, 1, 3) + nextC, _ := createConsumerValidator(3, 2, 3) + expectedUpdates = append(expectedUpdates, abci.ValidatorUpdate{PubKey: currentPublicKeyC, Power: 2}) + + // validator D exists in both `currentValidators` and `nextValidators` and it changes its consumer public key, so + // a validator update is generated to remove the old public key and another update to add the new public key + currentD, currentPublicKeyD := createConsumerValidator(4, 1, 4) + nextD, nextPublicKeyD := createConsumerValidator(4, 1, 5) + expectedUpdates = append(expectedUpdates, abci.ValidatorUpdate{PubKey: currentPublicKeyD, Power: 0}) + expectedUpdates = append(expectedUpdates, abci.ValidatorUpdate{PubKey: nextPublicKeyD, Power: 1}) + + // validator E exists in both `currentValidators` and `nextValidators` and it changes both its power and + // its consumer public key, so a validator update is generated to remove the old public key and another update to + // add the new public key with thew new power + currentE, currentPublicKeyE := createConsumerValidator(5, 1, 6) + nextE, nextPublicKeyE := createConsumerValidator(5, 2, 7) + expectedUpdates = append(expectedUpdates, abci.ValidatorUpdate{PubKey: currentPublicKeyE, Power: 0}) + expectedUpdates = append(expectedUpdates, abci.ValidatorUpdate{PubKey: nextPublicKeyE, Power: 2}) + + // validator F does not exist in `currentValidators` and hence an update is generated to add this new validator + nextF, nextPublicKeyF := createConsumerValidator(6, 1, 8) + expectedUpdates = append(expectedUpdates, abci.ValidatorUpdate{PubKey: nextPublicKeyF, Power: 1}) + + currentValidators := []types.ConsumerValidator{currentA, currentB, currentC, currentD, currentE} + nextValidators := []types.ConsumerValidator{nextB, nextC, nextD, nextE, nextF} + + actualUpdates := keeper.DiffValidators(currentValidators, nextValidators) + + // sort validators first to be able to compare + sortUpdates := func(updates []abci.ValidatorUpdate) { + sort.Slice(updates, func(i, j int) bool { + if updates[i].Power != updates[j].Power { + return updates[i].Power < updates[j].Power + } + return updates[i].PubKey.String() < updates[j].PubKey.String() + }) + } + + sortUpdates(expectedUpdates) + sortUpdates(actualUpdates) + require.Equal(t, expectedUpdates, actualUpdates) +} + +func TestDiffEdgeCases(t *testing.T) { + require.Empty(t, len(keeper.DiffValidators([]types.ConsumerValidator{}, []types.ConsumerValidator{}))) + + valA, publicKeyA := createConsumerValidator(1, 1, 1) + valB, publicKeyB := createConsumerValidator(2, 2, 2) + valC, publicKeyC := createConsumerValidator(3, 3, 3) + validators := []types.ConsumerValidator{valA, valB, valC} + + // we do not expect any validator updates if the `currentValidators` are the same with the `nextValidators` + require.Empty(t, len(keeper.DiffValidators(validators, validators))) + + // only have `nextValidators` that would generate validator updates for the validators to be added + expectedUpdates := []abci.ValidatorUpdate{{publicKeyA, 1}, {publicKeyB, 2}, {publicKeyC, 3}} + actualUpdates := keeper.DiffValidators([]types.ConsumerValidator{}, validators) + // sort validators first to be able to compare + sortUpdates := func(updates []abci.ValidatorUpdate) { + sort.Slice(updates, func(i, j int) bool { + if updates[i].Power != updates[j].Power { + return updates[i].Power < updates[j].Power + } + return updates[i].PubKey.String() < updates[j].PubKey.String() + }) + } + + sortUpdates(expectedUpdates) + sortUpdates(actualUpdates) + require.Equal(t, expectedUpdates, actualUpdates) + + // only have `currentValidators` that would generate validator updates for the validators to be removed + expectedUpdates = []abci.ValidatorUpdate{{publicKeyA, 0}, {publicKeyB, 0}, {publicKeyC, 0}} + actualUpdates = keeper.DiffValidators(validators, []types.ConsumerValidator{}) + sortUpdates(expectedUpdates) + sortUpdates(actualUpdates) + require.Equal(t, expectedUpdates, actualUpdates) + + // have nonempty `currentValidators` and `nextValidators`, but with empty intersection + // all old validators should be removed, all new validators should be added + expectedUpdates = []abci.ValidatorUpdate{{publicKeyA, 0}, {publicKeyB, 2}} + actualUpdates = keeper.DiffValidators(validators[0:1], validators[1:2]) + sortUpdates(expectedUpdates) + sortUpdates(actualUpdates) + require.Equal(t, expectedUpdates, actualUpdates) +} + +func TestSetConsumerValSet(t *testing.T) { + providerKeeper, ctx, ctrl, _ := testkeeper.GetProviderKeeperAndCtx(t, testkeeper.NewInMemKeeperParams(t)) + defer ctrl.Finish() + + chainID := "chainID" + + currentValidators := []types.ConsumerValidator{ + { + ProviderConsAddr: []byte("currentProviderConsAddr1"), + Power: 2, + ConsumerPublicKey: &crypto.PublicKey{ + Sum: &crypto.PublicKey_Ed25519{ + Ed25519: []byte{2}, + }, + }, + }, + { + ProviderConsAddr: []byte("currentProviderConsAddr2"), + Power: 3, + ConsumerPublicKey: &crypto.PublicKey{ + Sum: &crypto.PublicKey_Ed25519{ + Ed25519: []byte{3}, + }, + }, + }, + { + ProviderConsAddr: []byte("currentProviderConsAddr3"), + Power: 4, + ConsumerPublicKey: &crypto.PublicKey{ + Sum: &crypto.PublicKey_Ed25519{ + Ed25519: []byte{4}, + }, + }, + }, + } + + nextValidators := []types.ConsumerValidator{ + { + ProviderConsAddr: []byte("nextProviderConsAddr1"), + Power: 2, + ConsumerPublicKey: &crypto.PublicKey{ + Sum: &crypto.PublicKey_Ed25519{ + Ed25519: []byte{2}, + }, + }, + }, + { + ProviderConsAddr: []byte("nextProviderConsAddr2"), + Power: 3, + ConsumerPublicKey: &crypto.PublicKey{ + Sum: &crypto.PublicKey_Ed25519{ + Ed25519: []byte{3}, + }, + }, + }, + } + + // set the `currentValidators` for chain `chainID` + require.Empty(t, providerKeeper.GetConsumerValSet(ctx, chainID)) + for _, validator := range currentValidators { + providerKeeper.SetConsumerValidator(ctx, chainID, validator) + } + require.NotEmpty(t, providerKeeper.GetConsumerValSet(ctx, chainID)) + + providerKeeper.SetConsumerValSet(ctx, chainID, nextValidators) + nextCurrentValidators := providerKeeper.GetConsumerValSet(ctx, chainID) + + // sort validators first to be able to compare + sortValidators := func(validators []types.ConsumerValidator) { + sort.Slice(validators, func(i, j int) bool { + return bytes.Compare(validators[i].ProviderConsAddr, validators[j].ProviderConsAddr) < 0 + }) + } + + sortValidators(nextValidators) + sortValidators(nextCurrentValidators) + require.Equal(t, nextValidators, nextCurrentValidators) +} diff --git a/x/ccv/provider/types/genesis_test.go b/x/ccv/provider/types/genesis_test.go index 45d766fcfb..41a716757f 100644 --- a/x/ccv/provider/types/genesis_test.go +++ b/x/ccv/provider/types/genesis_test.go @@ -81,7 +81,7 @@ func TestValidateGenesisState(t *testing.T) { nil, types.NewParams(ibctmtypes.NewClientState("", ibctmtypes.DefaultTrustLevel, 0, 0, time.Second*40, clienttypes.Height{}, commitmenttypes.GetSDKSpecs(), []string{"ibc", "upgradedIBCState"}), - types.DefaultTrustingPeriodFraction, time.Hour, time.Hour, 30*time.Minute, time.Hour, "0.1", sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}), + types.DefaultTrustingPeriodFraction, time.Hour, time.Hour, 30*time.Minute, time.Hour, "0.1", sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}, 600), nil, nil, nil, @@ -102,7 +102,7 @@ func TestValidateGenesisState(t *testing.T) { nil, types.NewParams(ibctmtypes.NewClientState("", ibctmtypes.DefaultTrustLevel, 0, 0, time.Second*40, clienttypes.Height{}, commitmenttypes.GetSDKSpecs(), []string{"ibc", "upgradedIBCState"}), - types.DefaultTrustingPeriodFraction, time.Hour, time.Hour, 30*time.Minute, time.Hour, "0.1", sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}), + types.DefaultTrustingPeriodFraction, time.Hour, time.Hour, 30*time.Minute, time.Hour, "0.1", sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}, 600), nil, nil, nil, @@ -123,7 +123,7 @@ func TestValidateGenesisState(t *testing.T) { nil, types.NewParams(ibctmtypes.NewClientState("", ibctmtypes.DefaultTrustLevel, 0, 0, time.Second*40, clienttypes.Height{}, commitmenttypes.GetSDKSpecs(), []string{"ibc", "upgradedIBCState"}), - types.DefaultTrustingPeriodFraction, time.Hour, time.Hour, 30*time.Minute, time.Hour, "0.1", sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}), + types.DefaultTrustingPeriodFraction, time.Hour, time.Hour, 30*time.Minute, time.Hour, "0.1", sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}, 600), nil, nil, nil, @@ -144,7 +144,7 @@ func TestValidateGenesisState(t *testing.T) { nil, types.NewParams(ibctmtypes.NewClientState("", ibctmtypes.DefaultTrustLevel, 0, 0, time.Second*40, clienttypes.Height{}, commitmenttypes.GetSDKSpecs(), []string{"ibc", "upgradedIBCState"}), - types.DefaultTrustingPeriodFraction, time.Hour, time.Hour, 30*time.Minute, time.Hour, "0.1", sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}), + types.DefaultTrustingPeriodFraction, time.Hour, time.Hour, 30*time.Minute, time.Hour, "0.1", sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}, 600), nil, nil, nil, @@ -171,7 +171,7 @@ func TestValidateGenesisState(t *testing.T) { types.DefaultVscTimeoutPeriod, types.DefaultSlashMeterReplenishPeriod, types.DefaultSlashMeterReplenishFraction, - sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}), + sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}, 600), nil, nil, nil, @@ -198,7 +198,7 @@ func TestValidateGenesisState(t *testing.T) { types.DefaultVscTimeoutPeriod, types.DefaultSlashMeterReplenishPeriod, types.DefaultSlashMeterReplenishFraction, - sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}), + sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}, 600), nil, nil, nil, @@ -225,7 +225,7 @@ func TestValidateGenesisState(t *testing.T) { types.DefaultVscTimeoutPeriod, types.DefaultSlashMeterReplenishPeriod, types.DefaultSlashMeterReplenishFraction, - sdk.Coin{Denom: "stake", Amount: sdk.NewInt(1000000)}), + sdk.Coin{Denom: "stake", Amount: sdk.NewInt(1000000)}, 600), nil, nil, nil, @@ -252,7 +252,7 @@ func TestValidateGenesisState(t *testing.T) { types.DefaultVscTimeoutPeriod, types.DefaultSlashMeterReplenishPeriod, types.DefaultSlashMeterReplenishFraction, - sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}), + sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}, 600), nil, nil, nil, @@ -279,7 +279,7 @@ func TestValidateGenesisState(t *testing.T) { 0, // 0 vsc timeout here types.DefaultSlashMeterReplenishPeriod, types.DefaultSlashMeterReplenishFraction, - sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}), + sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}, 600), nil, nil, nil, @@ -306,7 +306,7 @@ func TestValidateGenesisState(t *testing.T) { types.DefaultVscTimeoutPeriod, 0, // 0 slash meter replenish period here types.DefaultSlashMeterReplenishFraction, - sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}), + sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}, 600), nil, nil, nil, @@ -333,7 +333,7 @@ func TestValidateGenesisState(t *testing.T) { types.DefaultVscTimeoutPeriod, types.DefaultSlashMeterReplenishPeriod, "1.15", - sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}), + sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}, 600), nil, nil, nil, @@ -685,7 +685,7 @@ func TestValidateGenesisState(t *testing.T) { nil, types.NewParams(ibctmtypes.NewClientState("", ibctmtypes.DefaultTrustLevel, 0, 0, time.Second*40, clienttypes.Height{}, commitmenttypes.GetSDKSpecs(), []string{"ibc", "upgradedIBCState"}), - types.DefaultTrustingPeriodFraction, time.Hour, time.Hour, 30*time.Minute, time.Hour, "0.1", sdk.Coin{Denom: "st", Amount: sdk.NewInt(10000000)}), + types.DefaultTrustingPeriodFraction, time.Hour, time.Hour, 30*time.Minute, time.Hour, "0.1", sdk.Coin{Denom: "st", Amount: sdk.NewInt(10000000)}, 600), nil, nil, nil, @@ -706,7 +706,7 @@ func TestValidateGenesisState(t *testing.T) { nil, types.NewParams(ibctmtypes.NewClientState("", ibctmtypes.DefaultTrustLevel, 0, 0, time.Second*40, clienttypes.Height{}, commitmenttypes.GetSDKSpecs(), []string{"ibc", "upgradedIBCState"}), - types.DefaultTrustingPeriodFraction, time.Hour, time.Hour, 30*time.Minute, time.Hour, "0.1", sdk.Coin{Denom: "stake", Amount: sdk.NewInt(-1000000)}), + types.DefaultTrustingPeriodFraction, time.Hour, time.Hour, 30*time.Minute, time.Hour, "0.1", sdk.Coin{Denom: "stake", Amount: sdk.NewInt(-1000000)}, 600), nil, nil, nil, diff --git a/x/ccv/provider/types/keys.go b/x/ccv/provider/types/keys.go index 615b901368..e9222ade98 100644 --- a/x/ccv/provider/types/keys.go +++ b/x/ccv/provider/types/keys.go @@ -120,7 +120,9 @@ const ( // on consumer chains to validator addresses on the provider chain ValidatorsByConsumerAddrBytePrefix - // KeyAssignmentReplacementsBytePrefix is the byte prefix that will store the key assignments that need to be replaced in the current block + // KeyAssignmentReplacementsBytePrefix was the byte prefix used to store the key assignments that needed to be replaced in the current block + // NOTE: This prefix is deprecated, but left in place to avoid consumer state migrations + // [DEPRECATED] KeyAssignmentReplacementsBytePrefix // ConsumerAddrsToPruneBytePrefix is the byte prefix that will store the mapping from VSC ids @@ -145,6 +147,9 @@ const ( // ProposedConsumerChainByteKey is the byte prefix storing the consumer chainId in consumerAddition gov proposal submitted before voting finishes ProposedConsumerChainByteKey + // ConsumerValidatorBytePrefix is the byte prefix used when storing for each consumer chain all the consumer validators in this epoch + ConsumerValidatorBytePrefix + // NOTE: DO NOT ADD NEW BYTE PREFIXES HERE WITHOUT ADDING THEM TO getAllKeyPrefixes() IN keys_test.go ) @@ -362,12 +367,6 @@ func ValidatorsByConsumerAddrKey(chainID string, addr ConsumerConsAddress) []byt return ChainIdAndConsAddrKey(ValidatorsByConsumerAddrBytePrefix, chainID, addr.ToSdkConsAddr()) } -// KeyAssignmentReplacementsKey returns the key under which the -// key assignments that need to be replaced in the current block are stored -func KeyAssignmentReplacementsKey(chainID string, addr ProviderConsAddress) []byte { - return ChainIdAndConsAddrKey(KeyAssignmentReplacementsBytePrefix, chainID, addr.ToSdkConsAddr()) -} - // ConsumerAddrsToPruneKey returns the key under which the // mapping from VSC ids to consumer validators addresses is stored func ConsumerAddrsToPruneKey(chainID string, vscID uint64) []byte { @@ -517,6 +516,12 @@ func ParseProposedConsumerChainKey(prefix byte, bz []byte) (uint64, error) { return proposalID, nil } +// ConsumerValidatorKey returns the key of consumer chain `chainID` and validator with `providerAddr` +func ConsumerValidatorKey(chainID string, providerAddr []byte) []byte { + prefix := ChainIdWithLenKey(ConsumerValidatorBytePrefix, chainID) + return append(prefix, providerAddr...) +} + // // End of generic helpers section // diff --git a/x/ccv/provider/types/keys_test.go b/x/ccv/provider/types/keys_test.go index 4d5ea58ff8..02faa9a640 100644 --- a/x/ccv/provider/types/keys_test.go +++ b/x/ccv/provider/types/keys_test.go @@ -56,6 +56,7 @@ func getAllKeyPrefixes() []byte { providertypes.VSCMaturedHandledThisBlockBytePrefix, providertypes.EquivocationEvidenceMinHeightBytePrefix, providertypes.ProposedConsumerChainByteKey, + providertypes.ConsumerValidatorBytePrefix, } } @@ -96,7 +97,6 @@ func getAllFullyDefinedKeys() [][]byte { providertypes.GlobalSlashEntryKey(providertypes.GlobalSlashEntry{}), providertypes.ConsumerValidatorsKey("chainID", providertypes.NewProviderConsAddress([]byte{0x05})), providertypes.ValidatorsByConsumerAddrKey("chainID", providertypes.NewConsumerConsAddress([]byte{0x05})), - providertypes.KeyAssignmentReplacementsKey("chainID", providertypes.NewProviderConsAddress([]byte{0x05})), providertypes.ConsumerAddrsToPruneKey("chainID", 88), providertypes.SlashLogKey(providertypes.NewProviderConsAddress([]byte{0x05})), providertypes.VSCMaturedHandledThisBlockKey(), diff --git a/x/ccv/provider/types/params.go b/x/ccv/provider/types/params.go index a580e60f41..30bce2d17f 100644 --- a/x/ccv/provider/types/params.go +++ b/x/ccv/provider/types/params.go @@ -36,6 +36,10 @@ const ( // that is replenished to the slash meter every replenish period. This param also serves as a maximum // fraction of total voting power that the slash meter can hold. DefaultSlashMeterReplenishFraction = "0.05" + + // DefaultBlocksPerEpoch defines the default blocks that constitute an epoch. Assuming we need 6 seconds per block, + // an epoch corresponds to 1 hour (6 * 600 = 3600 seconds). + DefaultBlocksPerEpoch = 600 ) // Reflection based keys for params subspace @@ -47,6 +51,7 @@ var ( KeySlashMeterReplenishPeriod = []byte("SlashMeterReplenishPeriod") KeySlashMeterReplenishFraction = []byte("SlashMeterReplenishFraction") KeyConsumerRewardDenomRegistrationFee = []byte("ConsumerRewardDenomRegistrationFee") + KeyBlocksPerEpoch = []byte("BlocksPerEpoch") ) // ParamKeyTable returns a key table with the necessary registered provider params @@ -64,6 +69,7 @@ func NewParams( slashMeterReplenishPeriod time.Duration, slashMeterReplenishFraction string, consumerRewardDenomRegistrationFee sdk.Coin, + blocksPerEpoch int64, ) Params { return Params{ TemplateClient: cs, @@ -74,6 +80,7 @@ func NewParams( SlashMeterReplenishPeriod: slashMeterReplenishPeriod, SlashMeterReplenishFraction: slashMeterReplenishFraction, ConsumerRewardDenomRegistrationFee: consumerRewardDenomRegistrationFee, + BlocksPerEpoch: blocksPerEpoch, } } @@ -104,6 +111,7 @@ func DefaultParams() Params { Denom: sdk.DefaultBondDenom, Amount: sdk.NewInt(10000000), }, + DefaultBlocksPerEpoch, ) } @@ -136,6 +144,9 @@ func (p Params) Validate() error { if err := ValidateCoin(p.ConsumerRewardDenomRegistrationFee); err != nil { return fmt.Errorf("consumer reward denom registration fee is invalid: %s", err) } + if err := ccvtypes.ValidateInt64(p.BlocksPerEpoch); err != nil { + return fmt.Errorf("blocks per epoch is invalid: %s", err) + } return nil } @@ -150,6 +161,7 @@ func (p *Params) ParamSetPairs() paramtypes.ParamSetPairs { paramtypes.NewParamSetPair(KeySlashMeterReplenishPeriod, p.SlashMeterReplenishPeriod, ccvtypes.ValidateDuration), paramtypes.NewParamSetPair(KeySlashMeterReplenishFraction, p.SlashMeterReplenishFraction, ccvtypes.ValidateStringFraction), paramtypes.NewParamSetPair(KeyConsumerRewardDenomRegistrationFee, p.ConsumerRewardDenomRegistrationFee, ValidateCoin), + paramtypes.NewParamSetPair(KeyBlocksPerEpoch, p.BlocksPerEpoch, ccvtypes.ValidatePositiveInt64), } } diff --git a/x/ccv/provider/types/params_test.go b/x/ccv/provider/types/params_test.go index 1de6b6fe54..4e72c233af 100644 --- a/x/ccv/provider/types/params_test.go +++ b/x/ccv/provider/types/params_test.go @@ -24,39 +24,39 @@ func TestValidateParams(t *testing.T) { {"custom valid params", types.NewParams( ibctmtypes.NewClientState("", ibctmtypes.DefaultTrustLevel, 0, 0, time.Second*40, clienttypes.Height{}, commitmenttypes.GetSDKSpecs(), []string{"ibc", "upgradedIBCState"}), - "0.33", time.Hour, time.Hour, time.Hour, 30*time.Minute, "0.1", sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}), true}, + "0.33", time.Hour, time.Hour, time.Hour, 30*time.Minute, "0.1", sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}, 1000), true}, {"custom invalid params", types.NewParams( ibctmtypes.NewClientState("", ibctmtypes.DefaultTrustLevel, 0, 0, 0, clienttypes.Height{}, nil, []string{"ibc", "upgradedIBCState"}), - "0.33", time.Hour, time.Hour, time.Hour, 30*time.Minute, "0.1", sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}), false}, + "0.33", time.Hour, time.Hour, time.Hour, 30*time.Minute, "0.1", sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}, 1000), false}, {"blank client", types.NewParams(&ibctmtypes.ClientState{}, - "0.33", time.Hour, time.Hour, time.Hour, 30*time.Minute, "0.1", sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}), false}, - {"nil client", types.NewParams(nil, "0.33", time.Hour, time.Hour, time.Hour, 30*time.Minute, "0.1", sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}), false}, + "0.33", time.Hour, time.Hour, time.Hour, 30*time.Minute, "0.1", sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}, 1000), false}, + {"nil client", types.NewParams(nil, "0.33", time.Hour, time.Hour, time.Hour, 30*time.Minute, "0.1", sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}, 1000), false}, // Check if "0.00" is valid or if a zero dec TrustFraction needs to return an error {"0 trusting period fraction", types.NewParams(ibctmtypes.NewClientState("", ibctmtypes.DefaultTrustLevel, 0, 0, time.Second*40, clienttypes.Height{}, commitmenttypes.GetSDKSpecs(), []string{"ibc", "upgradedIBCState"}), - "0.00", time.Hour, time.Hour, time.Hour, 30*time.Minute, "0.1", sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}), true}, + "0.00", time.Hour, time.Hour, time.Hour, 30*time.Minute, "0.1", sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}, 1000), true}, {"0 ccv timeout period", types.NewParams(ibctmtypes.NewClientState("", ibctmtypes.DefaultTrustLevel, 0, 0, time.Second*40, clienttypes.Height{}, commitmenttypes.GetSDKSpecs(), []string{"ibc", "upgradedIBCState"}), - "0.33", 0, time.Hour, time.Hour, 30*time.Minute, "0.1", sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}), false}, + "0.33", 0, time.Hour, time.Hour, 30*time.Minute, "0.1", sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}, 1000), false}, {"0 init timeout period", types.NewParams(ibctmtypes.NewClientState("", ibctmtypes.DefaultTrustLevel, 0, 0, time.Second*40, clienttypes.Height{}, commitmenttypes.GetSDKSpecs(), []string{"ibc", "upgradedIBCState"}), - "0.33", time.Hour, 0, time.Hour, 30*time.Minute, "0.1", sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}), false}, + "0.33", time.Hour, 0, time.Hour, 30*time.Minute, "0.1", sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}, 1000), false}, {"0 vsc timeout period", types.NewParams(ibctmtypes.NewClientState("", ibctmtypes.DefaultTrustLevel, 0, 0, time.Second*40, clienttypes.Height{}, commitmenttypes.GetSDKSpecs(), []string{"ibc", "upgradedIBCState"}), - "0.33", time.Hour, time.Hour, 0, 30*time.Minute, "0.1", sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}), false}, + "0.33", time.Hour, time.Hour, 0, 30*time.Minute, "0.1", sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}, 1000), false}, {"0 slash meter replenish period", types.NewParams(ibctmtypes.NewClientState("", ibctmtypes.DefaultTrustLevel, 0, 0, time.Second*40, clienttypes.Height{}, commitmenttypes.GetSDKSpecs(), []string{"ibc", "upgradedIBCState"}), - "0.33", time.Hour, time.Hour, 24*time.Hour, 0, "0.1", sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}), false}, + "0.33", time.Hour, time.Hour, 24*time.Hour, 0, "0.1", sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}, 1000), false}, {"slash meter replenish fraction over 1", types.NewParams(ibctmtypes.NewClientState("", ibctmtypes.DefaultTrustLevel, 0, 0, time.Second*40, clienttypes.Height{}, commitmenttypes.GetSDKSpecs(), []string{"ibc", "upgradedIBCState"}), - "0.33", time.Hour, time.Hour, 24*time.Hour, time.Hour, "1.5", sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}), false}, + "0.33", time.Hour, time.Hour, 24*time.Hour, time.Hour, "1.5", sdk.Coin{Denom: "stake", Amount: sdk.NewInt(10000000)}, 1000), false}, {"invalid consumer reward denom registration fee denom", types.NewParams(ibctmtypes.NewClientState("", ibctmtypes.DefaultTrustLevel, 0, 0, time.Second*40, clienttypes.Height{}, commitmenttypes.GetSDKSpecs(), []string{"ibc", "upgradedIBCState"}), - "0.33", time.Hour, time.Hour, 24*time.Hour, time.Hour, "0.1", sdk.Coin{Denom: "st", Amount: sdk.NewInt(10000000)}), false}, + "0.33", time.Hour, time.Hour, 24*time.Hour, time.Hour, "0.1", sdk.Coin{Denom: "st", Amount: sdk.NewInt(10000000)}, 1000), false}, {"invalid consumer reward denom registration fee amount", types.NewParams(ibctmtypes.NewClientState("", ibctmtypes.DefaultTrustLevel, 0, 0, time.Second*40, clienttypes.Height{}, commitmenttypes.GetSDKSpecs(), []string{"ibc", "upgradedIBCState"}), - "0.33", time.Hour, time.Hour, 24*time.Hour, time.Hour, "0.1", sdk.Coin{Denom: "stake", Amount: sdk.NewInt(-10000000)}), false}, + "0.33", time.Hour, time.Hour, 24*time.Hour, time.Hour, "0.1", sdk.Coin{Denom: "stake", Amount: sdk.NewInt(-10000000)}, 1000), false}, } for _, tc := range testCases { diff --git a/x/ccv/provider/types/provider.pb.go b/x/ccv/provider/types/provider.pb.go index 41a87e69f4..819154b75d 100644 --- a/x/ccv/provider/types/provider.pb.go +++ b/x/ccv/provider/types/provider.pb.go @@ -451,6 +451,8 @@ type Params struct { SlashMeterReplenishFraction string `protobuf:"bytes,7,opt,name=slash_meter_replenish_fraction,json=slashMeterReplenishFraction,proto3" json:"slash_meter_replenish_fraction,omitempty"` // The fee required to be paid to add a reward denom ConsumerRewardDenomRegistrationFee types2.Coin `protobuf:"bytes,9,opt,name=consumer_reward_denom_registration_fee,json=consumerRewardDenomRegistrationFee,proto3" json:"consumer_reward_denom_registration_fee"` + // The number of blocks that comprise an epoch. + BlocksPerEpoch int64 `protobuf:"varint,10,opt,name=blocks_per_epoch,json=blocksPerEpoch,proto3" json:"blocks_per_epoch,omitempty"` } func (m *Params) Reset() { *m = Params{} } @@ -542,6 +544,13 @@ func (m *Params) GetConsumerRewardDenomRegistrationFee() types2.Coin { return types2.Coin{} } +func (m *Params) GetBlocksPerEpoch() int64 { + if m != nil { + return m.BlocksPerEpoch + } + return 0 +} + // SlashAcks contains cons addresses of consumer chain validators // successfully slashed on the provider chain. type SlashAcks struct { @@ -1385,6 +1394,71 @@ func (m *ConsumerAddrsToPrune) GetConsumerAddrs() *AddressList { return nil } +// ConsumerValidator is used to facilitate epoch-based transitions. It contains relevant info for +// a validator that is opted in, in an epoch, on a consumer chain. +type ConsumerValidator struct { + // validator's consensus address on the provider chain + ProviderConsAddr []byte `protobuf:"bytes,1,opt,name=provider_cons_addr,json=providerConsAddr,proto3" json:"provider_cons_addr,omitempty"` + // voting power the validator has during this epoch + Power int64 `protobuf:"varint,2,opt,name=power,proto3" json:"power,omitempty"` + // public key the validator uses on the consumer chain during this epoch + ConsumerPublicKey *crypto.PublicKey `protobuf:"bytes,3,opt,name=consumer_public_key,json=consumerPublicKey,proto3" json:"consumer_public_key,omitempty"` +} + +func (m *ConsumerValidator) Reset() { *m = ConsumerValidator{} } +func (m *ConsumerValidator) String() string { return proto.CompactTextString(m) } +func (*ConsumerValidator) ProtoMessage() {} +func (*ConsumerValidator) Descriptor() ([]byte, []int) { + return fileDescriptor_f22ec409a72b7b72, []int{22} +} +func (m *ConsumerValidator) XXX_Unmarshal(b []byte) error { + return m.Unmarshal(b) +} +func (m *ConsumerValidator) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) { + if deterministic { + return xxx_messageInfo_ConsumerValidator.Marshal(b, m, deterministic) + } else { + b = b[:cap(b)] + n, err := m.MarshalToSizedBuffer(b) + if err != nil { + return nil, err + } + return b[:n], nil + } +} +func (m *ConsumerValidator) XXX_Merge(src proto.Message) { + xxx_messageInfo_ConsumerValidator.Merge(m, src) +} +func (m *ConsumerValidator) XXX_Size() int { + return m.Size() +} +func (m *ConsumerValidator) XXX_DiscardUnknown() { + xxx_messageInfo_ConsumerValidator.DiscardUnknown(m) +} + +var xxx_messageInfo_ConsumerValidator proto.InternalMessageInfo + +func (m *ConsumerValidator) GetProviderConsAddr() []byte { + if m != nil { + return m.ProviderConsAddr + } + return nil +} + +func (m *ConsumerValidator) GetPower() int64 { + if m != nil { + return m.Power + } + return 0 +} + +func (m *ConsumerValidator) GetConsumerPublicKey() *crypto.PublicKey { + if m != nil { + return m.ConsumerPublicKey + } + return nil +} + func init() { proto.RegisterType((*ConsumerAdditionProposal)(nil), "interchain_security.ccv.provider.v1.ConsumerAdditionProposal") proto.RegisterType((*ConsumerRemovalProposal)(nil), "interchain_security.ccv.provider.v1.ConsumerRemovalProposal") @@ -1408,6 +1482,7 @@ func init() { proto.RegisterType((*ValidatorConsumerPubKey)(nil), "interchain_security.ccv.provider.v1.ValidatorConsumerPubKey") proto.RegisterType((*ValidatorByConsumerAddr)(nil), "interchain_security.ccv.provider.v1.ValidatorByConsumerAddr") proto.RegisterType((*ConsumerAddrsToPrune)(nil), "interchain_security.ccv.provider.v1.ConsumerAddrsToPrune") + proto.RegisterType((*ConsumerValidator)(nil), "interchain_security.ccv.provider.v1.ConsumerValidator") } func init() { @@ -1415,113 +1490,117 @@ func init() { } var fileDescriptor_f22ec409a72b7b72 = []byte{ - // 1694 bytes of a gzipped FileDescriptorProto - 0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xb4, 0x58, 0xcd, 0x72, 0x1b, 0xc7, - 0x11, 0xe6, 0x12, 0x20, 0x45, 0x34, 0xf8, 0xa7, 0x25, 0x6d, 0x2d, 0x15, 0x06, 0xa4, 0xd6, 0xb1, - 0xc3, 0x94, 0xcb, 0x8b, 0x90, 0x4e, 0xaa, 0x5c, 0xaa, 0xb8, 0x5c, 0x24, 0x28, 0x5b, 0x14, 0x63, - 0x8b, 0x5e, 0x32, 0x54, 0x25, 0x39, 0x6c, 0x0d, 0x66, 0x47, 0xc0, 0x14, 0x17, 0x3b, 0xab, 0x99, - 0xd9, 0x95, 0x71, 0xc9, 0x39, 0x47, 0xe7, 0xe6, 0xca, 0x25, 0x4e, 0x5e, 0x20, 0xe7, 0xbc, 0x81, - 0x8f, 0x3e, 0xe6, 0x64, 0xa7, 0xa4, 0x63, 0x5e, 0x22, 0x35, 0xb3, 0xff, 0x20, 0xa1, 0x40, 0xe5, - 0xe4, 0x36, 0xdb, 0xd3, 0xfd, 0x75, 0xcf, 0x74, 0xf7, 0xd7, 0x03, 0xc0, 0x01, 0x0d, 0x25, 0xe1, - 0x78, 0x88, 0x68, 0xe8, 0x09, 0x82, 0x63, 0x4e, 0xe5, 0xb8, 0x8b, 0x71, 0xd2, 0x8d, 0x38, 0x4b, - 0xa8, 0x4f, 0x78, 0x37, 0xd9, 0x2f, 0xd6, 0x4e, 0xc4, 0x99, 0x64, 0xe6, 0x5b, 0x37, 0xd8, 0x38, - 0x18, 0x27, 0x4e, 0xa1, 0x97, 0xec, 0xdf, 0x7d, 0x7b, 0x1a, 0x70, 0xb2, 0xdf, 0x7d, 0x4e, 0x39, - 0x49, 0xb1, 0xee, 0x6e, 0x0e, 0xd8, 0x80, 0xe9, 0x65, 0x57, 0xad, 0x32, 0xe9, 0xce, 0x80, 0xb1, - 0x41, 0x40, 0xba, 0xfa, 0xab, 0x1f, 0x3f, 0xed, 0x4a, 0x3a, 0x22, 0x42, 0xa2, 0x51, 0x94, 0x29, - 0x74, 0x26, 0x15, 0xfc, 0x98, 0x23, 0x49, 0x59, 0x98, 0x03, 0xd0, 0x3e, 0xee, 0x62, 0xc6, 0x49, - 0x17, 0x07, 0x94, 0x84, 0x52, 0x79, 0x4d, 0x57, 0x99, 0x42, 0x57, 0x29, 0x04, 0x74, 0x30, 0x94, - 0xa9, 0x58, 0x74, 0x25, 0x09, 0x7d, 0xc2, 0x47, 0x34, 0x55, 0x2e, 0xbf, 0x32, 0x83, 0xed, 0xca, - 0x3e, 0xe6, 0xe3, 0x48, 0xb2, 0xee, 0x15, 0x19, 0x8b, 0x6c, 0xf7, 0x1d, 0xcc, 0xc4, 0x88, 0x89, - 0x2e, 0x51, 0xe7, 0x0f, 0x31, 0xe9, 0x26, 0xfb, 0x7d, 0x22, 0xd1, 0x7e, 0x21, 0xc8, 0xe3, 0xce, - 0xf4, 0xfa, 0x48, 0x94, 0x3a, 0x98, 0xd1, 0x2c, 0x6e, 0xfb, 0xfb, 0x45, 0xb0, 0x7a, 0x2c, 0x14, - 0xf1, 0x88, 0xf0, 0x43, 0xdf, 0xa7, 0xea, 0x48, 0x67, 0x9c, 0x45, 0x4c, 0xa0, 0xc0, 0xdc, 0x84, - 0x05, 0x49, 0x65, 0x40, 0x2c, 0x63, 0xd7, 0xd8, 0x6b, 0xb9, 0xe9, 0x87, 0xb9, 0x0b, 0x6d, 0x9f, - 0x08, 0xcc, 0x69, 0xa4, 0x94, 0xad, 0x79, 0xbd, 0x57, 0x15, 0x99, 0x5b, 0xb0, 0x94, 0xe6, 0x81, - 0xfa, 0x56, 0x43, 0x6f, 0xdf, 0xd2, 0xdf, 0x27, 0xbe, 0xf9, 0x09, 0xac, 0xd2, 0x90, 0x4a, 0x8a, - 0x02, 0x6f, 0x48, 0xd4, 0x6d, 0x58, 0xcd, 0x5d, 0x63, 0xaf, 0x7d, 0x70, 0xd7, 0xa1, 0x7d, 0xec, - 0xa8, 0x0b, 0x74, 0xb2, 0x6b, 0x4b, 0xf6, 0x9d, 0x87, 0x5a, 0xe3, 0xa8, 0xf9, 0xcd, 0x77, 0x3b, - 0x73, 0xee, 0x4a, 0x66, 0x97, 0x0a, 0xcd, 0x7b, 0xb0, 0x3c, 0x20, 0x21, 0x11, 0x54, 0x78, 0x43, - 0x24, 0x86, 0xd6, 0xc2, 0xae, 0xb1, 0xb7, 0xec, 0xb6, 0x33, 0xd9, 0x43, 0x24, 0x86, 0xe6, 0x0e, - 0xb4, 0xfb, 0x34, 0x44, 0x7c, 0x9c, 0x6a, 0x2c, 0x6a, 0x0d, 0x48, 0x45, 0x5a, 0xa1, 0x07, 0x20, - 0x22, 0xf4, 0x3c, 0xf4, 0x54, 0xb6, 0xad, 0x5b, 0x59, 0x20, 0x69, 0xa6, 0x9d, 0x3c, 0xd3, 0xce, - 0x45, 0x5e, 0x0a, 0x47, 0x4b, 0x2a, 0x90, 0x2f, 0xbf, 0xdf, 0x31, 0xdc, 0x96, 0xb6, 0x53, 0x3b, - 0xe6, 0x67, 0xb0, 0x1e, 0x87, 0x7d, 0x16, 0xfa, 0x34, 0x1c, 0x78, 0x11, 0xe1, 0x94, 0xf9, 0xd6, - 0x92, 0x86, 0xda, 0xba, 0x06, 0x75, 0x9c, 0x15, 0x4d, 0x8a, 0xf4, 0x95, 0x42, 0x5a, 0x2b, 0x8c, - 0xcf, 0xb4, 0xad, 0xf9, 0x39, 0x98, 0x18, 0x27, 0x3a, 0x24, 0x16, 0xcb, 0x1c, 0xb1, 0x35, 0x3b, - 0xe2, 0x3a, 0xc6, 0xc9, 0x45, 0x6a, 0x9d, 0x41, 0xfe, 0x1e, 0xee, 0x48, 0x8e, 0x42, 0xf1, 0x94, - 0xf0, 0x49, 0x5c, 0x98, 0x1d, 0xf7, 0x8d, 0x1c, 0xa3, 0x0e, 0xfe, 0x10, 0x76, 0x71, 0x56, 0x40, - 0x1e, 0x27, 0x3e, 0x15, 0x92, 0xd3, 0x7e, 0xac, 0x6c, 0xbd, 0xa7, 0x1c, 0x61, 0x5d, 0x23, 0x6d, - 0x5d, 0x04, 0x9d, 0x5c, 0xcf, 0xad, 0xa9, 0x7d, 0x9c, 0x69, 0x99, 0x8f, 0xe1, 0x27, 0xfd, 0x80, - 0xe1, 0x2b, 0xa1, 0x82, 0xf3, 0x6a, 0x48, 0xda, 0xf5, 0x88, 0x0a, 0xa1, 0xd0, 0x96, 0x77, 0x8d, - 0xbd, 0x86, 0x7b, 0x2f, 0xd5, 0x3d, 0x23, 0xfc, 0xb8, 0xa2, 0x79, 0x51, 0x51, 0x34, 0xdf, 0x03, - 0x73, 0x48, 0x85, 0x64, 0x9c, 0x62, 0x14, 0x78, 0x24, 0x94, 0x9c, 0x12, 0x61, 0xad, 0x68, 0xf3, - 0xdb, 0xe5, 0xce, 0x83, 0x74, 0xc3, 0x7c, 0x04, 0xf7, 0xa6, 0x3a, 0xf5, 0xf0, 0x10, 0x85, 0x21, - 0x09, 0xac, 0x55, 0x7d, 0x94, 0x1d, 0x7f, 0x8a, 0xcf, 0x5e, 0xaa, 0x76, 0x7f, 0xe9, 0x8f, 0x5f, - 0xef, 0xcc, 0x7d, 0xf5, 0xf5, 0xce, 0x9c, 0xfd, 0x77, 0x03, 0xee, 0xf4, 0x8a, 0x83, 0x8f, 0x58, - 0x82, 0x82, 0xff, 0x67, 0x83, 0x1d, 0x42, 0x4b, 0x48, 0x16, 0xa5, 0x25, 0xdd, 0x7c, 0x8d, 0x92, - 0x5e, 0x52, 0x66, 0x6a, 0xc3, 0xfe, 0x8b, 0x01, 0x9b, 0x0f, 0x9e, 0xc5, 0x34, 0x61, 0x18, 0xfd, - 0x4f, 0xf8, 0xe0, 0x14, 0x56, 0x48, 0x05, 0x4f, 0x58, 0x8d, 0xdd, 0xc6, 0x5e, 0xfb, 0xe0, 0x6d, - 0x27, 0x25, 0x27, 0xa7, 0xe0, 0xac, 0x8c, 0xa0, 0x9c, 0xaa, 0x77, 0xb7, 0x6e, 0x7b, 0x7f, 0xde, - 0x32, 0xec, 0xbf, 0x19, 0x70, 0x57, 0xdd, 0xf4, 0x80, 0xb8, 0xe4, 0x39, 0xe2, 0xfe, 0x31, 0x09, - 0xd9, 0x48, 0xfc, 0xe0, 0x38, 0x6d, 0x58, 0xf1, 0x35, 0x92, 0x27, 0x99, 0x87, 0x7c, 0x5f, 0xc7, - 0xa9, 0x75, 0x94, 0xf0, 0x82, 0x1d, 0xfa, 0xbe, 0xb9, 0x07, 0xeb, 0xa5, 0x0e, 0x57, 0xf9, 0x54, - 0xd7, 0xac, 0xd4, 0x56, 0x73, 0x35, 0x9d, 0x65, 0x62, 0xff, 0xdb, 0x80, 0xf5, 0x4f, 0x02, 0xd6, - 0x47, 0xc1, 0x79, 0x80, 0xc4, 0x50, 0x55, 0xd9, 0x58, 0xa5, 0x87, 0x93, 0xac, 0xbd, 0x75, 0x78, - 0x33, 0xa7, 0x47, 0x99, 0x69, 0xc2, 0xf9, 0x08, 0x6e, 0x17, 0x0d, 0x57, 0x54, 0x81, 0x3e, 0xcd, - 0xd1, 0xc6, 0x8b, 0xef, 0x76, 0xd6, 0xf2, 0x62, 0xeb, 0xe9, 0x8a, 0x38, 0x76, 0xd7, 0x70, 0x4d, - 0xe0, 0x9b, 0x1d, 0x68, 0xd3, 0x3e, 0xf6, 0x04, 0x79, 0xe6, 0x85, 0xf1, 0x48, 0x17, 0x50, 0xd3, - 0x6d, 0xd1, 0x3e, 0x3e, 0x27, 0xcf, 0x3e, 0x8b, 0x47, 0xe6, 0xfb, 0xf0, 0x66, 0x3e, 0x58, 0xbd, - 0x04, 0x05, 0x9e, 0xb2, 0x57, 0xd7, 0xc1, 0x75, 0x3d, 0x2d, 0xbb, 0x1b, 0xf9, 0xee, 0x25, 0x0a, - 0x94, 0xb3, 0x43, 0xdf, 0xe7, 0xf6, 0x3f, 0x16, 0x60, 0xf1, 0x0c, 0x71, 0x34, 0x12, 0xe6, 0x05, - 0xac, 0x49, 0x32, 0x8a, 0x02, 0x24, 0x89, 0x97, 0x92, 0x79, 0x76, 0xd2, 0x77, 0x35, 0xc9, 0x57, - 0x87, 0xa0, 0x53, 0x19, 0x7b, 0xc9, 0xbe, 0xd3, 0xd3, 0xd2, 0x73, 0x89, 0x24, 0x71, 0x57, 0x73, - 0x8c, 0x54, 0x68, 0x7e, 0x00, 0x96, 0xe4, 0xb1, 0x90, 0x25, 0xcd, 0x96, 0xfc, 0x92, 0xe6, 0xf2, - 0xcd, 0x7c, 0x3f, 0x65, 0xa6, 0x82, 0x57, 0x6e, 0x66, 0xd4, 0xc6, 0x0f, 0x61, 0xd4, 0x73, 0xd8, - 0x50, 0xe3, 0x68, 0x12, 0xb3, 0x39, 0x3b, 0xe6, 0x6d, 0x65, 0x5f, 0x07, 0xfd, 0x1c, 0xcc, 0x44, - 0xe0, 0x49, 0xcc, 0x85, 0xd7, 0x88, 0x33, 0x11, 0xb8, 0x0e, 0xe9, 0xc3, 0xb6, 0x50, 0xc5, 0xe7, - 0x8d, 0x88, 0xd4, 0xfc, 0x1c, 0x05, 0x24, 0xa4, 0x62, 0x98, 0x83, 0x2f, 0xce, 0x0e, 0xbe, 0xa5, - 0x81, 0x3e, 0x55, 0x38, 0x6e, 0x0e, 0x93, 0x79, 0xe9, 0x41, 0xe7, 0x66, 0x2f, 0x45, 0x82, 0x6e, - 0xe9, 0x04, 0xfd, 0xe8, 0x06, 0x88, 0x22, 0x4b, 0x02, 0xde, 0xa9, 0xcc, 0x11, 0xd5, 0xd5, 0x9e, - 0x6e, 0x28, 0x8f, 0x93, 0x81, 0x22, 0x5b, 0x94, 0x8e, 0x14, 0x42, 0x8a, 0x59, 0x98, 0xb1, 0x87, - 0x7a, 0xda, 0x14, 0xcc, 0xd1, 0x63, 0x34, 0xcc, 0x1e, 0x0c, 0x76, 0x39, 0x6e, 0x0a, 0x8e, 0x70, - 0x2b, 0x58, 0x1f, 0x13, 0xf2, 0xa8, 0xb9, 0xb4, 0xb4, 0xde, 0xb2, 0x7f, 0x06, 0x2d, 0xdd, 0xa2, - 0x87, 0xf8, 0x4a, 0x98, 0xdb, 0xd0, 0x52, 0xb5, 0x4e, 0x84, 0x20, 0xc2, 0x32, 0x74, 0x67, 0x97, - 0x02, 0x5b, 0xc2, 0xd6, 0xb4, 0xe7, 0x92, 0x30, 0x9f, 0xc0, 0xad, 0x88, 0xe8, 0x59, 0xae, 0x0d, - 0xdb, 0x07, 0x1f, 0x3a, 0x33, 0xbc, 0x5c, 0x9d, 0x69, 0x80, 0x6e, 0x8e, 0x66, 0xf3, 0xf2, 0x91, - 0x36, 0x31, 0x42, 0x84, 0x79, 0x39, 0xe9, 0xf4, 0x57, 0xaf, 0xe5, 0x74, 0x02, 0xaf, 0xf4, 0xf9, - 0x2e, 0xb4, 0x0f, 0xd3, 0x63, 0xff, 0x9a, 0x0a, 0x79, 0xfd, 0x5a, 0x96, 0xab, 0xd7, 0xf2, 0x08, - 0x56, 0xb3, 0xc9, 0x77, 0xc1, 0x34, 0xcd, 0x98, 0x3f, 0x06, 0xc8, 0x46, 0xa6, 0xa2, 0xa7, 0x94, - 0x88, 0x5b, 0x99, 0xe4, 0xc4, 0xaf, 0x4d, 0xb0, 0xf9, 0xda, 0x04, 0xb3, 0x5d, 0x58, 0xbb, 0x14, - 0xf8, 0x37, 0xf9, 0xb3, 0xe8, 0x71, 0x24, 0xcc, 0x37, 0x60, 0x51, 0x75, 0x46, 0x06, 0xd4, 0x74, - 0x17, 0x12, 0x81, 0x4f, 0x34, 0x17, 0x97, 0x4f, 0x2f, 0x16, 0x79, 0xd4, 0x17, 0xd6, 0xfc, 0x6e, - 0x63, 0xaf, 0xe9, 0xae, 0xc6, 0xa5, 0xf9, 0x89, 0x2f, 0xec, 0xdf, 0x42, 0xbb, 0x02, 0x68, 0xae, - 0xc2, 0x7c, 0x81, 0x35, 0x4f, 0x7d, 0xf3, 0x3e, 0x6c, 0x95, 0x40, 0x75, 0x72, 0x4d, 0x11, 0x5b, - 0xee, 0x9d, 0x42, 0xa1, 0xc6, 0xaf, 0xc2, 0x7e, 0x0c, 0x9b, 0x27, 0x65, 0x2b, 0x17, 0xd4, 0x5d, - 0x3b, 0xa1, 0x51, 0x9f, 0xd1, 0xdb, 0xd0, 0x2a, 0x7e, 0x5f, 0xe8, 0xd3, 0x37, 0xdd, 0x52, 0x60, - 0x8f, 0x60, 0xfd, 0x52, 0xe0, 0x73, 0x12, 0xfa, 0x25, 0xd8, 0x94, 0x0b, 0x38, 0x9a, 0x04, 0x9a, - 0xf9, 0xfd, 0x5a, 0xba, 0x63, 0xb0, 0x75, 0x89, 0x02, 0xea, 0x23, 0xc9, 0xf8, 0x39, 0x91, 0xe9, - 0x58, 0x3d, 0x43, 0xf8, 0x8a, 0x48, 0x61, 0xba, 0xd0, 0x0c, 0xa8, 0x90, 0x59, 0x65, 0x7d, 0x30, - 0xb5, 0xb2, 0x92, 0x7d, 0x67, 0x1a, 0xc8, 0x31, 0x92, 0x28, 0xeb, 0x48, 0x8d, 0x65, 0xff, 0x14, - 0x36, 0x3e, 0x45, 0x32, 0xe6, 0xc4, 0xaf, 0xe5, 0x78, 0x1d, 0x1a, 0x2a, 0x7f, 0x86, 0xce, 0x9f, - 0x5a, 0xaa, 0x29, 0x6f, 0x3d, 0xf8, 0x22, 0x62, 0x5c, 0x12, 0xff, 0xda, 0x8d, 0xbc, 0xe2, 0x7a, - 0xaf, 0x60, 0x43, 0x5d, 0x96, 0x20, 0xa1, 0xef, 0x15, 0xe7, 0x4c, 0xf3, 0xd8, 0x3e, 0xf8, 0xe5, - 0x4c, 0xdd, 0x31, 0xe9, 0x2e, 0x3b, 0xc0, 0xed, 0x64, 0x42, 0x2e, 0xec, 0x3f, 0x19, 0x60, 0x9d, - 0x92, 0xf1, 0xa1, 0x10, 0x74, 0x10, 0x8e, 0x48, 0x28, 0x15, 0xb3, 0x21, 0x4c, 0xd4, 0xd2, 0x7c, - 0x0b, 0x56, 0x8a, 0x49, 0xaa, 0x07, 0xa8, 0xa1, 0x07, 0xe8, 0x72, 0x2e, 0x54, 0x0d, 0x66, 0xde, - 0x07, 0x88, 0x38, 0x49, 0x3c, 0xec, 0x5d, 0x91, 0x71, 0x96, 0xc5, 0xed, 0xea, 0x60, 0x4c, 0x7f, - 0xfd, 0x39, 0x67, 0x71, 0x3f, 0xa0, 0xf8, 0x94, 0x8c, 0xdd, 0x25, 0xa5, 0xdf, 0x3b, 0x25, 0x63, - 0xf5, 0xd2, 0x89, 0xd8, 0x73, 0xc2, 0xf5, 0x34, 0x6b, 0xb8, 0xe9, 0x87, 0xfd, 0x67, 0x03, 0xee, - 0x14, 0xe9, 0xc8, 0xcb, 0xf5, 0x2c, 0xee, 0x2b, 0x8b, 0x57, 0xdc, 0xdb, 0xb5, 0x68, 0xe7, 0x6f, - 0x88, 0xf6, 0x23, 0x58, 0x2e, 0x1a, 0x44, 0xc5, 0xdb, 0x98, 0x21, 0xde, 0x76, 0x6e, 0x71, 0x4a, - 0xc6, 0xf6, 0x1f, 0x2a, 0xb1, 0x1d, 0x8d, 0x2b, 0xdc, 0xc7, 0xff, 0x4b, 0x6c, 0x85, 0xdb, 0x6a, - 0x6c, 0xb8, 0x6a, 0x7f, 0xed, 0x00, 0x8d, 0xeb, 0x07, 0xb0, 0xff, 0x6a, 0xc0, 0x66, 0xd5, 0xab, - 0xb8, 0x60, 0x67, 0x3c, 0x0e, 0xc9, 0xab, 0xbc, 0x97, 0xed, 0x37, 0x5f, 0x6d, 0xbf, 0x27, 0xb0, - 0x5a, 0x0b, 0x4a, 0x64, 0xb7, 0xf1, 0xf3, 0x99, 0x6a, 0xac, 0xc2, 0xae, 0xee, 0x4a, 0xf5, 0x1c, - 0xe2, 0xe8, 0xc9, 0x37, 0x2f, 0x3a, 0xc6, 0xb7, 0x2f, 0x3a, 0xc6, 0xbf, 0x5e, 0x74, 0x8c, 0x2f, - 0x5f, 0x76, 0xe6, 0xbe, 0x7d, 0xd9, 0x99, 0xfb, 0xe7, 0xcb, 0xce, 0xdc, 0xef, 0x3e, 0x1c, 0x50, - 0x39, 0x8c, 0xfb, 0x0e, 0x66, 0xa3, 0x6e, 0xf6, 0xd3, 0xbe, 0xf4, 0xf5, 0x5e, 0xf1, 0xbf, 0x47, - 0xf2, 0x8b, 0xee, 0x17, 0xf5, 0x7f, 0x55, 0xe4, 0x38, 0x22, 0xa2, 0xbf, 0xa8, 0x59, 0xe1, 0xfd, - 0xff, 0x04, 0x00, 0x00, 0xff, 0xff, 0x9f, 0x08, 0x80, 0x3d, 0x86, 0x11, 0x00, 0x00, + // 1755 bytes of a gzipped FileDescriptorProto + 0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0xb4, 0x58, 0x4f, 0x6f, 0x1b, 0xc7, + 0x15, 0xd7, 0x92, 0x94, 0x2c, 0x3e, 0x4a, 0x94, 0xb4, 0x52, 0xe2, 0x95, 0xab, 0x52, 0xf2, 0xa6, + 0x49, 0x55, 0xa4, 0x59, 0x56, 0x4a, 0x0b, 0x04, 0x46, 0x83, 0x40, 0xa2, 0x9c, 0x58, 0x56, 0x12, + 0x2b, 0x2b, 0x55, 0x46, 0xdb, 0xc3, 0x62, 0x38, 0x3b, 0x26, 0x07, 0x5a, 0xee, 0xac, 0x67, 0x66, + 0xd7, 0xe1, 0xa5, 0xe7, 0x1e, 0xd3, 0x5b, 0xd0, 0x4b, 0xd3, 0x02, 0x3d, 0xf7, 0x6b, 0xe4, 0x98, + 0x63, 0x4f, 0x49, 0x61, 0x1f, 0xfb, 0x25, 0x8a, 0x99, 0xfd, 0x4b, 0x4a, 0x72, 0x69, 0xb8, 0xbd, + 0xcd, 0xbe, 0x79, 0xef, 0xf7, 0xfe, 0xbf, 0x37, 0x24, 0xec, 0xd3, 0x50, 0x12, 0x8e, 0x87, 0x88, + 0x86, 0x9e, 0x20, 0x38, 0xe6, 0x54, 0x8e, 0xbb, 0x18, 0x27, 0xdd, 0x88, 0xb3, 0x84, 0xfa, 0x84, + 0x77, 0x93, 0xbd, 0xe2, 0xec, 0x44, 0x9c, 0x49, 0x66, 0xbe, 0x75, 0x8d, 0x8c, 0x83, 0x71, 0xe2, + 0x14, 0x7c, 0xc9, 0xde, 0x9d, 0xb7, 0x6f, 0x02, 0x4e, 0xf6, 0xba, 0xcf, 0x28, 0x27, 0x29, 0xd6, + 0x9d, 0x8d, 0x01, 0x1b, 0x30, 0x7d, 0xec, 0xaa, 0x53, 0x46, 0xdd, 0x1e, 0x30, 0x36, 0x08, 0x48, + 0x57, 0x7f, 0xf5, 0xe3, 0x27, 0x5d, 0x49, 0x47, 0x44, 0x48, 0x34, 0x8a, 0x32, 0x86, 0xce, 0x34, + 0x83, 0x1f, 0x73, 0x24, 0x29, 0x0b, 0x73, 0x00, 0xda, 0xc7, 0x5d, 0xcc, 0x38, 0xe9, 0xe2, 0x80, + 0x92, 0x50, 0x2a, 0xad, 0xe9, 0x29, 0x63, 0xe8, 0x2a, 0x86, 0x80, 0x0e, 0x86, 0x32, 0x25, 0x8b, + 0xae, 0x24, 0xa1, 0x4f, 0xf8, 0x88, 0xa6, 0xcc, 0xe5, 0x57, 0x26, 0xb0, 0x55, 0xb9, 0xc7, 0x7c, + 0x1c, 0x49, 0xd6, 0xbd, 0x24, 0x63, 0x91, 0xdd, 0xbe, 0x83, 0x99, 0x18, 0x31, 0xd1, 0x25, 0xca, + 0xff, 0x10, 0x93, 0x6e, 0xb2, 0xd7, 0x27, 0x12, 0xed, 0x15, 0x84, 0xdc, 0xee, 0x8c, 0xaf, 0x8f, + 0x44, 0xc9, 0x83, 0x19, 0xcd, 0xec, 0xb6, 0x7f, 0x58, 0x00, 0xab, 0xc7, 0x42, 0x11, 0x8f, 0x08, + 0x3f, 0xf0, 0x7d, 0xaa, 0x5c, 0x3a, 0xe5, 0x2c, 0x62, 0x02, 0x05, 0xe6, 0x06, 0xcc, 0x4b, 0x2a, + 0x03, 0x62, 0x19, 0x3b, 0xc6, 0x6e, 0xd3, 0x4d, 0x3f, 0xcc, 0x1d, 0x68, 0xf9, 0x44, 0x60, 0x4e, + 0x23, 0xc5, 0x6c, 0xd5, 0xf4, 0x5d, 0x95, 0x64, 0x6e, 0xc2, 0x62, 0x9a, 0x07, 0xea, 0x5b, 0x75, + 0x7d, 0x7d, 0x4b, 0x7f, 0x1f, 0xfb, 0xe6, 0x27, 0xd0, 0xa6, 0x21, 0x95, 0x14, 0x05, 0xde, 0x90, + 0xa8, 0x68, 0x58, 0x8d, 0x1d, 0x63, 0xb7, 0xb5, 0x7f, 0xc7, 0xa1, 0x7d, 0xec, 0xa8, 0x00, 0x3a, + 0x59, 0xd8, 0x92, 0x3d, 0xe7, 0x81, 0xe6, 0x38, 0x6c, 0x7c, 0xfb, 0xfd, 0xf6, 0x9c, 0xbb, 0x9c, + 0xc9, 0xa5, 0x44, 0xf3, 0x2e, 0x2c, 0x0d, 0x48, 0x48, 0x04, 0x15, 0xde, 0x10, 0x89, 0xa1, 0x35, + 0xbf, 0x63, 0xec, 0x2e, 0xb9, 0xad, 0x8c, 0xf6, 0x00, 0x89, 0xa1, 0xb9, 0x0d, 0xad, 0x3e, 0x0d, + 0x11, 0x1f, 0xa7, 0x1c, 0x0b, 0x9a, 0x03, 0x52, 0x92, 0x66, 0xe8, 0x01, 0x88, 0x08, 0x3d, 0x0b, + 0x3d, 0x95, 0x6d, 0xeb, 0x56, 0x66, 0x48, 0x9a, 0x69, 0x27, 0xcf, 0xb4, 0x73, 0x9e, 0x97, 0xc2, + 0xe1, 0xa2, 0x32, 0xe4, 0xab, 0x1f, 0xb6, 0x0d, 0xb7, 0xa9, 0xe5, 0xd4, 0x8d, 0xf9, 0x39, 0xac, + 0xc6, 0x61, 0x9f, 0x85, 0x3e, 0x0d, 0x07, 0x5e, 0x44, 0x38, 0x65, 0xbe, 0xb5, 0xa8, 0xa1, 0x36, + 0xaf, 0x40, 0x1d, 0x65, 0x45, 0x93, 0x22, 0x7d, 0xad, 0x90, 0x56, 0x0a, 0xe1, 0x53, 0x2d, 0x6b, + 0x7e, 0x01, 0x26, 0xc6, 0x89, 0x36, 0x89, 0xc5, 0x32, 0x47, 0x6c, 0xce, 0x8e, 0xb8, 0x8a, 0x71, + 0x72, 0x9e, 0x4a, 0x67, 0x90, 0xbf, 0x87, 0xdb, 0x92, 0xa3, 0x50, 0x3c, 0x21, 0x7c, 0x1a, 0x17, + 0x66, 0xc7, 0x7d, 0x23, 0xc7, 0x98, 0x04, 0x7f, 0x00, 0x3b, 0x38, 0x2b, 0x20, 0x8f, 0x13, 0x9f, + 0x0a, 0xc9, 0x69, 0x3f, 0x56, 0xb2, 0xde, 0x13, 0x8e, 0xb0, 0xae, 0x91, 0x96, 0x2e, 0x82, 0x4e, + 0xce, 0xe7, 0x4e, 0xb0, 0x7d, 0x9c, 0x71, 0x99, 0x8f, 0xe0, 0x27, 0xfd, 0x80, 0xe1, 0x4b, 0xa1, + 0x8c, 0xf3, 0x26, 0x90, 0xb4, 0xea, 0x11, 0x15, 0x42, 0xa1, 0x2d, 0xed, 0x18, 0xbb, 0x75, 0xf7, + 0x6e, 0xca, 0x7b, 0x4a, 0xf8, 0x51, 0x85, 0xf3, 0xbc, 0xc2, 0x68, 0xbe, 0x07, 0xe6, 0x90, 0x0a, + 0xc9, 0x38, 0xc5, 0x28, 0xf0, 0x48, 0x28, 0x39, 0x25, 0xc2, 0x5a, 0xd6, 0xe2, 0x6b, 0xe5, 0xcd, + 0xfd, 0xf4, 0xc2, 0x7c, 0x08, 0x77, 0x6f, 0x54, 0xea, 0xe1, 0x21, 0x0a, 0x43, 0x12, 0x58, 0x6d, + 0xed, 0xca, 0xb6, 0x7f, 0x83, 0xce, 0x5e, 0xca, 0x76, 0x6f, 0xf1, 0x8f, 0xdf, 0x6c, 0xcf, 0x7d, + 0xfd, 0xcd, 0xf6, 0x9c, 0xfd, 0x0f, 0x03, 0x6e, 0xf7, 0x0a, 0xc7, 0x47, 0x2c, 0x41, 0xc1, 0xff, + 0xb3, 0xc1, 0x0e, 0xa0, 0x29, 0x24, 0x8b, 0xd2, 0x92, 0x6e, 0xbc, 0x42, 0x49, 0x2f, 0x2a, 0x31, + 0x75, 0x61, 0xff, 0xc5, 0x80, 0x8d, 0xfb, 0x4f, 0x63, 0x9a, 0x30, 0x8c, 0xfe, 0x27, 0xf3, 0xe0, + 0x04, 0x96, 0x49, 0x05, 0x4f, 0x58, 0xf5, 0x9d, 0xfa, 0x6e, 0x6b, 0xff, 0x6d, 0x27, 0x1d, 0x4e, + 0x4e, 0x31, 0xb3, 0xb2, 0x01, 0xe5, 0x54, 0xb5, 0xbb, 0x93, 0xb2, 0xf7, 0x6a, 0x96, 0x61, 0xff, + 0xcd, 0x80, 0x3b, 0x2a, 0xd2, 0x03, 0xe2, 0x92, 0x67, 0x88, 0xfb, 0x47, 0x24, 0x64, 0x23, 0xf1, + 0xda, 0x76, 0xda, 0xb0, 0xec, 0x6b, 0x24, 0x4f, 0x32, 0x0f, 0xf9, 0xbe, 0xb6, 0x53, 0xf3, 0x28, + 0xe2, 0x39, 0x3b, 0xf0, 0x7d, 0x73, 0x17, 0x56, 0x4b, 0x1e, 0xae, 0xf2, 0xa9, 0xc2, 0xac, 0xd8, + 0xda, 0x39, 0x9b, 0xce, 0x32, 0xb1, 0xff, 0x6d, 0xc0, 0xea, 0x27, 0x01, 0xeb, 0xa3, 0xe0, 0x2c, + 0x40, 0x62, 0xa8, 0xaa, 0x6c, 0xac, 0xd2, 0xc3, 0x49, 0xd6, 0xde, 0xda, 0xbc, 0x99, 0xd3, 0xa3, + 0xc4, 0xf4, 0xc0, 0xf9, 0x08, 0xd6, 0x8a, 0x86, 0x2b, 0xaa, 0x40, 0x7b, 0x73, 0xb8, 0xfe, 0xfc, + 0xfb, 0xed, 0x95, 0xbc, 0xd8, 0x7a, 0xba, 0x22, 0x8e, 0xdc, 0x15, 0x3c, 0x41, 0xf0, 0xcd, 0x0e, + 0xb4, 0x68, 0x1f, 0x7b, 0x82, 0x3c, 0xf5, 0xc2, 0x78, 0xa4, 0x0b, 0xa8, 0xe1, 0x36, 0x69, 0x1f, + 0x9f, 0x91, 0xa7, 0x9f, 0xc7, 0x23, 0xf3, 0x7d, 0x78, 0x33, 0x5f, 0xac, 0x5e, 0x82, 0x02, 0x4f, + 0xc9, 0xab, 0x70, 0x70, 0x5d, 0x4f, 0x4b, 0xee, 0x7a, 0x7e, 0x7b, 0x81, 0x02, 0xa5, 0xec, 0xc0, + 0xf7, 0xb9, 0xfd, 0x62, 0x1e, 0x16, 0x4e, 0x11, 0x47, 0x23, 0x61, 0x9e, 0xc3, 0x8a, 0x24, 0xa3, + 0x28, 0x40, 0x92, 0x78, 0xe9, 0x30, 0xcf, 0x3c, 0x7d, 0x57, 0x0f, 0xf9, 0xea, 0x12, 0x74, 0x2a, + 0x6b, 0x2f, 0xd9, 0x73, 0x7a, 0x9a, 0x7a, 0x26, 0x91, 0x24, 0x6e, 0x3b, 0xc7, 0x48, 0x89, 0xe6, + 0x07, 0x60, 0x49, 0x1e, 0x0b, 0x59, 0x8e, 0xd9, 0x72, 0xbe, 0xa4, 0xb9, 0x7c, 0x33, 0xbf, 0x4f, + 0x27, 0x53, 0x31, 0x57, 0xae, 0x9f, 0xa8, 0xf5, 0xd7, 0x99, 0xa8, 0x67, 0xb0, 0xae, 0xd6, 0xd1, + 0x34, 0x66, 0x63, 0x76, 0xcc, 0x35, 0x25, 0x3f, 0x09, 0xfa, 0x05, 0x98, 0x89, 0xc0, 0xd3, 0x98, + 0xf3, 0xaf, 0x60, 0x67, 0x22, 0xf0, 0x24, 0xa4, 0x0f, 0x5b, 0x42, 0x15, 0x9f, 0x37, 0x22, 0x52, + 0xcf, 0xe7, 0x28, 0x20, 0x21, 0x15, 0xc3, 0x1c, 0x7c, 0x61, 0x76, 0xf0, 0x4d, 0x0d, 0xf4, 0x99, + 0xc2, 0x71, 0x73, 0x98, 0x4c, 0x4b, 0x0f, 0x3a, 0xd7, 0x6b, 0x29, 0x12, 0x74, 0x4b, 0x27, 0xe8, + 0x47, 0xd7, 0x40, 0x14, 0x59, 0x12, 0xf0, 0x4e, 0x65, 0x8f, 0xa8, 0xae, 0xf6, 0x74, 0x43, 0x79, + 0x9c, 0x0c, 0xd4, 0xb0, 0x45, 0xe9, 0x4a, 0x21, 0xa4, 0xd8, 0x85, 0xd9, 0xf4, 0x50, 0x4f, 0x9b, + 0x62, 0x72, 0xf4, 0x18, 0x0d, 0xb3, 0x07, 0x83, 0x5d, 0xae, 0x9b, 0x62, 0x46, 0xb8, 0x15, 0xac, + 0x8f, 0x09, 0x51, 0xdd, 0x5c, 0x59, 0x39, 0x24, 0x62, 0x78, 0xa8, 0x57, 0x62, 0xdd, 0x6d, 0x17, + 0xeb, 0xe5, 0xbe, 0xa2, 0x3e, 0x6c, 0x2c, 0x2e, 0xae, 0x36, 0xed, 0x9f, 0x41, 0x53, 0x37, 0xf3, + 0x01, 0xbe, 0x14, 0xe6, 0x16, 0x34, 0x55, 0x57, 0x10, 0x21, 0x88, 0xb0, 0x0c, 0x3d, 0x03, 0x4a, + 0x82, 0x2d, 0x61, 0xf3, 0xa6, 0x87, 0x95, 0x30, 0x1f, 0xc3, 0xad, 0x88, 0xe8, 0xad, 0xaf, 0x05, + 0x5b, 0xfb, 0x1f, 0x3a, 0x33, 0xbc, 0x71, 0x9d, 0x9b, 0x00, 0xdd, 0x1c, 0xcd, 0xe6, 0xe5, 0x73, + 0x6e, 0x6a, 0xd9, 0x08, 0xf3, 0x62, 0x5a, 0xe9, 0xaf, 0x5f, 0x49, 0xe9, 0x14, 0x5e, 0xa9, 0xf3, + 0x5d, 0x68, 0x1d, 0xa4, 0x6e, 0x7f, 0x4a, 0x85, 0xbc, 0x1a, 0x96, 0xa5, 0x6a, 0x58, 0x1e, 0x42, + 0x3b, 0xdb, 0x91, 0xe7, 0x4c, 0x0f, 0x24, 0xf3, 0xc7, 0x00, 0xd9, 0x72, 0x55, 0x83, 0x2c, 0x1d, + 0xd9, 0xcd, 0x8c, 0x72, 0xec, 0x4f, 0xec, 0xba, 0xda, 0xc4, 0xae, 0xb3, 0x5d, 0x58, 0xb9, 0x10, + 0xf8, 0x37, 0xf9, 0x03, 0xea, 0x51, 0x24, 0xcc, 0x37, 0x60, 0x41, 0xf5, 0x50, 0x06, 0xd4, 0x70, + 0xe7, 0x13, 0x81, 0x8f, 0xf5, 0xd4, 0x2e, 0x1f, 0x69, 0x2c, 0xf2, 0xa8, 0x2f, 0xac, 0xda, 0x4e, + 0x7d, 0xb7, 0xe1, 0xb6, 0xe3, 0x52, 0xfc, 0xd8, 0x17, 0xf6, 0x6f, 0xa1, 0x55, 0x01, 0x34, 0xdb, + 0x50, 0x2b, 0xb0, 0x6a, 0xd4, 0x37, 0xef, 0xc1, 0x66, 0x09, 0x34, 0x39, 0x86, 0x53, 0xc4, 0xa6, + 0x7b, 0xbb, 0x60, 0x98, 0x98, 0xc4, 0xc2, 0x7e, 0x04, 0x1b, 0xc7, 0x65, 0xd3, 0x17, 0x43, 0x7e, + 0xc2, 0x43, 0x63, 0x72, 0x9b, 0x6f, 0x41, 0xb3, 0xf8, 0x25, 0xa2, 0xbd, 0x6f, 0xb8, 0x25, 0xc1, + 0x1e, 0xc1, 0xea, 0x85, 0xc0, 0x67, 0x24, 0xf4, 0x4b, 0xb0, 0x1b, 0x02, 0x70, 0x38, 0x0d, 0x34, + 0xf3, 0x4b, 0xb7, 0x54, 0xc7, 0x60, 0xf3, 0x02, 0x05, 0xd4, 0x47, 0x92, 0xf1, 0x33, 0x22, 0xd3, + 0x05, 0x7c, 0x8a, 0xf0, 0x25, 0x91, 0xc2, 0x74, 0xa1, 0x11, 0x50, 0x21, 0xb3, 0xca, 0xfa, 0xe0, + 0xc6, 0xca, 0x4a, 0xf6, 0x9c, 0x9b, 0x40, 0x8e, 0x90, 0x44, 0x59, 0xef, 0x6a, 0x2c, 0xfb, 0xa7, + 0xb0, 0xfe, 0x19, 0x92, 0x31, 0x27, 0xfe, 0x44, 0x8e, 0x57, 0xa1, 0xae, 0xf2, 0x67, 0xe8, 0xfc, + 0xa9, 0xa3, 0x7a, 0x0f, 0x58, 0xf7, 0xbf, 0x8c, 0x18, 0x97, 0xc4, 0xbf, 0x12, 0x91, 0x97, 0x84, + 0xf7, 0x12, 0xd6, 0x55, 0xb0, 0x04, 0x09, 0x7d, 0xaf, 0xf0, 0x33, 0xcd, 0x63, 0x6b, 0xff, 0x57, + 0x33, 0x75, 0xc7, 0xb4, 0xba, 0xcc, 0x81, 0xb5, 0x64, 0x8a, 0x2e, 0xec, 0x3f, 0x19, 0x60, 0x9d, + 0x90, 0xf1, 0x81, 0x10, 0x74, 0x10, 0x8e, 0x48, 0x28, 0xd5, 0x0c, 0x44, 0x98, 0xa8, 0xa3, 0xf9, + 0x16, 0x2c, 0x17, 0x3b, 0x57, 0xaf, 0x5a, 0x43, 0xaf, 0xda, 0xa5, 0x9c, 0xa8, 0x1a, 0xcc, 0xbc, + 0x07, 0x10, 0x71, 0x92, 0x78, 0xd8, 0xbb, 0x24, 0xe3, 0x2c, 0x8b, 0x5b, 0xd5, 0x15, 0x9a, 0xfe, + 0x4e, 0x74, 0x4e, 0xe3, 0x7e, 0x40, 0xf1, 0x09, 0x19, 0xbb, 0x8b, 0x8a, 0xbf, 0x77, 0x42, 0xc6, + 0xea, 0x4d, 0x14, 0xb1, 0x67, 0x84, 0xeb, 0xbd, 0x57, 0x77, 0xd3, 0x0f, 0xfb, 0xcf, 0x06, 0xdc, + 0x2e, 0xd2, 0x91, 0x97, 0xeb, 0x69, 0xdc, 0x57, 0x12, 0x2f, 0x89, 0xdb, 0x15, 0x6b, 0x6b, 0xd7, + 0x58, 0xfb, 0x11, 0x2c, 0x15, 0x0d, 0xa2, 0xec, 0xad, 0xcf, 0x60, 0x6f, 0x2b, 0x97, 0x38, 0x21, + 0x63, 0xfb, 0x0f, 0x15, 0xdb, 0x0e, 0xc7, 0x95, 0xd9, 0xc7, 0xff, 0x8b, 0x6d, 0x85, 0xda, 0xaa, + 0x6d, 0xb8, 0x2a, 0x7f, 0xc5, 0x81, 0xfa, 0x55, 0x07, 0xec, 0xbf, 0x1a, 0xb0, 0x51, 0xd5, 0x2a, + 0xce, 0xd9, 0x29, 0x8f, 0x43, 0xf2, 0x32, 0xed, 0x65, 0xfb, 0xd5, 0xaa, 0xed, 0xf7, 0x18, 0xda, + 0x13, 0x46, 0x89, 0x2c, 0x1a, 0xbf, 0x98, 0xa9, 0xc6, 0x2a, 0xd3, 0xd5, 0x5d, 0xae, 0xfa, 0x21, + 0xec, 0xbf, 0x1b, 0xb0, 0x96, 0xdb, 0x58, 0x04, 0xcb, 0xfc, 0x39, 0x98, 0x85, 0x7b, 0xe5, 0xeb, + 0x2d, 0x2d, 0xa9, 0xd5, 0xfc, 0x26, 0x7f, 0xba, 0x95, 0xa5, 0x51, 0xab, 0x94, 0x86, 0xf9, 0x29, + 0xac, 0x17, 0x26, 0x47, 0x3a, 0x41, 0x33, 0x67, 0xb1, 0x78, 0x9f, 0x16, 0xa4, 0xc3, 0xc7, 0xdf, + 0x3e, 0xef, 0x18, 0xdf, 0x3d, 0xef, 0x18, 0xff, 0x7a, 0xde, 0x31, 0xbe, 0x7a, 0xd1, 0x99, 0xfb, + 0xee, 0x45, 0x67, 0xee, 0x9f, 0x2f, 0x3a, 0x73, 0xbf, 0xfb, 0x70, 0x40, 0xe5, 0x30, 0xee, 0x3b, + 0x98, 0x8d, 0xba, 0xd9, 0x9f, 0x15, 0x65, 0x4c, 0xde, 0x2b, 0xfe, 0xc9, 0x49, 0x7e, 0xd9, 0xfd, + 0x72, 0xf2, 0x7f, 0x22, 0x39, 0x8e, 0x88, 0xe8, 0x2f, 0xe8, 0xe9, 0xf5, 0xfe, 0x7f, 0x02, 0x00, + 0x00, 0xff, 0xff, 0x0a, 0xef, 0x81, 0x2b, 0x58, 0x12, 0x00, 0x00, } func (m *ConsumerAdditionProposal) Marshal() (dAtA []byte, err error) { @@ -1876,6 +1955,11 @@ func (m *Params) MarshalToSizedBuffer(dAtA []byte) (int, error) { _ = i var l int _ = l + if m.BlocksPerEpoch != 0 { + i = encodeVarintProvider(dAtA, i, uint64(m.BlocksPerEpoch)) + i-- + dAtA[i] = 0x50 + } { size, err := m.ConsumerRewardDenomRegistrationFee.MarshalToSizedBuffer(dAtA[:i]) if err != nil { @@ -2585,6 +2669,53 @@ func (m *ConsumerAddrsToPrune) MarshalToSizedBuffer(dAtA []byte) (int, error) { return len(dAtA) - i, nil } +func (m *ConsumerValidator) Marshal() (dAtA []byte, err error) { + size := m.Size() + dAtA = make([]byte, size) + n, err := m.MarshalToSizedBuffer(dAtA[:size]) + if err != nil { + return nil, err + } + return dAtA[:n], nil +} + +func (m *ConsumerValidator) MarshalTo(dAtA []byte) (int, error) { + size := m.Size() + return m.MarshalToSizedBuffer(dAtA[:size]) +} + +func (m *ConsumerValidator) MarshalToSizedBuffer(dAtA []byte) (int, error) { + i := len(dAtA) + _ = i + var l int + _ = l + if m.ConsumerPublicKey != nil { + { + size, err := m.ConsumerPublicKey.MarshalToSizedBuffer(dAtA[:i]) + if err != nil { + return 0, err + } + i -= size + i = encodeVarintProvider(dAtA, i, uint64(size)) + } + i-- + dAtA[i] = 0x1a + } + if m.Power != 0 { + i = encodeVarintProvider(dAtA, i, uint64(m.Power)) + i-- + dAtA[i] = 0x10 + } + if len(m.ProviderConsAddr) > 0 { + i -= len(m.ProviderConsAddr) + copy(dAtA[i:], m.ProviderConsAddr) + i = encodeVarintProvider(dAtA, i, uint64(len(m.ProviderConsAddr))) + i-- + dAtA[i] = 0xa + } + return len(dAtA) - i, nil +} + func encodeVarintProvider(dAtA []byte, offset int, v uint64) int { offset -= sovProvider(v) base := offset @@ -2774,6 +2905,9 @@ func (m *Params) Size() (n int) { } l = m.ConsumerRewardDenomRegistrationFee.Size() n += 1 + l + sovProvider(uint64(l)) + if m.BlocksPerEpoch != 0 { + n += 1 + sovProvider(uint64(m.BlocksPerEpoch)) + } return n } @@ -3053,6 +3187,26 @@ func (m *ConsumerAddrsToPrune) Size() (n int) { return n } +func (m *ConsumerValidator) Size() (n int) { + if m == nil { + return 0 + } + var l int + _ = l + l = len(m.ProviderConsAddr) + if l > 0 { + n += 1 + l + sovProvider(uint64(l)) + } + if m.Power != 0 { + n += 1 + sovProvider(uint64(m.Power)) + } + if m.ConsumerPublicKey != nil { + l = m.ConsumerPublicKey.Size() + n += 1 + l + sovProvider(uint64(l)) + } + return n +} + func sovProvider(x uint64) (n int) { return (math_bits.Len64(x|1) + 6) / 7 } @@ -4507,6 +4661,25 @@ func (m *Params) Unmarshal(dAtA []byte) error { return err } iNdEx = postIndex + case 10: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field BlocksPerEpoch", wireType) + } + m.BlocksPerEpoch = 0 + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowProvider + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + m.BlocksPerEpoch |= int64(b&0x7F) << shift + if b < 0x80 { + break + } + } default: iNdEx = preIndex skippy, err := skipProvider(dAtA[iNdEx:]) @@ -6327,6 +6500,145 @@ func (m *ConsumerAddrsToPrune) Unmarshal(dAtA []byte) error { } return nil } +func (m *ConsumerValidator) Unmarshal(dAtA []byte) error { + l := len(dAtA) + iNdEx := 0 + for iNdEx < l { + preIndex := iNdEx + var wire uint64 + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowProvider + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + wire |= uint64(b&0x7F) << shift + if b < 0x80 { + break + } + } + fieldNum := int32(wire >> 3) + wireType := int(wire & 0x7) + if wireType == 4 { + return fmt.Errorf("proto: ConsumerValidator: wiretype end group for non-group") + } + if fieldNum <= 0 { + return fmt.Errorf("proto: ConsumerValidator: illegal tag %d (wire type %d)", fieldNum, wire) + } + switch fieldNum { + case 1: + if wireType != 2 { + return fmt.Errorf("proto: wrong wireType = %d for field ProviderConsAddr", wireType) + } + var byteLen int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowProvider + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + byteLen |= int(b&0x7F) << shift + if b < 0x80 { + break + } + } + if byteLen < 0 { + return ErrInvalidLengthProvider + } + postIndex := iNdEx + byteLen + if postIndex < 0 { + return ErrInvalidLengthProvider + } + if postIndex > l { + return io.ErrUnexpectedEOF + } + m.ProviderConsAddr = append(m.ProviderConsAddr[:0], dAtA[iNdEx:postIndex]...) + if m.ProviderConsAddr == nil { + m.ProviderConsAddr = []byte{} + } + iNdEx = postIndex + case 2: + if wireType != 0 { + return fmt.Errorf("proto: wrong wireType = %d for field Power", wireType) + } + m.Power = 0 + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowProvider + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + m.Power |= int64(b&0x7F) << shift + if b < 0x80 { + break + } + } + case 3: + if wireType != 2 { + return fmt.Errorf("proto: wrong wireType = %d for field ConsumerPublicKey", wireType) + } + var msglen int + for shift := uint(0); ; shift += 7 { + if shift >= 64 { + return ErrIntOverflowProvider + } + if iNdEx >= l { + return io.ErrUnexpectedEOF + } + b := dAtA[iNdEx] + iNdEx++ + msglen |= int(b&0x7F) << shift + if b < 0x80 { + break + } + } + if msglen < 0 { + return ErrInvalidLengthProvider + } + postIndex := iNdEx + msglen + if postIndex < 0 { + return ErrInvalidLengthProvider + } + if postIndex > l { + return io.ErrUnexpectedEOF + } + if m.ConsumerPublicKey == nil { + m.ConsumerPublicKey = &crypto.PublicKey{} + } + if err := m.ConsumerPublicKey.Unmarshal(dAtA[iNdEx:postIndex]); err != nil { + return err + } + iNdEx = postIndex + default: + iNdEx = preIndex + skippy, err := skipProvider(dAtA[iNdEx:]) + if err != nil { + return err + } + if (skippy < 0) || (iNdEx+skippy) < 0 { + return ErrInvalidLengthProvider + } + if (iNdEx + skippy) > l { + return io.ErrUnexpectedEOF + } + iNdEx += skippy + } + } + + if iNdEx > l { + return io.ErrUnexpectedEOF + } + return nil +} func skipProvider(dAtA []byte) (n int, err error) { l := len(dAtA) iNdEx := 0 From d6041c03dbbb4c78a11332ad20a92edead07af04 Mon Sep 17 00:00:00 2001 From: insumity Date: Tue, 12 Mar 2024 14:50:03 +0100 Subject: [PATCH 2/5] moved changelogs under v4.0.0 --- .../features/provider/1516-introduce-epochs.md | 0 .../state-breaking/provider/1516-introduce-epochs.md | 0 2 files changed, 0 insertions(+), 0 deletions(-) rename .changelog/{unreleased => v4.0.0}/features/provider/1516-introduce-epochs.md (100%) rename .changelog/{unreleased => v4.0.0}/state-breaking/provider/1516-introduce-epochs.md (100%) diff --git a/.changelog/unreleased/features/provider/1516-introduce-epochs.md b/.changelog/v4.0.0/features/provider/1516-introduce-epochs.md similarity index 100% rename from .changelog/unreleased/features/provider/1516-introduce-epochs.md rename to .changelog/v4.0.0/features/provider/1516-introduce-epochs.md diff --git a/.changelog/unreleased/state-breaking/provider/1516-introduce-epochs.md b/.changelog/v4.0.0/state-breaking/provider/1516-introduce-epochs.md similarity index 100% rename from .changelog/unreleased/state-breaking/provider/1516-introduce-epochs.md rename to .changelog/v4.0.0/state-breaking/provider/1516-introduce-epochs.md From ba63a2a191d3cfe43fd4c6ed8f2bc84c3c43930a Mon Sep 17 00:00:00 2001 From: insumity Date: Tue, 12 Mar 2024 15:39:52 +0100 Subject: [PATCH 3/5] mbt from main --- Makefile | 5 +- go.mod | 1 + tests/e2e/config.go | 101 ------ tests/mbt/driver/core.go | 15 +- tests/mbt/driver/generate_more_traces.sh | 3 +- tests/mbt/driver/generate_traces.sh | 3 +- tests/mbt/driver/mbt_test.go | 121 ++++++-- tests/mbt/driver/stats.go | 2 + tests/mbt/model/README.md | 20 +- tests/mbt/model/ccv.qnt | 371 +++++------------------ tests/mbt/model/ccv_boundeddrift.qnt | 14 +- tests/mbt/model/ccv_model.qnt | 40 ++- tests/mbt/model/ccv_test.qnt | 1 + tests/mbt/run_invariants.sh | 2 +- 14 files changed, 247 insertions(+), 452 deletions(-) diff --git a/Makefile b/Makefile index 1b0ddec7f9..fb570aa370 100644 --- a/Makefile +++ b/Makefile @@ -129,9 +129,6 @@ test-trace: # Note: this is *not* using the Quint models to test the system, # this tests/verifies the Quint models *themselves*. verify-models: - quint test tests/mbt/model/ccv_test.qnt;\ - quint test tests/mbt/model/ccv_model.qnt;\ - quint run --invariant "all{ValidatorUpdatesArePropagatedInv,ValidatorSetHasExistedInv,SameVscPacketsInv,MatureOnTimeInv,EventuallyMatureOnProviderInv}" tests/mbt/model/ccv_model.qnt --max-steps 200 --max-samples 200 cd tests/mbt/model;\ ../run_invariants.sh @@ -245,4 +242,4 @@ build-docs: ############################################################################### e2e-traces: - cd tests/e2e; go test -timeout 30s -run ^TestWriteExamples -v \ No newline at end of file + cd tests/e2e; go test -timeout 30s -run ^TestWriteExamples -v diff --git a/go.mod b/go.mod index fd15000bb9..c767d26b2e 100644 --- a/go.mod +++ b/go.mod @@ -169,6 +169,7 @@ require ( require ( github.com/informalsystems/itf-go v0.0.1 github.com/spf13/viper v1.16.0 + golang.org/x/mod v0.11.0 google.golang.org/genproto/googleapis/api v0.0.0-20231212172506-995d672761c0 ) diff --git a/tests/e2e/config.go b/tests/e2e/config.go index 677a68e748..520cf04b10 100644 --- a/tests/e2e/config.go +++ b/tests/e2e/config.go @@ -4,8 +4,6 @@ import ( "fmt" "strconv" "time" - - "golang.org/x/mod/semver" ) var ( @@ -262,105 +260,6 @@ func SlashThrottleTestConfig() TestConfig { return tr } -// CompatibilityTestConfig returns a test configuration for a given version of a consumer and provider -func CompatibilityTestConfig(providerVersion, consumerVersion string) TestConfig { - // Base configuration is the default - testCfg := DefaultTestConfig() - - // get version dependent validator configs - testCfg.validatorConfigs = getValidatorConfigFromVersion(providerVersion, consumerVersion) - - var providerConfig, consumerConfig ChainConfig - if !semver.IsValid(consumerVersion) { - fmt.Println("Using default provider chain config") - consumerConfig = testCfg.chainConfigs[ChainID("consu")] - } else if semver.Compare(consumerVersion, "v3.0.0") < 0 { - fmt.Println("Using consumer chain config for v2.0.0") - consumerConfig = ChainConfig{ - ChainId: ChainID("consu"), - AccountPrefix: "cosmos", - BinaryName: "interchain-security-cd", - IpPrefix: "7.7.8", - VotingWaitTime: 20, - GenesisChanges: ".app_state.gov.voting_params.voting_period = \"20s\" | " + - ".app_state.slashing.params.signed_blocks_window = \"15\" | " + - ".app_state.slashing.params.min_signed_per_window = \"0.500000000000000000\" | " + - ".app_state.slashing.params.downtime_jail_duration = \"2s\" | " + - ".app_state.slashing.params.slash_fraction_downtime = \"0.010000000000000000\"", - } - } else if semver.Compare(consumerVersion, "v4.0.0") < 0 { - fmt.Println("Using consumer chain config for v3.x.x") - consumerConfig = ChainConfig{ - ChainId: ChainID("consu"), - AccountPrefix: "cosmos", - BinaryName: "interchain-security-cd", - IpPrefix: "7.7.8", - VotingWaitTime: 20, - GenesisChanges: ".app_state.gov.params.voting_period = \"20s\" | " + - ".app_state.slashing.params.signed_blocks_window = \"15\" | " + - ".app_state.slashing.params.min_signed_per_window = \"0.500000000000000000\" | " + - ".app_state.slashing.params.downtime_jail_duration = \"60s\" | " + - ".app_state.slashing.params.slash_fraction_downtime = \"0.010000000000000000\"", - } - } else { - fmt.Println("Using default consumer chain config") - consumerConfig = testCfg.chainConfigs[ChainID("consu")] - } - - // Get the provider chain config for a specific version - if !semver.IsValid(providerVersion) { - fmt.Println("Using default provider chain config") - providerConfig = testCfg.chainConfigs[ChainID("provi")] - } else if semver.Compare(providerVersion, "v3.0.0") < 0 { - fmt.Println("Using provider chain config for v2.x.x") - providerConfig = ChainConfig{ - ChainId: ChainID("provi"), - AccountPrefix: "cosmos", - BinaryName: "interchain-security-pd", - IpPrefix: "7.7.7", - VotingWaitTime: 20, - GenesisChanges: ".app_state.gov.voting_params.voting_period = \"20s\" | " + - // Custom slashing parameters for testing validator downtime functionality - // See https://docs.cosmos.network/main/modules/slashing/04_begin_block.html#uptime-tracking - ".app_state.slashing.params.signed_blocks_window = \"10\" | " + - ".app_state.slashing.params.min_signed_per_window = \"0.500000000000000000\" | " + - ".app_state.slashing.params.downtime_jail_duration = \"2s\" | " + - ".app_state.slashing.params.slash_fraction_downtime = \"0.010000000000000000\" | " + - ".app_state.provider.params.slash_meter_replenish_fraction = \"1.0\" | " + // This disables slash packet throttling - ".app_state.provider.params.slash_meter_replenish_period = \"3s\"", - } - } else if semver.Compare(providerVersion, "v4.0.0") <= 0 { - fmt.Println("Using provider chain config for v3.x.x") - providerConfig = ChainConfig{ - ChainId: ChainID("provi"), - AccountPrefix: "cosmos", - BinaryName: "interchain-security-pd", - IpPrefix: "7.7.7", - VotingWaitTime: 20, - GenesisChanges: ".app_state.gov.params.voting_period = \"20s\" | " + - // Custom slashing parameters for testing validator downtime functionality - // See https://docs.cosmos.network/main/modules/slashing/04_begin_block.html#uptime-tracking - ".app_state.slashing.params.signed_blocks_window = \"10\" | " + - ".app_state.slashing.params.min_signed_per_window = \"0.500000000000000000\" | " + - ".app_state.slashing.params.downtime_jail_duration = \"60s\" | " + - ".app_state.slashing.params.slash_fraction_downtime = \"0.010000000000000000\" | " + - ".app_state.provider.params.slash_meter_replenish_fraction = \"1.0\" | " + // This disables slash packet throttling - ".app_state.provider.params.slash_meter_replenish_period = \"3s\"", - } - } else { - fmt.Println("Using default provider chain config") - providerConfig = testCfg.chainConfigs[ChainID("provi")] - } - - testCfg.chainConfigs[ChainID("consu")] = consumerConfig - testCfg.chainConfigs[ChainID("provi")] = providerConfig - testCfg.name = string(CompatibilityTestCfg) - testCfg.containerConfig.InstanceName = fmt.Sprintf("%s_%s-%s", - testCfg.containerConfig.InstanceName, - consumerVersion, providerVersion) - return testCfg -} - func DefaultTestConfig() TestConfig { tr := TestConfig{ name: "default", diff --git a/tests/mbt/driver/core.go b/tests/mbt/driver/core.go index b49e98766a..803cd486f1 100644 --- a/tests/mbt/driver/core.go +++ b/tests/mbt/driver/core.go @@ -20,6 +20,7 @@ import ( abcitypes "github.com/cometbft/cometbft/abci/types" cmttypes "github.com/cometbft/cometbft/types" + "github.com/cometbft/cometbft/proto/tendermint/crypto" appConsumer "github.com/cosmos/interchain-security/v4/app/consumer" appProvider "github.com/cosmos/interchain-security/v4/app/provider" simibc "github.com/cosmos/interchain-security/v4/testutil/simibc" @@ -127,9 +128,13 @@ func (s *Driver) consumerPower(i int64, chain ChainId) (int64, error) { return v.Power, nil } +func (s *Driver) stakingValidator(i int64) (stakingtypes.Validator, bool) { + return s.providerStakingKeeper().GetValidator(s.ctx(PROVIDER), s.validator(i)) +} + // providerPower returns the power(=number of bonded tokens) of the i-th validator on the provider. func (s *Driver) providerPower(i int64) (int64, error) { - v, found := s.providerStakingKeeper().GetValidator(s.ctx(PROVIDER), s.validator(i)) + v, found := s.stakingValidator(i) if !found { return 0, fmt.Errorf("validator with id %v not found on provider", i) } else { @@ -374,6 +379,14 @@ func (s *Driver) setTime(chain ChainId, newTime time.Time) { testChain.App.BeginBlock(abcitypes.RequestBeginBlock{Header: testChain.CurrentHeader}) } +func (s *Driver) AssignKey(chain ChainId, valIndex int64, value crypto.PublicKey) error { + stakingVal, found := s.stakingValidator(valIndex) + if !found { + return fmt.Errorf("validator with id %v not found on provider", valIndex) + } + return s.providerKeeper().AssignConsumerKey(s.providerCtx(), string(chain), stakingVal, value) +} + // DeliverPacketToConsumer delivers a packet from the provider to the given consumer recipient. // It updates the client before delivering the packet. // Since the channel is ordered, the packet that is delivered is the first packet in the outbox. diff --git a/tests/mbt/driver/generate_more_traces.sh b/tests/mbt/driver/generate_more_traces.sh index 9af4da82e9..40589bb83b 100755 --- a/tests/mbt/driver/generate_more_traces.sh +++ b/tests/mbt/driver/generate_more_traces.sh @@ -9,4 +9,5 @@ go run ./... -modelPath=../model/ccv_boundeddrift.qnt -step stepBoundedDrift -in echo "Generating synced traces with maturations" go run ./... -modelPath=../model/ccv_sync.qnt -init initSync -step stepSync -invariant CanReceiveMaturations -traceFolder traces/sync_mat -numTraces 20 -numSteps 300 -numSamples 20 echo "Generating long synced traces without invariants" -go run ./... -modelPath=../model/ccv_sync.qnt -init initSync -step stepSync -traceFolder traces/sync_noinv -numTraces 20 -numSteps 500 -numSamples 1 \ No newline at end of file +go run ./... -modelPath=../model/ccv_sync.qnt -init initSync -step stepSync -traceFolder traces/sync_noinv -numTraces 20 -numSteps 500 -numSamples 1 +go run ./... -modelPath=../model/ccv_boundeddrift.qnt --step stepBoundedDriftKeyAssignment --traceFolder traces/bound_key -numTraces 20 -numSteps 100 -numSamples 20 \ No newline at end of file diff --git a/tests/mbt/driver/generate_traces.sh b/tests/mbt/driver/generate_traces.sh index ca0a6ba973..9f1134fb26 100755 --- a/tests/mbt/driver/generate_traces.sh +++ b/tests/mbt/driver/generate_traces.sh @@ -9,4 +9,5 @@ go run ./... -modelPath=../model/ccv_boundeddrift.qnt -step stepBoundedDrift -in echo "Generating synced traces with maturations" go run ./... -modelPath=../model/ccv_sync.qnt -init initSync -step stepSync -invariant CanReceiveMaturations -traceFolder traces/sync_mat -numTraces 1 -numSteps 300 -numSamples 20 echo "Generating long synced traces without invariants" -go run ./... -modelPath=../model/ccv_sync.qnt -init initSync -step stepSync -traceFolder traces/sync_noinv -numTraces 1 -numSteps 500 -numSamples 1 \ No newline at end of file +go run ./... -modelPath=../model/ccv_sync.qnt -init initSync -step stepSync -traceFolder traces/sync_noinv -numTraces 1 -numSteps 500 -numSamples 1 +go run ./... -modelPath=../model/ccv_boundeddrift.qnt --step stepBoundedDriftKeyAssignment --traceFolder traces/bound_key -numTraces 1 -numSteps 100 -numSamples 20 \ No newline at end of file diff --git a/tests/mbt/driver/mbt_test.go b/tests/mbt/driver/mbt_test.go index 01faa9d089..df748ea11f 100644 --- a/tests/mbt/driver/mbt_test.go +++ b/tests/mbt/driver/mbt_test.go @@ -15,10 +15,13 @@ import ( "github.com/kylelemons/godebug/pretty" "github.com/stretchr/testify/require" - sdktypes "github.com/cosmos/cosmos-sdk/types" - cmttypes "github.com/cometbft/cometbft/types" + tmencoding "github.com/cometbft/cometbft/crypto/encoding" + "github.com/cosmos/interchain-security/v4/testutil/integration" + + sdktypes "github.com/cosmos/cosmos-sdk/types" + providertypes "github.com/cosmos/interchain-security/v4/x/ccv/provider/types" ) @@ -69,6 +72,7 @@ func TestMBT(t *testing.T) { t.Logf("Number of sent packets: %v", stats.numSentPackets) t.Logf("Number of blocks: %v", stats.numBlocks) t.Logf("Number of transactions: %v", stats.numTxs) + t.Logf("Number of key assignments: %v", stats.numKeyAssignments) t.Logf("Average summed block time delta passed per trace: %v", stats.totalBlockTimePassedPerTrace/time.Duration(numTraces)) } @@ -117,6 +121,21 @@ func RunItfTrace(t *testing.T, path string) { t.Log("Chains are: ", chains) + // generate keys that can be assigned on consumers, according to the ConsumerAddresses in the trace + consumerAddressesExpr := params["ConsumerAddresses"].Value.(itf.ListExprType) + + _, _, consumerPrivVals, err := integration.CreateValidators(len(consumerAddressesExpr)) + require.NoError(t, err, "Error creating consumer signers") + + consumerAddrNamesToPrivVals := make(map[string]cmttypes.PrivValidator, len(consumerAddressesExpr)) + realAddrsToModelConsAddrs := make(map[string]string, len(consumerAddressesExpr)) + i := 0 + for address, privVal := range consumerPrivVals { + consumerAddrNamesToPrivVals[consumerAddressesExpr[i].Value.(string)] = privVal + realAddrsToModelConsAddrs[address] = consumerAddressesExpr[i].Value.(string) + i++ + } + // create params struct vscTimeout := time.Duration(params["VscTimeout"].Value.(int64)) * time.Second @@ -145,6 +164,15 @@ func RunItfTrace(t *testing.T, path string) { valSet, addressMap, signers, err := CreateValSet(initialValSet) require.NoError(t, err, "Error creating validator set") + // get the set of signers for consumers: the validator signers, plus signers for the assignable addresses + consumerSigners := make(map[string]cmttypes.PrivValidator, 0) + for consAddr, consPrivVal := range consumerPrivVals { + consumerSigners[consAddr] = consPrivVal + } + for consAddr, signer := range signers { + consumerSigners[consAddr] = signer + } + // get a slice of validators in the right order nodes := make([]*cmttypes.Validator, len(valNames)) for i, valName := range valNames { @@ -286,7 +314,7 @@ func RunItfTrace(t *testing.T, path string) { consumer.Value.(string), modelParams, driver.providerChain().Vals, - signers, + consumerSigners, nodes, valNames, driver.providerChain(), @@ -311,11 +339,8 @@ func RunItfTrace(t *testing.T, path string) { if len(consumersToStart) > 0 && consumer.ChainId == consumersToStart[len(consumersToStart)-1].Value.(string) { continue } - consumerChainId := consumer.ChainId - driver.path(ChainId(consumerChainId)).AddClientHeader(PROVIDER, driver.providerHeader()) - err := driver.path(ChainId(consumerChainId)).UpdateClient(consumerChainId, false) - require.True(t, err == nil, "Error updating client from %v on provider: %v", consumerChainId, err) + UpdateProviderClientOnConsumer(t, driver, consumer.ChainId) } case "EndAndBeginBlockForConsumer": @@ -329,13 +354,12 @@ func RunItfTrace(t *testing.T, path string) { _ = headerBefore driver.endAndBeginBlock(ChainId(consumerChain), 1*time.Nanosecond) + UpdateConsumerClientOnProvider(t, driver, consumerChain) + driver.endAndBeginBlock(ChainId(consumerChain), time.Duration(timeAdvancement)*time.Second-1*time.Nanosecond) // update the client on the provider - consumerHeader := driver.chain(ChainId(consumerChain)).LastHeader - driver.path(ChainId(consumerChain)).AddClientHeader(consumerChain, consumerHeader) - err := driver.path(ChainId(consumerChain)).UpdateClient(PROVIDER, false) - require.True(t, err == nil, "Error updating client from %v on provider: %v", consumerChain, err) + UpdateConsumerClientOnProvider(t, driver, consumerChain) case "DeliverVscPacket": consumerChain := lastAction["consumerChain"].Value.(string) @@ -371,8 +395,26 @@ func RunItfTrace(t *testing.T, path string) { expectError = false driver.DeliverPacketFromConsumer(ChainId(consumerChain), expectError) } - default: + case "KeyAssignment": + consumerChain := lastAction["consumerChain"].Value.(string) + node := lastAction["validator"].Value.(string) + consumerAddr := lastAction["consumerAddr"].Value.(string) + t.Log("KeyAssignment", consumerChain, node, consumerAddr) + stats.numKeyAssignments++ + + valIndex := getIndexOfString(node, valNames) + assignedPrivVal := consumerAddrNamesToPrivVals[consumerAddr] + assignedKey, err := assignedPrivVal.GetPubKey() + require.NoError(t, err, "Error getting pubkey") + + protoPubKey, err := tmencoding.PubKeyToProto(assignedKey) + require.NoError(t, err, "Error converting pubkey to proto") + + error := driver.AssignKey(ChainId(consumerChain), int64(valIndex), protoPubKey) + require.NoError(t, error, "Error assigning key") + + default: log.Fatalf("Error loading trace file %s, step %v: do not know action type %s", path, index, actionKind) } @@ -407,7 +449,7 @@ func RunItfTrace(t *testing.T, path string) { require.Equal(t, modelRunningConsumers, actualRunningConsumers, "Running consumers do not match") // check validator sets - provider current validator set should be the one from the staking keeper - CompareValidatorSets(t, driver, currentModelState, actualRunningConsumers) + CompareValidatorSets(t, driver, currentModelState, actualRunningConsumers, realAddrsToModelConsAddrs) // check times - sanity check that the block times match the ones from the model CompareTimes(driver, actualRunningConsumers, currentModelState, timeOffset) @@ -434,7 +476,27 @@ func RunItfTrace(t *testing.T, path string) { t.Log("🟢 Trace is ok!") } -func CompareValidatorSets(t *testing.T, driver *Driver, currentModelState map[string]itf.Expr, consumers []string) { +func UpdateProviderClientOnConsumer(t *testing.T, driver *Driver, consumerChainId string) { + driver.path(ChainId(consumerChainId)).AddClientHeader(PROVIDER, driver.providerHeader()) + err := driver.path(ChainId(consumerChainId)).UpdateClient(consumerChainId, false) + require.True(t, err == nil, "Error updating client from %v on provider: %v", consumerChainId, err) +} + +func UpdateConsumerClientOnProvider(t *testing.T, driver *Driver, consumerChain string) { + consumerHeader := driver.chain(ChainId(consumerChain)).LastHeader + driver.path(ChainId(consumerChain)).AddClientHeader(consumerChain, consumerHeader) + err := driver.path(ChainId(consumerChain)).UpdateClient(PROVIDER, false) + require.True(t, err == nil, "Error updating client from %v on provider: %v", consumerChain, err) +} + +func CompareValidatorSets( + t *testing.T, + driver *Driver, + currentModelState map[string]itf.Expr, + consumers []string, + // a map from real addresses to the names of those consumer addresses in the model + keyAddrsToModelConsAddrName map[string]string, +) { t.Helper() modelValSet := ValidatorSet(currentModelState, "provider") @@ -458,23 +520,28 @@ func CompareValidatorSets(t *testing.T, driver *Driver, currentModelState map[st pubkey, err := val.ConsPubKey() require.NoError(t, err, "Error getting pubkey") - consAddr := providertypes.NewConsumerConsAddress(sdktypes.ConsAddress(pubkey.Address().Bytes())) + consAddrModelName, ok := keyAddrsToModelConsAddrName[pubkey.Address().String()] + if ok { // the node has a key assigned, use the name of the consumer address in the model + consumerCurValSet[consAddrModelName] = val.Power + } else { // the node doesn't have a key assigned yet, get the validator moniker + consAddr := providertypes.NewConsumerConsAddress(sdktypes.ConsAddress(pubkey.Address().Bytes())) - // the consumer vals right now are CrossChainValidators, for which we don't know their mnemonic - // so we need to find the mnemonic of the consumer val now to enter it by name in the map + // the consumer vals right now are CrossChainValidators, for which we don't know their mnemonic + // so we need to find the mnemonic of the consumer val now to enter it by name in the map - // get the address on the provider that corresponds to the consumer address - providerConsAddr, found := driver.providerKeeper().GetValidatorByConsumerAddr(driver.providerCtx(), consumer, consAddr) - if !found { - providerConsAddr = providertypes.NewProviderConsAddress(consAddr.Address) - } + // get the address on the provider that corresponds to the consumer address + providerConsAddr, found := driver.providerKeeper().GetValidatorByConsumerAddr(driver.providerCtx(), consumer, consAddr) + if !found { + providerConsAddr = providertypes.NewProviderConsAddress(consAddr.Address) + } - // get the validator for that address on the provider - providerVal, found := driver.providerStakingKeeper().GetValidatorByConsAddr(driver.providerCtx(), providerConsAddr.Address) - require.True(t, found, "Error getting provider validator") + // get the validator for that address on the provider + providerVal, found := driver.providerStakingKeeper().GetValidatorByConsAddr(driver.providerCtx(), providerConsAddr.Address) + require.True(t, found, "Error getting provider validator") - // use the moniker of that validator - consumerCurValSet[providerVal.GetMoniker()] = val.Power + // use the moniker of that validator + consumerCurValSet[providerVal.GetMoniker()] = val.Power + } } require.NoError(t, CompareValSet(modelValSet, consumerCurValSet), "Validator sets do not match for consumer %v", consumer) } diff --git a/tests/mbt/driver/stats.go b/tests/mbt/driver/stats.go index 0d397571be..8b4c95a3dd 100644 --- a/tests/mbt/driver/stats.go +++ b/tests/mbt/driver/stats.go @@ -16,4 +16,6 @@ type Stats struct { numTxs int totalBlockTimePassedPerTrace time.Duration + + numKeyAssignments int } diff --git a/tests/mbt/model/README.md b/tests/mbt/model/README.md index f53900e77f..57f5b2eda9 100644 --- a/tests/mbt/model/README.md +++ b/tests/mbt/model/README.md @@ -31,16 +31,25 @@ All the logic in EndBlock/BeginBlock happens here, like updating the validator s * `EndAndBeginBlockForConsumer(chain: Chain, timeAdvancement: Time)`: On the consumer `chain`, ends the current block, and begins a new one. Again, all the logic in EndBlock/BeginBlock happens here, like validator set change maturations. * `DeliverVscPacket(receiver: Chain)`: Delivers a pending VSCPacket from the provider to the consumer `receiver`. * `DeliverVscMaturedPacket(receiver: Chain)`: Delivers a pending VSCMaturedPacket from the consumer `receiver` to the provider. +* `KeyAssignment(chain: Chain, validator: Node, consumerAddr: ConsumerAddr)` (only when running with `--step stepKeyAssignment`): Assigns the `consumerAddr` to the `validator` on the `chain`. Note that we use "key" and "consumerAddr" pretty much interchangeably, as the model makes no differentiation between private keys, public keys, addresses, etc, as it doesn't model the cryptography. ### State machines There are 3 different "state machine layers" that can be put on top of the core logic. +Some layers include extra logic, need other invariants, ... #### ccv_model.qnt This is the most general state machine layer. It allows the most behaviour, in particular it allows abitrary clock drift between chains, it allows starting and stopping consumer chains during runtime, etc. This layer is most useful for model checking, because it encompasses the most behaviour. +As an optional module, it can also include KeyAssignment. + +##### KeyAssignment + +To run with key assignment, specify the step flag: `--step stepKeyAssignment`. + +KeyAssignment also needs some different invariants, see below. #### ccv_boundeddrift.qnt This state machine layer is more restricted to generate more interesting traces: @@ -94,6 +103,13 @@ with a timestamp >= t + UnbondingPeriod on that consumer. - [X] EventuallyMatureOnProviderInv: If we send a VscPacket, this is eventually responded to by all consumers that were running at the time the packet was sent (and are still running). +Invariants only relevant when running with key assignment (`--step stepKeyAssignment`): +- [X] ValidatorSetHasExistedKeyAssignmentInv: Should replace ValidatorSetHasExistedInv when running with `--step stepKeyAssignment`. Validator sets are checked for equality under key assignment when checking whether they have existed. +- [X] SameVscPacketsKeyAssignmentInv: Should replace SameVscPacketsInv when running with `--step stepKeyAssignment`. VscPackets are checked for equality under key assignment when ensuring consumers receive the same ones. +- [X] KeyAssignmentRulesInv: Ensures the rules of key assignment are never violated. The two rules relevant for the model are: 1) validator A cannot assign consumer key K to consumer chain X if there is already a validator B (B!=A) +using K on the provider, and 2) validator A cannot assign consumer key K to consumer chain X if there is already a validator B using K on X + + Invariants can also be model-checked by Apalache, using this command: ``` quint verify --invariant ValidatorUpdatesArePropagatedInv,ValidatorSetHasExistedInv,SameVscPacketsInv,MatureOnTimeInv,EventuallyMatureOnProviderInv \ @@ -113,4 +129,6 @@ The available sanity checks are: - CanStopConsumer - CanTimeoutConsumer - CanSendVscPackets -- CanSendVscMaturedPackets \ No newline at end of file +- CanSendVscMaturedPackets +- CanAssignConsumerKey (only with `--step stepKeyAssignment`) +- CanHaveConsumerAddresses (only with `--step stepKeyAssignment`) \ No newline at end of file diff --git a/tests/mbt/model/ccv.qnt b/tests/mbt/model/ccv.qnt index 51def4c01e..2a81fe3a2d 100644 --- a/tests/mbt/model/ccv.qnt +++ b/tests/mbt/model/ccv.qnt @@ -1,6 +1,7 @@ // -*- mode: Bluespec; -*- module ccv_types { import Time.* from "./libraries/Time" + import extraSpells.* from "./libraries/extraSpells" type Node = str type Chain = str @@ -11,6 +12,11 @@ module ccv_types { // a list of validator sets per blocks, ordered by recency type VotingPowerHistory = List[ValidatorSet] + // For key assignment, to differentiate Nodes + // (on the provider) from the assigned + // keys/addresses on consumers + type ConsumerAddr = str + type VscPacket = { // the identifier for this packet @@ -240,6 +246,7 @@ module ccv { import Time.* from "./libraries/Time" import extraSpells.* from "./libraries/extraSpells" import ccv_types.* + import ccv_utils.* from "./ccv_utils" // =================== @@ -388,7 +395,7 @@ module ccv { } } else { // the packet has not timed out, so receive it on the consumer - val result = recvPacketOnConsumer(currentState, receiver, packet) + val result = recvPacketOnConsumer(currentState, receiver, packet, UnbondingPeriodPerChain.get(receiver)) val tmpState = result.newState if (result.hasError()) { (result, false) @@ -421,23 +428,36 @@ module ccv { // check for vsc timeouts val timedOutConsumers = getRunningConsumers(currentProviderState).filter( consumer => - val res = TimeoutDueToVscTimeout(currentState, consumer) + val res = TimeoutDueToVscTimeout(currentState, consumer, VscTimeout) res._1 ) + // for each consumer chain, apply the key assignment to the current validator set + val currentValSets = ConsumerChains.mapBy( + (consumer) => + currentProviderState.applyKeyAssignmentToValSet( + consumer, + currentProviderState.chainState.currentValidatorSet + ) + ) + // store the current validator set with the key assignments applied in the history + val newKeyAssignedValSetHistory = currentValSets.keys().mapBy( + (consumer) => + currentProviderState.keyAssignedValSetHistory + .getOrElse(consumer, List()) // get the existing history (empty list if no history yet) + .prepend(currentValSets.get(consumer)) // prepend the current validator set with key assignments applied + ) // run the shared core chainState logic val newChainState = currentProviderState.chainState.endAndBeginBlockShared(timeAdvancement) - val providerStateAfterTimeAdvancement = currentProviderState.with( - "chainState", newChainState - ) - + val providerStateAfterTimeAdvancement = + {...currentProviderState, chainState: newChainState, keyAssignedValSetHistory: newKeyAssignedValSetHistory} val tmpState = currentState.with( "providerState", providerStateAfterTimeAdvancement ) - // send vsc packets + // send vsc packets (will be a noop if no sends are necessary) val providerStateAfterSending = // if currentBlockHeight is a multiple of BlocksPerEpoch, send VscPackets if (providerStateAfterTimeAdvancement.chainState.currentBlockHeight % BlocksPerEpoch == 0) { @@ -466,19 +486,21 @@ module ccv { if (err != "") { Err(err) } else { - // for each consumer we just set to running, set its initial validator set to be the current one on the provider. + // for each consumer we just set to running, set its initial validator set to be the current one on the provider... val valSet = providerStateAfterConsumerAdvancement.chainState.currentValidatorSet val newConsumerStateMap = tmpState.consumerStates.keys().mapBy( (consumer) => if (consumersToStart.contains(consumer)) { + // ...modified by the key assignments for the consumer + val consValSet = applyKeyAssignmentToValSet(providerStateAfterConsumerAdvancement, consumer, valSet) val currentConsumerState: ConsumerState = tmpState.consumerStates.get(consumer) val newConsumerState: ConsumerState = currentConsumerState.with( "chainState", currentConsumerState.chainState.with( - "currentValidatorSet", valSet + "currentValidatorSet", consValSet ).with( "votingPowerHistory", - List(valSet) + List(consValSet) ).with( "lastTimestamp", providerStateAfterConsumerAdvancement.chainState.lastTimestamp @@ -555,36 +577,21 @@ module ccv { } } - // =================== - // UTILITY FUNCTIONS - // which do not hold the core logic of the protocol, but are still part of it - // =================== - - // Returns the new ConsumerStatusMap according to the consumers to stop - // and the consumers to time out. - // If a consumer is both stopped and timed out, it will be timed out. - // The second return is an error string: If it is not equal to "", - // it contains an error message, and the first return should be ignored. - pure def stopConsumers( - currentConsumerStatusMap: Chain -> str, - consumersToStop: Set[Chain], - consumersToTimeout: Set[Chain]): (Chain -> str, str) = { - val runningConsumers = currentConsumerStatusMap.keys().filter( - chain => currentConsumerStatusMap.get(chain) == RUNNING - ) - // all consumers to stop must be running right now, else we have an error - if (consumersToStop.exclude(runningConsumers).size() > 0) { - (currentConsumerStatusMap, "Cannot stop a consumer that is not running") + // Validator providerNode assigns their address for the consumer to be the consumerAddress. + pure def assignConsumerKey(currentState: ProtocolState, consumer: Chain, providerNode: Node, consumerAddr: ConsumerAddr): Result = { + // rule 1: validator A cannot assign consumer key K to consumer chain X + // if there is already a validator B (B!=A) using K on the provider + pure val provCurValSet = currentState.providerState.chainState.currentValidatorSet + if (provCurValSet.keys().exists(node => node != providerNode and node == consumerAddr)) { + Err("validator A cannot assign consumer key K to consumer chain X + if there is already a validator B (B!=A) using K on the provider") + } else { + // rule 2: validator A cannot assign consumer key K to consumer chain X if + // there is already a validator B using K on X + pure val valByConsAddr = currentState.providerState.consumerAddrToValidator.getOrElse(consumer, Map()) + if (valByConsAddr.keys().contains(consumerAddr)) { + Err("consumer key is already in use on the consumer chain") } else { - val newConsumerStatusMap = currentConsumerStatusMap.keys().mapBy( - (chain) => - if (consumersToTimeout.contains(chain)) { - TIMEDOUT - } else if (consumersToStop.contains(chain)) { - STOPPED - } else { - currentConsumerStatusMap.get(chain) - } // this key can be assigned // get the old assigned key @@ -623,277 +630,37 @@ module ccv { "consumersWithAddrAssignmentChangesInThisEpoch", consumersWithAddrAssignmentChangesInThisEpoch ) ) - (newConsumerStatusMap, "") - } - } - - // Returns the new ConsumerStatusMap according to the consumers to start. - // The second return is an error string: If it is not equal to "", - // it contains an error message, and the first return should be ignored. - pure def startConsumers( - currentConsumerStatusMap: Chain -> str, - consumersToStart: Set[Chain]): (Chain -> str, str) = { - val nonConsumers = currentConsumerStatusMap.keys().filter( - chain => currentConsumerStatusMap.get(chain) == NOT_CONSUMER - ) - // all consumers to start must be nonConsumers right now, otherwise we have an error - if (consumersToStart.exclude(nonConsumers).size() > 0) { - (currentConsumerStatusMap, "cannot start a consumer that is stopped or already a consumer") - } else { - val newConsumerStatusMap = currentConsumerStatusMap.keys().mapBy( - (chain) => - if (consumersToStart.contains(chain)) { - RUNNING - } else { - currentConsumerStatusMap.get(chain) - } + + pure val newvalidatorToConsumerAddr = currentState.providerState.validatorToConsumerAddr.put( + consumer, + currentState.providerState.validatorToConsumerAddr.getOrElse(consumer, Map()).put( + providerNode, + consumerAddr + ) ) - (newConsumerStatusMap, "") - } - } - - pure def StartStopConsumers( - currentConsumerStatusMap: Chain -> str, - consumersToStart: Set[Chain], - consumersToStop: Set[Chain], - consumersToTimeout: Set[Chain] - ): (Chain -> str, str) = { - // check if any consumer is both started and stopped - if (consumersToStart.intersect(consumersToStop).size() > 0) { - (currentConsumerStatusMap, "Cannot start and stop a consumer at the same time") - } else { - val res1 = currentConsumerStatusMap.startConsumers(consumersToStart) - val newConsumerStatus = res1._1 - val err1 = res1._2 - val res2 = newConsumerStatus.stopConsumers(consumersToStop, consumersToTimeout) - val err2 = res2._2 - if (err1 != "") { - (currentConsumerStatusMap, err1) - } else if (err2 != "") { - (currentConsumerStatusMap, err2) - } else { - (res2._1, "") - } - } - } - - - // Takes the currentValidatorSet and puts it as the newest set of the voting history - pure def enterCurValSetIntoBlock(chainState: ChainState): ChainState = { - chainState.with( - "votingPowerHistory", chainState.votingPowerHistory.prepend( - chainState.currentValidatorSet - ) - ) - } - // Advances the timestamp in the chainState by timeAdvancement - pure def advanceTime(chainState: ChainState, timeAdvancement: Time): ChainState = - { - ...chainState, - lastTimestamp: chainState.runningTimestamp, - runningTimestamp: chainState.runningTimestamp + timeAdvancement, - } - - // common logic to update the chain state, used by both provider and consumers. - pure def endAndBeginBlockShared(chainState: ChainState, timeAdvancement: Time): ChainState = { - chainState.enterCurValSetIntoBlock().advanceTime(timeAdvancement) - } - - // returns the providerState with the following modifications: - // * sends VscPackets to all running consumers, using the provided timestamp as sending time - // * increments the runningVscId - // This should only be called when the provider chain is ending a block, - // and only when the running validator set is considered to have changed - // and there is a consumer to send a packet to. - pure def sendVscPackets(providerState: ProviderState, sendingTimestamp: Time): ProviderState = { - val newSentPacketsPerConsumer = ConsumerChains.mapBy( - (consumer) => - // if validator set changed and the consumer is running, send a packet - if (providerState.providerValidatorSetChangedInThisBlock and - isRunningConsumer(consumer, providerState)) { - List({ - id: providerState.runningVscId, - validatorSet: providerState.chainState.currentValidatorSet, - sendingTime: sendingTimestamp, - timeoutTime: sendingTimestamp + CcvTimeout.get(PROVIDER_CHAIN) - }) - } else { - List() - } - ) - val newOutstandingPacketsToConsumer = ConsumerChains.mapBy( - (consumer) => - providerState.outstandingPacketsToConsumer.get(consumer).concat( - newSentPacketsPerConsumer.get(consumer) - ) - ) - val newSentVscPackets = ConsumerChains.mapBy( - (consumer) => - providerState.sentVscPacketsToConsumer.get(consumer).concat( - newSentPacketsPerConsumer.get(consumer) + pure val newconsumerAddrToValidator = currentState.providerState.consumerAddrToValidator.put( + consumer, + currentState.providerState.consumerAddrToValidator.getOrElse(consumer, Map()).put( + consumerAddr, + providerNode ) ) - { - ...providerState, - outstandingPacketsToConsumer: newOutstandingPacketsToConsumer, - sentVscPacketsToConsumer: newSentVscPackets, - providerValidatorSetChangedInThisBlock: false, - runningVscId: providerState.runningVscId + 1, - } - } - - // receives a given packet (sent by the provider) on the consumer. The arguments are the consumer chain that is receiving the packet, and the packet itself. - // To receive a packet, modify the running validator set (not the one entered into the block yet, - // but the candidate that would be put into the block if it ended now) - // and store the maturation time for the packet. - pure def recvPacketOnConsumer(currentState: ProtocolState, receiver: Chain, packet: VscPacket): Result = { - if(not(isRunningConsumer(receiver, currentState.providerState))) { - Err("Receiver is not currently a consumer - must have 'running' status!") - } else { - // update the running validator set, but not the history yet, - // as that only happens when the next block is started - val currentConsumerState: ConsumerState = currentState.consumerStates.get(receiver) - val newConsumerState: ConsumerState = - { - ...currentConsumerState, - chainState: currentConsumerState.chainState.with( - "currentValidatorSet", packet.validatorSet - ), - maturationTimes: currentConsumerState.maturationTimes.append( - ( - packet, - currentConsumerState.chainState.runningTimestamp + UnbondingPeriodPerChain.get(receiver) - ) - ), - receivedVscPackets: currentConsumerState.receivedVscPackets.prepend(packet) - } - val newConsumerStates = currentState.consumerStates.set(receiver, newConsumerState) - val newState = currentState.with( - "consumerStates", newConsumerStates - ) - Ok(newState) - } - } - - // receives a given packet on the provider. The arguments are the consumer chain that sent the packet, and the packet itself. - // To receive a packet, add it to the list of received maturations. - pure def recvPacketOnProvider(currentState: ProtocolState, sender: Chain, packet: VscMaturedPacket): Result = { - if (not(isRunningConsumer(sender, currentState.providerState))) { - Err("Sender is not currently a consumer - must have 'running' status!") - } else if (currentState.providerState.sentVscPacketsToConsumer.get(sender).head().id != packet.id) { - // the packet is not the oldest sentVscPacket, something went wrong - Err("Received maturation is not for the oldest sentVscPacket") - } else { - val currentReceivedMaturations = currentState.providerState.receivedMaturations - val newReceivedMaturations = currentReceivedMaturations.union(Set(packet)) - val newProviderState = currentState.providerState.with( - "receivedMaturations", newReceivedMaturations - ) - // prune the sentVscPacket - val newSentVscPacket = currentState.providerState.sentVscPacketsToConsumer.get(sender).tail() - val newState = currentState.with( - "providerState", - {...newProviderState, - sentVscPacketsToConsumer: currentState.providerState.sentVscPacketsToConsumer.set(sender, newSentVscPacket) - } - ) - Ok(newState) - } - } - - // removes the oldest outstanding packet from the consumer. on-chain, this would happen when the packet is acknowledged. - // only the oldest packet can be removed, since we model ordered channels. - pure def removeOutstandingPacketFromConsumer(currentState: ProtocolState, sender: Chain): ProtocolState = { - val currentOutstandingPackets = currentState.consumerStates.get(sender).outstandingPacketsToProvider - val newOutstandingPackets = currentOutstandingPackets.tail() - val newConsumerState = currentState.consumerStates.get(sender).with( - "outstandingPacketsToProvider", newOutstandingPackets - ) - val newConsumerStates = currentState.consumerStates.set(sender, newConsumerState) - val newState = currentState.with( - "consumerStates", newConsumerStates - ) - newState - } - - // removes the oldest outstanding packet (to the given consumer) from the provider. - // on-chain, this would happen when the packet is acknowledged. - // only the oldest packet can be removed, since we model ordered channels. - pure def removeOutstandingPacketFromProvider(currentState: ProtocolState, receiver: Chain): ProtocolState = { - val currentOutstandingPackets = currentState.providerState.outstandingPacketsToConsumer.get(receiver) - val newOutstandingPackets = currentOutstandingPackets.tail() - val newProviderState = currentState.providerState.with( - "outstandingPacketsToConsumer", - currentState.providerState.outstandingPacketsToConsumer.set(receiver, newOutstandingPackets) - ) - val newState = currentState.with( - "providerState", newProviderState - ) - newState - } - - // Returns a ProtocolState where the current validator set on the provider is set to - // newValidatorSet. - pure def setProviderValidatorSet(currentState: ProtocolState, newValidatorSet: ValidatorSet): ProtocolState = { - pure val newChainState = currentState.providerState.chainState.with( - "currentValidatorSet", newValidatorSet - ) - currentState.with( - "providerState", - currentState.providerState.with( - "chainState", newChainState - ) - ) - } - - // Returns true if the given chain is currently a running consumer, false otherwise. - pure def isRunningConsumer(chain: Chain, providerState: ProviderState): bool = { - val status = providerState.consumerStatus.get(chain) - status == RUNNING - } - - // Returns the set of all consumer chains that currently have the status RUNNING. - pure def getRunningConsumers(providerState: ProviderState): Set[Chain] = { - providerState.consumerStatus.keys().filter( - chain => providerState.consumerStatus.get(chain) == RUNNING - ) - } - // Returns the set of all consumer chains that currently have the status NOT_CONSUMER. - pure def getNonConsumers(providerState: ProviderState): Set[Chain] = { - providerState.consumerStatus.keys().filter( - chain => providerState.consumerStatus.get(chain) == NOT_CONSUMER - ) - } + pure val newProviderState = tmpStateAfterKeyAssignmentReplacement.providerState.with( + "validatorToConsumerAddr", newvalidatorToConsumerAddr + ).with( + "consumerAddrToValidator", newconsumerAddrToValidator + ) - // Returns whether the consumer has timed out due to the VscTimeout, and an error message. - // If the second return is not equal to "", the first return should be ignored. - // If it is equal to "", the first return will be true if the consumer has timed out and should be dropped, - // or false otherwise. - pure def TimeoutDueToVscTimeout(currentState: ProtocolState, consumer: Chain): (bool, str) = - // check for errors: the consumer is not running - if (not(isRunningConsumer(consumer, currentState.providerState))) { - (false, "Consumer is not currently a consumer - must have 'running' status!") - } else { - val providerState = currentState.providerState - val consumerState: ConsumerState = currentState.consumerStates.get(consumer) - - // has a packet been sent on the provider more than VscTimeout ago, but we have not received an answer since then? - val sentVscPacketsToConsumer = providerState.sentVscPacketsToConsumer.get(consumer) - if(sentVscPacketsToConsumer.length() > 0) { - val oldestSentVscPacket = sentVscPacketsToConsumer.head() // if length is 0, this is undefined, but we check for this before we use it - if(oldestSentVscPacket.sendingTime + VscTimeout < providerState.chainState.runningTimestamp) { - (true, "") - } else { - // no timeout yet, it has not been VscTimeout since that packet was sent - (false, "") - } - } else { - // no packet has been sent yet, so no timeout - (false, "") + Ok( + tmpStateAfterKeyAssignmentReplacement.with( + "providerState", newProviderState + ) + ) } } + } // =================== // ASSUMPTIONS ON MODEL PARAMETERS diff --git a/tests/mbt/model/ccv_boundeddrift.qnt b/tests/mbt/model/ccv_boundeddrift.qnt index 21a53074c0..24db1d3eaf 100644 --- a/tests/mbt/model/ccv_boundeddrift.qnt +++ b/tests/mbt/model/ccv_boundeddrift.qnt @@ -1,6 +1,7 @@ module ccv_boundeddrift { import ccv_model.* from "ccv_model" import ccv_types as Ccvt from "ccv" + import ccv_utils.* from "ccv_utils" import ccv from "ccv" import Time.* from "./libraries/Time" import extraSpells.* from "./libraries/extraSpells" @@ -32,7 +33,7 @@ module ccv_boundeddrift { // Given the name of a chain, gets a set with the chain states of all other chains. def GetOtherChainStates(advancingChain: Ccvt::Chain): Set[Ccvt::ChainState] = - val runCons = ccv::getRunningConsumers(currentState.providerState) + val runCons = getRunningConsumers(currentState.providerState) if (advancingChain == Ccvt::PROVIDER_CHAIN) { runCons.map(c => currentState.consumerStates.get(c).chainState) } else { @@ -48,7 +49,7 @@ module ccv_boundeddrift { } def GetRunningChainStates(): Set[Ccvt::ChainState] = - val runCons = ccv::getRunningConsumers(currentState.providerState) + val runCons = getRunningConsumers(currentState.providerState) val consumerChainStates = runCons.map(c => currentState.consumerStates.get(c).chainState) consumerChainStates.union(Set(currentState.providerState.chainState)) @@ -80,13 +81,18 @@ module ccv_boundeddrift { val consumerStatus = currentState.providerState.consumerStatus nondet consumersToStart = oneOf(nonConsumers.powerset()) // make it so we stop consumers only with small likelihood: - nondet stopConsumers = oneOf(1.to(100)) - nondet consumersToStop = if (stopConsumers <= consumerStopChance) oneOf(runningConsumers.powerset()) else Set() + nondet stopConsumersRand = oneOf(1.to(100)) + nondet consumersToStop = if (stopConsumersRand <= consumerStopChance) oneOf(runningConsumers.powerset()) else Set() nondet timeAdvancement = oneOf(possibleAdvancements) EndAndBeginBlockForProvider(timeAdvancement, consumersToStart, consumersToStop), } } + action stepBoundedDriftKeyAssignment = any { + stepBoundedDrift, + nondetKeyAssignment, + } + // INVARIANT // The maxDrift between chains is never exceeded. // This *should* be ensured by the step function. diff --git a/tests/mbt/model/ccv_model.qnt b/tests/mbt/model/ccv_model.qnt index 3665837c2a..1509ad439f 100644 --- a/tests/mbt/model/ccv_model.qnt +++ b/tests/mbt/model/ccv_model.qnt @@ -4,6 +4,7 @@ module ccv_model { import ccv_types.* from "./ccv" import Time.* from "./libraries/Time" import extraSpells.* from "./libraries/extraSpells" + import ccv_utils.* from "./ccv_utils" pure val consumerChainList = List("consumer1", "consumer2", "consumer3") pure val consumerChains = consumerChainList.toSet() @@ -15,6 +16,8 @@ module ccv_model { pure val epochLength = 3 pure val nodes = Set("node1", "node2", "node3", "node4", "node5", "node6", "node7", "node8", "node9", "node10") + // possible consumer addresses that nodes can assign their key to + pure val consumerAddresses = Set("consAddr1", "consAddr2", "consAddr3", "consAddr4", "consAddr5", "consAddr6", "consAddr7", "consAddr8", "consAddr9", "consAddr10") pure val InitialValidatorSet = nodes.mapBy(node => 100) import ccv( @@ -33,6 +36,7 @@ module ccv_model { TrustingPeriodPerChain: Chain -> Time, ConsumerChains: Set[Chain], Nodes: Set[Node], + ConsumerAddresses: Set[ConsumerAddr], InitialValidatorSet: Node -> int, BlocksPerEpoch: int, } @@ -59,6 +63,7 @@ module ccv_model { consumersToStop: Set[Chain], validator: Node, changeAmount: int, + consumerAddr: ConsumerAddr, } @@ -73,7 +78,7 @@ module ccv_model { // otherwise connections will break down. pure val timeAdvancements = Set(1 * Second, 1 * Day, 1 * Week - 1 * Hour) - pure def emptyAction = + pure def emptyAction: Action = { kind: "", consumerChain: "", @@ -82,6 +87,7 @@ module ccv_model { consumersToStop: Set(), validator: "", changeAmount: 0, + consumerAddr: "", } @@ -109,6 +115,8 @@ module ccv_model { ).with( "currentValidatorSet", InitialValidatorSet ) + ).with( + "keyAssignedValSetHistory", ConsumerChains.mapBy(chain => List(InitialValidatorSet)) ) currentState' = { providerState: providerStateWithConsumers, @@ -184,6 +192,7 @@ module ccv_model { params' = params, } + // stepCommon is the core functionality of steps that does not have anything to do with time. action stepCommon = any { nondet node = oneOf(nodes) @@ -253,6 +262,17 @@ module ccv_model { // UTILITY FUNCTIONS // ================== + pure def removeZeroPowers(valSet: ValidatorSet): ValidatorSet = + valSet.keys().fold( + Map(), + (acc, node) => + if (valSet.get(node) == 0) { + acc + } else { + acc.put(node, valSet.get(node)) + } + ) + pure def oldest(packets: Set[VscPacket]): VscPacket = val newestPossiblePacket: VscPacket = { id: 0, @@ -314,9 +334,11 @@ module ccv_model { // Every validator set on any consumer chain MUST either be or have been // a validator set on the provider chain. val ValidatorSetHasExistedInv = - runningConsumers.forall(chain => + runningConsumers.forall(chain => // for all running consumers currentState.consumerStates.get(chain).chainState.votingPowerHistory.toSet().forall( + // go through all its historical and current validator sets validatorSet => providerValidatorHistory.toSet().contains(validatorSet) + // and check that they are also historical or current validator sets on the provider ) ) @@ -337,10 +359,6 @@ module ccv_model { consumer => currentState.providerState.sentVscPacketsToConsumer.get(consumer).toSet().exists( packet => packet.validatorSet == providerValSetInCurBlock ) - // or the consumer was just started, which we detect by the consumer having a timestamp of 0 - // and the consumer having the validator set that was just sent in the block - or - (currentState.consumerStates.get(consumer).chainState.lastTimestamp == 0 and currentState.consumerStates.get(consumer).chainState.currentValidatorSet == providerValSetInCurBlock) ) // Every consumer chain receives the same sequence of @@ -387,8 +405,11 @@ module ccv_model { val lastTimeAdvancement = trace[trace.length()-1].timeAdvancement val lastBlockTime = currentState.consumerStates.get(ConsumerWithPotentialMaturations).chainState.lastTimestamp - lastTimeAdvancement val MatureOnTimeInv = + // if a consumer ended a block MaturationPrecondition implies + // then all matured packets need to have been processed and removed from the packets + // waiting to mature currentState.consumerStates.get(ConsumerWithPotentialMaturations).maturationTimes.toSet().forall( pair => val maturationTime = pair._2 @@ -405,7 +426,7 @@ module ccv_model { val EventuallyMatureOnProviderInv = runningConsumers.forall( consumer => { - val sentPackets = currentState.providerState.sentVscPacketsToConsumer.get(consumer).toSet() + val sentPackets = currentState.providerState.sentVscPacketsToConsumer.getOrElse(consumer, List()).toSet() sentPackets.forall( packet => // consumer still has time to respond @@ -417,6 +438,7 @@ module ccv_model { } ) + // ================= // SANITY CHECKS // ================= @@ -448,7 +470,7 @@ module ccv_model { val CanSendVscPackets = not(ConsumerChains.exists( consumer => - currentState.providerState.outstandingPacketsToConsumer.get(consumer).length() > 0 + currentState.providerState.outstandingPacketsToConsumer.getOrElse(consumer, List()).length() > 0 )) val CanReceiveVscPackets = @@ -516,7 +538,7 @@ module ccv_model { // consumer1 was started assert(currentState.providerState.consumerStatus.get("consumer1") == RUNNING), // but no packet was sent to consumer 1 - assert(currentState.providerState.outstandingPacketsToConsumer.get("consumer1").length() == 0), + assert(currentState.providerState.outstandingPacketsToConsumer.getOrElse("consumer1", List()).length() == 0), // the validator set on the provider was entered into the history assert(currentState.providerState.chainState.votingPowerHistory == List(InitialValidatorSet.put("node1", 150), InitialValidatorSet)), // change voting power on provider again diff --git a/tests/mbt/model/ccv_test.qnt b/tests/mbt/model/ccv_test.qnt index 9c21d8e541..ab47df50b7 100644 --- a/tests/mbt/model/ccv_test.qnt +++ b/tests/mbt/model/ccv_test.qnt @@ -5,6 +5,7 @@ module ccv_test { import ccv_types.* from "./ccv" import Time.* from "./libraries/Time" import extraSpells.* from "./libraries/extraSpells" + import ccv_utils.* from "./ccv_utils" pure val consumerChains = Set("sender", "receiver") pure val chains = consumerChains.union(Set(PROVIDER_CHAIN)) diff --git a/tests/mbt/run_invariants.sh b/tests/mbt/run_invariants.sh index 2d0b33bed9..afe079387e 100755 --- a/tests/mbt/run_invariants.sh +++ b/tests/mbt/run_invariants.sh @@ -38,4 +38,4 @@ run_invariant "CanSendVscPackets" "" '[violation]' run_invariant "CanSendVscMaturedPackets" "" '[violation]' run_invariant "CanAssignConsumerKey" "stepKeyAssignment" '[violation]' run_invariant "CanHaveConsumerAddresses" "stepKeyAssignment" '[violation]' -run_invariant "CanReceiveMaturations" "stepKeyAssignment" '[violation]' +run_invariant "CanReceiveMaturations" "stepKeyAssignment" '[violation]' \ No newline at end of file From 6bbb1ac472a9d6d25089f81ed2d863b43e684b80 Mon Sep 17 00:00:00 2001 From: insumity Date: Tue, 12 Mar 2024 15:47:31 +0100 Subject: [PATCH 4/5] fix markdown links --- .../adr-005-cryptographic-equivocation-verification.md | 3 +-- docs/docs/validators/overview.md | 8 ++++---- 2 files changed, 5 insertions(+), 6 deletions(-) diff --git a/docs/docs/adrs/adr-005-cryptographic-equivocation-verification.md b/docs/docs/adrs/adr-005-cryptographic-equivocation-verification.md index d4f9bd2bf8..7c2bd34cdd 100644 --- a/docs/docs/adrs/adr-005-cryptographic-equivocation-verification.md +++ b/docs/docs/adrs/adr-005-cryptographic-equivocation-verification.md @@ -138,8 +138,7 @@ either using its infraction height or its unsigned timestamp. Note that changes The underlying reason is that a malicious validator could take advantage of getting tombstoned to avoid being slashed on the provider ([see comment](https://github.com/cosmos/interchain-security/pull/1232#issuecomment-1693127641)). -- Currently, the endpoint can only handle _equivocation_ light client attacks. This is because the _lunatic_ attacks require the endpoint to possess the ability to dissociate which header is conflicted or trusted upon receiving a misbehavior message. Without this information, it's not possible to extract the Byzantine validators from the conflicting headers (see [comment](https://github.com/cosmos/interchain-security/pull/826#discussion_r1268668684)). In addition, "amnesia" attacks are ignored, similar to CometBFT (see [ADR-056](https://github.com/cometbft/cometbft/blob/main/docs/architecture/tendermint-core/adr-056-light-client-amnesia-attacks.md#decision)). - +- Currently, the endpoint can only handle _equivocation_ light client attacks. This is because the _lunatic_ attacks require the endpoint to possess the ability to dissociate which header is conflicted or trusted upon receiving a misbehavior message. Without this information, it's not possible to extract the Byzantine validators from the conflicting headers (see [comment](https://github.com/cosmos/interchain-security/pull/826#discussion_r1268668684)). In addition, "amnesia" attacks are ignored, similar to CometBFT (see [ADR-056](https://github.com/cometbft/cometbft/blob/main/docs/references/architecture/tendermint-core/adr-056-light-client-amnesia-attacks.md#decision)). ## Consequences diff --git a/docs/docs/validators/overview.md b/docs/docs/validators/overview.md index 4cac6e9582..a017f51a9e 100644 --- a/docs/docs/validators/overview.md +++ b/docs/docs/validators/overview.md @@ -85,7 +85,7 @@ At present, the consumer chain can report evidence about downtime infractions to :::info Causing a downtime infraction on any consumer chain will not incur a slash penalty. Instead, the offending validator will be jailed on the provider chain and consequently on all consumer chains. -To unjail, the validator must wait for the jailing period to elapse on the provider chain and [submit an unjail transaction](https://hub.cosmos.network/main/validators/validator-setup.html#unjail-validator) on the provider chain. After unjailing on the provider, the validator will be unjailed on all consumer chains. +To unjail, the validator must wait for the jailing period to elapse on the provider chain and [submit an unjail transaction](https://hub.cosmos.network/validators/validator-setup#unjail-validator) on the provider chain. After unjailing on the provider, the validator will be unjailed on all consumer chains. More information is available in [Downtime Slashing documentation](../features/slashing.md#downtime-infractions) ::: @@ -99,7 +99,7 @@ Validators can use different consensus keys on the provider and each of the cons For more information check out the [Key assignment overview and guide](../features/key-assignment.md) ## References: -- [Cosmos Hub Validators FAQ](https://hub.cosmos.network/main/validators/validator-faq.html) -- [Cosmos Hub Running a validator](https://hub.cosmos.network/main/validators/validator-setup.html) +- [Cosmos Hub Validators FAQ](https://hub.cosmos.network/validators/validator-faq) +- [Cosmos Hub Running a validator](https://hub.cosmos.network/validators/validator-setup) - [Startup Sequence](https://github.com/cosmos/testnets/blob/master/replicated-security/CONSUMER_LAUNCH_GUIDE.md#chain-launch) -- [Submit Unjailing Transaction](https://hub.cosmos.network/main/validators/validator-setup.html#unjail-validator) +- [Submit Unjailing Transaction](https://hub.cosmos.network/validators/validator-setup#edit-validator-description) From b4505098e2f649a21e54739e27fe8408a91e82b2 Mon Sep 17 00:00:00 2001 From: insumity Date: Tue, 12 Mar 2024 16:00:05 +0100 Subject: [PATCH 5/5] fix markdown links --- docs/docs/validators/joining-testnet.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/docs/validators/joining-testnet.md b/docs/docs/validators/joining-testnet.md index dcf654786d..dae0cd3755 100644 --- a/docs/docs/validators/joining-testnet.md +++ b/docs/docs/validators/joining-testnet.md @@ -12,7 +12,7 @@ The experience gained in the testnet will prepare you for validating interchain :::tip Provider and consumer chain represent distinct networks and infrastructures operated by the same validator set. -For general information about running cosmos-sdk based chains check out the [validator basics](https://hub.cosmos.network/main/validators/validator-setup.html) and [Running a Node section](https://docs.cosmos.network/main/run-node/run-node) of Cosmos SDK docs +For general information about running cosmos-sdk based chains check out the [validator basics](https://hub.cosmos.network/validators/validator-setup) and [Running a Node section](https://docs.cosmos.network/main/run-node/run-node) of Cosmos SDK docs ::: ## Joining the provider chain @@ -79,7 +79,7 @@ gaiad tx staking create-validator \ ``` :::tip -Check this [guide](https://hub.cosmos.network/main/validators/validator-setup.html#edit-validator-description) to edit your validator. +Check this [guide](https://hub.cosmos.network/validators/validator-setup#edit-validator-description) to edit your validator. ::: After this step, your validator is created and you can start your node and catch up to the rest of the network. It is recommended that you use `statesync` to catch up to the rest of the network.