Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: Allow validators outside the active set to validate on consumer chains #1878

Closed
Closed
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
0fcb57b
rough draft of one method of sending a reduced active set to the prov…
jtremback May 13, 2024
c0d8581
Add key for last consensus validator set on the provider
p-offtermatt May 14, 2024
b88cfb5
Refactor validator set storage to reduce duplication
p-offtermatt May 14, 2024
2e5a91a
Add draft ADR for active set validators
p-offtermatt May 14, 2024
2cddff5
Revert "Add draft ADR for active set validators"
p-offtermatt May 14, 2024
acdf06a
Add MaxProviderConsensusValidators param
p-offtermatt May 14, 2024
48f4085
Use MaxProviderConsensusValidators param
p-offtermatt May 14, 2024
5ccdb18
Utilize wrapper around staking module
p-offtermatt May 14, 2024
48a1aba
Use consensus power for throttle meter
p-offtermatt May 15, 2024
66139cc
Add new param to tests
p-offtermatt May 15, 2024
c11e65d
Start fixing genesis
p-offtermatt May 15, 2024
d21cf9a
Fix mock usage in tests
p-offtermatt May 15, 2024
87742aa
Make Genesis return consensus validators
p-offtermatt May 15, 2024
fb1a8e5
Add debugging info in InitGenesis
p-offtermatt May 15, 2024
b632aca
Add wrapped_genutil
p-offtermatt May 15, 2024
519a329
Fix wrapped genutil integration
p-offtermatt May 16, 2024
0285039
Start adding e2e test
p-offtermatt May 23, 2024
9e58c13
Adjust tests
p-offtermatt May 24, 2024
134d840
Merge branch 'main' into ph/max-consensus-validators
p-offtermatt May 24, 2024
70a6bf1
Adjust expected calls
p-offtermatt May 24, 2024
1265836
Set ProviderConsensusValSet in proposal test
p-offtermatt May 24, 2024
affd10f
Extract variable
p-offtermatt May 24, 2024
5ea92ea
Improve comment
p-offtermatt May 24, 2024
617b515
Merge branch 'main' into ph/max-consensus-validators
p-offtermatt Jun 5, 2024
9822bbf
Comment on slashing vs jailing
p-offtermatt Jun 5, 2024
b29b867
Update e2e test
p-offtermatt Jun 5, 2024
9b5e578
Remove commented code
p-offtermatt Jun 11, 2024
bfb83ab
Move createStakingValidator to testkeeper
p-offtermatt Jun 24, 2024
a7de72a
Address comments
p-offtermatt Jun 24, 2024
faff1a7
Address comments
p-offtermatt Jun 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions proto/interchain_security/ccv/provider/v1/provider.proto
Original file line number Diff line number Diff line change
Expand Up @@ -212,6 +212,10 @@ message Params {

// The number of blocks that comprise an epoch.
int64 blocks_per_epoch = 10;

// The maximal number of validators that will be passed
// to the consensus engine on the provider.
int64 max_provider_consensus_validators = 11;
Comment on lines +216 to +218
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tip

Codebase Verification

The search results indicate that the field max_provider_consensus_validators is defined in the provider.proto file but there is no evidence of its documentation or validation in the codebase.

  • Ensure that the new field max_provider_consensus_validators is properly documented.
  • Implement validation logic for max_provider_consensus_validators to ensure it is correctly configured and used.
Analysis chain

Ensure the new field max_provider_consensus_validators is properly documented and validated.

Scripts executed

The following scripts were executed for the analysis:


Script:

#!/bin/bash
# Description: Verify that the new field is used in the consensus engine configuration.

# Test: Search for the usage of `max_provider_consensus_validators`. Expect: Configurations or validations that use this field.
rg --type go $'max_provider_consensus_validators'

Length of output: 407



Script:

#!/bin/bash
# Description: Verify that the new field is documented and validated in the consensus engine configuration.

# Test: Search for documentation related to `max_provider_consensus_validators`.
rg --type go --type md --type proto 'max_provider_consensus_validators'

# Test: Search for validation logic related to `max_provider_consensus_validators`.
rg --type go 'validate.*max_provider_consensus_validators'

Length of output: 162



Script:

#!/bin/bash
# Description: Verify that the new field is documented and validated in the consensus engine configuration.

# Test: Search for documentation related to `max_provider_consensus_validators`.
rg --type go --type md --type 'proto' 'max_provider_consensus_validators'

# Test: Search for validation logic related to `max_provider_consensus_validators`.
rg --type go 'validate.*max_provider_consensus_validators'

Length of output: 162



Script:

#!/bin/bash
# Description: Verify that the new field is documented and validated in the consensus engine configuration.

# Test: Search for documentation related to `max_provider_consensus_validators`.
rg --type go --type md --glob '**/*.proto' 'max_provider_consensus_validators'

# Test: Search for validation logic related to `max_provider_consensus_validators`.
rg --type go 'validate.*max_provider_consensus_validators'

Length of output: 244

}

// SlashAcks contains cons addresses of consumer chain validators
Expand Down
7 changes: 7 additions & 0 deletions x/ccv/provider/keeper/params.go
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,12 @@ func (k Keeper) GetBlocksPerEpoch(ctx sdk.Context) int64 {
return b
}

func (k Keeper) GetMaxProviderConsensusValidators(ctx sdk.Context) int64 {
var m int64
k.paramSpace.Get(ctx, types.KeyMaxProviderConsensusValidators, &m)
return m
}

// GetParams returns the paramset for the provider module
func (k Keeper) GetParams(ctx sdk.Context) types.Params {
return types.NewParams(
Expand All @@ -97,6 +103,7 @@ func (k Keeper) GetParams(ctx sdk.Context) types.Params {
k.GetSlashMeterReplenishFraction(ctx),
k.GetConsumerRewardDenomRegistrationFee(ctx),
k.GetBlocksPerEpoch(ctx),
k.GetMaxProviderConsensusValidators(ctx),
)
}

Expand Down
47 changes: 47 additions & 0 deletions x/ccv/provider/keeper/provider_consensus.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
package keeper

import (
sdk "github.com/cosmos/cosmos-sdk/types"

"github.com/cosmos/interchain-security/v4/x/ccv/provider/types"
)

// SetLastProviderConsensusValidator sets the given validator to be stored
// as part of the last provider consensus validator set
func (k Keeper) SetLastProviderConsensusValidator(
ctx sdk.Context,
validator types.ConsumerValidator,
) {
k.setValidator(ctx, []byte{types.LastProviderConsensusValsPrefix}, validator)
}

// SetLastProviderConsensusValSet resets the stored last validator set sent to the consensus engine on the provider
// to the provided nextValidators.
func (k Keeper) SetLastProviderConsensusValSet(ctx sdk.Context, nextValidators []types.ConsumerValidator) {
k.setValSet(ctx, []byte{types.LastProviderConsensusValsPrefix}, nextValidators)
}

// DeleteLastProviderConsensusValidator removes the validator with `providerConsAddr` address
// from the stored last provider consensus validator set
func (k Keeper) DeleteLastProviderConsensusValidator(
ctx sdk.Context,
providerConsAddr types.ProviderConsAddress,
) {
k.deleteValidator(ctx, []byte{types.LastProviderConsensusValsPrefix}, providerConsAddr)
}

// DeleteLastProviderConsensusValSet deletes all the stored validators from the
// last provider consensus validator set
func (k Keeper) DeleteLastProviderConsensusValSet(
ctx sdk.Context,
) {
k.deleteValSet(ctx, []byte{types.LastProviderConsensusValsPrefix})
}

// GetLastProviderConsensusValSet returns the last stored
// validator set sent to the consensus engine on the provider
func (k Keeper) GetLastProviderConsensusValSet(
ctx sdk.Context,
) []types.ConsumerValidator {
return k.getValSet(ctx, []byte{types.LastProviderConsensusValsPrefix})
}
102 changes: 102 additions & 0 deletions x/ccv/provider/keeper/provider_consensus_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
package keeper_test

import (
"testing"

"github.com/cometbft/cometbft/proto/tendermint/crypto"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please reorder the imports according to the Golang style guide.

-	"github.com/cometbft/cometbft/proto/tendermint/crypto"
+	"github.com/stretchr/testify/require"
+	"github.com/cometbft/cometbft/proto/tendermint/crypto"

Committable suggestion was skipped due to low confidence.

Tools
golangci-lint

6-6: File is not gci-ed with --skip-generated -s standard -s default -s blank -s dot -s prefix(cosmossdk.io) -s prefix(github.com/cosmos/cosmos-sdk) -s prefix(github.com/cometbft/cometbft) -s prefix(github.com/cosmos/interchain-security) --custom-order (gci)

"github.com/stretchr/testify/require"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please reorder the imports according to the Golang style guide.

-	"github.com/stretchr/testify/require"
+	"github.com/cosmos/interchain-security/v4/testutil/keeper"
+	"github.com/cosmos/interchain-security/v4/x/ccv/provider/types"
+	"github.com/stretchr/testify/require"

Committable suggestion was skipped due to low confidence.

Tools
golangci-lint

8-8: File is not gci-ed with --skip-generated -s standard -s default -s blank -s dot -s prefix(cosmossdk.io) -s prefix(github.com/cosmos/cosmos-sdk) -s prefix(github.com/cometbft/cometbft) -s prefix(github.com/cosmos/interchain-security) --custom-order (gci)

testkeeper "github.com/cosmos/interchain-security/v4/testutil/keeper"
"github.com/cosmos/interchain-security/v4/x/ccv/provider/types"
)

func TestSetLastProviderConsensusValidator(t *testing.T) {
providerKeeper, ctx, ctrl, _ := testkeeper.GetProviderKeeperAndCtx(t, testkeeper.NewInMemKeeperParams(t))
defer ctrl.Finish()

validator := types.ConsumerValidator{
ProviderConsAddr: []byte("providerConsAddr"),
Power: 2,
ConsumerPublicKey: &crypto.PublicKey{},
}

providerKeeper.SetLastProviderConsensusValidator(ctx, validator)

// Retrieve the stored validator
storedValidator := providerKeeper.GetLastProviderConsensusValSet(ctx)[0]

require.Equal(t, validator, storedValidator, "stored validator does not match")
}

func TestSetLastProviderConsensusValSet(t *testing.T) {
providerKeeper, ctx, ctrl, _ := testkeeper.GetProviderKeeperAndCtx(t, testkeeper.NewInMemKeeperParams(t))
defer ctrl.Finish()

validator1 := types.ConsumerValidator{
ProviderConsAddr: []byte("providerConsAddr1"),
Power: 2,
ConsumerPublicKey: &crypto.PublicKey{},
}

validator2 := types.ConsumerValidator{
ProviderConsAddr: []byte("providerConsAddr2"),
Power: 3,
ConsumerPublicKey: &crypto.PublicKey{},
}

nextValidators := []types.ConsumerValidator{validator1, validator2}

providerKeeper.SetLastProviderConsensusValSet(ctx, nextValidators)

// Retrieve the stored validator set
storedValidators := providerKeeper.GetLastProviderConsensusValSet(ctx)
require.Equal(t, nextValidators, storedValidators, "stored validator set does not match")
}

func TestDeleteLastProviderConsensusValidator(t *testing.T) {
providerKeeper, ctx, ctrl, _ := testkeeper.GetProviderKeeperAndCtx(t, testkeeper.NewInMemKeeperParams(t))
defer ctrl.Finish()

validator := types.ConsumerValidator{
ProviderConsAddr: []byte("providerConsAddr"),
Power: 2,
ConsumerPublicKey: &crypto.PublicKey{},
}

providerKeeper.SetLastProviderConsensusValidator(ctx, validator)

// Delete the stored validator
providerKeeper.DeleteLastProviderConsensusValidator(ctx, types.NewProviderConsAddress(validator.ProviderConsAddr))

// Ensure the validator is deleted
storedValidators := providerKeeper.GetLastProviderConsensusValSet(ctx)
require.Empty(t, storedValidators, "validator set should be empty")
}

func TestDeleteLastProviderConsensusValSet(t *testing.T) {
providerKeeper, ctx, ctrl, _ := testkeeper.GetProviderKeeperAndCtx(t, testkeeper.NewInMemKeeperParams(t))
defer ctrl.Finish()

validator1 := types.ConsumerValidator{
ProviderConsAddr: []byte("providerConsAddr1"),
Power: 2,
ConsumerPublicKey: &crypto.PublicKey{},
}

validator2 := types.ConsumerValidator{
ProviderConsAddr: []byte("providerConsAddr2"),
Power: 3,
ConsumerPublicKey: &crypto.PublicKey{},
}

nextValidators := []types.ConsumerValidator{validator1, validator2}

providerKeeper.SetLastProviderConsensusValSet(ctx, nextValidators)

// Delete the stored validator set
providerKeeper.DeleteLastProviderConsensusValSet(ctx)

// Ensure the validator set is empty
storedValidators := providerKeeper.GetLastProviderConsensusValSet(ctx)
require.Empty(t, storedValidators, "validator set should be empty")
}
45 changes: 45 additions & 0 deletions x/ccv/provider/keeper/relay.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ import (
sdk "github.com/cosmos/cosmos-sdk/types"
stakingtypes "github.com/cosmos/cosmos-sdk/x/staking/types"

abci "github.com/cometbft/cometbft/abci/types"
"github.com/cosmos/interchain-security/v4/x/ccv/provider/types"
Comment on lines +15 to +16
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ensure the Keeper type is correctly defined and accessible in this file.

It appears that the Keeper type might not be defined or imported correctly, as indicated by the static analysis tool. Please verify the definition and import statements.

providertypes "github.com/cosmos/interchain-security/v4/x/ccv/provider/types"
ccv "github.com/cosmos/interchain-security/v4/x/ccv/types"
)
Expand Down Expand Up @@ -256,6 +258,49 @@ func (k Keeper) QueueVSCPackets(ctx sdk.Context) {
k.IncrementValidatorSetUpdateId(ctx)
}

func (k Keeper) ProviderValidatorUpdates(ctx sdk.Context) []abci.ValidatorUpdate {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method is quite important and is not called in any test.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Also add docstring.

// get the bonded validators from the staking module
bondedValidators := k.stakingKeeper.GetLastValidators(ctx)

// get the last validator set sent to consensus
currentValidators := k.GetLastProviderConsensusValSet(ctx)

MAX_CONSENSUS_VALIDATORS := 180 // TODO: make this a parameter

nextValidators := []types.ConsumerValidator{}
for _, val := range bondedValidators[:MAX_CONSENSUS_VALIDATORS] {
// create the validator from the staking validator
consAddr, err := val.GetConsAddr()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could potentially reuse CreateConsumerValidator with an empty chainID.

if err != nil {
k.Logger(ctx).Error("could not create consumer validator",
"validator", val.GetOperator().String(),
"error", err)
continue
}
pubKey, err := val.TmConsPublicKey()
if err != nil {
k.Logger(ctx).Error("could not create consumer validator",
"validator", val.GetOperator().String(),
"error", err)
continue
}
nextValidator := types.ConsumerValidator{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

General comment but maybe we should not be calling those **Consumer** validators anymore because they're also provider validators.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, agreed. Didn't think of a good, unambigious way for this yet

ProviderConsAddr: consAddr,
ConsumerPublicKey: &pubKey,
Power: k.stakingKeeper.GetLastValidatorPower(ctx, val.GetOperator()),
}

nextValidators = append(nextValidators, nextValidator)
}

// store the validator set we will send to consensus
k.SetLastProviderConsensusValSet(ctx, nextValidators)

valUpdates := DiffValidators(currentValidators, nextValidators)

return valUpdates
}

// BeginBlockCIS contains the BeginBlock logic needed for the Consumer Initiated Slashing sub-protocol.
func (k Keeper) BeginBlockCIS(ctx sdk.Context) {
// Replenish slash meter if necessary. This ensures the meter value is replenished before handling any slash packets,
Expand Down
94 changes: 94 additions & 0 deletions x/ccv/provider/keeper/validator_set_storage.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
package keeper

import (
"fmt"

sdk "github.com/cosmos/cosmos-sdk/types"

"github.com/cosmos/interchain-security/v4/x/ccv/provider/types"
)

// getValidatorKey constructs the key to access a given validator, stored under a given prefix.
func (k Keeper) getValidatorKey(prefix []byte, providerAddr types.ProviderConsAddress) []byte {
return append(prefix, providerAddr.ToSdkConsAddr()...)
}

// setValidator stores the given `validator` in the validator set stored under the given prefix.
func (k Keeper) setValidator(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All these methods should be tested.

ctx sdk.Context,
prefix []byte,
validator types.ConsumerValidator,
) {
store := ctx.KVStore(k.storeKey)
bz, err := validator.Marshal()
if err != nil {
panic(fmt.Errorf("failed to marshal ConsumerValidator: %w", err))

Check warning

Code scanning / CodeQL

Panic in BeginBock or EndBlock consensus methods Warning

Possible panics in BeginBock- or EndBlock-related consensus methods could cause a chain halt
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replace panics with proper error handling in the setValidator and getValSet methods to prevent potential chain halts if errors occur. This will enhance the robustness and fault tolerance of the consensus-related methods.

- panic(fmt.Errorf("failed to marshal ConsumerValidator: %w", err))
+ return fmt.Errorf("failed to marshal ConsumerValidator: %w", err)
- panic(fmt.Errorf("failed to unmarshal ConsumerValidator: %w", err))
+ return fmt.Errorf("failed to unmarshal ConsumerValidator: %w", err)

Also applies to: 89-89

Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change
panic(fmt.Errorf("failed to marshal ConsumerValidator: %w", err))
return fmt.Errorf("failed to marshal ConsumerValidator: %w", err)
Tools
GitHub Check: CodeQL

[warning] 26-26: Panic in BeginBock or EndBlock consensus methods
Possible panics in BeginBock- or EndBlock-related consensus methods could cause a chain halt

}

store.Set(k.getValidatorKey(prefix, types.NewProviderConsAddress(validator.ProviderConsAddr)), bz)
}

// setValSet resets the validator set stored under the given prefix to the provided `nextValidators`.
func (k Keeper) setValSet(ctx sdk.Context, prefix []byte, nextValidators []types.ConsumerValidator) {
k.deleteValSet(ctx, prefix)
for _, val := range nextValidators {
k.setValidator(ctx, prefix, val)
}
}

// deleteValidator removes validator with `providerAddr` address from the
// validator set stored under the given prefix.
func (k Keeper) deleteValidator(
ctx sdk.Context,
prefix []byte,
providerConsAddr types.ProviderConsAddress,
) {
store := ctx.KVStore(k.storeKey)
store.Delete(k.getValidatorKey(prefix, providerConsAddr))
}

// deleteValSet deletes all the stored consumer validators under the given prefix.
func (k Keeper) deleteValSet(
ctx sdk.Context,
prefix []byte,
) {
store := ctx.KVStore(k.storeKey)
iterator := sdk.KVStorePrefixIterator(store, prefix)

var keysToDel [][]byte
defer iterator.Close()
for ; iterator.Valid(); iterator.Next() {
keysToDel = append(keysToDel, iterator.Key())
}
for _, delKey := range keysToDel {
store.Delete(delKey)
}
}

// isValidator returns `true` if the validator with `providerAddr` exists
// in the validator set stored under the given prefix.
func (k Keeper) isValidator(ctx sdk.Context, prefix []byte, providerAddr types.ProviderConsAddress) bool {
store := ctx.KVStore(k.storeKey)
return store.Get(k.getValidatorKey(prefix, providerAddr)) != nil
}

// getValSet returns all the validators stored under the given prefix.
func (k Keeper) getValSet(
ctx sdk.Context,
key []byte,
) (validators []types.ConsumerValidator) {
store := ctx.KVStore(k.storeKey)
iterator := sdk.KVStorePrefixIterator(store, key)
defer iterator.Close()

for ; iterator.Valid(); iterator.Next() {
iterator.Value()
var validator types.ConsumerValidator
if err := validator.Unmarshal(iterator.Value()); err != nil {
panic(fmt.Errorf("failed to unmarshal ConsumerValidator: %w", err))

Check warning

Code scanning / CodeQL

Panic in BeginBock or EndBlock consensus methods Warning

Possible panics in BeginBock- or EndBlock-related consensus methods could cause a chain halt
}
validators = append(validators, validator)
}

return validators
}
Loading
Loading