Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace difftests with model-based testing using Quint #1368

Closed
wants to merge 83 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
83 commits
Select commit Hold shift + click to select a range
e2b6407
Start new attempt on Quint model of ICS
p-offtermatt Aug 17, 2023
283dc0a
Advance quint model
p-offtermatt Aug 23, 2023
2155a4f
Add first finished draft of model
p-offtermatt Aug 24, 2023
6ca6510
Add test run to model
p-offtermatt Aug 25, 2023
e27e74b
Rename model, add test, use powerset for nondeterminism
p-offtermatt Aug 25, 2023
50fae34
Reintroduce vsc changed tracking variables
p-offtermatt Aug 25, 2023
78e890a
Add text with what expliticly is modelled
p-offtermatt Aug 25, 2023
a1d57ef
Add bluespec to ccv.qnt
p-offtermatt Aug 25, 2023
6c91866
Add bluespec to expraSpells.qnt
p-offtermatt Aug 25, 2023
2126eac
Add docstring to extraSpells module
p-offtermatt Sep 5, 2023
1320b95
Start rewriting model
p-offtermatt Sep 18, 2023
5fa449c
Revert "Start rewriting model"
p-offtermatt Sep 18, 2023
c05d9e6
Start rewriting quint model
p-offtermatt Sep 18, 2023
4bbe033
Continue seperating logic in Quint model
p-offtermatt Sep 19, 2023
5ebab39
Start debugging cryptic error message
p-offtermatt Sep 20, 2023
5bca6fc
Start adding endAndBeginBlock defs
p-offtermatt Sep 20, 2023
4f77d68
Diagnose Quint parser bug
p-offtermatt Sep 21, 2023
fa233d6
Fix type in Quint
p-offtermatt Sep 21, 2023
66663bf
Add endBlock actions
p-offtermatt Sep 21, 2023
7b489fe
Start adding state machine module
p-offtermatt Sep 26, 2023
b992f8c
Save status with crashing effect checker
p-offtermatt Sep 26, 2023
baaddb7
Resolve issue by removing undefined field
p-offtermatt Sep 26, 2023
2c1341d
Remove add
p-offtermatt Sep 27, 2023
e09968d
Fix init
p-offtermatt Sep 27, 2023
55ab595
Snapshot spec with parser crasher
p-offtermatt Sep 28, 2023
0897e8c
Snapshot model
p-offtermatt Sep 28, 2023
d227aee
Start debugging tests
p-offtermatt Sep 28, 2023
09b1b87
Finish test for quint model
p-offtermatt Sep 28, 2023
96c101f
Add README and improve folder structure
p-offtermatt Sep 29, 2023
aa2989e
Fix import
p-offtermatt Sep 29, 2023
c291303
Add some invariants
p-offtermatt Sep 29, 2023
94a0acb
Refactor Consumer advancement
p-offtermatt Sep 29, 2023
8818531
Snapshot error
p-offtermatt Sep 29, 2023
633f8bb
Make time module upper case
p-offtermatt Sep 29, 2023
f28d074
Add invariants
p-offtermatt Sep 29, 2023
c0363f2
Clean up invariants
p-offtermatt Sep 29, 2023
b9d8caf
Add script to run many invariants
p-offtermatt Sep 29, 2023
e494a09
Update model
p-offtermatt Oct 2, 2023
289ccad
Update model for bug reporting]
p-offtermatt Oct 2, 2023
45cfc5c
Remove sanity check script
p-offtermatt Oct 2, 2023
4fdc2ff
Fix model and randomly run invariant checks
p-offtermatt Oct 2, 2023
99744ff
Remove trace
p-offtermatt Oct 2, 2023
1bc4610
Add model checking to README
p-offtermatt Oct 2, 2023
9e4f9c4
Add bluespec
p-offtermatt Oct 2, 2023
ca50ee2
Try fixed bluespec
p-offtermatt Oct 2, 2023
a80ce70
Fix bluespec definitions
p-offtermatt Oct 2, 2023
918e71a
Update tests/difference/core/quint_model/README.md
p-offtermatt Oct 5, 2023
8da4cfe
Update tests/difference/core/quint_model/README.md
p-offtermatt Oct 5, 2023
2a68236
Fix minor issues
p-offtermatt Oct 5, 2023
2ad4deb
Update tests/difference/core/quint_model/README.md
p-offtermatt Oct 6, 2023
9d40842
Update tests/difference/core/quint_model/README.md
p-offtermatt Oct 6, 2023
3eeb3fd
Update tests/difference/core/quint_model/README.md
p-offtermatt Oct 6, 2023
2000272
Correct verify command by adding \
p-offtermatt Oct 6, 2023
7cd826b
Add Inv to ValidatorUpdatesArePropagated
p-offtermatt Oct 6, 2023
7df2b1f
Update tests/difference/core/quint_model/README.md
p-offtermatt Oct 6, 2023
3633dc4
Apply comments
p-offtermatt Oct 10, 2023
dc7e687
Rename VSC to Vsc
p-offtermatt Oct 10, 2023
728b024
Return plain ProtocolState in cases where no error is returned anyways
p-offtermatt Oct 10, 2023
641d845
Remove unused defs
p-offtermatt Oct 10, 2023
c2aa6d0
Fix indentation
p-offtermatt Oct 10, 2023
75b1e11
Rename to isRunningConsumer
p-offtermatt Oct 10, 2023
f46554b
Unify naming for extraSpells
p-offtermatt Oct 10, 2023
3630846
Remove HasSubsequence
p-offtermatt Oct 10, 2023
90f6652
Run tests before running invariants
p-offtermatt Oct 10, 2023
13a32fc
Rename modules to have same name as files
p-offtermatt Oct 11, 2023
852dac4
Adjust module name in README and invariant script
p-offtermatt Oct 11, 2023
8aee401
Fix voting power change behaviour around 0
p-offtermatt Oct 11, 2023
d75d5c5
Adjust error message in test
p-offtermatt Oct 11, 2023
f294eec
Remove special treatment of 0 voting power
p-offtermatt Oct 11, 2023
4d87372
Rename sentVscPackets to sentVscPacketsToConsumer
p-offtermatt Oct 12, 2023
53d5bff
Update tests/difference/core/quint_model/README.md
p-offtermatt Oct 12, 2023
738a347
Resolve comments
p-offtermatt Oct 12, 2023
c7fe7c5
Merge branch 'ph/quint-model-v2' of https://github.com/cosmos/interch…
p-offtermatt Oct 12, 2023
3a03fcd
Adjust comment to fit actual time advancement
p-offtermatt Oct 12, 2023
d71dad3
Remove hasError field and make it a function
p-offtermatt Oct 13, 2023
455696a
Adjust docstring
p-offtermatt Oct 13, 2023
d422cb4
Remove unused timedout val
p-offtermatt Oct 13, 2023
6ed82c8
Update doc
p-offtermatt Oct 13, 2023
b817f43
Rename statemachine to model
p-offtermatt Oct 16, 2023
70cd774
Use ... syntax
p-offtermatt Oct 16, 2023
86324b7
Change Error type to string
p-offtermatt Oct 16, 2023
b856081
Start adding mbt using Quint model
p-offtermatt Oct 19, 2023
234a89e
Merge branch 'main' into ph/mbt
p-offtermatt Oct 19, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
module github.com/cosmos/interchain-security/v3

go 1.20
go 1.21.1

toolchain go1.21.3

require (
cosmossdk.io/errors v1.0.0
Expand Down Expand Up @@ -171,6 +173,7 @@ require (
)

require (
github.com/informalsystems/itf-go v0.0.1 // indirect
cosmossdk.io/log v1.2.1 // indirect
github.com/cockroachdb/errors v1.10.0 // indirect
github.com/cockroachdb/logtags v0.0.0-20230118201751-21c54148d20b // indirect
Expand Down
2 changes: 2 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -774,6 +774,8 @@ github.com/influxdata/promql/v2 v2.12.0/go.mod h1:fxOPu+DY0bqCTCECchSRtWfc+0X19y
github.com/influxdata/roaring v0.4.13-0.20180809181101-fc520f41fab6/go.mod h1:bSgUQ7q5ZLSO+bKBGqJiCBGAl+9DxyW63zLTujjUlOE=
github.com/influxdata/tdigest v0.0.0-20181121200506-bf2b5ad3c0a9/go.mod h1:Js0mqiSBE6Ffsg94weZZ2c+v/ciT8QRHFOap7EKDrR0=
github.com/influxdata/usage-client v0.0.0-20160829180054-6d3895376368/go.mod h1:Wbbw6tYNvwa5dlB6304Sd+82Z3f7PmVZHVKU637d4po=
github.com/informalsystems/itf-go v0.0.1 h1:lVvdg3v+IMWOsVfIvOOGy1hHFO5KxoS8b8EiwKLbQDg=
github.com/informalsystems/itf-go v0.0.1/go.mod h1:wgqaQ/yl2kbNlgw6GaleuHEefpZvkZo6Hc0jc8cGG9M=
github.com/jackpal/go-nat-pmp v1.0.2/go.mod h1:QPH045xvCAeXUZOxsnwmrtiCoxIr9eob+4orBN1SBKc=
github.com/jedisct1/go-minisign v0.0.0-20190909160543-45766022959e/go.mod h1:G1CVv03EnqU1wYL2dFwXxW2An0az9JTl/ZsqXQeBlkU=
github.com/jessevdk/go-flags v0.0.0-20141203071132-1679536dcc89/go.mod h1:4FA24M0QyGHXBuZZK/XkWh8h0e1EYbRYJSGM75WSRxI=
Expand Down
33 changes: 28 additions & 5 deletions tests/difference/core/quint_model/ccv_model.qnt
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,16 @@ module ccv_model {

import ccv(VscTimeout = 5 * Week, CcvTimeout = ccvTimeouts, UnbondingPeriodPerChain = unbondingPeriods, ConsumerChains = consumerChains).* from "./ccv"

type Parameters = {
VscTimeout: Time,
CcvTimeout: Chain -> Time,
UnbondingPeriodPerChain: Chain -> Time,
ConsumerChains: Set[Chain],
Nodes: Set[Node],
InitialValidatorSet: Node -> int,
}

var params: Parameters

var currentState: ProtocolState

Expand Down Expand Up @@ -126,14 +136,23 @@ module ccv_model {
consumerStates: consumerStates
},
trace' = List(emptyAction.with("kind", "init")),
params' = {
VscTimeout: VscTimeout,
CcvTimeout: CcvTimeout,
UnbondingPeriodPerChain: UnbondingPeriodPerChain,
ConsumerChains: ConsumerChains,
Nodes: nodes,
InitialValidatorSet: InitialValidatorSet,
}
}

action VotingPowerChange(validator: Node, newVotingPower: int): bool =
val result = votingPowerChange(currentState, validator, newVotingPower)
all {
result.hasError() == false,
currentState' = result.newState,
trace' = trace.append(emptyAction.with("kind", "VotingPowerChange").with("validator", validator).with("newVotingPower", newVotingPower))
trace' = trace.append(emptyAction.with("kind", "VotingPowerChange").with("validator", validator).with("newVotingPower", newVotingPower)),
params' = params,
}

// The receiver receives the next outstanding VscPacket from the provider.
Expand All @@ -144,7 +163,8 @@ module ccv_model {
all {
result.hasError() == false,
currentState' = result.newState,
trace' = trace.append(emptyAction.with("kind", "DeliverVscPacket").with("consumerChain", receiver))
trace' = trace.append(emptyAction.with("kind", "DeliverVscPacket").with("consumerChain", receiver)),
params' = params,
}

// The provider receives the next outstanding VscMaturedPacket from the sender.
Expand All @@ -155,7 +175,8 @@ module ccv_model {
all {
result.hasError() == false,
currentState' = result.newState,
trace' = trace.append(emptyAction.with("kind", "DeliverVscMaturedPacket").with("consumerChain", sender))
trace' = trace.append(emptyAction.with("kind", "DeliverVscMaturedPacket").with("consumerChain", sender)),
params' = params,
}

action EndAndBeginBlockForProvider(
Expand All @@ -166,7 +187,8 @@ module ccv_model {
all {
result.hasError() == false,
currentState' = result.newState,
trace' = trace.append(emptyAction.with("kind", "EndAndBeginBlockForProvider").with("timeAdvancement", timeAdvancement).with("consumersToStart", consumersToStart).with("consumersToStop", consumersToStop))
trace' = trace.append(emptyAction.with("kind", "EndAndBeginBlockForProvider").with("timeAdvancement", timeAdvancement).with("consumersToStart", consumersToStart).with("consumersToStop", consumersToStop)),
params' = params,
}

action EndAndBeginBlockForConsumer(
Expand All @@ -176,7 +198,8 @@ module ccv_model {
all {
result.hasError() == false,
currentState' = result.newState,
trace' = trace.append(emptyAction.with("kind", "EndAndBeginBlockForConsumer").with("consumerChain", chain).with("timeAdvancement", timeAdvancement))
trace' = trace.append(emptyAction.with("kind", "EndAndBeginBlockForConsumer").with("consumerChain", chain).with("timeAdvancement", timeAdvancement)),
params' = params,
}

// ==================
Expand Down
96 changes: 96 additions & 0 deletions tests/difference/core/quint_model/driver/common.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
package main

import (
"time"

sdk "github.com/cosmos/cosmos-sdk/types"
stakingtypes "github.com/cosmos/cosmos-sdk/x/staking/types"

tmproto "github.com/cometbft/cometbft/proto/tendermint/types"
tmtypes "github.com/cometbft/cometbft/types"
)

const (
P = "provider"
C = "consumer"
)

// ValStates represents the total delegation
// and bond status of a validator
type ValStates struct {
Delegation []int
Tokens []int
ValidatorExtraTokens []int
Status []stakingtypes.BondStatus
}

type InitState struct {
PKSeeds []string
NumValidators int
MaxValidators int
InitialDelegatorTokens int
SlashDoublesign sdk.Dec
SlashDowntime sdk.Dec
UnbondingP time.Duration
UnbondingC time.Duration
Trusting time.Duration
MaxClockDrift time.Duration
BlockInterval time.Duration
ConsensusParams *tmproto.ConsensusParams
ValStates ValStates
MaxEntries int
}

var initStateVar InitState

func init() {
// tokens === power
sdk.DefaultPowerReduction = sdk.NewInt(1)
initStateVar = InitState{
PKSeeds: []string{
// Fixed seeds are used to create the private keys for validators.
// The seeds are chosen to ensure that the resulting validators are
// sorted in descending order by the staking module.
"bbaaaababaabbaabababbaabbbbbbaaa",
"abbbababbbabaaaaabaaabbbbababaab",
"bbabaabaabbbbbabbbaababbbbabbbbb",
"aabbbabaaaaababbbabaabaabbbbbbba",
},
NumValidators: 4,
MaxValidators: 2,
InitialDelegatorTokens: 10000000000000,
SlashDoublesign: sdk.NewDec(0),
SlashDowntime: sdk.NewDec(0),
UnbondingP: time.Second * 70,
UnbondingC: time.Second * 50,
Trusting: time.Second * 49,
MaxClockDrift: time.Second * 10000,
BlockInterval: time.Second * 6,
ValStates: ValStates{
Delegation: []int{4000, 3000, 2000, 1000},
Tokens: []int{5000, 4000, 3000, 2000},
ValidatorExtraTokens: []int{1000, 1000, 1000, 1000},
Status: []stakingtypes.BondStatus{
stakingtypes.Bonded, stakingtypes.Bonded,
stakingtypes.Unbonded, stakingtypes.Unbonded,
},
},
MaxEntries: 1000000,
ConsensusParams: &tmproto.ConsensusParams{
Block: &tmproto.BlockParams{
MaxBytes: 9223372036854775807,
MaxGas: 9223372036854775807,
},
Evidence: &tmproto.EvidenceParams{
MaxAgeNumBlocks: 302400,
MaxAgeDuration: 504 * time.Hour, // 3 weeks is the max duration
MaxBytes: 10000,
},
Validator: &tmproto.ValidatorParams{
PubKeyTypes: []string{
tmtypes.ABCIPubKeyTypeEd25519,
},
},
},
}
}
Loading
Loading