Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: finality gadget server #13

Merged
merged 97 commits into from
Aug 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
97 commits
Select commit Hold shift + click to select a range
b497f53
docs: add readme
parketh Jul 9, 2024
c12bc2a
feat: scaffold initial services
parketh Jul 11, 2024
24a69dc
docs: update readme
parketh Jul 11, 2024
935c53c
chore: remove old comments
parketh Jul 11, 2024
088d0bf
feat: add mutex for db write operations
parketh Jul 12, 2024
5026dc9
feat: convert db write to tx with rollback
parketh Jul 12, 2024
c422e3f
chore: rename main -> demo
parketh Jul 16, 2024
fd6d939
Merge remote-tracking branch 'origin/main' into feat/verifier-daemon
parketh Jul 16, 2024
c7fca29
fix: update verifier for sdk configs
parketh Jul 16, 2024
36562c7
feat: run server inside verifier
parketh Jul 17, 2024
607236d
fix: remove hardcode sql script
parketh Jul 25, 2024
033ba5a
fix: sql to handle query non locally stored blocks
parketh Jul 25, 2024
6d2abd4
Revert "fix: sql to handle query non locally stored blocks"
parketh Jul 25, 2024
a0ec321
fix: remove incorrect error check
parketh Jul 25, 2024
102f9ff
debug: add test messages
parketh Jul 25, 2024
f959591
fix: handle latest finalised block 0 exception
parketh Jul 25, 2024
99a41ef
fix: log errors in demo process
parketh Jul 25, 2024
f50db6a
fix: move server port and pg conn into configs
parketh Jul 25, 2024
0bc78bc
feat: add `processNBlocks` for testing, improve block handling on res…
parketh Jul 26, 2024
1a197c1
fix: handle non-existent local block
parketh Jul 26, 2024
709e3e4
chore: remove debugging logs
parketh Jul 26, 2024
ed98fb9
Merge remote-tracking branch 'origin/main' into feat/verifier-daemon
parketh Jul 26, 2024
afbaa45
chore: fix lint errors
parketh Jul 26, 2024
ac51468
feat: refactor to bbolt db
parketh Jul 27, 2024
bdae82c
fix: throw if block height is 0
parketh Jul 28, 2024
43e997c
docs: update verifier readme
parketh Jul 28, 2024
79b58ee
fix: gitignore .db
parketh Jul 28, 2024
1030a5d
feat: create cli daemon and config file
parketh Jul 30, 2024
60275d9
feat: improve logging
parketh Jul 30, 2024
6c982ae
chore: consolidate verifier code
parketh Jul 30, 2024
faa7df1
fix: lint errors
parketh Jul 30, 2024
18bc36c
fix: handle server error
parketh Jul 30, 2024
0063e32
fix: gracefully handle server shutdown
parketh Jul 30, 2024
3970dc1
fix: gracefully shutdown verifier
parketh Jul 30, 2024
c64b520
fix: remove ProcessNBlocks
parketh Jul 30, 2024
4a400c8
fix: handle stop error
parketh Jul 30, 2024
b167bcd
fix: handle non-existent latest consecutively finalized block
parketh Jul 30, 2024
a7ee7dc
chore: rm deprecated pg handler
parketh Jul 30, 2024
43901f4
chore: fix formatting
parketh Jul 30, 2024
cf60731
fix: remove .db
parketh Jul 30, 2024
58fd023
chore: use relative path for db
parketh Jul 30, 2024
9a13ebb
docs: update readme
parketh Jul 30, 2024
00dd6df
chore: fix formatting
parketh Jul 30, 2024
71166c9
chore: clean unused deps
parketh Jul 30, 2024
f683157
feat: remove duplicate tracker for consecutively finalized block
parketh Jul 30, 2024
0fae16c
chore: restructure verifier as part of sdk
parketh Jul 30, 2024
8ca6a65
fix: remove isFinalized bool from db type
parketh Jul 30, 2024
f70295f
feat: refactor http -> grpc and rename verifier -> server
parketh Jul 31, 2024
cc19f28
feat: refactor http -> grpc and rename verifier -> server
parketh Jul 31, 2024
7a3ce41
Merge remote-tracking branch 'origin/main' into feat/verifier-daemon
parketh Aug 1, 2024
018e67e
fix: deps
parketh Aug 1, 2024
036cb07
fix: remove grpc server ping to fix name conflict
parketh Aug 1, 2024
370fce1
debug: add logging
parketh Aug 5, 2024
faab534
debug: add more logging and bug fixes
parketh Aug 5, 2024
910f07b
debug: add err logging for rpc client
parketh Aug 5, 2024
fc7ba0f
debug: use fmt for logging
parketh Aug 5, 2024
48a40d0
Revert "debug: use fmt for logging"
parketh Aug 5, 2024
be767cb
Revert "debug: add err logging for rpc client"
parketh Aug 5, 2024
4b8d292
Revert "debug: add more logging and bug fixes"
parketh Aug 5, 2024
e615f7a
Revert "debug: add logging"
parketh Aug 5, 2024
b187992
feat: add method to delete local DB
parketh Aug 5, 2024
c2b799e
fix: delete db path
parketh Aug 5, 2024
8e72d67
feat: allow cancel ProcessBlocks with ctx
parketh Aug 5, 2024
fb077df
chore: remove print
parketh Aug 5, 2024
4729404
update package name (#11)
lesterli Aug 2, 2024
62491f1
fix: update package name
parketh Aug 5, 2024
4ff43c7
fix: block status queries
parketh Aug 5, 2024
02a49df
feat: rename daemon and add build instructions
parketh Aug 6, 2024
e196544
fix: always persist DB state
parketh Aug 6, 2024
8e84098
chore: rename finality gadget
parketh Aug 6, 2024
fd21437
feat: normalise block hash for querying
parketh Aug 6, 2024
b6e4cf1
test: add db unit tests
parketh Aug 6, 2024
8dfd9cb
feat: refactor repo and merge finalitygadget <> sdk
parketh Aug 6, 2024
efe8b24
test: regenerate mocks
parketh Aug 6, 2024
57516df
chore: cleanup code and standardise client format
parketh Aug 6, 2024
5b71fc7
test: regenerate mocks
parketh Aug 6, 2024
b288c42
feat: use zap logger
parketh Aug 6, 2024
0f98df1
docs: update readme
parketh Aug 6, 2024
556b15b
feat: add getBlockByHash route to db
parketh Aug 6, 2024
4d7124e
feat: add get block routes to finalitygadget
parketh Aug 6, 2024
7328f3c
chore: consolidate errors
parketh Aug 6, 2024
8d37a70
test: add fg unit tests
parketh Aug 6, 2024
04a6c73
feat: add logger
parketh Aug 6, 2024
cfad1d3
fix: lint errors
parketh Aug 6, 2024
28dc39f
docs: add logger to readme
parketh Aug 6, 2024
e9eb77d
Merge remote-tracking branch 'origin/main' into feat/verifier-daemon
parketh Aug 6, 2024
c288422
restore expected_clients
parketh Aug 6, 2024
1252b28
remove unused quit chan
parketh Aug 8, 2024
e63afa9
feat: simplify grpc insertBlock response
parketh Aug 8, 2024
f4119cd
chore: config.toml.example
parketh Aug 8, 2024
d7bb5de
chore: rename fns for consistency
parketh Aug 8, 2024
d6bff6c
nit: fix comment
parketh Aug 8, 2024
4b9a0d8
remove DeleteDB api
parketh Aug 8, 2024
d6015bf
chore: rename fns for consistency
parketh Aug 8, 2024
e22c82f
remove InsertBlock from grpc
parketh Aug 8, 2024
4f1c348
nit: fix comments
parketh Aug 8, 2024
a7ac2fe
chore: rename fns for consistency
parketh Aug 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,6 @@
**/target/
.DS_Store
.DS_Store
**/.env
**/*.db
*.db
config.toml
7 changes: 6 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

MOCKS_DIR=./testutil/mocks

OPFGD_PKG := github.com/babylonlabs-io/finality-gadget/cmd/opfgd

mock-gen:
go install go.uber.org/mock/mockgen@latest
mockgen -source=sdk/client/expected_clients.go -package mocks -destination $(MOCKS_DIR)/expected_clients_mock.go
Expand All @@ -11,4 +13,7 @@ test:
go test -race ./... -v

lint:
golangci-lint run
golangci-lint run

install:
go install -trimpath $(OPFGD_PKG)
81 changes: 74 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,86 @@
# Babylon Finality Gadget

We proposed a [Babylon finality gadget](https://github.com/ethereum-optimism/specs/discussions/218) for OP-stack chains. The finality gadget depends on the EOTS data in a CosmWasm contract deployed on the Babylon chain.
The Babylon Finality Gadget is a program that can be run by users of OP stack L2s to track consecutive L2 block quorum and query the BTC-finalised status of blocks.

We have modified the OP-stack codebase to use the SDK in this codebase for additional finalty checks.
See our [proposal](https://github.com/ethereum-optimism/specs/discussions/218) on Optimism for more details.

In the future, we will also move the CosmWasm contract code here.
## Modules

## Dependencies
- `cmd` : entry point for `opfgd` finality gadget daemon
- `finalitygadget` : top-level umbrella module that exposes query methods and coordinates calls to other clients
- `client` : grpc client to query the finality gadget
- `server` : grpc server for the finality gadget
- `proto` : protobuf definitions for the grpc server
- `config` : configs for the finality gadget
- `btcclient` : wrapper around Bitcoin RPC client
- `bbnclient` : wrapper around Babylon RPC client
- `ethl2client` : wrapper around OP stack L2 ETH RPC client
- `cwclient` : client to query CosmWasm smart contract deployed on BabylonChain
- `db` : handler for local database to store finalized block state
- `types` : common types
- `log` : custom logger
- `testutil` : test utilities and helpers

The SDK requires a BTC RPC client defined in https://github.com/btcsuite/btcd/tree/master/rpcclient. We wrap it in our own BTC Client to make it easier to use.
## Instructions

## Usages
### Download and configuration

To run tests
To get started, clone the repository.

```bash
git clone https://github.com/babylonlabs-io/finality-gadget.git
```

Copy the `config.toml.example` file to `config.toml`:

```bash
cp config.toml.example config.toml
```

Configure the `config.toml` file with the following parameters:

```toml
L2RPCHost = # RPC URL of OP stack L2 chain
BitcoinRPCHost = # Bitcoin RPC URL
DBFilePath = # Path to local bbolt DB file
FGContractAddress = # Babylon finality gadget contract address
BBNChainID = # Babylon chain id
BBNRPCAddress = # Babylon RPC host URL
GRPCServerPort = # Port to run the gRPC server on
PollInterval = # Interval to poll for new L2 blocks
```

### Building and installing the binary

At the top-level directory of the project

```bash
make install
```

The above command will build and install the `opfgd` binary to
`$GOPATH/bin`.

If your shell cannot find the installed binaries, make sure `$GOPATH/bin` is in
the `$PATH` of your shell. Usually these commands will do the job

```bash
export PATH=$HOME/go/bin:$PATH
echo 'export PATH=$HOME/go/bin:$PATH' >> ~/.profile
```

### Running the daemon

To start the daemon, run:

```bash
opfgd start --cfg config.toml
```

### Running tests

To run tests:

```bash
make test
```
85 changes: 77 additions & 8 deletions sdk/bbnclient/bbnclient.go → bbnclient/bbnclient.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,29 @@ import (
"math"

"github.com/babylonlabs-io/babylon/client/query"
"github.com/babylonlabs-io/babylon/x/btcstaking/types"
bbntypes "github.com/babylonlabs-io/babylon/x/btcstaking/types"
sdkquerytypes "github.com/cosmos/cosmos-sdk/types/query"
)

type Client struct {
type BabylonClient struct {
*query.QueryClient
}

func (bbnClient *Client) QueryAllFpBtcPubKeys(consumerId string) ([]string, error) {
//////////////////////////////
// CONSTRUCTOR
//////////////////////////////

func NewBabylonClient(queryClient *query.QueryClient) *BabylonClient {
return &BabylonClient{
QueryClient: queryClient,
}
}

//////////////////////////////
// METHODS
//////////////////////////////

func (bbnClient *BabylonClient) QueryAllFpBtcPubKeys(consumerId string) ([]string, error) {
pagination := &sdkquerytypes.PageRequest{}
resp, err := bbnClient.QueryClient.QueryConsumerFinalityProviders(consumerId, pagination)
if err != nil {
Expand All @@ -27,7 +41,7 @@ func (bbnClient *Client) QueryAllFpBtcPubKeys(consumerId string) ([]string, erro
return pkArr, nil
}

func (bbnClient *Client) QueryFpPower(fpPubkeyHex string, btcHeight uint64) (uint64, error) {
func (bbnClient *BabylonClient) QueryFpPower(fpPubkeyHex string, btcHeight uint64) (uint64, error) {
totalPower := uint64(0)
pagination := &sdkquerytypes.PageRequest{}
// queries the BTCStaking module for all delegations of a finality provider
Expand Down Expand Up @@ -58,7 +72,7 @@ func (bbnClient *Client) QueryFpPower(fpPubkeyHex string, btcHeight uint64) (uin
return totalPower, nil
}

func (bbnClient *Client) QueryMultiFpPower(
func (bbnClient *BabylonClient) QueryMultiFpPower(
fpPubkeyHexList []string,
btcHeight uint64,
) (map[string]uint64, error) {
Expand All @@ -76,7 +90,7 @@ func (bbnClient *Client) QueryMultiFpPower(
}

// QueryEarliestActiveDelBtcHeight returns the earliest active BTC staking height
func (bbnClient *Client) QueryEarliestActiveDelBtcHeight(fpPkHexList []string) (uint64, error) {
func (bbnClient *BabylonClient) QueryEarliestActiveDelBtcHeight(fpPkHexList []string) (uint64, error) {
allFpEarliestDelBtcHeight := uint64(math.MaxUint64)

for _, fpPkHex := range fpPkHexList {
Expand All @@ -92,7 +106,7 @@ func (bbnClient *Client) QueryEarliestActiveDelBtcHeight(fpPkHexList []string) (
return allFpEarliestDelBtcHeight, nil
}

func (bbnClient *Client) QueryFpEarliestActiveDelBtcHeight(fpPubkeyHex string) (uint64, error) {
func (bbnClient *BabylonClient) QueryFpEarliestActiveDelBtcHeight(fpPubkeyHex string) (uint64, error) {
pagination := &sdkquerytypes.PageRequest{
Limit: 100,
}
Expand Down Expand Up @@ -144,14 +158,69 @@ func (bbnClient *Client) QueryFpEarliestActiveDelBtcHeight(fpPubkeyHex string) (
return earliestBtcHeight, nil
}

//////////////////////////////
// INTERNAL
//////////////////////////////

// we implemented exact logic as in GetStatus
// https://github.com/babylonlabs-io/babylon-private/blob/3d8f190c9b0c0795f6546806e3b8582de716cd60/x/btcstaking/types/btc_delegation.go#L90-L111
func (bbnClient *BabylonClient) isDelegationActive(
btcDel *bbntypes.BTCDelegationResponse,
btcHeight uint64,
) (bool, error) {
btccheckpointParams, err := bbnClient.QueryClient.BTCCheckpointParams()
if err != nil {
return false, err
}
btcstakingParams, err := bbnClient.QueryClient.BTCStakingParams()
if err != nil {
return false, err
}
kValue := btccheckpointParams.GetParams().BtcConfirmationDepth
wValue := btccheckpointParams.GetParams().CheckpointFinalizationTimeout
covQuorum := btcstakingParams.GetParams().CovenantQuorum
ud := btcDel.UndelegationResponse

if len(ud.GetDelegatorUnbondingSigHex()) > 0 {
return false, nil
}

// k is not involved in the `GetStatus` logic as Babylon will accept a BTC delegation request
// only when staking tx is k-deep on BTC.
//
// But the msg handler performs both checks 1) ensure staking tx is k-deep, and 2) ensure the
// staking tx's timelock has at least w BTC blocks left.
// (https://github.com/babylonlabs-io/babylon-private/blob/3d8f190c9b0c0795f6546806e3b8582de716cd60/x/btcstaking/keeper/msg_server.go#L283-L292)
//
// So after the msg handler accepts BTC delegation the 1st check is no longer needed
// the k-value check is added per
//
// So in our case, we need to check both to ensure the delegation is active
if btcHeight < btcDel.StartHeight+kValue || btcHeight+wValue > btcDel.EndHeight {
return false, nil
}

if uint32(len(btcDel.CovenantSigs)) < covQuorum {
return false, nil
}
if len(ud.CovenantUnbondingSigList) < int(covQuorum) {
return false, nil
}
if len(ud.CovenantSlashingSigs) < int(covQuorum) {
return false, nil
}

return true, nil
}

// The active delegation needs to satisfy:
// 1) the staking tx is k-deep in Bitcoin, i.e., start_height + k
// 2) it receives a quorum number of covenant committee signatures
//
// return math.MaxUint64 if the delegation is not active
//
// Note: the delegation can be unbounded and that's totally fine and shouldn't affect when the chain was activated
func getDelFirstActiveHeight(btcDel *types.BTCDelegationResponse, latestBtcHeight, kValue uint64, covQuorum uint32) uint64 {
func getDelFirstActiveHeight(btcDel *bbntypes.BTCDelegationResponse, latestBtcHeight, kValue uint64, covQuorum uint32) uint64 {
activationHeight := btcDel.StartHeight + kValue
// not activated yet
if latestBtcHeight < activationHeight || uint32(len(btcDel.CovenantSigs)) < covQuorum {
Expand Down
28 changes: 20 additions & 8 deletions sdk/btcclient/client.go → btcclient/btcclient.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,30 +10,38 @@ import (
"go.uber.org/zap"
)

type BTCClient struct {
type BitcoinClient struct {
client *rpcclient.Client
logger *zap.Logger
cfg *BTCConfig
}

func NewBTCClient(cfg *BTCConfig, logger *zap.Logger) (*BTCClient, error) {
//////////////////////////////
// CONSTRUCTOR
//////////////////////////////

func NewBitcoinClient(cfg *BTCConfig, logger *zap.Logger) (*BitcoinClient, error) {
c, err := rpcclient.New(cfg.ToConnConfig(), nil)
if err != nil {
return nil, err
}

return &BTCClient{
return &BitcoinClient{
client: c,
logger: logger,
cfg: cfg,
}, nil
}

//////////////////////////////
// METHODS
//////////////////////////////

type BlockCountResponse struct {
count int64
}

func (c *BTCClient) GetBlockCount() (uint64, error) {
func (c *BitcoinClient) GetBlockCount() (uint64, error) {
callForBlockCount := func() (*BlockCountResponse, error) {
count, err := c.client.GetBlockCount()
if err != nil {
Expand All @@ -51,7 +59,7 @@ func (c *BTCClient) GetBlockCount() (uint64, error) {
return uint64(blockCount.count), nil
}

func (c *BTCClient) GetBlockHashByHeight(height uint64) (*chainhash.Hash, error) {
func (c *BitcoinClient) GetBlockHashByHeight(height uint64) (*chainhash.Hash, error) {
callForBlockHash := func() (*chainhash.Hash, error) {
return c.client.GetBlockHash(int64(height))
}
Expand All @@ -64,7 +72,7 @@ func (c *BTCClient) GetBlockHashByHeight(height uint64) (*chainhash.Hash, error)
return blockHash, nil
}

func (c *BTCClient) GetBlockHeaderByHash(blockHash *chainhash.Hash) (*wire.BlockHeader, error) {
func (c *BitcoinClient) GetBlockHeaderByHash(blockHash *chainhash.Hash) (*wire.BlockHeader, error) {
callForBlockHeader := func() (*wire.BlockHeader, error) {
return c.client.GetBlockHeader(blockHash)
}
Expand All @@ -77,7 +85,7 @@ func (c *BTCClient) GetBlockHeaderByHash(blockHash *chainhash.Hash) (*wire.Block
return header, nil
}

func (c *BTCClient) GetBlockHeightByTimestamp(targetTimestamp uint64) (uint64, error) {
func (c *BitcoinClient) GetBlockHeightByTimestamp(targetTimestamp uint64) (uint64, error) {
// get the height of the most-work fully-validated chain
blockHeight, err := c.GetBlockCount()
if err != nil {
Expand Down Expand Up @@ -113,7 +121,7 @@ func (c *BTCClient) GetBlockHeightByTimestamp(targetTimestamp uint64) (uint64, e
return lowerBound - 1, nil
}

func (c *BTCClient) GetBlockTimestampByHeight(height uint64) (uint64, error) {
func (c *BitcoinClient) GetBlockTimestampByHeight(height uint64) (uint64, error) {
// get block hash by height
blockHash, err := c.GetBlockHashByHeight(height)
if err != nil {
Expand All @@ -129,6 +137,10 @@ func (c *BTCClient) GetBlockTimestampByHeight(height uint64) (uint64, error) {
return uint64(blockHeader.Timestamp.Unix()), nil
}

//////////////////////////////
// INTERNAL
//////////////////////////////

func clientCallWithRetry[T any](
call retry.RetryableFuncWithData[*T], logger *zap.Logger, cfg *BTCConfig,
) (*T, error) {
Expand Down
6 changes: 3 additions & 3 deletions sdk/btcclient/client_test.go → btcclient/btcclient_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ package btcclient
import (
"testing"

"github.com/babylonlabs-io/finality-gadget/log"
"github.com/stretchr/testify/require"
"go.uber.org/zap"
)

// TODO: 1) not rely on mainnet RPC; 2) add more tests for some other edge cases
Expand All @@ -13,12 +13,12 @@ func TestBtcClient(t *testing.T) {
var err error

// Create logger.
logger, err := zap.NewDevelopment()
logger, err := log.NewRootLogger("console", true)
require.Nil(t, err)

// Create BTC client
btcConfig := DefaultBTCConfig()
btc, err := NewBTCClient(btcConfig, logger)
btc, err := NewBitcoinClient(btcConfig, logger)
require.Nil(t, err)

// timestmap between block 848682 and 848683
Expand Down
3 changes: 1 addition & 2 deletions sdk/btcclient/config.go → btcclient/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,6 @@ func DefaultBTCConfig() *BTCConfig {
}
}


func (cfg *BTCConfig) ToConnConfig() *rpcclient.ConnConfig {
return &rpcclient.ConnConfig{
Host: cfg.RPCHost,
Expand All @@ -60,4 +59,4 @@ func (cfg *BTCConfig) ToConnConfig() *rpcclient.ConnConfig {
// we may need to re-consider it later if we need any notifications
HTTPPostMode: true,
}
}
}
Loading