feat: Implement BundleDataClient and Dataworker #1155

nicholaspai · 2024-01-23T15:00:01Z

Summary

Implements logic to propose and execute root bundles containing V3 and V3 events. This PR should be backwards compatible and can be deployed today.
the way I recommend to tackle reviewing this large PR is to focus on the BundleDataClient logic. This is the most complex bit and has the most branches.

DataworkerUtils

We should now invalidate bundle block ranges before we attempt to propose or dispute them if the SpokePool client does not have enough of a lookback to fetch all expired deposits in the latest bundle. See DataworkerUtils.blockRangesAreInvalidForSpokeClients
_buildRelayerRefundRoot, _buildPoolRebalanceRoot, _buildSlowFillRoot are all implemented to handle V3 BundleData objects in addition to preserving existing logic. You'll see that the logic is built to be naturally backwards compatible

BundleDataClient

Returns new v3 types: bundleFillsV3, bundleDepositsV3, bundleSlowFills, bundleExpiredDeposits, unexecutableV3SlowFills

Unit tests

Unit tests in Dataworker.loadData and Dataworker.buildRoots are illustrative of new logic in BundleDataClient and DataworkerUtils.

~~Note this PR will only work with across-protocol/sdk#535 as the SDK version~~
Updated now to v0.21.1

- We should now invalidate bundle block ranges before we attempt to propose or dispute them if the SpokePool client does not have enough of a lookback to fetch all expired deposits in the latest bundle. See `DataworkerUtils.blockRangesAreInvalidForSpokeClients` - Add algorithm in-comments in BundleDataClient for loading "newly expired" deposits for a bundle - This PR should be reviewed in tandem with SDK #TODO which has the required SDK changes to make this work

…lient These will be useful for the dataworker functions (beginning to be implemented in across-protocol/relayer#1155 ) that have to deal with expired deposits. We need to be able to know when a deposit is expired (comparing its fill deadline TIMESTAMP with a bundle BLOCK NUMBER), and also whether our spoke pool clients have enough of a lookback to fetch expired deposits for a bundle

src/clients/BundleDataClient.ts

src/dataworker/Dataworker.ts

src/dataworker/DataworkerUtils.ts

Co-authored-by: Paul <[email protected]>

These will be useful for the dataworker functions (beginning to be implemented in across-protocol/relayer#1155 ) that have to deal with expired deposits. We need to be able to know when a deposit is expired (comparing its fill deadline TIMESTAMP with a bundle BLOCK NUMBER), and also whether our spoke pool clients have enough of a lookback to fetch expired deposits for a bundle Signed-off-by: nicholaspai <[email protected]>

src/clients/BundleDataClient.ts

This reverts commit 3553aee.

src/dataworker/PoolRebalanceUtils.ts

src/clients/BundleDataClient.ts

src/dataworker/Dataworker.ts

src/dataworker/DataworkerUtils.ts

src/dataworker/PoolRebalanceUtils.ts

src/interfaces/index.ts

pxrl

Huge effort 👏

pxrl

A few issues in type narrowing in test.

test/Dataworker.buildRoots.ts

nicholaspai added 2 commits January 23, 2024 09:59

Update Dataworker.blockRangeUtils.ts

837ce80

nicholaspai mentioned this pull request Jan 23, 2024

feat: Add oldest time searched to SpokePoolClient across-protocol/sdk#506

Merged

nicholaspai and others added 5 commits January 23, 2024 11:45

Update Dataworker.blockRangeUtils.ts

a5245a8

Update BundleDataClient.ts

c32c07f

Merge branch 'master' into npai/v3-dataworker

caff4bb

Update Dataworker.ts

38e2876

Update BundleDataClient.ts

f49ed37

pxrl reviewed Jan 24, 2024

View reviewed changes

src/clients/BundleDataClient.ts Outdated Show resolved Hide resolved

src/clients/BundleDataClient.ts Show resolved Hide resolved

src/dataworker/Dataworker.ts Outdated Show resolved Hide resolved

src/dataworker/DataworkerUtils.ts Outdated Show resolved Hide resolved

nicholaspai and others added 3 commits January 24, 2024 10:44

Update src/clients/BundleDataClient.ts

3bcc0fb

Co-authored-by: Paul <[email protected]>

Update src/dataworker/Dataworker.ts

6442ff2

Co-authored-by: Paul <[email protected]>

fix

7f537ad

nicholaspai requested a review from pxrl January 24, 2024 15:46

nicholaspai added 4 commits January 24, 2024 15:34

Update BundleDataClient.ts

eda32bf

Update BundleDataClient.ts

bbcbdaf

fillDeadlineBuffers

9464da9

More fixes

017f51a

nicholaspai added 10 commits January 29, 2024 12:33

Update BundleDataClient.ts

9edbc79

Update BundleDataClient.ts

ff2871c

Merge branch 'master' into npai/v3-dataworker

e8cb57f

Update BundleDataClient.ts

8225f5c

Update BundleDataClient.ts

728875e

merge

fbe09bf

Update BundleDataClient.ts

ef6d646

Update BundleDataClient.ts

9fdb26e

Merge branch 'master' into npai/v3-dataworker

022d8ee

Update BundleDataClient.ts

e75f21f

nicholaspai changed the title ~~feat: Add helper methods for dealing with V3 expired-deposit logic~~ feat: Add skeleton for V3 BundleDataClient and Dataworker logic Feb 1, 2024

nicholaspai commented Feb 18, 2024

View reviewed changes

src/clients/BundleDataClient.ts Outdated Show resolved Hide resolved

nicholaspai and others added 16 commits February 18, 2024 18:32

Sanity check that SlowFIll fills are token equivalent events

3553aee

Update DataworkerUtils.ts

e0ba245

Revert "Sanity check that SlowFIll fills are token equivalent events"

0115b3a

This reverts commit 3553aee.

Update BundleDataClient.ts

c0f66a9

Remove optimization for SlowFill execution short-circuit

101111f

Update BundleDataClient.ts

3eee0d2

refactors

4119f21

Make types stricter

4093ceb

Lint

d23bbfa

Update BundleDataClient.ts

10e2f05

Merge branch 'master' into npai/v3-dataworker

edd69e7

fix types

2c289aa

Merge branch 'master' into npai/v3-dataworker

d161340

Add comments, refactor, mark v3RelayHash properties as optional

ec3a644

lint, fix types

65e7439

Merge branch 'master' into npai/v3-dataworker

faaa189

pxrl reviewed Feb 20, 2024

View reviewed changes

pxrl and others added 3 commits February 20, 2024 07:11

improve(BundleDataClient): Small optimisation (#1203)

bbdba7e

Move endblock timestamp calculation outside of array.some

2f86009

Paul nits

dc0a838

pxrl approved these changes Feb 20, 2024

View reviewed changes

nicholaspai added 6 commits February 20, 2024 15:19

WIP

634d818

lint

e7ae62d

Update BundleDataClient.ts

040a10c

Update ProfitClient.ts

cd276d5

Update to 0.22.0

288efb1

Update yarn.lock

e954e94

nicholaspai merged commit 3d61baa into master Feb 20, 2024
4 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Implement BundleDataClient and Dataworker #1155

feat: Implement BundleDataClient and Dataworker #1155

nicholaspai commented Jan 23, 2024 •

edited

Loading

pxrl left a comment

pxrl left a comment

feat: Implement BundleDataClient and Dataworker #1155

feat: Implement BundleDataClient and Dataworker #1155

Conversation

nicholaspai commented Jan 23, 2024 • edited Loading

Summary

DataworkerUtils

BundleDataClient

Unit tests

pxrl left a comment

Choose a reason for hiding this comment

pxrl left a comment

Choose a reason for hiding this comment

nicholaspai commented Jan 23, 2024 •

edited

Loading