Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bridge testnet #19871

Draft
wants to merge 19 commits into
base: testnet
Choose a base branch
from
Draft

Bridge testnet #19871

wants to merge 19 commits into from

Conversation

williampsmith
Copy link
Contributor

Description

Bridge testnet branch

Test plan

How did you test the new or updated feature?


Release notes

Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required.

For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates.

  • Protocol:
  • Nodes (Validators and Full nodes):
  • Indexer:
  • JSON-RPC:
  • GraphQL:
  • CLI:
  • Rust SDK:
  • REST API:

mwtian and others added 13 commits October 4, 2024 16:14
## Description 

Using a flat high timeouts in commit sync can stall fetching for too
long and hurt throughput, when slow hosts block fetching blocks for the
max timeout. Using timeouts that are too short can be problematic too.
Instead, start the request timeout at 10s and gradually increase it with
retries.

## Test plan 

CI
PT

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
## Description 
Create GraphQl staging build for `ci` and `devnet` environments.
See
https://linear.app/mysten-labs/issue/DVX-329/devnet-and-ci-graphql-built-with-staging-feature-enabled
for details

## Test plan 
Will be testing this after it lands
## Description

Temporarily re-enable `EventFilter::Any` as it is still in use in some
places. This is a partial revert of #19617.

## Test plan

CI

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [x] Nodes (Validators and Full nodes): Temporarily re-enable
`EventFilter::Any` as a kind of event subscription filter. Note that
subscriptions are deprecated. This means they are not officially
supported, nor actively maintained.
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
## Description

Ungate access to the parts of the GraphQL schema that depend on
`tx_affected_addresses` as this has now been fully rolled out in the
indexer

## Test plan

```
sui$ cargo nextest run -p sui-graphql-rpc
sui$ cargo nextest run -p sui-graphql-e2e-tests
```

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [x] GraphQL: Adds support for
`TransactionBlockFilter.affectedAddress`, and
`AddressTransactionBlockRelationship.AFFECTED` to find transactions
associated with any address they touch (sender, recipient, or payer).
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
## Description 

- fixed a bug that caused mainnet indexer to stop, also got reported in
#19542, specifically here when
tx has many input objects / affected objects / recipients / affected
addresses etc. the expanded query will exceed the PG parameter limit of
65535
- also added ingestion tests for big tx indices & event indices and
better error tracing


## Test plan 

added ingestion tests for tx and event indices

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:

## Description 

Describe the changes or additions included in this PR.

## Test plan 

How did you test the new or updated feature?

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
## Description 

Cherry-picked commits for adding Arden and Trace to the upcoming release

## Test plan 

How did you test the new or updated feature?

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:

---------

Co-authored-by: Vassilis Legakis <[email protected]>
## Description 

Describe the changes or additions included in this PR.

## Test plan 

How did you test the new or updated feature?

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
#19848)

## Description

Spotted that MovePackage queries were still using `objects_history`,
which is not going to work after that table gets pruned to two epochs.

This PR replaces queries to `objects_history` with queries to
`full_objects_history`, with support from `objects_version` and
`packages` to fetch the checkpoint sequence number where necessary.

## Test plan

Existing tests:

```
sui$ cargo nextest run -p sui-graphql-e2e-tests
```

And ran the following queries on the mainnet DB:

```graphql
query AllPackages($after: String) {
  packages(first: 5 after: $after) {
    pageInfo {
      hasNextPage
      endCursor
      startCursor
    }
    nodes {
      address
      version
    }
  }
}

query AllPackagesBack($before: String) {
  packages(last: 5 before: $before) {
    pageInfo {
      hasNextPage
      endCursor
      startCursor
    }
    nodes {
      address
      version
    }
  }
}

query SystemPackages($after: String) {
  packageVersions(address: "0x2" first: 5 after: $after) {
    pageInfo {
      hasNextPage
      endCursor
      startCursor
    }

    nodes {
      address
      version
    }
  }
}

query SystemPackagesBack($before: String) {
  packageVersions(address: "0x2" last: 5 before: $before) {
    pageInfo {
      hasNextPage
      endCursor
    }

    nodes {
      address
      version
    }
  }
}

query UserPackage($after: String) {
  packageVersions(
    address: "0xbc3df36be17f27ac98e3c839b2589db8475fa07b20657b08e8891e3aaf5ee5f9"
		first: 5
    after: $after
  ) {
    pageInfo {
      hasNextPage
      endCursor
      startCursor
    }
    nodes {
      address
      version
    }
  }
}
```

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
## Description 

The original limits on the object queue are set with 1s checkpoint
interval. Having the oldest transaction on an object queued for 1s is
almost tolerable. In hindsight it should be lower than the checkpoint
interval to ensure healthy system. Now checkpoint interval is targeting
0.2s - 0.25s, so lowering the limits further to 0.2s, to ensure smooth
checkpoint constructions.

## Test plan 

CI
PT
Reading `num_rejected_cert_during_overload` metric from validators.

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
…19856)

## Description 

Cherry pick #19853 

## Test plan 

How did you test the new or updated feature?

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
## Description 

as title. This state is not used today.

## Test plan 

How did you test the new or updated feature?

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
## Description 

Adds path to provide min timeout for sig aggregation with preference
such that, when specified, we will wait at least `min_timeout` to
collect as many sigs as possible before ordering based on the provided
preference.

Note that the const min timeout value provided here is for verifying
functionality for now, and should ideally be replaced with something
like P95 latency.

## Test plan 

Added unit test. Also ran on testnet node and observed that the client
running this generated a certificate with 11 signatures, while other
client generated a certificate with 28 signatures.

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
Copy link

vercel bot commented Oct 15, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
sui-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Nov 7, 2024 0:25am
3 Skipped Deployments
Name Status Preview Comments Updated (UTC)
multisig-toolkit ⬜️ Ignored (Inspect) Visit Preview Nov 7, 2024 0:25am
sui-kiosk ⬜️ Ignored (Inspect) Visit Preview Nov 7, 2024 0:25am
sui-typescript-docs ⬜️ Ignored (Inspect) Visit Preview Nov 7, 2024 0:25am

longbowlu and others added 5 commits October 18, 2024 14:53
## Description 

so we can use this in the dashboard

## Test plan 

How did you test the new or updated feature?

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
## Description 

as title. It also removes the transitive dependency of other crates on
sui-bridge.

## Test plan 

tests

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
## Description 

so we can monitor this in a dashboard.

## Test plan 

unit tests & test in production
---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
## Description 

1. run watchdogs on bridge nodes.
2. merges `sui-bridge-watchdog` to `sui-bridge` crate, so there is no
circular dependencies

## Test plan 

unit tests, will deploy locally and test

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
## Description 

In some cases, the client observes a finalized transaction before most
bridge authorities. In such cases, today the code will return an error
without retries, causing this validator to be skipped in terms of
signature aggregation. If bridge validators are using slow ethereum
fullnode providers, for example, this can be a large amount of the
committee, resulting in failed signature aggregation attempts in some
cases (due to not achieving quorum), or high gas costs in others (due to
not getting fewer higher staked validator signatures).

The solution here is to add retry logic in the map function.

## Test plan 

Will run on a testnet node and see what happens.

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
## Description 

Describe the changes or additions included in this PR.

## Test plan 

How did you test the new or updated feature?

---

## Release notes

Check each box that your changes affect. If none of the boxes relate to
your changes, release notes aren't required.

For each box you select, include information after the relevant heading
that describes the impact of your changes that a user might notice and
any actions they must take to implement updates.

- [ ] Protocol: 
- [ ] Nodes (Validators and Full nodes): 
- [ ] Indexer: 
- [ ] JSON-RPC: 
- [ ] GraphQL: 
- [ ] CLI: 
- [ ] Rust SDK:
- [ ] REST API:
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants