Implement Express Lane Timeboost

[rauljordan]

Background

At the time of writing, the Arbitrum sequencer is centralized and offers a first-come, first-serve transaction ordering policy. Txs have a current delay of approximately 250ms, which is the time the sequencer takes to produce an ordered list of txs to emit in the form of an L2 block. The current policy does not handle MEV that occurs naturally on L2, and leads to latency races offline to get faster access to the sequencer ingress server.

A new policy has been proposed, known as Express Lane Timeboost, which allows participants to bid for the rights of priority sequencing using their funds instead of hardware. In “rounds” that start at each minute mark, participants can submits bids to participate in a sealed, second-price auction for control of the next round’s “express lane”. During a round, all non-express lane txs get their first arrival timestamp delayed by some amount of time (250ms), while the express lane controller does not. The express lane controller can also choose to transfer their rights in a round.

The sequencer itself does not need to manage auctions, but simply needs to know the current round number and the address of the express lane controller for that round. From there, it can delay non-express lane txs by a nominal amount required by the protocol and validate that a tx should go through the express lane.

This PR contains the complete implementation of the system with all its components. The smart contract changes are contained within OffchainLabs/nitro-contracts/tree/express-lane-auction-all-merged.

Basic Readings

To read more about timeboost, see the AIP, the research specification, and design doc although the design doc is not fully updated yet.

Reviewing

Recommend to look at the basic readings, then look at system_tests/timeboost_test.go to understand how it all fits together. Then, look at bid validator and auctioneer. Finally, the sequencer changes.

Features

• Bidder client that allows participants to join the auction and submit bids to a bid validator
• Bid validator that receives bids over the internet, validates them, and inserts validated items into Redis stream
• Auctioneer server that consumes validated bids from Redis stream.
• Auctioneer at the 45 second mark, submits the top two bids to a privileged sequencer endpoint
• Ability to persist validated bids to a local DB (sqlite) in the auctioneer server
• System tests are added that assert express lane txs have an advantage in the emitted sequencer feed

Sequencer Changes

The changes to the sequencer hot path are quite simple. In a nutshell, if a transaction is received, it checks the following:
If timeboost is enabled AND there is an express lane controller set AND it is not coming from the express lane, it delays the tx's first arrival timestamp by some amount (250ms).

To determine if a transaction is a valid express lane tx, the sequencer runs a background thread called the expressLaneService, which is scraping events from the ExpressLaneAuction.sol smart contract. Express lane transactions arrive via a different sequencer endpoint than the normal one, called timeboost_sendExpressLaneTransaction. The message looks as follows:

{
  "type": "object",
  "properties": {
    "chainId": {
      "type": "bigInt",
      "description": "chain id of the target chain"
    },
    "round": {
      "type": "uint64",
      "description": "round number (0-indexed) for the round the bidder wants to become the controller of"
    },
    "auctionContractAddress": {
      "type": "address",
      "description": "hex string of the auction contract address that the bid corresponds to"
    },
    "sequenceNumber": {
      "type": "uint64",
      "description": "the per-round nonce of express lane submissions. Each submission to the express lane during a round increases this sequence number by one, and if submissions are received out of order, the sequencer will queue them for processing in order. This is reset to 0 at each round"
    },
    "transaction": {
      "type": "bytes",
      "description": "hex string of the RLP encoded transaction payload that submitter wishes to be sequenced through the express lane"
    },
    "options": {
      "type": "ArbitrumConditionalOptions",
      "description": "conditional options for Arbitrum transactions, supported by normal sequencer endpoint https://github.com/OffchainLabs/go-ethereum/blob/48de2030c7a6fa8689bc0a0212ebca2a0c73e3ad/arbitrum_types/txoptions.go#L71"
    },
    "signature": {
      "type": "bytes",
      "description": "Ethereum signature over the bytes encoding of (keccak256(TIMEBOOST_BID), padTo32Bytes(chainId), auctionContractAddress, uint64ToBytes(round), uint64ToBytes(sequenceNumber), transaction)"
    }
  },
}

The submission itself contains a tx payload, which MAY not be from the express lane controller. As long as the submission is signed by the controller, that is sufficient. Submissions have a specific nonce, called a sequence, to ensure that submissions are processed in order. This is different from the inner nonce of the payload tx. The sequencer keeps a queue of submissions and ensures it processes them in order. That is, if a submission N is received before N-1, it will get queued for submission once N arrives.

Bid Validator Architecture

Bids are limited to 5 bids per sender, but there are no limits to the number of bidders in a single round. To alleviate potential scaling concerns, we adopt a simple architecture of separating the bid validators from the auctioneer. The bid validators filter out invalid items and publish validated results to a Redis stream. In a simplified diagram, here's what it will look like:

Dependencies Added

• github.com/golang-jwt/jwt/v4 for the authenticated endpoint from the auctioneer to the sequencer
• github.com/stretchr/testify for testing utilities (will probably have to remove)
• github.com/mattn/go-sqlite3 for the bids DB
• github.com/jmoiron/sqlx for the bids DB
• github.com/DATA-DOG/go-sqlmock for testing the bids DB

Notes

There are several parts of this implementation that are likely not ideal:

Chicken and the egg problem in sequencer
Cannot start sequencer without express lane, but cannot deploy auction for express lane without starting sequencer. To solve this in tests, we have a separate func called StartExpressLaneService in the sequencer. In prod, we don’t have this issue because we can deploy the contracts before we upgrade the sequencer to timeboost, but what to do about tests?

Janky prioritizing of auction resolution txs
The sequencer exposes an authenticated endpoint auctioneer_submitAuctionResolutionTransaction over the JWT Auth RPC for the auctioneer to use. When the auctioneer is ready to resolve an auction, it submits a tx to this endpoint, which the sequencer verifies for integrity. Then, the sequencer does the following:

log.Info("Prioritizing auction resolution transaction from auctioneer", "txHash", tx.Hash().Hex())
s.timeboostAuctionResolutionTx = tx
s.createBlock(ctx)

it immediately tries to put the item in the queue and create block. It also sets the tx as a property of the sequencer struct, and in the createBlock func, if this field is not nil, it gets put at the top of the queue. This is a bit janky in how it works and perhaps inefficient. Is there another way to prioritize a tx in the sequencer?

Sequencer opens an http connection to itself
The sequencer has a thread called expressLaneService which reads events from the auction smart contracts on L2 to determine express lane controllers. Because the sequencer does not have filtersystem API access, we instead open an RPC client against itself so we can create an ethclient to read logs and data from onchain. This doesn't seem ideal

References

• Constitutional AIP Introducing Timeboost
• Research Specification
• Initial Implementation Design Document
• Bid Validator <> Auctioneer Architecture

[tsahee]

small thing I noticed

[tsahee]

this seems like it belongs inside the sequencer's Start function?

[tsahee]

this is great!
It deserves to be in a separate file, and should have a few lines documenting why it exists and which contract funs are supported.

[tsahee]

don't panic. return an error (same for CodeAt)

[tsahee]

buffer seems excessive. If we've filled thousands of logs something is wrong and the queue's capacity will only make it worse.

[tsahee]

Why do we filter for auction resolved? seems like all we really need is SetExpressLaneController. If there's anything to be understood from AuctionResolved it can be done by looking at events outside nitro

[tsahee]

I've made comments about it elsewhere:

• laways passing around initialTimestamp and roundDuration is really cumbersome
• I think you must take a timestamp when starting to process an incoming message and treat everything from that point as if that's the timestamp

[tsahee]

Are we doing anything with messagesBySequenceNumber? any reason to keep messages in the map if err == nil?

[tsahee]

first scan should probably start before current block because an older block might have info for the next round.
A static number of blocks that relates to 1-2 rounds is probably good enough.

[tsahee]

instead of putting logic under the timer we usually do

 for {
    ..logic..
    select {
      case <-ctx.Done()
      case <-time.After()
    }
 }

select could come either before or after loic

[tsahee]

this can be checked earlier, before we extract signature

[CLAassistant]

All committers have signed the CLA.