Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Express Lane Timeboost #2561

Open
wants to merge 165 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 76 commits
Commits
Show all changes
165 commits
Select commit Hold shift + click to select a range
9925308
added in timeboost items
rauljordan Jul 2, 2024
4c2c74f
add in
rauljordan Jul 2, 2024
d0ed9bf
system test
rauljordan Jul 2, 2024
4ff7410
Rename auction master to autonomous auctioneer
terencechain Jul 2, 2024
83e8e00
Sequence express lane transactions follow spec
terencechain Jul 2, 2024
4b873fa
Merge branch 'rename-auctioneer' into express-lane-timeboost
terencechain Jul 3, 2024
0a7ba8c
include master
rauljordan Jul 23, 2024
81a237b
update contracts repo and bindings
rauljordan Jul 23, 2024
0b7819b
building once more
rauljordan Jul 23, 2024
b829d9b
builds
rauljordan Jul 23, 2024
94c5653
autonomous auctioneer bin
rauljordan Jul 23, 2024
676b89e
receive bid test passing
rauljordan Jul 24, 2024
d7a91a8
edits and fix sig
rauljordan Jul 24, 2024
c5fad5c
passing test
rauljordan Jul 24, 2024
f54ac50
add to seq test
rauljordan Jul 24, 2024
4b7a910
system test
rauljordan Jul 24, 2024
9681909
Updated auctioneer with research spec
terencechain Jul 24, 2024
fc81996
Merge pull request #2521 from OffchainLabs/auctioneer
terencechain Jul 24, 2024
2fd3327
begin adding sequencer endpoint
rauljordan Jul 24, 2024
dcb5525
Clean up auctioneer part 1
terencechain Jul 25, 2024
32e5ffd
Use init for domain value
terencechain Jul 25, 2024
6fd97f4
Tie break bids and unit tests
terencechain Jul 25, 2024
d54f89b
Test resolve bids
terencechain Jul 27, 2024
c3d8c01
begin adding in endpoints
rauljordan Jul 29, 2024
08cc4f5
Merge branch 'express-lane-timeboost' of github.com:OffchainLabs/nitr…
rauljordan Jul 29, 2024
ea852cd
Merge branch 'express-lane-timeboost' into timeboost-endpoints
rauljordan Jul 29, 2024
02bdf45
validate express lane tx submission in sequencer
rauljordan Jul 29, 2024
d7eb164
express lane client send transaction
rauljordan Jul 29, 2024
4ad6fcb
adding in and building
rauljordan Jul 29, 2024
ca24d8b
auctioneer binary config
rauljordan Jul 29, 2024
083caf0
add back domain
rauljordan Jul 29, 2024
521e7a6
Merge branch 'express-lane-timeboost' into timeboost-endpoints
rauljordan Jul 29, 2024
0c6e15b
do not use 112
rauljordan Jul 29, 2024
98f8dcb
fix test
rauljordan Jul 29, 2024
93ba2cf
wire up the forwarder
rauljordan Jul 30, 2024
8f037de
passing
rauljordan Jul 30, 2024
396c6e9
auction is now happening on L2
rauljordan Jul 30, 2024
65e0dfc
big int chain id
rauljordan Jul 30, 2024
d317fa4
contracts on L2 passing test
rauljordan Jul 31, 2024
e7a0788
pass resolve auction
rauljordan Aug 1, 2024
889d567
tests and benchmark
rauljordan Aug 1, 2024
a33cb28
add sequence number management for express lane submissions
rauljordan Aug 2, 2024
52bbc58
add nonce precedence test
rauljordan Aug 2, 2024
471d79e
add test for too many bids
rauljordan Aug 2, 2024
3b8e6a8
validate bids passing
rauljordan Aug 2, 2024
79dc1ef
resolve auction
rauljordan Aug 2, 2024
91a9188
Add sqlite db for bids
terencechain Aug 2, 2024
5ee3048
Merge branch 'bids-db' into express-lane-timeboost
terencechain Aug 6, 2024
60bbb9f
using redis streams
rauljordan Aug 6, 2024
82860e6
auctioneer server binary
rauljordan Aug 6, 2024
bb52e7e
Merge branch 'express-lane-timeboost' of github.com:OffchainLabs/nitr…
rauljordan Aug 6, 2024
ca1b913
add configs for binary
rauljordan Aug 6, 2024
8a81d3e
use single config
rauljordan Aug 6, 2024
4e6b1e6
test passing for redis stream
rauljordan Aug 7, 2024
49ca6fa
specify the privileged sequencer endpoint for auctioneer
rauljordan Aug 7, 2024
840be36
privileged endpoint
rauljordan Aug 7, 2024
84ab0d1
move system test
rauljordan Aug 7, 2024
8fccf2c
use stopwaiter
rauljordan Aug 7, 2024
7bb8e3b
edit test
rauljordan Aug 7, 2024
1a41c25
fix up db and store bids correctly
rauljordan Aug 7, 2024
f997721
all tests passing once more
rauljordan Aug 7, 2024
6a0afaf
rem ctx
rauljordan Aug 7, 2024
9c4af2a
add jwt auth for auctioneer to sequencer
rauljordan Aug 7, 2024
40f4f24
authenticated, use call iteratively
rauljordan Aug 7, 2024
60fefb5
update contracts
rauljordan Aug 8, 2024
27b9a0c
test pass
rauljordan Aug 8, 2024
bd47e80
sequencer checks within auction closing
rauljordan Aug 8, 2024
c5fc048
sequencer endpoint in cfg
rauljordan Aug 8, 2024
91373de
add method to bid
rauljordan Aug 8, 2024
a9b2bdd
productionize bidder client
rauljordan Aug 8, 2024
4ed60df
tests passing
rauljordan Aug 8, 2024
b8b4483
productionize bidder client
rauljordan Aug 8, 2024
119e1bb
system test pass
rauljordan Aug 8, 2024
27e63db
contracts
rauljordan Aug 8, 2024
203a8ad
edit
rauljordan Aug 8, 2024
e7b0fa3
move express lane client to system test
rauljordan Aug 9, 2024
7396c62
Auctioneer metrics: bids and value (part 1)
terencechain Aug 12, 2024
02e3698
resolve confs
rauljordan Aug 19, 2024
c315dd8
tidy
rauljordan Aug 19, 2024
e6c6def
Merge branch 'express-lane-timeboost' of github.com:OffchainLabs/nitr…
rauljordan Aug 19, 2024
d40cee9
Update timeboost/ticker.go
rauljordan Aug 19, 2024
7b63dc9
condition
rauljordan Aug 19, 2024
1aaf4da
limit bids
rauljordan Aug 19, 2024
6705b06
last second leeway for bid processing
rauljordan Aug 19, 2024
ab55292
resolve wait a second
rauljordan Aug 19, 2024
7a70988
dont hardcode ports
rauljordan Aug 19, 2024
47697ed
Merge branch 'master' into express-lane-timeboost
rauljordan Aug 19, 2024
26f28d4
Merge branch 'master' into express-lane-timeboost
rauljordan Aug 19, 2024
d9307d8
Merge branch 'express-lane-timeboost' of github.com:OffchainLabs/nitr…
rauljordan Aug 19, 2024
e4892b2
geth pin
rauljordan Aug 19, 2024
4dc427a
add in new queue
rauljordan Aug 19, 2024
e2110ab
edits
rauljordan Aug 19, 2024
15e6087
Fix tie breaker and retry resolve bid tx
terencechain Aug 19, 2024
f75f242
edits
rauljordan Aug 20, 2024
ffd0766
commit
rauljordan Aug 20, 2024
a5c8777
rem redundant sig verify
rauljordan Aug 20, 2024
9d53ff7
Merge branch 'master' into express-lane-timeboost
rauljordan Aug 20, 2024
196bdc7
Fix tie breaker and better retry
terencechain Aug 20, 2024
e61c3b2
edit
rauljordan Aug 20, 2024
44c8179
Merge branch 'master' into express-lane-timeboost
rauljordan Aug 20, 2024
981bc6a
Fix duplicated word
terencechain Aug 20, 2024
8db0bd0
Add copyright to express lane service
terencechain Aug 21, 2024
ec185f5
Better popped the auction resolution tx log
terencechain Aug 21, 2024
d15e0c5
Fix auction-contract-address doesnt match the field SequencerEndpoint
terencechain Aug 21, 2024
f28896d
Remove unused conversions
terencechain Aug 21, 2024
3eec686
Fix express lane advantage to 200ms
terencechain Aug 23, 2024
98326dc
Update reserve price
terencechain Aug 21, 2024
3632c66
Filter transfer log
terencechain Aug 23, 2024
8122a49
Tristan's feedback
terencechain Aug 29, 2024
5c03cf0
Tristan's feedback
terencechain Sep 5, 2024
fa457b1
Fix autonomous-auctioner cli startup
Tristan-Wilson Oct 1, 2024
0ff4f7d
Start rpc stack after creating bid validator
Tristan-Wilson Oct 11, 2024
7a2eb14
Plumbing to be able to start timeboost in nitro
Tristan-Wilson Oct 11, 2024
d72fd38
Fix various linter and compilation issues
Tristan-Wilson Oct 14, 2024
2cd1ed0
Fix cyclic dependency in test
Tristan-Wilson Oct 14, 2024
f4efcd9
Move where timeboost is started to avoid circ dep
Tristan-Wilson Oct 15, 2024
651277d
Temp fix for timeboost startup race cond, fix NPE
Tristan-Wilson Oct 15, 2024
31ea7be
Retry initial call on express lane contract
Tristan-Wilson Oct 16, 2024
d1890da
Add bidder-client exe for timeboost deposits/bids
Tristan-Wilson Oct 18, 2024
103b395
Clear bidsPerSenderInRound when auction closes
Tristan-Wilson Oct 23, 2024
3431e2c
Use FilterSystem for ContractFilterer in ELS
Tristan-Wilson Oct 24, 2024
cad5b12
Fix SequenceNumber attribute and decode order
TucksonDev Oct 25, 2024
b6ea4f5
Fix panic when transferring express lane controller due to race in lr…
amsanghi Oct 28, 2024
d64ce27
Adapter for ContractFilterer
Tristan-Wilson Oct 28, 2024
9c9e1b9
Merge pull request #2757 from OffchainLabs/fix_race_lru
Tristan-Wilson Oct 29, 2024
1cb9731
Update execution/gethexec/express_lane_service.go
Tristan-Wilson Oct 29, 2024
fe0f243
Merge branch 'express-lane-timeboost' into express-lane-fix-sequence-…
Tristan-Wilson Oct 29, 2024
1d7a645
Merge pull request #2756 from TucksonDev/express-lane-fix-sequence-nu…
Tristan-Wilson Oct 29, 2024
0ae3c89
Merge remote-tracking branch 'origin/express-lane-timeboost' into exp…
Tristan-Wilson Oct 29, 2024
d299eea
Remove timeboost sequencer url config opt
Tristan-Wilson Oct 29, 2024
4567b0a
Fix typo causing panic
amsanghi Oct 30, 2024
d20c821
Merge pull request #2763 from OffchainLabs/fix_panic
Tristan-Wilson Oct 30, 2024
1ad12ca
Fix timeboost auction resolution queue handling
Tristan-Wilson Oct 30, 2024
181932e
Merge remote-tracking branch 'origin/master' into express-lane-timeboost
Tristan-Wilson Nov 1, 2024
aaf051f
Post merge fixes, mostly stricter linter from 1.23
Tristan-Wilson Nov 4, 2024
f653c95
Allow wider range of RoundTimingInfo + validation
Tristan-Wilson Nov 4, 2024
47ff5ee
Fix auction closed test
Tristan-Wilson Nov 5, 2024
f8e8aa9
Fix max bids OBOE
Tristan-Wilson Nov 5, 2024
71c8c84
Bind AuthPort to random number to avoid collision
Tristan-Wilson Nov 5, 2024
06dc0ab
Fix OBOE in validator in a better way, fix test
Tristan-Wilson Nov 5, 2024
8b7182d
Merge remote-tracking branch 'origin/master' into express-lane-timeboost
Tristan-Wilson Nov 7, 2024
31c950f
Merge branch 'express-lane-timeboost' into express-lane-timeboost-inp…
Tristan-Wilson Nov 7, 2024
7219c33
Set dep of arbitrator/langs/bf to match master
Tristan-Wilson Nov 8, 2024
75acee8
Merge branch 'express-lane-timeboost' into express-lane-timeboost-inp…
Tristan-Wilson Nov 8, 2024
fa93c2c
Forgot to commit RoundTimingInfo validation
Tristan-Wilson Nov 8, 2024
7f9e5b6
Reorder express lane tx validation
Tristan-Wilson Nov 8, 2024
b7d1b06
Rename delay param to isExpressLaneController
Tristan-Wilson Nov 12, 2024
436f831
Use ptr to sequencer instead of member fn for ES
Tristan-Wilson Nov 12, 2024
b991d76
Change log level for future EL sequence number
Tristan-Wilson Nov 12, 2024
f5ca4bf
Use interface instead of ptr to seq for tests
Tristan-Wilson Nov 13, 2024
23869a2
Fix reversed boolean condition
Tristan-Wilson Nov 15, 2024
7dfe40d
Merge remote-tracking branch 'origin/express-lane-timeboost' into exp…
Tristan-Wilson Nov 20, 2024
b13c89a
Merge pull request #2762 from OffchainLabs/express-lane-timeboost-rem…
Tristan-Wilson Nov 20, 2024
a812410
Merge branch 'express-lane-timeboost' into express-lane-timeboost-fix…
Tristan-Wilson Nov 20, 2024
6933b94
Merge pull request #2764 from OffchainLabs/express-lane-timeboost-fix…
Tristan-Wilson Nov 20, 2024
287f74a
Merge pull request #2775 from OffchainLabs/express-lane-timeboost-inp…
Tristan-Wilson Nov 20, 2024
066ff17
Merge remote-tracking branch 'origin/express-lane-timeboost' into exp…
Tristan-Wilson Nov 20, 2024
dc65eef
Merge remote-tracking branch 'origin/expres-lane-timeboost-fix-pr-com…
Tristan-Wilson Nov 20, 2024
969dc1d
Refactor stubPublisher
Tristan-Wilson Nov 20, 2024
5a02894
Change transactionPublisher interface
Tristan-Wilson Nov 20, 2024
cc75bf2
Remove uncessary locking
Tristan-Wilson Nov 21, 2024
9bf924f
Merge remote-tracking branch 'origin/master' into express-lane-timeboost
Tristan-Wilson Nov 21, 2024
7356aaf
Automatic import cleanup
Tristan-Wilson Nov 21, 2024
3b67813
Merge branch 'express-lane-timeboost' into expres-lane-timeboost-fix-…
Tristan-Wilson Nov 21, 2024
cee611d
Merge pull request #2787 from OffchainLabs/expres-lane-timeboost-fix-…
Tristan-Wilson Nov 21, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -294,6 +294,7 @@ FROM nitro-node-slim AS nitro-node
USER root
COPY --from=prover-export /bin/jit /usr/local/bin/
COPY --from=node-builder /workspace/target/bin/daserver /usr/local/bin/
COPY --from=node-builder /workspace/target/bin/autonomous-auctioneer /usr/local/bin/
COPY --from=node-builder /workspace/target/bin/datool /usr/local/bin/
COPY --from=nitro-legacy /home/user/target/machines /home/user/nitro-legacy/machines
RUN rm -rf /workspace/target/legacy-machines/latest
Expand Down
5 changes: 4 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,7 @@ push: lint test-go .make/fmt
all: build build-replay-env test-gen-proofs
@touch .make/all

build: $(patsubst %,$(output_root)/bin/%, nitro deploy relay daserver datool seq-coordinator-invalidate nitro-val seq-coordinator-manager)
build: $(patsubst %,$(output_root)/bin/%, nitro deploy relay daserver autonomous-auctioneer datool seq-coordinator-invalidate nitro-val seq-coordinator-manager)
@printf $(done)

build-node-deps: $(go_source) build-prover-header build-prover-lib build-jit .make/solgen .make/cbrotli-lib
Expand Down Expand Up @@ -256,6 +256,9 @@ $(output_root)/bin/relay: $(DEP_PREDICATE) build-node-deps
$(output_root)/bin/daserver: $(DEP_PREDICATE) build-node-deps
go build $(GOLANG_PARAMS) -o $@ "$(CURDIR)/cmd/daserver"

$(output_root)/bin/autonomous-auctioneer: $(DEP_PREDICATE) build-node-deps
go build $(GOLANG_PARAMS) -o $@ "$(CURDIR)/cmd/autonomous-auctioneer"

$(output_root)/bin/datool: $(DEP_PREDICATE) build-node-deps
go build $(GOLANG_PARAMS) -o $@ "$(CURDIR)/cmd/datool"

Expand Down
2 changes: 1 addition & 1 deletion arbitrator/langs/bf
Submodule bf updated 1 files
+282 −846 Cargo.lock
156 changes: 156 additions & 0 deletions cmd/autonomous-auctioneer/config.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
package main

import (
"fmt"

"reflect"
"time"

"github.com/ethereum/go-ethereum/node"
"github.com/ethereum/go-ethereum/p2p"
"github.com/ethereum/go-ethereum/rpc"
"github.com/offchainlabs/nitro/cmd/conf"
"github.com/offchainlabs/nitro/cmd/genericconf"
"github.com/offchainlabs/nitro/timeboost"
"github.com/offchainlabs/nitro/util/colors"
flag "github.com/spf13/pflag"
)

type AutonomousAuctioneerConfig struct {
AuctioneerServer timeboost.AuctioneerServerConfig `koanf:"auctioneer-server"`
BidValidator timeboost.BidValidatorConfig `koanf:"bid-validator"`
Persistent conf.PersistentConfig `koanf:"persistent"`
Conf genericconf.ConfConfig `koanf:"conf" reload:"hot"`
LogLevel string `koanf:"log-level" reload:"hot"`
LogType string `koanf:"log-type" reload:"hot"`
FileLogging genericconf.FileLoggingConfig `koanf:"file-logging" reload:"hot"`
HTTP genericconf.HTTPConfig `koanf:"http"`
WS genericconf.WSConfig `koanf:"ws"`
IPC genericconf.IPCConfig `koanf:"ipc"`
Metrics bool `koanf:"metrics"`
MetricsServer genericconf.MetricsServerConfig `koanf:"metrics-server"`
PProf bool `koanf:"pprof"`
PprofCfg genericconf.PProf `koanf:"pprof-cfg"`
}

var HTTPConfigDefault = genericconf.HTTPConfig{
Addr: "",
Port: genericconf.HTTPConfigDefault.Port,
API: []string{},
RPCPrefix: genericconf.HTTPConfigDefault.RPCPrefix,
CORSDomain: genericconf.HTTPConfigDefault.CORSDomain,
VHosts: genericconf.HTTPConfigDefault.VHosts,
ServerTimeouts: genericconf.HTTPConfigDefault.ServerTimeouts,
}

var WSConfigDefault = genericconf.WSConfig{
Addr: "",
Port: genericconf.WSConfigDefault.Port,
API: []string{},
RPCPrefix: genericconf.WSConfigDefault.RPCPrefix,
Origins: genericconf.WSConfigDefault.Origins,
ExposeAll: genericconf.WSConfigDefault.ExposeAll,
}

var IPCConfigDefault = genericconf.IPCConfig{
Path: "",
}

var AutonomousAuctioneerConfigDefault = AutonomousAuctioneerConfig{
Conf: genericconf.ConfConfigDefault,
LogLevel: "INFO",
LogType: "plaintext",
HTTP: HTTPConfigDefault,
WS: WSConfigDefault,
IPC: IPCConfigDefault,
Metrics: false,
MetricsServer: genericconf.MetricsServerConfigDefault,
PProf: false,
Persistent: conf.PersistentConfigDefault,
PprofCfg: genericconf.PProfDefault,
}

func AuctioneerConfigAddOptions(f *flag.FlagSet) {
timeboost.AuctioneerServerConfigAddOptions("auctioneer-server", f)
timeboost.BidValidatorConfigAddOptions("bid-validator", f)
conf.PersistentConfigAddOptions("persistent", f)
genericconf.ConfConfigAddOptions("conf", f)
f.String("log-level", AutonomousAuctioneerConfigDefault.LogLevel, "log level, valid values are CRIT, ERROR, WARN, INFO, DEBUG, TRACE")
f.String("log-type", AutonomousAuctioneerConfigDefault.LogType, "log type (plaintext or json)")
genericconf.FileLoggingConfigAddOptions("file-logging", f)
genericconf.HTTPConfigAddOptions("http", f)
genericconf.WSConfigAddOptions("ws", f)
genericconf.IPCConfigAddOptions("ipc", f)
genericconf.AuthRPCConfigAddOptions("auth", f)
f.Bool("metrics", AutonomousAuctioneerConfigDefault.Metrics, "enable metrics")
genericconf.MetricsServerAddOptions("metrics-server", f)
f.Bool("pprof", AutonomousAuctioneerConfigDefault.PProf, "enable pprof")
genericconf.PProfAddOptions("pprof-cfg", f)
}

func (c *AutonomousAuctioneerConfig) ShallowClone() *AutonomousAuctioneerConfig {
config := &AutonomousAuctioneerConfig{}
*config = *c
return config
}

func (c *AutonomousAuctioneerConfig) CanReload(new *AutonomousAuctioneerConfig) error {
var check func(node, other reflect.Value, path string)
var err error

check = func(node, value reflect.Value, path string) {
if node.Kind() != reflect.Struct {
return
}

for i := 0; i < node.NumField(); i++ {
fieldTy := node.Type().Field(i)
if !fieldTy.IsExported() {
continue
}
hot := fieldTy.Tag.Get("reload") == "hot"
dot := path + "." + fieldTy.Name

first := node.Field(i).Interface()
other := value.Field(i).Interface()

if !hot && !reflect.DeepEqual(first, other) {
err = fmt.Errorf("illegal change to %v%v%v", colors.Red, dot, colors.Clear)
} else {
check(node.Field(i), value.Field(i), dot)
}
}
}

check(reflect.ValueOf(c).Elem(), reflect.ValueOf(new).Elem(), "config")
return err
}

func (c *AutonomousAuctioneerConfig) GetReloadInterval() time.Duration {
return c.Conf.ReloadInterval
}

func (c *AutonomousAuctioneerConfig) Validate() error {
return nil
}
Comment on lines +133 to +135
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the intention for this? Is it a todo or should it be removed?


var DefaultAuctioneerStackConfig = node.Config{
DataDir: node.DefaultDataDir(),
HTTPPort: node.DefaultHTTPPort,
AuthAddr: node.DefaultAuthHost,
AuthPort: node.DefaultAuthPort,
AuthVirtualHosts: node.DefaultAuthVhosts,
HTTPModules: []string{timeboost.AuctioneerNamespace},
HTTPHost: "localhost",
HTTPVirtualHosts: []string{"localhost"},
HTTPTimeouts: rpc.DefaultHTTPTimeouts,
WSHost: "localhost",
WSPort: node.DefaultWSPort,
WSModules: []string{timeboost.AuctioneerNamespace},
GraphQLVirtualHosts: []string{"localhost"},
P2P: p2p.Config{
ListenAddr: "",
NoDiscovery: true,
NoDial: true,
},
}
Loading
Loading