Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: new bridge history apis #1024

Merged
merged 63 commits into from
Dec 28, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
f011fd5
feat: new bridge-history apis
colinlyguo Nov 16, 2023
bdbddc3
update go mod
colinlyguo Dec 6, 2023
0803dd9
refactor
colinlyguo Dec 6, 2023
e79900e
refactor
colinlyguo Dec 6, 2023
5902bee
fix
colinlyguo Dec 6, 2023
7629f06
change config default value
colinlyguo Dec 6, 2023
299f5e4
change Makefile
colinlyguo Dec 6, 2023
a6c72a2
fix
colinlyguo Dec 6, 2023
9bfefa0
update l1 start block height to first queue event
colinlyguo Dec 6, 2023
f031c38
Apply suggestions from code review
colinlyguo Dec 6, 2023
5936cd6
fixes
colinlyguo Dec 6, 2023
5622f8f
tweak
colinlyguo Dec 7, 2023
59c8470
fix batch index update
colinlyguo Dec 7, 2023
01d35d7
add fetching reverted relayed messages and fixes
colinlyguo Dec 7, 2023
a75813e
add PageSize limit
colinlyguo Dec 7, 2023
8587116
Update bridge-history-api/internal/config/config.go
colinlyguo Dec 7, 2023
cc5fd77
tweak
colinlyguo Dec 7, 2023
2f1abc0
address comments of new bridge history api (#1027)
colinlyguo Dec 8, 2023
3676368
fix: avoid duplicated batch updates (#1029)
colinlyguo Dec 11, 2023
7f5c823
Merge branch 'develop' into fix-bridge-history-api-write-db
colinlyguo Dec 11, 2023
c4c68ef
chore: auto version bump [bot]
colinlyguo Dec 11, 2023
bee9489
Merge branch 'develop' into fix-bridge-history-api-write-db
colinlyguo Dec 11, 2023
3a090ee
chore: auto version bump [bot]
colinlyguo Dec 11, 2023
faec5a5
trigger ci
colinlyguo Dec 11, 2023
e00d5a9
fix bugs
colinlyguo Dec 12, 2023
f11ce33
perf(bridge-history-api): add db indices and concurrent blocks fetchi…
colinlyguo Dec 12, 2023
cd883a3
add batch sync height in db
colinlyguo Dec 12, 2023
ebf7a46
remove useless logs
colinlyguo Dec 13, 2023
1d358fa
tweak logs
colinlyguo Dec 13, 2023
b534aed
Merge branch 'develop' into fix-bridge-history-api-write-db
colinlyguo Dec 13, 2023
9b5a7f7
fix go mod tidy
colinlyguo Dec 13, 2023
0da1355
tweak Makefile
colinlyguo Dec 13, 2023
7fe751f
fix ERROR: ON CONFLICT DO UPDATE command cannot affect row a second t…
colinlyguo Dec 13, 2023
04006b3
update message_type in relayed messages
colinlyguo Dec 13, 2023
3f9e500
Merge branch 'develop' into fix-bridge-history-api-write-db
colinlyguo Dec 13, 2023
420c96d
go mod tidy
colinlyguo Dec 13, 2023
1075d7d
chore: auto version bump [bot]
colinlyguo Dec 13, 2023
fef9f83
go mod tidy
colinlyguo Dec 13, 2023
bb1a7f6
add 0x prefix in merkle proof
colinlyguo Dec 13, 2023
5c1de26
split relayed failed to failed relayed and relayed tx reverted
colinlyguo Dec 13, 2023
33ec1e1
add withdraw root check
colinlyguo Dec 13, 2023
2f7e329
fix duplicated update reverted txs
colinlyguo Dec 14, 2023
7011c3e
feat: update some layer logic (#1050)
georgehao Dec 14, 2023
9484c1c
Merge branch 'develop' into fix-bridge-history-api-write-db
colinlyguo Dec 14, 2023
8d034de
feat: tweak some code (#1052)
georgehao Dec 15, 2023
a5ca547
bug fixes
colinlyguo Dec 16, 2023
5e2fe45
change start height
colinlyguo Dec 16, 2023
91d9926
fix a bug
colinlyguo Dec 17, 2023
ec6e618
fix another bug
colinlyguo Dec 17, 2023
fae11cc
add a case
colinlyguo Dec 17, 2023
60a4cde
add a clarify comment
colinlyguo Dec 17, 2023
3fc6534
add a comment
colinlyguo Dec 17, 2023
16d82f5
fix comment
colinlyguo Dec 17, 2023
e596a38
change some withdraw trie functions internal
colinlyguo Dec 17, 2023
8ee4dd8
Merge branch 'develop' into fix-bridge-history-api-write-db
colinlyguo Dec 18, 2023
7ddd1b7
fix a comment
colinlyguo Dec 18, 2023
842e256
fix a bug
colinlyguo Dec 18, 2023
6526878
filter L1 sentMessage in QueueTransaction
colinlyguo Dec 18, 2023
57e4987
feat(bridge-history-api): reorg handling (#1055)
colinlyguo Dec 19, 2023
dc1c18c
feat(bridge-history-api): add metrics (#1056)
colinlyguo Dec 22, 2023
bcd9489
refactor(bridge-history-api): rename some variables (#1057)
colinlyguo Dec 22, 2023
cad9e2b
Merge branch 'develop' into fix-bridge-history-api-write-db
colinlyguo Dec 22, 2023
3aa2c90
Merge branch 'develop' into fix-bridge-history-api-write-db
colinlyguo Dec 27, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions .github/workflows/docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ jobs:
tags: scrolltech/rollup-relayer:${{github.ref_name}}
# cache-from: type=gha,scope=${{ github.workflow }}
# cache-to: type=gha,scope=${{ github.workflow }}
bridgehistoryapi-cross-msg-fetcher:
bridgehistoryapi-fetcher:
runs-on: ubuntu-latest
steps:
- name: Checkout code
Expand All @@ -81,16 +81,16 @@ jobs:
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build and push bridgehistoryapi-cross-msg-fetcher docker
- name: Build and push bridgehistoryapi-fetcher docker
uses: docker/build-push-action@v2
with:
context: .
file: ./build/dockerfiles/bridgehistoryapi-cross-msg-fetcher.Dockerfile
file: ./build/dockerfiles/bridgehistoryapi-fetcher.Dockerfile
push: true
tags: scrolltech/bridgehistoryapi-cross-msg-fetcher:${{github.ref_name}}
tags: scrolltech/bridgehistoryapi-fetcher:${{github.ref_name}}
# cache-from: type=gha,scope=${{ github.workflow }}
# cache-to: type=gha,scope=${{ github.workflow }}
bridgehistoryapi-server:
bridgehistoryapi-api:
runs-on: ubuntu-latest
steps:
- name: Checkout code
Expand All @@ -102,13 +102,13 @@ jobs:
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build and push bridgehistoryapi-server docker
- name: Build and push bridgehistoryapi-api docker
uses: docker/build-push-action@v2
with:
context: .
file: ./build/dockerfiles/bridgehistoryapi-server.Dockerfile
file: ./build/dockerfiles/bridgehistoryapi-api.Dockerfile
push: true
tags: scrolltech/bridgehistoryapi-server:${{github.ref_name}}
tags: scrolltech/bridgehistoryapi-api:${{github.ref_name}}
# cache-from: type=gha,scope=${{ github.workflow }}
# cache-to: type=gha,scope=${{ github.workflow }}
coordinator-api:
Expand Down
25 changes: 17 additions & 8 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,11 +1,20 @@
.PHONY: fmt dev_docker build_test_docker run_test_docker clean
.PHONY: fmt dev_docker build_test_docker run_test_docker clean update

L2GETH_TAG=scroll-v5.1.6

help: ## Display this help message
@grep -h \
-E '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) | \
awk 'BEGIN {FS = ":.*?## "}; {printf "\033[36m%-30s\033[0m %s\n", $$1, $$2}'
update:
go work sync
cd $(PWD)/bridge-history-api/ && go get -u github.com/scroll-tech/go-ethereum@${L2GETH_TAG} && go mod tidy
cd $(PWD)/common/ && go get -u github.com/scroll-tech/go-ethereum@${L2GETH_TAG}&& go mod tidy
cd $(PWD)/coordinator/ && go get -u github.com/scroll-tech/go-ethereum@${L2GETH_TAG} && go mod tidy
cd $(PWD)/database/ && go get -u github.com/scroll-tech/go-ethereum@${L2GETH_TAG} && go mod tidy
cd $(PWD)/prover/ && go get -u github.com/scroll-tech/go-ethereum@${L2GETH_TAG}&& go mod tidy
cd $(PWD)/rollup/ && go get -u github.com/scroll-tech/go-ethereum@${L2GETH_TAG} && go mod tidy
cd $(PWD)/tests/integration-test/ && go get -u github.com/scroll-tech/go-ethereum@${L2GETH_TAG} && go mod tidy

lint: ## The code's format and security checks.
make -C rollup lint
Expand All @@ -17,13 +26,13 @@ lint: ## The code's format and security checks.

fmt: ## format the code
go work sync
cd $(PWD)/bridge-history-api/ && go get -u github.com/ethereum/go-ethereum@latest && go mod tidy
cd $(PWD)/common/ && go get -u github.com/scroll-tech/go-ethereum@${L2GETH_TAG} && go mod tidy
cd $(PWD)/coordinator/ && go get -u github.com/scroll-tech/go-ethereum@${L2GETH_TAG} && go mod tidy
cd $(PWD)/database/ && go get -u github.com/scroll-tech/go-ethereum@${L2GETH_TAG} && go mod tidy
cd $(PWD)/prover/ && go get -u github.com/scroll-tech/go-ethereum@${L2GETH_TAG} && go mod tidy
cd $(PWD)/rollup/ && go get -u github.com/scroll-tech/go-ethereum@${L2GETH_TAG} && go mod tidy
cd $(PWD)/tests/integration-test/ && go get -u github.com/scroll-tech/go-ethereum@${L2GETH_TAG} && go mod tidy
cd $(PWD)/bridge-history-api/ && go mod tidy
cd $(PWD)/common/ && go mod tidy
cd $(PWD)/coordinator/ && go mod tidy
cd $(PWD)/database/ && go mod tidy
cd $(PWD)/prover/ && go mod tidy
cd $(PWD)/rollup/ && go mod tidy
cd $(PWD)/tests/integration-test/ && go mod tidy

goimports -local $(PWD)/bridge-history-api/ -w .
goimports -local $(PWD)/common/ -w .
Expand Down
32 changes: 24 additions & 8 deletions bridge-history-api/Makefile
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
.PHONY: lint
REPO_ROOT_DIR=./..
IMAGE_VERSION=latest
PWD=$(shell pwd)

lint: ## Lint the files - used for CI
GOBIN=$(PWD)/build/bin go run ../build/lint.go
Expand All @@ -11,16 +12,31 @@ test:
bridgehistoryapi-db-cli:
go build -o $(PWD)/build/bin/bridgehistoryapi-db-cli ./cmd/db_cli

bridgehistoryapi-cross-msg-fetcher:
go build -o $(PWD)/build/bin/bridgehistoryapi-cross-msg-fetcher ./cmd/cross_msg_fetcher
bridgehistoryapi-fetcher:
go build -o $(PWD)/build/bin/bridgehistoryapi-fetcher ./cmd/fetcher

bridgehistoryapi-server:
go build -o $(PWD)/build/bin/bridgehistoryapi-server ./cmd/backend_server
bridgehistoryapi-api:
go build -o $(PWD)/build/bin/bridgehistoryapi-api ./cmd/api

db-docker:
reset-env:
if docker ps -a -q -f name=bridgehistoryapi-redis | grep -q . ; then \
docker stop bridgehistoryapi-redis; \
docker rm bridgehistoryapi-redis; \
fi
docker run --name bridgehistoryapi-redis -d -p 6379:6379 redis:latest
if docker ps -a -q -f name=bridgehistoryapi-history-db | grep -q . ; then \
docker stop bridgehistoryapi-history-db; \
docker rm bridgehistoryapi-history-db; \
fi
docker run --name bridgehistoryapi-history-db -p 5444:5432 -e POSTGRES_PASSWORD=123456 -e POSTGRES_DB=test -d postgres
until docker exec bridgehistoryapi-history-db pg_isready -h localhost -p 5432 -U postgres > /dev/null; do \
echo "Waiting for postgres to be ready..."; \
sleep 1; \
done
echo "Postgres is ready."
go build -o $(PWD)/build/bin/bridgehistoryapi-db-cli ./cmd/db_cli && $(PWD)/build/bin/bridgehistoryapi-db-cli reset

bridgehistoryapi-docker:
DOCKER_BUILDKIT=1 docker build -t scrolltech/bridgehistoryapi-cross-msg-fetcher:${IMAGE_VERSION} ${REPO_ROOT_DIR}/ -f ${REPO_ROOT_DIR}/build/dockerfiles/bridgehistoryapi-cross-msg-fetcher.Dockerfile
DOCKER_BUILDKIT=1 docker build -t scrolltech/bridgehistoryapi-server:${IMAGE_VERSION} ${REPO_ROOT_DIR}/ -f ${REPO_ROOT_DIR}/build/dockerfiles/bridgehistoryapi-server.Dockerfile
DOCKER_BUILDKIT=1 docker build -t scrolltech/bridgehistoryapi-db-cli:${IMAGE_VERSION} ${REPO_ROOT_DIR}/ -f ${REPO_ROOT_DIR}/build/dockerfiles/bridgehistoryapi-db-cli.Dockerfile
DOCKER_BUILDKIT=1 docker build -t scrolltech/bridgehistoryapi-fetcher:${IMAGE_VERSION} ${REPO_ROOT_DIR}/ -f ${REPO_ROOT_DIR}/build/dockerfiles/bridgehistoryapi-fetcher.Dockerfile
DOCKER_BUILDKIT=1 docker build -t scrolltech/bridgehistoryapi-api:${IMAGE_VERSION} ${REPO_ROOT_DIR}/ -f ${REPO_ROOT_DIR}/build/dockerfiles/bridgehistoryapi-api.Dockerfile
DOCKER_BUILDKIT=1 docker build -t scrolltech/bridgehistoryapi-db-cli:${IMAGE_VERSION} ${REPO_ROOT_DIR}/ -f ${REPO_ROOT_DIR}/build/dockerfiles/bridgehistoryapi-db-cli.Dockerfile
47 changes: 23 additions & 24 deletions bridge-history-api/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,30 +14,27 @@ Provide init, show version, rollback, and check status services of DB
./build/bin/bridgehistoryapi-db-cli [command]
```

### bridgehistoryapi-cross-msg-fetcher
### bridgehistoryapi-fetcher

Fetch the transactions from both l1 and l2
Fetch the transactions from both L1 and L2
```
cd ./bridge-history-api
make bridgehistoryapi-cross-msg-fetcher
./build/bin/bridgehistoryapi-cross-msg-fetcher
make bridgehistoryapi-fetcher
./build/bin/bridgehistoryapi-fetcher
```

### bridgehistoryapi-server
### bridgehistoryapi-api

provides REST APIs. Please refer to the API details below.
```
cd ./bridge-history-api
make bridgehistoryapi-server
./build/bin/bridgehistoryapi-server
make bridgehistoryapi-api
./build/bin/bridgehistoryapi-api
```

## APIs provided by bridgehistoryapi-server
## APIs provided by bridgehistoryapi-api

assume `bridgehistoryapi-server` listening on `https://localhost:8080`
can change this port by modifying `config.json`

1. `/txs`
1. `/api/txs`
```
// @Summary get all txs under the given address
// @Accept plain
Expand All @@ -49,34 +46,36 @@ can change this port by modifying `config.json`
// @Router /api/txs [get]
```

2. `/txsbyhashes`
2. `/api/l2/withdrawals`
```
// @Summary get txs by given tx hashes
// @Summary get all L2 withdrawals under given address
// @Accept plain
// @Produce plain
// @Param hashes query string array true "array of hashes list"
// @Success 200
// @Router /api/txsbyhashes [post]
// @Param address query string true "wallet address"
// @Param page_size query int true "page size"
// @Param page query int true "page"
// @Success 200
// @Router /api/l2/withdrawals [get]
```

3. `/claimable`
3. `/api/l2/unclaimed/withdrawals`
```
// @Summary get all claimable txs under the given address
// @Summary get all L2 unclaimed withdrawals under the given address
// @Accept plain
// @Produce plain
// @Param address query string true "wallet address"
// @Param page_size query int true "page size"
// @Param page query int true "page"
// @Success 200
// @Router /api/claimable [get]
// @Router /api/l2/unclaimed/withdrawals [get]
```

4. `/withdraw_root`
4. `/api/txsbyhashes`
```
// @Summary get withdraw_root of given batch index
// @Summary get txs by given tx hashes
// @Accept plain
// @Produce plain
// @Param batch_index query string true "batch_index"
// @Param hashes query string array true "array of hashes"
// @Success 200
// @Router /api/withdraw_root [get]
// @Router /api/txsbyhashes [post]
```
Loading
Loading