Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge main into dev branch 3 #18

Merged
merged 9 commits into from
Aug 8, 2024
4 changes: 2 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,14 @@ on:

jobs:
lint_test:
uses: babylonchain/.github/.github/workflows/reusable_go_lint_test.yml@v0.1.0
uses: babylonlabs-io/.github/.github/workflows/reusable_go_lint_test.yml@v0.2.0
with:
run-unit-tests: true
run-integration-tests: false
run-lint: false

docker_pipeline:
uses: babylonchain/.github/.github/workflows/reusable_docker_pipeline.yml@v0.1.0
uses: babylonlabs-io/.github/.github/workflows/reusable_docker_pipeline.yml@v0.2.0
secrets: inherit
with:
publish: false
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,15 +10,15 @@ on:

jobs:
lint_test:
uses: babylonchain/.github/.github/workflows/reusable_go_lint_test.yml@v0.1.0
uses: babylonlabs-io/.github/.github/workflows/reusable_go_lint_test.yml@v0.2.0
with:
run-unit-tests: true
run-integration-tests: false
run-lint: false

docker_pipeline:
needs: ["lint_test"]
uses: babylonchain/.github/.github/workflows/reusable_docker_pipeline.yml@v0.1.0
uses: babylonlabs-io/.github/.github/workflows/reusable_docker_pipeline.yml@v0.2.0
secrets: inherit
with:
publish: true
Expand Down
6 changes: 3 additions & 3 deletions LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,14 @@ License text copyright (c) 2017 MariaDB Corporation Ab, All Rights Reserved.

Parameters

Licensor: Babylonchain, Inc.
Licensor: Babylon Labs, Ltd.

Licensed Work: staking-api-service
The Licensed Work is (c) 2024 Babylonchain, Inc.
The Licensed Work is (c) 2024 Babylon Labs, Ltd.

Additional Use Grant: None.

Change Date: 2027-05-15 (March 15th, 2027]
Change Date: 2027-05-15 (March 15th, 2027)

Change License: Apache 2.0

Expand Down
25 changes: 20 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,21 @@ The primary infrastructure components include:
2. RabbitMQ
3. Redis cache (Work In Progress)

### Infra recommendation

Since the service is public-facing and exposes a set of endpoints,
it's crucial to apply additional protection mechanisms in front of the service.
Here are some recommendations beyond the basic configurations of request size
limit, header limit, request caching, and DDOS protection:

- Request size limit
- Header limit
- Request caching
- Rate limiting
- DDOS protection mechanism
- SSL/TLS
- ... other common API gateway setup for running with microservices.

### Key Features

- **Asynchronous Communication**: Enables decoupled, non-blocking inter-service
Expand All @@ -39,9 +54,9 @@ The standard staking path encompasses user-initiated staking through CLI/UI,
followed by waiting for the staking period (timelock) to expire.

1. **Transaction Submission**: User-submitted staking transactions are
confirmed and picked up by the [indexer](https://github.com/babylonchain/staking-indexer)
confirmed and picked up by the [indexer](https://github.com/babylonlabs-io/staking-indexer)
after receiving sufficient Bitcoin block confirmations.
2. **Event Queuing**: The indexer sends `ActiveStakingEvent` [messages](https://github.com/babylonchain/staking-queue-client/blob/main/client/schema.go#L24)
2. **Event Queuing**: The indexer sends `ActiveStakingEvent` [messages](https://github.com/babylonlabs-io/staking-queue-client/blob/main/client/schema.go#L24)
to the Active Event Queue for the staking API service to process.
3. **Processing and State Management**: The staking API service executes statistical calculations,
data transformation, and staking state management, inserting records into the `timelock_queue` collection.
Expand All @@ -64,12 +79,12 @@ stores it for further processing by the unbonding pipeline.
3. **Committee Co-Signing**: The unbonding pipeline collects additional signatures
from the covenant committee and submits the unbonding transaction to the Bitcoin network.
4. **Transaction Detection**: The unbonding transaction is detected by the staking-indexer
once it receives a sufficient number of confirmations and a corresponding [unbonding event](https://github.com/babylonchain/staking-queue-client/blob/main/client/schema.go#L70)
once it receives a sufficient number of confirmations and a corresponding [unbonding event](https://github.com/babylonlabs-io/staking-queue-client/blob/main/client/schema.go#L70)
is placed into the RabbitMQ queue.
5. **Processing and State Management**: Similar to the standard path, the staking API service
handles statistical updates, adjusts the staking state, and inserts a record into the `timelock_queue`.
6. **Finalization**: The expire-service processes items from the `timelock_queue` in
MongoDB and emits an [expired event](https://github.com/babylonchain/staking-queue-client/blob/main/client/schema.go#L130)
MongoDB and emits an [expired event](https://github.com/babylonlabs-io/staking-queue-client/blob/main/client/schema.go#L130)
to RabbitMQ for the staking-api-service to process. This marks completed staking
transactions as `unbonded`. This status displays to the user that the staking transaction is ready for withdrawal.

Expand All @@ -85,7 +100,7 @@ transactions as `unbonded`. This status displays to the user that the staking tr
1. Clone the repository:

```bash
git clone [email protected]:babylonchain/staking-api-service.git
git clone [email protected]:babylonlabs-io/staking-api-service.git
```

2. Run the service:
Expand Down
22 changes: 11 additions & 11 deletions cmd/staking-api-service/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,17 @@ import (
"context"
"fmt"

"github.com/babylonchain/staking-api-service/cmd/staking-api-service/cli"
"github.com/babylonchain/staking-api-service/cmd/staking-api-service/scripts"
"github.com/babylonchain/staking-api-service/internal/api"
"github.com/babylonchain/staking-api-service/internal/clients"
"github.com/babylonchain/staking-api-service/internal/config"
"github.com/babylonchain/staking-api-service/internal/db/model"
"github.com/babylonchain/staking-api-service/internal/observability/healthcheck"
"github.com/babylonchain/staking-api-service/internal/observability/metrics"
"github.com/babylonchain/staking-api-service/internal/queue"
"github.com/babylonchain/staking-api-service/internal/services"
"github.com/babylonchain/staking-api-service/internal/types"
"github.com/babylonlabs-io/staking-api-service/cmd/staking-api-service/cli"
"github.com/babylonlabs-io/staking-api-service/cmd/staking-api-service/scripts"
"github.com/babylonlabs-io/staking-api-service/internal/api"
"github.com/babylonlabs-io/staking-api-service/internal/clients"
"github.com/babylonlabs-io/staking-api-service/internal/config"
"github.com/babylonlabs-io/staking-api-service/internal/db/model"
"github.com/babylonlabs-io/staking-api-service/internal/observability/healthcheck"
"github.com/babylonlabs-io/staking-api-service/internal/observability/metrics"
"github.com/babylonlabs-io/staking-api-service/internal/queue"
"github.com/babylonlabs-io/staking-api-service/internal/services"
"github.com/babylonlabs-io/staking-api-service/internal/types"
"github.com/joho/godotenv"
"github.com/rs/zerolog/log"
)
Expand Down
10 changes: 5 additions & 5 deletions cmd/staking-api-service/scripts/replay_unprocessed_messages.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,10 @@ import (
"errors"
"fmt"

"github.com/babylonchain/staking-api-service/internal/config"
"github.com/babylonchain/staking-api-service/internal/db"
"github.com/babylonchain/staking-api-service/internal/queue"
queueClient "github.com/babylonchain/staking-queue-client/client"
"github.com/babylonlabs-io/staking-api-service/internal/config"
"github.com/babylonlabs-io/staking-api-service/internal/db"
"github.com/babylonlabs-io/staking-api-service/internal/queue"
queueClient "github.com/babylonlabs-io/staking-queue-client/client"
"github.com/rs/zerolog/log"
)

Expand Down Expand Up @@ -83,4 +83,4 @@ func processEventMessage(ctx context.Context, queues *queue.Queues, event Generi
fmt.Printf("Error: unknown event type: %v\n", event.EventType)
return fmt.Errorf("unknown event type: %v", event.EventType)
}
}
}
4 changes: 2 additions & 2 deletions contrib/images/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ DOCKER = $(shell which docker)
.PHONY: staking-api-service staking-api-service-rmi

staking-api-service:
$(DOCKER) build --tag babylonchain/staking-api-service -f staking-api-service/Dockerfile \
$(DOCKER) build --tag babylonlabs-io/staking-api-service -f staking-api-service/Dockerfile \
$(shell git rev-parse --show-toplevel)

staking-api-service-rmi:
$(DOCKER) rmi babylonchain/staking-api-service 2>/dev/null; true
$(DOCKER) rmi babylonlabs-io/staking-api-service 2>/dev/null; true
12 changes: 6 additions & 6 deletions contrib/images/staking-api-service/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,12 @@
openssh

# Build
WORKDIR /go/src/github.com/babylonchain/staking-api-service
WORKDIR /go/src/github.com/babylonlabs-io/staking-api-service
# Cache dependencies
COPY go.mod go.sum /go/src/github.com/babylonchain/staking-api-service/
COPY go.mod go.sum /go/src/github.com/babylonlabs-io/staking-api-service/

# Copy the rest of the files
COPY ./ /go/src/github.com/babylonchain/staking-api-service/
COPY ./ /go/src/github.com/babylonlabs-io/staking-api-service/


RUN LDFLAGS='-extldflags "-static" -v' \
Expand All @@ -31,18 +31,18 @@
make build

# Final minimal image with binary only
FROM alpine:3.16 as run

Check warning on line 34 in contrib/images/staking-api-service/Dockerfile

View workflow job for this annotation

GitHub Actions / docker_pipeline / docker_build

The 'as' keyword should match the case of the 'from' keyword

FromAsCasing: 'as' and 'FROM' keywords' casing do not match More info: https://docs.docker.com/go/dockerfile/rule/from-as-casing/

RUN addgroup --gid 1138 -S staking-api-service && adduser --uid 1138 -S staking-api-service -G staking-api-service
RUN apk add bash curl jq

# Label should match your github repo
LABEL org.opencontainers.image.source="https://github.com/babylonchain/staking-api-service:${VERSION}"
LABEL org.opencontainers.image.source="https://github.com/babylonlabs-io/staking-api-service:${VERSION}"

Check warning on line 40 in contrib/images/staking-api-service/Dockerfile

View workflow job for this annotation

GitHub Actions / docker_pipeline / docker_build

Variables should be defined before their use

UndefinedVar: Usage of undefined variable '$VERSION' More info: https://docs.docker.com/go/dockerfile/rule/undefined-var/


# Copy over binaries from the build-env
COPY --from=builder /go/src/github.com/babylonchain/staking-api-service/build/staking-api-service /bin/staking-api-service
COPY --from=builder /go/src/github.com/babylonchain/staking-api-service/contrib/images/staking-api-service/entrypoint.sh /bin/entrypoint.sh
COPY --from=builder /go/src/github.com/babylonlabs-io/staking-api-service/build/staking-api-service /bin/staking-api-service
COPY --from=builder /go/src/github.com/babylonlabs-io/staking-api-service/contrib/images/staking-api-service/entrypoint.sh /bin/entrypoint.sh

WORKDIR /home/staking-api-service
RUN chown -R staking-api-service /home/staking-api-service
Expand Down
2 changes: 1 addition & 1 deletion docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
version: "3.9"
services:
staking-api-service:
image: babylonchain/staking-api-service:latest
image: babylonlabs-io/staking-api-service:latest
container_name: staking-api-service
ports:
- "80:8090"
Expand Down
14 changes: 7 additions & 7 deletions docs/docs.go
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ const docTemplate = `{
"400": {
"description": "Error: Bad Request",
"schema": {
"$ref": "#/definitions/github_com_babylonchain_staking-api-service_internal_types.Error"
"$ref": "#/definitions/github_com_babylonlabs-io_staking-api-service_internal_types.Error"
}
}
}
Expand Down Expand Up @@ -139,7 +139,7 @@ const docTemplate = `{
"400": {
"description": "Error: Bad Request",
"schema": {
"$ref": "#/definitions/github_com_babylonchain_staking-api-service_internal_types.Error"
"$ref": "#/definitions/github_com_babylonlabs-io_staking-api-service_internal_types.Error"
}
}
}
Expand Down Expand Up @@ -176,7 +176,7 @@ const docTemplate = `{
"400": {
"description": "Error: Bad Request",
"schema": {
"$ref": "#/definitions/github_com_babylonchain_staking-api-service_internal_types.Error"
"$ref": "#/definitions/github_com_babylonlabs-io_staking-api-service_internal_types.Error"
}
}
}
Expand Down Expand Up @@ -224,7 +224,7 @@ const docTemplate = `{
"400": {
"description": "Error: Bad Request",
"schema": {
"$ref": "#/definitions/github_com_babylonchain_staking-api-service_internal_types.Error"
"$ref": "#/definitions/github_com_babylonlabs-io_staking-api-service_internal_types.Error"
}
}
}
Expand Down Expand Up @@ -258,7 +258,7 @@ const docTemplate = `{
"400": {
"description": "Invalid request payload",
"schema": {
"$ref": "#/definitions/github_com_babylonchain_staking-api-service_internal_types.Error"
"$ref": "#/definitions/github_com_babylonlabs-io_staking-api-service_internal_types.Error"
}
}
}
Expand Down Expand Up @@ -287,15 +287,15 @@ const docTemplate = `{
"400": {
"description": "Missing or invalid 'staking_tx_hash_hex' query parameter",
"schema": {
"$ref": "#/definitions/github_com_babylonchain_staking-api-service_internal_types.Error"
"$ref": "#/definitions/github_com_babylonlabs-io_staking-api-service_internal_types.Error"
}
}
}
}
}
},
"definitions": {
"github_com_babylonchain_staking-api-service_internal_types.Error": {
"github_com_babylonlabs-io_staking-api-service_internal_types.Error": {
"type": "object",
"properties": {
"err": {},
Expand Down
99 changes: 76 additions & 23 deletions docs/ordinals.md
Original file line number Diff line number Diff line change
@@ -1,41 +1,69 @@
# Ordinals in API system
# Detection of Inscriptions

The Babylon Staking API allows for the option to deploy additional endpoints
that check whether a UTXO contains an inscription or not, with the aim to help
staking applications identify whether they should avoid spending a particular UTXO.
This is accomplished through a connection to the
[Ordinal Service](https://github.com/ordinals/ord) and
a connection to the Unisat API.
Due to Unisat being a payed service and applying rate limits,
the API initially tries to get the status of a UTXO through the Ordinals Service,
and if that fails, then contacts the Unisat API,
effectively using it as a back-up mechanism to handle downtime from the Ordinals Service.
The Babylon Staking API provides optional endpoints to check whether a UTXO
contains an inscription. This functionality helps staking applications avoid
spending specific UTXOs that might hold inscriptions. The API connects to the
[Ordinal Service](https://github.com/ordinals/ord) and the Unisat API. Since
Unisat is a paid service with rate limits, the API first attempts to get the
UTXO status through the Ordinals Service. If that fails, it contacts the Unisat
API as a backup to handle Ordinals Service downtime.

NOTE: To enable the optional ordinal API endpoint, you will need to provide the
`ordinal` and `unisat` configurations under `assets`
**Note**: This is an approximate solution and should not be considered a
foolproof method. There may be false positives or negatives. If you intend to
use this service to detect inscriptions, please assume that the service may not
return entirely accurate results and implement additional fail-safe mechanisms
for inscription detection.

## Ordinal service Client
To enable the optional ordinal API endpoint, provide the `ordinal`
and `unisat` configurations under `assets`.

WIP
## Ordinal Service Client

The Ordinal Service Client is the primary method for checking inscriptions on
UTXOs. It connects directly to a running instance of the [Ordinal Service](https://github.com/ordinals/ord).

### Verification Process

1. The `verifyViaOrdinalService` function is called with a list of UTXOs.
2. It uses the `FetchUTXOInfos` method of the Ordinals client to get
information about the UTXOs.
- For each UTXO, the Ordinals endpoint `/output/{TXID}:{OUTPUT_ID}` is invoked,
where `TXID` is the transaction ID and `OUTPUT_ID` is the output index(i.e `vout`).
3. For each UTXO, it checks:
- If the `Runes` field is not empty and not "{}".
- If the `Inscriptions` field is not empty.
4. If either condition is true, the UTXO is marked as having an inscription.

### Latency

The exact latency depends on the hardware and network setup of the services.
As a reference, you can expect approximately 300ms for a steady 100 requests per second (rps).

## Unisat Service Client

You can find more information about Unisat's Ordinal/BRC-20/Runes related endpoints at:
https://docs.unisat.io/
More information about Unisat's Ordinal/BRC-20/Runes related endpoints can be
found at: [Unisat API Documentation](https://docs.unisat.io/).

In our service, we only utilize the following endpoint:
In our service, we utilize the following endpoint:
- `/v1/indexer/address/{{address}}/inscription-utxo-data`

### How to Use It

1. Log in via https://developer.unisat.io/account/login (create an account if you don't have one).
1. Log in at [Unisat Developer](https://developer.unisat.io/account/login)
(create an account if you don't have one).
2. Copy the `API-Key`.
3. Set the key as an environment variable named `UNISAT_TOKEN`.
4. Configure the values for `unisat.host`, `limit`, `timeout`, etc. Refer to `config-docker.yml`.
4. Configure the values for `unisat.host`, `limit`, `timeout`, etc. Refer
to `config-docker.yml`.
5. Ensure you also set up the `ordinals` configuration, as this is a dependency.
6. Call the POST endpoint `/v1/ordinals/verify-utxos` as shown in the example below:
7. The calls to unisat will only be triggered if the ordinal service is not responding or returning errors
```POST
6. Call the POST endpoint `/v1/ordinals/verify-utxos` as shown in the example below.
7. The Unisat API calls will only be triggered if the Ordinal Service is not
responding or returns errors.

## Example POST Request

POST /v1/ordinals/verify-utxos
```json
{
"utxos": [
{
Expand All @@ -53,3 +81,28 @@ In our service, we only utilize the following endpoint:
],
"address": "tb1pyqjxwcdv6pfcaj2l565ludclz2pwu2k5azs6uznz8kml74kkma6qm0gzlv"
}
```

Response:

```json
{
"data": [
{
"txid": "143c33b4ff4450a60648aec6b4d086639322cb093195226c641ae4f0ae33c3f5",
"vout": 0,
"inscription": true
},
{
"txid": "be3877c8dedd716f026cc77ef3f04f940b40b064d1928247cff5bb08ef1ba58e",
"vout": 1,
"inscription": false
},
{
"txid": "d7f65a37f59088b3b4e4bc119727daa0a0dd8435a645c49e6a665affc109539d",
"vout": 0,
"inscription": false
}
]
}
```
Loading
Loading