Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Dockerize DP 🐳 #22

Merged
merged 5 commits into from
Oct 7, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .build/docker/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
FROM oven/bun:alpine

WORKDIR /app

COPY . .

RUN bun install

# Expose the port the app runs on
EXPOSE 5384

# Entry script to handle conditional startup
COPY .build/docker/docker-entrypoint.sh ./docker-entrypoint.sh
RUN chmod +x ./docker-entrypoint.sh

ENTRYPOINT ["sh", "./docker-entrypoint.sh"]
22 changes: 22 additions & 0 deletions .build/docker/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
services:
seda-data-proxy:
build:
context: ../..
dockerfile: .build/docker/Dockerfile
ports:
- "5384:5384"
# environment:
# # Provide the private key if available
# SEDA_DATA_PROXY_PRIVATE_KEY: ${SEDA_DATA_PROXY_PRIVATE_KEY}
#
# volumes:
# # Mount config.json if it exists in the host folder
# - ./config.json:/app/config.json:ro
# # Mount a data proxy private key file
# - ./data-proxy-private-key.json:/app/data-proxy-private-key.json:ro
networks:
- proxy-network

networks:
proxy-network:
driver: bridge
53 changes: 53 additions & 0 deletions .build/docker/docker-entrypoint.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
#!/bin/bash

# Initialize flags for config.json and private key
CONFIG_EXISTS=false
PK_EXISTS=false

# Check if config.json exists
if [ -f /app/config.json ]; then
echo "config.json detected"
CONFIG_EXISTS=true
else
echo "config.json not found"
fi

# Check if data-proxy-private-key.json exists
if [ -f /app/data-proxy-private-key.json ]; then
echo "data-proxy-private-key.json detected"
PK_EXISTS=true
elif [ -n "$SEDA_DATA_PROXY_PRIVATE_KEY" ]; then
# If private key file does not exist, check if the private key is provided via environment variable
echo "Private key provided via environment variable"
echo "$SEDA_DATA_PROXY_PRIVATE_KEY" >/app/data-proxy-private-key.json
PK_EXISTS=true
Comment on lines +20 to +23
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this will work, as the key file is a JSON file with both the public and private key. I'd imagine most people would only supply the private key as an environment variable.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then how do we use $SEDA_DATA_PROXY_PRIVATE_KEY?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the env var is set the data-proxy uses that to construct the private/public key pair and won't even attempt to read the key file.

else
echo "No private key provided"
fi

run_bun_command() {
if ! bun "$@"; then
echo "Failed to run: bun $*"
exit 1
fi
}

# Determine the command to run based on the presence of config.json and private key
if [ "$CONFIG_EXISTS" = true ] && [ "$PK_EXISTS" = true ]; then
# Both config.json and private key are provided
echo "Running with config and private key"
RUN_CMD="start run --config /app/config.json --private-key-file /app/data-proxy-private-key.json"
elif [ "$CONFIG_EXISTS" = true ] && [ "$PK_EXISTS" = false ]; then
# Only config.json is provided
echo "Running with config only"
run_bun_command start init
RUN_CMD="start run --config /app/config.json"
Comment on lines +36 to +44
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the binary takes care of these checks, is there a reason to also do it in the entrypoint?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So that I know how to execute the binary.

e.g.

if [ "$CONFIG_EXISTS" = true ] && [ "$PK_EXISTS" = true ]; then => bun start init not required

Also, if I don't know which ones were provided do we really wanna guarantee graceful handling of non-existent files. Or this scenario => we call bun start run --config /app/config.json --private-key-file /app/data-proxy-private-key.json" but file app/data-proxy-private-key.json" doesn't exist?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not entirely sure what the best approach would be here... 🤔

I'm partial to letting the binary take care of everything as this reduces the maintenance burden on our end since the container doesn't really need to know anything.

If I want to initialise I should do docker run <IMAGE_NAME> init and pass a volume so I can persist the config and generated key.
For registration I can then do docker run <IMAGE_NAME> register <PAYOUT_ADDRESS> <AMOUNT> and click the link from the output to complete the registration.
And to run it just do docker run <IMAGE_NAME> run.

I guess that get a little messy with compose though, don't know how to do that in an elegant fashion.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My primary goal here was to dockerize in the best way possible that would enable first and foremost a functioning deployment.

Having a manual step in the deployment to devnetv4 would increase friction a lot.

Can we discuss these points on a call?

else
# Neither config.json nor private key is provided
echo "Running with --disable-proof"
run_bun_command start init
RUN_CMD="start run --disable-proof"
Comment on lines +46 to +49
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't make sense: --disable-proof should only skip validating whether the incoming request has the correct proof, it doesn't deal with the private key or config at all.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If neither of the two is provided, then my assumption was that we intend to run this in "debug" mode.

Otherwise, if we initiate bun start run when those two aren't provided and -dp isn't enabled the tool will call exit 2.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How would you like me to address third scenario, or do we have 3rd and 4th that I missed?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The data-proxy can not run when there is no config or no private key (file). --disable-proof only disables the request verification, so it should be a completely separate option.

I think the scenarios should be covered by the binary itself where it will error if it is unable to find a config and/or private key (file).

If someone wants to run it in debug mode they should pass the --disable-proof flag manually.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, passing it manually makes sense, maybe we could have an container env variable like FLAGS or so..

fi

# Execute the final command
run_bun_command $RUN_CMD
2 changes: 1 addition & 1 deletion .env.example
Original file line number Diff line number Diff line change
@@ -1 +1 @@
SEDA_DATA_PROXY_PRIVATE_KEY=
SEDA_DATA_PROXY_PRIVATE_KEY=
80 changes: 80 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
name: 🚀 Release

on:
push:
tags: ["*"]

permissions:
contents: write
packages: write

env:
REGISTRY: ghcr.io
IMAGE_NAME: sedaprotocol/seda-data-proxy

jobs:
build-and-push:
name: 🐳 Build and Push Docker Image
runs-on: ubuntu-latest
steps:
- name: 📥 Checkout code
uses: actions/checkout@v4

- name: 🏷️ Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
type=semver,pattern={{major}}
type=edge,branch=main

- name: 🛠️ Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: 🔐 Log in to the Container registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: 🏗️ Build and push Docker image
uses: docker/build-push-action@v5
with:
context: .
file: .build/docker/Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max

create-release:
name: 📦 Create GitHub Release
needs: build-and-push
runs-on: ubuntu-latest
steps:
- name: 📥 Checkout code
uses: actions/checkout@v4

- name: 📝 Generate Changelog
id: changelog
uses: TriPSs/[email protected]
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
output-file: false
skip-commit: true
skip-tag: true
skip-git-pull: true
git-push: false

- name: 🎉 Create GitHub Release
uses: ncipollo/release-action@v1
with:
allowUpdates: true
generateReleaseNotes: true
body: ${{ steps.changelog.outputs.changelog }}
token: ${{ secrets.GITHUB_TOKEN }}
2 changes: 1 addition & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: 'Test'
name: 🧪 Test
on: [pull_request, push]

jobs:
Expand Down
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -174,4 +174,4 @@ dist
# Finder (MacOS) folder config
.DS_Store
config.json
data-proxy-private-key.json
data-proxy-private-key.json
8 changes: 8 additions & 0 deletions .pre-commit-config.yaml
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless we standardise on something as a team I would expect these files to be ignored. In the explorer we use https://commitlint.js.org/guides/local-setup.html and Husky to manage git hooks, but given that a more diverse group works in this repo we should probably discuss it first :P

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't notice Husky was used. pre-commit wouldn't force anyone, nor would it be active locally before you call pre-commit install.

Where is the adequate spot to initiate this discussion? @Thomasvdam

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably in the engineering Slack channel, or opening a PR that introduces the tool.

The reason I commented on this file is that if we want to commit this file (and this workflow) we should also document it in the repo, otherwise it's just clutter.

Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.6.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
- id: check-added-large-files
31 changes: 31 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
.PHONY: build run stop up clean logs ssh

# Define the docker-compose file location
DOCKER_COMPOSE_FILE := .build/docker/docker-compose.yml

# Build the Docker image
build:
docker compose -f $(DOCKER_COMPOSE_FILE) build

# Run the Docker container
run:
docker compose -f $(DOCKER_COMPOSE_FILE) up

# Stop the Docker container
stop:
docker compose -f $(DOCKER_COMPOSE_FILE) down

# Build and run the Docker container
up: build run

# Clean up Docker resources
clean:
docker compose -f $(DOCKER_COMPOSE_FILE) down --rmi all --volumes --remove-orphans

# Show logs
logs:
docker compose -f $(DOCKER_COMPOSE_FILE) logs -f

# SSH into the running container
ssh:
docker compose -f $(DOCKER_COMPOSE_FILE) exec seda-data-proxy sh
2 changes: 1 addition & 1 deletion bunfig.toml
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
[test]
preload = ["./test-setup.ts"]
preload = ["./test-setup.ts"]