Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/donghoon/add docker compose #2

Merged
merged 4 commits into from
Dec 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .env.sample
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
BROKER_URI=amqp://<user_name>:<password>@<rabbitmq_container_name>:5672/<rabbitmq_vhost_name>
FIREBASE_DATABASE_URL=<firebase_database_url>
FIREBASE_STORAGE_BUCKET=<firebase_storage_bucket>
FIREBASE_APP_NAME=super-resolution
64 changes: 48 additions & 16 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,44 +2,54 @@

Serving Super Resolution Model Using FastAPI and Celery.

## For Developers

1. install dev package.
## How to start
### Using docker-compose(recommended)
1. Create RabbitMQ and FastAPI container(refer to [SR-FastAPI](https://github.com/ainize-team/SR-FastAPI))

2. Clone repository
```shell
pip install -r requirements.txt
pip install -r requirements-dev.txt
git clone https://github.com/ainize-team/SR-Worker
cd SR-Worker
```

2. install pre-commit.
3. Edit [docker-compose.yml](./docker-compose.yml) and [.env file](./.env.sample) for your project.
- Vhost for each worker is in [docker-compose.yml](./docker-compose.yml) and common RabbitMQ config is in [.env file](./.env.sample).

4. Run worker container
```shell
pre-commit install
docker-compose up -d

# If you want to run a specific worker container, write service name.
docker-compose up -d <service name>
```

## Installation
1. Run RabbitMQ image as a broker
### Using docker
1. Run RabbitMQ comtainer as a broker
```shell
docker run -d --name sr-rabbitmq -p 5672:5672 -p 8080:15672 --restart=unless-stopped rabbitmq:3.9.21-management
docker run -d --name sr-rabbitmq -p 5672:5672 -p 15672:15672 --restart=unless-stopped rabbitmq:3.11.2-management
```

2. Build docker image
2. Clone repository
```shell
git clone https://github.com/ainize-team/SR-Worker.git
git clone https://github.com/ainize-team/SR-Worker
cd SR-Worker
```

3. Build docker image
```shell
docker build -t sr-worker .
```

3. Run docker image
4. Run docker container
```shell
docker run -d --name <worker_container_name> \
--gpus='"device=0"' -e BROKER_URI=<broker_uri> \
-e DATABASE_URL=<firebase_realtime_database_url> \
-e STORAGE_BUCKET=<firebase_storage_url> \
-e FIREBASE_DATABASE_URL=<firebase_realtime_database_url> \
-e FIREBASE_STORAGE_BUCKET=<firebase_storage_url> \
-v <firebase_credential_path>:/app/key -v <model_local_path>:/app/model \
sr-worker
```
Or, you can use the env file to run as follows.
Or, you can use the [.env file](./.env.sample) to run as follows.
```shell
docker run -d --name <worker_container_name> \
--gpus='"device=0"' \
Expand All @@ -48,5 +58,27 @@ docker run -d --name <worker_container_name> \
sr-worker
```








## For Developers

1. install dev package.

```shell
pip install -r requirements.txt
pip install -r requirements-dev.txt
```

2. install pre-commit.

```shell
pre-commit install
```

## Test with FastAPI
- Check our [SR-FastAPI](https://github.com/ainize-team/SR-FastAPI) Repo.
29 changes: 29 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
version: '3'

services:
swinlr-large-x4:
build:
context: .
dockerfile: Dockerfile
container_name: swinlr-large-x4-worker
env_file:
- .env
environment:
- VHOST_NAME=swinlr-large-x4
volumes:
- <firebase_credential_path>:/app/key
- <model_local_path>:/app/model
networks:
- sr-fastapi_dev
restart: unless-stopped
deploy:
resources:
reservations:
devices:
- driver: nvidia
device_ids: [<gpu_id>]
capabilities: [gpu]

networks:
sr-fastapi_dev:
external: true
4 changes: 2 additions & 2 deletions src/configs/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@ class ModelSettings(BaseSettings):
class FirebaseSettings(BaseSettings):
firebase_app_name: str = "super-resolution"
cred_path: str = "/app/key/serviceAccountKey.json"
database_url: str
storage_bucket: str
firebase_database_url: str
firebase_storage_bucket: str


celery_worker_settings = CeleryWorkerSettings()
Expand Down
4 changes: 2 additions & 2 deletions src/worker.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
firebase_admin.initialize_app(
cred,
{
"databaseURL": firebase_settings.database_url,
"storageBucket": firebase_settings.storage_bucket,
"databaseURL": firebase_settings.firebase_database_url,
"storageBucket": firebase_settings.firebase_storage_bucket,
},
)
Loading