Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Still erroring out, but I am a bit stuck #43

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
open_energy_view/frontend/package-lock.json
*/node_modules
23 changes: 23 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
services:
rabbit:
build:
dockerfile: ./docker/service_rabbitmq/Dockerfile.rabbitmq
context: ./
environment:
- RABBITMQ_DEFAULT_USER=jp
- RABBITMQ_DEFAULT_PASS=admin
Comment on lines +7 to +8
Copy link
Owner

@JPHutchins JPHutchins Mar 5, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Elsewhere it looks for these without "DEFAULT" - so changing to:

    environment:
      - RABBITMQ_USER=jp
      - RABBITMQ_PASS=admin

seemed to resolve some of the errors

cpu:
tty: true
depends_on:
- rabbit
build:
context: ./
dockerfile: ./docker/service_web/Dockerfile.web
args:
NODE_OPTIONS: --openssl-legacy-provider
ports:
- 5001:5001
links:
- rabbit


5 changes: 5 additions & 0 deletions docker/service_rabbitmq/Dockerfile.rabbitmq
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
FROM rabbitmq:latest

copy ./docker/service_rabbitmq/init.sh /app/init.sh

CMD ["/app/init.sh"]
9 changes: 9 additions & 0 deletions docker/service_rabbitmq/init.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
#!/bin/sh

rabbitmqctl wait --timeout 60
rabbitmqctl add_user $RABBITMQ_USER $RABBITMQ_PASSWORD 2>/dev/null
rabbitmqctl set_user_tags $RABBITMQ_USER administrator
rabbitmqctl add_vhost myvhost
rabbitmqctl set_permissions -p / $RABBITMQ_USER ".*" ".*" ".*" ;

rabbitmq-server $@
25 changes: 25 additions & 0 deletions docker/service_web/Dockerfile.web
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
FROM node:latest as node

SHELL ["/bin/bash", "--login", "-c"]
WORKDIR /app
COPY . .

RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash
RUN nvm install 10.19.0
RUN nvm use 10


ARG NODE_OPTIONS
WORKDIR /app/open_energy_view/frontend
RUN npm install
RUN npm run build

WORKDIR /app

FROM python:3.9

COPY requirements.txt ./
COPY --from=node /app ./
RUN pip install --no-cache-dir -r requirements.txt

CMD ["./docker/service_web/init.sh"]
7 changes: 7 additions & 0 deletions docker/service_web/init.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#!/bin/sh



./run-wsgi-dev
./run-io-worker
./run-cpu-worker
Comment on lines +5 to +7
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the actual server these are run each as a separate sytemd process. Thus the instructions to run them each in a separate terminal.

This is all a bit misleading - the core of the app will work with just running the uwsgi server. The io and cpu workers process incoming data from PGE's servers - a work around to use multiple cores with python.

I'll also need to look for the flask entry point.

TLDR most devs would rather run the flask development server and not worry about the celery workers. I added these scripts so that I could simulate the live server - in fact, it basically IS the production setup, just run from terminals instead of systemd.

4 changes: 2 additions & 2 deletions run-cpu-worker
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/bin/bash
source venv/bin/activate
#source venv/bin/activate
export FLASK_CONFIG=config.DevConfig
export FLASK_APP=open_energy_view
export FLASK_ENV=production
Expand All @@ -22,5 +22,5 @@ export OAUTHLIB_INSECURE_TRANSPORT=True
export CERT_PATH=test/cert/cert.crt
export KEY_PATH=test/cert/private.key
export API_RESPONSE_KEY=xS5MqJ6N9CyH-hvqAGrmBVAxFMOyauMpdrdqCZa1eqo=
export IP_AND_PORT=$(hostname -I | awk '{print $1}'):5000
export IP_AND_PORT=$(hostname -I | awk '{print $1}'):5001
celery worker --app open_energy_view.celery_tasks --hostname=celery.cpus@%h --queues=cpu --loglevel=info --concurrency=1
2 changes: 1 addition & 1 deletion run-io-worker
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/bin/bash
source venv/bin/activate
#source venv/bin/activate

celery worker --app open_energy_view.celery_tasks --hostname=celery.io@%h --queues=io --loglevel=info --pool=gevent --concurrency=500
4 changes: 2 additions & 2 deletions run-wsgi-dev
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/bin/bash
source venv/bin/activate
#source venv/bin/activate
export FLASK_CONFIG=config.DevConfig
export FLASK_APP=open_energy_view
export FLASK_ENV=production
Expand All @@ -23,4 +23,4 @@ export CERT_PATH=test/cert/cert.crt
export KEY_PATH=test/cert/private.key
export API_RESPONSE_KEY=xS5MqJ6N9CyH-hvqAGrmBVAxFMOyauMpdrdqCZa1eqo=
export IP_AND_PORT=$(hostname -I | awk '{print $1}'):5000
./venv/bin/uwsgi --http 0.0.0.0:5000 --gevent 100 --wsgi-file wsgi.py --callable app
./venv/bin/uwsgi --http 0.0.0.0:5001 --gevent 100 --wsgi-file wsgi.py --callable app
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like in the docker environment a python venv is not needed. So changing this line to uwsgi --http 0.0.0.0:5001 --gevent 100 --wsgi-file wsgi.py --callable app seems to have it find uwsgi. pip installs it someplace in path luckily!