# Enhance CCDB Web and Database Deployment with Docker

## Summary

- Build a test database (`test_ccdb`) from `ccdb.mysql.sql` following proper SQL dump ordering.
- Add a custom `test_user` with `test_password` for local development and deployment.
- Utilize Docker Compose for a cohesive network setup, database volumes, and containerized builds.
- Streamline web deployment with a `requirements-web.txt` file, adding `cryptography` for SQLAlchemy provider compatibility.
- Abstracted the app component from `ccdbweb.py`'s `__main__` for Gunicorn-based deployment, exposing it to external IPs from the container.
- Goal: Enable seamless deployment and local testing of `ccdbweb`.

## Key Changes

### Docker Compose
- Defined services for:
  - `mysqldb`: Builds and initializes the database using MySQL with SQL dump files.
  - `webgui`: Builds the Flask-based `ccdbweb` GUI and connects to `mysqldb`.
- Volume setup:
  - Persist MySQL data with `mysql_data`.
  - Load initialization SQL files with `mysql_init_data`.
- Enabled network connectivity between containers using `app_network`.

### Dockerfiles
- **MySQL (`docker/mysqldb/Dockerfile`):**
  - Added SQL initialization scripts (`test_user_init.sql`, `ccdb.mysql.sql`) with numeric prefixes for proper execution order.
  - Exposed port `3306`.
- **Web GUI (`docker/webgui/Dockerfile`):**
  - Added `requirements-web.txt` for web-specific dependencies.
  - Used Gunicorn for serving the application with 4 worker processes.
  - Exposed port `5000`.

### Code Updates
- **`ccdbweb.py`:**
  - Abstracted Flask app creation to be compatible with Gunicorn.
  - Added `SQL_CONNECTION_STRING` for flexible database connectivity using environment variables.
- **`requirements-web.txt`:**
  - Added Flask, Bokeh, and Gunicorn dependencies.
- **`requirements.txt`:**
  - Added `cryptography` to address SQLAlchemy provider needs.

## Usage Instructions
To run the `ccdbweb` application locally with the test database:
1. Start the containers:
   ```bash
   cd docker
   docker-compose up --build
2. Check the website in a browser:
   ```bash
   localhost:5000

docker/docker-compose.yml

@@ -0,0 +1,43 @@
+version: '3.8'
+
+services:
+  mysqldb:
+    build:
+      context: .. # Use the parent directory as the context
+      dockerfile: docker/mysqldb/Dockerfile
+    container_name: mysqldb
+    environment:
+      MYSQL_ROOT_PASSWORD: root_password
+      # Set the MySQL database name so the ccdb.mysql.sql can proceed
+      MYSQL_DATABASE: test_ccdb
+    ports:
+      - "3306:3306" # Expose MySQL to the host
+    networks:
+      - app_network
+    volumes:
+      # Adjust the path for sql directory relative to docker-compose.yml
+      - mysql_data:/var/lib/mysql
+      - mysql_init_data:/docker-entrypoint-initdb.d
+
+  webgui:
+    build:
+      context: .. # Use the parent directory as the context
+      dockerfile: docker/webgui/Dockerfile
+    container_name: webgui
+    environment:
+      # dialect://username:password@host:port/database
+      SQL_CONNECTION_STRING: mysql://test_user:test_password@mysqldb:3306/test_ccdb
+    depends_on:
+      - mysqldb
+    ports:
+      - "5000:5000" # Expose web GUI
+    networks:
+      - app_network
+
+networks:
+  app_network:
+    driver: bridge
+
+volumes:
+  mysql_init_data:
+  mysql_data:

docker/mysqldb/Dockerfile

@@ -0,0 +1,13 @@
+# Use the official MySQL image as the base
+FROM mysql:latest
+
+# Set the MySQL root password
+# ENV MYSQL_ROOT_PASSWORD=root_password
+# Set the MySQL database name so the ccdb.mysql.sql can proceed
+# ENV MYSQL_DATABASE=test_ccdb
+
+# Copy SQL initialization scripts into the MySQL entrypoint initialization directory
+COPY sql/test_user_init.sql /docker-entrypoint-initdb.d/001-test_user_init.sql
+COPY sql/ccdb.mysql.sql /docker-entrypoint-initdb.d/002-ccdb.mysql.sql
+
+EXPOSE 3306

docker/webgui/Dockerfile

@@ -0,0 +1,32 @@
+# Use an official Python runtime as a parent image
+FROM python:3.10-slim
+
+# Set the working directory in the container
+WORKDIR /app
+
+# Copy the Python files and directories to the container
+COPY ./python/ /app
+
+# Set environment variables
+# Prevents Python from writing pyc files to disk
+ENV PYTHONDONTWRITEBYTECODE 1
+# Prevents Python from buffering stdout and stderr
+ENV PYTHONUNBUFFERED 1
+# Set the CCDB_HOME environment variable
+ENV CCDB_HOME=/app
+
+# Install system dependencies
+RUN apt-get update && apt-get install -y \
+    && rm -rf /var/lib/apt/lists/*
+
+COPY ./python/requirements.txt .
+COPY ./python/ccdb/webgui/requirements-web.txt .
+# Install Python dependencies
+RUN pip install --upgrade pip && \
+    pip install --no-cache-dir -r requirements-web.txt
+
+# Expose the port the app runs on
+EXPOSE 5000
+
+# Set the command to run the application with Gunicorn
+CMD ["gunicorn", "-w", "4", "-b", "0.0.0.0:5000", "ccdbweb:app"]

python/ccdb/webgui/__init__.py

@@ -1,3 +1,4 @@
+import os
 import ccdb
 from ccdb.model import User, TypeTable
 from ccdb.path_utils import parse_request, ParseRequestResult
@@ -54,7 +55,11 @@ def cerate_ccdb_flask_app(test_config=None):
     app = Flask(__name__, instance_relative_config=True)
     app.config.from_mapping(
         # a default secret that should be overridden by instance config
-        SQL_CONNECTION_STRING="mysql://[email protected]/ccdb2"
+        #SQL_CONNECTION_STRING="mysql://[email protected]/ccdb2"
+        SQL_CONNECTION_STRING = os.getenv(
+            "SQL_CONNECTION_STRING",
+            "mysql://test_user:test_user_password@localhost:3306/test_ccdb"
+        )
     )
 
     @app.before_request

python/ccdb/webgui/requirements-web.txt

@@ -0,0 +1,7 @@
+# Load the CCDB requirements
+-r requirements.txt
+# Requirements for the webgui
+Flask
+Bokeh
+bokeh_sampledata
+gunicorn

python/ccdbweb.py

@@ -1,8 +1,8 @@
-#python
+import ccdb.webgui
 
-if __name__ == "__main__":
+# Create the Flask app
+app = ccdb.webgui.cerate_ccdb_flask_app()
 
-    import ccdb.webgui
-    # import ccdb.ccdb_pyllapi
-    app = ccdb.webgui.cerate_ccdb_flask_app()
+if __name__ == "__main__":
+    # Run the app in debug mode when executed directly
     app.run(debug=True)

python/requirements.txt

@@ -5,6 +5,7 @@
 pymysql
 sqlalchemy
 six
+cryptography
 
 # web gui requirements
 Flask

sql/test_user_init.sql

@@ -0,0 +1,13 @@
+CREATE DATABASE IF NOT EXISTS test_ccdb;
+
+-- Create the user for external access
+CREATE USER 'test_user'@'%' IDENTIFIED BY 'test_password';
+
+-- Grant all privileges on the test database to the user
+GRANT ALL PRIVILEGES ON test_ccdb.* TO 'test_user'@'%';
+
+-- Apply the privilege changes
+FLUSH PRIVILEGES;
+
+-- Switch to the test database
+USE test_ccdb;