Individual docker scripts ci changes

[nedvedba]

Summary by Sourcery

Refactor CI configuration to utilize new script-based Docker container management, enhancing maintainability and reducing inline script complexity.

CI:

• Refactor CI scripts to use new container script generation for core, web, repo, and GCS components, replacing inline Docker run commands with script-based execution.

[sourcery-ai]

Reviewer's Guide by Sourcery

This pull request refactors the CI/CD pipeline for Docker container deployment, introducing new scripts for generating and managing container scripts for the metadata server, repository server, and GCS (Globus Connect Server). The changes aim to improve modularity, reduce redundancy, and enhance the maintainability of the deployment process.

File-Level Changes

Change	Details	Files
Refactored Docker container deployment scripts	Removed inline Docker run commands and replaced them with calls to generated scripts Added new script 'generate_metadata_container_scripts.sh' to create Docker run scripts for core and web containers Added new script 'generate_repo_container_scripts.sh' to create Docker run scripts for repository and GCS containers Introduced scripts for creating and removing a Docker network for DataFed containers Added scripts for running, stopping, and removing individual containers (core, web, nginx, repo, gcs)	.gitlab/end_to_end.yml scripts/generate_metadata_container_scripts.sh scripts/generate_repo_container_scripts.sh
Updated database configuration in generate_datafed.sh	Added support for configurable database port Set default database port to 8529 if not specified	scripts/generate_datafed.sh
Improved CI/CD pipeline configuration	Removed 'generate_datafed.sh' call from CI/CD stages Added calls to new container management scripts in CI/CD stages Updated environment variable handling for container deployment	.gitlab/end_to_end.yml

---

Tips

• Trigger a new Sourcery review by commenting @sourcery-ai review on the pull request.
• Continue your discussion with Sourcery by replying directly to review comments.
• You can change your review settings at any time by accessing your dashboard:
  • Enable or disable the Sourcery-generated pull request summary or reviewer's guide;
  • Change the review language;
• You can always contact us if you have any questions or feedback.

[sourcery-ai]

Hey @nedvedba - I've reviewed your changes - here's some feedback:

Overall Comments:

• Consider adding error handling in the CI pipeline to check for the existence and executability of the new scripts before running them.
• It would be beneficial to update or create documentation for the new environment variables and configuration options introduced in this change.

Here's what I looked at during the review

• 🟡 General issues: 2 issues found
• 🟢 Security: all looks good
• 🟢 Testing: all looks good
• 🟢 Complexity: all looks good
• 🟢 Documentation: all looks good

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment to tell me if it was helpful.}

[sourcery-ai]

suggestion: Consider centralizing environment variable handling

While sourcing the config file is good, consider creating a separate function to handle environment variables. This would make it easier to manage and update environment variables across all scripts.

# Function to load environment variables
load_environment_variables() {
    local config_file="${PROJECT_ROOT}/config/datafed.sh"
    if [[ -f "$config_file" ]]; then
        source "$config_file"
    else
        echo "Error: Config file not found: $config_file"
        exit 1
    fi
}

load_environment_variables

[sourcery-ai]

suggestion: Add error handling for Docker commands

Consider adding error checking after Docker commands. For example: 'if [ $? -ne 0 ]; then echo "Error: Docker command failed"; exit 1; fi'

docker run -d \
    --restart=always \
    --name datafed-nginx \
if [ $? -ne 0 ]; then
    echo "Error: Docker run command failed"
    exit 1
fi