Make use of postgres package

[bradsawadye]

Update the Jempi service to use the postgres database package

Summary by CodeRabbit

• New Features

  • Introduced PostgreSQL database initialization scripts that create tables and insert initial data.
  • Added new PostgreSQL configuration services for client-registry-jempi and dashboard-visualiser-superset.

• Configuration Updates

  • Updated environment variables for PostgreSQL configurations across multiple Docker Compose files.
  • Added new PostgreSQL service definitions and network configurations.
  • Removed outdated PostgreSQL service configurations for better performance and maintenance.

• Dependency Updates

  • Added PostgreSQL-related dependencies in package.json files for database configuration.

• Script Enhancements

  • Enhanced swarm.sh scripts to deploy PostgreSQL config importers for database setup.

[coderabbitai]

Walkthrough

The updates primarily focus on reconfiguring PostgreSQL settings across various Docker Compose files and scripts. They introduce new environment variables for PostgreSQL configuration, streamline the setup by removing redundant services, and add scripts for initializing and configuring PostgreSQL databases. These changes enhance the consistency and manageability of the PostgreSQL setup within the client registry and dashboard visualizer services.

Changes

Files/Paths	Change Summaries
.env.cluster	Added new PostgreSQL environment variables.
client-registry-jempi/docker-compose.api.yml	Updated PostgreSQL-related environment variables.
client-registry-jempi/docker-compose.combined-cluster.yml	Removed configurations for jempi-postgresql-02 and jempi-postgresql-03 services.
client-registry-jempi/docker-compose.combined-dev.yml	Removed jempi-postgresql-01 service definition.
client-registry-jempi/docker-compose.combined.yml	Updated PostgreSQL environment variables and added a new postgres service.
client-registry-jempi/importer/postgres/create-db.js	Added scripts to initialize PostgreSQL database and user.
client-registry-jempi/importer/postgres/docker-compose.config.yml	Introduced configuration for jempi_db_config service.
client-registry-jempi/importer/postgres/package.json	Added pg dependency for PostgreSQL configuration.
client-registry-jempi/package-metadata.json	Updated dependencies and added new PostgreSQL environment variables.
client-registry-jempi/swarm.sh	Added PostgreSQL database configuration in the initialize_package function.
dashboard-visualiser-superset/docker-compose.yml	Added a new postgres service and network configuration.
dashboard-visualiser-superset/importer/postgres/create-db.js	Added scripts to initialize PostgreSQL database and user.
dashboard-visualiser-superset/importer/postgres/docker-compose.config.yml	Introduced configuration for superset_db_config service.
dashboard-visualiser-superset/importer/postgres/package.json	Added pg dependency for PostgreSQL configuration.
dashboard-visualiser-superset/package-metadata.json	Updated dependencies and added new PostgreSQL environment variables.
dashboard-visualiser-superset/swarm.sh	Updated to deploy config importer for PostgreSQL setup.
test/cucumber/features/cluster-mode/jempi.cluster.feature	Removed starting and removal steps for jempi-postgresql-01 service.
test/cucumber/features/single-mode/jempi.feature	Removed jempi-postgresql-01 service from scenarios.

Poem

In clusters vast, the changes flow,
PostgreSQL's seeds we sow. 🌱
Tables set and users born,
In Docker's dawn, a bright new morn.
Scripts that hum and configs sing,
JeMPI and Superset take wing.
With every line, the future's clear,
Data's realm, we hold dear. 🚀🐇

---

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share

• X
• Mastodon
• Reddit
• LinkedIn

Tips

Chat

There are 3 ways to chat with CodeRabbit:

• Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  • I pushed a fix in commit <commit_id>.
  • Generate unit testing code for this file.
  • Open a follow-up GitHub issue for this discussion.
• Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
  • @coderabbitai generate unit testing code for this file.
  • @coderabbitai modularize this function.
• PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  • @coderabbitai generate interesting stats about this repository and render them as a table.
  • @coderabbitai show all the console.log statements in this repository.
  • @coderabbitai read src/utils.ts and generate unit testing code.
  • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
  • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

• @coderabbitai pause to pause the reviews on a PR.
• @coderabbitai resume to resume the paused reviews.
• @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
• @coderabbitai full review to do a full review from scratch and review all the files again.
• @coderabbitai summary to regenerate the summary of the PR.
• @coderabbitai resolve resolve all the CodeRabbit review comments.
• @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
• @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

• You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
• Please see the configuration documentation for more information.
• If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

• Visit our Documentation for detailed information on how to use CodeRabbit.
• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[coderabbitai]

Actionable comments posted: 2

Review Details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 679e2b6 and b9f7f36.

Files selected for processing (10)

• .env.cluster (2 hunks)
• client-registry-jempi/docker-compose.api.yml (5 hunks)
• client-registry-jempi/docker-compose.combined-cluster.yml (1 hunks)
• client-registry-jempi/docker-compose.combined-dev.yml (2 hunks)
• client-registry-jempi/docker-compose.combined.yml (5 hunks)
• client-registry-jempi/importer/postgres/create-db.js (1 hunks)
• client-registry-jempi/importer/postgres/docker-compose.config.yml (1 hunks)
• client-registry-jempi/importer/postgres/package.json (1 hunks)
• client-registry-jempi/package-metadata.json (2 hunks)
• client-registry-jempi/swarm.sh (1 hunks)

Files skipped from review due to trivial changes (5)

• client-registry-jempi/docker-compose.api.yml
• client-registry-jempi/docker-compose.combined-cluster.yml
• client-registry-jempi/docker-compose.combined-dev.yml
• client-registry-jempi/importer/postgres/docker-compose.config.yml
• client-registry-jempi/importer/postgres/package.json

Additional comments not posted (9)

.env.cluster (1)

103-103: The configuration for JEMPI_POSTGRES_DB is correctly set to point to multiple PostgreSQL instances, which is suitable for high availability and load balancing.

client-registry-jempi/package-metadata.json (2)

11-11: The addition of the database-postgres dependency aligns with the PR's objective to integrate PostgreSQL support.

---

64-74: The new PostgreSQL-related environment variables are correctly added and structured to support the integration with the PostgreSQL database.

client-registry-jempi/importer/postgres/create-db.js (1)

1-136: The script for setting up the PostgreSQL database is well-structured and uses best practices such as checking for existing databases and users before creation, which prevents potential errors during re-runs.

client-registry-jempi/docker-compose.combined.yml (4)

48-52: The updated PostgreSQL configuration for the jempi-controller service correctly reflects the new database settings and network requirements.

Also applies to: 76-76

---

83-87: The updated PostgreSQL configuration for the jempi-linker service correctly reflects the new database settings and network requirements.

Also applies to: 112-113

---

118-121: The updated PostgreSQL configuration for the jempi-bootstrapper service correctly reflects the new database settings and network requirements.

Also applies to: 135-135

---

150-152: The addition of the postgres network is correctly configured to facilitate communication between services and the PostgreSQL database.

client-registry-jempi/swarm.sh (1)

68-70: The addition of the PostgreSQL database configuration deployment in the initialize_package function is correctly implemented and aligns with the PR's objectives.

[coderabbitai]

Actionable comments posted: 1

Review Details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between b9f7f36 and eacce75.

Files selected for processing (8)

• .env.cluster (3 hunks)
• client-registry-jempi/importer/postgres/docker-compose.config.yml (1 hunks)
• dashboard-visualiser-superset/docker-compose.yml (2 hunks)
• dashboard-visualiser-superset/importer/postgres/create-db.js (1 hunks)
• dashboard-visualiser-superset/importer/postgres/docker-compose.config.yml (1 hunks)
• dashboard-visualiser-superset/importer/postgres/package.json (1 hunks)
• dashboard-visualiser-superset/package-metadata.json (2 hunks)
• dashboard-visualiser-superset/swarm.sh (2 hunks)

Files skipped from review due to trivial changes (1)

• dashboard-visualiser-superset/importer/postgres/package.json

Files skipped from review as they are similar to previous changes (2)

• .env.cluster
• client-registry-jempi/importer/postgres/docker-compose.config.yml

Additional comments not posted (4)

dashboard-visualiser-superset/importer/postgres/docker-compose.config.yml (1)

4-26: Ensure the Docker service configuration aligns with the intended database setup operations.

The configuration correctly sets up the environment for running the create-db.js script, including mounting necessary files and setting up the network. The restart policy is set to 'none', which is appropriate for a one-time configuration task.

dashboard-visualiser-superset/package-metadata.json (1)

Line range hint 7-33: Validate the updated dependencies and environment variables.

The addition of database-postgres to dependencies and the new PostgreSQL-related environment variables are consistent with the PR's objective to integrate PostgreSQL. Ensure that all services using these variables are updated accordingly.

dashboard-visualiser-superset/swarm.sh (1)

48-49: Verify the integration of PostgreSQL configuration in the deployment script.

The script correctly deploys the PostgreSQL configuration importer, ensuring that the database and user setup is handled as part of the service initialization. This aligns with the changes made in other parts of the system.

dashboard-visualiser-superset/docker-compose.yml (1)

Line range hint 39-79: Confirm the network configuration for the new PostgreSQL service.

The addition of the postgres network to the Superset service ensures that it can communicate with the newly integrated PostgreSQL service. This is crucial for the correct operation of the Superset application with the new database backend.

[coderabbitai]

Actionable comments posted: 0

Review Details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between eacce75 and 2abe67f.

Files selected for processing (1)

• dashboard-visualiser-superset/package-metadata.json (2 hunks)

Files skipped from review as they are similar to previous changes (1)

• dashboard-visualiser-superset/package-metadata.json

[coderabbitai]

Actionable comments posted: 0

Review Details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 2abe67f and bb2d67e.

Files selected for processing (1)

• client-registry-jempi/importer/postgres/create-db.js (1 hunks)

Files skipped from review as they are similar to previous changes (1)

• client-registry-jempi/importer/postgres/create-db.js

[brett-onions]

Look Good to Me

[drizzentic]

LGTM! We just need to ensure tests are also refactored to work with new package. Also left a small minor comment

[coderabbitai]

Actionable comments posted: 0

Review Details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between bb2d67e and 35480dd.

Files selected for processing (2)

• test/cucumber/features/cluster-mode/jempi.cluster.feature (2 hunks)
• test/cucumber/features/single-mode/jempi.feature (2 hunks)

Files skipped from review due to trivial changes (2)

• test/cucumber/features/cluster-mode/jempi.cluster.feature
• test/cucumber/features/single-mode/jempi.feature

[coderabbitai]

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 35480dd and f2c75ef.

Files selected for processing (1)

• client-registry-jempi/importer/postgres/create-db.js (1 hunks)

Files skipped from review as they are similar to previous changes (1)

• client-registry-jempi/importer/postgres/create-db.js