Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Flyway for database schema management (#543) #626

Merged
merged 7 commits into from
Dec 11, 2024
Merged

Conversation

eidottermihi
Copy link
Member

@eidottermihi eidottermihi commented Dec 6, 2024

Description

Adds flyway for db schema management. Hibernate's ddl-auto feature removed (as this is flyways job now).

Added a Maven local profile to manage the local PostgresDB from stack via Flyway Maven plugin (e.g. to do a clean -> mvn flyway:clean -Plocal)

Reference

Issues #543

Summary by CodeRabbit

  • New Features

    • Integrated Flyway for database migrations, enhancing schema management capabilities.
    • Added a new database table the_entity with defined attributes.
    • Introduced test data setup for the the_entity table.
  • Bug Fixes

    • Removed outdated Hibernate configuration settings for improved clarity.
  • Documentation

    • Added configuration details for Flyway migrations in both application.yml and application-local.yml.

@eidottermihi eidottermihi requested a review from a team as a code owner December 6, 2024 11:57
@eidottermihi eidottermihi linked an issue Dec 6, 2024 that may be closed by this pull request
2 tasks
Copy link
Contributor

coderabbitai bot commented Dec 6, 2024

Walkthrough

The changes introduced in this pull request primarily enhance the refarch-backend project by integrating Flyway for database migration management. Two new dependencies for Flyway are added to the pom.xml. The configuration files application-local.yml and application.yml are updated to include Flyway migration settings. Additionally, two SQL migration files are introduced: one for creating a new table (the_entity) and another for inserting test data into that table.

Changes

File Path Change Summary
refarch-backend/pom.xml Added dependencies for flyway-core and flyway-database-postgresql; introduced flyway-maven-plugin.
refarch-backend/src/main/resources/application-local.yml Removed ddl-auto setting; added Flyway migration locations for schema and test data.
refarch-backend/src/main/resources/application.yml Added jpa.hibernate.ddl-auto with value validate and Flyway migration locations.
refarch-backend/src/main/resources/db/migration/schema/V001__TheEntity_schema.sql Created the_entity table with text_attribute and id columns.
refarch-backend/src/main/resources/db/migration/testdata/R__Testdata.sql Added script to truncate the_entity and insert test data records.

Sequence Diagram(s)

sequenceDiagram
    participant Developer
    participant LocalDB as Local Database
    participant Flyway as Flyway Migration Tool

    Developer->>LocalDB: Configure local profile
    Developer->>Flyway: Run migrations
    Flyway->>LocalDB: Create table `the_entity`
    Flyway->>LocalDB: Insert test data into `the_entity`
Loading

🐇 "In the garden where data grows,
Flyway hops, as the schema flows.
With tables new and profiles bright,
Our backend's ready, what a sight!
Test data's in, all set to play,
Hooray for changes, hip-hip-hooray!" 🥕


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Experiment)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@github-actions github-actions bot added the Template: Backend Issues regarding the backend template. label Dec 6, 2024
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: ASSERTIVE

📥 Commits

Reviewing files that changed from the base of the PR and between 46b8f27 and 5241aca.

📒 Files selected for processing (3)
  • refarch-backend/pom.xml (2 hunks)
  • refarch-backend/src/main/resources/application-local.yml (0 hunks)
  • refarch-backend/src/main/resources/db/migration/V001__TheEntity_schema.sql (1 hunks)
💤 Files with no reviewable changes (1)
  • refarch-backend/src/main/resources/application-local.yml
🔇 Additional comments (4)
refarch-backend/src/main/resources/db/migration/V001__TheEntity_schema.sql (2)

1-5: 🧹 Nitpick (assertive)

Add an index for the primary key

While PostgreSQL automatically creates an index for the primary key constraint, it's a good practice to explicitly create it for better maintainability and documentation.

 create table the_entity (
     text_attribute varchar(8) not null,
     id uuid not null,
     primary key (id)
-);
+);
+create index idx_the_entity_id on the_entity(id);

Likely invalid or redundant comment.


2-2: Verify the varchar length restriction

The varchar(8) limit seems quite restrictive. Please verify if 8 characters will be sufficient for all use cases.

✅ Verification successful

Let me try a different approach to search for any usage or constraints of this field.


Let me search for the entity class and DTO to understand the complete usage pattern.


The varchar length of 8 is appropriate and validated

The length restriction of 8 characters for text_attribute is intentionally set and properly validated:

  • The entity class has @Size(min = 2, max = 8) validation constraint
  • The field is used in tests with Unicode characters ("Ä-é"), showing it can handle special characters within this length
  • The column definition matches the validation constraints in the entity class
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Search for any Entity classes or DTOs that might use this field
# to verify the expected length requirements

ast-grep --pattern 'class $_ {
  $$$
  @$$$
  private String text$_;
  $$$
}'

Length of output: 79


Script:

#!/bin/bash
# Search for files containing "text_attribute" to understand its usage
rg -i "text_attribute" -A 5 -B 5

# Also search for any SQL files that might reference this column
fd -e sql -x cat {} \; | rg -i "text_attribute" -A 5 -B 5

Length of output: 6127


Script:

#!/bin/bash
# Search for TheEntity class and DTO
fd -e java -x cat {} \; | rg -i "class TheEntity" -A 10 -B 2

# Also search for any validation annotations on the text attribute
ast-grep --pattern '@$_($$$)
private String textAttribute;'

Length of output: 2632

refarch-backend/pom.xml (2)

153-160: LGTM! Dependencies are correctly configured

The Flyway dependencies are properly added and will inherit versions from the Spring Boot parent, which is the recommended approach.


468-468: 🧹 Nitpick (assertive)

Review the clean operation configuration

The cleanDisabled property is set to false, which allows the database to be cleaned. While this might be acceptable for local development, it could be dangerous if accidentally used in other environments.

Consider adding a warning in the configuration comment about the risks of the clean operation, or implementing additional safeguards such as environment checks.

refarch-backend/pom.xml Outdated Show resolved Hide resolved
refarch-backend/pom.xml Outdated Show resolved Hide resolved
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: ASSERTIVE

📥 Commits

Reviewing files that changed from the base of the PR and between 5241aca and aa81594.

📒 Files selected for processing (4)
  • refarch-backend/src/main/resources/application-local.yml (2 hunks)
  • refarch-backend/src/main/resources/application.yml (1 hunks)
  • refarch-backend/src/main/resources/db/migration/schema/V001__TheEntity_schema.sql (1 hunks)
  • refarch-backend/src/main/resources/db/migration/testdata/R__Testdata.sql (1 hunks)
🧰 Additional context used
🪛 yamllint (1.35.1)
refarch-backend/src/main/resources/application-local.yml

[error] 20-20: wrong indentation: expected 6 but found 4

(indentation)

🔇 Additional comments (2)
refarch-backend/src/main/resources/application.yml (1)

4-6: Verify migration scripts organization.

The configuration shows only schema migrations. Let's verify if the migration scripts are properly organized.

✅ Verification successful

Let me check the content of the migration files to ensure proper organization and separation of concerns.


Migration scripts are properly organized and follow best practices

The migration scripts are well-organized with clear separation of concerns:

  • Schema migrations (V001__TheEntity_schema.sql) contain only DDL statements and follow versioned naming convention
  • Test data is properly isolated in a separate repeatable migration (R__Testdata.sql) under the testdata directory
  • No data manipulation statements are mixed within schema migrations
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check the organization of Flyway migration scripts

# Check if migration files follow the naming convention and are properly separated
echo "Checking migration files structure..."
find . -path "*/db/migration/*" -type f -name "*.sql" | while read -r file; do
  echo "Found migration: $file"
  # Check if it's a schema or data migration based on naming convention
  if [[ $file =~ V[0-9]+__.*\.sql ]]; then
    echo "  ✓ Valid version-based migration"
  else
    echo "  ✗ Invalid migration filename format"
  fi
done

# Look for potential data migrations mixed with schema migrations
echo -e "\nChecking for data/schema separation..."
find . -path "*/db/migration/schema/*.sql" -type f -exec grep -l "INSERT INTO\|UPDATE\|DELETE FROM" {} \;

Length of output: 1170


Script:

#!/bin/bash
# Check content of migration files
echo "=== Schema Migration Content ==="
cat ./refarch-backend/src/main/resources/db/migration/schema/V001__TheEntity_schema.sql

echo -e "\n=== Test Data Migration Content ==="
cat ./refarch-backend/src/main/resources/db/migration/testdata/R__Testdata.sql

Length of output: 773

refarch-backend/src/main/resources/application-local.yml (1)

7-8: LGTM! Appropriate Hibernate configuration for Flyway integration

Setting ddl-auto: validate is the correct approach when using Flyway for schema management. This ensures Hibernate only validates the schema against the entity mappings without attempting to modify it.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: ASSERTIVE

📥 Commits

Reviewing files that changed from the base of the PR and between 57b8534 and 481ebd2.

📒 Files selected for processing (1)
  • refarch-backend/pom.xml (2 hunks)
🔇 Additional comments (2)
refarch-backend/pom.xml (2)

152-160: LGTM! Dependencies are correctly configured.

The Flyway dependencies are properly configured and inherit versions from the Spring Boot parent POM, which is a good practice for version management.


451-455: 🧹 Nitpick (assertive)

Consider removing profile binding for better IDE integration.

As discussed in the previous review comments, binding the Flyway plugin to a profile might make it harder to use via IDE's Maven integration. Since the plugin is used manually, consider making it available without profile activation.

refarch-backend/pom.xml Outdated Show resolved Hide resolved
refarch-backend/pom.xml Outdated Show resolved Hide resolved
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: ASSERTIVE

📥 Commits

Reviewing files that changed from the base of the PR and between 481ebd2 and 7f744ca.

📒 Files selected for processing (1)
  • refarch-backend/pom.xml (2 hunks)
🔇 Additional comments (2)
refarch-backend/pom.xml (2)

152-160: LGTM: Flyway dependencies are properly configured.

The required Flyway dependencies for PostgreSQL integration are correctly added and their versions are managed by the Spring Boot parent.


456-456: 🧹 Nitpick (assertive)

Consider using a dedicated database name.

Using the default 'postgres' database is not recommended. Consider using a dedicated database name for the application.

-    <url>jdbc:postgresql://localhost:5432/postgres</url>
+    <url>jdbc:postgresql://localhost:5432/refarch_local</url>

Likely invalid or redundant comment.

refarch-backend/pom.xml Outdated Show resolved Hide resolved
refarch-backend/pom.xml Outdated Show resolved Hide resolved
refarch-backend/pom.xml Outdated Show resolved Hide resolved
@eidottermihi eidottermihi requested a review from devtobi December 11, 2024 10:30
Copy link
Member

@devtobi devtobi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@eidottermihi eidottermihi added the Type: Feature The issue is an feature label Dec 11, 2024
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: ASSERTIVE

📥 Commits

Reviewing files that changed from the base of the PR and between 070cc5f and 0a47e4a.

📒 Files selected for processing (1)
  • refarch-backend/src/main/resources/application-local.yml (1 hunks)
🔇 Additional comments (2)
refarch-backend/src/main/resources/application-local.yml (2)

Line range hint 11-13: Consider using environment variables for database credentials

For better security practices, consider externalizing the database credentials using environment variables, even in the local profile.

  datasource:
    url: jdbc:postgresql://localhost:5432/postgres
-    password: admin
-    username: admin
+    password: ${DB_PASSWORD:admin}
+    username: ${DB_USERNAME:admin}

Line range hint 4-8: Verify Hibernate validation mode in main configuration

Based on previous discussions, the ddl-auto: validate setting should be moved to the main application.yml. Let's verify this configuration:

✅ Verification successful

Hibernate validation mode is correctly configured

The ddl-auto: validate setting is properly configured in the main application.yml file, which is the recommended location for this production-critical configuration.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Check if Hibernate validation mode exists in the main application.yml
rg "ddl-auto.*validate" "refarch-backend/src/main/resources/application.yml"

# Also check for any remaining ddl-auto configurations
rg "ddl-auto" "refarch-backend/src/main/**/*.yml"

Length of output: 225


Script:

#!/bin/bash
# Let's try searching with correct path patterns
rg "ddl-auto" "refarch-backend/src/main/resources/"

Length of output: 125

@eidottermihi eidottermihi merged commit 5e0468b into main Dec 11, 2024
11 checks passed
@eidottermihi eidottermihi deleted the 543-flyway branch December 11, 2024 13:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Template: Backend Issues regarding the backend template. Type: Feature The issue is an feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature] Setup database migration (i.e. Flyway)
3 participants