Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

integrate docker compose #27062

Merged
merged 91 commits into from
Sep 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
91 commits
Select commit Hold shift + click to select a range
bac69c3
integrate docker compose
mshima Aug 27, 2024
ab7757d
use withMockedSource
mshima Aug 27, 2024
c93ac11
disable docker-compose integration by default
mshima Sep 7, 2024
5558e79
add github-build-matrix command
mshima Aug 28, 2024
2723bbc
fix tests
mshima Sep 7, 2024
7b6e518
switch to mock from node:test
mshima Sep 8, 2024
d73349e
fix global it
mshima Sep 8, 2024
3fdcd1c
add doc
mshima Sep 8, 2024
50db650
fix compilation issue
mshima Sep 8, 2024
b63471c
matchWritten* should not depend on esmocha
mshima Sep 9, 2024
e0da8d4
Merge remote-tracking branch 'upstream/main' into compose
mshima Sep 11, 2024
6fe987e
add temporary testcontainers workflow
mshima Sep 11, 2024
c3273c7
adjusts
mshima Sep 11, 2024
4ae7353
fix matrix
mshima Sep 11, 2024
68ca371
add clientTestFrameworks to cli
mshima Sep 11, 2024
ebca0ef
Update testcontainers.ts
mshima Sep 12, 2024
86ea4c4
adjust workflow
mshima Sep 12, 2024
40a5ecb
enable docker compose integration
mshima Sep 12, 2024
6a55b86
Update testcontainers.ts
mshima Sep 12, 2024
00b8e87
Update testcontainers-integration.yml
mshima Sep 12, 2024
104190a
Update generator.ts
mshima Sep 12, 2024
6c27b59
Update testcontainers-integration.yml
mshima Sep 12, 2024
b74dfa8
Merge branch 'jhipster:main' into compose
mshima Sep 12, 2024
8cf6268
use ci:e2e:dev script
mshima Sep 12, 2024
6261dfa
add password to postgresql
mshima Sep 12, 2024
d74d58a
remove whitespace
mshima Sep 12, 2024
5deae08
adjust containers supported by composer-compose integration
mshima Sep 12, 2024
9f8e17e
fix prettier
mshima Sep 12, 2024
d51e676
update snapshot
mshima Sep 12, 2024
a5aa5b1
add password to mariadb and mysql
mshima Sep 12, 2024
6914433
update snapshots
mshima Sep 12, 2024
5c21801
hide clientTestFrameworks cli option
mshima Sep 12, 2024
190c257
fix test
mshima Sep 12, 2024
6fdbfc6
fix neo4j with elasticsearch
mshima Sep 12, 2024
3e0e694
disable docker-compose integration for cassandra
mshima Sep 12, 2024
33d674d
drop password for mysql
mshima Sep 12, 2024
de0bb79
drop password for mariadb and postgresql
mshima Sep 12, 2024
190c068
update snapshot
mshima Sep 12, 2024
7b1b06e
add MARIADB_ALLOW_EMPTY_ROOT_PASSWORD again
mshima Sep 12, 2024
0c7bffa
add spring label to Cassandra migration
mshima Sep 12, 2024
e9cb681
move docker-compose dep back to profile
mshima Sep 12, 2024
b7185fc
add password to postgresql again
mshima Sep 12, 2024
06e590d
add docker-compose integration for cassandra
mshima Sep 12, 2024
9e773d0
update snapshot
mshima Sep 12, 2024
38cd5eb
adjusts
mshima Sep 12, 2024
2944ad6
fix neo4j import
mshima Sep 12, 2024
7058b15
require label to enable ci
mshima Sep 12, 2024
bf006f6
rename matrix sample name and use jdl
mshima Sep 12, 2024
1a29b11
Revert "add docker-compose integration for cassandra"
mshima Sep 12, 2024
34a86bb
fix jdl
mshima Sep 12, 2024
c279385
remove reactive mariadb sample
mshima Sep 12, 2024
3df0a95
Use baseName as password for postgresql
mshima Sep 12, 2024
b618477
cleanup
mshima Sep 12, 2024
17b2efc
set compose to false for mariadb + reactive
mshima Sep 12, 2024
ca7bc1d
use strong password in postgresql
mshima Sep 12, 2024
0ac0cf6
update snapshot
mshima Sep 12, 2024
144f9e1
generate a safe password for sonar
mshima Sep 13, 2024
89e247b
fix sonar recommendations option
mshima Sep 13, 2024
3c7bf00
enable docker-compose integration for keycloak
mshima Sep 13, 2024
560ae55
try to reenable disabled containers
mshima Sep 13, 2024
236ffa8
fix imports
mshima Sep 13, 2024
325618b
remove ./ from compose file
mshima Sep 13, 2024
6311fe0
skipJhipsterDependencies
mshima Sep 13, 2024
616c59d
containers adjusts
mshima Sep 13, 2024
17c961a
ignore sonar issue
mshima Sep 13, 2024
ba2749d
container adjusts
mshima Sep 13, 2024
bcb9421
disable docker compose for serviceDiscoveryType
mshima Sep 13, 2024
38adf5e
cleanup
mshima Sep 13, 2024
152acfd
lint fix
mshima Sep 13, 2024
faba084
sonar error
mshima Sep 13, 2024
2e2f032
add docker logger
mshima Sep 13, 2024
7d55aae
Merge remote-tracking branch 'upstream/main' into compose
mshima Sep 13, 2024
a22539f
adjusts for docker compose integration with service discovery
mshima Sep 13, 2024
f3fd6d3
update snapshot
mshima Sep 13, 2024
740a504
reduce log
mshima Sep 13, 2024
260849d
add h2 test
mshima Sep 13, 2024
3f72155
fix sonar property
mshima Sep 13, 2024
13e9ea7
use database docker compose file in microservices
mshima Sep 13, 2024
6d52440
prettier adjusts
mshima Sep 13, 2024
aa56565
update snapshot
mshima Sep 13, 2024
6ea3817
add -ntp
mshima Sep 13, 2024
d03312f
concurrently adjusts
mshima Sep 13, 2024
a46f317
concurrently adjusts
mshima Sep 13, 2024
48f240f
reduce kafka logs
mshima Sep 13, 2024
55fa4d2
adjusts
mshima Sep 13, 2024
fab41df
add FORCE_COLOR to workflow
mshima Sep 13, 2024
73efee0
fix test
mshima Sep 13, 2024
b22caa6
adjust resources filtering
mshima Sep 13, 2024
414d5aa
maven resources adjusts
mshima Sep 13, 2024
de4f367
cleanup leftover
mshima Sep 13, 2024
02b5b0e
fix microservice compose file with serviceDiscorvery
mshima Sep 13, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .blueprint/cli/commands.mts
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,10 @@ const defaultCommands = {
desc: 'Generate a test sample',
blueprint: '@jhipster/jhipster-dev',
},
'github-build-matrix': {
desc: 'Generate a matrix for GitHub Actions',
blueprint: '@jhipster/jhipster-dev',
},
'update-vscode': {
desc: 'Update generator-jhipster vscode files',
blueprint: '@jhipster/jhipster-dev',
Expand Down
14 changes: 14 additions & 0 deletions .blueprint/github-build-matrix/command.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
import type { JHipsterCommandDefinition } from '../../generators/index.js';

export default {
configs: {
workflow: {
description: 'Workflow',
argument: {
type: String,
},
scope: 'generator',
choices: ['testcontainers'],
},
},
} as const satisfies JHipsterCommandDefinition;
22 changes: 22 additions & 0 deletions .blueprint/github-build-matrix/generator.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
import BaseGenerator from '../../generators/base/index.js';
import { setGithubTaskOutput } from '../../lib/testing/index.js';
import { convertToGitHubMatrix } from './support/github-ci-matrix.js';
import { dockerComposeMatrix } from './samples/docker-compose-integration.js';

export default class extends BaseGenerator {
workflow;

constructor(args, opts, features) {
super(args, opts, { queueCommandTasks: true, ...features, jhipsterBootstrap: false });
}

get [BaseGenerator.WRITING]() {
return this.asWritingTaskGroup({
async buildMatrix() {
if (this.workflow === 'docker-compose-integration') {
setGithubTaskOutput('matrix', JSON.stringify(convertToGitHubMatrix(dockerComposeMatrix), null, 2));
}
},
});
}
}
2 changes: 2 additions & 0 deletions .blueprint/github-build-matrix/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
export { default } from './generator.js';
export { default as command } from './command.js';
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
import { extendMatrix, fromMatrix } from '../../../lib/testing/index.js';
import { convertOptionsToJDL } from '../support/jdl.js';

// Supported containers: https://github.com/spring-projects/spring-boot/tree/main/spring-boot-project/spring-boot-docker-compose/src/main/java/org/springframework/boot/docker/compose/service/connection
export const dockerComposeMatrix = Object.fromEntries(
[
...Object.entries(
extendMatrix(
{
...fromMatrix({
databaseType: ['cassandra', 'mongodb', 'neo4j'],
reactive: [undefined, true],
}),
...extendMatrix(
fromMatrix({
prodDatabaseType: ['postgresql', 'mysql', 'mariadb'],
reactive: [undefined],
}),
{ cacheProvider: ['no', 'redis', 'memcached'] },
),
...fromMatrix({
prodDatabaseType: ['postgresql', 'mysql', 'mariadb'],
reactive: [true],
}),
},
{
buildTool: ['maven', 'gradle'],
searchEngine: [undefined, 'elasticsearch'],
authenticationType: ['jwt', 'oauth2'],
serviceDiscoveryType: [undefined, 'eureka', 'consul'],
messageBroker: [undefined, 'kafka'],
},
),
),
['h2', { devDatabaseType: 'h2Disk' }],
].map(([key, value]) => [
key,
{
'cmd-e2e': 'npm run ci:e2e:dev',
args: 'jdl',
jdl: convertOptionsToJDL(value),
},
]),
);
13 changes: 13 additions & 0 deletions .blueprint/github-build-matrix/support/cli-args.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
import { kebabCase } from 'lodash-es';

export const convertToCliArgs = (opts: Record<string, any>): string => {
return Object.entries(opts)
.map(([key, value]) => {
key = kebabCase(key);
if (typeof value === 'boolean') {
return `--${value ? '' : 'no-'}${key}`;
}
return `--${key} ${value}`;
})
.join(' ');
};
38 changes: 38 additions & 0 deletions .blueprint/github-build-matrix/support/github-ci-matrix.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
import { RECOMMENDED_JAVA_VERSION, RECOMMENDED_NODE_VERSION } from '../../../generators/index.js';

type GitHubMatrix = {
os: string;
'node-version': string;
'java-version': string;
'default-environment': string;
'job-name': string;
args: string;
};

type GitHubMatrixOutput = {
include: GitHubMatrix[];
};

export const defaultEnvironment = {
os: 'ubuntu-latest',
'node-version': RECOMMENDED_NODE_VERSION,
'java-version': RECOMMENDED_JAVA_VERSION,
'default-environment': 'prod',
};

export const defaultEnvironmentMatrix = {
os: ['ubuntu-latest'],
'node-version': [RECOMMENDED_NODE_VERSION],
'java-version': [RECOMMENDED_JAVA_VERSION],
'default-environment': ['prod'],
};

export const convertToGitHubMatrix = (matrix: Record<string, any>): GitHubMatrixOutput => {
return {
include: Object.entries(matrix).map(([key, value]) => ({
'job-name': key,
...defaultEnvironment,
...value,
})),
};
};
11 changes: 11 additions & 0 deletions .blueprint/github-build-matrix/support/jdl.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
export const convertOptionsToJDL = (opts: Record<string, any>): string => {
return `application {
config {
testFrameworks [cypress]
${Object.entries(opts)
.filter(([_key, value]) => value !== undefined)
.map(([key, value]) => ` ${key} ${value}`)
.join('\n')}
}
}`;
};
109 changes: 109 additions & 0 deletions .github/workflows/docker-compose-integration.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
#
# Copyright the original author or authors from the JHipster project.
#
# This file is part of the JHipster project, see https://www.jhipster.tech/
# for more information.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

name: Testcontainers Integration
concurrency:
# Group PRs by head_ref, push to main branch by commit id, and others branch by ref.
group: ${{ github.workflow }}-${{ github.head_ref || (github.ref == 'refs/heads/main' && github.sha) || github.ref }}
cancel-in-progress: true
on:
pull_request:
types: [closed, opened, synchronize, reopened]
branches:
- '*'
jobs:
build-matrix:
runs-on: ubuntu-20.04
if: >-
contains(github.event.pull_request.labels.*.name, 'pr: docker-compose integration')
outputs:
matrix: ${{ steps.build.outputs.matrix }}
empty-matrix: ${{ steps.build.outputs.empty-matrix }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 1
- uses: actions/setup-node@v4
with:
node-version: '20'
- run: npm ci --ignore-scripts
- id: build
run: bin/jhipster.cjs github-build-matrix docker-compose-integration
applications:
name: ${{ matrix.job-name }}
needs: build-matrix
runs-on: ${{ matrix.os }}
defaults:
run:
working-directory: ${{ github.workspace }}/app
timeout-minutes: 40
strategy:
fail-fast: false
matrix: ${{fromJson(needs.build-matrix.outputs.matrix)}}
steps:
#----------------------------------------------------------------------
# Install all tools and check configuration
#----------------------------------------------------------------------
- name: 'SETUP: Checkout generator-jhipster'
uses: actions/checkout@v4
with:
path: generator-jhipster
fetch-depth: 2
- uses: jhipster/actions/setup-runner@v0
with:
node-version: ${{ matrix.node-version }}
java-version: ${{ matrix.java-version }}
npm-version: ${{ matrix.npm-version }}
maven-cache: true
gradle-cache: ${{ matrix.gradle-cache }}
binary-dir: ${{ github.workspace }}/generator-jhipster/bin
- run: npm ci --ignore-scripts
working-directory: ${{ github.workspace }}/generator-jhipster
- uses: jhipster/actions/build-jhipster-bom@v0
with:
jhipster-bom-ref: main
- name: Generate project
run: jhipster.cjs ${{ matrix.args }} --defaults
env:
JHIPSTER_DEPENDENCIES_VERSION: 0.0.0-CICD
JHI_SKIP_JHIPSTER_DEPENDENCIES: true
JHI_PROFILE: ${{ matrix.default-environment }}
JHI_JDL: ${{ matrix.jdl }}
- run: jhipster.cjs info
- run: ${{ matrix.cmd-e2e }}
id: e2e
- name: Upload cypress screenshots
uses: actions/upload-artifact@v4
if: always() && steps.e2e.outcome == 'failure'
with:
name: screenshots-${{ matrix.name }}
path: ${{ github.workspace }}app//*/cypress/screenshots
check-workflow:
permissions:
contents: none
runs-on: ubuntu-latest
needs: [applications]
if: always()
steps:
- run: |
echo '${{ toJSON(needs) }}'
if [ 'skipped' == '${{ needs.applications.result }}' ] || [ 'success' == '${{ needs.applications.result }}' ] || [ 'closed' == '${{ github.event.action }}' ]; then
exit 0
fi
exit 1
2 changes: 2 additions & 0 deletions .github/workflows/docker-image-publish-github-registry.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ on:
tags:
- 'v*.*.*'

env:
FORCE_COLOR: 1
jobs:
build:
runs-on: ubuntu-20.04
Expand Down
14 changes: 7 additions & 7 deletions generators/app/__snapshots__/generator.spec.ts.snap
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ Options:
--force-install Fail on install dependencies error (default: false)
--ask-answered Show prompts for already configured options (default: false)
--base-name <value> Application base name
--skip-jhipster-dependencies Don't write jhipster dependencies to package.json.
--skip-jhipster-dependencies Don't write jhipster dependencies to package.json. (env: JHI_SKIP_JHIPSTER_DEPENDENCIES)
--creation-timestamp <value> Project creation timestamp (used for reproducible builds)
--jdl-store <value> JDL store
--prettier-tab-width <value> Default tab width for prettier
Expand Down Expand Up @@ -287,7 +287,7 @@ exports[`generator - app with default config should match snapshot 1`] = `
"defaultPackaging": "jar",
"devDatabaseExtraOptions": "",
"devDatabaseName": "jhipster",
"devDatabasePassword": "",
"devDatabasePassword": "password",
"devDatabaseType": "postgresql",
"devDatabaseTypeH2Any": false,
"devDatabaseTypeH2Disk": false,
Expand Down Expand Up @@ -741,7 +741,7 @@ exports[`generator - app with default config should match snapshot 1`] = `
},
"prodDatabaseExtraOptions": "",
"prodDatabaseName": "jhipster",
"prodDatabasePassword": "",
"prodDatabasePassword": "password",
"prodDatabaseType": "postgresql",
"prodDatabaseTypeMariadb": false,
"prodDatabaseTypeMssql": false,
Expand Down Expand Up @@ -901,7 +901,7 @@ exports[`generator - app with gateway should match snapshot 1`] = `
"defaultPackaging": "jar",
"devDatabaseExtraOptions": "",
"devDatabaseName": "jhipster",
"devDatabasePassword": "",
"devDatabasePassword": "password",
"devDatabaseType": "postgresql",
"devDatabaseTypeH2Any": false,
"devDatabaseTypeH2Disk": false,
Expand Down Expand Up @@ -1351,7 +1351,7 @@ exports[`generator - app with gateway should match snapshot 1`] = `
},
"prodDatabaseExtraOptions": "",
"prodDatabaseName": "jhipster",
"prodDatabasePassword": "",
"prodDatabasePassword": "password",
"prodDatabaseType": "postgresql",
"prodDatabaseTypeMariadb": false,
"prodDatabaseTypeMssql": false,
Expand Down Expand Up @@ -1510,7 +1510,7 @@ exports[`generator - app with microservice should match snapshot 1`] = `
"defaultPackaging": "jar",
"devDatabaseExtraOptions": "",
"devDatabaseName": "jhipster",
"devDatabasePassword": "",
"devDatabasePassword": "password",
"devDatabaseType": "postgresql",
"devDatabaseTypeH2Any": false,
"devDatabaseTypeH2Disk": false,
Expand Down Expand Up @@ -1907,7 +1907,7 @@ exports[`generator - app with microservice should match snapshot 1`] = `
},
"prodDatabaseExtraOptions": "",
"prodDatabaseName": "jhipster",
"prodDatabasePassword": "",
"prodDatabasePassword": "password",
"prodDatabaseType": "postgresql",
"prodDatabaseTypeMariadb": false,
"prodDatabaseTypeMssql": false,
Expand Down
7 changes: 7 additions & 0 deletions generators/base/support/secret.ts
Original file line number Diff line number Diff line change
Expand Up @@ -52,3 +52,10 @@ export function createBase64Secret(len?: number | boolean, reproducible = false)
}
return Buffer.from(createSecret(len)).toString('base64');
}

/**
* Create a strong secret from a timestamp and a base name
*/
export function createSafeSecret(timestamp: number, baseName: string) {
return Buffer.from(`${timestamp}-${baseName}`).toString('base64');
}
1 change: 1 addition & 0 deletions generators/bootstrap-application-base/command.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ const command = {
skipJhipsterDependencies: {
description: "Don't write jhipster dependencies to package.json.",
type: Boolean,
env: 'JHI_SKIP_JHIPSTER_DEPENDENCIES',
scope: 'storage',
},
creationTimestamp: {
Expand Down
4 changes: 4 additions & 0 deletions generators/client/command.ts
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,10 @@ const command = {
},
clientTestFrameworks: {
description: 'Client test frameworks',
cli: {
type: Array,
hide: true,
},
prompt: ({ jhipsterConfigWithDefaults: config }) => ({
when: answers => [ANGULAR, REACT, VUE].includes(answers.clientFramework ?? config.clientFramework),
type: 'checkbox',
Expand Down
6 changes: 5 additions & 1 deletion generators/common/__snapshots__/generator.spec.ts.snap
Original file line number Diff line number Diff line change
Expand Up @@ -152,8 +152,12 @@ sonar.exclusions = src/main/webapp/content/**/*.*, src/main/webapp/i18n/*.js, \\
target/classes/static/**/*.*

sonar.issue.ignore.multicriteria = \\
S1192,S125,S3437,S4502,S4684,S5145,UndocumentedApi
S6437,S1192,S125,S3437,S4502,S4684,S5145,UndocumentedApi

# Rule https://rules.sonarsource.com/java/RSPEC-6437 is ignored, hardcoded
# passwords are provided for development purposes
sonar.issue.ignore.multicriteria.S6437.resourceKey = src/main/resources/config/*
sonar.issue.ignore.multicriteria.S6437.ruleKey = java:S6437
# Rule https://rules.sonarsource.com/java/RSPEC-3437 is ignored, as a
# JPA-managed field cannot be transient
sonar.issue.ignore.multicriteria.S3437.resourceKey = src/main/java/**/*
Expand Down
Loading
Loading