Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Jenkins CI environment for bulk Expression Atlas in GCP k8s cluster #158

Draft
wants to merge 35 commits into
base: develop
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
2c3833e
Add configuration files for Jenkins CI pipeline - WIP
ke4 Jul 21, 2023
f80c92c
Add volumes for gxa-data and expdesign
ke4 Sep 29, 2023
c57c106
configured test execution with the correct solr properties
ke4 Sep 29, 2023
ae248c9
Fix experiment file and experiment design file locations
ke4 Sep 29, 2023
70d44dd
Temporarily remove 2 missing experiments
ke4 Sep 29, 2023
572a221
Update atlas-web-core submodule
alfonsomunozpomer Sep 29, 2023
981f40e
Change exp design volume to rox
ke4 Sep 29, 2023
461b754
Lower pod template resources
alfonsomunozpomer Sep 29, 2023
a243da5
Remove workspace volume
alfonsomunozpomer Sep 29, 2023
5b03e28
Revert workspace volume and use RWO PVC for experiment design
alfonsomunozpomer Sep 29, 2023
3b01b1f
Update mockito version to align with the test code usage
ke4 Oct 2, 2023
144f28e
Update jacoco test report config
ke4 Oct 2, 2023
6733dc3
Use ROX volume for experiment design files
alfonsomunozpomer Oct 2, 2023
0e1b276
Increase ephemeral storage
alfonsomunozpomer Oct 2, 2023
c163521
Increase Jenkins agent workspace volume
alfonsomunozpomer Oct 2, 2023
0c11832
Increase memory size of Gradle pod
alfonsomunozpomer Oct 2, 2023
5de460d
Remove stale imports
alfonsomunozpomer Oct 2, 2023
1541c82
Use methods from Experiment instead of ExperimentDesign for the heatm…
alfonsomunozpomer Oct 2, 2023
5b1b323
Merge remote-tracking branch 'origin/chore/add_ci_env' into chore/add…
alfonsomunozpomer Oct 2, 2023
97c88a7
Update atlas-web-core submodule
alfonsomunozpomer Oct 2, 2023
8e05a57
Use new experiment design-related logic from the Experiment class
alfonsomunozpomer Oct 2, 2023
cc9500e
Increase memory request for Gradle pod
alfonsomunozpomer Oct 2, 2023
adddc83
Fix compile errors due to missing experiment design in the experiment…
alfonsomunozpomer Oct 2, 2023
748910e
Remove tests and update experiment design-related dependencies
alfonsomunozpomer Oct 3, 2023
62ddd68
Fix stubs in test
alfonsomunozpomer Oct 3, 2023
f37052c
Remove logging iteration that consumes the iterator (!)
alfonsomunozpomer Oct 3, 2023
e6fb583
Remove stale logger
alfonsomunozpomer Oct 3, 2023
2b702c4
Fix parameters in solr's docker compose file
ke4 Oct 3, 2023
124ddc3
Add missing properties for local docker environment
ke4 Oct 3, 2023
4f4c998
Merge remote-tracking branch 'origin/chore/add_ci_env' into chore/add…
ke4 Oct 3, 2023
dd8e575
Fix stubbings of experiment object as they’re used by the client
alfonsomunozpomer Oct 3, 2023
a669030
Merge remote-tracking branch 'origin/chore/add_ci_env' into chore/add…
alfonsomunozpomer Oct 3, 2023
0a59eec
Implement missing interface
alfonsomunozpomer Oct 3, 2023
cdd1114
Add mount for ontology/antomical_systems.txt and related files
alfonsomunozpomer Oct 4, 2023
1c8ad45
Remove unneeded change dir
ke4 Oct 19, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Jenkinsfile
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ pipeline {
agent {
kubernetes {
cloud 'gke-autopilot'
workspaceVolume dynamicPVC(storageClassName: 'premium-rwo', accessModes: 'ReadWriteOnce', requestsSize: '5Gi')
workspaceVolume dynamicPVC(storageClassName: 'premium-rwo', accessModes: 'ReadWriteOnce', requestsSize: '6Gi')
defaultContainer 'openjdk'
yamlFile 'jenkins-k8s-pod.yaml'
}
Expand Down
18 changes: 9 additions & 9 deletions buildSrc/src/main/groovy/atlas-web-app.java-conventions.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -64,8 +64,8 @@ dependencies {
testImplementation 'org.springframework:spring-test:5.1.5.RELEASE'
testImplementation 'com.h2database:h2:1.4.199'
testImplementation 'org.junit.jupiter:junit-jupiter-params:5.4.1'
testImplementation 'org.mockito:mockito-core:2.25.1'
testImplementation 'org.mockito:mockito-junit-jupiter:2.25.1'
testImplementation 'org.mockito:mockito-core:5.2.0'
testImplementation 'org.mockito:mockito-junit-jupiter:5.2.0'
testImplementation 'org.assertj:assertj-core:3.11.1'
testImplementation 'org.assertj:assertj-guava:3.2.0'
testImplementation 'org.hamcrest:java-hamcrest:2.0.0.0'
Expand Down Expand Up @@ -121,19 +121,19 @@ test {
jvmArgs '-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=*:5005'
}

if (project.hasProperty('excludeTests')) {
exclude project.property('excludeTests')
}

if (project.hasProperty('testResultsPath')) {
reporting.baseDir = "${buildDir}/reports/" + project.property('testResultsPath')
testResultsDirName = "${buildDir}/test-results/" + project.property('testResultsPath')
testReportDirName = project.property('testResultsPath')
testResultsDirName = project.property('testResultsPath')

jacoco {
destinationFile = file("${buildDir}/jacoco/${project.property('testResultsPath')}.exec")
}
}

if (project.hasProperty('excludeTests')) {
exclude project.property('excludeTests')
}

testLogging {
outputs.upToDateWhen {false}
showStandardStreams = true
Expand All @@ -148,7 +148,7 @@ jacocoTestReport {
executionData.setFrom fileTree(dir: "${buildDir}/jacoco", include: '*.exec' )

reports {
html.destination file("${buildDir}/jacoco/html")
html.outputLocation = layout.buildDirectory.dir("reports/jacoco")
}
}

8 changes: 4 additions & 4 deletions jenkins-k8s-pod.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,10 @@ spec:
image: openjdk:11
resources:
requests:
memory: "1.5Gi"
ephemeral-storage: "4Gi"
memory: "2.5Gi"
ephemeral-storage: "6Gi"
limits:
memory: "3Gi"
memory: "4Gi"
env:
- name: GRADLE_OPTS
value: "-Dorg.gradle.daemon=false"
Expand Down Expand Up @@ -76,7 +76,7 @@ spec:
readOnly: true
- name: gxa-expdesign
persistentVolumeClaim:
claimName: gxa-expdesign-rwo
claimName: gxa-expdesign-rox
readOnly: true
- name: bioentity-properties
persistentVolumeClaim:
Expand Down
Loading