Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inline Containerfile generation #2119

Merged
merged 1 commit into from
Oct 1, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -65,8 +65,8 @@ dev-image:
echo "ERROR: QUAY_USERNAME is not set"; \
exit 1; \
fi
docker build . -t quay.io/$(QUAY_USERNAME)/hacbs-jvm-controller:dev
docker push quay.io/$(QUAY_USERNAME)/hacbs-jvm-controller:dev
docker build . -t quay.io/$(QUAY_USERNAME)/hacbs-jvm-controller:"$${JBS_QUAY_IMAGE_TAG:-dev}"
docker push quay.io/$(QUAY_USERNAME)/hacbs-jvm-controller:"$${JBS_QUAY_IMAGE_TAG:-dev}"
Copy link
Collaborator Author

@rnc rnc Sep 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless the JBS_QUAY_IMAGE_TAG is specified we default to dev. With Tekton pipelines, if tag is latest the it will use PullAlways. Hence for pipelines setting JBS_QUAY_IMAGE_TAG=latest is advised. We should consider whether to change the default to latest (perhaps in another iteration).


dev: dev-image
cd java-components && mvn clean install -Dlocal -DskipTests -Ddev
Expand Down
9 changes: 6 additions & 3 deletions deploy/tasks/pre-build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -46,10 +46,13 @@ spec:
type: string
- name: RECIPE_IMAGE
description: The image from the build recipe to use
- name: BUILD_TOOL
description: The build tool to use.
- name: BUILD_PLUGINS
description: Comma separated list of build plugins that should be disabled.
default: ""
- name: BUILD_SCRIPT
description: The build script to embed with the Containerfile
- name: PREPROCESSOR_ARGS
description: The arguments for the build preprocessor
- name: ORAS_OPTIONS
type: string
description: Optional environment variable string for build-trusted-artifacts
Expand Down Expand Up @@ -103,7 +106,7 @@ spec:
memory: 512Mi
script: |
$(params.BUILD_SCRIPT)
/opt/jboss/container/java/run/run-java.sh $(params.PREPROCESSOR_ARGS)
/opt/jboss/container/java/run/run-java.sh $(params.BUILD_TOOL)-prepare $(workspaces.source.path)/source --recipe-image=$(params.RECIPE_IMAGE) --request-processor-image=$(params.JVM_BUILD_SERVICE_REQPROCESSOR_IMAGE) --disabled-plugins=$(params.BUILD_PLUGINS)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This completely inlines the request processor arguments (which makes it easier for PNC to use)

- name: create-pre-build-source
image: $(params.JVM_BUILD_SERVICE_REQPROCESSOR_IMAGE)
securityContext:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,21 +1,86 @@
package com.redhat.hacbs.container.build.preprocessor;

import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.List;

import io.quarkus.logging.Log;
import picocli.CommandLine;

/**
* We keep all the options the same between maven and gradle for now,
* to keep the pipeline setup simpler.
*
* We keep all the options the same between maven, gradle, sbt and ant for now to keep the pipeline setup simpler.
* Some of these may be ignored by different processors
*/
public abstract class AbstractPreprocessor implements Runnable {

@CommandLine.Parameters(description = "The directory to process")
protected Path buildRoot;

@CommandLine.Option(names = { "-dp", "--disable-plugin" }, paramLabel = "<plugin>", description = "The plugin to disable")
@CommandLine.Option(names = { "-dp", "--disabled-plugins" }, paramLabel = "<plugin>", description = "The plugin to disable", split=",")
protected List<String> disabledPlugins;

@CommandLine.Option(names = "--recipe-image", required = true)
String recipeImage;

@CommandLine.Option(names = "--request-processor-image", required = true)
String buildRequestProcessorImage;

protected enum ToolType {
ANT,
GRADLE,
MAVEN,
SBT
}

protected ToolType type;

@Override
public void run() {
Path jbsDirectory = Path.of(buildRoot.toString(), ".jbs");
//noinspection ResultOfMethodCallIgnored
jbsDirectory.toFile().mkdirs();

String containerFile = """
FROM %s
USER 0
WORKDIR /var/workdir
RUN mkdir -p /var/workdir/software/settings /original-content/marker
ARG CACHE_URL=""
ENV CACHE_URL=$CACHE_URL
COPY .jbs/run-build.sh /var/workdir
COPY . /var/workdir/workspace/source/
RUN /var/workdir/run-build.sh
""".formatted(recipeImage);

// TODO: This is a bit of a hack but as Ant doesn't deploy and the previous implementation relied upon using the
// BuildRequestProcessorImage we need to modify the Containerfile. In future the ant-build.sh should probably
// encapsulate this.
if (type == ToolType.ANT) {
// Don't think we need to mess with keystore as copy-artifacts is simply calling copy commands.
containerFile +=

Check warning on line 62 in java-components/build-request-processor/src/main/java/com/redhat/hacbs/container/build/preprocessor/AbstractPreprocessor.java

View check run for this annotation

Codecov / codecov/patch

java-components/build-request-processor/src/main/java/com/redhat/hacbs/container/build/preprocessor/AbstractPreprocessor.java#L62

Added line #L62 was not covered by tests
"""
FROM %s AS build-request-processor
USER 0
WORKDIR /var/workdir
COPY --from=0 /var/workdir/ /var/workdir/
RUN /opt/jboss/container/java/run/run-java.sh copy-artifacts --source-path=/var/workdir/workspace/source --deploy-path=/var/workdir/workspace/artifacts
FROM scratch
COPY --from=1 /var/workdir/workspace/artifacts /
""".formatted(buildRequestProcessorImage);

Check warning on line 71 in java-components/build-request-processor/src/main/java/com/redhat/hacbs/container/build/preprocessor/AbstractPreprocessor.java

View check run for this annotation

Codecov / codecov/patch

java-components/build-request-processor/src/main/java/com/redhat/hacbs/container/build/preprocessor/AbstractPreprocessor.java#L71

Added line #L71 was not covered by tests
} else {
containerFile +=
"""
FROM scratch
COPY --from=0 /var/workdir/workspace/artifacts /
""";
}
try {
Files.writeString(Paths.get(jbsDirectory.toString(), "Containerfile"), containerFile);
} catch (IOException e) {
Log.errorf("Unable to write Containerfile", e);
throw new RuntimeException(e);

Check warning on line 83 in java-components/build-request-processor/src/main/java/com/redhat/hacbs/container/build/preprocessor/AbstractPreprocessor.java

View check run for this annotation

Codecov / codecov/patch

java-components/build-request-processor/src/main/java/com/redhat/hacbs/container/build/preprocessor/AbstractPreprocessor.java#L81-L83

Added lines #L81 - L83 were not covered by tests
}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This inlines the Containerfile generation into the source directory for the next steps (the git archiving and image generation) in the task to use.

}
}
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,13 @@
*/
@CommandLine.Command(name = "ant-prepare")
public class AntPrepareCommand extends AbstractPreprocessor {

public AntPrepareCommand() {
type = ToolType.ANT;
}

Check warning on line 17 in java-components/build-request-processor/src/main/java/com/redhat/hacbs/container/build/preprocessor/ant/AntPrepareCommand.java

View check run for this annotation

Codecov / codecov/patch

java-components/build-request-processor/src/main/java/com/redhat/hacbs/container/build/preprocessor/ant/AntPrepareCommand.java#L15-L17

Added lines #L15 - L17 were not covered by tests

@Override
public void run() {

super.run();

Check warning on line 21 in java-components/build-request-processor/src/main/java/com/redhat/hacbs/container/build/preprocessor/ant/AntPrepareCommand.java

View check run for this annotation

Codecov / codecov/patch

java-components/build-request-processor/src/main/java/com/redhat/hacbs/container/build/preprocessor/ant/AntPrepareCommand.java#L21

Added line #L21 was not covered by tests
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,14 @@ public class GradlePrepareCommand extends AbstractPreprocessor {
"version.gradle"
};

public GradlePrepareCommand() {
type = ToolType.GRADLE;
}

@Override
public void run() {
try {
super.run();
setupInitScripts();
Files.walkFileTree(buildRoot, new SimpleFileVisitor<>() {

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,13 @@
@CommandLine.Command(name = "maven-prepare")
public class MavenPrepareCommand extends AbstractPreprocessor {

public MavenPrepareCommand() {
type = ToolType.MAVEN;
}

@Override
public void run() {
super.run();
try {
Files.walkFileTree(buildRoot, new SimpleFileVisitor<>() {

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,12 @@
@CommandLine.Command(name = "sbt-prepare")
public class SBTPrepareCommand extends AbstractPreprocessor {

public SBTPrepareCommand() {
type = ToolType.SBT;
}

Check warning on line 15 in java-components/build-request-processor/src/main/java/com/redhat/hacbs/container/build/preprocessor/sbt/SBTPrepareCommand.java

View check run for this annotation

Codecov / codecov/patch

java-components/build-request-processor/src/main/java/com/redhat/hacbs/container/build/preprocessor/sbt/SBTPrepareCommand.java#L13-L15

Added lines #L13 - L15 were not covered by tests

@Override
public void run() {
super.run();

Check warning on line 19 in java-components/build-request-processor/src/main/java/com/redhat/hacbs/container/build/preprocessor/sbt/SBTPrepareCommand.java

View check run for this annotation

Codecov / codecov/patch

java-components/build-request-processor/src/main/java/com/redhat/hacbs/container/build/preprocessor/sbt/SBTPrepareCommand.java#L19

Added line #L19 was not covered by tests
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ public List<String> getCommand() {
var args = buildInfoLocator.lookupDisabledPlugins(GRADLE);
var command = new ArrayList<String>(1 + 2 * args.size());
command.add("gradle-prepare");
command.add("--recipe-image=foobar");
command.add("--request-processor-image=barfoo");
args.forEach(arg -> {
command.add("-dp");
command.add(arg);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@ public List<String> getCommand() {
var args = buildInfoLocator.lookupDisabledPlugins(MAVEN);
var command = new ArrayList<String>(1 + 2 * args.size());
command.add("maven-prepare");
command.add("--recipe-image=foobar");
command.add("--request-processor-image=barfoo");
args.forEach(arg -> {
command.add("-dp");
command.add(arg);
Expand Down
17 changes: 15 additions & 2 deletions java-components/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,8 @@
<quarkus-quinoa.version>2.4.7</quarkus-quinoa.version>

<format.skip>false</format.skip>

<imageTag>dev</imageTag>
</properties>

<modules>
Expand Down Expand Up @@ -452,7 +454,7 @@
<configuration>
<systemProperties>
<quarkus.container-image.group>${env.QUAY_USERNAME}</quarkus.container-image.group>
<quarkus.container-image.tag>dev</quarkus.container-image.tag>
<quarkus.container-image.tag>${imageTag}</quarkus.container-image.tag>
<quarkus.container-image.build>true</quarkus.container-image.build>
<quarkus.container-image.push>true</quarkus.container-image.push>
<quarkus.package.type>mutable-jar</quarkus.package.type>
Expand Down Expand Up @@ -611,5 +613,16 @@
<module>management-console</module>
</modules>
</profile>
</profiles>
<profile>
<activation>
<activeByDefault>true</activeByDefault>
<property>
<name>env.JBS_QUAY_IMAGE_TAG</name>
</property>
</activation>
<properties>
<imageTag>${env.JBS_QUAY_IMAGE_TAG}</imageTag>
</properties>
</profile>
</profiles>
</project>
Loading
Loading