Feat-multi-arch-containers #5533

duncdrum · 2024-11-02T12:23:24Z

enables a long overdue feature: multi-arch containers for existdb/existdb
backport will remove a release blocker for 6.3.1

sidessteps docker maven plugin bug, deploy is green again.
see fabric8io/docker-maven-plugin#1835

fixes outdated JRE patches, bumps base image to Debian 12
see #5450
close #4151
close #5532
close #4153

dependencies are part of distroless base image enable experimental image push for testing

see eXist-db#5450 close eXist-db#4151 close eXist-db#5532 close eXist-db#4153

duncdrum · 2024-11-12T12:21:05Z

The codacy issue here is pointless. We are trying to generate moving tag latest images. So they are based on latest base images from upstream on purpose

duncdrum · 2024-11-12T14:54:44Z

Deploy in my fork:

images tested locally, all good.

dizzzz · 2024-11-12T18:53:23Z

Ubuntu build fails, any idea ?
Don't have a laptop around to check the logs ...

duncdrum · 2024-11-12T20:11:52Z

@dizzzz some timeout with building distribution packages, I restarted it. Pretty sure it's unrelated to the changes in here.

line-o

@duncdrum is this good to go? Should we wait for the back port to 6.x.x?

duncdrum · 2024-11-18T19:45:37Z

@line-o this is good to go. There are problems with the back port, but I m 99,9% certain that these will not affect the contents of this PR, which also fixes our broken ci.

line-o · 2024-11-18T19:52:52Z

@duncdrum would you have the time to edit the commit message of edfc10e

to

[bugfix] workaround to deploy images to dockerhub

fixes #5450

sidesteps [docker maven plugin bug](https://github.com/fabric8io/docker-maven-plugin/issues/1835)

If you do not have time I would volunteer to do this.

duncdrum · 2024-11-18T21:03:31Z

@line-o go ahead I ll be afk a lot this week. Thanks for the help

fixes eXist-db#5450 see [docker maven plugin bug](fabric8io/docker-maven-plugin#1835)

sonarcloud · 2024-11-19T12:18:56Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

line-o

LGTM

adamretter · 2024-11-19T13:53:50Z

exist-docker/src/main/resources-filtered/Dockerfile

-
-FROM gcr.io/distroless/java17:latest
-
-# Copy over dependencies for Apache FOP, missing from GCR's JRE


This change breaks XSL:FO in eXist-db when using Apache FOP. Unfortunately, the gcr.io/distroless/java17-debian12:latest image you have chosen to update to does not include these libraries. Please test a proper XSL-FO to PDF transformation with these changes, and you will see the error.

This PR switches from gcr.io/distroless/java17:latest to gcr.io/distroless/java17-debian12:latest
The lines stating they would be needed for FOP are commented out (lines 24-27 and 32-41) and select a different base image debian:bullseye-slim.
So, even if those additional libraries were needed for PDF-generation using XSL:FO they are already missing and nothing is changed from the status quo.
It, therefore, would be beneficial to create an additional Dockerfile that has all that is needed for this specific use-case. This could also be provided by a third party.
This is all just under the pretense that this is really needed for PDF generation and the adoption rate would also show us how common it is.

@line-o It looks like they are only commented out because they were recently done so in a prior Pull Request that looks like it broke this.

I pointed out the problems in #4895 would break Apache FOP here, here, and here.

The response was that the base image now contains the required libraries, however, I have now checked the base image, and it is clear that that is not the case. See below:

We can see in the above image that /usr/lib inside the base image does not contain the previously required libraries.

If they are no longer included that's a bug likely introduced when moving to debian 12. see GoogleContainerTools/distroless#321 or GoogleContainerTools/distroless#789 from 2019 and 2021 for relevant discussions. I will check the java-base image and move the discussion to the distroless repo

I don't see it as a bug with Debian 12 or GCR. Simply, GCR has never included these libraries as many Java Applications don't need them. However, we do need them for eXist-db, which is why previously we explicitly installed them into a base Debian image and then copied them into our derivative of GCR.

@duncdrum I can't see it in the GCR java-base images (see below)... Where are you seeing it?

@adamretter https://github.com/GoogleContainerTools/distroless/blob/main/java/BUILD#L117 here actually. I also remember and tested that the files were present after the discussions 3-4 years ago. If they no longer are it's a bug. Only libssl should no longer be included on Debian 12 Java images. I won't have time to look into this for another week or so. But I will check again. In the meantime I don't see how keeping a failing deploy tank running is helping. More fixes should come in a separate PR.

@duncdrum Interesting that the files even though listed in that BUILD file don't seem to have ever been in the published GCR images as far as I can see.

Just to be sure @adamretter You are checking in subsequent layers as well, and not seeing the dependencies?

You are checking in subsequent layers as well, and not seeing the dependencies?

@duncdrum Yes, I think so. What do you see when you look in the image?

adamretter · 2024-11-19T13:55:09Z

exist-docker/src/main/resources-filtered/Dockerfile-DEBUG

@@ -21,7 +21,7 @@
 #

 # Use JDK 17 in Debian Bullseye (as our production image gcr.io/distroless/java:17 is based on Debian Bullseye with just a JRE)


I think this comment is out of date if you are switching the non-debug build to bookworm also.

Yeah, better remove the name of a specific build of Debian from the comment, otherwise we will need to change it everytime the base image changes

exist-docker/src/main/resources-filtered/Dockerfile

[refactor] remove dead code

e3e3e85

dependencies are part of distroless base image enable experimental image push for testing

duncdrum added the docker issues related to our container images label Nov 2, 2024

duncdrum added this to the eXist-6.3.1 milestone Nov 2, 2024

duncdrum added the needs 6.x.x backport label Nov 2, 2024

[feature] add multi-arch images

75981d8

see eXist-db#5450 close eXist-db#4151 close eXist-db#5532 close eXist-db#4153

duncdrum force-pushed the feat-multi-arch-containers branch from e453994 to 75981d8 Compare November 2, 2024 12:35

[chore] bump base image

9bc97a1

duncdrum force-pushed the feat-multi-arch-containers branch 2 times, most recently from 44ac3cd to edfc10e Compare November 12, 2024 13:03

duncdrum marked this pull request as ready for review November 12, 2024 14:54

duncdrum requested a review from a team as a code owner November 12, 2024 14:54

line-o assigned duncdrum Nov 12, 2024

reinhapa approved these changes Nov 13, 2024

View reviewed changes

line-o reviewed Nov 18, 2024

View reviewed changes

duncdrum added 3 commits November 19, 2024 12:47

[bugfix] workaround docker maven plugin issue

bd809c9

fixes eXist-db#5450 see [docker maven plugin bug](fabric8io/docker-maven-plugin#1835)

[refactor] LegacyKeyValueFormat

5cf787a

[chore] no more experimental images

9b1a3cf

line-o mentioned this pull request Nov 19, 2024

Feature: deploy containers for x86 and arm #5561

Closed

line-o force-pushed the feat-multi-arch-containers branch from 6f7fff4 to 9b1a3cf Compare November 19, 2024 11:56

line-o self-requested a review November 19, 2024 12:29

line-o approved these changes Nov 19, 2024

View reviewed changes

adamretter suggested changes Nov 19, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feat-multi-arch-containers #5533

Feat-multi-arch-containers #5533

duncdrum commented Nov 2, 2024 •

edited

Loading

duncdrum commented Nov 12, 2024

duncdrum commented Nov 12, 2024

dizzzz commented Nov 12, 2024

duncdrum commented Nov 12, 2024

line-o left a comment

duncdrum commented Nov 18, 2024

line-o commented Nov 18, 2024 •

edited

Loading

duncdrum commented Nov 18, 2024

sonarcloud bot commented Nov 19, 2024

line-o left a comment

adamretter Nov 19, 2024 •

edited

Loading

line-o Nov 21, 2024

adamretter Nov 21, 2024

duncdrum Nov 21, 2024

adamretter Nov 21, 2024 •

edited

Loading

adamretter Nov 21, 2024

duncdrum Nov 21, 2024

adamretter Nov 21, 2024 •

edited

Loading

duncdrum Nov 22, 2024

adamretter Nov 23, 2024

adamretter Nov 19, 2024

line-o Nov 21, 2024


		FROM gcr.io/distroless/java17:latest

		# Copy over dependencies for Apache FOP, missing from GCR's JRE

		@@ -21,7 +21,7 @@
		#

		# Use JDK 17 in Debian Bullseye (as our production image gcr.io/distroless/java:17 is based on Debian Bullseye with just a JRE)

Feat-multi-arch-containers #5533

Are you sure you want to change the base?

Feat-multi-arch-containers #5533

Conversation

duncdrum commented Nov 2, 2024 • edited Loading

duncdrum commented Nov 12, 2024

duncdrum commented Nov 12, 2024

dizzzz commented Nov 12, 2024

duncdrum commented Nov 12, 2024

line-o left a comment

Choose a reason for hiding this comment

duncdrum commented Nov 18, 2024

line-o commented Nov 18, 2024 • edited Loading

duncdrum commented Nov 18, 2024

sonarcloud bot commented Nov 19, 2024

Quality Gate passed

line-o left a comment

Choose a reason for hiding this comment

adamretter Nov 19, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adamretter Nov 21, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

adamretter Nov 21, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

duncdrum commented Nov 2, 2024 •

edited

Loading

line-o commented Nov 18, 2024 •

edited

Loading

adamretter Nov 19, 2024 •

edited

Loading

adamretter Nov 21, 2024 •

edited

Loading

adamretter Nov 21, 2024 •

edited

Loading