Use Chainguard images for build and runtime

[dallemon]

this changes the base images to chainguard based ones
reduced the layers by moving most copy to a separate container

unsure how to replicate the issue mentioned here #2071 (comment)
however using the wolfi glibc image for runtime might work around it?

also updated my email :)

[ghost]

Edit: Seems like I misunderstood the purpose. My mistake.

[dallemon]

Sure, however the README.md would need a thorough updating to support it. :)

This PR simply uses some more secure base images without compromising the current functionality and with minimal changes.

[slingamn]

I'm not familiar with Chainguard or their governance structure. Could you say more about the security advantages with respect to our current base images?

[slingamn]

FWIW I can reproduce the faccessat2(2) issue using Docker version 19.03.6, build 369ce74a3c on Linux 5.4.x (Ubuntu 20.04). It sounds like that system call was added in Linux 5.8.

[slingamn]

Just curious, any reason to prefer glibc over musl here? It sounds like musl is the default: https://edu.chainguard.dev/chainguard/chainguard-images/reference/busybox/#image-variants

[dallemon]

Potential workaround for your issue with faccessat2(2) as I figured maybe it was an alpine/musl related issue and the default latest tag uses alpine and musl as a base instead of wolfi and glibc.

However if you can still reproduce it with this PR then it would seem it's not a very good workaround :)

[slingamn]

Oh, my mistake --- I reproduced the issue using Alpine, not this branch. I'm no longer very concerned about the issue, since Docker 20.10 should have an effective workaround (discussion on #2103)

[dallemon]

I'm not familiar with Chainguard or their governance structure. Could you say more about the security advantages with respect to our current base images?

So the obvious one is significantly less CVE's in their base images due to how they daily/regularly update everything.
Easily verifiable by using a vulnerability scanner.

The tools they've developed, melange and apko, also make it possible to create reproducible OCI images, something Docker still can't do, and by default also give you an SBOM when building software.

I've been following the project for a while now and it's growing fast.
I would recommend you checkout some of their blogposts or whitepapers. :)

[slingamn]

Nice, I didn't realize that Chainguard are the people who make Distroless :-)

That said, a lot of this feels like an example of the streetlight problem. In Chainguard's blog posts, they keep emphasizing that they're a good way to cut down on vulnerability scanner noise. Snyk does report a long list of vulnerabilities in Alpine 3.13 but in fact none of these are part of the Ergo container's attack surface, since Ergo does not invoke busybox or any userland native libraries. As far as we're concerned, these are all false positives. So if we were using Snyk, switching to Chainguard would make our lives easier, but we can also just not use Snyk :-)

Moreover, even if we were concerned about these vulnerabilities, switching to Chainguard wouldn't solve the problem by itself: we would also have to adjust our current release cycle to cut a new stable release of the Docker image every time that Chainguard updates.

IMO the real concern is not vulnerabilities in the Alpine userland, but compromise of either Docker Hub or cgr.dev that replaces the base image itself with malware. The correct mitigation here would either be to pin the hash of the base image, or use some kind of public-key signature verification workflow to validate the latest image. (Does Chainguard have such a workflow? I'm imagining a setup where we could hardcode their public key in the Dockerfile and require a successful verification of the base image's signature during each build.)

[dallemon]

i added pinned versions and a dependabot action that should update the sha256 when a new one is available.

[slingamn]

Sorry for the delay here. I think I'm going to merge #2104 instead of this for now, and reevaluate Chainguard later. The main reasons are (1) as mentioned previously, I don't think Busybox CVEs are actually exposed as part of Ergo's attack surface (2) for now I'm more comfortable with a community-governed project than a corporate-governed one.

[dallemon]

i will close this pr then as there is no interrest :)