image build: layer caching

[eguzki]

Trying to reduce the image building time, enable docker layer caching

[eguzki]

First build: https://github.com/Kuadrant/limitador/actions/runs/6315380975/job/17147791684 3hours

Second build (where cache should be using) https://github.com/Kuadrant/limitador/actions/runs/6318070335/job/17156232704 4 hours

looking at the logs every single layer is being built again

Furthermore, I learned, caching was introduced in buildah 1.27.0 https://github.com/containers/buildah/releases/tag/v1.27.0

and redhat-actions/buildah-build is using 1.23

/usr/bin/buildah version
  /usr/bin/buildah version
  Version:         1.23.1

redhat-actions/buildah-build#90

What I do not understand is then what is the layers input https://github.com/redhat-actions/buildah-build#action-inputs ?? it says literally: Set to true to cache intermediate layers during the build process.

Next try: Move all images builds to docker/build-and-push action and use the gha cache and only COPY the necessary things into the Dockerfile that are needed for each build (since if the payload is the EXACTLY the same then the layer will be re-used from cache)