Test and document binarySource=docker with Renovate image

[rarkins]

Need to map Docker sock into container first so that other containers can be run as siblings. It should work for composer, pipenv, go modules. Then document in the self hosting doc.

[rarkins]

This works for me on Mac: docker run --rm --user $UID -w pwd-vpwd:pwd -v /tmp:/tmp -v /var/run/docker.sock:/var/run/docker.sock renovate/renovate:slim owner/repo. Authentication is contained within a config.js in pwd but could also be passed via env.

It's essential that /tmp:/tmp is a mirror, e.g. /my/tmp:/tmp would not work.

[rarkins]

Cc @viceice

[viceice]

We should document some possibillities.

I'll try to create a rootless-dind sample. It will be a side container to renovate.

[rarkins]

Wouldn't the ideal solution be that you run Renovate (let's call it renovate/renovate:dind for now) and it can spawn "child" containers for renovate/go, renovate/pipenv, etc? i.e. no more side containers?

[viceice]

Yes, so we extend docker:stable-dind-rootless and add our specific stuff.

Bu we have to remember that dind-rootless currently needs --privileged mode.

[viceice]

But the daemon inside dind-rootless is running as user rootless. So this might be a feasible solution.

https://github.com/docker-library/docker/blob/c01ffa41486f70c34f020c769bc5bddf106367ea/19.03/dind-rootless/Dockerfile#L73

[rarkins]

Yes, I thought the idea was that the process inside the dind container gets all the privileges of the user running it on the host, but can't break out to root. So don't leave any privileged info within the user account on the host.

[rarkins]

It would be pretty awesome to have a slim Docker image of Renovate that can span go/composer/pipenv/etc within child containers.

[rarkins]

Seems to be based off alpine though, but shouldn't be too hard. I hope we can keep the child containers all as ubuntu-based.

[viceice]

we can use the https://get.docker.com/rootless script to install local rootless docker.

I'll try that, we can start the daemon from renovate because renovate is the entrypoint.

We can check for RENOVATE_BINARY_SOURCE=docker to start the daemon

[rarkins]

Do you mean you run the renovate slim image as a "regular" Docker image but then install run rootless Docker inside renovate slim?

[viceice]

maybe? or just start the dind side container manually?

[rarkins]

It's really nice to make them child containers rather than siblings so you don't have to worry about host and all containers having the same directory structure.

[viceice]

then we need to run the docker daemon from renovate prior to start any containers

[viceice]

start as child is more difficult. i'll se if i get that managed

[rarkins]

Let's assume that we discontinue the embedding of npm/yarn/pnpm as renovate dependencies and as a result there's a 90%+ likelihood that Renovate users will need child containers after that.

In that scenario:

• the slim container should be built with rootless docker preinstalled, because it would be needed in 90%+ of users anyway
• the entrypoint of this renovate slim image needs to start docker at the start of each run
• renovate will spawn child containers, and be oblivious that it is running inside a "normal" Docker container itself
• once Renovate is done, it might need to tear down rootless docker to exit cleanly?

[rarkins]

Is another possible option that we say you need to run renovate with --privileged and recommend that people do that with rootless docker instead of regular docker? i.e. that way the Renovate container doesn't need to be running a Docker daemon of its own.

[viceice]

thats a possibillity.

Currently experimenting with podman as a drop-in replacement for docker. Here we don't need a daemon.

[viceice]

Simply run renovate in renovate 🙈

[viceice]

Problem

Docker in Docker or Podman in Docker is pretty slow. Mostly this is an IO bottleneck.

Idea

• Add a volume for the Podman / Docker data to retain images and container.
• pull image and start tool container ( with cat command) on first run (when required)
• pull image on next run, if updated recreate tool container and prune obsolete images in background
• use docker exec for running tool

I hope this idea is understandable 😅

[micheelengronne]

I would also add the kubernetes usecase. Renovate integrated in a kube environment should delegate the container management to kube itself (in particular if kube has another runtime than docker like CRI-O).

[rarkins]

@micheelengronne it would be helpful if you can create an issue describing those requirements in detail.

[micheelengronne]

Ok, https://github.com/renovatebot/renovate/issues/6121

[rarkins]

Superseded by #6364