Processes in container run as root

[andreeaflorescu]

Running as root is unfortunately this is the default configuration when running a container. The simplest way to make the processes to run as something else is to provide a UID when running the container:
docker run --user 100

The problem is that the Rust toolchain is installed per user and as of now there is no official way of installing it system-wide (at least I didn't find one). Some more details in this issue: rust-lang/rustup#313

We need to find a way around this that is not adding complexity to running the container (like creating a user only for running the container) because this container is used by both automated systems (like Buildkite), but also by developers who want to test their code on their local machines.

[michael2012z]

Hi, @andreeaflorescu

Why is it a problem to run the container as root when it "is used by both automated systems (like Buildkite), but also by developers who want to test their code on their local machines"? And I haven't been able to connect the problem with the per-user Rust toolchain installation.

Do you mind explaining a bit more? :)

[andreeaflorescu]

One of the main pain points when running inside the container as root is that the build system leaves files on the disk with user root. To clean them up you need to be root (or the major hack that I did which was to run the container again just to clean the files). For example when running the container with Buildkite I would like to specify the user to be buildkite-agent so that the agent has permissions to remove any leftover files from previous runs. Same when running on a dev machine. I am not sure if this is possible, but that's what I had in mind.

[michael2012z]

Other projects also have similar trouble. In Moby, their workaround is almost as same as your hack. They change the ownership of generated files: docker run --rm -v /home/ubuntu/workspace/moby_PR-40394:/workspace busybox chown -R 1000:1000 /workspace

Maybe we can do "chown" in a different way (assuming I am now in the folder of a rust-vmm component and want to start a container to build and test):
docker run --device=/dev/kvm -it --security-opt seccomp=unconfined --volume $(pwd):/workspace rustvmm/dev:v3 bash -c "bash ; chown -R $(id -u):$(id -g) /workspace/target"
(Add a HOOK to chown after the bash exits. The generated files belong to the user after the container exits.)

Does it help?

[rn]

This assumes that you use volumes to get the source code into the container and that you build the artefacts directly in the volume. I don't know buildkite but maybe it can be set up differently.

[andreeaflorescu]

@rn I think that is the default behavior of the docker plugin in Buildkite. What is the recommend way to do this?

[rn]

I've never used buildkite so can't really comment. I had a quick look at https://github.com/buildkite-plugins/docker-buildkite-plugin and one options might be perform the artefact build as part of a docker build and extract the build result with a docker run -v. Do you have example buildkite configs to look at?

[andreeaflorescu]

@rn rust-vmm-ci has the common buildkite pipeline we are using. We don't configure anything related to docker, we just specify what docker container we want to use for running the tests.

I was wondering what is the best practice when using containers so that you don't end up with files created by root.