Docker Desktop's engine won't start while Virtualbox is in using

[dhay3]

Description

Hi Docker Desktop maintainers and expert users,

I've encountered a problem with Docker Desktop's engine, might need some helps.

Docker Desktop's engine won't start while Virtualbox is in using(one VM is running, or one VM is shutdown but the GUI stays), vice versa. But the default docker context installed by package manager is working as expected when Virtualbox is in using.

running engine: waiting for the Docker API: engine linux/qemu failed to run: running VM: qemu: process terminated unexpectedly: exit status 1

I've do all the stuffs mentioned in general-system-requirements section. And I do find some related comments eg. #80 (comment), #162 (comment). But, there is no distinct reason why that Docker Desktop cannot be used with Virtualbox or any resolutions. If I have not missed something.

Any hints would be highly appreciated.

Best regards

Reproduce

1. Open Virtualbox GUI and power on one machine or use vboxmanage startvm
2. Run Docker Desktop

Expected behavior

Docker Desktop should be work when Virtualbox is in using as the default context does.

docker version

Client:
Version: 27.2.1
API version: 1.47
Go version: go1.23.1
Git commit: 9e34c9bb39
Built: Fri Sep 13 21:18:02 2024
OS/Arch: linux/amd64
Context: desktop-linux
Cannot connect to the Docker daemon at unix:///home/cc/.docker/desktop/docker.sock. Is the docker daemon running?

docker info

Client:
Version: 27.2.1
Context: desktop-linux
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.16.2-desktop.1
Path: /usr/lib/docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.29.2-desktop.2
Path: /usr/lib/docker/cli-plugins/docker-compose
debug: Get a shell into any image or container (Docker Inc.)
Version: 0.0.34
Path: /usr/lib/docker/cli-plugins/docker-debug
desktop: Docker Desktop commands (Alpha) (Docker Inc.)
Version: v0.0.15
Path: /usr/lib/docker/cli-plugins/docker-desktop
dev: Docker Dev Environments (Docker Inc.)
Version: v0.1.2
Path: /usr/lib/docker/cli-plugins/docker-dev
extension: Manages Docker extensions (Docker Inc.)
Version: v0.2.25
Path: /usr/lib/docker/cli-plugins/docker-extension
feedback: Provide feedback, right in your terminal! (Docker Inc.)
Version: v1.0.5
Path: /usr/lib/docker/cli-plugins/docker-feedback
init: Creates Docker-related starter files for your project (Docker Inc.)
Version: v1.3.0
Path: /usr/lib/docker/cli-plugins/docker-init
sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
Version: 0.6.0
Path: /usr/lib/docker/cli-plugins/docker-sbom
scout: Docker Scout (Docker Inc.)
Version: v1.13.0
Path: /usr/lib/docker/cli-plugins/docker-scout

Server:
Containers: 1
Running: 0
Paused: 0
Stopped: 1
Images: 1
Server Version: 27.2.0
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 8fc6bcff51318944179630522a095cc9dbf9f353
runc version: v1.1.13-0-g58aa920
init version: de40ad0
Security Options:
seccomp
Profile: unconfined
cgroupns
Kernel Version: 6.10.4-linuxkit
Operating System: Docker Desktop
OSType: linux
Architecture: x86_64
CPUs: 16
Total Memory: 3.82GiB
Name: docker-desktop
ID: 7fcfde63-a5b5-4949-bc88-8d5f172e82bc
Docker Root Dir: /var/lib/docker
Debug Mode: false
HTTP Proxy: http.docker.internal:3128
HTTPS Proxy: http.docker.internal:3128
No Proxy: hubproxy.docker.internal
Labels:
com.docker.desktop.address=unix:///home/cc/.docker/desktop/docker-cli.sock
Experimental: false
Insecure Registries:
hubproxy.docker.internal:5555
127.0.0.0/8
Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile

Diagnostics ID

A5DA3A08-2769-4902-A16C-4C831A909E22/20241025075326

Additional Info

If the Docker Desktop has started first, Virtualbox VMS won't start with those errors

vboxmanage startvm Centos-TUI --type=headless
Waiting for VM "Centos-TUI" to power on...
VBoxManage: error: VirtualBox can't enable the AMD-V extension. Please disable the KVM kernel extension, recompile your kernel and reboot (VERR_SVM_IN_USE)
VBoxManage: error: Details: code NS_ERROR_FAILURE (0x80004005), component ConsoleWrap, interface IConsole

And /opt/docker-desktop/bin/com.docker.diagnose check shows like this

Starting diagnostics

[2024-10-25T08:23:27.949602300Z][com.docker.diagnose.ipc] 3e26c1fe-diagnose -> <HOME>/.docker/desktop/backend.sock BackendAPI
[2024-10-25T08:23:27.949773552Z][com.docker.diagnose.ipc] (78f8a88d) 3e26c1fe-diagnose C->S BackendAPI POST /idle/make-busy
[2024-10-25T08:23:27.950473237Z][com.docker.diagnose.ipc] (78f8a88d) 3e26c1fe-diagnose C<-S bde81f4f-BackendAPI POST /idle/make-busy (683.415µs): 0xc00079e090
[2024-10-25T08:23:28.950058509Z][com.docker.diagnose.ipc] (78f8a88d-1) 3e26c1fe-diagnose C->S BackendAPI GET /idle
[2024-10-25T08:23:28.950706537Z][com.docker.diagnose.ipc] (78f8a88d-1) 3e26c1fe-diagnose C<-S bde81f4f-BackendAPI GET /idle (649.241µs): {"busyReason":["timed activities: map[/idle/make-busy:28999819660]"],"timedActivities":{"/idle/make-busy":"float64"},"vmRunning":true}
[PASS] DD0039: are KVM user permissions configured?
[PASS] DD0018: does the host support virtualization?
[PASS] DD0001: is the application running?
[FAIL] DD0017: can a VM be started? vm has not started: vm has not started
[FAIL] DD0016: is the LinuxKit VM running? prereq failed: can a VM be started?
[FAIL] DD0004: is the Docker engine running? prereq failed: is the LinuxKit VM running?
[PASS] DD0015: are the binary symlinks installed?
[FAIL] DD0031: does the Docker API work? prereq failed: is the Docker engine running?
[PASS] DD0013: is the $PATH ok?
[PASS] DD0034: is Context set to a Docker Desktop context?
[FAIL] DD0003: is the Docker CLI working? prereq failed: is the Docker engine running?
[FAIL] DD0038: is the connection to Docker working? prereq failed: is the Docker engine running?
[FAIL] DD0014: are the backend processes running? prereq failed: is the LinuxKit VM running?
[FAIL] DD0007: is the backend responding? prereq failed: are the backend processes running?
[FAIL] DD0009: is the vpnkit API responding? prereq failed: are the backend processes running?
[FAIL] DD0010: is the Docker API proxy responding? prereq failed: are the backend processes running?
[FAIL] DD0030: is the image access management authorized? prereq failed: is the Docker engine running?
[PASS] DD0037: is the virtiofs setup correct?
[WARN] DD0036: is the credentials store configured correctly? Error: password store is empty. Try "pass init".

[PASS] DD0033: does the host have Internet access?
[PASS] DD0039: are KVM user permissions configured?
[PASS] DD0018: does the host support virtualization?
[PASS] DD0001: is the application running?
[WARN] DD0017: can a VM be started? vm has not started: vm has not started
[WARN] DD0016: is the LinuxKit VM running? prereq failed: can a VM be started?
[WARN] DD0004: is the Docker engine running? prereq failed: is the LinuxKit VM running?
[PASS] DD0015: are the binary symlinks installed?
[WARN] DD0031: does the Docker API work? prereq failed: is the Docker engine running?
[WARN] DD0032: do Docker networks overlap with host IPs? prereq failed: does the Docker API work?

Please note the following 6 warnings:

1 : The check: is the credentials store configured correctly?
    Produced the following warning: Error: password store is empty. Try "pass init".


The Credential Store is not set up correctly.

See https://docs.docker.com/desktop/get-started/#credentials-management-for-linux-users

2 : The check: can a VM be started?
    Produced the following warning: vm has not started: vm has not started

The Docker engine runs inside a Linux VM. Therefore we must be able to start Virtual Machines.

3 : The check: is the LinuxKit VM running?
    Produced the following warning: prereq failed: can a VM be started?

The Docker engine runs inside a Linux VM. Therefore the VM must be running.

4 : The check: is the Docker engine running?
    Produced the following warning: prereq failed: is the LinuxKit VM running?

The Docker engine manages all containers and images on the host. Check the dockerd.log to see why it failed to start.

5 : The check: does the Docker API work?
    Produced the following warning: prereq failed: is the Docker engine running?

If the Docker API is not available from the host then Docker Desktop will not work correctly.

6 : The check: do Docker networks overlap with host IPs?
    Produced the following warning: prereq failed: does the Docker API work?

If the subnet used by a Docker network overlaps with an IP used by the host, then containers
won't be able to contact the overlapping IP addresses.

Try configuring the IP address range used by networks: in your docker-compose.yml.
See https://docs.docker.com/compose/compose-file/compose-file-v2/#ipv4_address-ipv6_address


Please investigate the following 1 issue:

1 : The test: can a VM be started?
    Failed with: vm has not started: vm has not started

The Docker engine runs inside a Linux VM. Therefore we must be able to start Virtual Machines.

[filoucrackeur94220]

+1 same problem i'm on fedora 41 last version of docker virtualbox 7.0 or 7.1 same same result

[wesmorgan-mirantis]

As the original poster noted, the two are mutually incompatible; VirtualBox can't launch VMs if Docker Desktop is running. (For those who may be interested, the VirtualBox error is "VirtualBox can't operate in VMX root mode. Please disable the KVM kernel extension, recompile your kernel and reboot (VERR_VMX_IN_VMX_ROOT_MODE).")

The problem is that QEMU (under Docker Desktop) and VirtualBox both want to control CPU virtualization. Basically, whichever of the two runs first does its CPU virtualization and 'locks out' the other. I tried forcing qemu to use TCG instead of KVM (by creating and modifying /etc/qemu/qemu.conf), but Docker Desktop failed even faster on launch than it does when locked out by VirtualBox.

It is possible to build a version of VirtualBox that uses KVM instead of its own hypervisor - details here. I know of no one offering prebuilt binaries of this version.

[dhay3]

@wesmorgan-mirantis Thanks for your patient explanation. But there is another thing mentioned in the OP that I cannot figure out.

But the default docker context installed by the package manager works as expected when Virtualbox is in use.

What is the difference between Docker Desktop and pure Docker Engine? I do want to read the source code, but it's hard for me without professional knowledge.