[BUG] --pull policy flag does not complain about invalid values and silently swallows subsequent options

[liamjones]

Description

The new behaviour on --pull (from PR #10981 I think) just tripped me up.

The command I'd been using with previous versions was:

docker compose up --build --pull --detach && docker compose logs -f

Today I was finding the containers exiting when I ^C-ed from the logs. Initially I thought something was wrong with logs -f but it turned out it was never reaching the second command.

What was happening was --pull was reading --detach as its policy value so --detach was never getting to the up command. ^C was killing the up command, nothing to do with the logs command.

If up had failed to run and --pull had complained that "--detach is not a valid policy" the issue would have been more obvious.

Steps To Reproduce

1. Run docker compose up --pull --detach
2. ^C

Compose Version

Docker Compose version v2.21.0

Docker Environment

Client: Docker Engine - Community
 Version:    24.0.6
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.21.0
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 3
  Running: 3
  Paused: 0
  Stopped: 0
 Images: 276
 Server Version: 24.0.6
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 61f9fd88f79f081d64d6fa3bb1a0dc71ec870523
 runc version: v1.1.9-0-gccaecfc
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
 Kernel Version: 5.15.0-1047-aws
 Operating System: Ubuntu 20.04.6 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 2
 Total Memory: 7.674GiB
 Name: ip-172-31-45-113
 ID: e22db929-e459-4698-a63c-f6984d478327
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Anything else?

No response

[glours]

Hello @liamjones
I'm not able to reproduce and the PR you mentioned was introduced in Compose v2.22.0 and isn't present in v2.21.0 🤔
Can you try with the v2.22.0 and let me know if you still have the issue?

[liamjones]

Oh, interesting. v2.21.0 is the newest version available on the relevant server via apt currently (Ubuntu 20.04.6 LTS).

Let me check nothing weird is up with the installation on that box.

[liamjones]

I went back through the ubuntu install instructions. There was still an old docker-compose package installed so I removed that.

Behaviour is still the same. Interestingly, if that PR was only added in v2.22.0 I don't understand why I'm seeing this output from docker compose up --help? E.g. it lists the pull policy options:

user@host:~/jenkins$ docker compose up --help

Usage:  docker compose up [OPTIONS] [SERVICE...]

Create and start containers

Options:
      --abort-on-container-exit   Stops all containers if any container was stopped. Incompatible with -d
      --always-recreate-deps      Recreate dependent containers. Incompatible with --no-recreate.
      --attach stringArray        Restrict attaching to the specified services. Incompatible with --attach-dependencies.
      --attach-dependencies       Automatically attach to log output of dependent services.
      --build                     Build images before starting containers.
  -d, --detach                    Detached mode: Run containers in the background
      --dry-run                   Execute command in dry run mode
      --exit-code-from string     Return the exit code of the selected service container. Implies --abort-on-container-exit
      --force-recreate            Recreate containers even if their configuration and image haven't changed.
      --no-attach stringArray     Do not attach (stream logs) to the specified services.
      --no-build                  Don't build an image, even if it's missing.
      --no-color                  Produce monochrome output.
      --no-deps                   Don't start linked services.
      --no-log-prefix             Don't print prefix in logs.
      --no-recreate               If containers already exist, don't recreate them. Incompatible with --force-recreate.
      --no-start                  Don't start the services after creating them.
      --pull string               Pull image before running ("always"|"missing"|"never") (default "missing")
      --quiet-pull                Pull without printing progress information.
      --remove-orphans            Remove containers for services not defined in the Compose file.
  -V, --renew-anon-volumes        Recreate anonymous volumes instead of retrieving data from the previous containers.
      --scale scale               Scale SERVICE to NUM instances. Overrides the scale setting in the Compose file if present.
  -t, --timeout int               Use this timeout in seconds for container shutdown when attached or when containers are already running.
      --timestamps                Show timestamps.
      --wait                      Wait for services to be running|healthy. Implies detached mode.
      --wait-timeout int          Maximum duration to wait for the project to be running|healthy.

[liamjones]

and I think I misidentified the PR, that's changing it from always/missing/never. Looks like it was in like this from the start (9976077#diff-8d744271912deb0e53fc0cc830281171cb1c2273ad8e9568692fd316b7f0356cR75) so I'm not sure why I've only just started running into this issue today...

[liamjones]

@glours As 2.22.0 wasn't available on the server I tried reproducing locally on my mac (also with a completely different docker-compose file). I get the same behaviour. If I do docker compose up --pull --detach it attaches but doesn't complain that --detach isn't a valid --pull value). docker compose up --pull always --detach or docker compose up --detach --pull work as expected.

docker compose version:

Docker Compose version v2.22.0-desktop.2

docker info:

Client:
 Version:    24.0.6
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.11.2-desktop.5
    Path:     /Users/liam.jones/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.22.0-desktop.2
    Path:     /Users/liam.jones/.docker/cli-plugins/docker-compose
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.0
    Path:     /Users/liam.jones/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.20
    Path:     /Users/liam.jones/.docker/cli-plugins/docker-extension
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v0.1.0-beta.8
    Path:     /Users/liam.jones/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/liam.jones/.docker/cli-plugins/docker-sbom
  scan: Docker Scan (Docker Inc.)
    Version:  v0.26.0
    Path:     /Users/liam.jones/.docker/cli-plugins/docker-scan
  scout: Docker Scout (Docker Inc.)
    Version:  v1.0.7
    Path:     /Users/liam.jones/.docker/cli-plugins/docker-scout

Server:
 Containers: 17
  Running: 0
  Paused: 0
  Stopped: 17
 Images: 57
 Server Version: 24.0.6
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 8165feabfdfe38c65b599c4993d227328c231fca
 runc version: v1.1.8-0-g82f18fe
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.4.16-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 16
 Total Memory: 7.768GiB
 Name: docker-desktop
 ID: e1d7b639-f7ec-4f2e-8bbb-82250aee0439
 Docker Root Dir: /var/lib/docker
 Debug Mode: true
  File Descriptors: 45
  Goroutines: 71
  System Time: 2023-10-18T12:06:01.066077015Z
  EventsListeners: 12
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Experimental: true
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile