Skip to content

Commit

Permalink
fix(docker compose): Use 'docker compose' everywhere (#10916)
Browse files Browse the repository at this point in the history
* fix(docker compose): Use 'docker compose' everywhere

* Apply suggestions from code review

Co-authored-by: Charles Neill <[email protected]>

---------

Co-authored-by: Charles Neill <[email protected]>
  • Loading branch information
kiblik and cneill authored Sep 20, 2024
1 parent a643544 commit 8dfe373
Show file tree
Hide file tree
Showing 14 changed files with 53 additions and 97 deletions.
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ A clear and concise description of what you expected to happen.
- DefectDojo version (see footer) or commit message: [use `git show -s --format="[%ci] %h: %s [%d]"`]

**Logs**
Use `docker-compose logs` (or similar, depending on your deployment method) to get the logs and add the relevant sections here showing the error occurring (if applicable).
Use `docker compose logs` (or similar, depending on your deployment method) to get the logs and add the relevant sections here showing the error occurring (if applicable).

**Sample scan files**
If applicable, add sample scan files to help reproduce your problem.
Expand Down
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/support_request.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ A clear and concise description of what you expected to happen.
- DefectDojo version (see footer) or commit message: [use `git show -s --format="[%ci] %h: %s [%d]"`]

**Logs**
Use `docker-compose logs` (or similar, depending on your deployment method) to get the logs and add the relevant sections here showing the error occurring (if applicable).
Use `docker compose logs` (or similar, depending on your deployment method) to get the logs and add the relevant sections here showing the error occurring (if applicable).

**Sample scan files**
If applicable, add sample scan files to help reproduce your problem.
Expand Down
4 changes: 2 additions & 2 deletions docker/docker-compose-check.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,11 @@ current=$(docker compose version --short)

echo 'Checking docker compose version'
if [[ $main -lt 2 ]]; then
echo "$current is not a supported docker-compose version, please upgrade to the minimum supported version: 2.0"
echo "$current is not a supported 'docker compose' version, please upgrade to the minimum supported version: 2.0"
exit 1
elif [[ $main -eq 1 ]]; then
if [[ $minor -lt 28 ]]; then
echo "$current is not supported docker-compose version, please upgrade to minimal supported version:1.28"
echo "$current is not supported 'docker compose' version, please upgrade to minimal supported version:1.28"
exit 1
fi
fi
Expand Down
2 changes: 1 addition & 1 deletion docker/extra_settings/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ If a file if placed here, it will be copied on startup to `dojo/settings/local_s
For an example, see [template-local_settings](../../dojo/settings/template-local_settings)

Please note this copy action could fail if you have mounted the full `dojo/` folder, but that is owned by a different user/group.
That's why this copy action only happens in docker-compose release mode, and not in dev/debug/unit_tests/integration_tests modes.
That's why this copy action only happens in docker compose release mode, and not in dev/debug/unit_tests/integration_tests modes.

For advanced usage you can also place a `settings.dist.py` or `settings.py` file. These will also be copied on startup to dojo/settings.

Expand Down
14 changes: 7 additions & 7 deletions docs/content/en/contributing/how-to-write-a-parser.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ All commands assume that you're located at the root of the django-DefectDojo clo
- Checkout `dev` and make sure you're up to date with the latest changes.
- It's advised that you create a dedicated branch for your development, such as `git checkout -b parser-name`.

It is easiest to use the docker-compose deployment as it has hot-reload capbility for uWSGI.
It is easiest to use the docker compose deployment as it has hot-reload capbility for uWSGI.
Set up your environment to use the debug environment:

`$ docker/setEnv.sh debug`
Expand All @@ -27,7 +27,7 @@ Please have a look at [DOCKER.md](https://github.com/DefectDojo/django-DefectDoj
You will want to build your docker images locally, and eventually pass in your local user's `uid` to be able to write to the image (handy for database migration files). Assuming your user's `uid` is `1000`, then:

{{< highlight bash >}}
$ docker-compose build --build-arg uid=1000
$ docker compose build --build-arg uid=1000
{{< /highlight >}}

## Which files do you need to modify?
Expand Down Expand Up @@ -279,7 +279,7 @@ This ensures the file is closed at the end of the with statement, even if an exc

### Test database

To test your unit tests locally, you first need to grant some rights. Get your MySQL root password from the docker-compose logs, login as root and issue the following commands:
To test your unit tests locally, you first need to grant some rights. Get your MySQL root password from the docker compose logs, login as root and issue the following commands:

{{< highlight mysql >}}
MYSQL> grant all privileges on test_defectdojo.* to defectdojo@'%';
Expand All @@ -291,17 +291,17 @@ MYSQL> flush privileges;
This local command will launch the unit test for your new parser

{{< highlight bash >}}
$ docker-compose exec uwsgi bash -c 'python manage.py test unittests.tools.<your_unittest_py_file>.<main_class_name> -v2'
$ docker compose exec uwsgi bash -c 'python manage.py test unittests.tools.<your_unittest_py_file>.<main_class_name> -v2'
{{< /highlight >}}

Example for the blackduck hub parser:

{{< highlight bash >}}
$ docker-compose exec uwsgi bash -c 'python manage.py test unittests.tools.test_blackduck_csv_parser.TestBlackduckHubParser -v2'
$ docker compose exec uwsgi bash -c 'python manage.py test unittests.tools.test_blackduck_csv_parser.TestBlackduckHubParser -v2'
{{< /highlight >}}

{{% alert title="Information" color="info" %}}
If you want to run all unit tests, simply run `$ docker-compose exec uwsgi bash -c 'python manage.py test unittests -v2'`
If you want to run all unit tests, simply run `$ docker compose exec uwsgi bash -c 'python manage.py test unittests -v2'`
{{% /alert %}}

### Endpoint validation
Expand Down Expand Up @@ -330,7 +330,7 @@ In the event where you'd have to change the model, e.g. to increase a database c
* Create a new migration file in dojo/db_migrations by running and including as part of your PR

{{< highlight bash >}}
$ docker-compose exec uwsgi bash -c 'python manage.py makemigrations -v2'
$ docker compose exec uwsgi bash -c 'python manage.py makemigrations -v2'
{{< /highlight >}}

### Accept a different type of file to upload
Expand Down
4 changes: 2 additions & 2 deletions docs/content/en/getting_started/running-in-production.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ draft: false
weight: 4
---

## Production use with docker-compose
## Production use with docker compose

The docker-compose.yml file in this repository is fully functional to evaluate DefectDojo in your local environment.

Expand Down Expand Up @@ -76,7 +76,7 @@ Dockerfile.django-* for in-file references.

You can execute the following command to see the configuration:

`docker-compose exec celerybeat bash -c "celery -A dojo inspect stats"`
`docker compose exec celerybeat bash -c "celery -A dojo inspect stats"`
and see what is in effect.

#### Asynchronous Import
Expand Down
2 changes: 1 addition & 1 deletion docs/content/en/getting_started/upgrading/2.23.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,6 @@ There is a migration process built into the upgrade that will automatically conv
- If your deployment uses the MySQL containerized database, please see the following updates to run DefectDojo:
- Use of the helper script "dc-up": `./dc-up.sh mysql-rabbitmq` or `./dc-up.sh mysql-redis`
- Use of the helper script "dc-up-d": `./dc-up-d.sh mysql-rabbitmq` or `./dc-up-d.sh mysql-redis`
- Use of Docker Compose directly: `docker-compose --profile mysql-rabbitmq --env-file ./docker/environments/mysql-rabbitmq.env up` or `docker-compose --profile mysql-redis --env-file ./docker/environments/mysql-redis.env up`
- Use of Docker Compose directly: `docker compose --profile mysql-rabbitmq --env-file ./docker/environments/mysql-rabbitmq.env up` or `docker compose --profile mysql-redis --env-file ./docker/environments/mysql-redis.env up`

For all other changes, check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/2.23.0) for the contents of the release.
2 changes: 1 addition & 1 deletion docs/content/en/getting_started/upgrading/2.30.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ There are instructions for upgrading to 2.30.0 if you disabled `enable_auditlog`

Parameter `enable_auditlog` is not possible to set through System settings anymore. If you set this parameter or you need to change it to `False` (to disable audit logging), set environmental variable `DD_ENABLE_AUDITLOG` to `False`.

If you are using docker-compose, another EnvVar should be added to the `docker-compose.yml` file in all the containers ran by the django image. This should do the trick
If you are using docker compose, another EnvVar should be added to the `docker-compose.yml` file in all the containers ran by the django image. This should do the trick
```yaml
DD_ENABLE_AUDITLOG: ${DD_ENABLE_AUDITLOG:-False}
```
Expand Down
12 changes: 6 additions & 6 deletions docs/content/en/getting_started/upgrading/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@ draft: false
weight: 5
---

## Docker-compose
## Docker compose

When you deploy a vanilla docker-compose, it will create a persistent
When you deploy a vanilla docker compose, it will create a persistent
volume for your Postgres database. As long as your volume is there, you
should not lose any data.

Expand All @@ -19,7 +19,7 @@ DockerHub to update.
{{% /alert %}}


The generic upgrade method for docker-compose are as follows:
The generic upgrade method for docker compose are as follows:
- Pull the latest version

``` {.sourceCode .bash}
Expand All @@ -46,10 +46,10 @@ The generic upgrade method for docker-compose are as follows:
- Re-start DefectDojo, allowing for container recreation:
`./dc-up-d.sh`
- Database migrations will be run automatically by the initializer.
Check the output via `docker-compose logs initializer` or relevant k8s command
Check the output via `docker compose logs initializer` or relevant k8s command
- If you have the initializer disabled (or if you want to be on the
safe side), run the migration command:
`docker-compose exec uwsgi /bin/bash -c "python manage.py migrate"`
`docker compose exec uwsgi /bin/bash -c "python manage.py migrate"`
### Building your local images
Expand All @@ -64,7 +64,7 @@ first.
git merge origin/master
```
Then replace the first step of the above generic upgrade method for docker-compose with: `docker-compose build`
Then replace the first step of the above generic upgrade method for docker compose with: `docker compose build`
## godojo installations
Expand Down
6 changes: 3 additions & 3 deletions docs/content/en/integrations/jira.md
Original file line number Diff line number Diff line change
Expand Up @@ -167,19 +167,19 @@ optional arguments:
This can be executed from the uwsgi docker container using:

{{< highlight bash >}}
$ docker-compose exec uwsgi /bin/bash -c 'python manage.py jira_status_reconciliation'
$ docker compose exec uwsgi /bin/bash -c 'python manage.py jira_status_reconciliation'
{{< /highlight >}}

DEBUG output can be obtains via `-v 3`, but only after increasing the logging to DEBUG level in your settings.dist.py or local_settings.py file

{{< highlight bash >}}
$ docker-compose exec uwsgi /bin/bash -c 'python manage.py jira_status_reconciliation -v 3'
$ docker compose exec uwsgi /bin/bash -c 'python manage.py jira_status_reconciliation -v 3'
{{< /highlight >}}

At the end of the command a semicolon seperated CSV summary will be printed. This can be captured by redirecting stdout to a file:

{{< highlight bash >}}
$ docker-compose exec uwsgi /bin/bash -c 'python manage.py jira_status_reconciliation > jira_reconciliation.csv'
$ docker compose exec uwsgi /bin/bash -c 'python manage.py jira_status_reconciliation > jira_reconciliation.csv'
{{< /highlight >}}


Expand Down
2 changes: 1 addition & 1 deletion docs/content/en/integrations/ldap-authentication.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ Read the docs for Django Authentication with LDAP here: https://django-auth-ldap

#### docker-compose.yml

In order to pass the variables to the settings.dist.py file via docker, it's a good idea to add these to the docker-compose file.
In order to pass the variables to the settings.dist.py file via docker, it's a good idea to add these to the docker compose file.

You can do this by adding the following variables to the environment section for the uwsgi image:
```yaml
Expand Down
10 changes: 5 additions & 5 deletions docs/content/en/usage/features.md
Original file line number Diff line number Diff line change
Expand Up @@ -357,22 +357,22 @@ to the hashcode configuration or calculation logic. We will mention this in the
To regenerate the hashcodes, use the `dedupe` management command:

{{< highlight bash >}}
docker-compose exec uwsgi ./manage.py dedupe --hash_code_only
docker compose exec uwsgi ./manage.py dedupe --hash_code_only
{{< / highlight >}}

This will only regenerated the hashcodes, but will not run any deduplication logic on existing findings.
If you want to run deduplication again on existing findings to make sure any duplicates found by the new
hashcode config are marked as such, run:

{{< highlight bash >}}
docker-compose exec uwsgi ./manage.py dedupe
docker compose exec uwsgi ./manage.py dedupe
{{< / highlight >}}

The deduplication part of this command will run the deduplication for each finding in a celery task. If you want to
run the deduplication in the foreground process, use:

{{< highlight bash >}}
docker-compose exec uwsgi ./manage.py dedupe --dedupe_sync
docker compose exec uwsgi ./manage.py dedupe --dedupe_sync
{{< / highlight >}}

Please note the deduplication process is resource intensive and can take a long time to complete
Expand Down Expand Up @@ -502,10 +502,10 @@ You can of course change this default by modifying that stanza.
### Launching from the CLI

You can also invoke the SLA notification function from the CLI. For
example, if run from docker-compose:
example, if run from docker compose:

{{< highlight bash >}}
$ docker-compose exec uwsgi /bin/bash -c 'python manage.py sla_notifications'
$ docker compose exec uwsgi /bin/bash -c 'python manage.py sla_notifications'
{{< / highlight >}}

## Reports
Expand Down
62 changes: 9 additions & 53 deletions readme-docs/DOCKER.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Although Docker Compose is one of the supported installation methods to deploy a
# Prerequisites

* Docker version
* Installing with docker-compose requires at least Docker 19.03.0 and Docker Compose 1.28.0. See "Checking Docker versions" below for version errors during running docker-compose.
* Installing with docker compose requires at least Docker 19.03.0 and Docker Compose 1.28.0. See "Checking Docker versions" below for version errors during running docker compose.
* Proxies
* If you're behind a corporate proxy check https://docs.docker.com/network/proxy/ .

Expand Down Expand Up @@ -100,7 +100,7 @@ This will run the application based on merged configurations from docker-compose
* Hot-reloading for the **celeryworker** container is not yet implemented. When working on deduplication for example, restart the celeryworker container with:

```
docker-compose restart celeryworker
docker compose restart celeryworker
```

* The postgres port is forwarded to the host so that you can access your database from outside the container.
Expand All @@ -126,7 +126,7 @@ To find out the admin password, check the very beginning of the console
output of the initializer container by running:

```zsh
docker-compose logs initializer | grep "Admin password:"
docker compose logs initializer | grep "Admin password:"
```

Make sure you write down the first password generated as you'll need it when re-starting the application.
Expand All @@ -141,7 +141,7 @@ docker exec -it django-defectdojo-uwsgi-1 ./manage.py changepassword admin
```

# Logging
For docker-compose release mode the log level is INFO. In the other modes the log level is DEBUG. Logging is configured in `settings.dist.py` and can be tuned using a `local_settings.py`, see [template for local_settings.py](dojo/settings/template-local_settings). For example the deduplication logger can be set to DEBUG in a local_settings.py file:
For docker compose release mode the log level is INFO. In the other modes the log level is DEBUG. Logging is configured in `settings.dist.py` and can be tuned using a `local_settings.py`, see [template for local_settings.py](dojo/settings/template-local_settings). For example the deduplication logger can be set to DEBUG in a local_settings.py file:


```
Expand Down Expand Up @@ -251,7 +251,7 @@ To change the port:
- update `docker-compose.override.https.yml` or set DD_TLS_PORT in the environment)
- restart the application

NB: some third party software may require to change the exposed port in Dockerfile.nginx as they use docker-compose declarations to discover which ports to map when publishing the application.
NB: some third party software may require to change the exposed port in Dockerfile.nginx as they use docker compose declarations to discover which ports to map when publishing the application.


# Run the tests with Docker Compose
Expand Down Expand Up @@ -324,7 +324,7 @@ docker logs -f django-defectdojo_integration-tests_1

# Checking Docker versions

Run the following to determine the versions for docker and docker-compose:
Run the following to determine the versions for docker and docker compose:

```zsh
$ docker version
Expand All @@ -345,58 +345,14 @@ Server:
OS/Arch: linux/amd64
Experimental: false

$ docker-compose version
docker-compose version 1.18.0, build 8dd22a9
$ docker compose version
Docker Compose version 1.18.0, build 8dd22a9
docker-py version: 2.6.1
CPython version: 2.7.13
OpenSSL version: OpenSSL 1.0.1t 3 May 2016
```

In this case, both docker (version 17.09.0-ce) and docker-compose (1.18.0) need to be updated.
In this case, both docker (version 17.09.0-ce) and docker compose (1.18.0) need to be updated.

Follow [Docker's documentation](https://docs.docker.com/install/) for your OS to get the latest version of Docker. For the docker command, most OSes have a built-in update mechanism like "apt upgrade".

Docker Compose isn't packaged like Docker and you'll need to manually update an existing install if using Linux. For Linux, either follow the instructions in the [Docker Compose documentation](https://docs.docker.com/compose/install/) or use the shell script below. The script below will update docker-compose to the latest version automatically. You will need to make the script executable and have sudo privileges to upgrade docker-compose:

```zsh
#!/bin/bash

# Set location of docker-compose binary - shouldn't need to modify this
DESTINATION=/usr/local/bin/docker-compose

# Get latest docker-compose version
VERSION=$(curl --silent https://api.github.com/repos/docker/compose/releases/latest | jq .name -r)

# Output some info on what this is going to do
echo "Note: docker-compose version $VERSION will be downloaded from:"
echo "https://github.com/docker/compose/releases/download/${VERSION}/docker-compose-$(uname -s)-$(uname -m)"
echo "Enter sudo password to install docker-compose"

# Download and install latest docker compose
sudo curl -L https://github.com/docker/compose/releases/download/${VERSION}/docker-compose-$(uname -s)-$(uname -m) -o $DESTINATION
sudo chmod +x $DESTINATION

# Output new docker-compose version info
echo ""
docker-compose version
```

Running the script above will look like:

```zsh
$ vi update-docker-compose
$ chmod u+x update-docker-compose
$ ./update-docker-compose
Note: docker-compose version 1.24.0 will be downloaded from:
https://github.com/docker/compose/releases/download/1.24.0/docker-compose-Linux-x86_64
Enter sudo password to install docker-compose
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 617 0 617 0 0 1778 0 --:--:-- --:--:-- --:--:-- 1778
100 15.4M 100 15.4M 0 0 2478k 0 0:00:06 0:00:06 --:--:-- 2910k

docker-compose version 1.24.0, build 0aa59064
docker-py version: 3.7.2
CPython version: 3.6.8
OpenSSL version: OpenSSL 1.1.0j 20 Nov 2018
```
Loading

0 comments on commit 8dfe373

Please sign in to comment.