Merge pull request #120 from cdalvaro/feature/set_reactor_worker_threads

feat: Set the number of workers for the runner/wheel in the reactor
cdalvaro · Dec 31, 2021 · dfeea99 · dfeea99
2 parents 65d21d5 + 2194025
commit dfeea99
Show file tree

Hide file tree

Showing 13 changed files with 44 additions and 31 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -4,6 +4,10 @@ This file only reflects the changes that are made in this image.
 Please refer to the [Salt 3004 Release Notes](https://docs.saltstack.com/en/latest/topics/releases/3004.html)
 for the list of changes in SaltStack.
 
+**3004_5**
+
+- Set the number of workers for the runner/wheel in the reactor by setting `SALT_REACTOR_WORKER_THREADS` env variable.
+
 **3004_4**
 
 - Fix an issue restarting `salt-master` processes with `supervisorctl` when reloading config.

diff --git a/Dockerfile b/Dockerfile
@@ -7,7 +7,7 @@ ARG VCS_REF
 ENV SALT_VERSION="3004" \
     PYTHON_VERSION="3.9"
 
-ENV IMAGE_VERSION="${SALT_VERSION}_4"
+ENV IMAGE_VERSION="${SALT_VERSION}_5"
 
 ENV SALT_DOCKER_DIR="/etc/docker-salt" \
     SALT_ROOT_DIR="/etc/salt" \

diff --git a/README.md b/README.md
@@ -135,7 +135,7 @@ docker run --name salt_master -d \
 
 This image provides support for automatically restart `salt-master` when configuration files change.
 This support is disabled by default, but it can be enabled by setting the
-`SALT_RESTART_MASTER_ON_CONFIG_CHANGE` environment variable to `true`.
+`SALT_RESTART_MASTER_ON_CONFIG_CHANGE` environment variable to `True`.
 
 ### Custom States
 
@@ -213,7 +213,7 @@ The newly created keys will appear inside `keys/generated/new_master_sign` direc
 
 ### Salt API
 
-You can enable `salt-api` service by setting env variable `SALT_API_SERVICE_ENABLED` to `true`.
+You can enable `salt-api` service by setting env variable `SALT_API_SERVICE_ENABLED` to `True`.
 
 A self-signed SSL certificate will be automatically generated and the following configuration will be added to the master configuration file:
 
@@ -229,7 +229,7 @@ The container exposes port `8000` by default, although you can map this port to
 ```sh
 docker run --name salt_stack --detach \
     --publish 4505:4505 --publish 4506:4506 --publish 8000:8000 \
-    --env 'SALT_API_SERVICE_ENABLED=true' \
+    --env 'SALT_API_SERVICE_ENABLED=True' \
     --env 'SALT_API_USER_PASS=4wesome-Pass0rd' \
     --volume $(pwd)/roots/:/home/salt/data/srv/ \
     --volume $(pwd)/keys/:/home/salt/data/keys/ \
@@ -243,7 +243,7 @@ By default, user `salt_api` is created and you can set its password by setting t
 
 You can also change the salt-api _username_ by setting `SALT_API_USER`. It is possible to disable this user by explicitly setting this variable to an empty string: `SALT_API_USER=''` if you are going to use an `LDAP` server.
 
-As a security measure, if `SALT_API_SERVICE_ENABLED` is set to `true` and you don't disable `SALT_API_USER`, you'll be required to set `SALT_API_USER_PASS`. Otherwise initialization will fail and your Docker image won't work.
+As a security measure, if `SALT_API_SERVICE_ENABLED` is set to `True` and you don't disable `SALT_API_USER`, you'll be required to set `SALT_API_USER_PASS`. Otherwise initialization will fail and your Docker image won't work.
 
 With all that set, you'll be able to provide your _salt-api_ custom configuration by creating the `salt-api.conf` file inside your `conf` directory:
 
@@ -508,24 +508,25 @@ Below you can find a list with the available options that can be used to customi
 
 | Parameter                              | Description                                                                                                                                                                                                                |
 | :------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `DEBUG`                                | Set this to `true` to enable entrypoint debugging.                                                                                                                                                                         |
+| `DEBUG`                                | Set this to `True` to enable entrypoint debugging.                                                                                                                                                                         |
 | `TIMEZONE` / `TZ`                      | Set the container timezone. Defaults to `UTC`. Values are expected to be in Canonical format. Example: `Europe/Madrid`. See the list of [acceptable values](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones). |
-| `SALT_RESTART_MASTER_ON_CONFIG_CHANGE` | Set this to `true` to restart `salt-master` service when configuration files change. Default: `false`                                                                                                                      |
+| `SALT_RESTART_MASTER_ON_CONFIG_CHANGE` | Set this to `True` to restart `salt-master` service when configuration files change. Default: `False`                                                                                                                      |
 | `SALT_LOG_LEVEL`                       | The level of messages to send to the console. One of 'garbage', 'trace', 'debug', info', 'warning', 'error', 'critical'. Default: `warning`                                                                                |
 | `SALT_LOG_ROTATE_FREQUENCY`            | Logrotate frequency for salt logs. Available options are 'daily', 'weekly', 'monthly', and 'yearly'. Default: `weekly`                                                                                                     |
 | `SALT_LOG_ROTATE_RETENTION`            | Keep x files before deleting old log files. Defaults: `52`                                                                                                                                                                 |
 | `SALT_LEVEL_LOGFILE`                   | The level of messages to send to the log file. One of 'garbage', 'trace', 'debug', info', 'warning', 'error', 'critical'. Default: `warning`                                                                               |
-| `SALT_API_SERVICE_ENABLED`             | Enable `salt-api` service. Default: `false`                                                                                                                                                                                |
+| `SALT_API_SERVICE_ENABLED`             | Enable `salt-api` service. Default: `False`                                                                                                                                                                                |
 | `SALT_API_USER`                        | Set username for `salt-api` service. Default: `salt_api`                                                                                                                                                                   |
-| `SALT_API_USER_PASS`                   | `SALT_API_USER` password. Required if `SALT_API_SERVICE_ENBALED` is `true` and `SALT_API_USER` is not empty. _Unset_ by default                                                                                            |
+| `SALT_API_USER_PASS`                   | `SALT_API_USER` password. Required if `SALT_API_SERVICE_ENBALED` is `True` and `SALT_API_USER` is not empty. _Unset_ by default                                                                                            |
 | `SALT_API_CERT_CN`                     | Common name in the request. Default: `localhost`                                                                                                                                                                           |
 | `SALT_MASTER_SIGN_PUBKEY`              | Sign the master auth-replies with a cryptographic signature of the master's public key. Possible values: 'True' or 'False'. Default: `False`                                                                               |
 | `SALT_MASTER_USE_PUBKEY_SIGNATURE`     | Instead of computing the signature for each auth-reply, use a pre-calculated signature. This option requires `SALT_MASTER_SIGN_PUBKEY` set to 'True'. Possible values: 'True' or 'False'. Default: `True`                  |
 | `SALT_MASTER_SIGN_KEY_NAME`            | The customizable name of the signing-key-pair without suffix. Default: `master_sign`                                                                                                                                       |
 | `SALT_MASTER_PUBKEY_SIGNATURE`         | The name of the file in the master's pki-directory that holds the pre-calculated signature of the master's public-key. Default: `master_pubkey_signature`                                                                  |
 | `SALT_MASTER_ROOT_USER`                | Forces `salt-master` to be runned as `root` instead of `salt`. Default: `False`                                                                                                                                            |
 | `SALT_GITFS_SSH_PRIVATE_KEY`           | The name of the ssh private key for gitfs. Default: `gitfs_ssh`                                                                                                                                                            |
-| `SALT_GITFS_SSH_PUBLIC_KEY`            | The name of the ssh public key for gitfs. Default: `gitfs_ssh.pub`                                                                                                                                                         |
+| `SALT_GITFS_SSH_PUBLIC_KEY`            | The name of the ssh public key for gitfs. Default: `gitfs_ssh`.pub`                                                                                                                                                        |
+| `SALT_REACTOR_WORKER_THREADS`          | The number of workers for the runner/wheel in the reactor. Default: `10`.                                                                                                                                                  |
 | `PUID`                                 | Sets the uid for user `salt` to the specified uid. Default: `1000`.                                                                                                                                                        |
 | `PGID`                                 | Sets the gid for user `salt` to the specified gid. Default: `1000`.                                                                                                                                                        |
 | `USERMAP_UID` (**deprecated**)         | Same as `PUID`. Support will be removed in Salt 3005 release in favor of `PUID`.                                                                                                                                           |
@@ -581,7 +582,7 @@ You can restart containers services by running the following command:
 docker exec -it salt_master entrypoint.sh app:restart [salt-service]
 ```
 
-Where `salt-service` is one of: `salt-master` os `salt-api` (if `SALT_API_SERVICE_ENABLED` is set to `true`)
+Where `salt-service` is one of: `salt-master` os `salt-api` (if `SALT_API_SERVICE_ENABLED` is set to `True`)
 
 ## References
 

diff --git a/assets/build/functions.sh b/assets/build/functions.sh
@@ -7,7 +7,7 @@ set -e
 #   DESCRIPTION:  Echo debug information to stdout.
 #----------------------------------------------------------------------------------------------------------------------
 function log_debug() {
-  if [[ "${DEBUG}" == 'true' || "${ECHO_DEBUG}" == 'true' ]]; then
+  if [[ "${DEBUG,,}" == true || "${ECHO_DEBUG,,}" == true ]]; then
     echo "[DEBUG] - $*"
   fi
 }

diff --git a/assets/runtime/config/master.yml b/assets/runtime/config/master.yml
@@ -85,6 +85,12 @@ pillar_roots:
     - {{SALT_BASE_DIR}}/pillar
 
 
+######        Reactor Settings        #####
+###########################################
+# Configure the number of workers for the runner/wheel in the reactor.
+reactor_worker_threads: {{SALT_REACTOR_WORKER_THREADS}}
+
+
 #####          Syndic settings       #####
 ##########################################
 # The Salt syndic is used to pass commands through a master from a higher

diff --git a/assets/runtime/env-defaults.sh b/assets/runtime/env-defaults.sh
@@ -1,17 +1,18 @@
 #!/usr/bin/env bash
 
-DEBUG=${DEBUG:-false}
+DEBUG=${DEBUG:-False}
 TIMEZONE=${TIMEZONE:-${TZ:-UTC}}
 
-SALT_API_SERVICE_ENABLED=${SALT_API_SERVICE_ENABLED:-false}
+SALT_API_SERVICE_ENABLED=${SALT_API_SERVICE_ENABLED:-False}
 SALT_API_USER=${SALT_API_USER:-salt_api}
 SALT_API_CERT_CN=${SALT_API_CERT_CN:-localhost}
 
 SALT_LOG_ROTATE_FREQUENCY=${SALT_LOG_ROTATE_FREQUENCY:-weekly}
 SALT_LOG_ROTATE_RETENTION=${SALT_LOG_ROTATE_RETENTION:-52}
 
 # https://docs.saltstack.com/en/latest/ref/configuration/master.html
-SALT_RESTART_MASTER_ON_CONFIG_CHANGE=${SALT_RESTART_MASTER_ON_CONFIG_CHANGE:-false}
+SALT_RESTART_MASTER_ON_CONFIG_CHANGE=${SALT_RESTART_MASTER_ON_CONFIG_CHANGE:-False}
+SALT_REACTOR_WORKER_THREADS=${SALT_REACTOR_WORKER_THREADS:-10}
 
 #####         Logging settings       #####
 # https://docs.saltstack.com/en/latest/ref/configuration/master.html#master-logging-settings

diff --git a/assets/runtime/functions.sh b/assets/runtime/functions.sh
@@ -27,7 +27,7 @@ function exec_as_salt()
 #   DESCRIPTION:  Echo debug information to stdout.
 #----------------------------------------------------------------------------------------------------------------------
 function log_debug() {
-  if [[ "${DEBUG}" == 'true' || "${ECHO_DEBUG}" == 'true' ]]; then
+  if [[ "${DEBUG,,}" == true || "${ECHO_DEBUG,,}" == true ]]; then
     echo "[DEBUG] - $*"
   fi
 }
@@ -268,7 +268,8 @@ function configure_salt_master()
     SALT_BASE_DIR \
     SALT_CACHE_DIR \
     SALT_CONFS_DIR \
-    SALT_KEYS_DIR
+    SALT_KEYS_DIR \
+    SALT_REACTOR_WORKER_THREADS
 
   # Update keys configuration
   update_template "${SALT_ROOT_DIR}/master" \
@@ -285,7 +286,7 @@ function configure_salt_master()
 function configure_salt_api()
 {
   rm -f /etc/supervisor/conf.d/salt-api.conf
-  [[ ${SALT_API_SERVICE_ENABLED} == true ]] || return 0
+  [[ ${SALT_API_SERVICE_ENABLED,,} == true ]] || return 0
 
   if [[ -n "${SALT_API_USER}" ]]; then
 
@@ -483,7 +484,7 @@ ${SALT_LOGS_DIR}/salt/key {
 }
 EOF
 
-  if [[ "${SALT_API_SERVICE_ENABLED}" == true ]]; then
+  if [[ "${SALT_API_SERVICE_ENABLED,,}" == true ]]; then
     # configure salt-api log rotation
     cat >> /etc/logrotate.d/salt <<EOF
 
@@ -502,7 +503,7 @@ EOF
 function configure_config_reloader()
 {
   rm -f /etc/supervisor/conf.d/config-reloader.conf
-  [ "${SALT_RESTART_MASTER_ON_CONFIG_CHANGE}" == true ] || return 0
+  [ "${SALT_RESTART_MASTER_ON_CONFIG_CHANGE,,}" == true ] || return 0
 
   log_info "Configuring config reloader ..."
 

diff --git a/docker-compose.yml b/docker-compose.yml
@@ -24,6 +24,6 @@ services:
       PGID: 1000
       SALT_LOG_LEVEL: info
       ### salt-api settings
-      # SALT_API_SERVICE_ENABLED: 'true'
+      # SALT_API_SERVICE_ENABLED: 'True'
       # SALT_API_USER: salt_api
       # SALT_API_USER_PASS: 4wesome-Pass0rd
diff --git a/entrypoint.sh b/entrypoint.sh
@@ -7,7 +7,7 @@ set -o pipefail
 FUNCTIONS_FILE="${SALT_RUNTIME_DIR}/functions.sh"
 source "${FUNCTIONS_FILE}"
 
-[[ "${DEBUG}" == true ]] && set -x
+[[ "${DEBUG,,}" == true ]] && set -x
 
 case "${1}" in
   app:start|app:gen-signed-keys)

diff --git a/tests/basic/test.sh b/tests/basic/test.sh
@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 
 set -e
-[ "${DEBUG}" == true ] && set -vx
+[ "${DEBUG,,}" == true ] && set -vx
 
 echo "🧪 Running basic tests ..."
 
@@ -26,5 +26,5 @@ ok "salt version"
 
 # Test image calling healthcheck
 echo "==> Executing healthcheck ..."
-docker-exec /usr/local/sbin/healthcheck | grep -i 'true' || error "healthcheck"
+docker-exec /usr/local/sbin/healthcheck | grep -i true || error "healthcheck"
 ok "healthcheck"
diff --git a/tests/config-reloader/test.sh b/tests/config-reloader/test.sh
@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 
 set -e
-[ "${DEBUG}" == true ] && set -vx
+[ "${DEBUG,,}" == true ] && set -vx
 
 echo "🧪 Running config-reloader tests ..."
 
@@ -16,7 +16,7 @@ trap cleanup EXIT
 # Run test instance
 echo "==> Starting docker-salt-master (${PLATFORM}) ..."
 start_container_and_wait \
-  --env SALT_RESTART_MASTER_ON_CONFIG_CHANGE=true \
+  --env SALT_RESTART_MASTER_ON_CONFIG_CHANGE=True \
   --volume "${SCRIPT_PATH}/config":/home/salt/data/config:ro \
 || error "container started"
 ok "container started"

diff --git a/tests/gitfs/test.sh b/tests/gitfs/test.sh
@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 
 set -e
-[ "${DEBUG}" == true ] && set -vx
+[ "${DEBUG,,}" == true ] && set -vx
 
 echo "🧪 Running gitfs tests ..."
 
@@ -34,7 +34,7 @@ ok "container started"
 echo "==> Updating gitfs repositories ..."
 salt-run cache.clear_git_lock gitfs type=update
 UPDATE_REPOS="$( salt-run fileserver.update )"
-echo "${UPDATE_REPOS}" | grep -qi 'true' || error "update gitfs"
+echo "${UPDATE_REPOS}" | grep -qi true || error "update gitfs"
 ok "update gitfs"
 
 # Check pillars

diff --git a/tests/salt-api/test.sh b/tests/salt-api/test.sh
@@ -1,7 +1,7 @@
 #!/usr/bin/env bash
 
 set -e
-[ "${DEBUG}" == true ] && set -vx
+[ "${DEBUG,,}" == true ] && set -vx
 
 echo "🧪 Running salt-api tests ..."
 
@@ -37,7 +37,7 @@ ok "salt-api config created"
 echo "==> Starting docker-salt-master (${PLATFORM}) with salt-api config ..."
 start_container_and_wait \
   --publish 8000:8000 \
-  --env SALT_API_SERVICE_ENABLED=true \
+  --env SALT_API_SERVICE_ENABLED=True \
   --env SALT_API_USER_PASS="${SALTAPI_PASS}" \
   --volume "${SALTAPI_TMP_DIR}/config":/home/salt/data/config:ro \
 || error "container started"
@@ -61,7 +61,7 @@ curl -sSk "${SALTAPI_URL}" \
   -d client=runner \
   -d tgt='*' \
   -d fun=test.stream \
-| grep -i 'true' || error "curl command"
+| grep -i true || error "curl command"
 ok "curl command"
 
 # Install salt-pepper