Merge branch 'wireapp:develop' into develop

changelog.d/0-release-notes/configurable-argon

@@ -1,18 +1,33 @@
-Password hashing is now done using argon2id instead of scrypt. The argon2id parameters can be configured using these options:
+Password hashing can now be done using argon2id instead of scrypt. The argon2id parameters can be configured using these options:
 
 ```yaml
 brig:
   optSettings:
     setPasswordHashingOptions:
+      algorithm: argon2id
       iterations: ...
       memory: ... # memory needed in KiB
       parallelism: ...
 galley:
   settings:
     passwordHashingOptions:
+      algorithm: argon2id
       iterations: ...
       memory: ... # memory needed in KiB
       parallelism: ...
 ```
 
-These have default values, which should work for most deployments. Please see documentation on config-options for more. 
+The default option is still to use scrypt as moving to argon2id might require
+allocating more resources according to configured parameters.
+
+When configured to use argon2id, the DB will be migrated slowly over time as the
+users enter their passwords (either to login or to do other operations which
+require explicit password entry). This migration is **NOT** done in reverse,
+i.e., if a deployment started with argon2id as the algorithm then chose to move
+to scrypt, the passwords will not get rehashed automatically, instead the users
+will have to reset their passwords if that is desired.
+
+**NOTE** It is highly recommended to move to argon2id as it will be made the
+  only available choice for the `algorithm` config option in future.
+
+(#4291, ##)

changelog.d/2-features/add-config-for-pwd-hash

@@ -1 +1 @@
-Allow configuring Argon2id parameters
+Allow choosing hashing algorithm and configuring argon2id parameters (#4291, ##)

changelog.d/4-docs/fix-swagger-2

@@ -0,0 +1 @@
+Move docs from docs.wire.com to generated helper page served by brig

changelog.d/5-internal/fix-local-fed-v1

@@ -0,0 +1 @@
+Local integration tests of federation version V1 fixed

charts/brig/values.yaml

@@ -150,11 +150,12 @@ config:
     setDisabledAPIVersions: [ development ]
     setFederationStrategy: allowNone
     setFederationDomainConfigsUpdateFreq: 10
-    # Options for Argon2id version 19
     setPasswordHashingOptions:
-      iterations: 1
-      parallelism: 32
-      memory: 180224 # 176 MiB
+      algorithm: scrypt # or argon2id
+      # When algorithm is argon2id, these can be configured:
+      # iterations: <iterations>
+      # parallelism: <paralellism>
+      # memory: <memory used in KiB>
   smtp:
     passwordFile: /etc/wire/brig/secrets/smtp-password.txt
   proxy: {}

charts/galley/values.yaml

@@ -70,11 +70,12 @@ config:
     # The lifetime of a conversation guest link in seconds. Must be a value 0 < x <= 31536000 (365 days)
     # Default is 31536000 (365 days) if not set
     guestLinkTTLSeconds: 31536000
-    # Options for Argon2id version 19
     passwordHashingOptions:
-      iterations: 1
-      parallelism: 32
-      memory: 180224 # 176 MiB
+      algorithm: scrypt # or argon2id
+      # When algorithm is argon2id, these can be configured:
+      # iterations: <iterations>
+      # parallelism: <paralellism>
+      # memory: <memory used in KiB>
 
     # To disable proteus for new federated conversations:
     # federationProtocols: ["mls"]

deploy/dockerephemeral/federation-v0/nginz/conf/integration.conf

@@ -15,7 +15,5 @@ listen 8090;
 # But to also test tls forwarding, this port can be used.
 # This applies only locally, as for kubernetes (helm chart) based deployments,
 # TLS is terminated at the ingress level, not at nginz level
-listen 8443 ssl;
-listen [::]:8443 ssl;
-
-http2 on;
+listen 8443 ssl http2;
+listen [::]:8443 ssl http2;

deploy/dockerephemeral/federation-v1/nginz/conf/integration.conf

@@ -15,7 +15,5 @@ listen 8090;
 # But to also test tls forwarding, this port can be used.
 # This applies only locally, as for kubernetes (helm chart) based deployments,
 # TLS is terminated at the ingress level, not at nginz level
-listen 8443 ssl;
-listen [::]:8443 ssl;
-
-http2 on;
+listen 8443 ssl http2;
+listen [::]:8443 ssl http2;

deploy/dockerephemeral/run.sh

@@ -1,25 +1,28 @@
 #!/usr/bin/env bash
 
+# To start the federation v0, v1 backends, set ENABLE_FEDERATION_V0=1, ENABLE_FEDERATION_V1=1
+# in the env where this script is run
+
 set -e
 
 # run.sh should work no matter what is the current directory
-SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 DOCKER_FILE="$SCRIPT_DIR/docker-compose.yaml"
 FED_VERSIONS=(0 1)
 
-opts=( "--file" "$DOCKER_FILE" )
+opts=("--file" "$DOCKER_FILE")
 for v in "${FED_VERSIONS[@]}"; do
   var="ENABLE_FEDERATION_V$v"
   if [[ "${!var}" == 1 ]]; then
-    opts+=( "--file" "$SCRIPT_DIR/federation-v$v.yaml" )
+    opts+=("--file" "$SCRIPT_DIR/federation-v$v.yaml")
   fi
 done
 
 dc() {
   docker-compose "${opts[@]}" "$@"
 }
 
-cleanup () {
+cleanup() {
   dc down
 }
 

docs/src/developer/reference/config-options.md

@@ -719,52 +719,46 @@ optSettings:
   setOAuthMaxActiveRefreshTokens: 10
 ```
 
-#### Argon2id password hashing parameters
+#### Password hashing options
 
-Since release 5.6.0, wire-server hashes passwords with
-[argon2id](https://datatracker.ietf.org/doc/html/rfc9106) at rest.  If
-you do not do anything, the default parameters will be used, which
-are:
-
-```yaml
-    setPasswordHashingOptions:
-      iterations: 1
-      memory: 180224 # memory needed in kibibytes (1 kibibyte is 2^10 bytes)
-      parallelism: 32
-```
-
-The default will be adjusted to new developments in hashing algorithm
-security from time to time.
+Since release 5.6.0, wire-server can hash passwords with
+[argon2id](https://datatracker.ietf.org/doc/html/rfc9106) to be stored at rest.
+If you do not do anything, the deployment will still use scrypt.
 
 The password hashing options are set for brig and galley:
 
 ```yaml
 brig:
   optSettings:
     setPasswordHashingOptions:
+      algorithm: # argon2id or scrypt
+      # These options only apply to argon2id
       iterations: ...
       memory: ... # memory needed in KiB
       parallelism: ...
 galley:
   settings:
     passwordHashingOptions:
+      algorithm: # argon2id or scrypt
+      # These options only apply to argon2id
       iterations: ...
       memory: ... # memory needed in KiB
       parallelism: ...
 ```
 
-**Performance implications:** scrypt takes ~80ms on a realistic test
-system, and argon2id with default settings takes ~500ms. This is a
-runtime increase by a factor of ~6.  This happens every time a
-password is entered by the user: during login, password reset,
-deleting a device, etc. (It does **NOT** happen during any other
-cryptographic operations like session key update or message
-de-/encryption.)
+**Performance implications:** argon2id typically takes longer and uses more
+memory than scrypt. So when migrating to it brig and galley pods must be
+allocated more resouces according to the chosen paramters.
+
+When configured to use argon2id, the DB will be migrated slowly over time as the
+users enter their passwords (either to login or to do other operations which
+require explicit password entry). This migration is **NOT** done in reverse,
+i.e., if a deployment started with argon2id as the algorithm then chose to move
+to scrypt, the passwords already stored will not get rehashed automatically,
+however the users will still be able to use them to login.
 
-The settings are a trade-off between resilience against brute force
-attacks and password secrecy.  For most systems this should be safe
-and not need more hardware resources for brig, but you may want to
-form your own opinion.
+**NOTE** It is highly recommended to move to argon2id as it will be made the
+  only available choice for the `algorithm` config option in future.
 
 #### Disabling API versions
 

docs/src/understand/api-client-perspective/swagger.md

@@ -1,37 +1,27 @@
 (swagger-api-docs)=
 
-# Swagger API documentation
+# Swagger / OpenAPI documentation
 
-Our staging system provides [Swagger /
-OpenAPI](https://swagger.io/resources/open-api/) documentation of our HTTP REST
-API.
+Our staging system provides [OpenAPI
+3.0](https://swagger.io/resources/open-api/) documentation of our HTTP
+REST API under the following URL:
 
-The swagger docs are correct by construction (compiled from the server
-code), and they are complete up to bots/services and event notification
-payloads (as of 2023-01-16).
+[https://staging-nginz-https.zinfra.io/api/swagger-ui](https://staging-nginz-https.zinfra.io/api/swagger-ui)
 
-There are several ways to interpret this kind of documentation:
+There are several ways to interpret this documentation:
 
 - Read it as a reference
 - Generate client code from it
 - Interactively explore the API by making requests
 
-## Swagger docs (Swagger 2.0)
+To find the source code of end-points mentioned in the API, a *route
+internal ID* (field `operationId` in openapi) is provided for every
+end-point.  See {ref}`named-and-internal-route-ids` for details and
+usage.
 
-The [Swagger / OpenAPI 2.0](https://swagger.io/specification/v2/)
-documentation for endpoints depends on the API version.  For a list of
-all swagger docs for all supported API versions, [visit
-https://staging-nginz-https.zinfra.io/api/swagger-ui](https://staging-nginz-https.zinfra.io/api/swagger-ui).
+If you find anything you don't like or understand, please let us know!
 
-To learn which versions are supported, look at
-`https://<nginz-host>/api-version`.  ([See
-also.](../../developer/developer/api-versioning.md))
-
-If you want to get the raw json for the swagger (ie., for compiling it
-into client code in typescript, kotlin, swift, ...), replace
-`swagger-ui` with `swagger.json` in the above URL pattern.
-
-#### Example: doing it manually
+## Example
 
 To get the versions a backend (`staging-nginz-https.zinfra.io` in this case)
 supports, execute:
@@ -43,57 +33,3 @@ curl https://<nginz-host>/api-version
 
 The URL to open in your browser for the development version `4` is
 `https://<nginz-host>/v4/api/swagger-ui/`.
-
-### On-prem and test instances, versioning
-
-The above is valid for the official wire.com staging environment and
-includes both all released API versions and the current development
-version, which changes continuously until released.
-
-If you talk to any other backend, the development version may differ.
-Try to ask the backend you're talking if it exposes its docs itself:
-
-```
-curl https://nginz-https.<custom-staging>.example.com/<version>/api/swagger-ui/
-curl https://nginz-https.<custom-staging>.example.com/<version>/api/swagger.json
-```
-
-### Internal endpoints
-
-Swagger docs for internal endpoints are served per service. I.e. there's one for
-`brig`, one for `cannon`, etc.. This is because Swagger doesn't play well with
-multiple actions having the same combination of HTTP method and URL path.
-
-Internal APIs are not under version control.
-
-- Unversioned:
-    - [`brig` - **internal** (private)
-    endpoints](https://staging-nginz-https.zinfra.io/api-internal/swagger-ui/brig)
-    - [`cannon` - **internal** (private)
-    endpoints](https://staging-nginz-https.zinfra.io/api-internal/swagger-ui/cannon)
-    - [`cargohold` - **internal** (private)
-    endpoints](https://staging-nginz-https.zinfra.io/api-internal/swagger-ui/cargohold)
-    - [`galley` - **internal** (private)
-    endpoints](https://staging-nginz-https.zinfra.io/api-internal/swagger-ui/galley)
-    - [`gundeck` - **internal** (private)
-    endpoints](https://staging-nginz-https.zinfra.io/api-internal/swagger-ui/gundeck)
-    - [`spar` - **internal** (private)
-    endpoints](https://staging-nginz-https.zinfra.io/api-internal/swagger-ui/spar)
-
-The URL pattern is similar to that of public endpoints for latest version:
-`https://<nginz-host>/api-internal/swagger-ui/<service>`.
-
-If you want to get the raw json of the swagger:
-`https://<nginz-host>/api-internal/swagger-ui/<service>-swagger.json`.
-
-### Federation API
-
-- Unversioned
-  - [`brig` - Federation API](https://staging-nginz-https.zinfra.io/api-federation/swagger-ui/brig)
-  - [`galley` - Federation API](https://staging-nginz-https.zinfra.io/api-federation/swagger-ui/galley)
-  - [`cargohold` - Federation API](https://staging-nginz-https.zinfra.io/api-federation/swagger-ui/cargohold)
-
-### Finding the source code for an end-point
-
-A *route internal ID* is provided for every end-point.  See
-{ref}`named-and-internal-route-ids` for details and usage.

hack/helm_vars/wire-server/values.yaml.gotmpl

@@ -136,7 +136,8 @@ brig:
       setOAuthMaxActiveRefreshTokens: 10
       # These values are insecure, against anyone getting hold of the hash, 
       # but its not a concern for the integration tests.
-      setPasswordHashingOptions: 
+      setPasswordHashingOptions:
+        algorithm: argon2id
         iterations: 1
         parallelism: 4
         memory: 32 # This needs to be at least 8 * parallelism.
@@ -266,7 +267,8 @@ galley:
 
       # These values are insecure, against anyone getting hold of the hash, 
       # but its not a concern for the integration tests.
-      passwordHashingOptions: 
+      passwordHashingOptions:
+        algorithm: argon2id
         iterations: 1
         parallelism: 4
         memory: 32 # This needs to be at least 8 * parallelism.

integration/test/Test/MLS/One2One.hs

@@ -395,6 +395,8 @@ testMLSGhostOne2OneConv = do
 -- still be created but only by the user whose backend hosts this conversation.
 
 -- | See Note: [Federated 1:1 MLS Conversations]
+-- To run locally this test requires federation-v1 docker containers to be up and running.
+-- See `deploy/dockerephemeral/run.sh` and comment on `StaticFedDomain` in `Testlib/VersionedFed.hs` for more details.
 testMLSFederationV1ConvOnOldBackend :: App ()
 testMLSFederationV1ConvOnOldBackend = do
   alice <- randomUser OwnDomain def
@@ -447,6 +449,8 @@ testMLSFederationV1ConvOnOldBackend = do
     parsedMsg %. "message.content.sender.External" `shouldMatchInt` 0
 
 -- | See Note: Federated 1:1 MLS Conversations
+-- To run locally this test requires federation-v1 docker containers to be up and running.
+-- See `deploy/dockerephemeral/run.sh` and comment on `StaticFedDomain` in `Testlib/VersionedFed.hs` for more details.
 testMLSFederationV1ConvOnNewBackend :: App ()
 testMLSFederationV1ConvOnNewBackend = do
   alice <- randomUser OwnDomain def

integration/test/Test/Swagger.hs

@@ -84,7 +84,7 @@ testSwaggerToc = do
     get path = rawBaseRequest OwnDomain Brig Unversioned path >>= submit "GET"
 
     html :: String
-    html = "<html><head></head><body><h2>please pick an api version</h2><a href=\"/v0/api/swagger-ui/\">/v0/api/swagger-ui/</a><br><a href=\"/v1/api/swagger-ui/\">/v1/api/swagger-ui/</a><br><a href=\"/v2/api/swagger-ui/\">/v2/api/swagger-ui/</a><br><a href=\"/v3/api/swagger-ui/\">/v3/api/swagger-ui/</a><br><a href=\"/v4/api/swagger-ui/\">/v4/api/swagger-ui/</a><br><a href=\"/v5/api/swagger-ui/\">/v5/api/swagger-ui/</a><br><a href=\"/v6/api/swagger-ui/\">/v6/api/swagger-ui/</a><br><a href=\"/v7/api/swagger-ui/\">/v7/api/swagger-ui/</a><br></body>"
+    html = "<html><head></head><body><h1>OpenAPI 3.0 docs for all Wire APIs</h1>\n<p>This wire-server system provides <a href=\"https://swagger.io/resources/open-api/\">OpenAPI 3.0</a> documentation of our HTTP REST API.</p> <p>The openapi docs are correct by construction (compiled from the server code), and more or less complete.</p> <p>Some endpoints are version-controlled. </a href=\"/api-version\">Show all supported versions.</a> <a href=\"https://docs.wire.com/developer/developer/api-versioning.html\">find out more.</a>\n<h2>Public (all available versions)</h2>\nv0: \n<a href=\"/v0/api/swagger-ui\">swagger-ui</a>; \n<a href=\"/v0/api/swagger.json\">swagger.json</a>\n<br>\nv1: \n<a href=\"/v1/api/swagger-ui\">swagger-ui</a>; \n<a href=\"/v1/api/swagger.json\">swagger.json</a>\n<br>\nv2: \n<a href=\"/v2/api/swagger-ui\">swagger-ui</a>; \n<a href=\"/v2/api/swagger.json\">swagger.json</a>\n<br>\nv3: \n<a href=\"/v3/api/swagger-ui\">swagger-ui</a>; \n<a href=\"/v3/api/swagger.json\">swagger.json</a>\n<br>\nv4: \n<a href=\"/v4/api/swagger-ui\">swagger-ui</a>; \n<a href=\"/v4/api/swagger.json\">swagger.json</a>\n<br>\nv5: \n<a href=\"/v5/api/swagger-ui\">swagger-ui</a>; \n<a href=\"/v5/api/swagger.json\">swagger.json</a>\n<br>\nv6: \n<a href=\"/v6/api/swagger-ui\">swagger-ui</a>; \n<a href=\"/v6/api/swagger.json\">swagger.json</a>\n<br>\nv7: \n<a href=\"/v7/api/swagger-ui\">swagger-ui</a>; \n<a href=\"/v7/api/swagger.json\">swagger.json</a>\n<br>\n\n<h2>Internal (not versioned)</h2>\n<p>Openapi docs for internal endpoints are served per service. I.e. there's one for `brig`, one for `cannon`, etc..  This is because Openapi doesn't play well with multiple actions having the same combination of HTTP method and URL path.</p>\nbrig:<br>\n<a href=\"/api-internal/swagger-ui/brig\">swagger-ui</a>; \n<a href=\"/api-internal/swagger-ui/brig-swagger.json\">swagger.json</a>\n<br>\ngalley:<br>\n<a href=\"/api-internal/swagger-ui/galley\">swagger-ui</a>; \n<a href=\"/api-internal/swagger-ui/galley-swagger.json\">swagger.json</a>\n<br>\nspar:<br>\n<a href=\"/api-internal/swagger-ui/spar\">swagger-ui</a>; \n<a href=\"/api-internal/swagger-ui/spar-swagger.json\">swagger.json</a>\n<br>\ncargohold:<br>\n<a href=\"/api-internal/swagger-ui/cargohold\">swagger-ui</a>; \n<a href=\"/api-internal/swagger-ui/cargohold-swagger.json\">swagger.json</a>\n<br>\ngundeck:<br>\n<a href=\"/api-internal/swagger-ui/gundeck\">swagger-ui</a>; \n<a href=\"/api-internal/swagger-ui/gundeck-swagger.json\">swagger.json</a>\n<br>\ncannon:<br>\n<a href=\"/api-internal/swagger-ui/cannon\">swagger-ui</a>; \n<a href=\"/api-internal/swagger-ui/cannon-swagger.json\">swagger.json</a>\n<br>\nproxy:<br>\n<a href=\"/api-internal/swagger-ui/proxy\">swagger-ui</a>; \n<a href=\"/api-internal/swagger-ui/proxy-swagger.json\">swagger.json</a>\n<br>\n\n<h2>Federated API (backend-to-backend)</h2>\nbrig (v0):<br><a href=\"/v0/api-federation/swagger-ui/brig\">swagger-ui</a>; <a href=\"/v0/api-federation/swagger-ui/brig-swagger.json\">swagger.json</a><br>brig (v1):<br><a href=\"/v1/api-federation/swagger-ui/brig\">swagger-ui</a>; <a href=\"/v1/api-federation/swagger-ui/brig-swagger.json\">swagger.json</a><br>brig (v2):<br><a href=\"/v2/api-federation/swagger-ui/brig\">swagger-ui</a>; <a href=\"/v2/api-federation/swagger-ui/brig-swagger.json\">swagger.json</a><br><br>\ngalley (v0):<br><a href=\"/v0/api-federation/swagger-ui/galley\">swagger-ui</a>; <a href=\"/v0/api-federation/swagger-ui/galley-swagger.json\">swagger.json</a><br>galley (v1):<br><a href=\"/v1/api-federation/swagger-ui/galley\">swagger-ui</a>; <a href=\"/v1/api-federation/swagger-ui/galley-swagger.json\">swagger.json</a><br>galley (v2):<br><a href=\"/v2/api-federation/swagger-ui/galley\">swagger-ui</a>; <a href=\"/v2/api-federation/swagger-ui/galley-swagger.json\">swagger.json</a><br><br>\ncargohold (v0):<br><a href=\"/v0/api-federation/swagger-ui/cargohold\">swagger-ui</a>; <a href=\"/v0/api-federation/swagger-ui/cargohold-swagger.json\">swagger.json</a><br>cargohold (v1):<br><a href=\"/v1/api-federation/swagger-ui/cargohold\">swagger-ui</a>; <a href=\"/v1/api-federation/swagger-ui/cargohold-swagger.json\">swagger.json</a><br>cargohold (v2):<br><a href=\"/v2/api-federation/swagger-ui/cargohold\">swagger-ui</a>; <a href=\"/v2/api-federation/swagger-ui/cargohold-swagger.json\">swagger.json</a><br><br>\n\n</body></html>\n"
 
 data Swagger = SwaggerPublic | SwaggerInternal Service