From 1b01358dc89ec9b1e6e29dd2b4bbe60b4fb4e343 Mon Sep 17 00:00:00 2001 From: Christian Kirschnick Date: Thu, 4 Jun 2020 20:38:55 +0200 Subject: [PATCH 1/4] Set version to 0.5.10 (#448) * Introduce new version 0.5.9 * New 0.5.10-SNAPSHOT --- .mvn/maven.config | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.mvn/maven.config b/.mvn/maven.config index 13ebeaa741..a9954b4b44 100644 --- a/.mvn/maven.config +++ b/.mvn/maven.config @@ -1,4 +1,4 @@ --Drevision=0.5.9-SNAPSHOT +-Drevision=0.5.10-SNAPSHOT -Dlicense.projectName=Corona-Warn-App -Dlicense.inceptionYear=2020 -Dlicense.licenseName=apache_v2 From 196a273f366a35f838f317ee7fcb17f71e2c2c73 Mon Sep 17 00:00:00 2001 From: Thomas Klingbeil <64434904+tklingbeil@users.noreply.github.com> Date: Thu, 4 Jun 2020 20:44:51 +0200 Subject: [PATCH 2/4] Update documentation (#447) * Updating documentation * linting * Renaming Verification Backend to Verification Server * Update of documentation * Add links to other documents * Adjust verification process * move time form to present * remove trailing spaces Co-authored-by: Christian Kirschnick --- docs/architecture-overview.md | 107 ++++++++++++++++++++-------------- 1 file changed, 62 insertions(+), 45 deletions(-) diff --git a/docs/architecture-overview.md b/docs/architecture-overview.md index c6089548bd..432c9732aa 100644 --- a/docs/architecture-overview.md +++ b/docs/architecture-overview.md @@ -1,8 +1,13 @@ -# Architecture CWA Backend +# Architecture CWA Server -This document outlines the CWA backend architecture on high level. This document -does not necessarily reflect the current implementation status in this repository, as there are -still frequent changes to the exposure notification API by Google/Apple. +This document outlines the CWA server architecture on high level. This document +does not necessarily reflect the current implementation status in this repository, as development +is still ongoing. Also, details of the exposure notification API by Google/Apple might change +in the future. + +Please note: This overview document only focuses on the architecture of this component. +If you are interested in the full architectural overview, check out the [solution architecture](https://github.com/corona-warn-app/cwa-documentation/blob/master/solution_architecture.md) +in the [cwa-documentation](https://github.com/corona-warn-app/cwa-documentation) repository. ## Overview @@ -13,18 +18,19 @@ does not mean automatically that all features will be implemented. Main driver f Find the latest specifications of Google/Apple here: -- [Google Spec](https://static.googleusercontent.com/media/www.google.com/en//covid19/exposurenotifications/pdfs/Exposure-Key-File-Format-and-Verification.pdf) -- [Apple Spec](https://developer.apple.com/documentation/exposurenotification/setting_up_an_exposure_notification_server?changes=latest_beta) - +- [Exposure Key Export File Format and Verification](https://static.googleusercontent.com/media/www.google.com/en//covid19/exposurenotifications/pdfs/Exposure-Key-File-Format-and-Verification.pdf) +- [Setting Up an Exposure Notification Server (Apple)](https://developer.apple.com/documentation/exposurenotification/setting_up_an_exposure_notification_server?changes=latest_beta) +- [Apple Framework Specifications](https://developer.apple.com/documentation/exposurenotification?changes=latest) +- [Google Framework Specifications (1.3.2)](https://static.googleusercontent.com/media/www.google.com/en//covid19/exposurenotifications/pdfs/Android-Exposure-Notification-API-documentation-v1.3.2.pdf) On a high level, the application consists of two main parts, as shown below. ![Overview Diagram](./images/v4.png) -1. CWA Backend: Handles submission and aggregation/distribution of diagnosis keys and configuration files. -2. Verification Backend: Deals with test result verification and issues TANs. This backend is managed and deployed -separately. The repository for those components will be opened up to the community soon. +1. CWA Server: Handles submission and aggregation/distribution of diagnosis keys and configuration files. +2. Verification Server: Deals with test result retrieval and verification (including issuing TANs). +The components regarding the verification are managed and deployed separately. -This document outlines the CWA backend components, which are part of this repository. For the full architectural +This document outlines the CWA Server components, which are part of this repository. For the full architectural overview, check out the [solution architecture](https://github.com/corona-warn-app/cwa-documentation/blob/master/solution_architecture.md). ## Integration with Other Systems @@ -33,8 +39,8 @@ overview, check out the [solution architecture](https://github.com/corona-warn-a All mobile app relevant files will be stored on the S3 Object Store. Those files are: -- Aggregated files containing the diagnostic keys, which were reported in a specific interval (e.g. hourly). -- Daily aggregated files containing the exposure keys, which were reported for the respective days. +- Aggregated files containing the diagnosis keys, which were reported in a specific interval (e.g. hourly). +- Daily aggregated files containing the diagnosis keys, which were reported for the respective days. - Configuration files containing the [exposure configuration](https://developer.apple.com/documentation/exposurenotification/enexposureconfiguration) and CWA mobile app config. - Additional files regarding meta information of available files/structures/etc. @@ -42,55 +48,57 @@ The files will be pushed to an S3 compliant object store whenever new files beco The mobile application will use a CDN for fetching files, which mirrors all files as a transparent proxy present in the object store. -### Verification Backend +### Verification Server -Note: The verification backend design will be available soon as discussions are ongoing. - -The verification backend supports the user's journey beginning at scanning the QR code printed +The verification server supports the user's journey beginning at scanning the QR code printed on the documentation of the SARS-CoV-2 test until the upload of diagnosis keys when the user was tested positive. Testing -labs will update the Verification Backend when a SARS-CoV-2 test result is positive. Since the QR code -is linked to a test, the mobile application will be able to fetch a test-positive notification from -the Verification Backend, along with a TAN. The TAN will be used as an authorization token when +labs will provide the results of SARS-CoV-2 tests to the [test result server](https://github.com/corona-warn-app/cwa-testresult-server), +which in turn provides an interface to the [verification server](https://github.com/corona-warn-app/cwa-verification-server). +Since the GUID contained in the QR code is linked to a test, the mobile application is able to fetch the results from +the verification server and provide a notification to the user. After users have given their consent to upload their diagnosis key, +a TAN is fetched from the verification server. This TAN will be used as an authorization token when the user uploads the diagnosis keys of the past 14 days. -Therefore, from a CWA Backend perspective, the Verification Backend provides an endpoint for TAN verification. +Therefore, from a CWA Server perspective, the Verification Server provides an endpoint for TAN verification. + +A detailed description of the process can be found in the chapter ["Retrieval of lab results and verification process"](https://github.com/corona-warn-app/cwa-documentation/blob/master/solution_architecture.md#retrieval-of-lab-results-and-verification-process) of the solution architecture document. ## Security ### Endpoint Protection -The CWA backend exposes only one endpoint – the submission endpoint. +The CWA Server exposes only one endpoint – the submission endpoint. The endpoint is public (unauthenticated), and authorization for calls is granted to users who are passing a valid TAN. -The TAN verification cannot be done on CWA backend, but the task is delegated to the Verification Backend (see Verification Backend chapter in Integration with other Systems). +The TAN verification cannot be done on CWA Server, but the task is delegated to the verification server (see Verification Server chapter in *Integration with other Systems*). ### Authenticity -All files published by CWA will be digitally signed by CWA. +All files published by CWA will be digitally signed by CWA. This ensures that clients can trust the file they have received from the third-party CDN. -The proto files from Google/Apple already specify how signing should work. Each diagnosis key aggregate file -will be a zip file, containing two files - one carries the actual payload. The other one carries signature information. +The [protocol buffer](https://developers.google.com/protocol-buffers) files from Google/Apple already specify how signing should work. Each diagnosis key aggregate file +is a zip file, containing two files - one carries the actual payload. The other one carries signature information. -CWA will be required to share the public key with Google/Apple, so that the API on the mobile client -will be able verify against it. +CWA is be required to share the public key with Google/Apple, so that the API on the mobile client +is able verify against it. -### Fake Submissions +### Fake Submissions (Plausible Deniability) -In order to protect the privacy of the users, the mobile app needs to send fake submissions from time to time. -The server accepts the incoming calls and will treat them the same way as regular submissions. -The payload and behavior of fake and real requests must be similar, so that 3rd parties are unable to differentiate between those requests. -If a submission request marked as fake is received by the server, the caller will be presented with a successful result. -The CWA backend will not persist the entry on the database and will ensure that fake and real requests take the same amount of total response time, e.g. by delaying the response to the client, if necessary. +In order to protect the privacy of the users, the mobile app needs to send dummy submissions from time to time. +The server accepts the incoming calls and treats them the same way as regular submissions. +The payload and behavior of dummy and real requests must be similar, so that 3rd parties are unable to differentiate between those requests. +If a submission request marked as a dummy is received by the server, the caller will be presented with a successful result. +The CWA server does not persist the entry on the database and ensures that dummy and real requests take the same amount of total response time, e.g. by delaying the response to the client, if necessary. ## Services ### Submission Service The submission service's only task is to process uploaded diagnosis keys and persist them to the database after the TAN has been verified. -The actual task of the verification is handed over to the verification backend, which will provide the verification result back to CWA. +The actual task of the verification is handed over to the verification server, which provides the verification result back to CWA. After verification was successfully done, the diagnosis keys are persisted in the database, and will be published in the next batch. -The payload send by the mobile is defined as: +The payload to be sent by the mobile applications is defined as: ```protobuf message SubmissionPayload { @@ -129,16 +137,16 @@ type Publish struct { } ``` -As of this moment, due to DDP concerns it is unclear whether the device attestation-related fields are in scope of CWA. +Due to concerns regarding data privacy and protection, device attestation is currently not being used by CWA. ### Distribution Service The distribution service's objective is to publish all CWA-related files to the object store, from which -the clients will fetch their data from. There are three types of files. +the clients will fetch their data. There are three types of files. #### Key Export -Key Export files are files, which hold published diagnosis keys from SARS-CoV-2 positive tested users. +Key Export files are files, which hold published diagnosis keys from users that have tested positive for SARS-CoV-2. These files are based on the specification of Google/Apple and are generated in regular intervals. Each interval generates a `.zip` file, containing two files: @@ -147,8 +155,7 @@ Each interval generates a `.zip` file, containing two files: The file structure definition can be found [here](https://github.com/google/exposure-notifications-server/blob/master/internal/pb/export/export.proto). The distribution service is triggered by a CRON scheduler, currently set to 1 hour. However, this -will change, since the Google/Apple API will have rate-limiting in place for a maximum of 15 checks -per day. +will change, since the Exposure Notification APIs have a rate-limiting in place (Apple: 15 files/day, Google 20 API calls/day). In case there is an insufficient number of diagnosis keys for the target interval available, the creation of the file will be skipped in order to prevent attackers from being potentially able to @@ -163,18 +170,28 @@ Configuration files are needed for two use cases: 1. Exposure Configuration: In order to calculate a risk score for each exposure incident, the mobile API requires a list of the following parameters, requiring weights and levels: duration, days, attenuation and transmission. -The function and impact of those parameters is described on the [Apple Exposure Configuration Page](https://developer.apple.com/documentation/exposurenotification/enexposureconfiguration). +The function and impact of those parameters is described on the [Apple Exposure Configuration Page](https://developer.apple.com/documentation/exposurenotification/enexposureconfiguration) and in the chapter [*Risk score calculation*](https://github.com/corona-warn-app/cwa-documentation/blob/master/solution_architecture.md#risk-score-calculation) of the solution architecture document. 2. Mobile App Configuration: Provides configuration values needed for the CWA mobile app, which are not part of the exposure notification framework. These values are required for controlling the application behavior. #### Discovery -Files on CWA may be discovered with the help of index files. There will be one central index file, +Files on CWA may be discovered with the help of index files. There is one central index file, containing all available key export files on the server, separated by new-line. The index will be regenerated whenever new export files are distributed to the object store. ## Data Retention -The retention period is set to 14 days. Therefore, all keys whose _submission date_ is older than 14 days are removed from the system. This includes the persistency layer, as well as CDN/object store published files. -The retention mechanism is enforced by the Distribution Service. +The retention period is set to 14 days. Therefore, all keys whose _submission date_ is older than 14 days are removed from the system. This includes the database persistency layer, as well as files stored on the object store. +The retention mechanism is enforced automatically by the Distribution Service upon each distribution run (multiple runs per day). +No manual trigger or action is required. + +Data is deleted by normal means. For PostgreSQL, the identified rows will be deleted by normal __DELETE__ calls to the database, and +cleaned up when auto vacuuming is executed. +When data deletion is executed on the object store, the object store is instructed to delete all +files with the following prefix: + +`version/v1/diagnosis-keys/country/DE/` + +In which `` stands for the ISO formatted date (e.g. `2012-06-05`), and is before the retention cutoff date (today - 14 days). From 37e2907e7bb4794b5d6b04309cb948e724f5fda3 Mon Sep 17 00:00:00 2001 From: Pit Humke Date: Fri, 5 Jun 2020 07:12:07 +0200 Subject: [PATCH 3/4] Remove android-package from SignatureInfo (#452) --- .../services/distribution/config/DistributionServiceConfig.java | 1 - .../src/main/resources/application-signature-dev.yaml | 1 - .../src/main/resources/application-signature-prod.yaml | 1 - .../archive/decorator/signing/SigningDecoratorTest.java | 2 +- services/distribution/src/test/resources/application.yaml | 1 - 5 files changed, 1 insertion(+), 5 deletions(-) diff --git a/services/distribution/src/main/java/app/coronawarn/server/services/distribution/config/DistributionServiceConfig.java b/services/distribution/src/main/java/app/coronawarn/server/services/distribution/config/DistributionServiceConfig.java index 4393e93c6e..102a29bf49 100644 --- a/services/distribution/src/main/java/app/coronawarn/server/services/distribution/config/DistributionServiceConfig.java +++ b/services/distribution/src/main/java/app/coronawarn/server/services/distribution/config/DistributionServiceConfig.java @@ -379,7 +379,6 @@ public void setSecurityProvider(String securityProvider) { public SignatureInfo getSignatureInfo() { return SignatureInfo.newBuilder() .setAppBundleId(this.getAppBundleId()) - .setAndroidPackage(this.getAndroidPackage()) .setVerificationKeyVersion(this.getVerificationKeyVersion()) .setVerificationKeyId(this.getVerificationKeyId()) .setSignatureAlgorithm(this.getAlgorithmOid()) diff --git a/services/distribution/src/main/resources/application-signature-dev.yaml b/services/distribution/src/main/resources/application-signature-dev.yaml index 40c9fb1a29..4e8627dffa 100644 --- a/services/distribution/src/main/resources/application-signature-dev.yaml +++ b/services/distribution/src/main/resources/application-signature-dev.yaml @@ -3,4 +3,3 @@ services: distribution: signature: app-bundle-id: de.rki.coronawarnapp-dev - android-package: de.rki.coronawarnapp.dev diff --git a/services/distribution/src/main/resources/application-signature-prod.yaml b/services/distribution/src/main/resources/application-signature-prod.yaml index 3d669d97cd..bad267d030 100644 --- a/services/distribution/src/main/resources/application-signature-prod.yaml +++ b/services/distribution/src/main/resources/application-signature-prod.yaml @@ -3,4 +3,3 @@ services: distribution: signature: app-bundle-id: de.rki.coronawarnapp - android-package: de.rki.coronawarnapp diff --git a/services/distribution/src/test/java/app/coronawarn/server/services/distribution/assembly/structure/archive/decorator/signing/SigningDecoratorTest.java b/services/distribution/src/test/java/app/coronawarn/server/services/distribution/assembly/structure/archive/decorator/signing/SigningDecoratorTest.java index b465b9cd8e..9322b9ed1b 100644 --- a/services/distribution/src/test/java/app/coronawarn/server/services/distribution/assembly/structure/archive/decorator/signing/SigningDecoratorTest.java +++ b/services/distribution/src/test/java/app/coronawarn/server/services/distribution/assembly/structure/archive/decorator/signing/SigningDecoratorTest.java @@ -125,7 +125,7 @@ void checkSignatureInfo() { SignatureInfo signatureInfo = signatureList.getSignaturesList().get(0).getSignatureInfo(); assertThat(signatureInfo.getAppBundleId()).isEqualTo("de.rki.coronawarnapp"); - assertThat(signatureInfo.getAndroidPackage()).isEqualTo("de.rki.coronawarnapp"); + assertThat(signatureInfo.getAndroidPackage()).isEmpty(); assertThat(signatureInfo.getSignatureAlgorithm()).isEqualTo("1.2.840.10045.4.3.2"); assertThat(signatureInfo.getVerificationKeyId()).isEqualTo("262"); assertThat(signatureInfo.getVerificationKeyVersion()).isEqualTo("v1"); diff --git a/services/distribution/src/test/resources/application.yaml b/services/distribution/src/test/resources/application.yaml index 41b78c45c4..8f10f8e666 100644 --- a/services/distribution/src/test/resources/application.yaml +++ b/services/distribution/src/test/resources/application.yaml @@ -32,7 +32,6 @@ services: app-config-file-name: app_config signature: app-bundle-id: de.rki.coronawarnapp - android-package: de.rki.coronawarnapp verification-key-id: 262 verification-key-version: v1 algorithm-oid: 1.2.840.10045.4.3.2 From 912628a95e3279e3b2ba72cf19ae41a7b293ba6f Mon Sep 17 00:00:00 2001 From: Ole Lilienthal Date: Fri, 5 Jun 2020 07:25:43 +0200 Subject: [PATCH 4/4] Introduce new version 0.5.10 --- .mvn/maven.config | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.mvn/maven.config b/.mvn/maven.config index a9954b4b44..4ccb765ad3 100644 --- a/.mvn/maven.config +++ b/.mvn/maven.config @@ -1,4 +1,4 @@ --Drevision=0.5.10-SNAPSHOT +-Drevision=0.5.10 -Dlicense.projectName=Corona-Warn-App -Dlicense.inceptionYear=2020 -Dlicense.licenseName=apache_v2