Skip to content

statisticsnorway/pseudo-service

Repository files navigation

Dapla Pseudo Service

(De/)pseudonymization and export endpoints.

Browse the API docs as:

... or parse the Open API specs for yourself 🤓

Examples

Pseudonymize JSON file and stream back the result

curl "${root_url}/pseudonymize/file" \
--header "Authorization: Bearer ${dapla_auth_token}" \
--form 'data=@src/test/resources/data/15k.json' \
--form 'request={
  "targetContentType": "application/json",
  "pseudoConfig": {
    "rules": [
      {
        "name": "allthenumbers",
        "pattern": "**/*nummer",
        "func": "fpe-anychar(secret1)"
      }
    ]
  }
}'

Depseudonymize JSON file and stream back the result as CSV

curl "${root_url}/depseudonymize/file" \
--header "Authorization: Bearer ${dapla_auth_token}" \
--form 'data=@src/test/resources/data/15k-pseudonymized.json' \
--form 'request={
  "targetContentType": "text/csv",
  "pseudoConfig": {
    "rules": [
      {
        "name": "allthenumbers",
        "pattern": "**/*nummer",
        "func": "fpe-anychar(secret1)"
      }
    ]
  }
}'

Depseudonymize JSON file and download a zipped CSV-file

curl "${root_url}/depseudonymize/file" \
--header "Authorization: Bearer ${dapla_auth_token}" \
--form 'data=@src/test/resources/data/15k-pseudonymized.json' \
--form 'request={
  "targetContentType": "text/csv",
  "pseudoConfig": {
    "rules": [
      {
        "name": "allthenumbers",
        "pattern": "**/*nummer",
        "func": "fpe-anychar(secret1)"
      }
    ]
  },
  "compression": {
    "password": "kensentme"
  }
}'

Depseudonymize archive with multiple JSON files and download a zipped CSV-file

curl --output depseudonymized.zip "${root_url}/depseudonymize/file" \
--header "Authorization: Bearer ${dapla_auth_token}" \
--form 'data=@src/test/resources/data/multiple-json-files.zip' \
--form 'request={
  "targetContentType": "text/csv",
  "pseudoConfig": {
    "rules": [
      {
        "name": "id",
        "pattern": "**/*identifikator*",
        "func": "fpe-fnr(secret1)"
      }
    ]
  },
  "compression": {
    "password": "kensentme"
  }
}'

A note regarding encrypted archives

Standard zip encryption is weak. Thus, for enhanced security, all compressed archives are password encrypted using AES256. You might need to use a non-standard unzip utility in order to decompress these files. A good alternative is 7zip.

To unzip using 7zip:

7z x <my-archive.zip>

Pseudo rules

Pseudo rules are defined by:

  • name (used only for logging purposes)
  • pattern - glob pattern that matches fields to be (de)/pseudonymized.
  • func - references a pseudo function (such as fpe-anychar, fpe-fnr or fpe-digits). The function references the pseudo secret to be used.

Development

See Makefile for details/examples of common dev tasks.

build-all                      Build all and create docker image
build-mvn                      Build project and install to you local maven repo
build-docker                   Build dev docker image
init-local-config              Creates configuration files in the local directory
run-local                      Run the app locally (without docker)
release-dryrun                 Simulate a release in order to detect any issues
release                        Release a new version. Update POMs and tag the new version in git