diff --git a/README.md b/README.md index 7f5a6e8..e394fba 100644 --- a/README.md +++ b/README.md @@ -3,53 +3,92 @@ [![chat][badge-chat]][badge-url-chat] ## Synopsis -This middleware enables the use of [Crypt4GH][crypt4gh] files as inputs for tasks that are run in [TES][tes] -implementations (e.g., [funnel][funnel], [TESK][tesk]). +This proof-of-concept middleware enables the use of [Crypt4GH][crypt4gh] files as inputs for tasks that are +run in [TES][tes] implementations (e.g., [funnel][funnel], [TESK][tesk]). + ## Description +Currently, there are no implementations of TES that natively support the use of files encrypted with Crypt4GH. +This middleware supports the use of Crypt4GH files by prepending the list of executors in a TES request with a +decryption executor. This decryption executor decrypts the contents of any Crypt4GH files and places them in a volume +so that subsequent executors can work on the decrypted contents. + +## Implementation Details + +### Middleware +The middleware alters the initial TES request such that a decryption executor and a new volume (`/vol/crypt/`) are added +to the request. Since the decryption executor places all input files in `/vol/crypt/`, all input paths in subsequent +executors are altered to `/vol/crypt/{filename}`. + +request-diagram + +### Decryption +The functionality of the decryption executor lies in [`decrypt.py`][decrypt]. This script moves all input files to a +specified output directory (in this case, `/vol/crypt/`). If a Crypt4GH file is detected and the secret key used to +encrypt it is provided, the executor decrypts the contents of the Crypt4GH file and places it in `/vol/crypt/`. +Subsequent executors then refer to the files in `/vol/crypt/`, not their original locations. + +workflow-diagram + +## Important Considerations +You __should not use this middleware in untrusted environments__, as it requires transmission of secret keys and stores +the decrypted contents of Crypt4GH files on disk. This middleware is meant to be used with a [Trusted Execution +Environment (TEE)][TEE], which allows for the secure transmission and storage of data. Integration with TEEs is a work +in progress. ## Installation +```bash + pip install poetry + poetry install +``` ## Testing -### Prerequisites -This middleware requires a running instance of [proTES](https://github.com/elixir-cloud-aai/proTES) -(running on localhost:8090 by default).\ -\ -Download the required dependencies\ -`pip install requirements_dev.txt` +### Requirements +Tests require a running TES instance and an S3 bucket containing the input files. Instances of [Funnel][funnel] +and [MinIO][minio] were used in development. `TES_URL` in `tests/tasks/test_tasks.py` can be altered depending on the +TES instance being used. + ### Run Tests -Run tests using pytest\ -`pytest tests` +Run tests using pytest. +```bash +poetry run pytest tests +``` ## Contributing - -## Code of Conduct - -## Versioning +This project is a community effort and lives off your contributions, be it in the form of bug reports, feature requests, +discussions, ideas, fixes, or other code changes. Please read these [guidelines][guidelines] if you want to contribute. +And please mind the [code of conduct][code-of-conduct] for all interactions with the community. ## License - This project is distributed under the [Apache License 2.0][badge-license], a copy of which is also available in [`LICENSE`][license]. ## Contact - The project is maintained by [ELIXIR Cloud & AAI][elixir-cloud-aai], a Driver Project of the [Global Alliance for Genomics and Health (GA4GH)][ga4gh], under -the umbrella of the [ELIXIR][elixir] [Compute Platform][elixir-compute]. - -- For filing bug reports, feature requests or other code-related issues, please +the umbrella of the [ELIXIR][elixir] [Compute Platform][elixir-compute]. For filing bug reports, feature requests or other code-related issues, please make use of the project's [issue tracker](https://github.com/elixir-cloud-aai/protes-middleware-crypt4gh/issues). +[![GA4GH logo](images/logo-ga4gh.png)](https://www.ga4gh.org/) +[![ELIXIR logo](images/logo-elixir.png)](https://www.elixir-europe.org/) +[![ELIXIR Cloud & AAI logo](images/logo-elixir-cloud.png)](https://elixir-europe.github.io/cloud/) + [badge-license]: https://img.shields.io/badge/license-Apache%202.0-blue.svg [badge-chat]: https://img.shields.io/static/v1?label=chat&message=Slack&color=ff6994 [badge-url-license]: [badge-url-chat]: https://elixir-cloud.slack.com/archives/C04RLFJNF7U +[code-of-conduct]: https://github.com/elixir-cloud-aai/elixir-cloud-aai/blob/dev/CODE_OF_CONDUCT.md [crypt4gh]: https://www.ga4gh.org/news_item/crypt4gh-a-secure-method-for-sharing-human-genetic-data/ +[decrypt]: https://github.com/elixir-cloud-aai/protes-middleware-crypt4gh/blob/main/crypt4gh_middleware/decrypt.py [elixir]: https://elixir-europe.org/ [elixir-cloud-aai]: https://elixir-cloud.dcc.sib.swiss/ [elixir-compute]: https://elixir-europe.org/platforms/compute [funnel]: https://ohsu-comp-bio.github.io/funnel/ [ga4gh]: https://ga4gh.org/ +[guidelines]: https://elixir-cloud-aai.github.io/guides/guide-contributor/ [license]: LICENSE +[minio]: https://min.io/ +[request]: [tes]: https://github.com/ga4gh/task-execution-schemas [tesk]: https://github.com/elixir-cloud-aai/TESK +[TEE]: https://f1000research.com/posters/13-194 +[workflow]: diff --git a/images/logo-elixir-cloud.png b/images/logo-elixir-cloud.png new file mode 100644 index 0000000..81ddf93 Binary files /dev/null and b/images/logo-elixir-cloud.png differ diff --git a/images/logo-elixir-cloud.svg b/images/logo-elixir-cloud.svg new file mode 100644 index 0000000..7ab05b9 --- /dev/null +++ b/images/logo-elixir-cloud.svg @@ -0,0 +1,109 @@ + + + + + + + + + + image/svg+xml + + + + + + + + + + + + + + + + + diff --git a/images/logo-elixir.png b/images/logo-elixir.png new file mode 100644 index 0000000..e5ebb6c Binary files /dev/null and b/images/logo-elixir.png differ diff --git a/images/logo-elixir.svg b/images/logo-elixir.svg new file mode 100644 index 0000000..012063d --- /dev/null +++ b/images/logo-elixir.svg @@ -0,0 +1,75 @@ + + + + + + + + + + image/svg+xml + + + + + + + + + + diff --git a/images/logo-ga4gh.png b/images/logo-ga4gh.png new file mode 100644 index 0000000..9e7faad Binary files /dev/null and b/images/logo-ga4gh.png differ diff --git a/images/logo-ga4gh.svg b/images/logo-ga4gh.svg new file mode 100644 index 0000000..eec7a2e --- /dev/null +++ b/images/logo-ga4gh.svg @@ -0,0 +1,375 @@ + + + + + + + + + + image/svg+xml + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/images/request.png b/images/request.png new file mode 100644 index 0000000..27d3a1c Binary files /dev/null and b/images/request.png differ diff --git a/images/workflow.png b/images/workflow.png new file mode 100644 index 0000000..5a2b61c Binary files /dev/null and b/images/workflow.png differ