Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use standard streams #23

Merged
merged 11 commits into from
Oct 7, 2024
Merged

Use standard streams #23

merged 11 commits into from
Oct 7, 2024

Conversation

artfuldev
Copy link
Contributor

@artfuldev artfuldev commented Oct 7, 2024

In this PR, we make use of standard streams (stdin/stdout) for the CLI, and make the results pipe-able. The structure of the JSON also closely mirrors the POST request expectations from the /analyze and /anonymize and /deanonymize endpoints from presidio itself, so it can be used interchangeably.

Example usage:

$ cat sample.txt
My name is Don Stark and my phone number is 212-555-5555
$ cat sample.txt | pii analyze
{
  "text": "My name is Don Stark and my phone number is 212-555-5555\n",
  "analyzer_results": [
    {
      "entity_type": "PERSON",
      "start": 11,
      "end": 20,
      "score": 1.0,
      "analysis_explanation": null,
      "recognition_metadata": {
        "recognizer_identifier": "Flair Analytics_6067929808",
        "recognizer_name": "Flair Analytics"
      }
    },
    {
      "entity_type": "PHONE_NUMBER",
      "start": 44,
      "end": 56,
      "score": 0.75,
      "analysis_explanation": null,
      "recognition_metadata": {
        "recognizer_name": "PhoneRecognizer",
        "recognizer_identifier": "PhoneRecognizer_6071836752"
      }
    }
  ]
}
$ cat sample.txt | pii analyze | pii anonymize
{
  "text": "My name is <PERSON> and my phone number is <PHONE_NUMBER>\n",
  "anonymizer_results": [
    {
      "start": 43,
      "end": 57,
      "entity_type": "PHONE_NUMBER",
      "text": "<PHONE_NUMBER>",
      "operator": "replace"
    },
    {
      "start": 11,
      "end": 19,
      "entity_type": "PERSON",
      "text": "<PERSON>",
      "operator": "replace"
    }
  ]
}
$ cat sample.txt | pii analyze | curl -X POST -H "Content-Type: application/json" --data-binary @- http://localhost:5001/anonymize
{"text": "My name is <PERSON> and my phone number is <PHONE_NUMBER>\n", "items": [{"start": 43, "end": 57, "entity_type": "PHONE_NUMBER", "text": "<PHONE_NUMBER>", "operator": "replace"}, {"start": 11, "end": 19, "entity_type": "PERSON", "text": "<PERSON>", "operator": "replace"}]}%
$ cat sample.txt | pii analyze | pii anonymize --vaulturl "http://127.0.0.1:8200" --vaultkey "orders"
{
  "text": "My name is vault:v1:tGK/Dw64a6/JugzIUw65/44ZMSqFpE5esq9M6Aygh5nQ0oVW2g== and my phone number is vault:v1:Ejfol9OGBHtV+QSgV/ZoLJKiJLTjOSu5EfO2hfQNWcPIoibDjD0CCQ==\n",
  "anonymizer_results": [
    {
      "start": 96,
      "end": 161,
      "entity_type": "PHONE_NUMBER",
      "text": "vault:v1:Ejfol9OGBHtV+QSgV/ZoLJKiJLTjOSu5EfO2hfQNWcPIoibDjD0CCQ==",
      "operator": "vault_encrypt"
    },
    {
      "start": 11,
      "end": 72,
      "entity_type": "PERSON",
      "text": "vault:v1:tGK/Dw64a6/JugzIUw65/44ZMSqFpE5esq9M6Aygh5nQ0oVW2g==",
      "operator": "vault_encrypt"
    }
  ]
}
$ cat sample.txt | pii analyze | pii anonymize --vaulturl "http://127.0.0.1:8200" --vaultkey "orders" | pii deanonymize --vaulturl "http://127.0.0.1:8200" --vaultkey "orders"
{"text": "My name is Don Stark and my phone number is 212-555-5555\n", "items": [{"start": 44, "end": 56, "entity_type": "PHONE_NUMBER", "text": "212-555-5555", "operator": "vault_decrypt"}, {"start": 11, "end": 20, "entity_type": "PERSON", "text": "Don Stark", "operator": "vault_decrypt"}]}

@artfuldev artfuldev requested a review from akshaykarle October 7, 2024 11:18
@artfuldev artfuldev merged commit 7a924b4 into main Oct 7, 2024
1 check passed
@artfuldev artfuldev deleted the use-standard-streams branch October 7, 2024 11:51
@artfuldev
Copy link
Contributor Author

Fixes #20

@artfuldev artfuldev mentioned this pull request Oct 7, 2024
@artfuldev artfuldev linked an issue Oct 7, 2024 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add stdio-based CLI
1 participant