Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SK-1038 BYOT Strict Mode in Insert #106

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@

All notable changes to this project will be documented in this file.

## [1.14.0] - 2023-09-11
## Added
- Support for different BYOT modes in Insert method.

## [1.13.0] - 2023-09-04
### Added
- Added new Query method.
Expand Down
81 changes: 77 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,17 +149,24 @@ All Vault APIs must be invoked using a client instance.

### Insert data into the vault

To insert data into your vault use the `insert(records: dict, options: InsertOptions)` method. The `records` parameter is a dictionary that requires a `records` key and takes an array of records to insert into the vault. The `options` parameter takes a dictionary of optional parameters for the insertion. This includes an option to return tokenized data, upsert records and continue on error.
To insert data into your vault use the `insert(records: dict, options: InsertOptions)` method. The `records` parameter is a dictionary that requires a `records` key and takes an array of records to insert into the vault. The `options` parameter takes a dictionary of optional parameters for the insertion. The optional parameter includes:

Insert Options
```python
# Optional, indicates whether you return tokens for inserted data. Defaults to 'true'.
tokens: bool
# Optional, indicates Upsert support in the vault.
upsert: [UpsertOption]
# Optional, decides whether to continue if error encountered or not
continueOnError: bool
# Optional, decides the byot mode
byot: Skyflow.BYOT
```

Notes:
- For supported `byot` types, see [Skyflow.BYOT](#byot)
- `byot` defaults to `BYOT.DISABLE`

Insert call schema
```python
from skyflow.vault import InsertOptions, UpsertOption
Expand Down Expand Up @@ -216,13 +223,14 @@ Skyflow returns tokens for the record you just inserted.
"fields": {
"cardNumber": "f3907186-e7e2-466f-91e5-48e12c2bcbc1",
"cvv": "1989cb56-63da-4482-a2df-1f74cd0dd1a5",
"skyflow_id": "d863633c-8c75-44fc-b2ed-2b58162d1117"
},
}
]
}
```

**Insert call [example](https://github.com/skyflowapi/skyflow-python/blob/main/samples/insert_with_continue_on_error_sample.py) with continueOnError option**
**Insert call [example](https://github.com/skyflowapi/skyflow-python/blob/main/samples/insert_with_continue_on_error_sample.py) with `continueOnError` option**

```python
client.insert(
Expand Down Expand Up @@ -256,22 +264,79 @@ Sample Response
"table": "cards",
"fields": {
"card_number": "f37186-e7e2-466f-91e5-48e2bcbc1",
"full_name": "1989cb56-63a-4482-adf-1f74cd1a5"
"full_name": "1989cb56-63a-4482-adf-1f74cd1a5",
"skyflow_id": "3daf1a7f-bc7f-4fc9-8c56-a6e4e93231e6"
}
}
],
"errors": [
{
"error": {
"code": 404,
"description": "Object Name pii_field was not found for Vault - requestId : id1234"
"description": "Object Name pii_field was not found for Vault - requestId : af4aad11-f276-474d-b626-c75c8b35d49e"
}
}
]
}

```

**Insert call [example](https://github.com/skyflowapi/skyflow-python/blob/main/samples/insert_byot_sample.py) with `byot` option**

```python
client.insert(
{
"records": [
{
"table": "cards",
"fields": {
"card_number": "4111111111111111",
"full_name": "john doe"
},
"tokens": {
"card_number": "8486-6981-3757-9998"
}
},
{
"table": "cards",
"fields": {
"card_number": "4242424242424200"
"full_name": "jane doe"
},
"tokens": {
"card_number": "9426-6911-3750-7998"
}
}
]
}, InsertOptions(tokens=True, byot=BYOT.ENABLE)
)
```

Sample Response:

```json
{
"records": [
{
"table": "cards",
"fields": {
"card_number": "8486-6981-3757-9998",
"full_name": "1989cb56-63a-4482-adf-1f74cd1a5",
"skyflow_id": "3daf1a7f-bc7f-4fc9-8c56-a6e4e93231e6"
}
},
{
"table": "cards",
"fields": {
"card_number": "9426-6911-3750-7998",
"full_name": "1041c13c-1a3c-4a09-a556-686d8bba307b",
"skyflow_id": "9a567659-fe5c-4ecc-80aa-059a9644138e"
}
}
],
}
```

**Insert call [example](https://github.com/skyflowapi/skyflow-python/blob/main/samples/insert_upsert_sample.py) with `upsert` options**

```python
Expand Down Expand Up @@ -302,12 +367,20 @@ Skyflow returns tokens, with `upsert` support, for the record you just inserted.
"fields": {
"cardNumber": "f3907186-e7e2-466f-91e5-48e12c2bcbc1",
"cvv": "1989cb56-63da-4482-a2df-1f74cd0dd1a5",
"skyflow_id": "60b32788-12ec-4dd7-9da5-0146c3afbe11"
},
}
]
}
```

#### BYOT
There are 3 accepted values in Skyflow.BYOT:

- `DISABLE`
- `ENABLE`
- `ENABLE_STRICT`

### Detokenize

To retrieve tokens from your vault, you can use the `Detokenize(records: dict, options: DetokenizeOptions)` method.The records parameter takes a dictionary that contains the `records` key that takes an array of records to return. The options parameter is a `DetokenizeOptions` object that provides further options, including `continueOnError` operation, for your detokenize call, as shown below:
Expand Down
41 changes: 41 additions & 0 deletions samples/insert_byot_sample.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
'''
Copyright (c) 2022 Skyflow, Inc.
'''
from skyflow.errors import SkyflowError
from skyflow.service_account import generate_bearer_token, is_expired
from skyflow.vault import Client, InsertOptions, Configuration, BYOT

# cache token for reuse
bearerToken = ''

def token_provider():
global bearerToken
if is_expired(bearerToken):
bearerToken, _ = generate_bearer_token('<YOUR_CREDENTIALS_FILE_PATH>')
return bearerToken


try:
config = Configuration(
'<YOUR_VAULT_ID>', '<YOUR_VAULT_URL>', token_provider)
client = Client(config)

options = InsertOptions(tokens=True, byot=BYOT.ENABLE)

data = {
"records": [
{
"table": "<TABLE_NAME>",
"fields": {
"<FIELDNAME>": "<VALUE>"
},
"tokens": {
"<FIELDNAME>": "<VALUE>"
}
}
]
}
response = client.insert(data, options=options)
print('Response:', response)
except SkyflowError as e:
print('Error Occurred:', e)
8 changes: 8 additions & 0 deletions tests/vault/test_insert.py
Original file line number Diff line number Diff line change
Expand Up @@ -584,6 +584,14 @@ def testValidUpsertOptions(self):
self.assertEqual(
e.message, SkyflowErrorMessages.EMPTY_UPSERT_OPTION_COLUMN.value % 0)

def testInvalidByotModeTypePassed(self):
try:
options = InsertOptions(byot='BYOT.DISABLE')
getInsertRequestBody(self.data, options)
self.fail("Should have thrown an error")
except SkyflowError as e:
self.assertEqual(e.message, SkyflowErrorMessages.INVALID_BYOT_TYPE.value % (type(options.byot)))

def testTokensPassedWithByotModeDisable(self):
try:
options = InsertOptions(byot=BYOT.DISABLE)
Expand Down