KSM-472 Add keeper_get_record action

Added a new action to return all the fields in a record as a dictionary.
The key in the dictionary is the field label, or field type if the label is missing.
The key is normalized to remove any non-alphanumeric characters.
Non-alphanumeric characters replaceds with underscores.

To use this action must be used with the 'register' attribute in order to store
the results in memory. The results can be accessed the same way other stored
structures are stored in Ansbile.

integration/keeper_secrets_manager_ansible/README.md

@@ -5,6 +5,7 @@ This module contains plugins that allow your Ansible automations to use Keeper S
 * `keeper_cache_records` - Generate a cache to use with other actions.
 * `keeper_copy` - Similar to `ansible.builtin.copy`. Uses the KSM vault for the source/content.
 * `keeper_get` - Retrieve secrets from a record.
+* `keeper_get_record` - Retrieve records as a dictionary.
 * `keeper_set` - Update an existing record from Ansible information.
 * `keeper_init` - Initialize a KSM configuration from a one-time access token.
 * `keeper_cleanup` - Remove the cache file, if being used.
@@ -18,6 +19,11 @@ For more information see our official documentation page https://docs.keeper.io/
 
 # Changes
 
+## 1.2.2
+* Add action `keeper_get_record` to return record as a dictionary.
+* Clean up comments.
+* Update pinned KSM SDK version to 16.6.3.
+
 ## 1.2.1
 * Add action `keeper_remove` to remove secrets from the Keeper Vault
 * Update pinned KSM SDK version to 16.6.2.

integration/keeper_secrets_manager_ansible/ansible_galaxy/keepersecurity/keeper_secrets_manager/README.md

@@ -64,9 +64,10 @@ If you omit the `collections` , you will need to use the full plugin name.
 
 * `keepersecurity.keeper_secrets_manager.keeper_cache_records` - Generate a cache to use with other actions.
 * `keepersecurity.keeper_secrets_manager.keeper_copy` - Copy file, or value, from your vault to a remote server.
-* `keepersecurity.keeper_secrets_manager.keeper_get` - Get a value from your vault.
+* `keepersecurity.keeper_secrets_manager.keeper_get` - Get a value from a record.
+* `keepersecurity.keeper_secrets_manager.keeper_get_record` - Get record as a dictionary.
 * `keepersecurity.keeper_secrets_manager.keeper_set` - Set a value of an existing record in your vault.
-* `keepersecurity.keeper_secrets_manager.keeper_create` - Create a new record in your vault.
+* `keepersecurity.keeper_secrets_manager.keeper_create` - Create a new record.
 * `keepersecurity.keeper_secrets_manager.keeper_remove` - Remove a record from your vault.
 * `keepersecurity.keeper_secrets_manager.keeper_password` - Generate a random password.
 * `keepersecurity.keeper_secrets_manager.keeper_cleanup` - Clean up Keeper related files.
@@ -118,8 +119,13 @@ configuration file or even a playbook.
 
 # Changes
 
+## 1.2.2
+* Add action `keeper_get_record` to return entire record as dictionary.
+* Clean up comments in code.
+* Update pinned KSM SDK version to 16.6.3.
+
 ## 1.2.1
-* Add action `keeper_remove` to remove secrets from the Keeper Vault
+* Add action `keeper_remove` to remove secrets from the Keeper Vault.
 * Update pinned KSM SDK version to 16.6.2.
 
 ## 1.2.0

integration/keeper_secrets_manager_ansible/ansible_galaxy/keepersecurity/keeper_secrets_manager/meta/runtime.yml

@@ -5,6 +5,7 @@ action_groups:
     - keeper_cache_records
     - keeper_copy
     - keeper_get
+    - keeper_get_record
     - keeper_set
     - keeper_cleanup
     - keeper_create

integration/keeper_secrets_manager_ansible/ansible_galaxy/tower_execution_environment/requirements.txt

@@ -1,3 +1,3 @@
 importlib_metadata
-keeper-secrets-manager-core>=16.6.2
+keeper-secrets-manager-core>=16.6.3
 keeper-secrets-manager-helper

integration/keeper_secrets_manager_ansible/keeper_secrets_manager_ansible/__init__.py

@@ -20,7 +20,6 @@
 import re
 import json
 import random
-from re import sub
 from enum import Enum
 import traceback
 import pickle
@@ -63,7 +62,7 @@ def get_enum(value):
 
 
 class KeeperAnsible:
-    """ A class containing common method used by the Ansible plugin and also talked to Keeper Python SDK
+    """ A class containing a common method used by the Ansible plugin and also talked to Keeper Python SDK
     """
 
     KEY_PREFIX = "keeper"
@@ -100,7 +99,8 @@ def fail_json(msg, **kwargs):
 
     def __init__(self, task_vars, action_module=None, task_attributes=None, force_in_memory=False):
 
-        """ Build the config used by the Keeper Python SDK
+        """
+        Build the config used by the Keeper Python SDK
 
         The configuration is mainly read from a JSON file.
 
@@ -138,7 +138,7 @@ def __init__(self, task_vars, action_module=None, task_attributes=None, force_in
         self.secret_values = []
 
         def camel_case(text):
-            text = sub(r"([_\-])+", " ", text).title().replace(" ", "")
+            text = re.sub(r"([_\-])+", " ", text).title().replace(" ", "")
             return text[0].lower() + text[1:]
 
         try:
@@ -200,8 +200,9 @@ def camel_case(text):
                 # not to leave config files lying around.
                 in_memory_storage = True
 
-                # If we have parameter with a Base64 config, use it for the config_option and force
+                # If we have a parameter with a Base64 config, use it for the config_option and force
                 # the config to be in memory.
+
                 base64_key = KeeperAnsible.keeper_key(KeeperAnsible.KEY_CONFIG_BASE64)
                 if base64_key in task_vars:
                     config_option = task_vars.get(base64_key)
@@ -219,8 +220,9 @@ def camel_case(text):
                         if keeper_key in task_vars:
                             config_option[camel_key] = task_vars[keeper_key]
 
-                    # Token is the odd ball. we need it to be client key in the SDK config. SDK will remove it
-                    # when it is done.
+                    # The token is the odd ball.
+                    # We need it to be client key in the SDK config.
+                    # SDK will remove it when it is done.
                     token_key = KeeperAnsible.keeper_key(KeeperAnsible.TOKEN_KEY)
                     if token_key in task_vars:
                         config_option[KeeperAnsible.CONFIG_CLIENT_KEY] = task_vars[token_key]
@@ -231,7 +233,7 @@ def camel_case(text):
                     elif token_key in task_vars:
                         config_option[KeeperAnsible.CONFIG_CLIENT_KEY] = task_vars[token_key]
 
-                    # If no variables were passed in throw an error.
+                    # If no variables were passed in, throw an error.
                     if len(config_option) == 0:
                         raise AnsibleError("There is no config file and the Ansible variable contain no config keys."
                                            " Will not be able to connect to the Keeper server.")
@@ -256,7 +258,8 @@ def camel_case(text):
                     elif os.path.isfile(self.config_file) is False:
                         self.config_created = True
 
-                    # Write the variables we have to a JSON file. If we are in here config_option is a dictionary,
+                    # Write the variables we have to a JSON file.
+                    # If we are in here, config_option is a dictionary,
                     # not a Base64 string.
                     with open(self.config_file, "w") as fh:
                         json.dump(config_option, fh, indent=4)
@@ -408,7 +411,7 @@ def get_records_from_vault(self, uids=None, titles=None, encrypt=False):
             if titles is not None:
                 records = self.client.get_secrets()
 
-            # If we are getting uid we need only select amount.
+            # If we are getting uid, we need only select amount.
             else:
                 records = self.client.get_secrets(uids)
         except Exception as err:
@@ -486,8 +489,10 @@ def remove_record(self, uids=None, titles=None, cache=None):
 
     @staticmethod
     def _gather_secrets(obj):
-        """ Walk the secret structure and get values. These should just be str, list, and dict. Warn if the SDK
-        return something different.
+        """
+        Walk the secret structure and get values.
+        These should just be str, list, and dict.
+        Warn if the SDK returns something different.
         """
         result = []
         if type(obj) is str:
@@ -504,7 +509,8 @@ def _gather_secrets(obj):
         return result
 
     def stash_secret_value(self, value):
-        """ Parse the result of the secret retrieval and add values to list of secret values.
+        """
+        Parse the result of the secret retrieval and add values to a list of secret values.
         """
         for secret_value in self._gather_secrets(value):
             if secret_value not in self.secret_values:
@@ -549,7 +555,7 @@ def get_value(self, field_type, key, uid=None, title=None, allow_array=False, ar
 
         self.stash_secret_value(values)
 
-        # If we want the entire array, then just return what we got from the field.
+        # If we want the entire array, then return what we got from the field.
         if allow_array is True:
             return values
 
@@ -571,6 +577,45 @@ def get_value(self, field_type, key, uid=None, title=None, allow_array=False, ar
 
         return value
 
+    def get_dict(self, uid=None, title=None, cache=None, allow=None):
+
+        record = self.get_record(uids=uid, titles=title, cache=cache)
+
+        record_dict = {}
+        for field_section in ["fields", "custom"]:
+            for field in record.dict.get(field_section, []):
+                label = field.get("label")
+                type = field.get("type")
+                value = field.get("value", [])
+                key = label if label is not None else type
+
+                if key is None or key == "":
+                    display.vvvvv("record contains a field without a label or type.")
+                    continue
+
+                # Scrub the label to make it a clean key.
+                # Only allow alphanumerics, remove runs of _, remove _ at the start and end of the key.
+                key = re.sub('[^0-9a-zA-Z]+', '_', key)
+                key = re.sub('_+', '_', key)
+                key = re.sub('^_+', '', key)
+                key = re.sub('_+$', '', key)
+
+                if key == "":
+                    display.vvvvv("record contains a field that has no alphanumeric characters. cannot use this field.")
+                    continue
+
+                if key in record_dict:
+                    count = len([x for x in record_dict if x.startswith(key)])
+                    key = key + "_" + str(count)
+
+                if allow is not None and key not in allow:
+                    continue
+
+                record_dict[key] = value
+                self.stash_secret_value(value)
+
+        return record_dict
+
     def set_value(self, field_type, key, value, uid=None, title=None, cache=None):
 
         record = self.get_record(uids=uid, titles=title, cache=cache)
@@ -589,13 +634,14 @@ def set_value(self, field_type, key, value, uid=None, title=None, cache=None):
     @staticmethod
     def get_field_type_enum_and_key(args):
 
-        """ Get the field type enum and field key in the Ansible args for a task.
+        """
+        Get the field type enum and field key in the Ansible args for a task.
 
-        For a task that, only allowed one of the allowed field, this method will find the type of field and
+        For a task that only allowed one of the allowed fields, this method will find the type of field and
         the key/label for that field.
 
-        If multiple fields types are specified, an error will be thrown. If no fields are found, an error will be
-        thrown.
+        If multiple fields types are specified, an error will be thrown.
+        If no fields are found, an error will be thrown.
 
         The method will return the KeeperFieldType enum for the field type and the name of the field in Keeper that
         the task requires.
@@ -617,37 +663,42 @@ def get_field_type_enum_and_key(args):
         return KeeperFieldType.get_enum(field_type[0]), field_key
 
     def add_secret_values_to_results(self, results):
-        """ If the 'redact' stdout callback is being used, add the secrets to the results dictionary. The redact
-        stdout callback will remove it from the results. It will use value to remove values from stdout.
+        """
+        If the 'redact' stdout callback is being used, add the secrets to the result dictionary.
+        The redacted stdout callback will remove it from the results.
+        It will use value to remove values from stdout.
         """
 
-        # If we are using the redact stdout callback, add the secrets we retrieve to the special key. The redact
-        # stdout callback will make sure the value is not in the stdout.
+        # If we are using the redacted stdout callback, add the secrets we retrieve to the special key.
+        # The redacted stdout callback will make sure the value is not in the stdout.
         if self.has_redact is True:
             results["_secrets"] = self.secret_values
         return results
 
     @staticmethod
     def password_complexity_translation(**kwargs):
         """
-        Generate a password complexity dictionary
+        Generate a password complexity dictionary.
 
-        Password complexity differ from place to place :(
+        Password complexity differs from place to place.
 
-        This is in more tune with the Vault UI since most service just want a specific set of characters, but not
-        a quantity. And some characters are illegal for specific services. Neither the SDK and Vault UI address this.
+        This is in more tune with the Vault UI since most services just want a specific set of characters, but not
+        a quantity.
+        And some characters are illegal for specific services.
+        Neither the SDK and Vault UI address this.
         So this is the third standard.
 
-        kwargs
+        Kwargs
 
         * length - Length of the password
         * allow_lowercase - Allow lowercase letters. Default is True.
         * allow_uppercase - Allow uppercase letters. Default is True.
         * allow_digits - Allow digits. Default is True.
         * allow_symbols - Allow symbols. Default is True
-        * filter_characters - An array of characters not to use. Some servies don't like some characters.
+        * filter_characters - An array of characters not to use. Some services don't like some characters.
 
-        The length is divided by the allowed characters. So with a length of 64, each would get 16 of each character.
+        The length is divided by the allowed characters.
+        So with a length of 64, each would get 16 of all characters.
         If the length cannot be unevenly divided, additional will be added to the first allowed character in the above
         list.
 
@@ -679,7 +730,7 @@ def password_complexity_translation(**kwargs):
         complexity = {
             "length": length,
 
-            # This is not part of the standard, however it's important because some service will not accept certain
+            # This is not part of the standard, however, it's important because some service will not accept certain
             # characters.
             "filter_characters": filter_characters
         }
@@ -713,7 +764,7 @@ def replacement_char(**kwargs):
 
         attempt = 0
         while True:
-            # If allow everything, then just get a lowercase letter
+            # If allow everything, then get a lowercase letter
             if all_true is True:
                 new_char = "abcdefghijklmnopqrstuvwxyz"[random.randint(0, 25)]
 
@@ -736,7 +787,8 @@ def replacement_char(**kwargs):
             if new_char not in kwargs.get("filter_characters"):
                 break
 
-            # Ok, some user might go crazy and filter out every letter, digit, and symbol and cause an infinite loop.
+            # Ok, some user might go overboard and filter out every letter, digit, and symbol
+            # and cause an infinite loop.
             # If we can't find a good character after 25 attempts, error out.
             attempt += 1
             if attempt > 25:
@@ -759,7 +811,8 @@ def generate_password(**kwargs):
         # The SDK generate_password doesn't know what the filter_characters is, remove it for now.
         filter_characters = kwargs.pop("filter_characters", None)
 
-        # The SDK uses these a params, record complexity use the ones on the right. Translate them.
+        # The SDK uses these a params, record complexity uses the ones on the right.
+        # Translate them.
         kwargs["uppercase"] = kwargs.pop("caps", None)
         kwargs["special_characters"] = kwargs.pop("special", None)
 

integration/keeper_secrets_manager_ansible/keeper_secrets_manager_ansible/plugins/action_plugins/keeper_get.py

@@ -103,16 +103,16 @@
 
 EXAMPLES = r'''
 - name: Get login name
-  keeper_copy:
+  keeper_get:
     uid: XXX
     field: login
   register: my_login_value
 - name: Get login name via notation
-  keeper_copy:
+  keeper_get:
     notation: XXX/field/login
   register: my_login_value
 - name: Get custom field
-  keeper_copy:
+  keeper_get:
     uid: XXX
     custom_field: Custom Label
   register: my_custom_value