Merge pull request #107 from networktocode/develop

Schema Enforcer v1.1.0 release

.travis.yml

@@ -1,11 +1,12 @@
 ---
 # Add additional stages in the order of execution here, and then under the job:include: key
 stages:
-  - "lint"
-  - "test"
-  - "test-without-ansible"
-
-if: "type IN (pull_request)"  # Add in "branch" as an option if desired for branch testing as well
+  - name: "lint"
+  - name: "test"
+  - name: "deploy-github"
+    if: "tag IS present"
+  - name: "deploy-pypi"
+    if: "tag IS present"
 
 language: "python"
 python:
@@ -19,31 +20,50 @@ services:
 # Env, before_script, and script for test stage
 env:
   matrix:
-    - "ANSIBLE_VER=2.8.18"
-    - "ANSIBLE_VER=2.9.17"
-    - "ANSIBLE_VER=2.10.5"
+    - "ANSIBLE_VER=2.8.20 ANSIBLE_PACKAGE=ansible"
+    - "ANSIBLE_VER=2.9.20 ANSIBLE_PACKAGE=ansible"
+    - "ANSIBLE_VER=2.10.7 ANSIBLE_PACKAGE=ansible"
+    - "ANSIBLE_VER=2.10.8 ANSIBLE_PACKAGE=ansible-base"
 before_script:
   - "pip install invoke toml"
 script:
-  - "invoke build --nocache"
+  - "invoke build --no-cache"
   - "invoke pytest"
+  - "invoke pytest-without-ansible"
 
 jobs:
   include:
     - stage: "lint"
       before_script:
         - "pip install invoke toml"
-        - "invoke build --nocache"
+        - "invoke build --no-cache"
       script:
         - "invoke black"
-        - "invoke bandit"  # Bandit fails to function on > Py3.8 https://github.com/PyCQA/bandit/issues/639
+        - "invoke bandit"
         - "invoke pydocstyle"
         - "invoke flake8"
         - "invoke yamllint"
         - "invoke pylint"
 
-    - stage: "test-without-ansible"
+    - stage: "deploy-github"
       before_script:
-        - "pip install invoke toml"
-        - "invoke build --without-ansible --nocache"
-      script: "invoke pytest-without-ansible"
+        - "pip install poetry"
+      script:
+        - "poetry version $TRAVIS_TAG"
+        - "poetry build"
+      deploy:
+        provider: "releases"
+        api_key: "$GITHUB_AUTH_TOKEN"
+        file_glob: true
+        file: "dist/*"
+        skip_cleanup: true
+        "on":
+          all_branches: true
+
+    - stage: "deploy-pypi"
+      before_script:
+        - "pip install poetry"
+      script:
+        - "poetry version $TRAVIS_TAG"
+        - "poetry config pypi-token.pypi $PYPI_TOKEN"
+        - "poetry publish --build"

CHANGELOG.md

@@ -1,5 +1,20 @@
 # Changelog
 
+## v1.1.0 - 2021-05-25
+
+### Adds
+
+- [Custom Validators](docs/custom_validators.md)
+- [Automatic mapping of schemas to data files](docs/mapping_data_files_to_schemas.md)
+- Automatic implementation of draft7 format checker to support [IPv4 and IPv6 format declarations](https://json-schema.org/understanding-json-schema/reference/string.html#id12) in a JSON Schema definition [#94](https://github.com/networktocode/schema-enforcer/issues/94)
+
+### Changes
+
+- Removes Ansible as a mandatory dependency [#90](https://github.com/networktocode/schema-enforcer/issues/90)
+- `docs/mapping_schemas.md` renamed to `docs/mapping_data_files_to_schemas.md`
+- Simplifies the invoke tasks used for development
+- Schema enforcer now exits if an invalid schema is found while loading schemas [#99](https://github.com/networktocode/schema-enforcer/issues/99)
+
 ## v1.0.0 - 2021-01-26
 
 Schema Enforcer Initial Release

Dockerfile

@@ -6,15 +6,17 @@ RUN pip install --upgrade pip \
   && pip install poetry
 
 WORKDIR /local
-COPY pyproject.toml /local
-
-ARG ANSIBLE_VER="ignore"
+# Poetry fails install without README.md being copied.
+COPY pyproject.toml poetry.lock README.md /local/
+COPY schema_enforcer /local/schema_enforcer
 
 RUN poetry config virtualenvs.create false \
-  && poetry install --no-interaction --no-ansi \
-  # If ANSIBLE_VER is set (not default), uninstall the ansible version poetry installed and install the declared ansible version.
-  && if [ ! "$ANSIBLE_VER" = "ignore" ]; then pip uninstall -yq ansible ansible-base && pip install ansible==$ANSIBLE_VER; fi
-
-FROM base as without_ansible
+  && poetry install --no-interaction --no-ansi
 
-RUN pip uninstall -yq ansible ansible-base
+# -----------------------------------------------------------------------------
+# Defines stage with ansible installed
+# -----------------------------------------------------------------------------
+FROM base as with_ansible
+ARG ANSIBLE_PACKAGE
+ARG ANSIBLE_VER
+RUN pip install $ANSIBLE_PACKAGE==$ANSIBLE_VER

README.md

@@ -29,7 +29,7 @@ Schema Enforcer requires that two different elements be defined by the user:
 - Schema Definition Files: These are files which define the schema to which a given set of data should adhere.
 - Structured Data Files: These are files which contain data that should adhere to the schema defined in one (or multiple) of the schema definition files.
 
-> Note: Data which needs to be validated against a schema definition can come in the form of Structured Data Files or Ansible host vars. In the interest of brevity and simplicity, this README.md contains discussion only of Structured Data Files -- for more information on how to use `schema-enforcer` with ansible host vars, see [the ansible_command README](docs/ansible_command.md)
+> Note: Data which needs to be validated against a schema definition can come in the form of Structured Data Files or Ansible host vars. Ansible is not installed by default when schema-enforcer is installed. In order to use Ansible features, ansible must already be available or must be declared as an optional dependency when schema-enforcer upon installation. In the interest of brevity and simplicity, this README.md contains discussion only of Structured Data Files -- for more information on how to use `schema-enforcer` with ansible host vars, see [the ansible_command README](docs/ansible_command.md)
 
 When `schema-enforcer` runs, it assumes directory hierarchy which should be in place from the folder in which the tool is run.
 
@@ -121,7 +121,7 @@ To run the schema validations, the command `schema-enforcer validate` can be run
 
 ```shell
 bash$ schema-enforcer validate
-schema-enforcer validate            
+schema-enforcer validate
 ALL SCHEMA VALIDATION CHECKS PASSED
 ```
 
@@ -140,14 +140,14 @@ If we modify one of the addresses in the `chi-beijing-rt1/dns.yml` file so that
 
 ```yaml
 bash$ cat chi-beijing-rt1/dns.yml
-# jsonschema: schemas/dns_servers       
+# jsonschema: schemas/dns_servers
 ---
 dns_servers:
   - address: true
   - address: "10.2.2.2"
 ```
 ```shell
-bash$ test-schema validate            
+bash$ test-schema validate
 FAIL | [ERROR] True is not of type 'string' [FILE] ./chi-beijing-rt1/dns.yml [PROPERTY] dns_servers:0:address
 bash$ echo $?
 1
@@ -160,7 +160,7 @@ When a structured data file fails schema validation, `schema-enforcer` exits wit
 Schema enforcer will work with default settings, however, a `pyproject.toml` file can be placed at the root of the path in which `schema-enforcer` is run in order to override default settings or declare configuration for more advanced features. Inside of this `pyproject.toml` file, `tool.schema_enfocer` sections can be used to declare settings for schema enforcer. Take for example the `pyproject.toml` file in example 2.
 
 ```shell
-bash$ cd examples/example2 && tree -L 2         
+bash$ cd examples/example2 && tree -L 2
 .
 ├── README.md
 ├── hostvars
@@ -194,7 +194,8 @@ bash$ cat pyproject.toml
 Detailed documentation can be found in the README.md files inside of the `docs/` directory.
 - ["Introducing Schema Enforcer" blog post](https://blog.networktocode.com/post/introducing_schema_enforcer/)
 - [Using a pyproject.toml file for configuration](docs/configuration.md)
+- [Mapping Structured Data Files to Schema Files](docs/mapping_data_files_to_schemas.md)
 - [The `ansible` command](docs/ansible_command.md)
 - [The `validate` command](docs/validate_command.md)
-- [Mapping Structured Data Files to Schema Files](docs/mapping_schemas.md)
 - [The `schema` command](docs/schema_command.md)
+- [Implementing custom validators](docs/custom_validators.md)

docs/ansible_command.md

@@ -9,6 +9,8 @@ The `ansible` command is used to check ansible inventory for adherence to a sche
 
 If all checks pass, `schema-enforcer` will inform the user that all tests have passed.
 
+> NOTE | Schema enforcer does not come with ansible pre-installed, rather it is an optional dependency. The user can install schema enforcer bundled with ansible using one of `pip install schema-enforcer[ansible-base]` or `pip install schema-enforcer[ansible]`. Likewise, if ansible is already installed inside of the active python environment, the ansible package which is already installed will be used.
+
 ## How the inventory is loaded
 
 When the `schema-enforcer ansible` command is run, an ansible inventory is constructed. Each host's properties are extracted from the ansible inventory into a single data structure per host, then this data structure is validated against all applicable schemas. For instance, take a look at the following example:

docs/configuration.md

@@ -26,8 +26,7 @@ schema_file_exclude_filenames = []
 data_file_search_directories = ["./"]
 data_file_extensions = [".json", ".yaml", ".yml"]
 data_file_exclude_filenames = [".yamllint.yml", ".travis.yml"]
-
-ansible_inventory = None
+data_file_automap = true
 
 [tools.schema_enforcer.schema_mapping]
 ```
@@ -42,10 +41,12 @@ The table below enumerates each individual setting, it's expected type, it's def
 | definition_directory | string | "definitions" | The directory in which to search for schema definition references. These definitions are can be referenced by the schema files in the "schema_directory". This directory should be nested in the "main_directory" |
 | schema_directory | string | "schemas" | The directory in which to search for schemas. This directory should be nested in the "main_directory" |
 | test_directory | string | "tests" | The directory in which to search for valid and invalid unit tests for schemas |
+| validator_directory | string | "validators" | The directory in which schema-enforcer searches for custom validators |
 | schema_file_extensions | list | [".json", ".yaml", ".yml"] | The extensions to use when searching for schema definition files |
 | schema_file_exclude_filenames | list | [] | The list of filenames to exclude when searching for schema files in the `schema_directory` directory |
 | data_file_search_directories | list | ["./"] The paths at which to start searching for files with structured data in them to validate against defined schemas. This path is relative to the directory in which `schema-enforcer` is executed.
 | data_file_extensions | list | [".json", ".yaml", ".yml"] | The extensions to use when searching for structured data files |
 | data_file_exclude_filenames | list | [".yamllint.yml", ".travis.yml"] | The list of filenames to exclude when searching for structured data files |
+| data_file_automap | bool | true | Whether or not to map top level keys in a data file to the top level properties defined in a schema |
 | ansible_inventory | str | None | The ansible inventory file to use when building an inventory of hosts against which to check for schema adherence |
 | schema_mapping | dict | {} | A mapping of structured data file names (keys) to lists of schema IDs (values) against which the data file should be checked for adherence |

docs/custom_validators.md

@@ -0,0 +1,158 @@
+# Implementing custom validators
+
+With custom validators, you can implement business logic in Python. Schema-enforcer will automatically
+load your plugins from the `validator_directory` and run them against your host data.
+
+The validator plugin provides two base classes: ModelValidation and JmesPathModelValidation. The former can be used
+when you want to implement all logic and the latter can be used as a shortcut for jmespath validation.
+
+## BaseValidation
+
+Use this class to implement arbitrary validation logic in Python. In order to work correctly, your Python script must meet
+the following criteria:
+
+1. Exist in the `validator_directory` dir.
+2. Include a subclass of the BaseValidation class to correctly register with schema-enforcer.
+3. Ensure you call `super().__init__()` in your class `__init__` if you override.
+4. Provide a class method in your subclass with the following signature:
+`def validate(data: dict, strict: bool):`
+
+   * Data is a dictionary of variables on a per-host basis.
+   * Strict is set to true when the strict flag is set via the CLI. You can use this to offer strict validation behavior
+   or ignore it if not needed.
+
+The name of your class will be used as the schema-id for mapping purposes. You can override the default schema ID
+by providing a class-level `id` variable.
+
+Helper functions are provided to add pass/fail results:
+
+```
+def add_validation_error(self, message: str, **kwargs):
+    """Add validator error to results.
+    Args:
+      message (str): error message
+      kwargs (optional): additional arguments to add to ValidationResult when required
+    """
+
+def add_validation_pass(self, **kwargs):
+    """Add validator pass to results.
+    Args:
+      kwargs (optional): additional arguments to add to ValidationResult when required
+    """
+```
+In most cases, you will not need to provide kwargs. However, if you find a use case that requires updating other fields
+in the ValidationResult, you can send the key/value pairs to update the result directly. This is for advanced users only.
+
+## JmesPathModelValidation
+
+Use this class for basic validation using [jmespath](https://jmespath.org/) expressions to query specific values in your data. In order to work correctly, your Python script must meet
+the following criteria:
+
+1. Exist in the `validator_directory` dir.
+2. Include a subclass of the JmesPathModelValidation class to correctly register with schema-enforcer.
+3. Provide the following class level variables:
+
+   * `top_level_properties`: Field for mapping of validator to data
+   * `id`: Schema ID to use for reporting purposes (optional - defaults to class name)
+   * `left`: Jmespath expression to query your host data
+   * `right`: Value or a compiled jmespath expression
+   * `operator`: Operator to use for comparison between left and right hand side of expression
+   * `error`: Message to report when validation fails
+
+### Supported operators:
+
+The class provides the following operators for basic use cases:
+
+```
+"gt": int(left) > int(right),
+"gte": int(left) >= int(right),
+"eq": left == right,
+"lt": int(left) < int(right),
+"lte": int(left) <= int(right),
+"contains": right in left,
+```
+
+If you require additional logic or need to compare other types, use the BaseValidation class and create your own validate method.
+
+### Examples:
+
+#### Basic
+```
+from schema_enforcer.schemas.validator import JmesPathModelValidation
+
+class CheckInterface(JmesPathModelValidation):  # pylint: disable=too-few-public-methods
+    top_level_properties = ["interfaces"]
+    id = "CheckInterface"  # pylint: disable=invalid-name
+    left = "interfaces.*[@.type=='core'][] | length([?@])"
+    right = 2
+    operator = "gte"
+    error = "Less than two core interfaces"
+```
+
+#### With compiled jmespath expression
+```
+import jmespath
+from schema_enforcer.schemas.validator import JmesPathModelValidation
+
+
+class CheckInterfaceIPv4(JmesPathModelValidation):  # pylint: disable=too-few-public-methods
+    top_level_properties = ["interfaces"]
+    id = "CheckInterfaceIPv4"  # pylint: disable=invalid-name
+    left = "interfaces.*[@.type=='core'][] | length([?@])"
+    right = jmespath.compile("interfaces.* | length([[email protected]=='core'][].ipv4)")
+    operator = "eq"
+    error = "All core interfaces do not have IPv4 addresses"
+```
+
+## Running validators
+
+Custom validators are run with `schema-enforcer validate` and `schema-enforcer ansible` commands.
+
+You map validators to keys in your data with `top_level_properties` in your subclass or with `schema_enforcer_schema_ids`
+in your data. Schema-enforcer uses the same process to map custom validators and schemas. Refer to the "Mapping Schemas" documentation
+for more details.
+
+### Example - top_level_properties
+
+The CheckInterface validator has a top_level_properties of "interfaces":
+
+```
+class CheckInterface(JmesPathModelValidation):  # pylint: disable=too-few-public-methods
+    top_level_properties = ["interfaces"]
+```
+
+With automapping enabled, this validator will apply to any host with a top-level `interfaces` key in the Ansible host_vars data:
+
+```
+---
+hostname: "az-phx-pe01"
+pair_rtr: "az-phx-pe02"
+interfaces:
+  MgmtEth0/0/CPU0/0:
+    ipv4: "172.16.1.1"
+  Loopback0:
+    ipv4: "192.168.1.1"
+    ipv6: "2001:db8:1::1"
+  GigabitEthernet0/0/0/0:
+    ipv4: "10.1.0.1"
+    ipv6: "2001:db8::"
+    peer: "az-phx-pe02"
+    peer_int: "GigabitEthernet0/0/0/0"
+    type: "core"
+  GigabitEthernet0/0/0/1:
+    ipv4: "10.1.0.37"
+    ipv6: "2001:db8::12"
+    peer: "co-den-p01"
+    peer_int: "GigabitEthernet0/0/0/2"
+    type: "core"
+```
+
+### Example - manual mapping
+
+Alternatively, you can manually map a validator in your Ansible host vars or other data files.
+
+```
+schema_enforcer_automap_default: false
+schema_enforcer_schema_ids:
+  - "CheckInterface"
+```