From ecf70314dfa113b7bda5f16a8bc4ea0160bea088 Mon Sep 17 00:00:00 2001 From: Adrian Velonis Date: Fri, 20 Dec 2024 12:31:12 -0600 Subject: [PATCH] PD-5477: CPC overview --- ...hecks-custom-policy-checks-isolation.flsnp | 6 ++++ .../custom-policy-checks/home.htm | 30 ++++++++++++++----- .../custom-policy-checks/tutorial.htm | 2 +- 3 files changed, 30 insertions(+), 8 deletions(-) create mode 100644 Content/Z_Resources/Snippets/note/checks-custom-policy-checks-isolation.flsnp diff --git a/Content/Z_Resources/Snippets/note/checks-custom-policy-checks-isolation.flsnp b/Content/Z_Resources/Snippets/note/checks-custom-policy-checks-isolation.flsnp new file mode 100644 index 000000000..0feed8cf9 --- /dev/null +++ b/Content/Z_Resources/Snippets/note/checks-custom-policy-checks-isolation.flsnp @@ -0,0 +1,6 @@ + + + +

Custom policy checks are not isolated and can interact both with local file systems and network utilities like the targeted database. It is a best practice to review all checks prior to execution to ensure they only affect the intended object(s).

+ + \ No newline at end of file diff --git a/Content/liquibase-pro/policy-checks/custom-policy-checks/home.htm b/Content/liquibase-pro/policy-checks/custom-policy-checks/home.htm index 768a147dd..19dd1697a 100644 --- a/Content/liquibase-pro/policy-checks/custom-policy-checks/home.htm +++ b/Content/liquibase-pro/policy-checks/custom-policy-checks/home.htm @@ -9,23 +9,38 @@

Custom Policy Checks

Custom Policy Checks are Python scripts that allow you run advanced policies using the  Policy Checks framework. Custom policy checks allow you to enforce compliance for a wide array of security, code standards, data quality, and more.

-

While it's possible to configure the behavior of many built-in policy checks using regular expressions, those checks only work for predefined use-cases. Sometimes, they might not be robust enough for your needs. In contrast, you can use custom policy checks to create unique checks for any situation in your workflow.

+

While it's possible to configure the behavior of many built-in policy checks using regular expressions (such as PatternAFollowedByPatternB), those checks may not be robust enough for complex governance requirements. In contrast, you can use custom policy checks to create unique checks for any situation in your workflow.

This is a feature, so you need a License Key to use it.

+

Business benefits

+

Quickstart: Create a custom policy check

-

For a step-by-step tutorial, see Create a Custom Policy Check.

+

For a detailed, step-by-step tutorial, see Create a Custom Policy Check. This tutorial explains:

+

checks framework

The logic of a custom policy check is stored in a Python (.py) script file in your working directory. You can write and modify that script without using commands: it's just a Python file. You should keep the logic for different checks in different Python files.

You can integrate your Python file into by using the same Policy Checks Commands you would use to copy a built-in policy check. Specifically, there is a built-in policy check called CustomChecksTemplate that you can copy and customize to specify your own logic. In the CLI, you specify a short name for your custom check in , the Python script containing the check logic to run, and other configurations like the check's severity level.

Since a custom policy check runs in the policy checks framework, you can deploy custom policy checks into your CI/CD automation the same way as built-in checks. This allows your code to be robust but keeps your processes simple.

+

Python script pseudocode

-

Your Python script contains:

+

Any Python script you use in the custom policy check framework includes:

  1. Imports of useful helper scripts to access the API.
  2. Clearly named variables that call on functions in the helper scripts (for ease of use).
  3. The logic of your custom check; the code for to run against your or database.
  4. A default return code (False).
-

The following examples show you the structure of a custom Python check script:

+

Here's that structure in Python pseudocode:

  • Changelog-scoped pseudocode @@ -60,8 +75,9 @@

    Python script pseudocode

    Remember that your Python script should test a single database object. runs it iteratively across your entire database.

Guidance on writing Python code

+

If you want to get started, see Create a Custom Policy Check. Here are some tips to make writing custom policy checks more successful:

Helper scripts

-

has a public API that you can import in your Python scripts in order to access a library of helper scripts. For example, there are helper scripts to find the author of a and to determine whether a database object is a table.

+

has a public API that you can import in your Python scripts in order to access a library of helper scripts. For example, there are helper scripts to find the author of a and to determine whether a database object is a table. Built-in helper scripts are included when you download the checks extension.

Using helper scripts makes it easier to access information in resources and your database. This way, you can keep your Python scripts concise. For a list of helper scripts and functions you can call on, see Checks Python API Helper Scripts.

Looping

When you execute the checks run command, always runs every enabled policy check against every object the check targets. This works the same for custom policy checks. For example, runs any custom policy check with a --checks-scope setting of:

@@ -69,7 +85,7 @@

Looping

  • changelog: once per in that . If the uses include or includeAll to reference child s, the script also runs once per in each of those s.
  • database: once per database object. generates a database snapshot once and then runs your script against every object in the snapshot. runs your check against the schema(s) you specify.
    • -

    If you write a custom policy check that is individually performance-heavy, and you have a very large or database, may take a long time to run your custom check iteratively.

    +

    If you write a custom policy check that is individually performance-heavy, and you have a very large or database, may take a long time to run your custom check against all objects.

    Example custom policy checks

    For executable custom policy check samples, see this documentation page: <link to page>

    @@ -77,7 +93,7 @@

    Example custom policy checks

    Limitations

    • When you run checks run --checks-scope=database, generates a database snapshot. If your database contains a very large number of complex objects, the snapshot command may take a long time to execute.
      • If you experience memory limitations while trying to generate the snapshot for database-scoped checks, see Memory Limits of Inspecting Large Schemas.
      • If you want to manually create a snapshot to run database-scoped checks against an offline version of your database, see the instructions on checks-scope.
      • -
    • As of 4.30.0, you can only use custom policy checks to analyze your and database. You cannot use custom policy checks against other resources like properties files and flow files.
      • +
    • As of 4.30.0, you can only use the custom policy checks framework to analyze your and database. You cannot natively use custom policy checks against other resources like properties files and flow files.
    \ No newline at end of file diff --git a/Content/liquibase-pro/policy-checks/custom-policy-checks/tutorial.htm b/Content/liquibase-pro/policy-checks/custom-policy-checks/tutorial.htm index a24fd6042..aad369c19 100644 --- a/Content/liquibase-pro/policy-checks/custom-policy-checks/tutorial.htm +++ b/Content/liquibase-pro/policy-checks/custom-policy-checks/tutorial.htm @@ -127,7 +127,7 @@

    Run your new check

    To run your custom check, you must use the checks run command. provides additional security configuration parameters for this command to ensure you do not accidentally execute Python code on your database:

    • Custom policy checks are disabled by default. Using the checks run command, you must set --checks-scripts-enabled=true in the CLI or set LIQUIBASE_COMMAND_CHECKS_RUN_CHECKS_SCRIPTS_ENABLED=TRUE via environment variable.
      • -
    • Custom policy check Python scripts can filter to specific directory paths. Using the checks run command, you can set the --checks-scripts-path parameter, LIQUIBASE_COMMAND_CHECKS_RUN_CHECKS_SCRIPTS_PATH environment variable, and other standard methods.
      • When set, the check's Python script must be in the specified path(s) to execute successfully.
      • If you don't set a script path, accepts any script path.

      Custom policy checks are not isolated and can interact with both local file systems and network utilities like the targeted database. We recommend reviewing these checks prior to execution to ensure they only affect the intended object(s).

      • +
    • Custom policy check Python scripts can filter to specific directory paths. Using the checks run command, you can set the --checks-scripts-path parameter, LIQUIBASE_COMMAND_CHECKS_RUN_CHECKS_SCRIPTS_PATH environment variable, and other standard methods.
      • When set, the check's Python script must be in the specified path(s) to execute successfully.
      • If you don't set a script path, accepts any script path.

    For example, if you enable custom checks via the CLI and want to run all policy checks, including your new check:

    liquibase checks run --checks-scripts-enabled=true

    If you instead only want to run policy checks with the scope database (such as this check), you must set the --checks-scope parameter to database:

    liquibase checks run --checks-scope=database --checks-scripts-enabled=true