From 070b146906a37ebf6f5e8ec22b4ebb340d3c3afa Mon Sep 17 00:00:00 2001 From: omertzionitreeverse Date: Sat, 20 Jul 2024 16:32:28 -0400 Subject: [PATCH] Added sumary for demo to replace "use cases" section, added explanations under error messages to enhance clarity --- .../hooks-schema-and-pii-validation.ipynb | 30 +++++++++++++++++-- 1 file changed, 27 insertions(+), 3 deletions(-) diff --git a/00_notebooks/hooks-schema-and-pii-validation.ipynb b/00_notebooks/hooks-schema-and-pii-validation.ipynb index cdc6acfbf..ebe89a8c1 100644 --- a/00_notebooks/hooks-schema-and-pii-validation.ipynb +++ b/00_notebooks/hooks-schema-and-pii-validation.ipynb @@ -9,10 +9,8 @@ "\n", "# Using multiple [Lua hooks](https://docs.lakefs.io/howto/hooks/lua.html) in lakeFS (similar to GitHub Actions)\n", "\n", - "Use Cases:\n", "\n", - "1. Don't allow PII data\n", - "2. Don't allow unintended schema changes" + "In this demo, you’ll learn how lakeFS uses Lua hooks to enforce data governance rules for PII protection and schema change prevention. These hooks run pre-merge checks to ensure that only compliant, PII-free data is merged. If schema changes are attempted, the hooks will validate these changes to maintain data governance compliance.\n" ] }, { @@ -566,6 +564,14 @@ "```" ] }, + { + "cell_type": "markdown", + "id": "cdf6ddbb", + "metadata": {}, + "source": [ + "This error highlights how the pre-merge hook prevents merging data containing blocked PII columns, specifically 'user_id'. Next, we'll modify the schema to remove the PII column and reattempt data ingestion, testing the hooks' ability to prevent unauthorized schema changes.\n" + ] + }, { "cell_type": "markdown", "id": "38cab646-5817-4a79-ae68-ca5d29cc3d34", @@ -964,11 +970,29 @@ "```" ] }, + { + "cell_type": "markdown", + "id": "b721ddba", + "metadata": {}, + "source": [ + "By blocking PII columns and preventing unauthorized schema changes, lakeFS ensures data protection. These measures help maintain data integrity and prevent the exposure of sensitive information, securing users' data while streamlining data version control.\n" + ] + }, + { + "cell_type": "markdown", + "id": "7b0a8a06", + "metadata": {}, + "source": [ + "---" + ] + }, { "cell_type": "markdown", "id": "c26a0587-8c74-4cbe-a213-d766ea23cb65", "metadata": {}, "source": [ + "\n", + "\n", "## You can also review all Actions in lakeFS UI\n", "\n", "👉🏻 http://localhost:8000/repositories/schema-and-pii-validation-example/actions\n",