Skip to content

Commit

Permalink
Merge pull request #216 from treeverse/doc-improvement-hooks
Browse files Browse the repository at this point in the history
Documentation improvement for hooks-schema-and-pii-validation.ipynb
  • Loading branch information
omertzionitreeverse authored Jul 23, 2024
2 parents 156b656 + 070b146 commit 646f778
Showing 1 changed file with 27 additions and 3 deletions.
30 changes: 27 additions & 3 deletions 00_notebooks/hooks-schema-and-pii-validation.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,8 @@
"\n",
"# Using multiple [Lua hooks](https://docs.lakefs.io/howto/hooks/lua.html) in lakeFS (similar to GitHub Actions)\n",
"\n",
"Use Cases:\n",
"\n",
"1. Don't allow PII data\n",
"2. Don't allow unintended schema changes"
"In this demo, you’ll learn how lakeFS uses Lua hooks to enforce data governance rules for PII protection and schema change prevention. These hooks run pre-merge checks to ensure that only compliant, PII-free data is merged. If schema changes are attempted, the hooks will validate these changes to maintain data governance compliance.\n"
]
},
{
Expand Down Expand Up @@ -566,6 +564,14 @@
"```"
]
},
{
"cell_type": "markdown",
"id": "cdf6ddbb",
"metadata": {},
"source": [
"This error highlights how the pre-merge hook prevents merging data containing blocked PII columns, specifically 'user_id'. Next, we'll modify the schema to remove the PII column and reattempt data ingestion, testing the hooks' ability to prevent unauthorized schema changes.\n"
]
},
{
"cell_type": "markdown",
"id": "38cab646-5817-4a79-ae68-ca5d29cc3d34",
Expand Down Expand Up @@ -964,11 +970,29 @@
"```"
]
},
{
"cell_type": "markdown",
"id": "b721ddba",
"metadata": {},
"source": [
"By blocking PII columns and preventing unauthorized schema changes, lakeFS ensures data protection. These measures help maintain data integrity and prevent the exposure of sensitive information, securing users' data while streamlining data version control.\n"
]
},
{
"cell_type": "markdown",
"id": "7b0a8a06",
"metadata": {},
"source": [
"---"
]
},
{
"cell_type": "markdown",
"id": "c26a0587-8c74-4cbe-a213-d766ea23cb65",
"metadata": {},
"source": [
"\n",
"\n",
"## You can also review all Actions in lakeFS UI\n",
"\n",
"👉🏻 http://localhost:8000/repositories/schema-and-pii-validation-example/actions\n",
Expand Down

0 comments on commit 646f778

Please sign in to comment.