This module deploys the required resources for a multi-workspace Overwatch deployment :
- Storage account and container to store Overwatch ETL output
- Role assignment of the SPN to the storage account created above
- Databricks secret scope backed with AKV to store the secrets needed on the main Overwatch workspace
- Databricks mount point to the container created above
- Databricks Overwatch notebook runner
- Databricks job that will run Overwatch with the notebook above
No requirements.
| Name | Version |
|---|---|
| azuread | n/a |
| azurerm | n/a |
| databricks | n/a |
No modules.
| Name | Type |
|---|---|
| azurerm_role_assignment.data-contributor-role | resource |
| azurerm_storage_account.ow-sa | resource |
| azurerm_storage_data_lake_gen2_filesystem.overwatch-db | resource |
| databricks_job.overwatch | resource |
| databricks_mount.overwatch_db | resource |
| databricks_notebook.overwatch_etl | resource |
| databricks_secret_scope.overwatch-akv | resource |
| azuread_service_principal.overwatch-spn | data source |
| azurerm_databricks_workspace.overwatch-ws | data source |
| azurerm_key_vault.existing-kv | data source |
| azurerm_key_vault_secret.spn-key | data source |
| azurerm_resource_group.rg | data source |
| Name | Description | Type | Default | Required |
|---|---|---|---|---|
| akv_name | Azure Key-Vault name | string |
n/a | yes |
| cron_job_schedule | Cron expression to schedule the Overwatch Job | string |
"0 0 8 * * ?" |
no |
| cron_timezone_id | Timezone for the cron schedule | string |
"Europe/Brussels" |
no |
| databricks_secret_scope_name | Databricks secret scope name (backed by Azure Key-Vault) | string |
n/a | yes |
| latest_dbr_lts | Latest DBR LTS version | string |
n/a | yes |
| overwatch_job_notification_email | Overwatch Job Notification Email | string |
"[email protected]" |
no |
| overwatch_spn_app_id | Azure SPN ID used to create the mount points | string |
n/a | yes |
| overwatch_version | Overwatch library maven version | string |
"overwatch_2.12:0.7.1.0" |
no |
| overwatch_ws_name | Overwatch workspace name | string |
n/a | yes |
| ow_sa_name | The name of the Overwatch ETL storage account | string |
n/a | yes |
| random_string | Random string used as a suffix for the resources names | string |
n/a | yes |
| rg_name | Resource group name | string |
n/a | yes |
| tenant_id | Azure Tenant ID | string |
n/a | yes |
| Name | Description |
|---|---|
| databricks_mount_db_name | Mount point name to the storage account where Overwatch will be writing the results |
| etl_storage_prefix | Overwatch ETL storage prefix, which represents a mount point to the ETL storage account |