From 384066249358358a05dead9b6438369f9267b2da Mon Sep 17 00:00:00 2001 From: Mike Alfare Date: Mon, 4 Dec 2023 14:55:08 -0500 Subject: [PATCH 1/8] add materialized view configuration --- .../resource-configs/bigquery-configs.md | 97 +++++++++++++++++++ 1 file changed, 97 insertions(+) diff --git a/website/docs/reference/resource-configs/bigquery-configs.md b/website/docs/reference/resource-configs/bigquery-configs.md index ffbaa37c059..eabefe90d4e 100644 --- a/website/docs/reference/resource-configs/bigquery-configs.md +++ b/website/docs/reference/resource-configs/bigquery-configs.md @@ -718,3 +718,100 @@ Views with this configuration will be able to select from objects in `project_1. The `grant_access_to` config is not thread-safe when multiple views need to be authorized for the same dataset. The initial `dbt run` operation after a new `grant_access_to` config is added should therefore be executed in a single thread. Subsequent runs using the same configuration will not attempt to re-apply existing access grants, and can make use of multiple threads. + + +## Materialized views + +The BigQuery adapter supports [materialized views](https://cloud.google.com/bigquery/docs/materialized-views-intro) +with the following configuration parameters: + +| Parameter | Type | Default | Change Monitoring Support | Reference | +|------------------------------|----------------------|---------|---------------------------|--------------------------------------------------| +| `cluster_by` | LIST[STRING] | `None` | DROP/CREATE | [Clustering](#clustering-clause) | +| `partition_by` | DICT | `None` | DROP/CREATE | [Partitioning](#partition-clause) | +| `enable_refresh` | BOOLEAN | `True` | ALTER | [Auto-refresh](#auto-refresh) | +| `refresh_interval_minutes` | FLOAT | `30` | ALTER | [Auto-refresh](#auto-refresh) | +| `max_staleness` (in Preview) | INTERVAL | `None` | ALTER | [Auto-refresh](#auto-refresh) | +| `description` | STRING | `None` | ALTER | | +| `labels` | DICT[STRING, STRING] | `None` | ALTER | [Labels](#specifying-labels) | +| `hours_to_expiration` | INTEGER | `None` | ALTER | [Table expiration](controlling-table-expiration) | +| `kms_key_name` | STRING | `None` | ALTER | [KMS encryption](#using-kms-encryption) | + +#### Sample model file: + + + +```sql +{{ config( + materialized='materialized_view', + cluster_by=['', ...], + partition_by={ + 'field': '', + 'data_type': '', + + # only if `data_type` is not 'int64' + 'granularity': '' + + # only if `data_type` is 'int64' + 'range': { + 'start': , + 'end': , + 'interval': , + } + }, + + # auto-refresh options + enable_refresh=, + refresh_interval_minutes=, + max_staleness='', + + # additional options + description='', + labels={'': '', ...}, + hours_to_expiration=, + kms_key_name='', +) }} + +select * from {{ ref('my_base_table') }} +``` + + + +Many of these parameters correspond to their table counterparts and have been linked above. +The set of parameters which are unique to materialized views covers auto-refresh functionality, which is covered [below](#auto-refresh). + +Find more information about these parameters in the BigQuery docs: +- [CREATE MATERIALIZED VIEW statement](https://cloud.google.com/bigquery/docs/reference/standard-sql/data-definition-language#create_materialized_view_statement) +- [materialized_view_option_list](https://cloud.google.com/bigquery/docs/reference/standard-sql/data-definition-language#materialized_view_option_list) + +### Auto-refresh + +| Parameter | Type | Default | Change Monitoring Support | +|-------------------------------|----------|---------|---------------------------| +| `enable_refresh` | BOOLEAN | `True` | ALTER | +| `max_staleness` (in Preview) | INTERVAL | `None` | ALTER | +| `refresh_interval_minutes` | FLOAT | `30` | ALTER | + +BigQuery supports [automatic refresh](https://cloud.google.com/bigquery/docs/materialized-views-manage#automatic_refresh) configuration for materialized views. +By default, a materialized view will automatically refresh within 5 minutes of changes in the base table, but no more frequently than every 30 minutes. +BigQuery only officially supports configuration of the frequency (the "30 minutes" part); +however, there is a feature in preview that allows for configuration of the staleness (the "5 minutes" part). +dbt will monitor these parameters for changes and apply them using an `ALTER` statement. + +Find more information about these parameters in the BigQuery docs: +- [materialized_view_option_list](https://cloud.google.com/bigquery/docs/reference/standard-sql/data-definition-language#materialized_view_option_list) +- [max_staleness](https://cloud.google.com/bigquery/docs/materialized-views-create#max_staleness) + +### Limitations + +As with most data platforms, there are limitations associated with materialized views. Some worth noting include: + +- Materialized view SQL has a [limited feature set](https://cloud.google.com/bigquery/docs/materialized-views-create#supported-mvs) +- Materialized view SQL cannot be updated; the materialized view must go through a `--full-refresh` (DROP/CREATE) +- The `partition_by` clause on a materialized view must match that of the underlying base table +- While materialized views can have descriptions, materialized view *columns* cannot +- Recreating/dropping the base table requires recreating/dropping the materialized view + +Find more information about materialized view limitations in Google's BigQuery [docs](https://cloud.google.com/bigquery/docs/materialized-views-intro#limitations). + + From 0256471955d8480994f515a9e98092317939ea8d Mon Sep 17 00:00:00 2001 From: Mike Alfare Date: Tue, 5 Dec 2023 13:19:21 -0500 Subject: [PATCH 2/8] add required column --- .../resource-configs/bigquery-configs.md | 32 +++++++++---------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/website/docs/reference/resource-configs/bigquery-configs.md b/website/docs/reference/resource-configs/bigquery-configs.md index eabefe90d4e..9d2b4abed75 100644 --- a/website/docs/reference/resource-configs/bigquery-configs.md +++ b/website/docs/reference/resource-configs/bigquery-configs.md @@ -725,17 +725,17 @@ The `grant_access_to` config is not thread-safe when multiple views need to be a The BigQuery adapter supports [materialized views](https://cloud.google.com/bigquery/docs/materialized-views-intro) with the following configuration parameters: -| Parameter | Type | Default | Change Monitoring Support | Reference | -|------------------------------|----------------------|---------|---------------------------|--------------------------------------------------| -| `cluster_by` | LIST[STRING] | `None` | DROP/CREATE | [Clustering](#clustering-clause) | -| `partition_by` | DICT | `None` | DROP/CREATE | [Partitioning](#partition-clause) | -| `enable_refresh` | BOOLEAN | `True` | ALTER | [Auto-refresh](#auto-refresh) | -| `refresh_interval_minutes` | FLOAT | `30` | ALTER | [Auto-refresh](#auto-refresh) | -| `max_staleness` (in Preview) | INTERVAL | `None` | ALTER | [Auto-refresh](#auto-refresh) | -| `description` | STRING | `None` | ALTER | | -| `labels` | DICT[STRING, STRING] | `None` | ALTER | [Labels](#specifying-labels) | -| `hours_to_expiration` | INTEGER | `None` | ALTER | [Table expiration](controlling-table-expiration) | -| `kms_key_name` | STRING | `None` | ALTER | [KMS encryption](#using-kms-encryption) | +| Parameter | Type | Required | Default | Change Monitoring Support | Reference | +|------------------------------|----------------------|----------|---------|---------------------------|--------------------------------------------------| +| `cluster_by` | LIST[STRING] | NO | | DROP/CREATE | [Clustering](#clustering-clause) | +| `partition_by` | DICT | NO | | DROP/CREATE | [Partitioning](#partition-clause) | +| `enable_refresh` | BOOLEAN | NO | `True` | ALTER | [Auto-refresh](#auto-refresh) | +| `refresh_interval_minutes` | FLOAT | NO | `30` | ALTER | [Auto-refresh](#auto-refresh) | +| `max_staleness` (in Preview) | INTERVAL | NO | | ALTER | [Auto-refresh](#auto-refresh) | +| `description` | STRING | NO | | ALTER | | +| `labels` | DICT[STRING, STRING] | NO | | ALTER | [Labels](#specifying-labels) | +| `hours_to_expiration` | INTEGER | NO | | ALTER | [Table expiration](controlling-table-expiration) | +| `kms_key_name` | STRING | NO | | ALTER | [KMS encryption](#using-kms-encryption) | #### Sample model file: @@ -786,11 +786,11 @@ Find more information about these parameters in the BigQuery docs: ### Auto-refresh -| Parameter | Type | Default | Change Monitoring Support | -|-------------------------------|----------|---------|---------------------------| -| `enable_refresh` | BOOLEAN | `True` | ALTER | -| `max_staleness` (in Preview) | INTERVAL | `None` | ALTER | -| `refresh_interval_minutes` | FLOAT | `30` | ALTER | +| Parameter | Type | Required | Default | Change Monitoring Support | +|-------------------------------|----------|----------|---------|---------------------------| +| `enable_refresh` | BOOLEAN | NO | `True` | ALTER | +| `refresh_interval_minutes` | FLOAT | NO | `30` | ALTER | +| `max_staleness` (in Preview) | INTERVAL | NO | | ALTER | BigQuery supports [automatic refresh](https://cloud.google.com/bigquery/docs/materialized-views-manage#automatic_refresh) configuration for materialized views. By default, a materialized view will automatically refresh within 5 minutes of changes in the base table, but no more frequently than every 30 minutes. From 88d681930b0ed629ed3c5e464cc65ec692d4d036 Mon Sep 17 00:00:00 2001 From: Mike Alfare Date: Tue, 5 Dec 2023 13:23:01 -0500 Subject: [PATCH 3/8] add on_configuration_change --- .../resource-configs/bigquery-configs.md | 24 ++++++++++--------- 1 file changed, 13 insertions(+), 11 deletions(-) diff --git a/website/docs/reference/resource-configs/bigquery-configs.md b/website/docs/reference/resource-configs/bigquery-configs.md index 9d2b4abed75..97acb73ce33 100644 --- a/website/docs/reference/resource-configs/bigquery-configs.md +++ b/website/docs/reference/resource-configs/bigquery-configs.md @@ -725,17 +725,18 @@ The `grant_access_to` config is not thread-safe when multiple views need to be a The BigQuery adapter supports [materialized views](https://cloud.google.com/bigquery/docs/materialized-views-intro) with the following configuration parameters: -| Parameter | Type | Required | Default | Change Monitoring Support | Reference | -|------------------------------|----------------------|----------|---------|---------------------------|--------------------------------------------------| -| `cluster_by` | LIST[STRING] | NO | | DROP/CREATE | [Clustering](#clustering-clause) | -| `partition_by` | DICT | NO | | DROP/CREATE | [Partitioning](#partition-clause) | -| `enable_refresh` | BOOLEAN | NO | `True` | ALTER | [Auto-refresh](#auto-refresh) | -| `refresh_interval_minutes` | FLOAT | NO | `30` | ALTER | [Auto-refresh](#auto-refresh) | -| `max_staleness` (in Preview) | INTERVAL | NO | | ALTER | [Auto-refresh](#auto-refresh) | -| `description` | STRING | NO | | ALTER | | -| `labels` | DICT[STRING, STRING] | NO | | ALTER | [Labels](#specifying-labels) | -| `hours_to_expiration` | INTEGER | NO | | ALTER | [Table expiration](controlling-table-expiration) | -| `kms_key_name` | STRING | NO | | ALTER | [KMS encryption](#using-kms-encryption) | +| Parameter | Type | Required | Default | Change Monitoring Support | Reference | +|------------------------------|----------------------|----------|-----------|---------------------------|--------------------------------------------------| +| `on_configuration_change` | STRING | NO | `'apply'` | N/A | | +| `cluster_by` | LIST[STRING] | NO | | DROP/CREATE | [Clustering](#clustering-clause) | +| `partition_by` | DICT | NO | | DROP/CREATE | [Partitioning](#partition-clause) | +| `enable_refresh` | BOOLEAN | NO | `True` | ALTER | [Auto-refresh](#auto-refresh) | +| `refresh_interval_minutes` | FLOAT | NO | `30` | ALTER | [Auto-refresh](#auto-refresh) | +| `max_staleness` (in Preview) | INTERVAL | NO | | ALTER | [Auto-refresh](#auto-refresh) | +| `description` | STRING | NO | | ALTER | | +| `labels` | DICT[STRING, STRING] | NO | | ALTER | [Labels](#specifying-labels) | +| `hours_to_expiration` | INTEGER | NO | | ALTER | [Table expiration](controlling-table-expiration) | +| `kms_key_name` | STRING | NO | | ALTER | [KMS encryption](#using-kms-encryption) | #### Sample model file: @@ -744,6 +745,7 @@ with the following configuration parameters: ```sql {{ config( materialized='materialized_view', + on_configuration_change='{ apply | continue | fail }' cluster_by=['', ...], partition_by={ 'field': '', From f50eb361a0066c9a264361d3bbce909ac0240eb2 Mon Sep 17 00:00:00 2001 From: Mike Alfare Date: Thu, 7 Dec 2023 12:20:56 -0500 Subject: [PATCH 4/8] update to use appropriate style guidelines --- .../resource-configs/bigquery-configs.md | 165 +++++++++++++----- 1 file changed, 126 insertions(+), 39 deletions(-) diff --git a/website/docs/reference/resource-configs/bigquery-configs.md b/website/docs/reference/resource-configs/bigquery-configs.md index 97acb73ce33..f9b447fdc0b 100644 --- a/website/docs/reference/resource-configs/bigquery-configs.md +++ b/website/docs/reference/resource-configs/bigquery-configs.md @@ -725,60 +725,147 @@ The `grant_access_to` config is not thread-safe when multiple views need to be a The BigQuery adapter supports [materialized views](https://cloud.google.com/bigquery/docs/materialized-views-intro) with the following configuration parameters: -| Parameter | Type | Required | Default | Change Monitoring Support | Reference | -|------------------------------|----------------------|----------|-----------|---------------------------|--------------------------------------------------| -| `on_configuration_change` | STRING | NO | `'apply'` | N/A | | -| `cluster_by` | LIST[STRING] | NO | | DROP/CREATE | [Clustering](#clustering-clause) | -| `partition_by` | DICT | NO | | DROP/CREATE | [Partitioning](#partition-clause) | -| `enable_refresh` | BOOLEAN | NO | `True` | ALTER | [Auto-refresh](#auto-refresh) | -| `refresh_interval_minutes` | FLOAT | NO | `30` | ALTER | [Auto-refresh](#auto-refresh) | -| `max_staleness` (in Preview) | INTERVAL | NO | | ALTER | [Auto-refresh](#auto-refresh) | -| `description` | STRING | NO | | ALTER | | -| `labels` | DICT[STRING, STRING] | NO | | ALTER | [Labels](#specifying-labels) | -| `hours_to_expiration` | INTEGER | NO | | ALTER | [Table expiration](controlling-table-expiration) | -| `kms_key_name` | STRING | NO | | ALTER | [KMS encryption](#using-kms-encryption) | - -#### Sample model file: - - +| Parameter | Type | Required | Default | Change Monitoring Support | +|--------------------------------------------------------|------------------------|----------|---------|---------------------------| +| `on_configuration_change` | `` | no | `apply` | n/a | +| [`cluster_by`](#clustering-clause) | `[]` | no | `none` | drop/create | +| [`partition_by`](#partition-clause) | `{}` | no | `none` | drop/create | +| [`enable_refresh`](#auto-refresh) | `` | no | `true` | alter | +| [`refresh_interval_minutes`](#auto-refresh) | `` | no | `30` | alter | +| [`max_staleness`](#auto-refresh) (in Preview) | `` | no | `none` | alter | +| `description` | `` | no | `none` | alter | +| [`labels`](#specifying-labels) | `{: }` | no | `none` | alter | +| [`hours_to_expiration`](#controlling-table-expiration) | `` | no | `none` | alter | +| [`kms_key_name`](#using-kms-encryption) | `` | no | `none` | alter | -```sql + + + + + + +directly to the materialized view in place. + +```yaml +models: + [](/reference/resource-configs/resource-path): + [+](/reference/resource-configs/plus-prefix)[materialized](/reference/resource-configs/materialized): materialized_view + [+](/reference/resource-configs/plus-prefix)on_configuration_change: apply | continue | fail + [+](/reference/resource-configs/plus-prefix)[cluster_by](#clustering-clause): | [] + [+](/reference/resource-configs/plus-prefix)[partition_by](#partition-clause): + - field: + - data_type: timestamp | date | datetime | int64 + # only if `data_type` is not 'int64' + - granularity: hour | day | month | year + # only if `data_type` is 'int64' + - range: + - start: + - end: + - interval: + [+](/reference/resource-configs/plus-prefix)[enable_refresh](#auto-refresh): true | false + [+](/reference/resource-configs/plus-prefix)[refresh_interval_minutes](#auto-refresh): + [+](/reference/resource-configs/plus-prefix)[max_staleness](#auto-refresh): + [+](/reference/resource-configs/plus-prefix)description: + [+](/reference/resource-configs/plus-prefix)[labels](#specifying-labels): {: } + [+](/reference/resource-configs/plus-prefix)[hours_to_expiration](#acontrolling-table-expiration): + [+](/reference/resource-configs/plus-prefix)[kms_key_name](##using-kms-encryption): +``` + + + + + + + + + + +```yaml +version: 2 + +models: + - name: [] + config: + [materialized](/reference/resource-configs/materialized): materialized_view + on_configuration_change: apply | continue | fail + [cluster_by](#clustering-clause): | [] + [partition_by](#partition-clause): + - field: + - data_type: timestamp | date | datetime | int64 + # only if `data_type` is not 'int64' + - granularity: hour | day | month | year + # only if `data_type` is 'int64' + - range: + - start: + - end: + - interval: + [enable_refresh](#auto-refresh): true | false + [refresh_interval_minutes](#auto-refresh): + [max_staleness](#auto-refresh): + description: + [labels](#specifying-labels): {: } + [hours_to_expiration](#acontrolling-table-expiration): + [kms_key_name](##using-kms-encryption): +``` + + + + + + + + + + +```jinja {{ config( materialized='materialized_view', - on_configuration_change='{ apply | continue | fail }' - cluster_by=['', ...], + on_configuration_change="apply" | "continue" | "fail", + cluster_by="" | [""], partition_by={ - 'field': '', - 'data_type': '', + "field": "", + "data_type": "timestamp" | "date" | "datetime" | "int64", # only if `data_type` is not 'int64' - 'granularity': '' + "granularity": "hour" | "day" | "month" | "year, # only if `data_type` is 'int64' - 'range': { - 'start': , - 'end': , - 'interval': , + "range": { + "start": , + "end": , + "interval": , } }, # auto-refresh options - enable_refresh=, + enable_refresh= true | false, refresh_interval_minutes=, - max_staleness='', + max_staleness="", # additional options - description='', - labels={'': '', ...}, - hours_to_expiration=, - kms_key_name='', + description="", + labels={ + "": "", + }, + hours_to_expiration=, + kms_key_name="", ) }} - -select * from {{ ref('my_base_table') }} ``` + + + + Many of these parameters correspond to their table counterparts and have been linked above. The set of parameters which are unique to materialized views covers auto-refresh functionality, which is covered [below](#auto-refresh). @@ -788,11 +875,11 @@ Find more information about these parameters in the BigQuery docs: ### Auto-refresh -| Parameter | Type | Required | Default | Change Monitoring Support | -|-------------------------------|----------|----------|---------|---------------------------| -| `enable_refresh` | BOOLEAN | NO | `True` | ALTER | -| `refresh_interval_minutes` | FLOAT | NO | `30` | ALTER | -| `max_staleness` (in Preview) | INTERVAL | NO | | ALTER | +| Parameter | Type | Required | Default | Change Monitoring Support | +|------------------------------|--------------|----------|---------|---------------------------| +| `enable_refresh` | `` | no | `true` | alter | +| `refresh_interval_minutes` | `` | no | `30` | alter | +| `max_staleness` (in Preview) | `` | no | `none` | alter | BigQuery supports [automatic refresh](https://cloud.google.com/bigquery/docs/materialized-views-manage#automatic_refresh) configuration for materialized views. By default, a materialized view will automatically refresh within 5 minutes of changes in the base table, but no more frequently than every 30 minutes. From 20ae17beeecb7d2eced51a81f1f157421139b372 Mon Sep 17 00:00:00 2001 From: Mike Alfare Date: Fri, 8 Dec 2023 15:34:31 -0500 Subject: [PATCH 5/8] add reference links to config block version --- .../resource-configs/bigquery-configs.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/website/docs/reference/resource-configs/bigquery-configs.md b/website/docs/reference/resource-configs/bigquery-configs.md index f9b447fdc0b..ea20be4fb41 100644 --- a/website/docs/reference/resource-configs/bigquery-configs.md +++ b/website/docs/reference/resource-configs/bigquery-configs.md @@ -827,10 +827,10 @@ models: ```jinja {{ config( - materialized='materialized_view', + [materialized](/reference/resource-configs/materialized)='materialized_view', on_configuration_change="apply" | "continue" | "fail", - cluster_by="" | [""], - partition_by={ + [cluster_by](#clustering-clause)="" | [""], + [partition_by](#partition-clause)={ "field": "", "data_type": "timestamp" | "date" | "datetime" | "int64", @@ -846,17 +846,17 @@ models: }, # auto-refresh options - enable_refresh= true | false, - refresh_interval_minutes=, - max_staleness="", + [enable_refresh](#auto-refresh)= true | false, + [refresh_interval_minutes](#auto-refresh)=, + [max_staleness](#auto-refresh)="", # additional options description="", - labels={ + [labels](#specifying-labels)={ "": "", }, - hours_to_expiration=, - kms_key_name="", + [hours_to_expiration](#acontrolling-table-expiration)=, + [kms_key_name](##using-kms-encryption)="", ) }} ``` From 279238f500dbdef1a30ea098a4d48318757aa0a7 Mon Sep 17 00:00:00 2001 From: Mike Alfare <13974384+mikealfare@users.noreply.github.com> Date: Tue, 12 Dec 2023 18:06:04 -0500 Subject: [PATCH 6/8] Update website/docs/reference/resource-configs/bigquery-configs.md Co-authored-by: Doug Beatty <44704949+dbeatty10@users.noreply.github.com> --- website/docs/reference/resource-configs/bigquery-configs.md | 1 - 1 file changed, 1 deletion(-) diff --git a/website/docs/reference/resource-configs/bigquery-configs.md b/website/docs/reference/resource-configs/bigquery-configs.md index ea20be4fb41..9e61a8d5265 100644 --- a/website/docs/reference/resource-configs/bigquery-configs.md +++ b/website/docs/reference/resource-configs/bigquery-configs.md @@ -752,7 +752,6 @@ with the following configuration parameters: -directly to the materialized view in place. ```yaml models: From 1fc78210e06413cc8722b6d323298cdd92735ffe Mon Sep 17 00:00:00 2001 From: Mike Alfare Date: Wed, 13 Dec 2023 12:04:45 -0500 Subject: [PATCH 7/8] add link to description docs --- .../resource-configs/bigquery-configs.md | 30 +++++++++---------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/website/docs/reference/resource-configs/bigquery-configs.md b/website/docs/reference/resource-configs/bigquery-configs.md index 9e61a8d5265..d47562ec2bb 100644 --- a/website/docs/reference/resource-configs/bigquery-configs.md +++ b/website/docs/reference/resource-configs/bigquery-configs.md @@ -725,18 +725,18 @@ The `grant_access_to` config is not thread-safe when multiple views need to be a The BigQuery adapter supports [materialized views](https://cloud.google.com/bigquery/docs/materialized-views-intro) with the following configuration parameters: -| Parameter | Type | Required | Default | Change Monitoring Support | -|--------------------------------------------------------|------------------------|----------|---------|---------------------------| -| `on_configuration_change` | `` | no | `apply` | n/a | -| [`cluster_by`](#clustering-clause) | `[]` | no | `none` | drop/create | -| [`partition_by`](#partition-clause) | `{}` | no | `none` | drop/create | -| [`enable_refresh`](#auto-refresh) | `` | no | `true` | alter | -| [`refresh_interval_minutes`](#auto-refresh) | `` | no | `30` | alter | -| [`max_staleness`](#auto-refresh) (in Preview) | `` | no | `none` | alter | -| `description` | `` | no | `none` | alter | -| [`labels`](#specifying-labels) | `{: }` | no | `none` | alter | -| [`hours_to_expiration`](#controlling-table-expiration) | `` | no | `none` | alter | -| [`kms_key_name`](#using-kms-encryption) | `` | no | `none` | alter | +| Parameter | Type | Required | Default | Change Monitoring Support | +|-------------------------------------------------------------|------------------------|----------|---------|---------------------------| +| `on_configuration_change` | `` | no | `apply` | n/a | +| [`cluster_by`](#clustering-clause) | `[]` | no | `none` | drop/create | +| [`partition_by`](#partition-clause) | `{}` | no | `none` | drop/create | +| [`enable_refresh`](#auto-refresh) | `` | no | `true` | alter | +| [`refresh_interval_minutes`](#auto-refresh) | `` | no | `30` | alter | +| [`max_staleness`](#auto-refresh) (in Preview) | `` | no | `none` | alter | +| [`description`](/reference/resource-properties/description) | `` | no | `none` | alter | +| [`labels`](#specifying-labels) | `{: }` | no | `none` | alter | +| [`hours_to_expiration`](#controlling-table-expiration) | `` | no | `none` | alter | +| [`kms_key_name`](#using-kms-encryption) | `` | no | `none` | alter | [+](/reference/resource-configs/plus-prefix)[max_staleness](#auto-refresh): - [+](/reference/resource-configs/plus-prefix)description: + [+](/reference/resource-configs/plus-prefix)[description](/reference/resource-properties/description): [+](/reference/resource-configs/plus-prefix)[labels](#specifying-labels): {: } [+](/reference/resource-configs/plus-prefix)[hours_to_expiration](#acontrolling-table-expiration): [+](/reference/resource-configs/plus-prefix)[kms_key_name](##using-kms-encryption): @@ -809,7 +809,7 @@ models: [enable_refresh](#auto-refresh): true | false [refresh_interval_minutes](#auto-refresh): [max_staleness](#auto-refresh): - description: + [description](/reference/resource-properties/description): [labels](#specifying-labels): {: } [hours_to_expiration](#acontrolling-table-expiration): [kms_key_name](##using-kms-encryption): @@ -850,7 +850,7 @@ models: [max_staleness](#auto-refresh)="", # additional options - description="", + [description](/reference/resource-properties/description)="", [labels](#specifying-labels)={ "": "", }, From 776e159ed5fc67e138965b071253ac44eaee54b2 Mon Sep 17 00:00:00 2001 From: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> Date: Wed, 13 Dec 2023 12:36:52 -0500 Subject: [PATCH 8/8] Apply suggestions from code review --- .../resource-configs/bigquery-configs.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/website/docs/reference/resource-configs/bigquery-configs.md b/website/docs/reference/resource-configs/bigquery-configs.md index d47562ec2bb..d3497a02caf 100644 --- a/website/docs/reference/resource-configs/bigquery-configs.md +++ b/website/docs/reference/resource-configs/bigquery-configs.md @@ -866,7 +866,7 @@ models: Many of these parameters correspond to their table counterparts and have been linked above. -The set of parameters which are unique to materialized views covers auto-refresh functionality, which is covered [below](#auto-refresh). +The set of parameters unique to materialized views covers [auto-refresh functionality](#auto-refresh). Find more information about these parameters in the BigQuery docs: - [CREATE MATERIALIZED VIEW statement](https://cloud.google.com/bigquery/docs/reference/standard-sql/data-definition-language#create_materialized_view_statement) @@ -881,9 +881,9 @@ Find more information about these parameters in the BigQuery docs: | `max_staleness` (in Preview) | `` | no | `none` | alter | BigQuery supports [automatic refresh](https://cloud.google.com/bigquery/docs/materialized-views-manage#automatic_refresh) configuration for materialized views. -By default, a materialized view will automatically refresh within 5 minutes of changes in the base table, but no more frequently than every 30 minutes. -BigQuery only officially supports configuration of the frequency (the "30 minutes" part); -however, there is a feature in preview that allows for configuration of the staleness (the "5 minutes" part). +By default, a materialized view will automatically refresh within 5 minutes of changes in the base table, but not more frequently than once every 30 minutes. +BigQuery only officially supports the configuration of the frequency (the "once every 30 minutes" frequency); +however, there is a feature in preview that allows for the configuration of the staleness (the "5 minutes" refresh). dbt will monitor these parameters for changes and apply them using an `ALTER` statement. Find more information about these parameters in the BigQuery docs: @@ -894,11 +894,11 @@ Find more information about these parameters in the BigQuery docs: As with most data platforms, there are limitations associated with materialized views. Some worth noting include: -- Materialized view SQL has a [limited feature set](https://cloud.google.com/bigquery/docs/materialized-views-create#supported-mvs) -- Materialized view SQL cannot be updated; the materialized view must go through a `--full-refresh` (DROP/CREATE) -- The `partition_by` clause on a materialized view must match that of the underlying base table -- While materialized views can have descriptions, materialized view *columns* cannot -- Recreating/dropping the base table requires recreating/dropping the materialized view +- Materialized view SQL has a [limited feature set](https://cloud.google.com/bigquery/docs/materialized-views-create#supported-mvs). +- Materialized view SQL cannot be updated; the materialized view must go through a `--full-refresh` (DROP/CREATE). +- The `partition_by` clause on a materialized view must match that of the underlying base table. +- While materialized views can have descriptions, materialized view *columns* cannot. +- Recreating/dropping the base table requires recreating/dropping the materialized view. Find more information about materialized view limitations in Google's BigQuery [docs](https://cloud.google.com/bigquery/docs/materialized-views-intro#limitations).