Skip to content

Commit

Permalink
Allow for local compile of incremental models. (duneanalytics#2403)
Browse files Browse the repository at this point in the history
* base is_incremental

* add force-incremental variable
  • Loading branch information
0xRobin authored and jeff-dude committed Jan 9, 2023
1 parent 61428a1 commit bac63f2
Show file tree
Hide file tree
Showing 2 changed files with 26 additions and 2 deletions.
18 changes: 18 additions & 0 deletions macros/dune/is_incremental.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
{% macro is_incremental() %}
{#-- allow for command line compile of incremental models #}
{#-- Usage: dbt compile --vars '{force-incremental: True}' #}
{% if var('force-incremental', False) %}
{{ return(True) }}
{% endif %}

{#-- do not run introspective queries in parsing #}
{% if not execute %}
{{ return(False) }}
{% else %}
{% set relation = adapter.get_relation(this.database, this.schema, this.table) %}
{{ return(relation is not none
and relation.type == 'table'
and model.config.materialized == 'incremental'
and not should_full_refresh()) }}
{% endif %}
{% endmacro %}
10 changes: 8 additions & 2 deletions macros/dune/macros_schema.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ macros:
description: "Custom schema name"
- name: node
type: column name or expression
description: "Node"
description: "Node"

- name: generate_alias_name
description: "This overrides the dbt-core function for setting aliases. For PR test, the alias is set to the user schema and file name. "
Expand Down Expand Up @@ -38,4 +38,10 @@ macros:
- name: spark__create_csv_table
description: >
Same as create_table_as but for seeds with spark package
Same as create_table_as but for seeds with spark package
- name: is_incremental
description: >
Overrides the core is_incremental marco to allow the usage of a force-incremental command line variable.
This enables us to easily get the compiled incremental models.
Example usage: dbt compile --vars '{force-incremental: True}'

0 comments on commit bac63f2

Please sign in to comment.