Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Materialized Views not updating cache #9959

Merged
merged 20 commits into from
Apr 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
81b713b
initial push of adding cache functions for materialized views
McKnight-42 Apr 16, 2024
86d07e4
update
McKnight-42 Apr 16, 2024
9f400fd
add cache_renamed
McKnight-42 Apr 16, 2024
6f7c01b
Merge branch '1.7.latest' of github.com:dbt-labs/dbt into mcknight/127
McKnight-42 Apr 16, 2024
342d54b
remove current attempt of add cache_renamed
McKnight-42 Apr 17, 2024
51231a6
add checks for add, and drop to see if relation exists in dispatch macro
McKnight-42 Apr 17, 2024
28e1f47
move calls from distpach macro into the default call/postgres specifi…
McKnight-42 Apr 18, 2024
5ed2d74
split up relation into parts to build up to_relation
McKnight-42 Apr 18, 2024
7e56792
move cache_dropped back to the dispatch original dispatch macro
McKnight-42 Apr 19, 2024
d6f61dc
move cache_dropped back to the dispatch original dispatch macro
McKnight-42 Apr 19, 2024
70049f5
readd removed space between macros
McKnight-42 Apr 19, 2024
64c9cab
move cache_added back to distpach macro
McKnight-42 Apr 19, 2024
39f88fe
remove extra curly brace
McKnight-42 Apr 19, 2024
5f0f049
revert get_relation call back get cache_renamed
McKnight-42 Apr 19, 2024
96bd884
move of cache_renamed to get_rename_sql
McKnight-42 Apr 23, 2024
8352986
remove updated cache_changes and remove Relation.MaterializedView fro…
McKnight-42 Apr 23, 2024
ceff6f2
add changelog
McKnight-42 Apr 23, 2024
4135f65
Merge branch '1.7.latest' into mcknight/127
McKnight-42 Apr 23, 2024
ad2c976
add simple test case, and doc string
McKnight-42 Apr 23, 2024
6fa2619
Merge branch 'mcknight/127' of github.com:dbt-labs/dbt into mcknight/127
McKnight-42 Apr 23, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changes/unreleased/Fixes-20240423-172407.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
kind: Fixes
body: remove materialized views from renambeable relation and remove a quote
time: 2024-04-23T17:24:07.249421-05:00
custom:
Author: McKnight-42
Issue: "127"
1 change: 0 additions & 1 deletion plugins/postgres/dbt/adapters/postgres/relation.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,6 @@ class PostgresRelation(BaseRelation):
{
RelationType.View,
RelationType.Table,
RelationType.MaterializedView,
}
)
)
Expand Down
2 changes: 1 addition & 1 deletion plugins/postgres/dbt/include/postgres/macros/adapters.sql
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@
on {{ relation }} {% if index_config.type -%}
using {{ index_config.type }}
{%- endif %}
({{ comma_separated_columns }});
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removing ;as it doubles up with one at end of statement causing error

({{ comma_separated_columns }})
{%- endmacro %}

{% macro postgres__create_schema(relation) -%}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
import pytest
from dbt.tests.util import run_dbt

SEED = """
order_id,customer_id,total_amount,order_date
1,101,50.00,2024-04-01
2,102,75.00,2024-04-02
3,103,100.00,2024-04-03
4,101,30.00,2024-04-04
5,104,45.00,2024-04-05
""".strip()

ORDERS = """
-- models/orders.sql
{{
config(
materialized='materialized_view'
)
}}

SELECT
order_id,
customer_id,
total_amount,
order_date
FROM
{{ ref('source_orders') }}
"""

PRODUCT_SALES = """
{{
config(
materialized='materialized_view'
)
}}

SELECT
order_id,
SUM(total_amount) AS total_sales_amount
FROM
{{ ref('orders') }}
GROUP BY
order_id
"""


class TestPostgresTestRefreshMaterializedView:
"""
this test addresses a issue in postgres around materialized views,
and renaming against a model who has dependent models that are also materialized views
related pr: https://github.com/dbt-labs/dbt-core/pull/9959
"""

@pytest.fixture(scope="class")
def models(self):
yield {"orders.sql": ORDERS, "product_sales.sql": PRODUCT_SALES}

@pytest.fixture(scope="class")
def seeds(self):
yield {"source_orders.csv": SEED}

def test_postgres_refresh_dependent_naterialized_views(self, project):
run_dbt(["seed"])
run_dbt(["run", "--full-refresh"])
run_dbt(["run", "--full-refresh"])
1 change: 0 additions & 1 deletion tests/unit/test_renamed_relations.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,5 @@ def test_renameable_relation():
{
RelationType.View,
RelationType.Table,
RelationType.MaterializedView,
}
)