-
Notifications
You must be signed in to change notification settings - Fork 43
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Snowflake materialized view fails on 2nd run #14
Comments
@jtcohen6 I noticed this same thing, running dbt |
Thanks for raising the issue @sandopolus, and for confirmation @matt-winkler. I don't love this behavior from Snowflake: Basically:
Back in v0.17, dbt-snowflake switched from using As I see it, we have two options for changes to the
What do you two think? |
@jtcohen6 oof. My instinct was to remove the |
The other potential option to use I tried to override some of the implementation to try and use show views to get more info back. But i was unsuccessful in my attempts. |
@sandopolus That's a fair point! |
@jtcohen6 This not just happens when a model is switched, but every second run -- even if you change nothing.
First run, all fine - MV gets created. This is not a "<1% case". It renders the solution unusable on Snowflake, imho.. |
can confirm the behavior as seen by @ingolevin, every subsequent run after the first returns I can currently work around it as described by removing the |
Materialized Views have been supported in dbt since dbt-core 1.7 (November 2023). See dbt docs: materialized views for more information. The version of materialized views in this experimental repo is is long-overdue for official deprecation -- it has not been meaningfully maintained or improved for over three years (see #11). If you haven't already, you should try out the official implementation! In the case of Snowflake, we implemented Dynamic Tables instead of Materialized Views which works in much the same way but better. Should you encounter an issue, you can then open bug or enhancement ticket on either dbt-core or the adapter repo for your current data platform. For now, I'm going to close this ticket as "wont_do" |
@dataders I think that it is worth considering the list of known limitations on Dynamic Tables. To say that Dynamic Tables are simply "better" than Materialized Views neglects the non-trivial number of cases for which a Materialized View may be the best or only option. In my case, I'd like to use a Materialized View, not a Dynamic Table, to maintain a flattened view on top of an external table containing semi-structured data. Unfortunately, it looks like I will have to write my own adapter as Dynamic Tables cannot reference external tables. I am hopeful that Snowflake will some day combined these two concepts into a single construct as they are so similar, but until then it would be nice if your team just supported both inside of core instead of simply ignoring the Materialized View concept because Dynamic Tables are seen as "better"? |
DBT Version: 0.20.0
If i create a simple materialized view in snowflake and do dbt run it creates the Materialized View fine.
But on the 2nd run of dbt run it fails because it thinks there is a view rather than a materialized view present.
This causes it to try and run the drop view command
I am new to DBT and have been trying to look into it more
It seems like the following query is used to lookup the information about the object
show terse objects in <DB>.<SCHEMA>
When i run this in the snowflake console it is returning info about the objects, but it lists both views and materialized views as just view. Which i assume is causing the problem.
I have been trying to modify the macros and adaptors to work around the problem. But still in the learning phase of this project. So i am bumping into quite a few unknowns at the moment.
The text was updated successfully, but these errors were encountered: