-
Notifications
You must be signed in to change notification settings - Fork 179
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Respect transient
config for dbt Python models
#802
Conversation
@dbeatty10 While this is running, have you thought about what tests would be needed, or how existing testing would cover this change? |
I hadn't given the testing any thought. Just poked around a little bit, and there are a couple tests for python models here. But none of them appeared to verify the underlying table type that Snowflake is using. Tried a quick search for So up to you how to approach the testing here. One option would be to outline the cases we care about and do a set of manual tests to verify those cases. |
* add transient configs * Backwards-compatible handling of `temporary` and `transient` configs for dbt python models * Fix Jinja syntax errors * add test for table_type on python models --------- Co-authored-by: jeremyyeo <[email protected]> Co-authored-by: colin-rogers-dbt <[email protected]> Co-authored-by: Mike Alfare <[email protected]> Co-authored-by: Mike Alfare <[email protected]>
resolves #776
branched from #777
No docs needed
Problem
Python models don't respect
transient
config.temporary
).table_type
parameter isn't specified here.A side problem is that we are currently using the
create_temp_table
parameter which is deprecated.Solution
The changes in this PR will allow Python tables to respect the
transient
config - in line with SQL tables described in (1) above. It also stops using the deprecatedcreate_temp_table
parameter in favor of usingtable_type
instead.This PR adopts the interpretation that
py_write_table
is meant to be a private helper macro of the table materialization. As such, it's a "naive" helper rather than being stand-alone method.Under that interpretation, we could probably justify making a backwards-incompatible change to the method signature. But out of an abundance of caution, this PR keeps
py_write_table
as backwards-compatible. The one caveat is that any usage of the old signature will not respecttransient
config.Alternative solution
Alternatively, we could just go ahead and make the backwards-incompatible change and declare that
py_write_table
has always been a private macro only intended to be called from within thetable
materialization. It will just create whichever type of table it is passed from within the materialization.Checklist