Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Make warn_unenforced and warn_unsupported constraint options on the project level #10159

Closed
3 tasks done
jsnb-devoted opened this issue May 16, 2024 · 6 comments
Closed
3 tasks done
Labels
enhancement New feature or request

Comments

@jsnb-devoted
Copy link

Is this your first time submitting a feature request?

  • I have read the expectations for open source contributors
  • I have searched the existing issues, and I could not find an existing issue for this feature
  • I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion

Describe the feature

I would love to be able to suppress the warning:

18:38:06  [WARNING]: The constraint type primary_key is not enforced by
snowflake. The constraint will be included in this model's DDL statement, but it
will not guarantee anything about the underlying data. Set 'warn_unenforced:
false' on this constraint to ignore this warning.

for dbt run/build. Our users are aware that the constraint is unenforced so the logs are taking up space.

Describe alternatives you've considered

We are trying to require primary keys on all 3k models -- we could also require that we warn_unenforced an all of those primary key yml keys. That is extra work for for us and it adds 3k lines of bytes to our schema files.

I saw a few potentially related issues that were closed in versions after 1.6: #7067 + #6754 -- but I'm not sure this actually addresses the feature. We are on dbt-core 1.6.7 and dbt-snowflake 1.6.8. Maybe this was already addressed?

Who will this benefit?

Anyone using constraints on Snowflake/BQ/Redshift etc

Are you interested in contributing this feature?

If it is reasonably contained I could take a crack at it with some guidance. I did a cursory read of the constraints code and I couldn't really sort it out.

Anything else?

No response

@jsnb-devoted jsnb-devoted added enhancement New feature or request triage labels May 16, 2024
@dbeatty10
Copy link
Contributor

Thanks for reaching out about this @jsnb-devoted !

Wanna take a look at the silence parameter to the warn_error_options flag that was added to v1.8 and see if it would fit the use case you are describing?

@jtcohen6
Copy link
Contributor

jtcohen6 commented May 16, 2024

Hey @jsnb-devoted !

@dbeatty10 You beat me to it :) I just confirmed that this worked for me locally:

$ dbt run -s model_a --warn-error-options '{"silence": ['ConstraintNotEnforced']}'

This can also be set:

@jsnb-devoted
Copy link
Author

That looks like what I need. I'll add it to the list of things to check out when we make it to 1.8. Thanks yall.

@dbeatty10
Copy link
Contributor

Sure thing @jsnb-devoted !

I'm going to close this as resolved, but please reach out if run into any issues or have any follow-up feedback.

@dbernett-amplify
Copy link

dbernett-amplify commented Oct 10, 2024

@dbeatty10 Happy Thursday!

I've tried this in our project and I can't get it to work. A few things...

  1. I actually don't see the original warning posted above at all anymore (though we definitely used to get it). This is true regardless of whether or not I incorporate any changes into my dbt_project.yml:
[WARNING]: The constraint type primary_key is not enforced by
snowflake. The constraint will be included in this model's DDL statement, but it
will not guarantee anything about the underlying data. Set 'warn_unenforced:
false' on this constraint to ignore this warning.

I definitely have models that include a primary_key constraint and that don't specify warn_unenforced. Like this one:
Screenshot 2024-10-10 at 12 33 34 PM

Did something change with this?

  1. We do still get this warning:
[WARNING]: Constraint types are not supported for view materializations
and will be ignored.  Set 'warn_unsupported: false' on this constraint to ignore
this warning.
  1. I've tried adding the following to my dbt_project.yml:
flags:
  warn_error_options:
    silence:
      - ConstraintNotEnforced
      - ConstraintNotSupported

But it doesn't seem to change anything. The not supported warning above still appears. The not enforced one does not (though, again, it doesn't appear regardless of whether or not I have this in the project.yml).

I also want to note that it's a little tricky to test these in the IDE because they don't show up as warnings when you run or build a model but rather during parsing (which is hidden unless you click the little "ready" pill). But then they show up when you run a job.

On a related note, the warning itself could be more informative. 😬 When you're staring at something like this screenshot, and your project has 2,000 models, it's a little hard to know which ones need to be updated. 😃 I mean, if we can turn the warning off entirely, that'd be even better. But if we are going to get the warning, would be nice to know which model is generating the warning. Thank you!! Appreciate your help!
Screenshot 2024-10-10 at 12 26 25 PM

@dbernett-amplify
Copy link

Following up in case anyone else comes across this. Looks like we needed to add:
- UnsupportedConstraintMaterialization
to the list of errors that we're suppressing.

Also, while I found that removing ConstraintNotEnforced had no effect in the IDE, I did find that if I removed it, then our production jobs would throw the unenforced constraint warning. So we also need to leave that one in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants