Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add enabled config to models without it #12

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

santina-brohen
Copy link

@santina-brohen santina-brohen commented Feb 27, 2024

Please provide your name and company

Santina Brohen, Pleo

Link the issue/feature request which this PR is meant to address

Link to issue here.

Some context here as well:
Currently, the package expects certain sources to be available. Some models, but not all, include the enabled config, which makes it possible to skip the model if the source is not available. For example, stg_zuora__credit_balance_adjustment includes the following: {{ config(enabled=var('zuora__using_credit_balance_adjustment', true)) }} (code here). Of the 19 models, only 4 can be disabled:

  • stg_zuora__credit_balance_adjustment
  • stg_zuora__refund
  • stg_zuora__refund_invoice_payment
  • stg_zuora__taxation_item

At Pleo, we are not ingesting all the sources that are expected by the package, so even when leveraging the enabled config in the models where it's available, we are still left with models raising errors.

Detail what changes this PR introduces and how this addresses the issue/feature request linked above.

With this PR, it is possible to disable any model, not just the subset listed above. Having the ability to disable models for which the sources are not ingested via Fivetran is a great feature, and it opens up use of this package even when only a few Fivetran sources are available.

How did you validate the changes introduced within this PR?

As recommended by you here, I followed this dbt Discourse article to contribute.

Before making the changes, we had four models that raised errors because the sources are not available. I ran dbt build --select +zuora_source and the following shows the error:

Screenshot 2024-02-26 at 16 08 21

After making the changes and disabling the models for sources we do not ingest, I ran dbt build --select +zuora_source again; no errors were raised.

Screenshot 2024-02-27 at 16 06 12

Screenshot 2024-02-27 at 16 04 27

I then removed one of the new variables to ensure that it still correctly raised an error; it did:

Screenshot 2024-02-27 at 16 08 12
Note: zuora__using_contact: false is commented out.

Screenshot 2024-02-27 at 16 09 10

Which warehouse did you use to develop these changes?
BigQuery

Did you update the CHANGELOG?

  • Yes

Did you update the dbt_project.yml files with the version upgrade (please leverage standard semantic versioning)? (In both your main project and integration_tests)

  • Yes

Provide an emoji that best describes your current mood

🦄

Feedback

We are so excited you decided to contribute to the Fivetran community dbt package! We continue to work to improve the packages and would greatly appreciate your feedback on our existing dbt packages or what you'd like to see next.

PR Template

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant