Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a "dev mode" variable to toggle on materialization of stg and int models #38

Open
katy-sadowski opened this issue Jun 11, 2024 · 2 comments

Comments

@katy-sadowski
Copy link
Collaborator

Currently, all stg and int models are materialized in the database. This clutters the schema and is not desirable for production analytic use cases. However, developers (and analysts in some cases) may want/need to inspect the contents of these models.

We can use a dbt_project variable that users pass into the command line that will enable materialization of stg and int models (I think it should be disabled by default).

@lawrenceadams
Copy link
Collaborator

I've tried this before and the solution that we used was to instead have a post run hook or a macro that drops all objects except the core tables we want i.e. the final OMOP tables.

At the moment our final dbt_synthea_dev schema holds 85 tables. We could rationalize those to be views instead? This may have a performance hit downstream.

Another approach would be to have our OMOP models be in their own schema - but I am unsure if that's really better // its just more complex

@katy-sadowski
Copy link
Collaborator Author

The post-run hook to drop all but final OMOP models sounds like a great idea to me!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants