Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rely on dbt-core's global project's materialized view implementation #477

Closed
dataders opened this issue Oct 11, 2023 · 2 comments
Closed
Labels
enhancement New feature or request

Comments

@dataders
Copy link
Contributor

dataders commented Oct 11, 2023

related to: #473

part of our internal implementation is to draw distinction between the following:

  • relation: a data warehouse object that can be created, renamed, updated, dropped
  • materialization: the workflow by which dbt maps user-provided SELECT statement and configuration to a relation.

one goal of this refactor is that adapter maintainers need not concern themselves so much with implementing custom materializations. Instead just scaffold out the SQL API for CRUD operations on relations. In the case of materialized views, we also require information about:

  • the parameters users may provide
  • the format they may take so that we can make the minimal amount of changes should the existing relation already exist
  • what alter operations exist when configuration changes exist

The result is that the default materialization is both simple and adapter-extensible (see core/dbt/include/global_project/macros/materializations/models/materialized_view.sql#L1-L22)

@benc-db
Copy link
Collaborator

benc-db commented Nov 10, 2023

@dataders are the required hooks shipped in 1.7.0 (or 1.7.1) despite dropping as an adapter requirement? In the next week or so I will probably be picking this back up.

@benc-db
Copy link
Collaborator

benc-db commented Jan 12, 2024

Unfortunately we were not able to, as dbt-core assumes we can execute multiple sql statements in a single execution by using a semicolon...I believe this is coming, but isn't here yet, so had to roll our own materialization based on dbt-core's.

@benc-db benc-db closed this as completed Jan 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants