Change dbt columns materialization in bigquery #450
Conversation
|
👋 @ofek1weiss |
| @@ -0,0 +1,66 @@ | |||
| {% macro get_dbt_columns_query(is_on_run_end=false) %} | |||
| {% if is_on_run_end %} | |||
| {% set get_relation = elementary.get_elementary_relation %} | |||
There was a problem hiding this comment.
Not sure if we care, but just a thought, if a user installs Elementary and doesn't run our models, this would return None and then the query would be from None.
There was a problem hiding this comment.
if he does not run our models, this wont run as well
| @@ -0,0 +1,66 @@ | |||
| {% macro get_dbt_columns_query(is_on_run_end=false) %} | |||
There was a problem hiding this comment.
The true distinction that you want to make is whether you're in a model build context or not so I'd rename this to in_model_build_context or something like that. For instance, in the on-run-start, materializations, or other places, it's still meaningful.
| @@ -26,6 +26,10 @@ | |||
| {% do elementary.file_log('[{}] Artifacts already ran.'.format(artifacts_model)) %} | |||
| {% endif %} | |||
| {% endfor %} | |||
| {% if elementary.get_dbt_columns_materialization() != "view" %} | |||
| {% do adapter.commit() %} | |||
There was a problem hiding this comment.
Is this needed? There's should_commit=true for the uploads.
There was a problem hiding this comment.
but we need to commit before running the dbt_columns query, as it queries the other artifacts
There was a problem hiding this comment.
Yes, but I'm saying that the there are no pending transactions at this point because of the should_commit=True and each model commits when it finishes executing.
| {% macro upload_dbt_columns() %} | ||
| {% set relation = elementary.get_elementary_relation('dbt_columns') %} | ||
| {% if execute and relation %} | ||
| {% set query %} |
There was a problem hiding this comment.
are you sure this macro exists?, i searched and did not find it
There was a problem hiding this comment.
What do you mean? There's a link in my comment, lol.
macros/edr/dbt_artifacts/dbt_columns/get_dbt_columns_materialization.sql
Show resolved
Hide resolved
No description provided.