[BUG] table and column comments not being added with vault_insert_by_rank

[erikwilkins-doosan]

Describe the bug
After setting "persist_docs" in dbt_project.yml, yml descriptions are not being added to Snowflake when model is using the vault_insert_by_rank materialization

Environment
dbt version: dbt_cloud v1.5
automate_dv version: 0.9.6
Database/Platform: Snowflake

To Reproduce
Steps to reproduce the behavior:

1. Add the following to dbt_project.yml under models: to add yml comments to table and column
   +persist_docs:
   relation: true
   columns: true
2. Setup model to use vault_insert_by_rank
   {{ config(materialized='vault_insert_by_rank', rank_column='automate_dv_rank', rank_source_models='STAGE MODEL') }}
3. Build Model
4. Check COMMENT column on INFORMATION_SCHEMA.COLUMNS in Snowflake where TABLE_NAME = 'Model'

Expected behavior
COMMENTs to be sent to Snowflake similar to dbt standard customizations when database object is created

AB#5357

[DVAlexHiggs]

Hello!

Apologies for the (massively) delayed response on this. This issue is due to a feature implemented in dbt which we haven't kept up-to-date with. persist_docs is handled by calling helper macros provided by dbt in custom materialisations, and we just haven't added that functionality in. Another example of this is the on_schema_change functionality.

For the custom materialisations we offer, this needs to happen. Though we plan to deprecate these materialisations eventually, we want to get them to a better state before we do so.

I've added the 'dbt-approach' label to highlight what I've described above.

[erikwilkins-doosan]

@DVAlexHiggs , thanks for responding to this. after this issue was created and after upgrading AutomateDV version to latest version for improved Satellite insert methodology, we pivoted from using vault_insert_by_rank custom materialization back to incremental . At this time, I see no need to leave this issue open.

[DVAlexHiggs]

@DVAlexHiggs , thanks for responding to this. after this issue was created and after upgrading AutomateDV version to latest version for improved Satellite insert methodology, we pivoted from using vault_insert_by_rank custom materialization back to incremental . At this time, I see no need to leave this issue open.

Thanks for the quick reply! That makes sense and we're glad you were able to transition away from this to use incremental, as that was one of the aims of the improvements to the Satellite macro. I'm going to leave this open until I've discussed with the team internally as to whether we still want to make these changes prior to deprecation.

[DVAlexHiggs]

Hello,
Apologies for the long time coming on this one. With the release of v0.11.0 today, we've decided to deprecate our vault_insert_by_x materialisations and focus on improving other areas of AutomateDV, so we won't be making that change.

If anyone comes across this thread and would like any guidance, please reach out to me on Slack, LinkedIn or via email. Thanks!