Explorer update: column level lineage #4767
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
github-actions
bot
added
content
| Column level lineage is available to dbt Cloud Enterprise accounts that can use Explorer. It’s also available through the Discovery API. | ||
|
|
||
| :::tip Beta | ||
| Column-level lineage is now available in beta. Check it out! We'd love to [know what you think](https://docs.google.com/forms/d/e/1FAIpQLSdpCbVkGY9QwfExFonpWE4DTOKi3fQxBGLD0wwKYpkMjgcE7g/viewform)! |
There was a problem hiding this comment.
i didn't use "open beta" here, aligned with the betas for Project recs and Model perf pages. lemme know if this doesn't work tho! happy to change
|
|
||
| - **Parsing error** — Error occurs when the SQL is ambiguous or too complex for parsing. An example of ambiguous parsing scenarios are _complex_ lateral joins. | ||
| - **Python error** — Error occurs when a Python model is used within the lineage. Due to the nature of Python models, it's not possible to parse and determine the lineage. | ||
| - **Unknown error** — Error occurs when the lineage can't be determined for an unknown reason. An example of this would be if a dbt best practice is not being followed, like using hardcoded table names instead of `ref` statements. |
There was a problem hiding this comment.
"improper use of dbt" read a bit strong to me so trying this wording instead. this might be a bit of a mouthful tho
|
|
||
| The project recommendations beta feature is now available in dbt Explorer! Check it out! | ||
|
|
||
| ::: |
There was a problem hiding this comment.
callouts at the top of pages affect SEO so as a best practice we're moving them to a lower location on the page but making sure they're still before the fold
|
|
||
| <Lightbox src="/img/docs/collaborate/dbt-explorer/example-cll.png" title="Example of the Columns tab and where to expand for the CLL"/> | ||
|
|
||
| <LoomVideo id='278c948ba387457884cc6b9545793685' /> |
There was a problem hiding this comment.
@dave-connors-3 : i'm using the loom video from the notion draft. we can update the link if you end up rerecording it, np
There was a problem hiding this comment.
@dave-connors-3 feel free to re-record, maybe on Fri or Mon? i'd like to see your take, and we may want to wait for more UX improvements to land
|
|
||
| ## What you can use column level lineage for {#use-cases} | ||
|
|
||
| Click on these tabs to learn more about the CLL use cases, the analysis you can do, and the results you can achieve: |
There was a problem hiding this comment.
Explore the sections below to learn more about why and how you can use column-level lineage.
There was a problem hiding this comment.
updated this. for accessibility, "below" is not a great word choice so worked around that
| <TabItem value="collaboration" label="Collaboration and efficiency"> | ||
|
|
||
| When exploring your data products, navigating column lineage allows analytics engineers and data analysts to more easily navigate and understand the origin and usage of their data, enabling them to make better decisions with higher confidence. | ||
| </TabItem> |
There was a problem hiding this comment.
Another use case - thoughts @dave-connors-3 ?
Auditing
Column-level awareness makes it easier for data developers to see how columns flow through the project when examining how data is used in a dbt project.
|
|
||
| <Lightbox src="/img/docs/collaborate/dbt-explorer/example-cll.png" title="Example of the Columns tab and where to expand for the CLL"/> | ||
|
|
||
| <LoomVideo id='278c948ba387457884cc6b9545793685' /> |
There was a problem hiding this comment.
@dave-connors-3 feel free to re-record, maybe on Fri or Mon? i'd like to see your take, and we may want to wait for more UX improvements to land
|
|
||
| <LoomVideo id='278c948ba387457884cc6b9545793685' /> | ||
|
|
||
| ## Column level lineage use cases {#use-cases} |
There was a problem hiding this comment.
updated heading title so it shows up higher (ie "weighted" more) in our search list as well as having a tidy URL. here's a comparison where the "distinct" example is at the top of the search list:
What are you changing in this pull request and why?
Create subpage for new column level lineage feature in dbt Explorer.
New page:
/docs/collaborate/column-level-lineageUpdated page:
Cleaned up (removed) unused files:
catalog-sidebar-v1.giflineage-v1.gifmodel-resource-details-v1.gifsearch-v1.gifChecklist
Adding or removing pages (delete if not applicable):
website/sidebars.jswebsite/static/_redirectsnpm run buildto update the links that point to deleted pages