Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

model query and auto-exposures beta #5896

Merged
merged 52 commits into from
Aug 12, 2024
Merged
Show file tree
Hide file tree
Changes from 48 commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
ed87fea
add drafts
mirnawong1 Jul 31, 2024
0a31c47
add imgs
mirnawong1 Aug 2, 2024
86ac451
add more images and snippet
mirnawong1 Aug 5, 2024
c9a322f
add intro
mirnawong1 Aug 5, 2024
8b4e386
add
mirnawong1 Aug 6, 2024
a5d1e36
add to docs
mirnawong1 Aug 6, 2024
14ed521
add carousel and section in mainpage
mirnawong1 Aug 6, 2024
a6c5507
Merge branch 'current' into add-exposures-model-query
mirnawong1 Aug 6, 2024
fb8d4c4
be more explicit
mirnawong1 Aug 6, 2024
fa7e980
Merge branch 'current' into add-exposures-model-query
mirnawong1 Aug 7, 2024
80d3991
Merge branch 'current' into add-exposures-model-query
mirnawong1 Aug 8, 2024
ff84bab
add data tile
mirnawong1 Aug 9, 2024
beef033
add data health tile
mirnawong1 Aug 9, 2024
3abb09e
move legacy to new
mirnawong1 Aug 9, 2024
679d8d3
img tweaks
mirnawong1 Aug 9, 2024
81ef0c2
add
mirnawong1 Aug 9, 2024
98ea7b1
Merge branch 'current' into add-exposures-model-query
mirnawong1 Aug 9, 2024
a68c13c
Merge branch 'add-exposures-model-query' of github.com:dbt-labs/docs.…
mirnawong1 Aug 9, 2024
68b1741
Update website/docs/docs/build/exposures.md
mirnawong1 Aug 9, 2024
1c69bb7
Update website/docs/docs/deploy/monitor-jobs.md
mirnawong1 Aug 9, 2024
df00551
Update website/docs/guides/productionize-your-dbt-databricks-project.md
mirnawong1 Aug 9, 2024
b120765
fix links
mirnawong1 Aug 9, 2024
43913c6
remoe
mirnawong1 Aug 9, 2024
66a681d
add link to data health
mirnawong1 Aug 9, 2024
9258555
add img
mirnawong1 Aug 9, 2024
57f33aa
Update website/docs/docs/collaborate/auto-exposures.md
mirnawong1 Aug 9, 2024
c9f121b
Update model-query-history.md
mirnawong1 Aug 9, 2024
41b7f59
Update model-query-history.md
mirnawong1 Aug 9, 2024
d3cd3c9
Update model-query-history.md
mirnawong1 Aug 9, 2024
24fe382
Update configure-auto-exposures.md
mirnawong1 Aug 9, 2024
cabb450
Update website/docs/docs/collaborate/model-query-history.md
mirnawong1 Aug 9, 2024
2b5b8b5
Update website/docs/docs/collaborate/data-tile.md
mirnawong1 Aug 9, 2024
ef4c023
Update website/docs/docs/collaborate/data-tile.md
mirnawong1 Aug 9, 2024
974a22d
Update website/docs/docs/collaborate/data-tile.md
mirnawong1 Aug 9, 2024
4cca4ab
Update website/docs/docs/collaborate/data-tile.md
mirnawong1 Aug 9, 2024
cc98470
Update website/docs/docs/collaborate/data-tile.md
mirnawong1 Aug 9, 2024
e5d7d6a
Update website/docs/docs/collaborate/data-tile.md
mirnawong1 Aug 9, 2024
62a25a4
Update website/docs/docs/collaborate/model-query-history.md
mirnawong1 Aug 9, 2024
39934ce
natalie's feedback
mirnawong1 Aug 9, 2024
ffa49f5
Merge branch 'add-exposures-model-query' of github.com:dbt-labs/docs.…
mirnawong1 Aug 9, 2024
d3cff70
Merge branch 'current' into add-exposures-model-query
mirnawong1 Aug 9, 2024
13add4f
Update configure-auto-exposures.md
mirnawong1 Aug 9, 2024
8c9d46a
Update auto-exposures.md
mirnawong1 Aug 9, 2024
1c43db9
Update data-tile.md
mirnawong1 Aug 9, 2024
6713d4e
Update model-query-history.md
mirnawong1 Aug 9, 2024
58a2b9b
Merge branch 'current' into add-exposures-model-query
mirnawong1 Aug 9, 2024
84fe9bd
Update website/docs/docs/cloud-integrations/configure-auto-exposures.md
runleonarun Aug 10, 2024
0d476e8
Merge branch 'current' into add-exposures-model-query
mirnawong1 Aug 12, 2024
30f9f88
Merge branch 'current' into add-exposures-model-query
mirnawong1 Aug 12, 2024
4d0a103
space
mirnawong1 Aug 12, 2024
f83ae2f
Merge branch 'current' into add-exposures-model-query
mirnawong1 Aug 12, 2024
e1b0e25
Apply suggestions from code review
runleonarun Aug 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion website/docs/docs/build/exposures.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,4 +78,4 @@ When we generate the dbt Explorer site, you'll see the exposure appear:

* [Exposure properties](/reference/exposure-properties)
* [`exposure:` selection method](/reference/node-selection/methods#the-exposure-method)
* [Dashboard status tiles](/docs/deploy/dashboard-status-tiles)
* [Data health tiles](/docs/collaborate/data-tile)
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
---
title: "Auto-exposures"
id: "configure-auto-exposures"
sidebar_label: "Configure auto-exposures"
description: "Import and auto-generate exposures from dashboards and understand how models are used in downstream tools for a richer lineage."
image: /img/docs/cloud-integrations/auto-exposures/explorer-lineage2.jpg
---

# Configure auto-exposures <Lifecycle status='beta' />

As a data team, it’s critical that you have context into the downstream use cases and users of your data products. [Auto-exposures](/docs/collaborate/auto-exposures) integrates natively with Tableau and [auto-generates downstream lineage](/docs/collaborate/auto-exposures#view-auto-exposures-in-dbt-explorer) in dbt Explorer for a richer experience.

runleonarun marked this conversation as resolved.
Show resolved Hide resolved
Auto-exposures help data teams optimize their efficiency and ensure data quality by:

- Helping users understand how their models are used in downstream analytics tools to inform investments and reduce incidents — ultimately building trust and confidence in data products.
- Importing and auto-generating exposures based on Tableau dashboards, with user-defined curation.
- Enabling the active exposure work to run models based on when exposures are updated or need to be updated, improving timeliness and reducing costs.

## Prerequisites

To access the features, you should meet the following:

1. You have a dbt Cloud account on the [Enterprise plan](https://www.getdbt.com/pricing/).
2. You have set up a [production](/docs/deploy/deploy-environments#set-as-production-environment) deployment environment for each project you want to explore, with at least one successful job run.
3. You have [admin permissions](/docs/cloud/manage-access/enterprise-permissions) in dbt Cloud to edit project settings or production environment settings
4. Use Tableau as your BI tool and can enable metadata permissions or work with an admin to do so. Compatible with Tableau Cloud or Tableau Server with the Metadata API enabled.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We may need an update for Tableau server based on what we find out from customers about Whitelisting the IP address (just an additional callout of the extra step for Server). I'll know more on Monday. Is it fine to make the update post it going live if needed?

Copy link
Collaborator

@runleonarun runleonarun Aug 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we can totally update after it goes live! I won't resolve this comment until we hear back though.


## Set up in Tableau

This section of the document explains the steps you need to set up the auto-exposures integration with Tableau. Once you've set this up in Tableau and dbt Cloud, you can view the [auto-exposures](/docs/collaborate/auto-exposures#view-auto-exposures-in-dbt-explorer) in dbt Explorer.

To set up [personal access tokens (PATs)](/docs/dbt-cloud-apis/user-tokens#using-the-new-personal-access-tokens) needed for auto exposures, ask a site admin to configure it for the account.

1. Ensure you or a site admin enables PATs for the account in Tableau.
<Lightbox src="/img/docs/cloud-integrations/auto-exposures/tableau-enable-pat.jpg" title="Enable PATs for the account in Tableau"/>

2. Create a PAT that you can add to dbt Cloud to pull in Tableau metadata for auto exposures.
<Lightbox src="/img/docs/cloud-integrations/auto-exposures/tableau-create-pat.jpg" title="Create PATs for the account in Tableau"/>

3. Copy the **Secret** and the **Token name** and enter them in dbt Cloud. The secret is only displayed once, so store it in a safe location (like a password manager).
<Lightbox src="/img/docs/cloud-integrations/auto-exposures/tableau-copy-token.jpg" title="Copy the secret and token name to enter them in dbt Cloud"/>

4. Copy the **Server URL** and **Sitename**. You can find these in the URL while logged into Tableau.
<Lightbox src="/img/docs/cloud-integrations/auto-exposures/tablueau-serverurl.jpg" title="Locate the Server URL and Sitename in Tableau"/>

For example, if the full URL is: `10az.online.tableau.com/#/site/dbtlabspartner/explore`:
- The **Server URL** is the first part of the URL, in this case: `10az.online.tableau.com`
- The **Sitename** is right after the `site` in the URL, in this case: `dbtlabspartner`

5. You should now be ready to set up auto-exposures in dbt Cloud after copying the following items, which you'll need during the dbt Cloud setup: ServerURL, Sitename, Token name, and Secret.

## Set up in dbt Cloud <Lifecycle status="enterprise"/>

1. In dbt Cloud, navigate to the project you want to add the auto-exposures to and then select **Settings**.
2. Under the **Exposures** section, select **Add integration** to add the Tableau connection.
<Lightbox src="/img/docs/cloud-integrations/auto-exposures/cloud-add-integration.jpg" title="Select Add Integration to add the Tableau connection."/>
3. Enter the details for the exposure connection you collected from Tableau in the [previous step](#set-up-in-tableau) and click **Continue**. Note that all fields are case-sensitive.
<Lightbox src="/img/docs/cloud-integrations/auto-exposures/cloud-integration-details.jpg" title="Enter the details for the exposure connection."/>
4. Select the collections you want to include for auto exposures.

dbt Cloud automatically imports and syncs any workbook within the selected collections. New additions to the collections will be added to the lineage in dbt Cloud during the next automatic sync (usually once per day).
<Lightbox src="/img/docs/cloud-integrations/auto-exposures/cloud-select-collections.jpg" title="Select the collections you want to include for auto exposures."/>
5. Click **Save**. dbt Cloud imports everything in this collection and you can continue to view them in Explorer using the next steps.

For more information on how to view and use auto-exposures, refer to [View auto-exposures from dbt Explorer](/docs/collaborate/auto-exposures) page.

<Lightbox src="/img/docs/cloud-integrations/auto-exposures/explorer-lineage2.jpg" width="100%" title="View from the dbt Explorer in your Project lineage view, displayed with the Tableau icon."/>

## Refresh auto-exposures in jobs

:::info Coming soon
Soon, you’ll also be able to use auto-exposures trigger refresh of the data used in your Tableau dashboards from within dbt Cloud. Stay tuned for more on this soon!
:::
16 changes: 11 additions & 5 deletions website/docs/docs/cloud-integrations/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,21 @@
title: "About dbt Cloud integrations"
sidebar_label: "About dbt Cloud integrations"
pagination_prev: null
pagination_next: "docs/cloud-integrations/snowflake-native-app"
pagination_next: "docs/cloud-integrations/configure-auto-exposures"
hide_table_of_contents: true
---

Many data applications integrate with dbt Cloud, enabling you to leverage the power of dbt for a variety of use cases and workflows.


## Integrations with dbt

<div className="grid--2-col">
<div className="grid--3-col">

<Card
title="Configure auto-exposures (beta)"
body="Import and auto-generate exposures from dashboards to understand how models are used in downstream tools for a richer downstream lineage."
link="/docs/cloud-integrations/configure-auto-exposures"
icon="dbt-bit"/>

<Card
title="dbt Snowflake Native App (preview)"
Expand All @@ -19,9 +25,9 @@ Many data applications integrate with dbt Cloud, enabling you to leverage the po
icon="snowflake"/>

<Card
title="Semantic layer integrations"
title="dbt Semantic layer integrations"
body="Review a wide range of partners you can integrate and query with the dbt Semantic Layer."
link="/docs/cloud-integrations/avail-sl-integrations"
icon="dbt-bit"/>

</div>
</div>
20 changes: 20 additions & 0 deletions website/docs/docs/collaborate/auto-exposures.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
---
title: "Auto-exposures"
sidebar_label: "Auto-exposures"
description: "Import and auto-generate exposures from dashboards and understand how models are used in downstream tools for a richer lineage."
pagination_prev: null
pagination_next: "docs/collaborate/data-tile"
image: /img/docs/cloud-integrations/auto-exposures/explorer-lineage.jpg
---

# Auto-exposures <Lifecycle status='beta' />

As a data team, it’s critical that you have context into the downstream use cases and users of your data products. Auto-exposures integrates natively with Tableau (Power BI coming soon) and auto-generates downstream lineage in dbt Explorer for a richer experience.
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved

runleonarun marked this conversation as resolved.
Show resolved Hide resolved
Auto-exposures helps users understand how their models are used in downstream analytics tools to inform investments and reduce incidents — ultimately building trust and confidence in data products. It imports and auto-generates exposures based on Tableau dashboards, with user-defined curation.

For more information on how to set up auto-exposures, prerequisites, and more &mdash; refer to [configure auto-exposures in Tableau and dbt Cloud](/docs/cloud-integrations/configure-auto-exposures).

import ViewExposures from '/snippets/_auto-exposures-view.md';

<ViewExposures/>
Loading
Loading