-
Notifications
You must be signed in to change notification settings - Fork 984
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
clarify build on docs content and move doc to dag section #5633
Changes from all commits
00d3861
aca3fa0
0bd46db
f4968db
b0f07cb
483bfe3
887f10e
da380bb
414be7d
50d1d28
1cb86a7
f3a6ce0
a8a44c6
5715d3d
d2125e2
14cd7c7
be268ba
131576d
b2583ae
0a021c7
e39c75d
4181ee5
844ca28
b7a07c2
c64658f
5573db7
b3a737b
a7f7bac
5e4cea3
db6371a
9b5f349
43fae64
588a6d5
acd92b9
5a87091
055c28a
0434e44
abe6c22
dd3eeed
2e906a0
9484e45
e016b66
33607a6
4ed3bd3
3427e7f
d9d21de
a2f9870
167ec1c
96d8f4b
cf55222
b929a93
1b85ed9
1ad82b7
ec09a7e
8414445
779505c
38da545
246e41f
50609eb
2390941
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,11 +1,12 @@ | ||
--- | ||
title: "About documentation" | ||
title: "Documentation" | ||
description: "Learn how good documentation for your dbt models helps stakeholders discover and understand your datasets." | ||
id: "documentation" | ||
pagination_next: "docs/collaborate/build-and-view-your-docs" | ||
pagination_prev: null | ||
--- | ||
|
||
Good documentation for your dbt models will help downstream consumers discover and understand the datasets you curate for them. | ||
dbt provides a way to generate documentation for your dbt project and render it as a website. | ||
|
||
## Related documentation | ||
|
||
* [Declaring properties](/reference/configs-and-properties) | ||
|
@@ -19,18 +20,12 @@ pagination_prev: null | |
|
||
## Overview | ||
|
||
Good documentation for your dbt models will help downstream consumers discover and understand the datasets which you curate for them. | ||
|
||
dbt provides a way to generate documentation for your dbt project and render it as a website. The documentation for your project includes: | ||
dbt provides a way to generate documentation for your dbt project. The documentation for your project includes: | ||
* **Information about your project**: including model code, a DAG of your project, any tests you've added to a column, and more. | ||
* **Information about your <Term id="data-warehouse" />**: including column data types, and <Term id="table" /> sizes. This information is generated by running queries against the information schema. | ||
|
||
Importantly, dbt also provides a way to add **descriptions** to models, columns, sources, and more, to further enhance your documentation. | ||
|
||
Here's an example docs site: | ||
|
||
<Lightbox src="/img/docs/building-a-dbt-project/dbt-docs-screenshot.png" title="Auto-generated dbt documentation website"/> | ||
|
||
## Adding descriptions to your project | ||
To add descriptions to your project, use the `description:` key in the same files where you declare [tests](/docs/build/data-tests), like so: | ||
|
||
|
@@ -60,13 +55,19 @@ models: | |
|
||
</File> | ||
|
||
|
||
## Generating project documentation | ||
You can generate a documentation site for your project (with or without descriptions) using the CLI. | ||
|
||
First, run `dbt docs generate` — this command tells dbt to compile relevant information about your dbt project and warehouse into `manifest.json` and `catalog.json` files respectively. To see the documentation for all columns and not just columns described in your project, ensure that you have created the models with `dbt run` beforehand. | ||
The default documentation experience in dbt Cloud is [dbt Explorer](/docs/collaborate/explore-projects), available on [Team or Enterprise plans](https://www.getdbt.com/pricing/). Use dbt Explorer to view your project's resources (such as models, tests, and metrics), its [metadata](/docs/collaborate/explore-projects#generate-metadata), and their lineage to gain a better understanding of its latest production state. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @cafzal introducing explorer content here - what do you think? also moved this page from collaborate -> build your dag. however, there is also an option to have a 'doc your project' folder under the 'build' section. and add doc examples for other resources per Dave's suggestion. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. sounds good! |
||
|
||
Then, run `dbt docs serve` to use these `.json` files to populate a local website. | ||
dbt Cloud developer and dbt Core users can use [dbt Docs](/docs/collaborate/build-and-view-your-docs#dbt-docs), which generates basic documentation, but it doesn't offer the same speed, metadata, or visibility as dbt Explorer. | ||
|
||
Generate documentation for you project by following these steps: | ||
|
||
1. Run `dbt docs generate` — this command tells dbt to compile relevant information about your dbt project and warehouse into `manifest.json` and `catalog.json` files, respectively. | ||
2. Ensure that you have created the models with `dbt run` to view the documentation for all columns, not just those described in your project. | ||
3. Run `dbt docs serve` if you're developing locally to use these `.json` files to populate a local website. | ||
|
||
To view a resource, its metadata, and what commands are needed in dbt Explorer, refer to [generate metadata](/docs/collaborate/explore-projects#generate-metadata) for more details. | ||
|
||
## FAQs | ||
<FAQ path="Project/example-projects" alt_header="Are there any example dbt documentation sites?"/> | ||
|
@@ -75,8 +76,7 @@ Then, run `dbt docs serve` to use these `.json` files to populate a local websit | |
<FAQ path="Docs/sharing-documentation" /> | ||
<FAQ path="Docs/document-other-resources" /> | ||
|
||
|
||
## Using Docs Blocks | ||
## Using docs blocks | ||
### Syntax | ||
To declare a docs block, use the jinja `docs` tag. Docs blocks can contain arbitrary markdown, but they must be uniquely named. Their names may contain uppercase and lowercase letters (A-Z, a-z), digits (0-9), and underscores (_), but can't start with a digit. | ||
|
||
|
@@ -128,9 +128,11 @@ models: | |
|
||
In the resulting documentation, `'{{ doc("table_events") }}'` will be expanded to the markdown defined in the `table_events` docs block. | ||
|
||
|
||
## Setting a custom overview | ||
*Currently available for dbt Docs only.* | ||
|
||
The "overview" shown in the documentation website can be overridden by supplying your own docs block called `__overview__`. By default, dbt supplies an overview with helpful information about the docs site itself. Depending on your needs, it may be a good idea to override this docs block with specific information about your company style guide, links to reports, or information about who to contact for help. To override the default overview, create a docs block that looks like this: | ||
The "overview" shown in the dbt Docs website can be overridden by supplying your own docs block called `__overview__`. By default, dbt supplies an overview with helpful information about the docs site itself. Depending on your needs, it may be a good idea to override this docs block with specific information about your company style guide, links to reports, or information about who to contact for help. To override the default overview, create a docs block that looks like this: | ||
|
||
<File name='models/overview.md'> | ||
|
||
|
@@ -148,6 +150,7 @@ as well as the repo for this project \[here](https://github.com/dbt-labs/mrr-pla | |
</File> | ||
|
||
### Custom project-level overviews | ||
*Currently available for dbt Docs only.* | ||
|
||
You can set different overviews for each dbt project/package included in your documentation site | ||
by creating a docs block named `__[project_name]__`. For example, in order to define | ||
|
@@ -174,31 +177,46 @@ up to page views and sessions. | |
</File> | ||
|
||
## Navigating the documentation site | ||
Using the docs interface, you can navigate to the documentation for a specific model. That might look something like this: | ||
|
||
Use [dbt Explorer](/docs/collaborate/explore-projects) for a richer documentation experience and more interactive experience for understanding your project's resources and lineage. Available on [Team or Enterprise plans](https://www.getdbt.com/pricing/). | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @cafzal introducing explorer content here - what do you think? |
||
|
||
For additional details on how to explore your lineage and navigate your resources, refer to [dbt Explorer](/docs/collaborate/explore-projects). | ||
|
||
<Lightbox src="/img/docs/collaborate/dbt-explorer/example-model-details.png" width="100%" title="Example of resource details" /> | ||
|
||
<Expandable alt_header="For dbt Docs"> | ||
If you're using the dbt Docs interface, you can navigate to the documentation for a specific model. That might look something like this: | ||
|
||
<Lightbox src="/img/docs/building-a-dbt-project/testing-and-documentation/f2221dc-Screen_Shot_2018-08-14_at_6.29.55_PM.png" title="Auto-generated documentation for a dbt model"/> | ||
|
||
Here, you can see a representation of the project structure, a markdown description for a model, and a list of all of the columns (with documentation) in the model. | ||
|
||
From a docs page, you can click the green button in the bottom-right corner of the webpage to expand a "mini-map" of your DAG. This pane (shown below) will display the immediate parents and children of the model that you're exploring. | ||
From the dbt Docs page, you can click the green button in the bottom-right corner of the webpage to expand a "mini-map" of your DAG. This pane (shown below) will display the immediate parents and children of the model that you're exploring. | ||
|
||
<Lightbox src="/img/docs/building-a-dbt-project/testing-and-documentation/ec77c45-Screen_Shot_2018-08-14_at_6.31.56_PM.png" title="Opening the DAG mini-map"/> | ||
|
||
In this example, the `fct_subscription_transactions` model only has one direct parent. By clicking the "Expand" button in the top-right corner of the window, we can pivot the graph horizontally and view the full <Term id="data-lineage">lineage</Term> for our model. This lineage is filterable using the `--select` and `--exclude` flags, which are consistent with the semantics of [model selection syntax](/reference/node-selection/syntax). Further, you can right-click to interact with the DAG, jump to documentation, or share links to your graph visualization with your coworkers. | ||
|
||
<Lightbox src="/img/docs/building-a-dbt-project/testing-and-documentation/ac97fba-Screen_Shot_2018-08-14_at_6.35.14_PM.png" title="The full lineage for a dbt model"/> | ||
|
||
</Expandable> | ||
|
||
## Deploying the documentation site | ||
|
||
With dbt Cloud, use [dbt Explorer](/docs/collaborate/explore-projects) automatically retrieves the metadata updates after each job run in the production or staging deployment environment so it always has the latest results for your project — meaning it's always automatically updated after each job run. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @cafzal introducing explorer content here - what do you think? |
||
|
||
:::caution Security | ||
|
||
The `dbt docs serve` command is only intended for local/development hosting of the documentation site. Please use one of the methods listed below (or similar) to ensure that your documentation site is hosted securely! | ||
|
||
::: | ||
|
||
#### For dbt Docs users | ||
|
||
dbt's documentation website was built to make it easy to host on the web. The site is "static,” meaning you don't need any "dynamic" servers to serve the docs. You can host your documentation in several ways: | ||
|
||
* Use [dbt Cloud](/docs/collaborate/documentation) | ||
* Use [dbt Cloud's](/docs/collaborate/build-and-view-your-docs) default documentation experience with [dbt Explorer](/docs/collaborate/explore-projects). | ||
* Host on [Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/dev/WebsiteHosting.html) (optionally [with IP access restrictions](https://docs.aws.amazon.com/AmazonS3/latest/dev/example-bucket-policies.html#example-bucket-policies-use-case-3)) | ||
* Publish with [Netlify](https://discourse.getdbt.com/t/publishing-dbt-docs-to-netlify/121) | ||
* Use your own web server like Apache/Nginx | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we move this to the bottom while we're here, since that's where we usually place it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
tbh i did think that but the other 'build' docs have this format so left it so it would be consistent for users.