diff --git a/website/docs/docs/collaborate/explore-projects.md b/website/docs/docs/collaborate/explore-projects.md index 05326016fab..7cddc87cb0b 100644 --- a/website/docs/docs/collaborate/explore-projects.md +++ b/website/docs/docs/collaborate/explore-projects.md @@ -2,7 +2,7 @@ title: "Explore your dbt projects" sidebar_label: "Explore dbt projects" description: "Learn about dbt Explorer and how to interact with it to understand, improve, and leverage your data pipelines." -pagination_next: "docs/collaborate/explore-multiple-projects" +pagination_next: "docs/collaborate/model-performance" pagination_prev: null --- diff --git a/website/docs/docs/collaborate/model-performance.md b/website/docs/docs/collaborate/model-performance.md new file mode 100644 index 00000000000..950703359c1 --- /dev/null +++ b/website/docs/docs/collaborate/model-performance.md @@ -0,0 +1,41 @@ +--- +title: "Model performance" +sidebar_label: "Model performance" +description: "Learn about ." +--- + +dbt Explorer provides metadata on dbt Cloud runs for in-depth model performance and quality analysis. This feature assists in reducing infrastructure costs and saving time for data teams by highlighting where to fine-tune projects and deployments — such as model refactoring or job configuration adjustments. + + + +:::tip Public preview + +The model performance feature is now available in dbt Explorer! Check it out! +::: + +## The Performance overview page + +You can pinpoint areas for performance enhancement by using the Performance overview page. This page presents a comprehensive analysis across all project models and displays the longest-running models, those most frequently executed, and the ones with the highest failure rates during runs/tests. Data can be segmented by environment and job type which can offer insights into: + +- Most executed models (total count). +- Models with the longest execution time (average duration). +- Models with the most failures, detailing run failures (percentage and count) and test failures (percentage and count). + +Each data point links to individual models in Explorer. + + + +You can view historical metadata for up to the past three months. Select the time horizon using the filter, which defaults to a two-week lookback. + + + +## The Model performance tab + +You can view trends in execution times, counts, and failures by using the Model performance tab for historical performance analysis. Daily execution data includes: + +- Average model execution time. +- Model execution counts, including failures/errors (total sum). + +Clicking on a data point reveals a table listing all job runs for that day, with each row providing a direct link to the details of a specific run. + + \ No newline at end of file diff --git a/website/docs/docs/collaborate/project-recommendations.md b/website/docs/docs/collaborate/project-recommendations.md new file mode 100644 index 00000000000..2874c646e60 --- /dev/null +++ b/website/docs/docs/collaborate/project-recommendations.md @@ -0,0 +1,50 @@ +--- +title: "Project recommendations" +sidebar_label: "Project recommendations" +description: "dbt Explorer provides recommendations that you can take to improve the quality of your dbt project." +--- + +:::tip Public preview + +The project recommendations feature is now available in dbt Explorer! Check it out! + +::: + +dbt Explorer provides recommendations about your project from the `dbt_project_evaluator` [package](https://hub.getdbt.com/dbt-labs/dbt_project_evaluator/latest/) using metadata from the Discovery API. + +Explorer also offers a global view, showing all the recommendations across the project for easy sorting and summarizing. + +These recommendations provide insight into how you can build a more well documented, well tested, and well built project, leading to less confusion and more trust. + +The Recommendations overview page includes two top-level metrics measuring the test and documentation coverage of the models in your project. + +- **Model test coverage** — The percent of models in your project (models not from a package or imported via dbt Mesh) with at least one dbt test configured on them. +- **Model documentation coverage** — The percent of models in your project (models not from a package or imported via dbt Mesh) with a description. + + + +## List of rules + +| Category | Name | Description | Package Docs Link | +| --- | --- | --- | --- | +| Modeling | Direct Join to Source | Model that joins both a model and source, indicating a missing staging model | [GitHub](https://dbt-labs.github.io/dbt-project-evaluator/0.8/rules/modeling/#direct-join-to-source) | +| Modeling | Duplicate Sources | More than one source node corresponds to the same data warehouse relation | [GitHub](https://dbt-labs.github.io/dbt-project-evaluator/0.8/rules/modeling/#duplicate-sources) | +| Modeling | Multiple Sources Joined | Models with more than one source parent, indicating lack of staging models | [GitHub](https://dbt-labs.github.io/dbt-project-evaluator/0.8/rules/modeling/#multiple-sources-joined) | +| Modeling | Root Model | Models with no parents, indicating potential hardcoded references and need for sources | [GitHub](https://dbt-labs.github.io/dbt-project-evaluator/0.8/rules/modeling/#root-models) | +| Modeling | Source Fanout | Sources with more than one model child, indicating a need for staging models | [GitHub](https://dbt-labs.github.io/dbt-project-evaluator/0.8/rules/modeling/#source-fanout) | +| Modeling | Unused Source | Sources that are not referenced by any resource | [GitHub](https://dbt-labs.github.io/dbt-project-evaluator/0.8/rules/modeling/#unused-sources) | +| Performance | Exposure Dependent on View | Exposures with at least one model parent materialized as a view, indicating potential query performance issues | [GitHub](https://dbt-labs.github.io/dbt-project-evaluator/0.8/rules/performance/#exposure-parents-materializations) | +| Testing | Missing Primary Key Test | Models with insufficient testing on the grain of the model. | [GitHub](https://dbt-labs.github.io/dbt-project-evaluator/0.8/rules/testing/#missing-primary-key-tests) | +| Documentation | Undocumented Models | Models without a model-level description | [GitHub](https://dbt-labs.github.io/dbt-project-evaluator/0.8/rules/documentation/#undocumented-models) | +| Documentation | Undocumented Source | Sources (collections of source tables) without descriptions | [GitHub](https://dbt-labs.github.io/dbt-project-evaluator/0.8/rules/documentation/#undocumented-sources) | +| Documentation | Undocumented Source Tables | Source tables without descriptions | [GitHub](https://dbt-labs.github.io/dbt-project-evaluator/0.8/rules/documentation/#undocumented-source-tables) | +| Governance | Public Model Missing Contract | Models with public access that do not have a model contract to ensure the data types | [GitHub](https://dbt-labs.github.io/dbt-project-evaluator/0.8/rules/governance/#public-models-without-contracts) | + + +## The Recommendations tab + +Models, sources and exposures each also have a Recommendations tab on their resource details page, with the specific recommendations that correspond to that resource: + + + + diff --git a/website/sidebars.js b/website/sidebars.js index 4653b028ef9..598fffc7f0d 100644 --- a/website/sidebars.js +++ b/website/sidebars.js @@ -424,6 +424,8 @@ const sidebarSettings = { link: { type: "doc", id: "docs/collaborate/explore-projects" }, items: [ "docs/collaborate/explore-projects", + "docs/collaborate/model-performance", + "docs/collaborate/project-recommendations", "docs/collaborate/explore-multiple-projects", ], }, diff --git a/website/static/img/docs/collaborate/dbt-explorer/ex-2-week-default.png b/website/static/img/docs/collaborate/dbt-explorer/ex-2-week-default.png new file mode 100644 index 00000000000..e4d6a0cfc30 Binary files /dev/null and b/website/static/img/docs/collaborate/dbt-explorer/ex-2-week-default.png differ diff --git a/website/static/img/docs/collaborate/dbt-explorer/example-model-performance-tab.png b/website/static/img/docs/collaborate/dbt-explorer/example-model-performance-tab.png new file mode 100644 index 00000000000..ad0dae2ba5b Binary files /dev/null and b/website/static/img/docs/collaborate/dbt-explorer/example-model-performance-tab.png differ diff --git a/website/static/img/docs/collaborate/dbt-explorer/example-performance-overview-page.png b/website/static/img/docs/collaborate/dbt-explorer/example-performance-overview-page.png new file mode 100644 index 00000000000..4b6d841294c Binary files /dev/null and b/website/static/img/docs/collaborate/dbt-explorer/example-performance-overview-page.png differ diff --git a/website/static/img/docs/collaborate/dbt-explorer/example-recommendations-overview.png b/website/static/img/docs/collaborate/dbt-explorer/example-recommendations-overview.png new file mode 100644 index 00000000000..c919dbf1dca Binary files /dev/null and b/website/static/img/docs/collaborate/dbt-explorer/example-recommendations-overview.png differ diff --git a/website/static/img/docs/collaborate/dbt-explorer/example-recommendations-tab.png b/website/static/img/docs/collaborate/dbt-explorer/example-recommendations-tab.png new file mode 100644 index 00000000000..493930c35db Binary files /dev/null and b/website/static/img/docs/collaborate/dbt-explorer/example-recommendations-tab.png differ