Updates to dbt Explorer (#4537)

## What are you changing in this pull request and why? Updates to dbt Explorer - Update this page https://docs.getdbt.com/docs/collaborate/explore-projects with the latest changes - Create a new subpage and move this section https://docs.getdbt.com/docs/collaborate/explore-projects#about-project-level-lineage and all its subsections to it - Create a release note about this update ## Checklist - [x] Review the [Content style guide](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/content-style-guide.md) so my content adheres to these guidelines. - [x] For [docs versioning](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#about-versioning), review how to [version a whole page](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#adding-a-new-version) and [version a block of content](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#versioning-blocks-of-content). - [ ] Needs review from PM Adding new pages (delete if not applicable): - ~~[ ] Add page to `website/sidebars.js`~~ n/a for release notes - [x] Provide a unique filename for the new page
dbt-labs · Nov 28, 2023 · bcbab8a · bcbab8a
2 parents 6d1d0ad + f5543c0
commit bcbab8a
Show file tree

Hide file tree

Showing 4 changed files with 111 additions and 62 deletions.
diff --git a/website/docs/docs/collaborate/explore-multiple-projects.md b/website/docs/docs/collaborate/explore-multiple-projects.md
@@ -0,0 +1,46 @@
+---
+title: "Explore multiple projects"
+sidebar_label: "Explore multiple projects"
+description: "Learn about project-level lineage in dbt Explorer and its uses."
+pagination_next: null
+---
+
+You can also view all the different projects and public models in the account, where the public models are defined, and how they are used to gain a better understanding about your cross-project resources.
+
+The resource-level lineage graph for a given project displays the cross-project relationships in the DAG. The different icons indicate whether you’re looking at an upstream producer project (parent) or a downstream consumer project (child).
+
+When you view an upstream (parent) project, its public models display a counter icon in the upper right corner indicating how many downstream (child) projects depend on them. Selecting a model reveals the lineage indicating the projects dependent on that model. These counts include all projects listing the upstream one as a dependency in its `dependencies.yml`, even without a direct `{{ ref() }}`. Selecting a project node from a public model opens its detailed lineage graph, which is subject to your [permission](/docs/cloud/manage-access/enterprise-permissions).
+
+<Lightbox src="/img/docs/collaborate/dbt-explorer/cross-project-lineage-parent.png" width="80%" height="100" title="Cross-project lineage in a parent project"/>
+
+When viewing a downstream (child) project that imports and refs public models from upstream (parent) projects, public models will show up in the lineage graph and display an icon on the graph edge that indicates what the relationship is to a model from another project. Hovering over this icon indicates the specific dbt Cloud project that produces that model. Double-clicking on a model from another project opens the resource-level lineage graph of the parent project, which is subject to your permissions.
+
+
+<Lightbox src="/img/docs/collaborate/dbt-explorer/cross-project-lineage-child.png" width="85%" height="100" title="Cross-project lineage in a child project"/>
+
+## Explore the project-level lineage graph
+
+For cross-project collaboration, you can interact with the DAG in all the same ways as described in [Explore your project's lineage](/docs/collaborate/explore-projects#project-lineage) but you can also interact with it at the project level and view the details. 
+
+To get a list view of all the projects, select the account name at the top of the **Explore** page near the navigation bar. This view includes a public model list, project list, and a search bar for project searches. You can also view the project-level lineage graph by clicking the Lineage view icon in the page's upper right corner.
+
+If you have permissions for a project in the account, you can view all public models used across the entire account. However, you can only view full public model details and private models if you have permissions for a project where the models are defined.
+
+From the project-level lineage graph, you can:
+
+- Click the Lineage view icon (in the graph’s upper right corner) to view the cross-project lineage graph.
+- Click the List view icon (in the graph’s upper right corner) to view the project list.
+    - Select a project from the **Projects** tab to switch to that project’s main **Explore** page.
+    - Select a model from the **Public Models** tab to view the [model’s details page](/docs/collaborate/explore-projects#view-resource-details).
+    - Perform searches on your projects with the search bar.
+- Select a project node in the graph (double-clicking) to switch to that particular project’s lineage graph.
+
+When you select a project node in the graph, a project details panel opens on the graph’s right-hand side where you can:
+
+- View counts of the resources defined in the project.
+- View a list of its public models, if any.
+- View a list of other projects that uses the project, if any.
+- Click **Open Project Lineage** to switch to the project’s lineage graph.
+- Click the Share icon to copy the project panel link to your clipboard so you can share the graph with someone.
+
+<LoomVideo id='606f02e1cce343eba7e1061d6273ff0a?t=1' />
diff --git a/website/docs/docs/collaborate/explore-projects.md b/website/docs/docs/collaborate/explore-projects.md
@@ -2,7 +2,7 @@
 title: "Explore your dbt projects"
 sidebar_label: "Explore dbt projects"
 description: "Learn about dbt Explorer and how to interact with it to understand, improve, and leverage your data pipelines."
-pagination_next: null
+pagination_next: "docs/collaborate/explore-multiple-projects"
 pagination_prev: null
 ---
 
@@ -53,17 +53,23 @@ To interact with the full lineage graph, you can:
 - Hover over any item in the graph to display the resource’s name and type.
 - Zoom in and out on the graph by mouse-scrolling.
 - Grab and move the graph and the nodes.
+- Right click on a node (context menu) to:
+    - Refocus on the node, including its parent and child nodes
+    - Refocus on the node and its children only
+    - Refocus on the node and it parents only
+    - View the node's [resource details](#view-resource-details) page
+
 - Select a resource to highlight its relationship with other resources in your project. A panel opens on the graph’s right-hand side that displays a high-level summary of the resource’s details. The side panel includes a **General** tab for information like description, materialized type, and other details.
     - Click the Share icon in the side panel to copy the graph’s link to your clipboard.
     - Click the View Resource icon in the side panel to [view the resource details](#view-resource-details). 
-- [Search and select specific resources](#search-resources) or a subset of the DAG using selectors and graph operators. For example:
+- [Search and select specific resources](#search-resources) or a subset of the DAG using [selectors](/reference/node-selection/methods) and [graph operators](/reference/node-selection/graph-operators). This can help you narrow the focus on the resources that interest you. For example:
     - `+[RESOURCE_NAME]` &mdash; Displays all parent nodes of the resource
     - `resource_type:model [RESOURCE_NAME]` &mdash; Displays all models matching the name search
 
 - [View resource details](#view-resource-details) by selecting a node (double-clicking) in the graph.
 - Click the List view icon in the graph's upper right corner to return to the main **Explore** page. 
 
-<LoomVideo id='1c2224ee7d6e4bbca850345e9a7b79aa' />
+<LoomVideo id='4f85654b5cc443478299d2bb1e9211f6?t=0' />
 
 
 ## Search for resources {#search-resources}
@@ -74,9 +80,15 @@ Select a node (single-click) in the lineage graph to highlight its relationship
 ### Search with keywords
 When searching with keywords, dbt Explorer searches through your resource metadata (such as resource type, resource name, column name, source name, tags, schema, database, version, alias/identifier, and package name) and returns any matches.
 
-### Search with selector methods 
+- Keyword search features a side panel (to the right of the main section) to filter search results by resource type. 
+- Use this panel to select specific resource tags or model access levels under the **Models** option. 
+  - For example, a search for "sale" returns results that include all resources with the keyword "sale" in their metadata. Filtering by **Models** and **Sources** refines these results to only include models or sources.
+
+- When searching for an exact column name, the results show all relational nodes containing that column in their schemas. If there's a match, a notice in the search result indicates the resource contains the specified column.
+
+### Search with selectors 
 
-You can search with [selector methods](/reference/node-selection/methods). Below are the selectors currently available in dbt Explorer:   
+You can search with [selectors](/reference/node-selection/methods). Below are the selectors currently available in dbt Explorer:   
 
 - `fqn:` &mdash; Find resources by [file or fully qualified name](/reference/node-selection/methods#the-fqn-method). This selector is the search bar's default. If you want to use the default, it's unnecessary to add `fqn:` before the search term.
 - `source:` &mdash; Find resources by a specified [source](/reference/node-selection/methods#the-source-method).
@@ -91,23 +103,15 @@ You can search with [selector methods](/reference/node-selection/methods). Below
 
 </VersionBlock>
 
-### Search with graph operators 
+Because the results of selectors are immutable, the filter side panel is not available with this search method. 
 
-You can use [graph operators](/reference/node-selection/graph-operators) on keywords or selector methods. For example, `+orders` returns all the parents of `orders`. 
-
-### Search with set operators 
+When searching with selector methods, you can also use [graph operators](/reference/node-selection/graph-operators). For example, `+orders` returns all the parents of `orders`. This functionality is not available for keyword search.
 
 You can use multiple selector methods in your search query with [set operators](/reference/node-selection/set-operators). A space implies a union set operator and a comma for an intersection. For example:
 - `resource_type:metric,tag:nightly` &mdash; Returns metrics with the tag `nightly`
 - `+snowplow_sessions +fct_orders` &mdash; Returns resources that are parent nodes of either `snowplow_sessions` or `fct_orders`
 
-### Search with both keywords and selector methods
-
-You can use keyword search to highlight results that are filtered by the selector search. For example, if you don't have a resource called `customers`, then `resource_type:metric customers` returns all the metrics in your project and highlights those that are related to the term `customers` in the name, in a column, tagged as customers, and so on.
-
-When searching in this way, the selectors behave as filters that you can use to narrow the search and keywords as a way to find matches within those filtered results.
-
-<LoomVideo id='7ff433c392dd48d68311e17ce2adc57f' />
+<LoomVideo id='712c8b74a6e242699ad2d32e2b0693a5' />
 
 ## Browse with the sidebar
 
@@ -120,7 +124,7 @@ To browse using a different view, you can choose one of these options from the *
 - **File Tree** &mdash; All resources in the project organized by the file in which they are defined. This mirrors the file tree in your dbt project repository.
 - **Database** &mdash; All resources in the project organized by the database and schema in which they are built. This mirrors your data platform's structure that represents the [applied state](/docs/dbt-cloud-apis/project-state) of your project. 
 
-<LoomVideo id='5d4a4b7904184b76bc907edad1555163' />
+<LoomVideo id='3bd3b16b22ab49ea809c9a863be06d05' />
 
 ## View model versions
 
@@ -132,7 +136,7 @@ You can view the definition and latest run results of any resource in your proje
 The details (metadata) available to you depends on the resource’s type, its definition, and the [commands](/docs/deploy/job-commands) that run within jobs in the production environment. 
 
 
-<LoomVideo id='2b2ae3e376fa46e4b344eb5becfd730e' />
+<LoomVideo id='cefbb113427e45fd8106607476ec0676' />
 
 ### Example of model details
 
@@ -143,11 +147,11 @@ An example of the details you might get for a model:
     - **Lineage** graph &mdash; The model’s lineage graph that you can interact with. The graph includes one parent node and one child node from the model. Click the Expand icon in the graph's upper right corner to view the model in full lineage graph mode.
     - **Description** section &mdash; A [description of the model](/docs/collaborate/documentation#adding-descriptions-to-your-project).
     - **Recent** section &mdash; Information on the last time the model ran, how long it ran for, whether the run was successful, the job ID, and the run ID.
-    - **Tests** section &mdash; [Tests](/docs/build/tests) for the model.
+    - **Tests** section &mdash; [Tests](/docs/build/tests) for the model, including a status indicator for the latest test status. A :white_check_mark: denotes a passing test. 
     - **Details** section &mdash; Key properties like the model’s relation name (for example, how it’s represented and how you can query it in the data platform: `database.schema.identifier`); model governance attributes like access, group, and if contracted; and more.
     - **Relationships** section &mdash; The nodes the model **Depends On**, is **Referenced by**, and (if applicable) is **Used by** for projects that have declared the models' project as a dependency.
 - **Code** tab &mdash; The source code and compiled code for the model.
-- **Columns** tab &mdash; The available columns in the model. This tab also shows tests results (if any) that you can select to view the test's details page. A :white_check_mark: denotes a passing test. 
+- **Columns** tab &mdash; The available columns in the model. This tab also shows tests results (if any) that you can select to view the test's details page. A :white_check_mark: denotes a passing test. To filter the columns in the resource, you can use the search bar that's located at the top of the columns view. 
 
 
 ### Example of exposure details
@@ -189,47 +193,6 @@ An example of the details you might get for each source table within a source co
     - **Relationships** section &mdash; A table that lists all the sources used with their freshness status, the timestamp of when freshness was last checked, and the timestamp of when the source was last loaded.
 - **Columns** tab &mdash; The available columns in the source. This tab also shows tests results (if any) that you can select to view the test's details page. A :white_check_mark: denotes a passing test.
 
-## About project-level lineage
-You can also view all the different projects and public models in the account, where the public models are defined, and how they are used to gain a better understanding about your cross-project resources.
-
-When viewing the resource-level lineage graph for a given project that uses cross-project references, you can see cross-project relationships represented in the DAG. The iconography is slightly different depending on whether you're viewing the lineage of an upstream producer project or a downstream consumer project. 
-
-When viewing an upstream (parent) project that produces public models that are imported by downstream (child) projects, public models will have a counter icon in their upper right corner that indicates the number of projects that declare the current project as a dependency. Selecting that model reveals the lineage to show the specific projects that are dependent on this model. Projects show up in this counter if they declare the parent project as a dependency in its `dependencies.yml` regardless of whether or not there's a direct `{{ ref() }}` against the public model. Selecting a project node from a public model opens the resource-level lineage graph for that project, which is subject to your permissions.  
-
-<Lightbox src="/img/docs/collaborate/dbt-explorer/cross-project-lineage-parent.png" width="85%" height="100" title="Cross-project lineage in a parent project"/>
-
-When viewing a downstream (child) project that imports and refs public models from upstream (parent) projects, public models will show up in the lineage graph and display an icon on the graph edge that indicates what the relationship is to a model from another project. Hovering over this icon indicates the specific dbt Cloud project that produces that model. Double-clicking on a model from another project opens the resource-level lineage graph of the parent project, which is subject to your permissions.
-
-
-<Lightbox src="/img/docs/collaborate/dbt-explorer/cross-project-lineage-child.png" width="85%" height="100" title="Cross-project lineage in a child project"/>
-
-### Explore the project-level lineage graph
-
-For cross-project collaboration, you can interact with the DAG in all the same ways as described in [Explore your project's lineage](#project-lineage) but you can also interact with it at the project level and view the details. 
-
-To get a list view of all the projects, select the account name at the top of the **Explore** page near the navigation bar. This view includes a public model list, project list, and a search bar for project searches. You can also view the project-level lineage graph by clicking the Lineage view icon in the page's upper right corner.
-
-If you have permissions for a project in the account, you can view all public models used across the entire account. However, you can only view full public model details and private models if you have permissions for a project where the models are defined.
-
-From the project-level lineage graph, you can:
-
-- Click the Lineage view icon (in the graph’s upper right corner) to view the cross-project lineage graph.
-- Click the List view icon (in the graph’s upper right corner) to view the project list.
-    - Select a project from the **Projects** tab to switch to that project’s main **Explore** page.
-    - Select a model from the **Public Models** tab to view the [model’s details page](#view-resource-details).
-    - Perform searches on your projects with the search bar.
-- Select a project node in the graph (double-clicking) to switch to that particular project’s lineage graph.
-
-When you select a project node in the graph, a project details panel opens on the graph’s right-hand side where you can:
-
-- View counts of the resources defined in the project.
-- View a list of its public models, if any.
-- View a list of other projects that uses the project, if any.
-- Click **Open Project Lineage** to switch to the project’s lineage graph.
-- Click the Share icon to copy the project panel link to your clipboard so you can share the graph with someone.
-
-<LoomVideo id='606f02e1cce343eba7e1061d6273ff0a?t=1' />
-
 ## Related content
 - [Enterprise permissions](/docs/cloud/manage-access/enterprise-permissions) 
 - [About model governance](/docs/collaborate/govern/about-model-governance)

diff --git a/website/docs/docs/dbt-versions/release-notes/02-Nov-2023/explorer-updates-rn.md b/website/docs/docs/dbt-versions/release-notes/02-Nov-2023/explorer-updates-rn.md
@@ -0,0 +1,32 @@
+---
+title: "Enhancement: New features and UI changes to dbt Explorer"
+description: "November 2023: New features and UI changes to dbt Explorer, including a new filter panel, improved lineage graph, and detailed resource information."
+sidebar_label: "Enhancement: New features and UI changes to dbt Explorer"
+sidebar_position: 08
+tags: [Nov-2023]
+---
+
+dbt Labs is excited to announce the latest features and UI updates to dbt Explorer!
+
+For more details, refer to [Explore your dbt projects](/docs/collaborate/explore-projects).
+
+## The project's lineage graph 
+
+- The search bar in the full lineage graph is now more prominent. 
+- It's easier to navigate across projects using the breadcrumbs.
+- The new context menu (right click) makes it easier to focus on a node or to view its lineage.
+
+<LoomVideo id='1d28cc6443a94f00bbcb43d6ab3ad0d9' />
+
+## Search improvements 
+
+- When searching with keywords, a new side panel UI helps you filter search results by resource type, tag, column, and other key properties (instead of manually defining selectors). 
+- Search result logic is clearly explained. For instance, indicating whether a resource contains a column name (exact match only).
+
+<LoomVideo id='5c4dc30afa824897a8c8b3ed37cee782?t=14' />
+
+## Resource details
+- Model test result statuses are now displayed on the model details page. 
+- Column names can now be searched within the list.
+
+<LoomVideo id='04f3e35cdc0c41109e91f7692d432059' />
diff --git a/website/sidebars.js b/website/sidebars.js
@@ -415,7 +415,15 @@ const sidebarSettings = {
       link: { type: "doc", id: "docs/collaborate/collaborate-with-others" },
       items: [
         "docs/collaborate/collaborate-with-others",
-        "docs/collaborate/explore-projects",
+        {
+          type: "category",
+          label: "Explore dbt projects",
+          link: { type: "doc", id: "docs/collaborate/explore-projects" },
+          items: [
+            "docs/collaborate/explore-projects",
+            "docs/collaborate/explore-multiple-projects",
+          ],
+        },
         {
           type: "category",
           label: "Git version control",