Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

semantic manifest artifact #6267

Merged
merged 4 commits into from
Oct 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 0 additions & 6 deletions website/docs/docs/dbt-cloud-apis/sl-api-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,10 +48,4 @@ plan="dbt Cloud Team or Enterprise"
link="/docs/dbt-cloud-apis/sl-python"
icon="dbt-bit"/>

<Card
title="Semantic manifest"
body="Learn about the semantic manifest.json file and how you can use artifacts to gain insights about your Semantic Layer."
link="/docs/dbt-cloud-apis/sl-manifest"
icon="dbt-bit"/>

</div>
2 changes: 1 addition & 1 deletion website/docs/reference/artifacts/dbt-artifacts.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ dbt has produced artifacts since the release of dbt-docs in v0.11.0. Starting in
### When are artifacts produced? <Lifecycle status="team,enterprise"/>

Most dbt commands (and corresponding RPC methods) produce artifacts:
- [semantic manifest](/docs/dbt-cloud-apis/sl-manifest): produced whenever your dbt project is parsed
- [semantic manifest](/reference/artifacts/sl-manifest): produced whenever your dbt project is parsed
- [manifest](/reference/artifacts/manifest-json): produced by commands that read and understand your project
- [run results](/reference/artifacts/run-results-json): produced by commands that run, compile, or catalog nodes in your DAG
- [catalog](catalog-json): produced by `docs generate`
Expand Down
2 changes: 1 addition & 1 deletion website/docs/reference/artifacts/other-artifacts.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ Each of those points in time contains the `name` and `type` of each node and `su

### semantic_manifest.json

The [`semantic_manifest.json`](/docs/dbt-cloud-apis/sl-manifest) file is useful as an internal interface between `dbt-core` and MetricFlow. As such, it functions as a behind-the-scenes bridge for interaction between the two systems. You can find all of the `semantic_manifest.json` information in the [`semantic_manifest.json`](/docs/dbt-cloud-apis/sl-manifest).
The [`semantic_manifest.json`](/reference/artifacts/sl-manifest) file is useful as an internal interface between `dbt-core` and MetricFlow. As such, it functions as a behind-the-scenes bridge for interaction between the two systems. You can find all of the `semantic_manifest.json` information in the [`semantic_manifest.json`](/reference/artifacts/sl-manifest).

There are two reasons why `semantic_manifest.json` exists alongside `manifest.json`:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,26 +7,24 @@ sidebar_label: "Semantic manifest"
pagination_next: null
---

**Produced by:** Any command that parses your project. This includes all commands _except_ [`deps`](/reference/commands/deps), [`clean`](/reference/commands/clean), [`debug`](/reference/commands/debug), and [`init`](/reference/commands/init).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this require a manifest version similar to the manifest.json?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it does! however i've asked about that here and here bc semantic manifest is missing from the schema.getdbt.com site as well. once i get guidance, I'll add it myself however i didn't want to block this pr for that.


dbt creates an [artifact](/reference/artifacts/dbt-artifacts) file called the _Semantic Manifest_ (`semantic_manifest.json`), which MetricFlow requires to build and run metric queries properly for the dbt Semantic Layer. This artifact contains comprehensive information about your dbt Semantic Layer. It is an internal file that acts as the integration point with MetricFlow.

By using the semantic manifest produced by dbt Core, MetricFlow will instantiate a data flow plan and generate SQL from Semantic Layer query requests. It's a valuable reference that you can use to understand the structure and details of your data models.

Similar to the [`manifest.json` file](/reference/artifacts/manifest-json), the `semantic_manifest.json` file also lives in the [target directory](/reference/global-configs/json-artifacts) of your dbt project where dbt stores various artifacts (such as compiled models and tests) generated during the execution of your project.

## How it's produced

Just like `manifest.json`, the `semantic_manifest.json` is produced whenever your dbt project is parsed. All dbt commands will parse your project and create a `semantic_manifest.json` file, _except_ [`deps`](/reference/commands/deps), [`clean`](/reference/commands/clean), [`debug`](/reference/commands/debug), and [`init`](/reference/commands/init).


## Top level keys
## Top-level keys

Top-level keys for the semantic manifest are:
- `semantic_models` &mdash; Starting points of data with entities, dimensions, and measures, and correspond to models in your dbt project.
- `metrics` &mdash; Functions combining measures, constraints, and so on to define quantitative indicators.
- `project_configuration` &mdash; Contains information around your project configurations

<details>
<summary>Example <code>target/semantic_manifest.json</code> file </summary>
### Example

<File name="target/semantic_manifest.json">

```json
{
Expand Down Expand Up @@ -112,7 +110,7 @@ Top-level keys for the semantic manifest are:
}
```

</details>
</File>

## Related docs

Expand Down
2 changes: 1 addition & 1 deletion website/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -727,7 +727,6 @@ const sidebarSettings = {
"docs/dbt-cloud-apis/sl-jdbc",
"docs/dbt-cloud-apis/sl-graphql",
"docs/dbt-cloud-apis/sl-python",
"docs/dbt-cloud-apis/sl-manifest",
],
},
],
Expand Down Expand Up @@ -1174,6 +1173,7 @@ const sidebarSettings = {
"reference/artifacts/run-results-json",
"reference/artifacts/catalog-json",
"reference/artifacts/sources-json",
"reference/artifacts/sl-manifest",
"reference/artifacts/other-artifacts",
],
},
Expand Down
2 changes: 1 addition & 1 deletion website/snippets/_sl-run-prod-job.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ This section explains how you can perform a job run in your deployment environme
3. To create a new environment, navigate to **Deploy** in the navigation menu, select **Environments**, and then select **Create new environment**.
4. Fill in your deployment credentials with your Snowflake username and password. You can name the schema anything you want. Click **Save** to create your new production environment.
5. [Create a new deploy job](/docs/deploy/deploy-jobs#create-and-schedule-jobs) that runs in the environment you just created. Go back to the **Deploy** menu, select **Jobs**, select **Create job**, and click **Deploy job**.
6. Set the job to run a `dbt parse` job to parse your projects and generate a [`semantic_manifest.json` artifact](/docs/dbt-cloud-apis/sl-manifest) file. Although running `dbt build` isn't required, you can choose to do so if needed.
6. Set the job to run a `dbt parse` job to parse your projects and generate a [`semantic_manifest.json` artifact](/reference/artifacts/sl-manifest) file. Although running `dbt build` isn't required, you can choose to do so if needed.
7. Run the job by clicking the **Run now** button. Monitor the job's progress in real-time through the **Run summary** tab.

Once the job completes successfully, your dbt project, including the generated documentation, will be fully deployed and available for use in your production environment. If any issues arise, review the logs to diagnose and address any errors.
Expand Down
5 changes: 5 additions & 0 deletions website/vercel.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@
"cleanUrls": true,
"trailingSlash": false,
"redirects": [
{
"source": "/docs/dbt-cloud-apis/sl-manifest",
"destination": "/reference/artifacts/sl-manifest",
"permanent": true
},
{
"source": "/docs/cloud/dbt-assist-data",
"destination": "/docs/cloud/dbt-copilot-data",
Expand Down
Loading