Skip to content

Commit

Permalink
Merge branch 'current' into mwong-mega-sl-clou-cli
Browse files Browse the repository at this point in the history
  • Loading branch information
mirnawong1 authored Oct 11, 2023
2 parents c1619a7 + 3a9d15e commit de22f98
Show file tree
Hide file tree
Showing 17 changed files with 96 additions and 5 deletions.
23 changes: 23 additions & 0 deletions website/docs/docs/build/semantic-models.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,29 @@ semantic_models:
type: categorical
```
<VersionBlock firstVersion="1.7">
Semantic models support configs in either the schema file or at the project level.
Semantic model config in `models/semantic.yml`:
```yml
semantic_models:
- name: orders
config:
enabled: true | false
group: some_group
```

Semantic model config in `dbt_project.yml`:
```yml
semantic_models:
my_project_name:
+enabled: true | false
+group: some_group
```

</VersionBlock>

### Name

Define the name of the semantic model. You must define a unique name for the semantic model. The semantic graph will use this name to identify the model, and you can update it at any time. Avoid using double underscores (__) in the name as they're not supported.
Expand Down
12 changes: 11 additions & 1 deletion website/docs/docs/cloud/dbt-cloud-ide/lint-format.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,11 @@ With the dbt Cloud IDE, you can seamlessly use [SQLFluff](https://sqlfluff.com/)
- Works with Jinja and SQL,
- Comes with built-in [linting rules](https://docs.sqlfluff.com/en/stable/rules.html). You can also [customize](#customize-linting) your own linting rules.
- Empowers you to [enable linting](#enable-linting) with options like **Lint** (displays linting errors and recommends actions) or **Fix** (auto-fixes errors in the IDE).
- Displays a **Code Quality** tab to view code errors, and provides code quality visibility and management.
- Displays a **Code Quality** tab to view code errors, and provides code quality visibility and management.

:::info Ephemeral models not supported
Linting doesn't support ephemeral models in dbt v1.5 and lower. Refer to the [FAQs](#faqs) for more info.
:::

### Enable linting

Expand Down Expand Up @@ -223,6 +227,12 @@ Currently, running SQLFluff commands from the terminal isn't supported.
Make sure you're on a development branch. Formatting or Linting isn't available on "main" or "read-only" branches.
</details>

<details>
<summary>Why is there inconsistent SQLFluff behavior when running outside the dbt Cloud IDE (such as a GitHub Action)?</summary>
&mdash; Double-check your SQLFluff version matches the one in dbt Cloud IDE (found in the <b>Code Quality</b> tab after a lint operation). <br /><br />
&mdash; If your lint operation passes despite clear rule violations, confirm you're not linting models with ephemeral models. Linting doesn't support ephemeral models in dbt v1.5 and lower.
</details>

## Related docs

- [User interface](/docs/cloud/dbt-cloud-ide/ide-user-interface)
Expand Down
8 changes: 8 additions & 0 deletions website/docs/docs/collaborate/govern/project-dependencies.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,3 +100,11 @@ There are a few cases where installing another internal project as a package can
- Coordinated changes &mdash; In development, if you wanted to test the effects of a change to a public model in an upstream project (`jaffle_finance.monthly_revenue`) on a downstream model (`jaffle_marketing.roi_by_channel`) _before_ introducing changes to a staging or production environment, you can install the `jaffle_finance` package as a package within `jaffle_marketing`. The installation can point to a specific git branch, however, if you find yourself frequently needing to perform end-to-end testing across both projects, we recommend you re-examine if this represents a stable interface boundary.

These are the exceptions, rather than the rule. Installing another team's project as a package adds complexity, latency, and risk of unnecessary costs. By defining clear interface boundaries across teams, by serving one team's public models as "APIs" to another, and by enabling practitioners to develop with a more narrowly-defined scope, we can enable more people to contribute, with more confidence, while requiring less context upfront.

## FAQs

<details>
<summary>Can I define private packages in the <code>dependencies.yml</code> file?</summary>

If you're using private packages with the [git token method](/docs/build/packages#git-token-method), you must define them in the `packages.yml` file instead of the `dependencies.yml` file. This is because conditional rendering (like Jinja-in-yaml) is not supported.
</details>
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ The following profile fields are optional to set up. They let you configure your
| Profile field | Example | Description |
| ----------------------------- | -------------------------------- | ----------------------------------------------------------------------------------------------------------- |
| `threads` | `8` | How many threads dbt should use (default is `1`) |
| `roles` | `system: analyst` | Catalog roles can be set under the optional `roles` parameter using following format: `catalog: role`. |
| `roles` | `system: analyst` | Catalog roles can be set under the optional `roles` parameter using the following format: `catalog: role`. |
| `session_properties` | `query_max_run_time: 4h` | Sets Trino session properties used in the connection. Execute `SHOW SESSION` to see available options |
| `prepared_statements_enabled` | `true` or `false` | Enable usage of Trino prepared statements (used in `dbt seed` commands) (default: `true`) |
| `retries` | `10` | Configure how many times all database operation is retried when connection issues arise (default: `3`) |
Expand Down
2 changes: 1 addition & 1 deletion website/docs/docs/dbt-cloud-apis/user-tokens.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ permissions of the user the that they were created for.
You can find your User API token in the Profile page under the `API Access`
label.

<Lightbox src="/img/api-access-profile.png" title="Finding your API token in your dbt Cloud Profile" />
<Lightbox src="/img/api-access-profile.jpg" title="Finding your API token in your dbt Cloud Profile" />

## FAQs

Expand Down
7 changes: 7 additions & 0 deletions website/docs/docs/use-dbt-semantic-layer/gsheets.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,3 +51,10 @@ To use the filter functionality, choose the dimension you want to filter by and
- If it's a categorical dimension, type in the dimension value you want to filter by (no quotes needed) and press enter.
- Continue adding additional filters as needed with AND and OR. If it's a time dimension, choose the operator and select from the calendar.



**Limited Use Policy Disclosure**

The dbt Semantic Layer for Sheet's use and transfer to any other app of information received from Google APIs will adhere to [Google API Services User Data Policy](https://developers.google.com/terms/api-services-user-data-policy), including the Limited Use requirements.


12 changes: 12 additions & 0 deletions website/docs/reference/commands/cmd-docs.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,18 @@ The command is responsible for generating your project's documentation website b
dbt docs generate
```

<VersionBlock firstVersion="1.7">

Use the `--select` argument to limit the nodes included within `catalog.json`. When this flag is provided, step (3) will be restricted to the selected nodes. All other nodes will be excluded. Step (2) is unaffected.

**Example**:
```shell
dbt docs generate --select +orders
```

</VersionBlock>


Use the `--no-compile` argument to skip re-compilation. When this flag is provided, `dbt docs generate` will skip step (2) described above.

**Example**:
Expand Down
11 changes: 11 additions & 0 deletions website/docs/reference/commands/init.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,10 +17,21 @@ Then, it will:
- Create a new folder with your project name and sample files, enough to get you started with dbt
- Create a connection profile on your local machine. The default location is `~/.dbt/profiles.yml`. Read more in [configuring your profile](/docs/core/connect-data-platform/connection-profiles).

<VersionBlock firstVersion="1.7">

When using `dbt init` to initialize your project, include the `--profile` flag to specify an existing `profiles.yml` as the `profile:` key to use instead of creating a new one. For example, `dbt init --profile`.



If the profile does not exist in `profiles.yml` or the command is run inside an existing project, the command raises an error.

</VersionBlock>

## Existing project

If you've just cloned or downloaded an existing dbt project, `dbt init` can still help you set up your connection profile so that you can start working quickly. It will prompt you for connection information, as above, and add a profile (using the `profile` name from the project) to your local `profiles.yml`, or create the file if it doesn't already exist.


## profile_template.yml

`dbt init` knows how to prompt for connection information by looking for a file named `profile_template.yml`. It will look for this file in two places:
Expand Down
24 changes: 22 additions & 2 deletions website/docs/reference/node-selection/test-selection-examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,14 +19,14 @@ Run generic tests only:


```bash
dbt test --select "test_type:generic"
dbt test --select "test_type:generic"
```

Run singular tests only:


```bash
dbt test --select "test_type:singular"
dbt test --select "test_type:singular"
```

In both cases, `test_type` checks a property of the test itself. These are forms of "direct" test selection.
Expand Down Expand Up @@ -102,8 +102,10 @@ It will only include tests whose references are each within the selected nodes.
Put another way, it will prevent tests from running if one or more of its parents is unselected.

```shell

dbt test --select "orders" --indirect-selection=cautious
dbt build --select "orders" --indirect-selection=cautious

```

</TabItem>
Expand Down Expand Up @@ -137,8 +139,10 @@ It will only include tests whose references are each within the selected nodes.
Put another way, it will prevent tests from running if one or more of its parents is unselected.

```shell

dbt test --select "orders" --indirect-selection=cautious
dbt build --select "orders" --indirect-selection=cautious

```

</TabItem>
Expand All @@ -154,6 +158,7 @@ This is useful in the same scenarios as "cautious", but also includes when a tes
```shell
dbt test --select "orders" --indirect-selection=buildable
dbt build --select "orders" --indirect-selection=buildable

```

</TabItem>
Expand Down Expand Up @@ -189,6 +194,7 @@ Put another way, it will prevent tests from running if one or more of its parent
```shell
dbt test --select "orders" --indirect-selection=cautious
dbt build --select "orders" --indirect-selection=cautious

```

</TabItem>
Expand All @@ -213,8 +219,10 @@ dbt build --select "orders" --indirect-selection=buildable
This mode will only include tests whose references are each within the selected nodes and will ignore all tests from attached nodes.

```shell

dbt test --select "orders" --indirect-selection=empty
dbt build --select "orders" --indirect-selection=empty

```

</TabItem>
Expand All @@ -235,6 +243,9 @@ The following examples should feel somewhat familiar if you're used to executing
```bash
# Run tests on a model (indirect selection)
dbt test --select "customers"

# Run tests on two or more specific models (indirect selection)
dbt test --select "customers orders"

# Run tests on all models in the models/staging/jaffle_shop directory (indirect selection)
dbt test --select "staging.jaffle_shop"
Expand All @@ -258,10 +269,14 @@ The following examples should feel somewhat familiar if you're used to executing

```bash
# tests on all sources

dbt test --select "source:*"

# tests on one source
dbt test --select "source:jaffle_shop"

# tests on two or more specific sources
dbt test --select "source:jaffle_shop source:raffle_bakery"

# tests on one source table
dbt test --select "source:jaffle_shop.customers"
Expand All @@ -276,10 +291,12 @@ Through the combination of direct and indirect selection, there are many ways to


```bash

dbt test --select "assert_total_payment_amount_is_positive" # directly select the test by name
dbt test --select "payments,test_type:singular" # indirect selection, v1.2
dbt test --select "payments,test_type:data" # indirect selection, v0.18.0
dbt test --select "payments" --data # indirect selection, earlier versions

```


Expand All @@ -295,6 +312,7 @@ Through the combination of direct and indirect selection, there are many ways to

# Run tests on all snapshots, which use the 'snapshot' materialization
dbt test --select "config.materialized:snapshot"

```

Note that this functionality may change in future versions of dbt.
Expand Down Expand Up @@ -323,6 +341,7 @@ models:

```bash
dbt test --select "tag:my_column_tag"

```

Currently, tests "inherit" tags applied to columns, sources, and source tables. They do _not_ inherit tags applied to models, seeds, or snapshots. In all likelihood, those tests would still be selected indirectly, because the tag selects its parent. This is a subtle distinction, and it may change in future versions of dbt.
Expand Down Expand Up @@ -351,4 +370,5 @@ models:

```bash
dbt test --select "tag:my_test_tag"

```
Binary file modified website/static/img/Filtering.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified website/static/img/Paginate.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added website/static/img/api-access-profile.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed website/static/img/api-access-profile.png
Binary file not shown.
Binary file modified website/static/img/dbt-cloud-project-setup-flow-next.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified website/static/img/delete_projects_from_dbt_cloud_20221023.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified website/static/img/node_color_example.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed website/static/img/sample_email_data.png
Binary file not shown.

0 comments on commit de22f98

Please sign in to comment.