Skip to content

Commit

Permalink
Merge branch 'current' into update-ratio-metrics
Browse files Browse the repository at this point in the history
  • Loading branch information
mirnawong1 authored Oct 2, 2024
2 parents d6eb0d4 + a1ca53a commit c6ab151
Show file tree
Hide file tree
Showing 7 changed files with 21 additions and 52 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ After the initial release I started to expand to cover the rest of the dbt Cloud

In this example we’ll download a `catalog.json` artifact from the latest run of a dbt Cloud job using `dbt-cloud run list` and `dbt-cloud get-artifact` and then write a simple Data Catalog CLI application using the same tools that are used in `dbt-cloud-cli` (i.e., `click` and `pydantic`). Let’s dive right in!

The first command we need is the `dbt-cloud run list` which uses an [API endpoint](https://docs.getdbt.com/dbt-cloud/api-v2-legacy#/operations/List%20Runs) that returns runs sorted by creation date, with the most recent run appearing first. The command returns a JSON response that has one top-level attribute `data` that contains a list of runs. We’ll need to extract the `id` attribute of the first one and to do that we use [jq](https://stedolan.github.io/jq/):
The first command we need is the `dbt-cloud run list` which uses an [API endpoint](https://docs.getdbt.com/dbt-cloud/api-v2#/operations/List%20Runs) that returns runs sorted by creation date, with the most recent run appearing first. The command returns a JSON response that has one top-level attribute `data` that contains a list of runs. We’ll need to extract the `id` attribute of the first one and to do that we use [jq](https://stedolan.github.io/jq/):

```
latest_run_id=$(dbt-cloud run list --job-id $DBT_CLOUD_JOB_ID | jq .data[0].id -r)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -96,4 +96,4 @@ dbt Cloud will always connect to your data platform from the IP addresses specif

Be sure to allow traffic from these IPs in your firewall, and include them in any database grants.

Allowing these IP addresses only enables the connection to your <Term id="data-warehouse" />. However, you might want to send API requests from your restricted network to the dbt Cloud API. For example, you could use the API to send a POST request that [triggers a job to run](https://docs.getdbt.com/dbt-cloud/api-v2-legacy#operation/triggerRun). Using the dbt Cloud API requires that you allow the `cloud.getdbt.com` subdomain. For more on the dbt Cloud architecture, see [Deployment architecture](/docs/cloud/about-cloud/architecture).
Allowing these IP addresses only enables the connection to your <Term id="data-warehouse" />. However, you might want to send API requests from your restricted network to the dbt Cloud API. Using the dbt Cloud API requires allowing the `cloud.getdbt.com` subdomain. For more on the dbt Cloud architecture, see [Deployment architecture](/docs/cloud/about-cloud/architecture).
32 changes: 0 additions & 32 deletions website/docs/docs/dbt-cloud-apis/migrating-to-v2.md

This file was deleted.

4 changes: 2 additions & 2 deletions website/docs/docs/dbt-versions/2022-release-notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,9 +117,9 @@ Read more about the new [Cloud IDE features](/docs/cloud/dbt-cloud-ide/develop-i

### List Steps API endpoint deprecation warning

On October 14th, 2022 dbt Labs is deprecating the [List Steps](https://docs.getdbt.com/dbt-cloud/api-v2-legacy#tag/Runs/operation/listSteps) API endpoint. From October 14th, any GET requests to this endpoint will fail. Please prepare to stop using the List Steps endpoint as soon as possible.
On October 14th, 2022 dbt Labs is deprecating the List Steps API endpoint. From October 14th, any GET requests to this endpoint will fail. Please prepare to stop using the List Steps endpoint as soon as possible.

dbt Labs will continue to maintain the [Get Run](https://docs.getdbt.com/dbt-cloud/api-v2-legacy#tag/Runs/operation/getRunById) endpoint, which is a viable alternative depending on the use case.
dbt Labs will continue to maintain the [Retrieve Run](https://docs.getdbt.com/dbt-cloud/api-v2#/operations/Retrieve%20Run) endpoint, which is a viable alternative depending on the use case.

You can fetch run steps for an individual run with a GET request to the following URL, replacing `YOUR_ACCESS_URL` with the [appropriate Access URL](/docs/cloud/about-cloud/access-regions-ip-addresses) for your region and plan:

Expand Down
6 changes: 3 additions & 3 deletions website/docs/docs/dbt-versions/2023-release-notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -691,7 +691,7 @@ Archived release notes for dbt Cloud from 2023

**Beginning June 1, 2023,** developers on the dbt Cloud multi-tenant application will be able to self-serve access to their account’s run history through the dbt Cloud user interface (UI) and API for only 365 days, on a rolling basis. Older run history will be available for download by reaching out to Customer Support. We're seeking to minimize the amount of metadata we store while maximizing application performance.

Specifically, all `GET` requests to the dbt Cloud [Runs endpoint](https://docs.getdbt.com/dbt-cloud/api-v2-legacy#tag/Runs) will return information on runs, artifacts, logs, and run steps only for the past 365 days. Additionally, the run history displayed in the dbt Cloud UI will only show runs for the past 365 days.
Specifically, all `GET` requests to the dbt Cloud [Runs endpoint](https://docs.getdbt.com/dbt-cloud/api-v2#/operations/List%20Runs) will return information on runs, artifacts, logs, and run steps only for the past 365 days. Additionally, the run history displayed in the dbt Cloud UI will only show runs for the past 365 days.

<Lightbox src="/img/docs/dbt-cloud/rn-run-history.jpg" width="100%" title="The dbt Cloud UI displaying a Run History"/>

Expand Down Expand Up @@ -808,7 +808,7 @@ Archived release notes for dbt Cloud from 2023

:::

For more info, refer to our [documentation](/dbt-cloud/api-v2-legacy#tag/Runs/operation/listRunsForAccount).
For more info, refer to our [documentation](https://docs.getdbt.com/dbt-cloud/api-v2#/operations/List%20Runs).

</Expandable>

Expand Down Expand Up @@ -936,7 +936,7 @@ Archived release notes for dbt Cloud from 2023

This maximum limit applies to [multi-tenant instances](/docs/cloud/about-cloud/access-regions-ip-addresses) only, and _does not_ apply to single tenant instances.

Refer to the [Pagination](https://docs.getdbt.com/dbt-cloud/api-v2-legacy#section/Pagination) section for more information on this change.
Refer to the [Pagination](https://docs.getdbt.com/dbt-cloud/api-v2#/) section of the overview for more information on this change.

</Expandable>

Expand Down
15 changes: 8 additions & 7 deletions website/docs/reference/resource-configs/snowflake-configs.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,13 +33,14 @@ flags:
</File>


The following configurations are supported:

| Field | Type | Required | Description | Sample input |
|-------|------|----------|-------------|-------|
|Table Format | String | Yes | Configures the object table format. | `iceberg` |
|External volume | String | Yes | Specifies the identifier (name) of the external volume where the Iceberg table stores its metadata files and data in Parquet format. If you don’t specify this parameter, the Iceberg table defaults to the external volume for the schema, database, or account. The schema takes precedence over the database, and the database takes precedence over the account.| `my_s3_bucket` |
|Base location Subpath |String | No | Defines the directory path for Snowflake to write the table data and metadata files. If you change this after creating the table, it will write a new table but leave the existing one. | `jaffle_marketing_folder` |
The following configurations are supported.
For more information, check out the Snowflake reference for [`CREATE ICEBERG TABLE` (Snowflake as the catalog)](https://docs.snowflake.com/en/sql-reference/sql/create-iceberg-table-snowflake).

| Field | Type | Required | Description | Sample input | Note |
| --------------------- | ------ | -------- | -------------------------------------------------------------------------------------------------------------------------- | ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Table Format | String | Yes | Configures the objects table format. | `iceberg` | `iceberg` is the only accepted value. |
| External volume | String | Yes(*) | Specifies the identifier (name) of the external volume where Snowflake writes the Iceberg table's metadata and data files. | `my_s3_bucket` | *You don't need to specify this if the account, database, or schema already has an associated external volume. [More info](https://docs.snowflake.com/en/sql-reference/sql/create-iceberg-table-snowflake#:~:text=Snowflake%20Table%20Structures.-,external_volume) |
| Base location Subpath | String | No | An optional suffix to add to the `base_location` path that dbt automatically specifies. | `jaffle_marketing_folder` | We recommend that you do not specify this. Modifying this parameter results in a new Iceberg table. See [Base Location](#base-location) for more info. |

### Example configuration

Expand Down
12 changes: 6 additions & 6 deletions website/docs/reference/snapshot-configs.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,12 +94,12 @@ snapshots:
snapshots:
- name: <string>
config:
database: <string>
schema: <string>
unique_key: <column_name_or_expression>
strategy: timestamp | check
updated_at: <column_name>
check_cols: [<column_name>] | all
[database](/reference/resource-configs/database): <string>
[schema](/reference/resource-configs/schema): <string>
[unique_key](/reference/resource-configs/unique_key): <column_name_or_expression>
[strategy](/reference/resource-configs/strategy): timestamp | check
[updated_at](/reference/resource-configs/updated_at): <column_name>
[check_cols](/reference/resource-configs/check_cols): [<column_name>] | all

```
</File>
Expand Down

0 comments on commit c6ab151

Please sign in to comment.