From 50b7961f15d0311d7fe94c9446c635ce5cd99ca8 Mon Sep 17 00:00:00 2001 From: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> Date: Wed, 2 Oct 2024 11:05:10 -0400 Subject: [PATCH 1/3] Adding feedback to iceberg table (#6212) ## What are you changing in this pull request and why? See https://github.com/dbt-labs/docs.getdbt.com/pull/6162#discussion_r1783306283 ## Checklist - [ ] I have reviewed the [Content style guide](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/content-style-guide.md) so my content adheres to these guidelines. - [ ] The topic I'm writing about is for specific dbt version(s) and I have versioned it according to the [version a whole page](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#adding-a-new-version) and/or [version a block of content](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#versioning-blocks-of-content) guidelines. - [ ] I have added checklist item(s) to this list for anything anything that needs to happen before this PR is merged, such as "needs technical review" or "change base branch." --------- Co-authored-by: Amy Chen <46451573+amychen1776@users.noreply.github.com> --- .../resource-configs/snowflake-configs.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/website/docs/reference/resource-configs/snowflake-configs.md b/website/docs/reference/resource-configs/snowflake-configs.md index 809667ef685..a8cc06ece9f 100644 --- a/website/docs/reference/resource-configs/snowflake-configs.md +++ b/website/docs/reference/resource-configs/snowflake-configs.md @@ -33,13 +33,14 @@ flags: -The following configurations are supported: - -| Field | Type | Required | Description | Sample input | -|-------|------|----------|-------------|-------| -|Table Format | String | Yes | Configures the object table format. | `iceberg` | -|External volume | String | Yes | Specifies the identifier (name) of the external volume where the Iceberg table stores its metadata files and data in Parquet format. If you don’t specify this parameter, the Iceberg table defaults to the external volume for the schema, database, or account. The schema takes precedence over the database, and the database takes precedence over the account.| `my_s3_bucket` | -|Base location Subpath |String | No | Defines the directory path for Snowflake to write the table data and metadata files. If you change this after creating the table, it will write a new table but leave the existing one. | `jaffle_marketing_folder` | +The following configurations are supported. +For more information, check out the Snowflake reference for [`CREATE ICEBERG TABLE` (Snowflake as the catalog)](https://docs.snowflake.com/en/sql-reference/sql/create-iceberg-table-snowflake). + +| Field | Type | Required | Description | Sample input | Note | +| --------------------- | ------ | -------- | -------------------------------------------------------------------------------------------------------------------------- | ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Table Format | String | Yes | Configures the objects table format. | `iceberg` | `iceberg` is the only accepted value. | +| External volume | String | Yes(*) | Specifies the identifier (name) of the external volume where Snowflake writes the Iceberg table's metadata and data files. | `my_s3_bucket` | *You don't need to specify this if the account, database, or schema already has an associated external volume. [More info](https://docs.snowflake.com/en/sql-reference/sql/create-iceberg-table-snowflake#:~:text=Snowflake%20Table%20Structures.-,external_volume) | +| Base location Subpath | String | No | An optional suffix to add to the `base_location` path that dbt automatically specifies. | `jaffle_marketing_folder` | We recommend that you do not specify this. Modifying this parameter results in a new Iceberg table. See [Base Location](#base-location) for more info. | ### Example configuration From 809c4d6801b6837aafd8b8d664e2b0217bc8f11e Mon Sep 17 00:00:00 2001 From: Doug Beatty <44704949+dbeatty10@users.noreply.github.com> Date: Wed, 2 Oct 2024 09:32:45 -0600 Subject: [PATCH 2/3] Hyperlinks for snapshot configs for property YAML files (#6213) [Preview](https://docs-getdbt-com-git-dbeatty10-patch-2-dbt-labs.vercel.app/reference/snapshot-configs#snapshot-specific-configurations) ## What are you changing in this pull request and why? I accidentally overlooked adding hyperlinks during the PR that added this example, so adding them now! ## Checklist - [x] I have reviewed the [Content style guide](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/content-style-guide.md) so my content adheres to these guidelines. --- website/docs/reference/snapshot-configs.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/website/docs/reference/snapshot-configs.md b/website/docs/reference/snapshot-configs.md index ed4e990953a..fd5969a4c8c 100644 --- a/website/docs/reference/snapshot-configs.md +++ b/website/docs/reference/snapshot-configs.md @@ -94,12 +94,12 @@ snapshots: snapshots: - name: config: - database: - schema: - unique_key: - strategy: timestamp | check - updated_at: - check_cols: [] | all + [database](/reference/resource-configs/database): + [schema](/reference/resource-configs/schema): + [unique_key](/reference/resource-configs/unique_key): + [strategy](/reference/resource-configs/strategy): timestamp | check + [updated_at](/reference/resource-configs/updated_at): + [check_cols](/reference/resource-configs/check_cols): [] | all ``` From a1ca53a392a8c09863d425af76b75f777190f00e Mon Sep 17 00:00:00 2001 From: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> Date: Wed, 2 Oct 2024 12:26:09 -0400 Subject: [PATCH 3/3] Changing old API links (#6199) ## What are you changing in this pull request and why? Changes legacy API links to the newer V2 API. Removed an old migration page ## Checklist - [ ] I have reviewed the [Content style guide](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/content-style-guide.md) so my content adheres to these guidelines. - [ ] The topic I'm writing about is for specific dbt version(s) and I have versioned it according to the [version a whole page](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#adding-a-new-version) and/or [version a block of content](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#versioning-blocks-of-content) guidelines. - [ ] I have added checklist item(s) to this list for anything anything that needs to happen before this PR is merged, such as "needs technical review" or "change base branch." --------- Co-authored-by: Ly Nguyen <107218380+nghi-ly@users.noreply.github.com> --- ...dbt-cloud-api-calls-using-dbt-cloud-cli.md | 2 +- .../about-connections.md | 2 +- .../docs/dbt-cloud-apis/migrating-to-v2.md | 32 ------------------- .../docs/dbt-versions/2022-release-notes.md | 4 +-- .../docs/dbt-versions/2023-release-notes.md | 6 ++-- 5 files changed, 7 insertions(+), 39 deletions(-) delete mode 100644 website/docs/docs/dbt-cloud-apis/migrating-to-v2.md diff --git a/website/blog/2022-05-03-making-dbt-cloud-api-calls-using-dbt-cloud-cli.md b/website/blog/2022-05-03-making-dbt-cloud-api-calls-using-dbt-cloud-cli.md index 6758a28638c..f75675ba7ee 100644 --- a/website/blog/2022-05-03-making-dbt-cloud-api-calls-using-dbt-cloud-cli.md +++ b/website/blog/2022-05-03-making-dbt-cloud-api-calls-using-dbt-cloud-cli.md @@ -113,7 +113,7 @@ After the initial release I started to expand to cover the rest of the dbt Cloud In this example we’ll download a `catalog.json` artifact from the latest run of a dbt Cloud job using `dbt-cloud run list` and `dbt-cloud get-artifact` and then write a simple Data Catalog CLI application using the same tools that are used in `dbt-cloud-cli` (i.e., `click` and `pydantic`). Let’s dive right in! -The first command we need is the `dbt-cloud run list` which uses an [API endpoint](https://docs.getdbt.com/dbt-cloud/api-v2-legacy#/operations/List%20Runs) that returns runs sorted by creation date, with the most recent run appearing first. The command returns a JSON response that has one top-level attribute `data` that contains a list of runs. We’ll need to extract the `id` attribute of the first one and to do that we use [jq](https://stedolan.github.io/jq/): +The first command we need is the `dbt-cloud run list` which uses an [API endpoint](https://docs.getdbt.com/dbt-cloud/api-v2#/operations/List%20Runs) that returns runs sorted by creation date, with the most recent run appearing first. The command returns a JSON response that has one top-level attribute `data` that contains a list of runs. We’ll need to extract the `id` attribute of the first one and to do that we use [jq](https://stedolan.github.io/jq/): ``` latest_run_id=$(dbt-cloud run list --job-id $DBT_CLOUD_JOB_ID | jq .data[0].id -r) diff --git a/website/docs/docs/cloud/connect-data-platform/about-connections.md b/website/docs/docs/cloud/connect-data-platform/about-connections.md index 650752b558c..8bec408af2e 100644 --- a/website/docs/docs/cloud/connect-data-platform/about-connections.md +++ b/website/docs/docs/cloud/connect-data-platform/about-connections.md @@ -96,4 +96,4 @@ dbt Cloud will always connect to your data platform from the IP addresses specif Be sure to allow traffic from these IPs in your firewall, and include them in any database grants. -Allowing these IP addresses only enables the connection to your . However, you might want to send API requests from your restricted network to the dbt Cloud API. For example, you could use the API to send a POST request that [triggers a job to run](https://docs.getdbt.com/dbt-cloud/api-v2-legacy#operation/triggerRun). Using the dbt Cloud API requires that you allow the `cloud.getdbt.com` subdomain. For more on the dbt Cloud architecture, see [Deployment architecture](/docs/cloud/about-cloud/architecture). +Allowing these IP addresses only enables the connection to your . However, you might want to send API requests from your restricted network to the dbt Cloud API. Using the dbt Cloud API requires allowing the `cloud.getdbt.com` subdomain. For more on the dbt Cloud architecture, see [Deployment architecture](/docs/cloud/about-cloud/architecture). diff --git a/website/docs/docs/dbt-cloud-apis/migrating-to-v2.md b/website/docs/docs/dbt-cloud-apis/migrating-to-v2.md deleted file mode 100644 index 72616f4b19c..00000000000 --- a/website/docs/docs/dbt-cloud-apis/migrating-to-v2.md +++ /dev/null @@ -1,32 +0,0 @@ ---- -title: "Migrating to dbt Cloud Administrative API v2" -description: "You should migrate to API v2 while we deprecate API v4 " -sidebar_label: "Migrating to API v2" -id: "migrating-to-v2" ---- - - -In an attempt to provide an improved dbt Cloud Administrative API experience, the dbt Cloud API v4 will be deprecated by April 30th, 2023. We suggest you migrate to dbt Cloud Administrative API v2. When migrating from API v4 to API v2, there are a few differences you should consider when querying your dbt Cloud account. - -## Key differences - -When using the [List runs](/dbt-cloud/api-v2-legacy#tag/Runs) endpoint, you can include triggered runs and sort by ID. You can use the following request in v2 to get a similar response as v4, replacing the `{accountId}` with your own and `{YOUR_ACCESS_URL}` with the appropriate [Access URL](https://docs.getdbt.com/docs/cloud/about-cloud/access-regions-ip-addresses) for your region and plan: - -```shell -GET https://{YOUR_ACCESS_URL}/api/v2/accounts/{accountId}/runs/?include_related=[%22trigger%22]&order_by=-id -``` -For example, if your region is EMEA multi-tenant and your account ID is `001`, your endpoint would be: - -```shell -GET https://emea.dbt.com/api/v2/accounts/001/runs/?include_related=[%22trigger%22]&order_by=-id` -``` - -Differences in responses include: - -| Property description | API v4 | API v2 | -|---------------------|-----------|-------------| -| Reverse sort order when you use sort by `-id` | Defaults to order by most recent | Defaults to order by least recent | -| Update to timestamps | Unix timestamps | ISO strings | -| Update to IDs: `id`, `environment_id`, `account_id`, `project_id`, `job_id` | Values are the same, but they are strings | Values are the same, but they are numeric | -| New property for returning runs with the specified status | `status` property | Maps to `status_humanized` | -| New property for including related field with run | `replace` property | Maps to the `trigger` property | diff --git a/website/docs/docs/dbt-versions/2022-release-notes.md b/website/docs/docs/dbt-versions/2022-release-notes.md index e7c5e1290c1..b46c259a6d8 100644 --- a/website/docs/docs/dbt-versions/2022-release-notes.md +++ b/website/docs/docs/dbt-versions/2022-release-notes.md @@ -117,9 +117,9 @@ Read more about the new [Cloud IDE features](/docs/cloud/dbt-cloud-ide/develop-i ### List Steps API endpoint deprecation warning -On October 14th, 2022 dbt Labs is deprecating the [List Steps](https://docs.getdbt.com/dbt-cloud/api-v2-legacy#tag/Runs/operation/listSteps) API endpoint. From October 14th, any GET requests to this endpoint will fail. Please prepare to stop using the List Steps endpoint as soon as possible. +On October 14th, 2022 dbt Labs is deprecating the List Steps API endpoint. From October 14th, any GET requests to this endpoint will fail. Please prepare to stop using the List Steps endpoint as soon as possible. -dbt Labs will continue to maintain the [Get Run](https://docs.getdbt.com/dbt-cloud/api-v2-legacy#tag/Runs/operation/getRunById) endpoint, which is a viable alternative depending on the use case. +dbt Labs will continue to maintain the [Retrieve Run](https://docs.getdbt.com/dbt-cloud/api-v2#/operations/Retrieve%20Run) endpoint, which is a viable alternative depending on the use case. You can fetch run steps for an individual run with a GET request to the following URL, replacing `YOUR_ACCESS_URL` with the [appropriate Access URL](/docs/cloud/about-cloud/access-regions-ip-addresses) for your region and plan: diff --git a/website/docs/docs/dbt-versions/2023-release-notes.md b/website/docs/docs/dbt-versions/2023-release-notes.md index 60f4fd42929..ec635a051dc 100644 --- a/website/docs/docs/dbt-versions/2023-release-notes.md +++ b/website/docs/docs/dbt-versions/2023-release-notes.md @@ -691,7 +691,7 @@ Archived release notes for dbt Cloud from 2023 **Beginning June 1, 2023,** developers on the dbt Cloud multi-tenant application will be able to self-serve access to their account’s run history through the dbt Cloud user interface (UI) and API for only 365 days, on a rolling basis. Older run history will be available for download by reaching out to Customer Support. We're seeking to minimize the amount of metadata we store while maximizing application performance. - Specifically, all `GET` requests to the dbt Cloud [Runs endpoint](https://docs.getdbt.com/dbt-cloud/api-v2-legacy#tag/Runs) will return information on runs, artifacts, logs, and run steps only for the past 365 days. Additionally, the run history displayed in the dbt Cloud UI will only show runs for the past 365 days. + Specifically, all `GET` requests to the dbt Cloud [Runs endpoint](https://docs.getdbt.com/dbt-cloud/api-v2#/operations/List%20Runs) will return information on runs, artifacts, logs, and run steps only for the past 365 days. Additionally, the run history displayed in the dbt Cloud UI will only show runs for the past 365 days. @@ -808,7 +808,7 @@ Archived release notes for dbt Cloud from 2023 ::: - For more info, refer to our [documentation](/dbt-cloud/api-v2-legacy#tag/Runs/operation/listRunsForAccount). + For more info, refer to our [documentation](https://docs.getdbt.com/dbt-cloud/api-v2#/operations/List%20Runs). @@ -936,7 +936,7 @@ Archived release notes for dbt Cloud from 2023 This maximum limit applies to [multi-tenant instances](/docs/cloud/about-cloud/access-regions-ip-addresses) only, and _does not_ apply to single tenant instances. - Refer to the [Pagination](https://docs.getdbt.com/dbt-cloud/api-v2-legacy#section/Pagination) section for more information on this change. + Refer to the [Pagination](https://docs.getdbt.com/dbt-cloud/api-v2#/) section of the overview for more information on this change.