Skip to content

Commit

Permalink
Add athena adapter preview (#5858)
Browse files Browse the repository at this point in the history
## What are you changing in this pull request and why?

Adds Amazon Athena adapter configuration page for dbt Cloud (in Preview)
Updates list of available dbt Cloud connections
Adds a release note for the Preview

## Checklist
<!--
Uncomment when publishing docs for a prerelease version of dbt:
- [ ] Add versioning components, as described in [Versioning
Docs](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#versioning-entire-pages)
- [ ] Add a note to the prerelease version [Migration
Guide](https://github.com/dbt-labs/docs.getdbt.com/tree/current/website/docs/docs/dbt-versions/core-upgrade)
-->
- [ ] Review the [Content style
guide](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/content-style-guide.md)
so my content adheres to these guidelines.
- [ ] For [docs
versioning](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#about-versioning),
review how to [version a whole
page](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#adding-a-new-version)
and [version a block of
content](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#versioning-blocks-of-content).
- [ ] Add a checklist item for anything that needs to happen before this
PR is merged, such as "needs technical review" or "change base branch."

Adding or removing pages (delete if not applicable):
- [ ] Add/remove page in `website/sidebars.js`
- [ ] Provide a unique filename for new pages
- [ ] Add an entry for deleted pages in `website/vercel.json`
- [ ] Run link testing locally with `npm run build` to update the links
that point to deleted pages

---------

Co-authored-by: Amy Chen <[email protected]>
Co-authored-by: Mirna Wong <[email protected]>
  • Loading branch information
3 people authored Aug 2, 2024
1 parent 2299540 commit 43a10d1
Show file tree
Hide file tree
Showing 4 changed files with 47 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ pagination_prev: null
---
dbt Cloud can connect with a variety of data platform providers including:
- [AlloyDB](/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb)
- [Amazon Athena (Beta)](/docs/cloud/connect-data-platform/connect-amazon-athena)
- [Amazon Redshift](/docs/cloud/connect-data-platform/connect-redshift-postgresql-alloydb)
- [Apache Spark](/docs/cloud/connect-data-platform/connect-apache-spark)
- [Azure Synapse Analytics](/docs/cloud/connect-data-platform/connect-azure-synapse-analytics)
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
---
title: "Connect Amazon Athena"
id: connect-amazon-athena
description: "Configure the Amazon Athena data platform connection in dbt Cloud."
sidebar_label: "Connect Amazon Athena"
---

# Connect Amazon Athena <Lifecycle status="beta" />

:::note beta

This is a beta feature with limited availability. A public preview will follow shortly, for wider early access. For more information, check out our [product lifecycle](/docs/dbt-versions/product-lifecycles#dbt-cloud) page.

:::

Your environment(s) must be on ["Keep on latest version"](/docs/dbt-versions/versionless-cloud) to use the Amazon Athena connection.

Connect dbt Cloud to Amazon's Athena interactive query service to build your dbt project. The following are the required and optional fields for configuring the Athena connection:

| Field | Option | Description | Type | Required? | Example |
| ----------------------------- | ---------------- | ----------------------------------------------------------------------------------- | ------ | --------- | ------- |
| AWS region name | region_name | AWS region of your Athena instance | String | Required | eu-west-1 |
| Database (catalog) | database | Specify the database (Data catalog) to build models into (lowercase only) | String | Required | awsdatacatalog |
| AWS S3 staging directory | s3_staging_dir | S3 location to store Athena query results and metadata | String | Required | s3://bucket/dbt/ |
| Athena workgroup | work_group | Identifier of Athena workgroup | String | Optional | my-custom-workgroup |
| Athena Spark workgroup | spark_work_group | Identifier of Athena Spark workgroup for running Python models | String | Optional | my-spark-workgroup |
| AWS S3 data directory | s3_data_dir | Prefix for storing tables, if different from the connection's s3_staging_dir | String | Optional | s3://bucket2/dbt/ |
| AWS S3 data naming convention | s3_data_naming | How to generate table paths in s3_data_dir | String | Optional | schema_table_unique |
| AWS S3 temp tables prefix | s3_tmp_table_dir | Prefix for storing temporary tables, if different from the connection's s3_data_dir | String | Optional | s3://bucket3/dbt/ |
| Poll interval | poll_interval | Interval in seconds to use for polling the status of query results in Athena | Integer| Optional | 5 |
| Query retries | num_retries | Number of times to retry a failing query | Integer| Optional | 3 |
| Boto3 retries | num_boto3_retries| Number of times to retry boto3 requests (e.g. deleting S3 files for materialized tables)| Integer | Optional | 5 |
| Iceberg retries | num_iceberg_retries| Number of times to retry iceberg commit queries to fix ICEBERG_COMMIT_ERROR | Integer | Optional | 0 |

### Development credentials

Enter your _development_ (not deployment) credentials with the following fields:

| Field | Option | Description | Type | Required | Example |
| --------------------- | --------------------- | -------------------------------------------------------------------------- | ------ | -------- | -------- |
| AWS Access Key ID | aws_access_key_id | Access key ID of the user performing requests | String | Required | AKIAIOSFODNN7EXAMPLE |
| AWS Secret Access Key | aws_secret_access_key | Secret access key of the user performing requests | String | Required | wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY |
| Schema | schema | Specify the schema (Athena database) to build models into (lowercase only) | String | Required | dbt |
| Threads | threads | | Integer| Optional | 3 |
1 change: 1 addition & 0 deletions website/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ const sidebarSettings = {
},
items: [
"docs/cloud/connect-data-platform/about-connections",
"docs/cloud/connect-data-platform/connect-amazon-athena",
"docs/cloud/connect-data-platform/connect-azure-synapse-analytics",
"docs/cloud/connect-data-platform/connect-microsoft-fabric",
"docs/cloud/connect-data-platform/connect-starburst-trino",
Expand Down
2 changes: 1 addition & 1 deletion website/snippets/_adapters-trusted.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@

<Card
title="Athena"
body="<ul><li><a href='/docs/core/connect-data-platform/athena-setup'>Install with dbt Core </a> </li> </ul><br /><br /><a href=https://badge.fury.io/py/dbt-athena-community><img src=https://badge.fury.io/py/dbt-athena-community.svg/></a>"
body="<ul><li><a href='/docs/cloud/connect-data-platform/connect-amazon-athena'>Set up in dbt Cloud (beta) </a><br /></li><li><a href='/docs/core/connect-data-platform/athena-setup'>Install with dbt Core </a> </li> </ul><br /><br /><a href=https://badge.fury.io/py/dbt-athena-community><img src=https://badge.fury.io/py/dbt-athena-community.svg/></a>"
pills='["dbt Core"]'
icon="athena"/>

Expand Down

0 comments on commit 43a10d1

Please sign in to comment.