Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Extrica adapter documentation to DBT docs #4616

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion website/docs/docs/community-adapters.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,4 @@ Community adapters are adapter plugins contributed and maintained by members of
| [TiDB](/docs/core/connect-data-platform/tidb-setup) | [Firebolt](/docs/core/connect-data-platform/firebolt-setup) | [MindsDB](/docs/core/connect-data-platform/mindsdb-setup)
| [Vertica](/docs/core/connect-data-platform/vertica-setup) | [AWS Glue](/docs/core/connect-data-platform/glue-setup) | [MySQL](/docs/core/connect-data-platform/mysql-setup) |
| [Upsolver](/docs/core/connect-data-platform/upsolver-setup) | [Databend Cloud](/docs/core/connect-data-platform/databend-setup) | [fal - Python models](/docs/core/connect-data-platform/fal-setup) |
| [TimescaleDB](https://dbt-timescaledb.debruyn.dev/) | | |
| [TimescaleDB](https://dbt-timescaledb.debruyn.dev/) | [Extrica](/docs/core/connect-data-platform/extrica-setup) | |
87 changes: 87 additions & 0 deletions website/docs/docs/core/connect-data-platform/extrica-setup.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
---
title: "Extrica Setup"
description: "Read this guide to learn about the Extrica Trino Query Engine setup in dbt."
id: "extrica-setup"
meta:
maintained_by: Extrica, Trianz
authors: Gaurav Mittal, Viney Kumar, Mohammed Feroz, and Mrinal Mayank
github_repo: 'extricatrianz/dbt-extrica'
pypi_package: 'dbt-extrica'
min_core_version: 'v1.7.2'
cloud_support: 'Not Supported'
min_supported_version: 'n/a'
platform_name: 'Extrica'
config_page: '/reference/resource-configs/extrica-configs'
---
<h2> Overview of {frontMatter.meta.pypi_package} </h2>

<ul>
<li><strong>Maintained by</strong>: {frontMatter.meta.maintained_by}</li>
<li><strong>Authors</strong>: {frontMatter.meta.authors}</li>
<li><strong>GitHub repo</strong>: <a href={`https://github.com/${frontMatter.meta.github_repo}`}>{frontMatter.meta.github_repo}</a></li>
<li><strong>PyPI package</strong>: <code>{frontMatter.meta.pypi_package}</code> <a href={`https://badge.fury.io/py/${frontMatter.meta.pypi_package}`}><img src={`https://badge.fury.io/py/${frontMatter.meta.pypi_package}.svg`}/></a></li>
<li><strong>Supported dbt Core version</strong>: {frontMatter.meta.min_core_version} and newer</li>
<li><strong>dbt Cloud support</strong>: {frontMatter.meta.cloud_support}</li>
<li><strong>Minimum data platform version</strong>: {frontMatter.meta.min_supported_version}</li>
</ul>
<h2> Installing {frontMatter.meta.pypi_package} </h2>

Use `pip` to install the adapter, which automatically installs `dbt-core` and any additional dependencies. Use the following command for installation:

<code>python -m pip install {frontMatter.meta.pypi_package}</code>

<h2> Configuring {frontMatter.meta.pypi_package} </h2>

<p>For {frontMatter.meta.platform_name}-specifc configuration please refer to <a href={frontMatter.meta.config_page}>{frontMatter.meta.platform_name} Configuration</a> </p>


<h2> Connecting to {frontMatter.meta.platform_name} </h2>

#### Example profiles.yml
Here is a example of a dbt-extrica profile parameters. At a minimum, you need to specify `type`, `method`, `username`, `password` `host`, `port`, `schema`, `catalog` and `threads`.
<File name='~/.dbt/profiles.yml'>

```yaml
<profile-name>:
outputs:
dev:
type: extrica
method: jwt
username: [username for jwt auth]
password: [password for jwt auth]
host: [extrica hostname]
port: [port number]
schema: [dev_schema]
catalog: [catalog_name]
threads: [1 or more]

prod:
type: extrica
method: jwt
username: [username for jwt auth]
password: [password for jwt auth]
host: [extrica hostname]
port: [port number]
schema: [dev_schema]
catalog: [catalog_name]
threads: [1 or more]
target: dev

```
</File>

#### Description of Extrica Profile Fields

| Parameter | Type | Description |
|------------|----------|------------------------------------------|
| type | string | Specifies the type of dbt adapter (Extrica). |
| method | jwt | Authentication method for JWT authentication. |
| username | string | Username for JWT authentication. The obtained JWT token is used to initialize a trino.auth.JWTAuthentication object. |
| password | string | Password for JWT authentication. The obtained JWT token is used to initialize a trino.auth.JWTAuthentication object. |
| host | string | The host parameter specifies the hostname or IP address of the Extrica's Trino server. |
| port | integer | The port parameter specifies the port number on which the Extrica's Trino server is listening. |
| schema | string | Schema or database name for the connection. |
| catalog | string | Name of the catalog representing the data source. |
| threads | integer | Number of threads for parallel execution of queries. (1 or more |

<p>For further info, refer to the GitHub repository: <a href={`https://github.com/${frontMatter.meta.github_repo}`}>{frontMatter.meta.github_repo}</a></p>
98 changes: 98 additions & 0 deletions website/docs/reference/resource-configs/extrica-configs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
---
title: "Extrica configurations"
id: "extrica-configs"
---
## Configuring Extrica

#### Install dbt-extrica adapter

```sh
pip install dbt-extrica
```
#### Initialize dbt project
```sh
dbt init
```
* <b> Select Adapters </b> : After running the `dbt init` command, you'll be prompted to select adapters from a list of available adapters. Choose the appropriate adapter for your project.
* <b> Modify Profiles.yml </b> : The dbt init command will create a project structure in current dir and a .dbt folder inside users of your system. Inside .dbt folder, you'll find a profiles.yml file.

#### Configure Profiles.yml
* Open the profiles.yml file in a text editor.
* Locate the section for your selected adapter and project (e.g., extrica).
* Add the necessary connection details such as host, port, user, password, etc.
* Save and close the profiles.yml file.

#### Example profiles.yml
Here is a example of a dbt-extrica profile parameters. At a minimum, you need to specify `type`, `method`, `username`, `password` `host`, `port`, `schema`, `catalog` and `threads`.
<File name='~/.dbt/profiles.yml'>

```yaml
<profile-name>:
outputs:
dev:
type: extrica
method: jwt
username: [username for jwt auth]
password: [password for jwt auth]
host: [extrica hostname]
port: [port number]
schema: [dev_schema]
catalog: [catalog_name]
threads: [1 or more]

prod:
type: extrica
method: jwt
username: [username for jwt auth]
password: [password for jwt auth]
host: [extrica hostname]
port: [port number]
schema: [dev_schema]
catalog: [catalog_name]
threads: [1 or more]
target: dev

```
</File>

#### Check connection is successful
```sh
dbt debug
```
#### To run all models use below command
```sh
dbt run
```
## Model Example:
This dbt model demonstrates a basic transformation workflow, starting with the selection of relevant columns from
the table. The SELECT statement combines the results to present the final transformed data.
<File name='hive_partition_by.sql'>

```sql
{{ config(materialized='table') }}

with customer_insurance_data as (

SELECT
hospital_sales.sale_id_en,
hospital_sales.sales_date,
hospital_sales.hospital_id_en,
hospital_sales.product_id_en,
customer_insurance.customerid,
customer_insurance.gender,
customer_insurance.annual_premium_cost
FROM
orarcle112.annuity.hospital_product_sales_encrypt_csv hospital_sales
INNER JOIN
orarcle112.annuity.insurance_customer_encrypt_csv customer_insurance
ON
hospital_sales.hospital_id_en = customer_insurance.hospital_id_en
)

select *
from customer_insurance_data
```

</File>


2 changes: 2 additions & 0 deletions website/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,7 @@ const sidebarSettings = {
"docs/core/connect-data-platform/decodable-setup",
"docs/core/connect-data-platform/upsolver-setup",
"docs/core/connect-data-platform/starrocks-setup",
"docs/core/connect-data-platform/extrica-setup",
],
},
],
Expand Down Expand Up @@ -723,6 +724,7 @@ const sidebarSettings = {
"reference/resource-configs/oracle-configs",
"reference/resource-configs/upsolver-configs",
"reference/resource-configs/starrocks-configs",
"reference/resource-configs/extrica-configs",
],
},
{
Expand Down