Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add python sdk for SL #5801

Merged
merged 27 commits into from
Jul 19, 2024
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
023b804
add new python sdk
mirnawong1 Jul 18, 2024
b66e147
add
mirnawong1 Jul 18, 2024
100be53
add
mirnawong1 Jul 18, 2024
87d87ac
final updates
mirnawong1 Jul 18, 2024
8c5b5b6
update link
mirnawong1 Jul 18, 2024
166d66e
add link
mirnawong1 Jul 18, 2024
3eacaa6
Merge branch 'current' into add-python-sdk
mirnawong1 Jul 18, 2024
fa6ac49
fold lucas feedback
mirnawong1 Jul 18, 2024
177c85a
Merge branch 'add-python-sdk' of https://github.com/dbt-labs/docs.get…
mirnawong1 Jul 18, 2024
159c046
Update website/docs/docs/dbt-cloud-apis/sl-api-overview.md
mirnawong1 Jul 18, 2024
1342cb0
Update website/docs/docs/dbt-cloud-apis/sl-python-sdk.md
mirnawong1 Jul 18, 2024
6c0c054
Update sl-python-sdk.md
mirnawong1 Jul 18, 2024
79f6401
Merge branch 'current' into add-python-sdk
mirnawong1 Jul 18, 2024
77f0115
Update website/docs/docs/dbt-cloud-apis/sl-api-overview.md
mirnawong1 Jul 19, 2024
ffcbc50
Merge branch 'current' into add-python-sdk
mirnawong1 Jul 19, 2024
b0b72f4
Update website/docs/docs/dbt-cloud-apis/sl-python-sdk.md
mirnawong1 Jul 19, 2024
02748ee
Update website/docs/docs/dbt-cloud-apis/sl-python-sdk.md
mirnawong1 Jul 19, 2024
7187f77
Update website/docs/docs/dbt-cloud-apis/sl-python-sdk.md
mirnawong1 Jul 19, 2024
2173b82
Update website/docs/docs/dbt-cloud-apis/sl-python-sdk.md
mirnawong1 Jul 19, 2024
14ec49d
Update website/docs/docs/dbt-cloud-apis/sl-python-sdk.md
mirnawong1 Jul 19, 2024
2b5d06b
Update website/docs/docs/dbt-cloud-apis/sl-python-sdk.md
mirnawong1 Jul 19, 2024
8421577
Update sl-python-sdk.md
mirnawong1 Jul 19, 2024
5ea5392
Update sl-python-sdk.md
mirnawong1 Jul 19, 2024
7005f67
Update sl-python-sdk.md
mirnawong1 Jul 19, 2024
eaa01ee
Update website/docs/docs/dbt-cloud-apis/sl-python-sdk.md
mirnawong1 Jul 19, 2024
c66b616
Merge branch 'current' into add-python-sdk
mirnawong1 Jul 19, 2024
0e9ca7a
Merge branch 'current' into add-python-sdk
mirnawong1 Jul 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions website/docs/docs/dbt-cloud-apis/sl-api-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,12 @@ plan="dbt Cloud Team or Enterprise"
link="/docs/dbt-cloud-apis/sl-graphql"
icon="dbt-bit"/>

<Card
title="Python SDK"
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved
body="Use the Python SDK to interact with the dbt Semantic Layer using Python."
link="/docs/dbt-cloud-apis/sl-python"
icon="dbt-bit"/>

<Card
title="Semantic manifest"
body="Learn about the semantic manifest.json file and how you can use artifacts to gain insights about your Semantic Layer."
Expand Down
138 changes: 138 additions & 0 deletions website/docs/docs/dbt-cloud-apis/sl-python-sdk.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
---
title: "Python SDK"
id: sl-python
description: "Learn how to use the dbt Semantic Layer SDK for Python library to interact with the dbt Semantic Layer."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're calling it the Python SDK everywhere else, but the Semantic Layer SDK here. Might be a little confusing

tags: [Semantic Layer, APIs]
keywords: [dbt Cloud, API, dbt Semantic Layer, python]
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved
sidebar_label: "Python SDK"
---

mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved
The [`dbt-sl-sk` Python software development kit](https://github.com/dbt-labs/semantic-layer-sdk-python) (SDK) is a Python library that provides you with easy access to the dbt Semantic Layer with Python. It allows developers to interact with the Semantic Layer APIs and query metrics and dimensions in downstream tools.
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved

## Installation

To install the SDK, you'll need to specify optional dependencies depending on whether you want to use it synchronously (backed by [requests](https://github.com/psf/requests/)) or with asynchronous ([asyncio](https://docs.python.org/3/library/asyncio.html) backed by [aiohttp](https://github.com/aio-libs/aiohttp/)).
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved

<Tabs>
<TabItem value="sync" label="Sync installation">

Sync installation means your program waits for each task to finish before moving on to the next one.

It's simpler and easier to understand. Suitable for smaller tasks or when your program doesn't need to handle many tasks at the same time.
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved

```bash
pip install dbt-sl-sdk[sync]
```
</TabItem>

<TabItem value="async" label="Async installation">

Async installation means your program can start a task and then move on to other tasks while waiting for the first one to finish.

It can handle many tasks at once without waiting, making it faster and more efficient for larger tasks or when you need to manage multiple tasks at the same time.
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved

```bash
pip install dbt-sl-sdk[async]
```
</TabItem>
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved
</Tabs>

## Usage
To run operations against the Semantic Layer APIs, instantiate (create an instance of) a `SemanticLayerClient` with your specific [API connection parameters](/docs/dbt-cloud-apis/sl-api-overview):

```python
from dbtsl import SemanticLayerClient

client = SemanticLayerClient(
environment_id=123,
auth_token="<your-semantic-layer-api-token>",
host="semantic-layer.cloud.getdbt.com",
)

# query the first metric by `metric_time`
def main():
with client.session():
metrics = client.metrics()
table = client.query(
metrics=[metrics[0].name],
group_by=["metric_time"],
)
print(table)

main()
```

**Note**: All method calls that reach out to the APIs need to be within a `client.session()` context manager. This allows the client to establish a connection to the APIs only once and reuse the same connection between API calls.
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved

### asyncio usage
If you're using asyncio, import `AsyncSemanticLayerClient` from `dbtsl.asyncio`. The APIs of `SemanticLayerClient` and `AsyncSemanticLayerClient` are identical, but the async version has async methods that you need to `await`.
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved

```python
import asyncio
from dbtsl.asyncio import AsyncSemanticLayerClient

client = AsyncSemanticLayerClient(
environment_id=123,
auth_token="<your-semantic-layer-api-token>",
host="semantic-layer.cloud.getdbt.com",
)

async def main():
async with client.session():
metrics = await client.metrics()
table = await client.query(
metrics=[metrics[0].name],
group_by=["metric_time"],
)
print(table)

asyncio.run(main())

```

## Integrate with dataframe libraries

The SDK returns all query data as [pyarrow](https://arrow.apache.org/docs/python/index.html) tables.

To use the data with libraries like [pandas](https://pandas.pydata.org/) or [polars](https://pola.rs/), manually convert the data into the desired format:
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved

#### If you're using pandas
Copy link
Contributor

@matthewshaver matthewshaver Jul 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be easier to link to

Suggested change
#### If you're using pandas
#### Pandas

Copy link
Contributor Author

@mirnawong1 mirnawong1 Jul 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i totally understand your bit here and that was my original header but i do also like writing to the user (as mentioned by @runleonarun previously) so opted for the 'you're using'. I'm happy to keep it as is since its an h4 unless you have strong feelings.


```python
# ... initialize client

arrow_table = client.query(...)
pandas_df = arrow_table.to_pandas()

```

#### If you're using polars
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#### If you're using polars
#### Polars

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as above


```python
import polars as pl

# ... initialize client

arrow_table = client.query(...)
polars_df = pl.from_arrow(arrow_table)
```

## Usage examples
For additional usage examples, check out our [usage examples](https://github.com/dbt-labs/semantic-layer-sdk-python/tree/main/examples), some which include:
mirnawong1 marked this conversation as resolved.
Show resolved Hide resolved

- [Fetching dimension values sync](https://github.com/dbt-labs/semantic-layer-sdk-python/blob/main/examples/fetch_dimension_values_sync.py)
- Fetching metrics [async](https://github.com/dbt-labs/semantic-layer-sdk-python/blob/main/examples/fetch_metric_async.py) and [sync](https://github.com/dbt-labs/semantic-layer-sdk-python/blob/main/examples/fetch_metric_sync.py)
- [List saved queries async](https://github.com/dbt-labs/semantic-layer-sdk-python/blob/main/examples/list_saved_queries_async.py)

## Disable telemetry
By default, the SDK sends some [platform-related information](https://github.com/dbt-labs/semantic-layer-sdk-python/blob/main/dbtsl/env.py) to dbt Labs. To opt out, set the `PLATFORM.anonymous` attribute to `True`:

```python
from dbtsl.env import PLATFORM
PLATFORM.anonymous = True

# ... initialize client
```

## Contribute
To contribute to this project, check out our [contribution guidelines](https://github.com/dbt-labs/semantic-layer-sdk-python/blob/main/CONTRIBUTING.md) and open a GitHub [issue](https://github.com/dbt-labs/semantic-layer-sdk-python/issues) or [pull request](https://github.com/dbt-labs/semantic-layer-sdk-python/pulls).
1 change: 1 addition & 0 deletions website/sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -669,6 +669,7 @@ const sidebarSettings = {
"docs/dbt-cloud-apis/sl-api-overview",
"docs/dbt-cloud-apis/sl-jdbc",
"docs/dbt-cloud-apis/sl-graphql",
"docs/dbt-cloud-apis/sl-python",
"docs/dbt-cloud-apis/sl-manifest",
],
},
Expand Down
Loading