Skip to content

Commit

Permalink
Merge branch 'current' into runleonarun-patch-12
Browse files Browse the repository at this point in the history
  • Loading branch information
runleonarun authored Nov 21, 2023
2 parents bf69bcb + b6f1910 commit 0720952
Show file tree
Hide file tree
Showing 64 changed files with 166 additions and 1,046 deletions.
2 changes: 1 addition & 1 deletion website/blog/2021-11-29-open-source-community-growth.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ For starters, I want to know how much conversation is occurring across the vario

There are a ton of metrics that can be tracked in any GitHub project — committers, pull requests, forks, releases — but I started pretty simple. For each of the projects we participate in, I just want to know how the number of GitHub stars grows over time, and whether the growth is accelerating or flattening out. This has become a key performance indicator for open source communities, for better or for worse, and keeping track of it isn't optional.

Finally, I want to know how much Marquez and OpenLineage are being used. It used to be that when you wanted to consume a bit of tech, you'd download a file. Folks like me who study user behavior would track download counts as if they were stock prices. This is no longer the case; today, our tech is increasingly distributed through package managers and image repositories. Docker Hub and PyPI metrics have therefore become good indicators of consumption. Docker image pulls and runs of `pip install` are the modern day download and, as noisy as these metrics are, they indicate a similar level of user commitment.
Finally, I want to know how much Marquez and OpenLineage are being used. It used to be that when you wanted to consume a bit of tech, you'd download a file. Folks like me who study user behavior would track download counts as if they were stock prices. This is no longer the case; today, our tech is increasingly distributed through package managers and image repositories. Docker Hub and PyPI metrics have therefore become good indicators of consumption. Docker image pulls and runs of `python -m pip install` are the modern day download and, as noisy as these metrics are, they indicate a similar level of user commitment.

To summarize, here are the metrics I decided to track (for now, anyway):
- Slack messages (by user/ by community)
Expand Down
4 changes: 2 additions & 2 deletions website/blog/2022-04-14-add-ci-cd-to-bitbucket.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@ pipelines:
artifacts: # Save the dbt run artifacts for the next step (upload)
- target/*.json
script:
- pip install -r requirements.txt
- python -m pip install -r requirements.txt
- mkdir ~/.dbt
- cp .ci/profiles.yml ~/.dbt/profiles.yml
- dbt deps
Expand Down Expand Up @@ -208,7 +208,7 @@ pipelines:
# Set up dbt environment + dbt packages. Rather than passing
# profiles.yml to dbt commands explicitly, we'll store it where dbt
# expects it:
- pip install -r requirements.txt
- python -m pip install -r requirements.txt
- mkdir ~/.dbt
- cp .ci/profiles.yml ~/.dbt/profiles.yml
- dbt deps
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ You probably agree that the latter example is definitely more elegant and easier

In addition to CLI commands that interact with a single dbt Cloud API endpoint there are composite helper commands that call one or more API endpoints and perform more complex operations. One example of composite commands are `dbt-cloud job export` and `dbt-cloud job import` where, under the hood, the export command performs a `dbt-cloud job get` and writes the job metadata to a <Term id="json" /> file and the import command reads job parameters from a JSON file and calls `dbt-cloud job create`. The export and import commands can be used in tandem to move dbt Cloud jobs between projects. Another example is the `dbt-cloud job delete-all` which fetches a list of all jobs using `dbt-cloud job list` and then iterates over the list prompting the user if they want to delete the job. For each job that the user agrees to delete a `dbt-cloud job delete` is performed.

To install the CLI in your Python environment run `pip install dbt-cloud-cli` and you’re all set. You can use it locally in your development environment or e.g. in a GitHub actions workflow.
To install the CLI in your Python environment run `python -m pip install dbt-cloud-cli` and you’re all set. You can use it locally in your development environment or e.g. in a GitHub actions workflow.

## How the project came to be

Expand Down Expand Up @@ -310,7 +310,7 @@ The `CatalogExploreCommand.execute` method implements the interactive exploratio
I’ve included the app in the latest version of dbt-cloud-cli so you can test it out yourself! To use the app you need install dbt-cloud-cli with extra dependencies:

```bash
pip install dbt-cloud-cli[demo]
python -m pip install dbt-cloud-cli[demo]
```

Now you can the run app:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -79,12 +79,12 @@ Depending on which database you’ve chosen, install the relevant database adapt

```text
# install adaptor for duckdb
pip install dbt-duckdb
python -m pip install dbt-duckdb
# OR
# install adaptor for postgresql
pip install dbt-postgres
python -m pip install dbt-postgres
```

### Step 4: Setup dbt profile
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,8 @@ We'll use pip to install MetricFlow and our dbt adapter:
python -m venv [virtual environment name]
source [virtual environment name]/bin/activate
# install dbt and MetricFlow
pip install "dbt-metricflow[adapter name]"
# e.g. pip install "dbt-metricflow[snowflake]"
python -m pip install "dbt-metricflow[adapter name]"
# e.g. python -m pip install "dbt-metricflow[snowflake]"
```

Lastly, to get to the pre-Semantic Layer starting state, checkout the `start-here` branch.
Expand Down
4 changes: 2 additions & 2 deletions website/docs/docs/build/metricflow-commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ MetricFlow is compatible with Python versions 3.8, 3.9, 3.10, and 3.11.

MetricFlow is a dbt package that allows you to define and query metrics in your dbt project. You can use MetricFlow to query metrics in your dbt project in the dbt Cloud CLI, dbt Cloud IDE, or dbt Core.

**Note** &mdash; MetricFlow commands aren't supported in dbt Cloud jobs yet. However, you can add MetricFlow validations with your git provider (such as GitHub Actions) by installing MetricFlow (`pip install metricflow`). This allows you to run MetricFlow commands as part of your continuous integration checks on PRs.
**Note** &mdash; MetricFlow commands aren't supported in dbt Cloud jobs yet. However, you can add MetricFlow validations with your git provider (such as GitHub Actions) by installing MetricFlow (`python -m pip install metricflow`). This allows you to run MetricFlow commands as part of your continuous integration checks on PRs.

<Tabs>

Expand Down Expand Up @@ -54,7 +54,7 @@ You can install [MetricFlow](https://github.com/dbt-labs/metricflow#getting-star

1. Create or activate your virtual environment `python -m venv venv`
2. Run `pip install dbt-metricflow`
* You can install MetricFlow using PyPI as an extension of your dbt adapter in the command line. To install the adapter, run `pip install "dbt-metricflow[your_adapter_name]"` and add the adapter name at the end of the command. For example, for a Snowflake adapter run `pip install "dbt-metricflow[snowflake]"`
* You can install MetricFlow using PyPI as an extension of your dbt adapter in the command line. To install the adapter, run `python -m pip install "dbt-metricflow[your_adapter_name]"` and add the adapter name at the end of the command. For example, for a Snowflake adapter run `python -m pip install "dbt-metricflow[snowflake]"`

**Note**, you'll need to manage versioning between dbt Core, your adapter, and MetricFlow.

Expand Down
6 changes: 3 additions & 3 deletions website/docs/docs/cloud/cloud-cli-installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -155,9 +155,9 @@ If you already have dbt Core installed, the dbt Cloud CLI may conflict. Here are
- Uninstall the dbt Cloud CLI using the command: `pip uninstall dbt`
- Reinstall dbt Core using the following command, replacing "adapter_name" with the appropriate adapter name:
```shell
pip install dbt-adapter_name --force-reinstall
python -m pip install dbt-adapter_name --force-reinstall
```
For example, if I used Snowflake as an adapter, I would run: `pip install dbt-snowflake --force-reinstall`
For example, if I used Snowflake as an adapter, I would run: `python -m pip install dbt-snowflake --force-reinstall`
--------
Expand Down Expand Up @@ -243,7 +243,7 @@ To update, follow the same process explained in [Windows](/docs/cloud/cloud-cli-
To update:
- Make sure you're in your virtual environment
- Run `pip install --upgrade dbt`.
- Run `python -m pip install --upgrade dbt`.
</TabItem>
Expand Down
2 changes: 1 addition & 1 deletion website/docs/docs/connect-adapters.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Explore the fastest and most reliable way to deploy dbt using dbt Cloud, a hoste

Install dbt Core, an open-source tool, locally using the command line. dbt communicates with a number of different data platforms by using a dedicated adapter plugin for each. When you install dbt Core, you'll also need to install the specific adapter for your database, [connect to dbt Core](/docs/core/about-core-setup), and set up a `profiles.yml` file.

With a few exceptions [^1], you can install all [Verified adapters](/docs/supported-data-platforms) from PyPI using `pip install adapter-name`. For example to install Snowflake, use the command `pip install dbt-snowflake`. The installation will include `dbt-core` and any other required dependencies, which may include both other dependencies and even other adapter plugins. Read more about [installing dbt](/docs/core/installation).
With a few exceptions [^1], you can install all [Verified adapters](/docs/supported-data-platforms) from PyPI using `python -m pip install adapter-name`. For example to install Snowflake, use the command `python -m pip install dbt-snowflake`. The installation will include `dbt-core` and any other required dependencies, which may include both other dependencies and even other adapter plugins. Read more about [installing dbt](/docs/core/installation).

[^1]: Here are the two different adapters. Use the PyPI package name when installing with `pip`

Expand Down
14 changes: 3 additions & 11 deletions website/docs/docs/core/connect-data-platform/alloydb-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,18 +14,10 @@ meta:
config_page: '/reference/resource-configs/postgres-configs'
---

## Overview of AlloyDB support
import SetUpPages from '/snippets/_setup-pages-intro.md';

<SetUpPages meta={frontMatter.meta} />

<ul>
<li><strong>Maintained by</strong>: {frontMatter.meta.maintained_by}</li>
<li><strong>Authors</strong>: {frontMatter.meta.authors}</li>
<li><strong>GitHub repo</strong>: <a href={`https://github.com/${frontMatter.meta.github_repo}`}>{frontMatter.meta.github_repo}</a><a href={`https://github.com/${frontMatter.meta.github_repo}`}><img src={`https://img.shields.io/github/stars/${frontMatter.meta.github_repo}?style=for-the-badge`}/></a></li>
<li><strong>PyPI package</strong>: <code>{frontMatter.meta.pypi_package}</code> <a href={`https://badge.fury.io/py/${frontMatter.meta.pypi_package}`}><img src={`https://badge.fury.io/py/${frontMatter.meta.pypi_package}.svg`}/></a></li>
<li><strong>Slack channel</strong>: <a href={frontMatter.meta.slack_channel_link}>{frontMatter.meta.slack_channel_name}</a></li>
<li><strong>Supported dbt Core version</strong>: {frontMatter.meta.min_core_version} and newer</li>
<li><strong>dbt Cloud support</strong>: {frontMatter.meta.cloud_support}</li>
<li><strong>Minimum data platform version</strong>: {frontMatter.meta.min_supported_version}</li>
</ul>

## Profile Configuration

Expand Down
27 changes: 3 additions & 24 deletions website/docs/docs/core/connect-data-platform/athena-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,32 +15,11 @@ meta:
config_page: '/reference/resource-configs/no-configs'
---

<h2> Overview of {frontMatter.meta.pypi_package} </h2>
<!--The following code uses a component and the built-in docusaurus markdown partials file, which contains reusable content assigned in the meta frontmatter. For this page, the partial file is _setup-pages-intro.md. You have to include the 'import' code and then assign the component as needed. -->

<ul>
<li><strong>Maintained by</strong>: {frontMatter.meta.maintained_by}</li>
<li><strong>Authors</strong>: {frontMatter.meta.authors}</li>
<li><strong>GitHub repo</strong>: <a href={`https://github.com/${frontMatter.meta.github_repo}`}>{frontMatter.meta.github_repo}</a><a href={`https://github.com/${frontMatter.meta.github_repo}`}><img src={`https://img.shields.io/github/stars/${frontMatter.meta.github_repo}?style=for-the-badge`}/></a></li>
<li><strong>PyPI package</strong>: <code>{frontMatter.meta.pypi_package}</code> <a href={`https://badge.fury.io/py/${frontMatter.meta.pypi_package}`}><img src={`https://badge.fury.io/py/${frontMatter.meta.pypi_package}.svg`}/></a></li>
<li><strong>Slack channel</strong>: <a href={frontMatter.meta.slack_channel_link}>{frontMatter.meta.slack_channel_name}</a></li>
<li><strong>Supported dbt Core version</strong>: {frontMatter.meta.min_core_version} and newer</li>
<li><strong>dbt Cloud support</strong>: {frontMatter.meta.cloud_support}</li>
<li><strong>Minimum data platform version</strong>: {frontMatter.meta.min_supported_version}</li>
</ul>
import SetUpPages from '/snippets/_setup-pages-intro.md';

<h2> Installing {frontMatter.meta.pypi_package} </h2>

pip is the easiest way to install the adapter:

<code>pip install {frontMatter.meta.pypi_package}</code>

<p>Installing <code>{frontMatter.meta.pypi_package}</code> will also install <code>dbt-core</code> and any other dependencies.</p>

<h2> Configuring {frontMatter.meta.pypi_package} </h2>

<p>For {frontMatter.meta.platform_name}-specifc configuration please refer to <a href={frontMatter.meta.config_page}>{frontMatter.meta.platform_name} Configuration</a> </p>

<p>For further info, refer to the GitHub repository: <a href={`https://github.com/${frontMatter.meta.github_repo}`}>{frontMatter.meta.github_repo}</a></p>
<SetUpPages meta={frontMatter.meta} />

## Connecting to Athena with dbt-athena

Expand Down
27 changes: 3 additions & 24 deletions website/docs/docs/core/connect-data-platform/azuresynapse-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,32 +24,11 @@ Refer to [Microsoft Fabric Synapse Data Warehouse](/docs/core/connect-data-platf

:::

<h2> Overview of {frontMatter.meta.pypi_package} </h2>
<!--The following code uses a component and the built-in docusaurus markdown partials file, which contains reusable content assigned in the meta frontmatter. For this page, the partial file is _setup-pages-intro.md. You have to include the 'import' code and then assign the component as needed. -->

<ul>
<li><strong>Maintained by</strong>: {frontMatter.meta.maintained_by}</li>
<li><strong>Authors</strong>: {frontMatter.meta.authors}</li>
<li><strong>GitHub repo</strong>: <a href={`https://github.com/${frontMatter.meta.github_repo}`}>{frontMatter.meta.github_repo}</a><a href={`https://github.com/${frontMatter.meta.github_repo}`}><img src={`https://img.shields.io/github/stars/${frontMatter.meta.github_repo}?style=for-the-badge`}/></a></li>
<li><strong>PyPI package</strong>: <code>{frontMatter.meta.pypi_package}</code> <a href={`https://badge.fury.io/py/${frontMatter.meta.pypi_package}`}><img src={`https://badge.fury.io/py/${frontMatter.meta.pypi_package}.svg`}/></a></li>
<li><strong>Slack channel</strong>: <a href={frontMatter.meta.slack_channel_link}>{frontMatter.meta.slack_channel_name}</a></li>
<li><strong>Supported dbt Core version</strong>: {frontMatter.meta.min_core_version} and newer</li>
<li><strong>dbt Cloud support</strong>: {frontMatter.meta.cloud_support}</li>
<li><strong>Minimum data platform version</strong>: {frontMatter.meta.min_supported_version}</li>
</ul>
import SetUpPages from '/snippets/_setup-pages-intro.md';

<h2> Installing {frontMatter.meta.pypi_package} </h2>

pip is the easiest way to install the adapter:

<code>pip install {frontMatter.meta.pypi_package}</code>

<p>Installing <code>{frontMatter.meta.pypi_package}</code> will also install <code>dbt-core</code> and any other dependencies.</p>

<h2> Configuring {frontMatter.meta.pypi_package} </h2>

<p>For {frontMatter.meta.platform_name}-specifc configuration please refer to <a href={frontMatter.meta.config_page}>{frontMatter.meta.platform_name} Configuration</a> </p>

<p>For further info, refer to the GitHub repository: <a href={`https://github.com/${frontMatter.meta.github_repo}`}>{frontMatter.meta.github_repo}</a></p>
<SetUpPages meta={frontMatter.meta} />

:::info Dedicated SQL only

Expand Down
28 changes: 2 additions & 26 deletions website/docs/docs/core/connect-data-platform/bigquery-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,33 +18,9 @@ meta:

<Snippet path="warehouse-setups-cloud-callout" />

<h2> Overview of {frontMatter.meta.pypi_package} </h2>

<ul>
<li><strong>Maintained by</strong>: {frontMatter.meta.maintained_by}</li>
<li><strong>Authors</strong>: {frontMatter.meta.authors}</li>
<li><strong>GitHub repo</strong>: <a href={`https://github.com/${frontMatter.meta.github_repo}`}>{frontMatter.meta.github_repo}</a> <a href={`https://github.com/${frontMatter.meta.github_repo}`}><img src={`https://img.shields.io/github/stars/${frontMatter.meta.github_repo}?style=for-the-badge`}/></a></li>
<li><strong>PyPI package</strong>: <code>{frontMatter.meta.pypi_package}</code> <a href={`https://badge.fury.io/py/${frontMatter.meta.pypi_package}`}><img src={`https://badge.fury.io/py/${frontMatter.meta.pypi_package}.svg`}/></a></li>
<li><strong>Slack channel</strong>: <a href={frontMatter.meta.slack_channel_link}>{frontMatter.meta.slack_channel_name}</a></li>
<li><strong>Supported dbt Core version</strong>: {frontMatter.meta.min_core_version} and newer</li>
<li><strong>dbt Cloud support</strong>: {frontMatter.meta.cloud_support}</li>
<li><strong>Minimum data platform version</strong>: {frontMatter.meta.min_supported_version}</li>
</ul>

<h2> Installing {frontMatter.meta.pypi_package} </h2>

pip is the easiest way to install the adapter:

<code>pip install {frontMatter.meta.pypi_package}</code>

<p>Installing <code>{frontMatter.meta.pypi_package}</code> will also install <code>dbt-core</code> and any other dependencies.</p>

<h2> Configuring {frontMatter.meta.pypi_package} </h2>

<p>For {frontMatter.meta.platform_name}-specifc configuration please refer to <a href={frontMatter.meta.config_page}>{frontMatter.meta.platform_name} Configuration</a> </p>

<p>For further info, refer to the GitHub repository: <a href={`https://github.com/${frontMatter.meta.github_repo}`}>{frontMatter.meta.github_repo}</a></p>
import SetUpPages from '/snippets/_setup-pages-intro.md';

<SetUpPages meta={frontMatter.meta} />

## Authentication Methods

Expand Down
29 changes: 2 additions & 27 deletions website/docs/docs/core/connect-data-platform/clickhouse-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,34 +17,9 @@ meta:

Some core functionality may be limited. If you're interested in contributing, check out the source code for each repository listed below.

import SetUpPages from '/snippets/_setup-pages-intro.md';

<h2> Overview of {frontMatter.meta.pypi_package} </h2>

<ul>
<li><strong>Maintained by</strong>: {frontMatter.meta.maintained_by}</li>
<li><strong>Authors</strong>: {frontMatter.meta.authors}</li>
<li><strong>GitHub repo</strong>: <a href={`https://github.com/${frontMatter.meta.github_repo}`}>{frontMatter.meta.github_repo}</a><a href={`https://github.com/${frontMatter.meta.github_repo}`}><img src={`https://img.shields.io/github/stars/${frontMatter.meta.github_repo}?style=for-the-badge`}/></a></li>
<li><strong>PyPI package</strong>: <code>{frontMatter.meta.pypi_package}</code> <a href={`https://badge.fury.io/py/${frontMatter.meta.pypi_package}`}><img src={`https://badge.fury.io/py/${frontMatter.meta.pypi_package}.svg`}/></a></li>
<li><strong>Slack channel</strong>: <a href={frontMatter.meta.slack_channel_link}>{frontMatter.meta.slack_channel_name}</a></li>
<li><strong>Supported dbt Core version</strong>: {frontMatter.meta.min_core_version} and newer</li>
<li><strong>dbt Cloud support</strong>: {frontMatter.meta.cloud_support}</li>
<li><strong>Minimum data platform version</strong>: {frontMatter.meta.min_supported_version}</li>
</ul>


<h2> Installing {frontMatter.meta.pypi_package} </h2>

pip is the easiest way to install the adapter:

<code>pip install {frontMatter.meta.pypi_package}</code>

<p>Installing <code>{frontMatter.meta.pypi_package}</code> will also install <code>dbt-core</code> and any other dependencies.</p>

<h2> Configuring {frontMatter.meta.pypi_package} </h2>

<p>For {frontMatter.meta.platform_name}-specifc configuration please refer to <a href={frontMatter.meta.config_page}>{frontMatter.meta.platform_name} Configuration</a> </p>

<p>For further info, refer to the GitHub repository: <a href={`https://github.com/${frontMatter.meta.github_repo}`}>{frontMatter.meta.github_repo}</a></p>
<SetUpPages meta={frontMatter.meta} />

## Connecting to ClickHouse with **dbt-clickhouse**

Expand Down
Loading

0 comments on commit 0720952

Please sign in to comment.