Skip to content

Commit

Permalink
Merge branch 'current' into current
Browse files Browse the repository at this point in the history
  • Loading branch information
mirnawong1 authored Jan 8, 2024
2 parents 5846110 + 515840b commit 81979c4
Show file tree
Hide file tree
Showing 1,726 changed files with 13,778 additions and 10,797 deletions.
20 changes: 9 additions & 11 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,23 @@
## What are you changing in this pull request and why?
<!---
Describe your changes and why you're making them. If linked to an open
Describe your changes and why you're making them. If related to an open
issue or a pull request on dbt Core, then link to them here!
To learn more about the writing conventions used in the dbt Labs docs, see the [Content style guide](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/content-style-guide.md).
-->

## Checklist
<!--
Uncomment if you're publishing docs for a prerelease version of dbt (delete if not applicable):
Uncomment when publishing docs for a prerelease version of dbt:
- [ ] Add versioning components, as described in [Versioning Docs](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#versioning-entire-pages)
- [ ] Add a note to the prerelease version [Migration Guide](https://github.com/dbt-labs/docs.getdbt.com/tree/current/website/docs/docs/dbt-versions/core-upgrade)
-->
- [ ] Review the [Content style guide](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/content-style-guide.md) and [About versioning](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#adding-a-new-version) so my content adheres to these guidelines.
- [ ] Review the [Content style guide](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/content-style-guide.md) so my content adheres to these guidelines.
- [ ] For [docs versioning](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#about-versioning), review how to [version a whole page](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#adding-a-new-version) and [version a block of content](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#versioning-blocks-of-content).
- [ ] Add a checklist item for anything that needs to happen before this PR is merged, such as "needs technical review" or "change base branch."

Adding new pages (delete if not applicable):
- [ ] Add page to `website/sidebars.js`
- [ ] Provide a unique filename for the new page

Removing or renaming existing pages (delete if not applicable):
- [ ] Remove page from `website/sidebars.js`
- [ ] Add an entry `website/static/_redirects`
- [ ] [Ran link testing](https://github.com/dbt-labs/docs.getdbt.com#running-the-cypress-tests-locally) to update the links that point to the deleted page
Adding or removing pages (delete if not applicable):
- [ ] Add/remove page in `website/sidebars.js`
- [ ] Provide a unique filename for new pages
- [ ] Add an entry for deleted pages in `website/static/_redirects`
- [ ] Run link testing locally with `npm run build` to update the links that point to deleted pages
17 changes: 17 additions & 0 deletions .github/workflows/asana-connection.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
name: Show PR Status in Asana
on:
pull_request:
types: [opened, reopened]

jobs:
create-asana-attachment-job:
runs-on: ubuntu-latest
name: Create pull request attachments on Asana tasks
steps:
- name: Create pull request attachments
uses: Asana/create-app-attachment-github-action@latest
id: postAttachment
with:
asana-secret: ${{ secrets.ASANA_SECRET }}
- name: Log output status
run: echo "Status is ${{ steps.postAttachment.outputs.status }}"
8 changes: 8 additions & 0 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,16 @@ jobs:
uses: actions/setup-node@v3
with:
node-version: '18.12.0'

- name: Cache Node Modules
uses: actions/cache@v3
id: cache-node-mods
with:
path: website/node_modules
key: node-modules-cache-v3-${{ hashFiles('**/package.json', '**/package-lock.json') }}

- name: Install Packages
if: steps.cache-node-mods.outputs.cache-hit != 'true'
run: cd website && npm ci

- name: Run ESLint
Expand Down
2 changes: 1 addition & 1 deletion contributing/adding-page-components.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
## Using warehouse components

You can use the following components to provide code snippets for each supported warehouse. You can see a real-life example in the docs page [Initialize your project](/quickstarts/databricks?step=6).
You can use the following components to provide code snippets for each supported warehouse. You can see a real-life example in the docs page [Initialize your project](/guides/databricks?step=6).

Identify code by labeling with the warehouse names:

Expand Down
7 changes: 5 additions & 2 deletions contributing/content-style-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -284,7 +284,7 @@ If the list starts getting lengthy and dense, consider presenting the same conte

A bulleted list with introductory text:

> A dbt project is a directory of `.sql` and .yml` files. The directory must contain at a minimum:
> A dbt project is a directory of `.sql` and `.yml` files. The directory must contain at a minimum:
>
> - Models: A model is a single `.sql` file. Each model contains a single `select` statement that either transforms raw data into a dataset that is ready for analytics or, more often, is an intermediate step in such a transformation.
> - A project file: A `dbt_project.yml` file, which configures and defines your dbt project.
Expand Down Expand Up @@ -360,7 +360,7 @@ Otherwise, the text will appear squished and provide users with a bad experience
- `<divclassName="grid--5-col">`: creates 5 columns cards (use sparingly)
- You can't create cards with 6 or more columns as that would provide users a poor experience.

Refer to [dbt Cloud features](/docs/cloud/about-cloud/dbt-cloud-features) and [Quickstarts](/docs/quickstarts/overview) as examples.
Refer to [dbt Cloud features](/docs/cloud/about-cloud/dbt-cloud-features) and [Quickstarts](/docs/guides) as examples.

### Create cards

Expand Down Expand Up @@ -479,6 +479,9 @@ Some common Latin abbreviations and other words to use instead:
| i.e. | that is | Use incremental models when your dbt runs are becoming too slow (that is, don't start with incremental models) |
| e.g. | <ul><li>for example</li><li>like</li></ul> | <ul><li>Join both the dedicated #adapter-ecosystem channel in dbt Slack and the channel for your adapter's data store (for example, #db-sqlserver and #db-athena)</li><li>Using Jinja in SQL provides a way to use control structures (like `if` statements and `for` loops) in your queries </li></ul> |
| etc. | <ul><li>and more</li><li>and so forth</li></ul> | <ul><li>A continuous integration environment running pull requests in GitHub, GitLab, and more</li><li>While reasonable defaults are provided for many such operations (like `create_schema`, `drop_schema`, `create_table`, and so forth), you might need to override one or more macros when building a new adapter</li></ul> |
| N.B. | note | Note: State-based selection is a powerful, complex feature. |

https://www.thoughtco.com/n-b-latin-abbreviations-in-english-3972787

### Prepositions

Expand Down
6 changes: 0 additions & 6 deletions package-lock.json

This file was deleted.

6 changes: 3 additions & 3 deletions website/blog/2021-02-05-dbt-project-checklist.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,7 @@ This post is the checklist I created to guide our internal work, and I’m shari
* [Sources](/docs/build/sources/)
* [Refs](/reference/dbt-jinja-functions/ref/)
* [tags](/reference/resource-configs/tags/)
* [Jinja docs](/guides/advanced/using-jinja)
* [Jinja docs](/guides/using-jinja)

## ✅ Testing & Continuous Integration
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Expand All @@ -156,7 +156,7 @@ This post is the checklist I created to guide our internal work, and I’m shari

**Useful links**

* [Version control](/guides/legacy/best-practices#version-control-your-dbt-project)
* [Version control](/best-practices/best-practice-workflows#version-control-your-dbt-project)
* [dbt Labs' PR Template](/blog/analytics-pull-request-template)

## ✅ Documentation
Expand Down Expand Up @@ -252,7 +252,7 @@ Thanks to Christine Berger for her DAG diagrams!

**Useful links**

* [How we structure our dbt Project](/guides/best-practices/how-we-structure/1-guide-overview)
* [How we structure our dbt Project](/best-practices/how-we-structure/1-guide-overview)
* [Coalesce DAG Audit Talk](https://www.youtube.com/watch?v=5W6VrnHVkCA&t=2s)
* [Modular Data Modeling Technique](https://getdbt.com/analytics-engineering/modular-data-modeling-technique/)
* [Understanding Threads](/docs/running-a-dbt-project/using-threads)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -159,4 +159,4 @@ All of the above configurations “work”. And as detailed, they each solve for
2. Figure out what may be a pain point in the future and try to plan for it from the beginning.
3. Don’t over-complicate things until you have the right reason. As I said in my Coalesce talk: **don’t drag your skeletons from one closet to another** 💀!

**Note:** Our attempt in writing guides like this and [How we structure our dbt projects](/guides/best-practices/how-we-structure/1-guide-overview) aren’t to try to convince you that our way is right; it is to hopefully save you the hundreds of hours it has taken us to form those opinions!
**Note:** Our attempt in writing guides like this and [How we structure our dbt projects](/best-practices/how-we-structure/1-guide-overview) aren’t to try to convince you that our way is right; it is to hopefully save you the hundreds of hours it has taken us to form those opinions!
2 changes: 1 addition & 1 deletion website/blog/2021-11-22-dbt-labs-pr-template.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ Checking for things like modularity and 1:1 relationships between sources and st

#### Validation of models:

This section should show something to confirm that your model is doing what you intended it to do. This could be a [dbt test](/docs/build/tests) like uniqueness or not null, or could be an ad-hoc query that you wrote to validate your data. Here is a screenshot from a test run on a local development branch:
This section should show something to confirm that your model is doing what you intended it to do. This could be a [dbt test](/docs/build/data-tests) like uniqueness or not null, or could be an ad-hoc query that you wrote to validate your data. Here is a screenshot from a test run on a local development branch:

![test validation](/img/blog/pr-template-test-validation.png "dbt test validation")

Expand Down
2 changes: 1 addition & 1 deletion website/blog/2021-11-22-primary-keys.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ In the days before testing your data was commonplace, you often found out that y

## How to test primary keys with dbt

Today, you can add two simple [dbt tests](/docs/build/tests) onto your primary keys and feel secure that you are going to catch the vast majority of problems in your data.
Today, you can add two simple [dbt tests](/docs/build/data-tests) onto your primary keys and feel secure that you are going to catch the vast majority of problems in your data.

Not surprisingly, these two tests correspond to the two most common errors found on your primary keys, and are usually the first tests that teams testing data with dbt implement:

Expand Down
2 changes: 1 addition & 1 deletion website/blog/2021-11-23-how-to-upgrade-dbt-versions.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,7 @@ Once your compilation issues are resolved, it's time to run your job for real, t

After that, make sure that your CI environment in dbt Cloud or your orchestrator is on the right dbt version, then open a PR.

If you're using [Slim CI](https://docs.getdbt.com/docs/guides/best-practices#run-only-modified-models-to-test-changes-slim-ci), keep in mind that artifacts aren't necessarily compatible from one version to another, so you won't be able to use it until the job you defer to has completed a run with the upgraded dbt version. This doesn't impact our example because support for Slim CI didn't come out until 0.18.0.
If you're using [Slim CI](https://docs.getdbt.com/docs/best-practices#run-only-modified-models-to-test-changes-slim-ci), keep in mind that artifacts aren't necessarily compatible from one version to another, so you won't be able to use it until the job you defer to has completed a run with the upgraded dbt version. This doesn't impact our example because support for Slim CI didn't come out until 0.18.0.

## Step 7. Merge and communicate

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ So let’s all commit to sharing our hard won knowledge with each other—and in

The purpose of this blog is to double down on our long running commitment to contributing to the knowledge loop.

From early posts like ‘[The Startup Founders Guide to Analytics’](https://thinkgrowth.org/the-startup-founders-guide-to-analytics-1d2176f20ac1) to foundational guides like [‘How We Structure Our dbt Projects](/guides/best-practices/how-we-structure/1-guide-overview)’, we’ve had a long standing goal of working with the community to create practical, hands-on tutorials and guides which distill the knowledge we’ve been able to collectively gather.
From early posts like ‘[The Startup Founders Guide to Analytics’](https://thinkgrowth.org/the-startup-founders-guide-to-analytics-1d2176f20ac1) to foundational guides like [‘How We Structure Our dbt Projects](/best-practices/how-we-structure/1-guide-overview)’, we’ve had a long standing goal of working with the community to create practical, hands-on tutorials and guides which distill the knowledge we’ve been able to collectively gather.

dbt as a product is based around the philosophy that even the most complicated problems can be broken down into modular, reusable components, then mixed and matched to create something novel.

Expand Down
6 changes: 3 additions & 3 deletions website/blog/2021-11-29-dbt-airflow-spiritual-alignment.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,8 +90,8 @@ So instead of getting bogged down in defining roles, let’s focus on hard skill
The common skills needed for implementing any flavor of dbt (Core or Cloud) are:

* SQL: ‘nuff said
* YAML: required to generate config files for [writing tests on data models](/docs/build/tests)
* [Jinja](/guides/advanced/using-jinja): allows you to write DRY code (using [macros](/docs/build/jinja-macros), for loops, if statements, etc)
* YAML: required to generate config files for [writing tests on data models](/docs/build/data-tests)
* [Jinja](/guides/using-jinja): allows you to write DRY code (using [macros](/docs/build/jinja-macros), for loops, if statements, etc)

YAML + Jinja can be learned pretty quickly, but SQL is the non-negotiable you’ll need to get started.

Expand Down Expand Up @@ -176,7 +176,7 @@ Instead you can now use the following command:
`dbt build –select result:error+ –defer –state <previous_state_artifacts>` … and that’s it!


You can see more examples [here](https://docs.getdbt.com/docs/guides/best-practices#run-only-modified-models-to-test-changes-slim-ci).
You can see more examples [here](https://docs.getdbt.com/docs/best-practices#run-only-modified-models-to-test-changes-slim-ci).


This means that whether you’re actively developing or you simply want to rerun a scheduled job (because of, say, permission errors or timeouts in your database), you now have a unified approach to doing both.
Expand Down
2 changes: 1 addition & 1 deletion website/blog/2021-11-29-open-source-community-growth.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ For starters, I want to know how much conversation is occurring across the vario

There are a ton of metrics that can be tracked in any GitHub project — committers, pull requests, forks, releases — but I started pretty simple. For each of the projects we participate in, I just want to know how the number of GitHub stars grows over time, and whether the growth is accelerating or flattening out. This has become a key performance indicator for open source communities, for better or for worse, and keeping track of it isn't optional.

Finally, I want to know how much Marquez and OpenLineage are being used. It used to be that when you wanted to consume a bit of tech, you'd download a file. Folks like me who study user behavior would track download counts as if they were stock prices. This is no longer the case; today, our tech is increasingly distributed through package managers and image repositories. Docker Hub and PyPI metrics have therefore become good indicators of consumption. Docker image pulls and runs of `pip install` are the modern day download and, as noisy as these metrics are, they indicate a similar level of user commitment.
Finally, I want to know how much Marquez and OpenLineage are being used. It used to be that when you wanted to consume a bit of tech, you'd download a file. Folks like me who study user behavior would track download counts as if they were stock prices. This is no longer the case; today, our tech is increasingly distributed through package managers and image repositories. Docker Hub and PyPI metrics have therefore become good indicators of consumption. Docker image pulls and runs of `python -m pip install` are the modern day download and, as noisy as these metrics are, they indicate a similar level of user commitment.

To summarize, here are the metrics I decided to track (for now, anyway):
- Slack messages (by user/ by community)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ In addition to learning the basic pieces of dbt, we're familiarizing ourselves w

If we decide not to do this, we end up missing out on what the dbt workflow has to offer. If you want to learn more about why we think analytics engineering with dbt is the way to go, I encourage you to read the [dbt Viewpoint](/community/resources/viewpoint#analytics-is-collaborative)!

In order to learn the basics, we’re going to [port over the SQL file](/guides/migration/tools/refactoring-legacy-sql) that powers our existing "patient_claim_summary" report that we use in our KPI dashboard in parallel to our old transformation process. We’re not ripping out the old plumbing just yet. In doing so, we're going to try dbt on for size and get used to interfacing with a dbt project.
In order to learn the basics, we’re going to [port over the SQL file](/guides/refactoring-legacy-sql) that powers our existing "patient_claim_summary" report that we use in our KPI dashboard in parallel to our old transformation process. We’re not ripping out the old plumbing just yet. In doing so, we're going to try dbt on for size and get used to interfacing with a dbt project.

**Project Appearance**

Expand All @@ -87,7 +87,7 @@ The most important thing we’re introducing when your project is an infant is t

* Introduce modularity with [{{ ref() }}](/reference/dbt-jinja-functions/ref) and [{{ source() }}](/reference/dbt-jinja-functions/source)

* [Document](/docs/collaborate/documentation) and [test](/docs/build/tests) your first models
* [Document](/docs/collaborate/documentation) and [test](/docs/build/data-tests) your first models

![image alt text](/img/blog/building-a-mature-dbt-project-from-scratch/image_3.png)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ In short, a jaffle is:

*See above: Tasty, tasty jaffles.*

Jaffle Shop is a demo repo referenced in [dbt’s Getting Started Guide](/quickstarts), and its jaffles hold a special place in the dbt community’s hearts, as well as on Data Twitter™.
Jaffle Shop is a demo repo referenced in [dbt’s Getting Started Guide](/guides), and its jaffles hold a special place in the dbt community’s hearts, as well as on Data Twitter™.

![jaffles on data twitter](/img/blog/2022-02-08-customer-360-view/image_1.png)

Expand Down
4 changes: 2 additions & 2 deletions website/blog/2022-04-14-add-ci-cd-to-bitbucket.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@ pipelines:
artifacts: # Save the dbt run artifacts for the next step (upload)
- target/*.json
script:
- pip install -r requirements.txt
- python -m pip install -r requirements.txt
- mkdir ~/.dbt
- cp .ci/profiles.yml ~/.dbt/profiles.yml
- dbt deps
Expand Down Expand Up @@ -208,7 +208,7 @@ pipelines:
# Set up dbt environment + dbt packages. Rather than passing
# profiles.yml to dbt commands explicitly, we'll store it where dbt
# expects it:
- pip install -r requirements.txt
- python -m pip install -r requirements.txt
- mkdir ~/.dbt
- cp .ci/profiles.yml ~/.dbt/profiles.yml
- dbt deps
Expand Down
2 changes: 1 addition & 1 deletion website/blog/2022-04-19-complex-deduplication.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,7 @@ select * from filter_real_diffs

> *What happens in this step? You check your data because you are thorough!*
Good thing dbt has already built this for you. Add a [unique test](/docs/build/tests#generic-tests) to your YAML model block for your `grain_id` in this de-duped staging model, and give it a dbt test!
Good thing dbt has already built this for you. Add a [unique test](/docs/build/data-tests#generic-data-tests) to your YAML model block for your `grain_id` in this de-duped staging model, and give it a dbt test!

```yaml
models:
Expand Down
Loading

0 comments on commit 81979c4

Please sign in to comment.