Skip to content

Commit

Permalink
cleaned README, removed profiles.yml
Browse files Browse the repository at this point in the history
  • Loading branch information
tnightengale committed Dec 1, 2020
1 parent 91f8149 commit 26a9919
Show file tree
Hide file tree
Showing 5 changed files with 17 additions and 42 deletions.
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,5 @@ target/
dbt_modules/
logs/
.env
.user.yml
.user.yml
profiles.yml
23 changes: 9 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,4 @@
# Quality dbt!
** NOTE: This package is in its alpha release.**

# dbt Meta Testing
This dbt package contains macros to assert test and documentation coverage from `dbt_project.yml` configuration settings.

## Install
Expand All @@ -9,14 +7,14 @@ Include in `packages.yml`:
```yaml
packages:
- git: "https://github.com/tnightengale/quality-assurance-dbt"
revision: <for latest release, see https://github.com/tnightengale/quality-assurance-dbt/releases>
revision: 0.1.2
```
For latest release, see https://github.com/tnightengale/quality-assurance-dbt/releases.
## Configurations
This package features two meta configs that can be applied to a dbt project: `+required_tests` and `+required_docs`. Read the dbt documentation [here](https://docs.getdbt.com/reference/model-configs) to learn more about model configurations in dbt.

### **Require Tests**
### **Required Tests**
To require test coverage, define the `+required_tests` configuration on a model path in `dbt_project.yml`:
```yaml
# dbt_project.yml
Expand All @@ -29,7 +27,7 @@ models:
+required_tests: {"unique": 1}
```

The `+required_tests` config must be either a `dict` or `None`. All the regular configuration override rules are the same. For example, individual model configs will override configs from the `dbt_project.yml`:
The `+required_tests` config must be either a `dict` or `None`. All the regular dbt configuration hierarchy rules apply. For example, individual model configs will override configs from the `dbt_project.yml`:
```sql
-- /models/marts/core/your_model.sql
{{
Expand Down Expand Up @@ -76,7 +74,7 @@ Encountered an error while running operation: Compilation Error in macro require
usr@home dbt-meta-testing $ %
```

### **Require Docs**
### **Required Docs**
To require documentation coverage, define the `+required_docs` configuration on a model path in `dbt_project.yml`:
```yaml
# dbt_project.yml
Expand All @@ -88,7 +86,7 @@ models:
The `+required_docs` config must be a `bool`.

When applied to a model, this config will ensure 3 things:
1. The _model_ has a non-empty descriptions:
1. The _model_ has a non-empty description
2. The _columns_ in the model are specified in the model `.yml`
3. The _columns_ specified in the model `.yml` have non-empty descriptions

Expand Down Expand Up @@ -173,25 +171,22 @@ For example, to run only changed models using dbt's Slim CI feature:
$ dbt run-operation required_tests --vars "{'models':'$(dbt list -m state:modified --state <filepath>)'}"
```

Alternatively, if you use git-diff to find changed models, a space delimited string of model names will work as well:
Alternatively, it's possible to use `git diff` to find changed models; a space delimited string of model names will work as well:
```bash
$ dbt run-operation required_tests --vars "{'models':'model1 model2 model3'}"
```

### required_tests ([source](macros/required_tests.sql))
Validates that models meet the `+required_tests` configurations applied in `dbt_project.yml`.

Can be used as a `run-operation` early in the CI/CD pipeline.
Validates that models meet the `+required_tests` configurations applied in `dbt_project.yml`. Typically used only as a `run-operation` in a CI pipeline.

Usage:
```
$ dbt run-operation required_tests [--vars "{'models': '<space_delimited_models>'}"]
```

### required_docs ([source](macros/required_tests.sql))
Validates that models meet the `+required_docs` configurations applied in `dbt_project.yml`.
Validates that models meet the `+required_docs` configurations applied in `dbt_project.yml`. Typically used only as a `run-operation` in a CI pipeline.

Can be used as a `run-operation` early in the CI/CD pipeline.

Usage:
```
Expand Down
21 changes: 6 additions & 15 deletions dbt_project.yml
Original file line number Diff line number Diff line change
@@ -1,18 +1,13 @@

# Name your project! Project names should contain only lowercase characters
# and underscores. A good package name should reflect your organization's
# name or the intended use of these models
# Project name.
name: 'dbt_meta_testing'
version: '1.0.0'
version: '0.1.2'
config-version: 2

# This setting configures which "profile" dbt uses for this project.
# The "profile" dbt uses for this project.
profile: 'dbt_meta_testing'

# These configurations specify where dbt should look for different types of files.
# The `source-paths` config, for example, states that models in this project can be
# found in the "models/" directory. You probably won't need to change these!

# Configuration paths.
source-paths: ["models"]
# source-paths: ["test_models"] # TO DO: Configure via scripts for integration testing
analysis-paths: ["analysis"]
Expand All @@ -21,15 +16,11 @@ data-paths: ["data"]
macro-paths: ["macros"]
snapshot-paths: ["snapshots"]

target-path: "target" # directory which will store compiled SQL files
clean-targets: # directories to be removed by `dbt clean`
target-path: "target"
clean-targets:
- "target"
- "dbt_modules"


# Configuring models
# Full documentation: https://docs.getdbt.com/docs/configuring-models

# Configured for the logger macro.
vars:
logging_level: INFO
Expand Down
12 changes: 0 additions & 12 deletions profiles.yml

This file was deleted.

Empty file removed snapshots/.gitkeep
Empty file.

0 comments on commit 26a9919

Please sign in to comment.