Skip to content

Commit

Permalink
Create contributors guide (#39)
Browse files Browse the repository at this point in the history
While there, fix some inconsistencies in the documentation website, and
update the readme.
  • Loading branch information
matthieucan authored Jun 7, 2024
1 parent db5f048 commit 341eefa
Show file tree
Hide file tree
Showing 8 changed files with 175 additions and 94 deletions.
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ and this project adheres to

## [Unreleased]

- Create contributors guide for the documentation website.

## [0.1.1] - 2023-06-03

### Added
Expand Down
104 changes: 23 additions & 81 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,92 +7,34 @@
[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/dbt-score.svg)](https://pypi.org/project/dbt-score)
[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](https://makeapullrequest.com)

Linter for dbt model metadata.
## What is `dbt-score`?

## Development
`dbt-score` is a linter for dbt model metadata.

### Prerequisites
[dbt](https://getdbt.com/) (Data Build Tool) is a great framework for creating,
building, organizing, testing and documenting _data models_, i.e. data sets
living in a database or a data warehouse. Through a declarative approach, it
allows data practitioners to build data with a methodology inspired by software
development practices.

You'll need the following prerequisites:
This leads to data models being bundled with a lot of metadata, such as
documentation, data tests, access control information, column types and
constraints, 3rd party integrations... Not to mention any other metadata that
organizations need, fully supported through the `meta` parameter.

- Any Python version starting from 3.11
- [pre-commit](https://pre-commit.com/)
- [PDM](https://pdm-project.org/2.12/)
At scale, with hundreds or thousands of data models, all this metadata can
become confusing, disparate, and inconsistent. It's hard to enforce good
practices and maintain them in continuous integration systems. This is where
`dbt-score` plays its role: by allowing data teams to programatically define and
enforce metadata rules, in an easy and scalable manner.

Configure development environment running these commands from the project's
root:
## Documentation

```shell
pre-commit install
pdm install --group :all
```
Everything you need (and more) can be found in
[`dbt-score` documentation website](https://dbt-score.picnic.tech/).

The pdm command will install all project's dependency groups, including all the
dependencies needed for development purposes.
## Contributing

### Lint

`dbt_score` uses:

- [ruff](https://docs.astral.sh/ruff/) for fast linting and formatting.
- [mypy](https://mypy.readthedocs.io/en/stable/) for type checking.
- [pre-commit-hooks](https://github.com/pre-commit/pre-commit-hooks).
- [prettier-hooks](https://github.com/pre-commit/mirrors-prettier).

Cheatsheet:

```shell
pdm run ruff check .
pdm run ruff check --fix
pdm run mypy .
pdm run tox -e lint
```

### Test

`dbt_score` uses:

- [pytest](https://docs.pytest.org/) as a main test framework.
- [coverage](https://coverage.readthedocs.io/en/latest/index.html) for test
coverage.
- [tox](https://tox.wiki/en/latest/) for testing against multiple Python
versions.

Cheatsheet:

```shell
pdm run tox -e py
pdm run pytest
pdm run coverage run -m pytest
```

### Docs

`dbt_score` uses:

- [mkdocs](https://www.mkdocs.org/) for docs generation.
- [mkdocstrings](https://mkdocstrings.github.io/) for automatic docs from
sources.

Cheatsheet:

```shell
pdm run mkdocs build
pdm run mkdocs serve
```

### Pre-commit

Cheatsheet:

Execute hooks manually:

```shell
pre-commit run --all-files
```

Create a commit bypassing hooks:

```shell
git commit --no-verify
```
Would you like to contribute to `dbt-score`? That's great news! Please follow
[the guide on the documentation website](https://dbt-score.picnic.tech/contributors_guide).
🚀
141 changes: 141 additions & 0 deletions docs/CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
# Contributor's guide

`dbt-score` is free software, your contributions are welcome! 🚀

## Reporting bugs

If you encountered a bug, check the
[issue tracker on Github](https://github.com/PicnicSupermarket/dbt-score/issues)
to see if it is already known. If not, feel free to
[open a new issue](https://github.com/PicnicSupermarket/dbt-score/issues/new).
Share all relevant details, especially how to reproduce the problem.

We'd love to hear from you and help make `dbt-score` as stable as we can.

## Developing new features

### Adding rules

The linting rules bundled with `dbt-score` aim to be as generic as possible, and
apply to a large majority of dbt projects.

This is why they are not very opinionated: for example, we believe documenting
data models is important, and hope it's not a controversial opinion ;) Of
course, it's always possible to disable any rule.

If you think a new rule should be created in `dbt-score`, feel free to
[open an issue](https://github.com/PicnicSupermarket/dbt-score/issues/new) to
discuss it first - this might save you some work in case maintainers don't see a
fit.

If your rule idea is not generic and applies to your own project and logic,
`dbt-score` has been designed to fully [support custom rules](create_rules.md).
Create as many as you need for your purposes!

### Fixing bugs

We love bug squashing! You can open a pull request to fix any bug you encounter
in `dbt-score`. If the changes are large enough, refer to the next section
first - discussing a solution in a Github issue is always a good idea to avoid
unnecessary work and orchestrate efforts.

### Adding or changing core features

Before implementing or changing a new feature, we kindly ask you to
[open a Github issue](https://github.com/PicnicSupermarket/dbt-score/issues/new/)
to get the maintainers' opinion on that feature. It might have been already
considered, discussed, or already in the works.

We aim to maintain a high code coverage in `dbt-score`'s unit tests, so new
features should be properly tested for happy and unhappy paths.

If the feature has direct impact on users, it should also be reflected in the
documentation website.

## Development environment

### Prerequisites

You'll need the following:

- Any Python version starting from 3.11
- [pre-commit](https://pre-commit.com/) (recommended)
- [PDM](https://pdm-project.org/2.12/)

After cloning the repository with git, configure your development environment by
running these commands from the project's root:

```shell
pre-commit install
pdm install --group :all
```

The pdm command will install all project's dependency groups, including all the
dependencies needed for development purposes.

### Lint

`dbt-score` uses:

- [ruff](https://docs.astral.sh/ruff/) for fast linting and formatting.
- [mypy](https://mypy.readthedocs.io/en/stable/) for type checking.
- [pre-commit-hooks](https://github.com/pre-commit/pre-commit-hooks).
- [prettier-hooks](https://github.com/pre-commit/mirrors-prettier).

Cheatsheet:

```shell
pdm run ruff check .
pdm run ruff check --fix
pdm run mypy .
pdm run tox -e lint
```

### Test

`dbt-score` uses:

- [pytest](https://docs.pytest.org/) as a main test framework.
- [coverage](https://coverage.readthedocs.io/en/latest/index.html) for test
coverage.
- [tox](https://tox.wiki/en/latest/) for testing against multiple Python
versions.

Cheatsheet:

```shell
pdm run tox -e py
pdm run pytest
pdm run coverage run -m pytest
```

### Docs

`dbt-score` uses:

- [mkdocs](https://www.mkdocs.org/) for docs generation.
- [mkdocstrings](https://mkdocstrings.github.io/) for automatic docs from
sources.

Cheatsheet:

```shell
pdm run mkdocs build
pdm run mkdocs serve
```

### Pre-commit

Cheatsheet:

Execute hooks manually:

```shell
pre-commit run --all-files
```

Create a commit bypassing hooks:

```shell
git commit --no-verify
```
6 changes: 3 additions & 3 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ All badges except `wip` can be configured with the following option:
score required for a model to be rewarded with a certain badge.

The default values can be found in the
[BadgeConfig](/reference/config/#dbt_score.config.BadgeConfig).
[BadgeConfig](reference/config.md#dbt_score.config.BadgeConfig).

#### Rule configuration

Expand All @@ -81,10 +81,10 @@ Every rule can be configured with the following option:
of 4.

Some rules have additional configuration options, e.g.
[sql_has_reasonable_number_of_lines](/rules/generic/#sql_has_reasonable_number_of_lines).
[sql_has_reasonable_number_of_lines](rules/generic.md#sql_has_reasonable_number_of_lines).
Depending on the rule, the options will have different names, types and default
values. In the case of the
[sql_has_reasonable_number_of_lines](/rules/generic/#sql_has_reasonable_number_of_lines),
[sql_has_reasonable_number_of_lines](rules/generic.md#sql_has_reasonable_number_of_lines),
the `max_lines` option can be configured.

## Command line
Expand Down
8 changes: 3 additions & 5 deletions docs/create_rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,9 @@

In order to lint and score models, `dbt-score` uses a set of rules that are
applied to each model. A rule can pass or fail when it is run. Based on the
severity of the rule models are scored. `dbt-score` has a set of rules enabled
by default, which can be found [here](reference/rules/generic.md).
severity of the rule, models are scored with the weighted average of the rules
results. Note that `dbt-score` comes bundled with a
[set of default rules](rules/generic.md).

On top of the generic rules, it's possible to add your own rules. Two ways exist
to create a new rule:
Expand All @@ -19,7 +20,6 @@ The `@rule` decorator can be used to easily create a new rule:
```python
from dbt_score import Model, rule, RuleViolation


@rule
def has_description(model: Model) -> RuleViolation | None:
"""A model should have a description."""
Expand All @@ -36,7 +36,6 @@ The severity of a rule can be set using the `severity` argument:
```python
from dbt_score import rule, Severity


@rule(severity=Severity.HIGH)
```

Expand All @@ -48,7 +47,6 @@ class:
```python
from dbt_score import Model, Rule, RuleViolation


class HasDescription(Rule):
description = "A model should have a description."

Expand Down
4 changes: 2 additions & 2 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ encourage) good practices.

## Example

```shell
$ dbt-score lint
```
> dbt-score lint
🥇 customers (score: 10.0)
OK dbt_score.rules.generic.has_description
OK dbt_score.rules.generic.has_owner: Model lacks an owner.
Expand Down
3 changes: 0 additions & 3 deletions docs/reference/rules/generic.md

This file was deleted.

1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,4 +38,5 @@ nav:
- reference/formatters/index.md
- reference/formatters/human_readable_formatter.md
- reference/formatters/manifest_formatter.md
- Contributor's guide: CONTRIBUTING.md
- Changelog: https://github.com/PicnicSupermarket/dbt-score/blob/master/CHANGELOG.md

0 comments on commit 341eefa

Please sign in to comment.