Skip to content

Commit

Permalink
Merge remote-tracking branch 'origin/main' into refactor/pydantic-v2
Browse files Browse the repository at this point in the history
  • Loading branch information
sujuka99 committed Oct 9, 2023
2 parents 320ad31 + 74dd373 commit 22300ad
Show file tree
Hide file tree
Showing 119 changed files with 2,357 additions and 1,660 deletions.
50 changes: 30 additions & 20 deletions .github/actions/update-docs/action.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
name: "Update documentation in gh-pages"
description: |
Compile markdown documents to html and deploy to docs branch. If a semver tag is given, this action strips the patch.
Compile markdown documents to html and deploy to docs branch. If a semver version is given, this action strips the patch.
It then pushes the <major>.<minor> as the latest alias to the documentation branch with mike.
In case no tag is given, it pushes to 'dev'.
inputs:
username:
Expand All @@ -14,8 +13,12 @@ inputs:
token:
description: "GitHub Token (must be a PAT for repository dispatch)"
required: true
tag:
description: "Version tag"
version:
description: "Version name to be deployed by mike"
required: true
release:
description: "Determines if the set version is a stable and latest version, otherwise it is a dev version. (Default false)"
default: 'false'
required: false

runs:
Expand All @@ -27,23 +30,30 @@ runs:
poetry-version: "1.5.1"
python-version: "3.10"

- name: Update documentation branch with mike
- name: Install docs dependencies
shell: bash
env:
USERNAME: ${{ inputs.username }}
EMAIL: ${{ inputs.email }}
TOKEN: ${{ inputs.token }}
NEW_TAG: ${{ inputs.tag }}
run: |
poetry install --with docs
git config --local user.name ${USERNAME}
git config --local user.email ${EMAIL}
git config --local user.password ${TOKEN}
git pull # make sure docs branch is up-to-date
if [ -z "$NEW_TAG" ]; then
poetry run mike deploy dev --push --rebase --config-file ./docs/mkdocs.yml
else
new_tag=${NEW_TAG%.*}
poetry run mike deploy "$new_tag" latest --update-aliases --push --rebase --config-file ./docs/mkdocs.yml
fi
- name: Update ${{ github.head_ref }} branch
shell: bash
run: |
git config --local user.name ${{ inputs.username }}
git config --local user.email ${{ inputs.email }}
git config --local user.password ${{ inputs.token }}
git pull
- name: Deploy ${{ inputs.version }} version of the documentation with mike
shell: bash
if: ${{ inputs.release == 'false' }}
run: |
poetry run mike deploy ${{ inputs.version }} --push --rebase --config-file ./docs/mkdocs.yml
- name: Deploy ${{ inputs.version }} version (latest) of the documentation with mike
shell: bash
if: ${{ inputs.release == 'true' }}
run: |
sem_version=${{ inputs.version }}
major_minor_version=${sem_version%.*}
poetry run mike deploy "$major_minor_version" latest --update-aliases --push --rebase --config-file ./docs/mkdocs.yml
17 changes: 17 additions & 0 deletions .github/ruff-matcher.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"problemMatcher": [
{
"owner": "ruff",
"pattern": [
{
"regexp": "^(.*):(\\d+):(\\d+):\\s([\\da-zA-Z]+)\\s(.*)$",
"file": 1,
"line": 2,
"column": 3,
"code": 4,
"message": 5
}
]
}
]
}
Empty file added .github/scripts/__init__.py
Empty file.
67 changes: 57 additions & 10 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -35,11 +35,16 @@ jobs:
- name: Install dependencies
run: poetry install --no-interaction

- name: Lint (flake8)
run: poetry run pre-commit run flake8 --all-files --show-diff-on-failure

- name: Order of imports (isort)
run: poetry run pre-commit run isort --all-files --show-diff-on-failure
- name: Lint (ruff)
shell: bash
run: |
if [[ "$RUNNER_OS" == "Linux" && "${{ matrix.python-version }}" == "3.10" ]]
then
echo "::add-matcher::.github/ruff-matcher.json"
poetry run ruff check . --config pyproject.toml --output-format text --no-fix
else
poetry run pre-commit run ruff --all-files --show-diff-on-failure
fi;
- name: Formatting (black)
run: poetry run pre-commit run black --all-files --show-diff-on-failure
Expand All @@ -59,16 +64,15 @@ jobs:
- name: Generate pipeline definitions
run: poetry run pre-commit run gen-docs-components --all-files --show-diff-on-failure

# TODO: enable when PEP 604 incompatibilty is in typer is resolved https://github.com/tiangolo/typer/issues/348
# See https://github.com/tiangolo/typer/pull/522
# - name: Syntax (pyupgrade)
# run: poetry run pre-commit run --hook-stage manual pyupgrade --all-files

- name: Test
run: poetry run pytest tests

- name: Install docs dependencies
run: poetry install --with docs

- name: Check markdown formatting
uses: dprint/[email protected]
if: runner.os == 'Linux'

- name: Test docs build (mkdocs)
run: poetry run mkdocs build -f docs/mkdocs.yml
Expand All @@ -79,3 +83,46 @@ jobs:
uses: bakdata/ci-templates/.github/workflows/[email protected]
secrets:
pypi-token: ${{ secrets.TEST_PYPI_TOKEN }}

publish-docs-from-main:
runs-on: ubuntu-22.04
if: ${{ github.ref == 'refs/heads/main' }}
needs: [test]
steps:
- uses: actions/checkout@v3

- name: Publish docs from main branch
uses: ./.github/actions/update-docs
with:
username: ${{ secrets.GH_USERNAME }}
email: ${{ secrets.GH_EMAIL }}
token: ${{ secrets.GH_TOKEN }}
version: main

publish-dev-docs-from-pr:
runs-on: ubuntu-22.04
if: ${{ github.event_name == 'pull_request' }}
needs: [test]
steps:
- uses: actions/checkout@v3
with:
ref: ${{ github.head_ref }}

# Checks to see if any files in the PR match one of the listed file types.
# This will return true if there's a file in docs folder that was added, deleted, or modified in the PR.
- name: Check if files in docs folder have changed
uses: dorny/paths-filter@v2
id: docs-changes
with:
filters: |
docs:
- added|deleted|modified: 'docs/**'
- name: Publish dev docs from PR
if: steps.docs-changes.outputs.docs == 'true'
uses: ./.github/actions/update-docs
with:
username: ${{ secrets.GH_USERNAME }}
email: ${{ secrets.GH_EMAIL }}
token: ${{ secrets.GH_TOKEN }}
version: dev
1 change: 1 addition & 0 deletions .github/workflows/lint.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ on:
- main
paths:
- .github/**
- actions/**

jobs:
actionlint:
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,8 @@ jobs:
- name: Update gh-pages
uses: ./.github/actions/update-docs
with:
tag: ${{ needs.create-github-release-push-tag.outputs.release-version }}
username: ${{ secrets.GH_USERNAME }}
email: ${{ secrets.GH_EMAIL }}
token: ${{ secrets.GH_TOKEN }}
version: ${{ needs.create-github-release-push-tag.outputs.release-version }}
release: true
38 changes: 15 additions & 23 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,43 +1,28 @@
repos:
- repo: https://github.com/pycqa/isort
rev: 5.12.0
hooks:
- id: isort
args: ["--settings", "setup.cfg"]
exclude: ^tests/.*snapshots/
- repo: local
hooks:
- id: ruff
name: ruff
entry: ruff check .
args: [ --config, pyproject.toml, --fix, --show-fixes, --exit-non-zero-on-fix ]
language: system
types_or: [python]
require_serial: true # run once for all files
pass_filenames: false
- id: black
name: black
entry: black
language: system
types_or: [python, pyi]
require_serial: true # run once for all files
exclude: ^tests/.*snapshots/
- repo: https://github.com/pycqa/flake8
rev: 5.0.4
hooks:
- id: flake8
args: ["--config", "setup.cfg"]
exclude: ^tests/.*snapshots/
- repo: local
hooks:
- id: pyright
name: pyright
entry: pyright
language: system
types: [python]
require_serial: true # run once for all files
exclude: ^tests/.*snapshots/
- repo: https://github.com/asottile/pyupgrade
rev: v3.1.0
hooks:
- id: pyupgrade
stages: [manual]
args: ["--py310-plus"]
exclude: ^tests/.*snapshots/
- repo: local
hooks:
- id: gen-schema
name: gen-schema
entry: python hooks/gen_schema.py
Expand Down Expand Up @@ -81,3 +66,10 @@ repos:
| kpops/components/.*\.py
)$
require_serial: true
- id: dprint
name: dprint
entry: dprint fmt
language: system
types: [markdown]
require_serial: true
pass_filenames: false
113 changes: 113 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,117 @@
# Changelog
## [2.0.9](https://github.com/bakdata/kpops/releases/tag/2.0.9) - Release Date: [2023-09-19]

### 🐛 Fixes

- Fix Kafka connect config name for deletion - [#361](https://github.com/bakdata/kpops/pull/361)


### 📝 Documentation

- Fix link to kpops-examples - [#357](https://github.com/bakdata/kpops/pull/357)






## [2.0.8](https://github.com/bakdata/kpops/releases/tag/2.0.8) - Release Date: [2023-09-06]

### 🐛 Fixes

- Fix config.yaml overriding environment variables - [#353](https://github.com/bakdata/kpops/pull/353)


### 🏭 Refactor

- Refactor component prefix & name - [#326](https://github.com/bakdata/kpops/pull/326)

- Remove unnecessary condition during inflate - [#328](https://github.com/bakdata/kpops/pull/328)






## [2.0.7](https://github.com/bakdata/kpops/releases/tag/2.0.7) - Release Date: [2023-08-31]

### 🐛 Fixes

- Print only rendered templates when `--template` flag is set - [#350](https://github.com/bakdata/kpops/pull/350)


### 📝 Documentation

- Add migration guide - [#352](https://github.com/bakdata/kpops/pull/352)






## [2.0.6](https://github.com/bakdata/kpops/releases/tag/2.0.6) - Release Date: [2023-08-30]

### 🏭 Refactor

- Simplify deployment with local Helm charts - [#349](https://github.com/bakdata/kpops/pull/349)






## [2.0.5](https://github.com/bakdata/kpops/releases/tag/2.0.5) - Release Date: [2023-08-30]

### 🐛 Fixes

- Fix versioning of docs when releasing - [#346](https://github.com/bakdata/kpops/pull/346)






## [2.0.4](https://github.com/bakdata/kpops/releases/tag/2.0.4) - Release Date: [2023-08-29]

### 🐛 Fixes

- Fix GitHub ref variable for pushing docs to main branch - [#343](https://github.com/bakdata/kpops/pull/343)


### 📝 Documentation

- Add `dprint` as the markdown formatter - [#337](https://github.com/bakdata/kpops/pull/337)

- Publish pre-release docs for PRs & main branch - [#339](https://github.com/bakdata/kpops/pull/339)

- Align docs colours - [#345](https://github.com/bakdata/kpops/pull/345)


### 🌀 Miscellaneous

- Exclude abstract components from pipeline schema - [#332](https://github.com/bakdata/kpops/pull/332)






## [2.0.3](https://github.com/bakdata/kpops/releases/tag/2.0.3) - Release Date: [2023-08-24]

### 🐛 Fixes

- Fix GitHub action error in non-Python projects - [#340](https://github.com/bakdata/kpops/pull/340)


### 🌀 Miscellaneous

- Lint GitHub action - [#342](https://github.com/bakdata/kpops/pull/342)






## [2.0.2](https://github.com/bakdata/kpops/releases/tag/2.0.2) - Release Date: [2023-08-23]

### 📝 Documentation
Expand Down
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
[![Build status](https://github.com/bakdata/kpops/actions/workflows/ci.yaml/badge.svg)](https://github.com/bakdata/kpops/actions/workflows/ci.yaml)
[![pypi](https://img.shields.io/pypi/v/kpops.svg)](https://pypi.org/project/kpops)
[![versions](https://img.shields.io/pypi/pyversions/kpops.svg)](https://github.com/bakdata/kpops)
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
[![license](https://img.shields.io/github/license/bakdata/kpops.svg)](https://github.com/bakdata/kpops/blob/main/LICENSE)

## Key features
Expand All @@ -21,7 +22,7 @@ the [documentation](https://bakdata.github.io/kpops/latest).

## Install KPOps

KPOps comes as a [PyPI package](https://pypi.org/project/kpops/).
KPOps comes as a [PyPI package](https://pypi.org/project/kpops/).
You can install it with [pip](https://github.com/pypa/pip):

```shell
Expand Down
Loading

0 comments on commit 22300ad

Please sign in to comment.