Skip to content

Latest commit

 

History

History
225 lines (155 loc) · 3.88 KB

DEVELOPMENT.md

File metadata and controls

225 lines (155 loc) · 3.88 KB
Raw

Development Instructions

Setup

Installing poetry:

pipx install poetry

Installing development dependencies:

poetry install

Open a terminal within the project's virtual environment:

poetry shell

See more at https://python-poetry.org/docs/cli#shell.

Installing pre-commit hooks:

pre-commit install

Running kaskade:

kaskade

Run textual console:

textual console --port 7342
textual run --port 7342 --dev -c kaskade admin -b localhost:19092
textual run --port 7342 --dev -c kaskade consumer -b localhost:19092 -t my-topic

Scripts

Unit tests:

python -m scripts.tests

E2E tests:

python -m scripts.tests --e2e

Applying code styles:

python -m scripts.styles

Running code analysis:

python -m scripts.analyze

Generate banner:

python -m scripts.banner

Docker

Build docker:

python -m scripts.docker

Image tag sauljabin/kaskade:latest.

Run with docker (create a config.yml file):

docker run --rm -it --network sandbox sauljabin/kaskade:latest admin -b kafka1:9092

Release

Help:

python -m scripts.bump --help

Upgrade (major.minor.patch):

python -m scripts.bump patch

More info at https://python-poetry.org/docs/cli/#version and https://semver.org/. For changelog management check https://github.com/sauljabin/changeloggh.

Manual Tests

Run local sandbox:

docker compose -f docker-compose.confluent.yml up -d
docker compose -f docker-compose.redpanda.yml up -d
docker compose -f docker-compose.apicurio.yml up -d

Stop sandbox:

docker compose -f docker-compose.confluent.yml down -v
docker compose -f docker-compose.redpanda.yml down -v
docker compose -f docker-compose.apicurio.yml down -v

Use the docker-compose file you need.

Populate kafka:

python -m scripts.sandbox

Read help messages:

kaskade admin --help
kaskade consumer --help

Test admin:

kaskade admin -b localhost:19092

Test consumer without deserialization:

kaskade consumer -b localhost:19092 --from-beginning -t string

Test consumer with nulls:

kaskade consumer -b localhost:19092 --from-beginning -k string -v string -t null

Test consumer with deserializers:

kaskade consumer -b localhost:19092 --from-beginning -k string -v string -t string
kaskade consumer -b localhost:19092 --from-beginning -k string -v integer -t integer
kaskade consumer -b localhost:19092 --from-beginning -k string -v long -t long
kaskade consumer -b localhost:19092 --from-beginning -k string -v float -t float
kaskade consumer -b localhost:19092 --from-beginning -k string -v double -t double
kaskade consumer -b localhost:19092 --from-beginning -k string -v boolean -t boolean
kaskade consumer -b localhost:19092 --from-beginning -k string -v json -t json

Test consumer with Schema Registry (Confluent and Redpanda):

kaskade consumer -b localhost:19092 --from-beginning -t avro \
        -k string -v registry \
        --registry url=http://localhost:18081

Test consumer with Apicurio Registry:

kaskade consumer -b localhost:19092 --from-beginning -t avro \
        -k string -v registry \
        --registry url=http://localhost:18081/apis/ccompat/v7

Test consumer avro without Schema Registry:

kaskade consumer -b localhost:19092 --from-beginning -t avro \
        -k string -v avro \
        --avro value=tests/avro/user.avsc

Test consumer with protobuf:

Install protoc with brew install protobuf.
Update descriptor with python -m scripts.protobuf.

kaskade consumer -b localhost:19092 --from-beginning -t protobuf \
        -k string -v protobuf \
        --protobuf descriptor=tests/protobuf/user.desc \
        --protobuf value=User