Skip to content

Commit

Permalink
remove python
Browse files Browse the repository at this point in the history
  • Loading branch information
sauljabin committed Jul 12, 2024
1 parent 96bebb8 commit 1a21cf1
Show file tree
Hide file tree
Showing 9 changed files with 37 additions and 54 deletions.
16 changes: 10 additions & 6 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ ENV GRADLE_BIN "${GRADLE_HOME}/bin"
ENV CONFLUENT_VERSION "$CONFLUENT_VERSION"
ENV CONFLUENT_HOME "/opt/confluent"
ENV CONFLUENT_BIN "${CONFLUENT_HOME}/bin"
ENV PATH "${PATH}:${CONFLUENT_BIN}:${GRADLE_BIN}:/root/.local/bin"
ENV PATH "${PATH}:${CONFLUENT_BIN}:${GRADLE_BIN}"
ENV COLORTERM "truecolor"
ENV TERM "xterm-256color"

Expand All @@ -26,11 +26,7 @@ RUN apt update \
postgresql-client \
mysql-client \
mosquitto-clients \
python-is-python3 \
python3 \
pipx \
&& rm -rf /var/lib/apt/lists/* \
&& pipx install kaskade
&& rm -rf /var/lib/apt/lists/*

RUN wget -q "http://packages.confluent.io/archive/$(echo "${CONFLUENT_VERSION}" | cut -c 1-3)/confluent-community-${CONFLUENT_VERSION}.zip" -O /tmp/confluent.zip \
&& unzip /tmp/confluent.zip -d /tmp \
Expand All @@ -41,3 +37,11 @@ RUN wget -q "https://services.gradle.org/distributions/gradle-8.8-bin.zip" -O /t
&& unzip /tmp/gradle.zip -d /tmp \
&& mv "/tmp/gradle-${GRADLE_VERSION}" "${GRADLE_HOME}" \
&& rm /tmp/gradle.zip

RUN mkdir -p "${CONFLUENT_HOME}/logs" \
&& chmod 777 "${CONFLUENT_HOME}/logs"

RUN mkdir -p "/home/ubuntu/.gradle" \
&& chmod 777 "/home/ubuntu/.gradle"

USER ubuntu
2 changes: 1 addition & 1 deletion docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -206,7 +206,7 @@ services:
CLUSTER_ID: ${CLUSTER_ID}
volumes:
- .:/kafka-sandbox
- gradle_cache:/root/.gradle
- gradle_cache:/home/ubuntu/.gradle
entrypoint: /bin/bash
tty: true

Expand Down
26 changes: 6 additions & 20 deletions md/json-producer-and-consumer.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ Producer:
```java
if (useSchemaRegistry) {
props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, KafkaJsonSchemaSerializer.class);
props.put(AbstractKafkaSchemaSerDeConfig.SCHEMA_REGISTRY_URL_CONFIG, "http://schema-registry:8081");
props.put(AbstractKafkaSchemaSerDeConfig.SCHEMA_REGISTRY_URL_CONFIG, SCHEMA_REGISTRY_URL);
} else {
props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, KafkaJsonSerializer.class);
}
Expand All @@ -37,7 +37,7 @@ Consumer:
```java
if (useSchemaRegistry) {
props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, KafkaJsonSchemaDeserializer.class);
props.put(AbstractKafkaSchemaSerDeConfig.SCHEMA_REGISTRY_URL_CONFIG, "http://schema-registry:8081");
props.put(AbstractKafkaSchemaSerDeConfig.SCHEMA_REGISTRY_URL_CONFIG, SCHEMA_REGISTRY_URL);
} else {
props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, KafkaJsonDeserializer.class);
}
Expand All @@ -63,30 +63,16 @@ kafka-topics --create --bootstrap-server kafka1:9092 \
--topic client.schema.users
```

### Produce

Produce **without** Schema Registry:
### Produce/Consume without Schema Registry

```bash
gradle kafka-json-clients:run --args="produce client.users 100"
```

Produce **with** Schema Registry:

```bash
gradle kafka-json-clients:run --args="produce -s client.schema.users 100"
```

### Consume

Consume **without** Schema Registry:

```bash
gradle kafka-json-clients:run --args="consume client.users"
```

Consume **with** Schema Registry:
### Produce/Consume with Schema Registry

```bash
gradle kafka-json-clients:run --args="produce -s client.schema.users 100"
gradle kafka-json-clients:run --args="consume -s client.schema.users"
```
```
10 changes: 5 additions & 5 deletions md/kafka-connect-mqtt-example.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,14 @@ docker compose --profile mqtt up -d
In one terminal, subscribe to mqtt topics:

```bash
mosquitto_sub -h mosquitto -t 'house/+/brightness'
mosquitto_sub -h mosquitto -t "house/+/brightness"
```

In another terminal, publish messages:

```bash
mosquitto_pub -h mosquitto -t 'house/room/brightness' -m '800LM'
mosquitto_pub -h mosquitto -t 'house/kitchen/brightness' -m '1000LM'
mosquitto_pub -h mosquitto -t "house/room/brightness" -m "800LM"
mosquitto_pub -h mosquitto -t "house/kitchen/brightness" -m "1000LM"
```

### Create Source Connector
Expand Down Expand Up @@ -47,8 +47,8 @@ kafka-console-consumer --from-beginning --group connect.mqtt \
In another terminal, publish new messages to the MQTT broker:

```bash
mosquitto_pub -h mosquitto -t 'house/room/brightness' -m '810LM'
mosquitto_pub -h mosquitto -t 'house/kitchen/brightness' -m '1020LM'
mosquitto_pub -h mosquitto -t "house/room/brightness" -m "810LM"
mosquitto_pub -h mosquitto -t "house/kitchen/brightness" -m "1020LM"
```

Deleting the connector:
Expand Down
2 changes: 1 addition & 1 deletion md/kafka-mqtt-proxy.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ kafka-topics --create \
Publish using mqtt proxy:

```bash
mosquitto_pub -h kafka-mqtt -p 1884 -t 'house/room/temperature' -m '20C'
mosquitto_pub -h kafka-mqtt -p 1884 -t "house/room/temperature" -m "20C"
```

Check the data:
Expand Down
7 changes: 7 additions & 0 deletions md/ksqldb-queries.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,3 +38,10 @@ ksql -f kafka-ksqldb/ksql/insert-orders.ksql http://ksqldb:8088
```bash
ksql -e "PRINT 'ksqldb.order_sizes' FROM BEGINNING;" http://ksqldb:8088
```

### Drop

```bash
ksql -e "DROP STREAM ORDERSIZES;" http://ksqldb:8088
ksql -e "DROP STREAM ORDERS;" http://ksqldb:8088
```
2 changes: 1 addition & 1 deletion md/ksqldb-tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ One interesting feature that ksqlDB has is the test runner, it allows you to tes
ksql-test-runner -e kafka-ksqldb-extensions/extensions/ \
-s kafka-ksqldb/ksql/create-orders.ksql \
-i kafka-ksqldb/tests/orders-input.json \
-o kafka-ksqldb/tests/orders-output.json | grep '>>>'
-o kafka-ksqldb/tests/orders-output.json | grep ">>>"
```


Expand Down
24 changes: 5 additions & 19 deletions md/protobuf-producer-and-consumer.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ Producer:
```java
if (useSchemaRegistry) {
props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, KafkaProtobufSerializer.class);
props.put(AbstractKafkaSchemaSerDeConfig.SCHEMA_REGISTRY_URL_CONFIG, "http://schema-registry:8081");
props.put(AbstractKafkaSchemaSerDeConfig.SCHEMA_REGISTRY_URL_CONFIG, SCHEMA_REGISTRY_URL);
} else {
// ProtobufSerializer is a custom class
props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, ProtobufSerializer.class);
Expand All @@ -37,7 +37,7 @@ Consumer:
```java
if (useSchemaRegistry) {
props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, KafkaProtobufDeserializer.class);
props.put(AbstractKafkaSchemaSerDeConfig.SCHEMA_REGISTRY_URL_CONFIG, "http://schema-registry:8081");
props.put(AbstractKafkaSchemaSerDeConfig.SCHEMA_REGISTRY_URL_CONFIG, SCHEMA_REGISTRY_URL);
} else {
// ProtobufDeserializer is a custom class
props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, ProtobufDeserializer.class);
Expand Down Expand Up @@ -66,30 +66,16 @@ kafka-topics --create --bootstrap-server kafka1:9092 \
--topic client.schema.invoices
```

### Produce

Produce **without** Schema Registry:
### Produce/Consume without Schema Registry

```bash
gradle kafka-protobuf-clients:run --args="produce client.invoices 100"
```

Produce **with** Schema Registry:

```bash
gradle kafka-protobuf-clients:run --args="produce -s client.schema.invoices 100"
```

### Consume

Consume **without** Schema Registry:

```bash
gradle kafka-protobuf-clients:run --args="consume client.invoices"
```

Consume **with** Schema Registry:
### Produce/Consume with Schema Registry

```bash
gradle kafka-protobuf-clients:run --args="produce -s client.schema.invoices 100"
gradle kafka-protobuf-clients:run --args="consume -s client.schema.invoices"
```
2 changes: 1 addition & 1 deletion md/sandbox-environment.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
For opening the sandbox environment just run:

```bash
docker compose exec -u $(id -u):$(id -g) cli bash
docker compose exec cli bash
```

> [!IMPORTANT]
Expand Down

0 comments on commit 1a21cf1

Please sign in to comment.