-
Notifications
You must be signed in to change notification settings - Fork 1.8k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #57548 from gabriel-rh/SRVKE-1366-namespaced-kafka…
…-broker SRVKE-1366 Namespaced Kafka broker
- Loading branch information
Showing
3 changed files
with
97 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,70 @@ | ||
// Module included in the following assemblies: | ||
// | ||
// * serverless/eventing/brokers/kafka-broker.adoc | ||
|
||
:_content-type: PROCEDURE | ||
[id="serverless-create-kafka-namespaced-broker-{context}"] | ||
= Creating a Knative broker for Apache Kafka that uses an isolated data plane | ||
|
||
:FeatureName: The Knative Broker implementation for Apache Kafka with isolated data plane | ||
include::snippets/technology-preview.adoc[leveloffset=+2] | ||
|
||
To create a `KafkaNamespaced` broker, you must set the `eventing.knative.dev/broker.class` annotation to `KafkaNamespaced`. | ||
|
||
.Prerequisites | ||
|
||
* The {ServerlessOperatorName}, Knative Eventing, and the `KnativeKafka` custom resource are installed on your {product-title} cluster. | ||
* You have access to an Apache Kafka instance, such as link:https://access.redhat.com/documentation/en-us/red_hat_amq/7.6/html/amq_streams_on_openshift_overview/kafka-concepts_str#kafka-concepts-key_str[Red Hat AMQ Streams], and have created a Kafka topic. | ||
* You have created a project, or have access to a project, with the appropriate roles and permissions to create applications and other workloads in {product-title}. | ||
* You have installed the OpenShift CLI (`oc`). | ||
.Procedure | ||
|
||
. Create an Apache Kafka-based broker by using a YAML file: | ||
+ | ||
[source,yaml] | ||
---- | ||
apiVersion: eventing.knative.dev/v1 | ||
kind: Broker | ||
metadata: | ||
annotations: | ||
eventing.knative.dev/broker.class: KafkaNamespaced <1> | ||
name: default | ||
namespace: my-namespace <2> | ||
spec: | ||
config: | ||
apiVersion: v1 | ||
kind: ConfigMap | ||
name: my-config <2> | ||
... | ||
---- | ||
<1> To use the Apache Kafka broker with isolated data planes, the broker class value must be `KafkaNamespaced`. | ||
<2> The referenced `ConfigMap` object `my-config` must be in the same namespace as the `Broker` object, in this case `my-namespace`. | ||
|
||
. Apply the Apache Kafka-based broker YAML file: | ||
+ | ||
[source,terminal] | ||
---- | ||
$ oc apply -f <filename> | ||
---- | ||
|
||
[IMPORTANT] | ||
==== | ||
The `ConfigMap` object in `spec.config` must be in the same namespace as the `Broker` object: | ||
[source,yaml] | ||
---- | ||
apiVersion: v1 | ||
kind: ConfigMap | ||
metadata: | ||
name: my-config | ||
namespace: my-namespace | ||
data: | ||
... | ||
---- | ||
==== | ||
|
||
After the creation of the first `Broker` object with the `KafkaNamespaced` class, the `kafka-broker-receiver` and `kafka-broker-dispatcher` deployments are created in the namespace. Subsequently, all brokers with the `KafkaNamespaced` class in the same namespace will use the same data plane. If no brokers with the `KafkaNamespaced` class exist in the namespace, the data plane in the namespace is deleted. |
25 changes: 25 additions & 0 deletions
25
modules/serverless-kafka-broker-with-isolated-dataplane.adoc
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
// Module included in the following assemblies: | ||
// | ||
// * serverless/eventing/brokers/kafka-broker.adoc | ||
|
||
:_content-type: CONCEPT | ||
[id="serverless-kafka-broker-with-isolated-dataplane_{context}"] | ||
= Knative Broker implementation for Apache Kafka with isolated data plane | ||
|
||
:FeatureName: The Knative Broker implementation for Apache Kafka with isolated data plane | ||
include::snippets/technology-preview.adoc[leveloffset=+2] | ||
|
||
The Knative Broker implementation for Apache Kafka has 2 planes: | ||
|
||
Control plane:: Consists of controllers that talk to the Kubernetes API, watch for custom objects, and manage the data plane. | ||
|
||
Data plane:: The collection of components that listen for incoming events, talk to Apache Kafka, and send events to the event sinks. The Knative Broker implementation for Apache Kafka data plane is where events flow. The implementation consists of `kafka-broker-receiver` and `kafka-broker-dispatcher` deployments. | ||
|
||
When you configure a Broker class of `Kafka`, the Knative Broker implementation for Apache Kafka uses a shared data plane. This means that the `kafka-broker-receiver` and `kafka-broker-dispatcher` deployments in the `knative-eventing` namespace are used for all Apache Kafka Brokers in the cluster. | ||
|
||
However, when you configure a Broker class of `KafkaNamespaced`, the Apache Kafka broker controller creates a new data plane for each namespace where a broker exists. This data plane is used by all `KafkaNamespaced` brokers in that namespace. This provides isolation between the data planes, so that the `kafka-broker-receiver` and `kafka-broker-dispatcher` deployments in the user namespace are only used for the broker in that namespace. | ||
|
||
[IMPORTANT] | ||
==== | ||
As a consequence of having separate data planes, this security feature creates more deployments and uses more resources. Unless you have such isolation requirements, use a *regular* Broker with a class of `Kafka`. | ||
==== |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters