From 050d3cfe4d9f6473dfa0d8256817f4a96a6df7e3 Mon Sep 17 00:00:00 2001 From: Gabriel McGoldrick Date: Wed, 22 Mar 2023 10:39:34 +0000 Subject: [PATCH] SRVKE-1366 Namespaces Kafka broker --- ...erless-create-kafka-namespaced-broker.adoc | 70 +++++++++++++++++++ ...-kafka-broker-with-isolated-dataplane.adoc | 25 +++++++ serverless/eventing/brokers/kafka-broker.adoc | 3 +- 3 files changed, 97 insertions(+), 1 deletion(-) create mode 100644 modules/serverless-create-kafka-namespaced-broker.adoc create mode 100644 modules/serverless-kafka-broker-with-isolated-dataplane.adoc diff --git a/modules/serverless-create-kafka-namespaced-broker.adoc b/modules/serverless-create-kafka-namespaced-broker.adoc new file mode 100644 index 000000000000..1f8a7df019d0 --- /dev/null +++ b/modules/serverless-create-kafka-namespaced-broker.adoc @@ -0,0 +1,70 @@ +// Module included in the following assemblies: +// +// * serverless/eventing/brokers/kafka-broker.adoc + +:_content-type: PROCEDURE +[id="serverless-create-kafka-namespaced-broker-{context}"] += Creating a Knative broker for Apache Kafka that uses an isolated data plane + +:FeatureName: The Knative Broker implementation for Apache Kafka with isolated data plane +include::snippets/technology-preview.adoc[leveloffset=+2] + +To create a `KafkaNamespaced` broker, you must set the `eventing.knative.dev/broker.class` annotation to `KafkaNamespaced`. + +.Prerequisites + +* The {ServerlessOperatorName}, Knative Eventing, and the `KnativeKafka` custom resource are installed on your {product-title} cluster. + +* You have access to an Apache Kafka instance, such as link:https://access.redhat.com/documentation/en-us/red_hat_amq/7.6/html/amq_streams_on_openshift_overview/kafka-concepts_str#kafka-concepts-key_str[Red Hat AMQ Streams], and have created a Kafka topic. + +* You have created a project, or have access to a project, with the appropriate roles and permissions to create applications and other workloads in {product-title}. + +* You have installed the OpenShift CLI (`oc`). + +.Procedure + +. Create an Apache Kafka-based broker by using a YAML file: ++ +[source,yaml] +---- +apiVersion: eventing.knative.dev/v1 +kind: Broker +metadata: + annotations: + eventing.knative.dev/broker.class: KafkaNamespaced <1> + name: default + namespace: my-namespace <2> +spec: + config: + apiVersion: v1 + kind: ConfigMap + name: my-config <2> +... +---- +<1> To use the Apache Kafka broker with isolated data planes, the broker class value must be `KafkaNamespaced`. +<2> The referenced `ConfigMap` object `my-config` must be in the same namespace as the `Broker` object, in this case `my-namespace`. + +. Apply the Apache Kafka-based broker YAML file: ++ +[source,terminal] +---- +$ oc apply -f +---- + +[IMPORTANT] +==== +The `ConfigMap` object in `spec.config` must be in the same namespace as the `Broker` object: + +[source,yaml] +---- +apiVersion: v1 +kind: ConfigMap +metadata: + name: my-config + namespace: my-namespace +data: + ... +---- +==== + +After the creation of the first `Broker` object with the `KafkaNamespaced` class, the `kafka-broker-receiver` and `kafka-broker-dispatcher` deployments are created in the namespace. Subsequently, all brokers with the `KafkaNamespaced` class in the same namespace will use the same data plane. If no brokers with the `KafkaNamespaced` class exist in the namespace, the data plane in the namespace is deleted. diff --git a/modules/serverless-kafka-broker-with-isolated-dataplane.adoc b/modules/serverless-kafka-broker-with-isolated-dataplane.adoc new file mode 100644 index 000000000000..4e12750a7e65 --- /dev/null +++ b/modules/serverless-kafka-broker-with-isolated-dataplane.adoc @@ -0,0 +1,25 @@ +// Module included in the following assemblies: +// +// * serverless/eventing/brokers/kafka-broker.adoc + +:_content-type: CONCEPT +[id="serverless-kafka-broker-with-isolated-dataplane_{context}"] += Knative Broker implementation for Apache Kafka with isolated data plane + +:FeatureName: The Knative Broker implementation for Apache Kafka with isolated data plane +include::snippets/technology-preview.adoc[leveloffset=+2] + +The Knative Broker implementation for Apache Kafka has 2 planes: + +Control plane:: Consists of controllers that talk to the Kubernetes API, watch for custom objects, and manage the data plane. + +Data plane:: The collection of components that listen for incoming events, talk to Apache Kafka, and send events to the event sinks. The Knative Broker implementation for Apache Kafka data plane is where events flow. The implementation consists of `kafka-broker-receiver` and `kafka-broker-dispatcher` deployments. + +When you configure a Broker class of `Kafka`, the Knative Broker implementation for Apache Kafka uses a shared data plane. This means that the `kafka-broker-receiver` and `kafka-broker-dispatcher` deployments in the `knative-eventing` namespace are used for all Apache Kafka Brokers in the cluster. + +However, when you configure a Broker class of `KafkaNamespaced`, the Apache Kafka broker controller creates a new data plane for each namespace where a broker exists. This data plane is used by all `KafkaNamespaced` brokers in that namespace. This provides isolation between the data planes, so that the `kafka-broker-receiver` and `kafka-broker-dispatcher` deployments in the user namespace are only used for the broker in that namespace. + +[IMPORTANT] +==== +As a consequence of having separate data planes, this security feature creates more deployments and uses more resources. Unless you have such isolation requirements, use a *regular* Broker with a class of `Kafka`. +==== diff --git a/serverless/eventing/brokers/kafka-broker.adoc b/serverless/eventing/brokers/kafka-broker.adoc index 6c4d059e7f95..15bb436175ad 100644 --- a/serverless/eventing/brokers/kafka-broker.adoc +++ b/serverless/eventing/brokers/kafka-broker.adoc @@ -16,7 +16,8 @@ If your {ServerlessProductName} deployment is not configured to use Kafka broker include::modules/serverless-kafka-broker.adoc[leveloffset=+2] include::modules/serverless-kafka-broker-with-kafka-topic.adoc[leveloffset=+2] - +include::modules/serverless-kafka-broker-with-isolated-dataplane.adoc[leveloffset=+2] +include::modules/serverless-create-kafka-namespaced-broker.adoc[leveloffset=+2] // kafka broker general configmap include::modules/serverless-kafka-broker-configmap.adoc[leveloffset=+1]