1. Kafka Configuration

Download kafka

1. Download and unpack kafka open source from confluent.io

Result, a ´confluent´ directory containing a set of sub directories like bin, etc, logs, share and so on. Consider ´confluent´ as current directory for the other steps.

Kafka broker

2. Go to ./etc/kafka and open server.properties in an editor

Enter the following at the top

authorizer.class.name=no.nav.common.security.authorization.SimpleLDAPAuthorizer
super.users=User:srvkafkabroker;User:srvkafkasregistry

listeners=SASL_PLAINTEXT://localhost:9092
security.inter.broker.protocol=SASL_PLAINTEXT
sasl.mechanism.inter.broker.protocol=PLAIN
sasl.enabled.mechanisms=PLAIN 

3. Add the following file ´kafka_server_jaas_example.conf´ to ./etc/kafka

Content

KafkaServer {
   no.nav.common.security.plain.PlainLoginModule required
   username="srvkafkabroker"
   password="broker";
};

4. Add the following line to ./bin/kafka-server-start, let it be the 2nd last line, just before exec ...

export KAFKA_OPTS=$"-Djava.security.auth.login.config=$base_dir/../etc/kafka/kafka_server_jaas_example.conf"

Schema registry

5. Go to ./etc/schema-registry and open schema-registry.properties in an editor Enter the following at the top

kafkastore.security.protocol=SASL_PLAINTEXT
kafkastore.sasl.mechanism=PLAIN 

6. Add the following file ´kafka_client_jaas_example.conf´ to ./etc/kafka

Content

KafkaClient {
   org.apache.kafka.common.security.plain.PlainLoginModule required
   username="srvkafkasregistry"
   password="sregistry";
};

7. Add the following line to ./bin/schema-registry-start, let it be the 2nd last line, just before exec ...

export SCHEMA_REGISTRY_OPTS=$"-Djava.security.auth.login.config=$(dirname $0)/../etc/kafka/kafka_client_jaas_example.conf"

Disable kafka REST and Connect

In order to get faster start and stop, it is ok to comment out REST and Connect in the confluent startup script.

8. Open ./bin/confluent in an editor and comment start/stop of REST and Connect

Details

declare -a services=(
    "zookeeper"
    "kafka"
    "schema-registry"
#    "kafka-rest"
#    "connect"
)

declare -a rev_services=(
#   "connect"
#   "kafka-rest"
    "schema-registry"
    "kafka"
    "zookeeper"
)