From 0d8f9cf609d6b41d8e9d094711db4d3dbaf1afad Mon Sep 17 00:00:00 2001 From: Piotr Belke Date: Sun, 8 Sep 2024 11:10:11 +0200 Subject: [PATCH] IKC-388 Documentation --- docs/FAQ.md | 24 ++++---- docs/FEATURES.md | 28 +++++++--- docs/README.md | 33 ++++++++--- docs/configuration/CUSTOM_CONTEXT_PATH.md | 10 +++- docs/configuration/DATABASE.md | 11 ++-- docs/configuration/JMX.md | 24 +++++--- docs/configuration/LOGGING.md | 15 +++-- docs/configuration/WEBSOCKET.md | 4 +- docs/configuration/kafka/AWS_MSK.md | 13 +++-- docs/configuration/kafka/SASL_PLAIN.md | 6 +- docs/configuration/kafka/TLS.md | 13 +++-- .../schema-registry/SCHEMA_REGISTRY_SSL.md | 9 ++- .../SCHEMA_REGISTRY_SSL_BASIC_AUTH.md | 10 +++- docs/configuration/security/AUTHENTICATION.md | 4 +- docs/configuration/security/AUTHORIZATION.md | 32 ++++++----- docs/configuration/security/GITHUB.md | 2 - docs/configuration/security/LDAP.md | 3 + .../security/LOCAL_AUTHENTICATION.md | 6 +- docs/installation/DEPLOYMENT.md | 56 +++++++++++++------ docs/installation/DEVELOPMENT.md | 22 +++++--- 20 files changed, 216 insertions(+), 109 deletions(-) diff --git a/docs/FAQ.md b/docs/FAQ.md index 661a53b4..46519f0e 100644 --- a/docs/FAQ.md +++ b/docs/FAQ.md @@ -15,56 +15,56 @@ com.consdata.kouncil.KouncilRuntimeException: java.util.concurrent.ExecutionExce ``` follow one of the below fixes. -If you run Kafka in terminal (not using docker) you can: +If you run Kafka in terminal (not using Docker) you can: - 1. use your host IP address. In that case you would have to modify IP address of listeners and advertised.listeners in server.properties. Kouncil docker run command would look like this (replace to yours IP address): + 1. Use your host IP address - in this case you need to modify the IP address of listeners and advertised.listeners in server.properties. Kouncil Docker run command would look like this (replace with yours IP address): ```bash docker run -p 80:8080 -e bootstrapServers=":9092" -e kouncil.auth.active-provider="inmemory" consdata/kouncil:latest ``` Kouncil will be available via: http://localhost/login - 2. use IP address of the docker bridge network. To find that IP address you have to run docker network inspect bridge in terminal and under Config use value assigned to Gateway. Also you have to modify IP address of listeners and advertised.listeners in server.properties to the found IP address of the gateway. If you run Kouncil in Windows or Mac you should be able to use host.docker.internal as IP address of bootstrapServer. + 2. Use the IP address of the Docker bridge network - to find that IP address, run Docker network, inspect the bridge in the terminal, and under Config, use the value assigned to Gateway. You also need to modify the IP addresses of listeners and advertised.listeners in server.properties to the IP address of the gateway. If you run Kouncil on Windows or Mac, you should be able to use host.docker.internal as the IP address of bootstrapServer. ```bash docker run -p 80:8080 -e bootstrapServers="host.docker.internal:9092" -e kouncil.auth.active-provider="inmemory" consdata/kouncil:latest ``` - But if you are using Linux you will have to use --add-host: + If you are using Linux, you will have to use --add-host: ```bash docker run -p 80:8080 -e bootstrapServers="host.docker.internal:9092" --add-host=host.docker.internal:host-gateway -e kouncil.auth.active-provider="inmemory" consdata/kouncil:latest ``` Kouncil will be available via: http://localhost/login - 3. add --network host to Kouncil docker command. It will look like this (as you can see I also removed publish flag as this is discarded by docker when using host network mode): + 3. Add --network host to Kouncil Docker command. It will look like this (as you can see, I removed the publish flag as it is discarded by Docker when using host network mode): ```bash docker run -e bootstrapServers="localhost:9092" --network host -e kouncil.auth.active-provider="inmemory" consdata/kouncil:latest ``` Kouncil will be available via: http://localhost:8080/login -If you run Kafka using docker container you have to put both containers in the same network, so they can reach out to each other. Firstly create new network using +If you run Kafka using a Docker container, you have to put both containers in the same network, so they can communicate with each other. First, create a new network using ```bash docker network create --driver bridge ``` -Then use this network name in run command/docker compose files, for example Kouncil docker run command will look like this: +Then use this network name in run command/Docker compose files, for example, Kouncil Docker run command will look like this: ```bash docker run -p 80:8080 -e bootstrapServers=":9092" --network="" -e kouncil.auth.active-provider="inmemory" consdata/kouncil:latest ``` -Also, you should use Kafka docker container name as IP address (see ) +Also, you should use Kafka Docker container name as an IP address (see ) ## I logged in and I see only brokers and consumer groups. You logged in as an administrator user, `admin`. In this case we have few possible solutions: - 1. Log in as a user with editor role (in default configuration it will be the `editor` user) - 2. Pass environment variable, `-e`, `kouncil.authorization.role-editor` in `docker run` command, which will include the group name for the `admin` user. This value will be used instead of the value from default configuration. Docker run command will look like this: + 1. Log in as a user with the editor’s role (in default configuration, it will be the `editor` user) + 2. Pass the environment variable, `-e`, `kouncil.authorization.role-editor` in the `docker run` command, with the value of the group name for the admin user. This value will be used instead of the value from default configuration. The Docker run command will look like this: ```bash docker run -p 80:8080 -e kouncil.authorization.role-editor="editor_group;admin_group" consdata/kouncil:latest ``` - 3. In docker run command mount volume which will have a custom configuration file. Docker run command will look like this: + 3. In the Docker run command, mount a volume with a custom configuration file. The command will look like this: ```bash docker run -p 80:8080 -v :/config/ consdata/kouncil:latest ``` ## I don't see any resolution to my issue -Please [reach out to us](mailto:kouncil@consdata.com) +Please [reach out to us](https://kouncil.io/contact-us/) diff --git a/docs/FEATURES.md b/docs/FEATURES.md index 3d036b30..57af4e46 100644 --- a/docs/FEATURES.md +++ b/docs/FEATURES.md @@ -1,10 +1,15 @@ # Features -Here are some of the main features of Kouncil. This list is not exhaustive, check out our [demo app](https://kouncil-demo.web.app/) or [quickly install Kouncil](README.md#quick-start) to experience them first-hand. +Here are some of the most important features of Kouncil. This list is not exhaustive. Check out +our [demo app](https://kouncil-demo.web.app/) or [quickly install Kouncil](README.md#quick-start) to +experience all features first-hand. ## Advanced record browsing in table format -Thanks to Kouncil's convenient way of presenting records in table format even large amounts of complex messages can be easily browsed. You can choose between browsing a single partition or a topic as a whole. If you wish to examine any of the messages more closely you can view its source, copy it to clipboard, or even post it again. +Thanks to Kouncil's convenient way of presenting records in a table format, even large amounts of +complex messages can be easily browsed. You can choose between browsing a single partition or a +topic as a whole. If you wish to examine any of the messages more closely, you can view its source, +copy it to a clipboard, or even post it again.

@@ -16,27 +21,34 @@ Thanks to Kouncil's convenient way of presenting records in table format even la ## Multiple cluster support -If your config spans across multiple Kafka clusters it's no problem for Kouncil. You can switch between them at any time, without having to restart or reconfigure anything. +If your config spans across multiple Kafka clusters, it's not a problem for Kouncil. You can switch +between clusters at any time, without having to restart or reconfigure anything. ## Consumer monitoring -Monitoring your consumer groups is one of the most important things when dealing with Kafka. Are my consumers even connected to Kafka? Do they process events? If so, how fast? How long until they finish their workload? Kouncil can help you answer all those questions. +Monitoring your consumer groups is one of the most important things when dealing with Kafka. Are my +consumers even connected to Kafka? Do they process events? If so, how fast? How long until they +finish their work? Kouncil can help you answer all those questions.

- ## Cluster monitoring -Monitoring your cluster's health can be as important as monitoring your consumer groups. Kouncil shows not only which brokers are currently connected to the cluster, but also their current resource consumption (using Kouncil's [advanced config](installation/DEPLOYMENT.md#docker---advanced-configuration)) +Monitoring your cluster's health can be as important as monitoring your consumer groups. +Kouncil shows the brokers that are currently connected to the cluster and their current resource +consumption (using +Kouncil's [advanced config](installation/DEPLOYMENT.md#docker---advanced-configuration))

-## Event Tracking -Event Tracking enables monitoring and visualizing the path of a given event or process by means of Kafka topics. +## Event tracking + +Event tracking enables monitoring and visualizing the path of a given event or process across the +Kafka topics.

diff --git a/docs/README.md b/docs/README.md index 1555d50b..c8233ca3 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,8 +1,14 @@ # Kouncil for Apache Kafka -Kouncil lets you monitor and manage your Apache Kafka clusters using a modern web interface. It's free & open source kafka web UI, [feature-rich](FEATURES.md#features) and [easy to set up](#quick-start)! This simple kafka tool makes your DATA detectible, helps to troubleshoot problems and deliver optimal solutions. Yoy can easily monitor brokers and their condition, consumer groups and their pace along with the current lag or simply view the content of topics in real time. +Kouncil lets you monitor and manage your Apache Kafka clusters using a modern web interface.It's +an [easy-to-set-up](#quick-start), [feature-rich](FEATURES.md#features), free and open-source Kafka +web UI. This simple Kafka tool makes your DATA detectible, helps troubleshoot problems and deliver +optimal solutions. You can easily monitor brokers and their condition, consumer groups and their +pace along with the current lag, or view the content of topics in real time. + +Here are some of **[Kouncil's](https://kouncil.io) main features**. For a more comprehensive list +check out the [features section](FEATURES.md#features). -Here are some of **the main features of [Kouncil](https://kouncil.io)**. For a more comprehensive list check out the [features section](FEATURES.md#features). * Advanced record browsing in table format * Multiple cluster support * Cluster monitoring @@ -11,7 +17,8 @@ Here are some of **the main features of [Kouncil](https://kouncil.io)**. For a m ## Demo app -If you wish to simply check out Kouncil in action, without having to install it, we've prepared a demo site showcasing the main features of Kouncil. The demo site can be found [here](https://kouncil-demo.web.app/) +Check out Kouncil in action without installing it. We've prepared a demo site showcasing the main +features of Kouncil, which can be found [here]. (https://kouncil-demo.web.app/) ## Quick start @@ -20,15 +27,25 @@ The easiest way to start working with Kouncil is by using Docker: ```bash docker run -d -p 80:8080 -e bootstrapServers="kafka1:9092" -e kouncil.auth.active-provider="inmemory" consdata/kouncil:latest ``` -There is only two required environment variables: `bootstrapServers` which should point to one of the brokers in your Kafka cluster and `kouncil.auth.active-provider` which specified authentication mode. For example, if your cluster consists of three brokers - kafka1:9092, kafka2:9092, kafka3:9092 - you only have to specify one of them (`-e bootstrapServers="kafka1:9092"`), and you are good to go, Kouncil will automatically do the rest! +There are only two required environment variables: `bootstrapServers`, which should point to one of +the brokers in your Kafka cluster, and `kouncil.auth.active-provider`, which specifies +authentication mode. For example, if your cluster consists of three brokers - kafka1:9092, kafka2: +9092, kafka3:9092 - you only have to specify one of them (`-e bootstrapServers="kafka1:9092"`), and +you are good to go. Kouncil will automatically do the rest. -Additionally, Kouncil supports multiple clusters. Hosts specified in `bootstrapServers` may point to brokers in several clusters, and Kouncil will recognize that properly. Brokers should be separated using comma, i.e.: `docker run -d -p 80:8080 -e bootstrapServers="kafka1:9092,kafka1.another.cluster:8001" -e kouncil.auth.active-provider="inmemory" consdata/kouncil:latest` +Additionally, Kouncil supports multiple clusters. Hosts specified in `bootstrapServers` may point to +brokers in several clusters, and Kouncil will recognize that properly. Brokers should be separated +using a comma, +i.e.: `docker run -d -p 80:8080 -e bootstrapServers="kafka1:9092,kafka1.another.cluster:8001" -e kouncil.auth.active-provider="inmemory" consdata/kouncil:latest` -After the `docker run` command head to [http://localhost](http://localhost). +After the `docker run` command, head to [http://localhost](http://localhost). Images for Kouncil are hosted here: https://hub.docker.com/r/consdata/kouncil. -For more advanced configuration consult the [Deployment](installation/DEPLOYMENT.md#deployment) section. +For more advanced configuration, consult the [Deployment](installation/DEPLOYMENT.md#deployment) +section. ## Authentication -Default credentials to log in to Kouncil are admin/admin. For more authentication option head out to [Authentication](configuration/security/AUTHENTICATION.md) + +Default credentials to log in to Kouncil are admin/admin. For more authentication options, check +out [Authentication](configuration/security/AUTHENTICATION.md) diff --git a/docs/configuration/CUSTOM_CONTEXT_PATH.md b/docs/configuration/CUSTOM_CONTEXT_PATH.md index 0d3ecbc6..18e4863a 100644 --- a/docs/configuration/CUSTOM_CONTEXT_PATH.md +++ b/docs/configuration/CUSTOM_CONTEXT_PATH.md @@ -1,8 +1,12 @@ ## Custom context path -If you want to expose Kouncil in custom context path you need to set Spring's `kouncil.context-path` parameter. -In docker run command it will look like this +If you want to expose Kouncil in a custom context path, you need to set +Spring's `kouncil.context-path` parameter. +In Docker run command it will look like this + ```bash docker run -d -p 80:8080 -e bootstrapServers="kafka1:9092" -e kouncil.context-path="/console" consdata/kouncil:latest ``` -After that, visit [http://localhost/console](http://localhost/console) in your browser, and you should be greeted by a login screen. + +After that, visit [http://localhost/console](http://localhost/console) in your browser, and you +should see a login screen. diff --git a/docs/configuration/DATABASE.md b/docs/configuration/DATABASE.md index bf8b480e..abd39331 100644 --- a/docs/configuration/DATABASE.md +++ b/docs/configuration/DATABASE.md @@ -1,24 +1,27 @@ ## Database configuration Currently, Kouncil supports two databases: + * PostgreSQL * H2 If no database is specified with below properties, H2 in-memory database will be used. ### Configuration properties + * `spring.datasource.url` JDBC URL of the database * `spring.datasource.username` login username of the database * `spring.datasource.password` login password of the database -* `spring.jpa.hibernate.default_schema` sets default schema +* `spring.jpa.hibernate.default_schema` sets default schema ### Example + ```yaml spring: datasource: - url: jdbc:postgresql://localhost:5432/ - username: postgres - password: password + url: jdbc:postgresql://localhost:5432/ + username: postgres + password: password jpa: hibernate: default_schema: kouncil diff --git a/docs/configuration/JMX.md b/docs/configuration/JMX.md index 99f3f105..4f58d2e7 100644 --- a/docs/configuration/JMX.md +++ b/docs/configuration/JMX.md @@ -1,6 +1,8 @@ ## Advanced config - JMX monitoring -If your Kafka brokers expose JMX metrics Kouncil can take advantage of that, displaying additional metrics. This is done using advanced config, where you can specify JMX parameters for each broker, like so: +If your Kafka brokers expose JMX metrics, Kouncil can leverage them to display additional metrics. +This is done using advanced config, where you can specify JMX parameters for each broker, as +follows: ```yaml kouncil: @@ -17,7 +19,10 @@ kouncil: port: 9094 jmxPort: 5090 ``` -This example assumes that broker does not require any kind of authentication to access JMX metrics - you only need to specify JMX port. If that's not the case, and JMX authentication is turned on, you can also specify JMX user and password: + +This example assumes that the broker does not require authentication to access JMX metrics - you +only need to specify the JMX port. If JMX authentication is enabled, you can also specify the JMX +username and password: ```yaml kouncil: @@ -40,7 +45,10 @@ kouncil: jmxUser: jmxAdmin jmxPassword: jmxPassword ``` -It quickly becomes clear, that in many cases those properties (`jmxPort`, `jmxUser`, `jmxPassword`) will be identical for each of the brokers inside the cluster. For that reason, you can also specify them on a cluster level, and they will propagate to each broker: + +It quickly becomes clear that, in many cases, the properties (`jmxPort`, `jmxUser`, `jmxPassword`) +will be identical for all brokers within the cluster. For that reason, you can specify them at the +cluster level, and they will be propagated to each broker: ```yaml kouncil: @@ -51,15 +59,18 @@ kouncil: jmxPassword: jmxPassword brokers: - host: 192.10.0.1 - port: 9092 + port: 9092 - host: 192.10.0.2 port: 9093 - host: 192.10.0.3 port: 9094 ``` -All brokers inside `transaction-cluster` will share the same JMX configuration (`jmxPort` = `5088`, `jmxUser` = `jmxAdmin`, `jmxPassword` = `jmxPassword`). -Propagation of JMX parameters works independently for each of those parameters. For example, each of the brokers may have the same JMX user and password, but different port: +All brokers within the `transaction-cluster` will share the same JMX +configuration (`jmxPort` = `5088`, `jmxUser` = `jmxAdmin`, `jmxPassword` = `jmxPassword`). + +Propagation of JMX parameters works independently for each parameter. For example, while all brokers +may share the same JMX user and password, they can have different ports. ```yaml kouncil: @@ -79,4 +90,3 @@ kouncil: jmxPort: 5090 ``` -In the case of both simple and advanced configuration being present, the advanced configuration takes precedence. diff --git a/docs/configuration/LOGGING.md b/docs/configuration/LOGGING.md index 31ae0d84..b8bf1c23 100644 --- a/docs/configuration/LOGGING.md +++ b/docs/configuration/LOGGING.md @@ -1,12 +1,19 @@ ## Logging -Kouncil supports logging to external log file. We suggest to use logback. If you want you can use our provided `logback.xml` file as it is or use it as a reference to create your custom one. -If you use provided `logback.xml` logs will be placed under logs/kouncil.log + +Kouncil supports logging to an external log file, and we recommend using Logback. You can either use +the provided `logback.xml` file as-is or use it as a reference to create your custom one. +If you use the provided `logback.xml`, logs will be placed under logs/kouncil.log ```bash docker run -d -p 80:8080 -e bootstrapServers="kafka1:9092" -e logging.config="path_to_your_logback_xml_file_in_docker_container" -v path_to_your_local_logback_xml_folder:/path_to_your_container_logback_xml_folder consdata/kouncil:latest ``` -If you want the logs to be accessible outside the docker container, you could pass another volume in docker run command like this: + +If you want the logs to be accessible outside the Docker container, you can add another volume to +the Docker run command like this: + ```bash -v path_to_your_local_logback_xml_folder:path_to_docker_container_logs ``` -Also `path_to_docker_container_logs` should be equal to path in `appender/file` parameter in `logback.xml` + +Ensure that `path_to_docker_container_logs` matches the path specified in the `appender/file` +parameter in your `logback.xml`. diff --git a/docs/configuration/WEBSOCKET.md b/docs/configuration/WEBSOCKET.md index dd415372..6a3f643c 100644 --- a/docs/configuration/WEBSOCKET.md +++ b/docs/configuration/WEBSOCKET.md @@ -1,5 +1,7 @@ ## WebSocket allowed origins configuration -By default, WebSocket allowed origins are set to *, which can be inefficient from the security point of view. You can easily narrow it down, setting `allowedOrigins` environment variable like that: + +By default, WebSocket allowed origins are set to *, which can be insecure. You can easily narrow it +down by setting the allowedOrigins environment variable as follows: ```bash docker run -d -p 80:8080 -e bootstrapServers="kafka1:9092" -e allowedOrigins="http://localhost:*, https://yourdomain.com" consdata/kouncil:latest diff --git a/docs/configuration/kafka/AWS_MSK.md b/docs/configuration/kafka/AWS_MSK.md index 3b00cdda..3cbbaaae 100644 --- a/docs/configuration/kafka/AWS_MSK.md +++ b/docs/configuration/kafka/AWS_MSK.md @@ -3,10 +3,13 @@ {% hint style="warning" %} **WARNING** -This configuration will be deprecated in version 1.9 and removed in 1.10. In version 1.9 this configuration will be used to preload your clusters. +This configuration will be deprecated in version 1.9 and removed in 1.10. In version 1.9 this +configuration will be used to preload your clusters. {% endhint %} -If one of your brokers in cluster environment is located in Amazon MSK cluster you should specify `saslMechanism`, `saslProtocol`, `saslJassConfig` and `saslCallbackHandler` for this broker, like this: +If one of your brokers in a cluster environment is located in an Amazon MSK cluster, you should +specify `saslMechanism`, `saslProtocol`, `saslJassConfig`, and `saslCallbackHandler` for this +broker, like this: ```yaml kouncil: @@ -25,5 +28,7 @@ kouncil: port: 9094 ``` -Above configuration is using IAM access to Amazon MSK cluster and you should provide `AWS_SECRET_ACCESS_KEY` and `AWS_ACCESS_KEY_ID` as environment variables to Kouncil. -And this two values should be generated to the user which has access to Amazon MSK cluster and his username should be provided in `awsProfileName` in Kouncil configuration. +The above configuration uses IAM access to the Amazon MSK cluster, and you should +provide `AWS_SECRET_ACCESS_KEY` and `AWS_ACCESS_KEY_ID` as environment variables to Kouncil. +These two values should be generated for the user with access to the Amazon MSK cluster, and their +user name should be provided in `awsProfileName` in the Kouncil configuration. diff --git a/docs/configuration/kafka/SASL_PLAIN.md b/docs/configuration/kafka/SASL_PLAIN.md index 482e03c9..ee645837 100644 --- a/docs/configuration/kafka/SASL_PLAIN.md +++ b/docs/configuration/kafka/SASL_PLAIN.md @@ -3,10 +3,12 @@ {% hint style="warning" %} **WARNING** -This configuration will be deprecated in version 1.9 and removed in 1.10. In version 1.9 this configuration will be used to preload your clusters. +This configuration will be deprecated in version 1.9 and removed in 1.10. In version 1.9 this +configuration will be used to preload your clusters. {% endhint %} -If one of your brokers in cluster environment needs SASL authentication you should specify `saslMechanism`, `saslProtocol` and `saslJassConfig` for this broker, like this: +If one of your brokers in a cluster environment requires SASL authentication, you should +specify `saslMechanism`, `saslProtocol` and `saslJassConfig` for that broker. For example: ```yaml kouncil: diff --git a/docs/configuration/kafka/TLS.md b/docs/configuration/kafka/TLS.md index 554f50b6..cfee1f04 100644 --- a/docs/configuration/kafka/TLS.md +++ b/docs/configuration/kafka/TLS.md @@ -3,11 +3,14 @@ {% hint style="warning" %} **WARNING** -This configuration will be deprecated in version 1.9 and removed in 1.10. In version 1.9 this configuration will be used to preload your clusters. +This configuration will be deprecated in version 1.9 and removed in 1.10. In version 1.9 this +configuration will be used to preload your clusters. {% endhint %} -Let's assume that your Kafka is secured and you need mTLS to connect. You need to provide a client truststore, containing CA public certificate and keystore with both client private key and CA signed certificate. -Then add "kafka" node to your yaml with the following values: +Assuming your Kafka is secured and requires mTLS to connect, you need to provide a client truststore +containing the CA's public certificate, and a keystore with both the client's private key and the +CA-signed certificate. +Then, add a kafka node to your YAML configuration with the following values: ```yaml kouncil: @@ -23,6 +26,6 @@ kouncil: keystore-location: file:///config/keystore/client.keystore.jks key-password: secret brokers: - - host: 192.10.0.1 - port: 9092 + - host: 192.10.0.1 + port: 9092 ``` diff --git a/docs/configuration/schema-registry/SCHEMA_REGISTRY_SSL.md b/docs/configuration/schema-registry/SCHEMA_REGISTRY_SSL.md index 4bc21283..0fd30073 100644 --- a/docs/configuration/schema-registry/SCHEMA_REGISTRY_SSL.md +++ b/docs/configuration/schema-registry/SCHEMA_REGISTRY_SSL.md @@ -1,12 +1,15 @@ ## Advanced config - SSL Schema registry {% hint style="warning" %} -**WARNING** +**WARNING** -This configuration will be deprecated in version 1.9 and removed in 1.10. In version 1.9 this configuration will be used to preload your clusters. +This configuration will be deprecated in version 1.9 and removed in 1.10. In version 1.9 this +configuration will be used to preload your clusters. {% endhint %} -Let's assume that your SchemaRegistry is secured and you need SSL to connect. You need to provide a client truststore, containing CA public certificate and keystore with both client private key and CA signed certificate. +Assuming your SchemaRegistry is secured and requires SSL for connection, you need to provide a +client truststore that contains the CA's public certificate, and a keystore that includes both the +client's private key and the CA-signed certificate. ```yaml kouncil: diff --git a/docs/configuration/schema-registry/SCHEMA_REGISTRY_SSL_BASIC_AUTH.md b/docs/configuration/schema-registry/SCHEMA_REGISTRY_SSL_BASIC_AUTH.md index 5dcdddb8..b02fd6d9 100644 --- a/docs/configuration/schema-registry/SCHEMA_REGISTRY_SSL_BASIC_AUTH.md +++ b/docs/configuration/schema-registry/SCHEMA_REGISTRY_SSL_BASIC_AUTH.md @@ -3,11 +3,15 @@ {% hint style="warning" %} **WARNING** -This configuration will be deprecated in version 1.9 and removed in 1.10. In version 1.9 this configuration will be used to preload your clusters. +This configuration will be deprecated in version 1.9 and removed in 1.10. In version 1.9 this +configuration will be used to preload your clusters. {% endhint %} -Let's assume that your SchemaRegistry is secured and you need SSL and BASIC authentication to connect. You need to provide a client truststore, containing CA public certificate and keystore with both client private key and CA signed certificate. -And fot the BASIC authentication you need to provide user-info which will be use to authenticate when Kouncil will connect to Schema Registry. +Assuming your SchemaRegistry is secured and requires both SSL and BASIC authentication for +connection, you need to provide a client truststore that contains the CA's public certificate, and a +keystore that includes both the client's private key and the CA-signed certificate. For BASIC +authentication, you need to provide user credentials that will be used to authenticate when Kouncil +connects to the SchemaRegistry. ```yaml kouncil: diff --git a/docs/configuration/security/AUTHENTICATION.md b/docs/configuration/security/AUTHENTICATION.md index ca35e47f..5cf37c16 100644 --- a/docs/configuration/security/AUTHENTICATION.md +++ b/docs/configuration/security/AUTHENTICATION.md @@ -1,2 +1,4 @@ ## Authentication -Kouncil supports multiple authentication methods along with LDAP, Active Directory and SSO. There are a lot of different configuration scenarios. Here are examples of most common ones: + +Kouncil supports multiple authentication methods along with LDAP, Active Directory, and SSO. There +are a lot of different configuration scenarios. Here are examples of the most common ones: diff --git a/docs/configuration/security/AUTHORIZATION.md b/docs/configuration/security/AUTHORIZATION.md index a8f6c635..3c62f527 100644 --- a/docs/configuration/security/AUTHORIZATION.md +++ b/docs/configuration/security/AUTHORIZATION.md @@ -8,21 +8,23 @@ configuration will be used to preload user groups and based on that create permi group. {% endhint %} -Kouncil allows to restrict access to pages and functions for users. To achieve that you should add -to you configuration file appropriate properties. -There are three of -them, `kouncil.authorization.role-admin`, `kouncil.authorization.role-editor`, `kouncil.authorization.role-viewer`. -Each one of this will allow user to do specific actions in Kouncil. Users with roles assigned to: +## Authorization + +Kouncil enables restricting access to pages and functions. To achieve that you should add +appropriate properties to your configuration file. +There are three: `kouncil.authorization.role-admin`, `kouncil.authorization.role-editor`, +and`kouncil.authorization.role-viewer`. +Each one of these allows users to do specific actions in Kouncil. Users with roles assigned to: -* `kouncil.authorization.role-admin` can view brokers and consumer groups pages. -* `kouncil.authorization.role-editor` can view topics, event tracker pages and sent messages to - topics. -* `kouncil.authorization.role-viewer` can only view topics and event tracker pages. +* `kouncil.authorization.role-admin` can view brokers and consumer groups pages +* `kouncil.authorization.role-editor` can view topics, event tracker pages, and send messages to + topics +* `kouncil.authorization.role-viewer` can only view topics and event tracker pages -As a values in these parameters you should provide semicolon (`;`) separated list of groups defined -in selected authentication provider (`inmemory`, `LDAP`, `AD`, `SSO`). +As a value in these parameters you should provide a list of groups defined in the selected +authentication provider (`inmemory`, `LDAP`, `AD`, `SSO`), separated by a semicolon (`;`). -For the default configuration, we have defined user groups: +For the default configuration, we have defined the user groups: ```yaml kouncil: @@ -33,9 +35,9 @@ kouncil: ``` These will be used in any authentication method unless you override them in any of your -configuration files, which are used by Kouncil. +configuration files used by Kouncil. -Example roles configuration: +Example of role configuration: ```yaml kouncil: @@ -45,7 +47,7 @@ kouncil: role-viewer: KOUNCIL_VIEWER;viewer_group ``` -To get LDAP user groups in LDAP configuration we have to add few additional +To get LDAP user groups in LDAP configuration, add a few additional parameters (`group-search-base`, `group-search-filter`, `group-role-attribute`) ```yaml diff --git a/docs/configuration/security/GITHUB.md b/docs/configuration/security/GITHUB.md index ed261015..a963cd00 100644 --- a/docs/configuration/security/GITHUB.md +++ b/docs/configuration/security/GITHUB.md @@ -3,8 +3,6 @@ In Kouncil you can configure GitHub SSO by adding below snippet to your configuration file. You have to replace `your-client-id`, `your-client-secret` and `your-application-url`. -* Github SSO - ```yaml kouncil: auth: diff --git a/docs/configuration/security/LDAP.md b/docs/configuration/security/LDAP.md index 70b49935..5060c8c9 100644 --- a/docs/configuration/security/LDAP.md +++ b/docs/configuration/security/LDAP.md @@ -3,6 +3,7 @@ Below you can find configuration snippets for authentication using LDAP, LDAPS and AD: * LDAPS authentication for all users. + ```yaml kouncil: auth: @@ -14,6 +15,7 @@ kouncil: ``` * LDAP authentication with a technical user for users who belong to a KOUNCIL group. + ```yaml kouncil: auth: @@ -27,6 +29,7 @@ kouncil: ``` * Active Directory authentication for users who belong to a KOUNCIL group. + ```yaml kouncil: auth: diff --git a/docs/configuration/security/LOCAL_AUTHENTICATION.md b/docs/configuration/security/LOCAL_AUTHENTICATION.md index 3977e70f..dcbf765e 100644 --- a/docs/configuration/security/LOCAL_AUTHENTICATION.md +++ b/docs/configuration/security/LOCAL_AUTHENTICATION.md @@ -1,8 +1,8 @@ ### Local authentication -Simplest user authentication is using in-memory provider. Do not require any configuration. Only for the test purposes! Default -uses are superuser, admin, editor, viewer. Default password for each of these users is equal to -username. +The simplest in-memory provider. It does not require any configuration. Only for test purposes! The +default uses are admin, editor, and viewer. The default password for each of these users is user +name. ```yaml kouncil: diff --git a/docs/installation/DEPLOYMENT.md b/docs/installation/DEPLOYMENT.md index f81f503f..68fa04b9 100644 --- a/docs/installation/DEPLOYMENT.md +++ b/docs/installation/DEPLOYMENT.md @@ -1,18 +1,25 @@ # Deployment -There are two ways in which Kouncil can be configured: +Kouncil can be configured in two ways: + * simple - suitable for most cases, relying solely on `docker run` parameters -* advanced - suitable for larger configurations. Provided as an external file, and thus can be tracked in version control. It also exposes additional configuration options, which are not available in the simple configuration +* advanced - suitable for larger configurations, provided as an external file, allowing version + control tracking and providing additional configuration options not available in the simple setup + +In the case of both simple and advanced configuration being present, the advanced configuration +takes precedence. ## Docker - simple configuration -Simple configuration is passed directly into `docker run` command using `bootstrapServers` environment variable, just as we've seen in [Quick start](../README.md#quick-start): +Simple configuration is passed directly into `docker run` command using `bootstrapServers` +environment variable, just as we've seen in [Quick start](../README.md#quick-start): ```bash docker run -d -p 80:8080 -e bootstrapServers="kafka1:9092" consdata/kouncil:latest ``` -`bootstrapServers` variable expects a comma-separated list of brokers, each belonging to a different cluster. Kouncil only needs to know about a single broker from the cluster in order to work. +`bootstrapServers` variable expects a comma-separated list of brokers, each belonging to a different +cluster. Kouncil only needs to know about a single broker from the cluster in order to work. The simplest possible configuration looks like this: @@ -20,48 +27,60 @@ The simplest possible configuration looks like this: docker run -d -p 80:8080 -e bootstrapServers="kafka1:9092" consdata/kouncil:latest ``` -After that, visit [http://localhost](http://localhost) in your browser, and you should be greeted with a list of topics from your cluster. +Next, visit [http://localhost](http://localhost) in your browser, and you should be greeted with a +list of topics from your cluster. -If you have multiple clusters and wish to manage them all with Kouncil, you can do so by simply specifying one broker from each cluster using comma-separated list: +If you have multiple clusters and wish to manage them all with Kouncil, you can do so by simply +specifying one broker from each cluster using a comma-separated list: ```bash docker run -d -p 80:8080 -e bootstrapServers="kafka1:9092,kafka1.another.cluster:8001" consdata/kouncil:latest ``` -If you want to set Schema Registry url use `schemaRegistryUrl` environment variable, for instance: +If you want to set Schema Registry URL, use `schemaRegistryUrl` environment variable, for instance: + ```bash docker run -d -p 80:8080 -e bootstrapServers="kafka1:9092" -e schemaRegistryUrl="http://schema.registry:8081" consdata/kouncil:latest ``` -This url will be used for every cluster in `boostrapServers` variable. If you want to be more specific go to [Advanced configuration](#docker---advanced-configuration). -If you want to set list of headers to keep while resending events from one topic to another you can use `resendHeadersToKeep` environment variable and pass list of comma-seperated header names, for example: +This URL will be used for every cluster in `boostrapServers` variable. If you want to be more +specific, go to [Advanced configuration](#docker---advanced-configuration). + +If you want to set a list of headers to keep while resending events from one topic to another, you +can use `resendHeadersToKeep` environment variable and pass the list of comma-separated header +names, for example: + ```bash docker run -d -p 80:8080 -e bootstrapServers="kafka1:9092" -e resendHeadersToKeep="requestId,version" consdata/kouncil:latest ``` -In order to change the port on which Kouncil listens for connections, just modify the `-p` argument, like so: +To change the port on which Kouncil listens for connections, just modify the `-p` argument, like so: ```bash docker run -d -p 7070:8080 -e bootstrapServers="kafka1:9092" consdata/kouncil:latest ``` -That will cause Kouncil to listen on port `7070`. +It will cause Kouncil to listen on port `7070`. ## Docker - advanced configuration -If you have many Kafka clusters, configuring them using `bootstrapServers` may become cumbersome. It is also impossible to express more sophisticated configuration options using such a simple configuration pattern. +If you have many Kafka clusters, configuring them using `bootstrapServers` may become cumbersome. It +is also impossible to express more sophisticated configuration options using such a simple +configuration pattern. -To address these issues Kouncil allows you to provide an external configuration in a yaml file. +To address these issues, Kouncil allows you to provide an external configuration in a YAML file. -Kouncil expects this configuration file to be named `kouncil.yaml`. After that it's only a matter of binding a directory containing that file into docker - let's say your `kouncil.yaml` lives in `/home/users/test/Kouncil/config/`, that's how your `docker run` should look: +Kouncil expects this configuration file to be named `kouncil.yaml`. It's only a matter of binding a +directory containing that file with Docker. Let's say your `kouncil.yaml` lives +in `/home/users/test/Kouncil/config/` - this is what your `docker run` should look like: ```bash docker run -p 80:8080 -v /home/users/test/Kouncil/config/:/config/ consdata/kouncil:latest ``` -Format of `kouncil.yaml` is described below. +The format of `kouncil.yaml` is described below. -## Advanced config example +## Advanced configuration example ```yaml kouncil: @@ -85,4 +104,7 @@ kouncil: - host: kouncil.kafka.local port: 8002 ``` -This example shows two clusters, named `transaction-cluster` and `kouncil` respectively. Each cluster needs to have its name specified. After that comes a list of brokers that make up this cluster - each of which consisting of the broker's host and port on which it's listening on. + +This example shows two clusters named `transaction-cluster` and `kouncil`. Each cluster requires a +specified name, followed by a list of brokers that make up the cluster. Each broker entry includes +the broker's host and the port it is listening on. diff --git a/docs/installation/DEVELOPMENT.md b/docs/installation/DEVELOPMENT.md index 097acc68..e2ad1b4f 100644 --- a/docs/installation/DEVELOPMENT.md +++ b/docs/installation/DEVELOPMENT.md @@ -1,11 +1,15 @@ # Local Development ## Running the project -For the backend, run KouncilApplication passing parameter ```bootstrapServers=localhost:9092``` pointing to any of your Kafka brokers and ```spring.config.name=kouncil```. -For the frontend, having node and yarn installed, run ```yarn``` and ```yarn start``` +For the backend, run KouncilApplication passing parameter ```bootstrapServers=localhost:9092``` +pointing to any of your Kafka brokers and ```spring.config.name=kouncil```. + +For the frontend, with Node.js and Yarn installed, run ```yarn``` and ```yarn start``` + +For the local Kafka, create docker-compose.yml (KAFKA_ADVERTISED_HOST_NAME should match your Docker +host IP) -For the local Kafka, create docker-compose.yml (KAFKA_ADVERTISED_HOST_NAME should match your docker host IP) ```yaml version: "2" @@ -31,17 +35,21 @@ run ```docker-compose up -d``` more info: https://hub.docker.com/r/bitnami/kafka/ -By default, authentication is set to inmemory. Default users role configuration is described here [Authorization](../configuration/security/AUTHORIZATION.md). -You can modify this to match your needs, for example if you want to have editor role on an admin user you have to add `admin-group` to the `role-editor`. +By default, authentication is set to inmemory. The default users role configuration is described +here [Authorization](../configuration/security/AUTHORIZATION.md). +You can modify this to match your needs. For example if you want to assign the editor role to an +admin user, you need to add `admin-group` to the `role-editor`. ## Release -To release just push to release branch: +To release, simply push to release branch: + ```bash git push origin master:release ``` -after a successful release, remember to merge back to master: +after a successful release, remember to merge back into the master: + ```bash git merge origin/release ```