Skip to content

Commit

Permalink
Merge pull request #283 from Aiven-Open/minor-changes
Browse files Browse the repository at this point in the history
Update readme with quickstart steps
  • Loading branch information
muralibasani authored May 24, 2024
2 parents 8ed2745 + 9f01015 commit 6ab08e5
Show file tree
Hide file tree
Showing 2 changed files with 21 additions and 2 deletions.
23 changes: 21 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Aiven's OpenSearch® Connector for Apache Kafka®
# Aiven's OpenSearch® Sink Connector for Apache Kafka®

[![Build Status](https://github.com/aiven/opensearch-connector-for-apache-kafka/actions/workflows/master_push_workflow.yml/badge.svg)](https://github.com/aiven/opensearch-connector-for-apache-kafka/actions)

Expand All @@ -11,7 +11,14 @@ The code was forked and all classes were renamed.

# Documentation

## How to install
## Install and configure required software:
1. Install and Run Kafka (https://kafka.apache.org/quickstart)
2. Install and Run Kafka Connect (https://kafka.apache.org/documentation/#connect_running)
3. Install and Run OpenSearch (https://opensearch.org/docs/latest/install-and-configure/install-opensearch/index/)
- If OpenSearch is running on SSL, make sure to copy the credentials and copy them to woker properties as mentioned in Quickstart guide below
4. Install Aiven's OpenSearch® Sink Connector (see below)

## How to install Aiven's OpenSearch® Sink Connector

1. Connector plugins are packaged in zip/tar format to be released
2. Users download plugins from GitHub releases or build binaries from source
Expand Down Expand Up @@ -75,6 +82,18 @@ curl http://localhost:8083/connector-plugins | jq .

[OpenSearch® Sink Connector Configuration Options](docs/opensearch-sink-connector-config-options.rst)

## QuickStart guide

1. Ensure that the required software is installed and running
2. Add OpenSearch Sink connector to Kafka Connect: Follow the 'How to install' instructions to add the OpenSearch sink connector to Kafka Connect. Example worker config is located here https://github.com/Aiven-Open/opensearch-connector-for-apache-kafka/blob/main/config/quickstart-opensearch.properties
3. Verify plugin installation : Visit http://localhost:8083/connectors to confirm that the OpenSearch sink connector is listed
4. Check ACLs (If Enabled): If ACLs are enabled on Kafka, ensure there are no authorization exceptions for the topic and group resources. Example of adding acls : https://kafka.apache.org/documentation/#security_authz_examples
5. Produce Events: Produce JSON-formatted events to the Kafka topic specified in the worker properties.
6. Index Creation: An index will be created in OpenSearch with the same name as the Kafka topic.
7. Create Index Pattern: Create an index pattern in OpenSearch.
8. Discover Events: Events produced to the Kafka topic can now be discovered in OpenSearch.
9. Trouble shooting: If there are any deserialization errors in the connector logs, try setting schema.ignore to true.

# Contribute

[Source Code](https://github.com/aiven/aiven-kafka-connect-opensearch)
Expand Down
File renamed without changes.

0 comments on commit 6ab08e5

Please sign in to comment.