Duplicate records Inserted

[dbhatia2]

Here Setup
Maxwell is reading from mysql binary logs and sending it to kafka.

• From kafka, we are using this kafka-connect-mongodb, version 9 to import to Mongodb
• listening at kafka consumer, we only see 1-1 message.
• In Mongo for every message is produced, we are seeing 2 entries pushed.

Can you please suggest, Let me know if you need more logs.

more mongo_connector_configs.json.bkp
{
"name":"mongo-connector-",
"config" :{
"connector.class":"com.startapp.data.MongoSinkConnector",
"tasks.max":"5",
"db.host":"",
"db.port":"27017",
"db.name":"*",
"db.collections":"mysqlcdc",
"write.batch.enabled":"true",
"write.batch.size":"200",
"connect.use_schema":"false",
"topics":"maxwell"
}
}

more /tmp/connect-distributed.properties|grep -v '#'

bootstrap.servers=localhost:9092

group.id=connect-cluster

key.converter=org.apache.kafka.connect.json.JsonConverter
value.converter=org.apache.kafka.connect.json.JsonConverter
key.converter.schemas.enable=false
value.converter.schemas.enable=false

internal.key.converter=org.apache.kafka.connect.json.JsonConverter
internal.value.converter=org.apache.kafka.connect.json.JsonConverter
internal.key.converter.schemas.enable=false
internal.value.converter.schemas.enable=false

offset.storage.topic=mongo-connect-offsets
offset.flush.interval.ms=10000
config.storage.topic=mongo-connect-configs

bin/kafka-topics.sh --list --zookeeper localhost:2181
__consumer_offsets
maxwell
mongo-connect-configs
mongo-connect-offs - marked for deletion
mongo-connect-offsets
test - marked for deletion

Thanks
Dk

[itayhasson12]

Hi!
Have you solved this bug?