Skip to content

Commit

Permalink
Rst (#23)
Browse files Browse the repository at this point in the history
* Added documentation.

* Changed to 0.10.2.1-cp2.
  • Loading branch information
jcustenborder authored Jul 18, 2017
1 parent f14906f commit 9c753e6
Show file tree
Hide file tree
Showing 4 changed files with 59 additions and 9 deletions.
54 changes: 54 additions & 0 deletions docs/introduction.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
==================
Spooldir Connector
==================

This Kafka Connect connector provides the capability to watch a directory for files and read the data as new files are
written to the input directory. Each of the records in the input file will be converted based on the user supplied schema.

The CSVRecordProcessor supports reading CSV or TSV files. It can convert a CSV on the fly to the strongly typed Kafka
Connect data types. It currently has support for all of the schema types and logical types that are supported in Kafka 0.10.x.
If you couple this with the Avro converter and Schema Registry by Confluent, you will be able to process csv files to
strongly typed Avro data in real time.



.. toctree::
:maxdepth: 1
:caption: Source Connectors:
:hidden:
:glob:

sources/*


.. toctree::
:maxdepth: 1
:caption: Sink Connectors:
:hidden:
:glob:

sinks/*


.. toctree::
:maxdepth: 1
:caption: Transformations:
:hidden:
:glob:

transformations/*


.. toctree::
:maxdepth: 0
:caption: Schemas:
:hidden:

schemas







8 changes: 2 additions & 6 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -19,14 +19,10 @@
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<prerequisites>
<maven>3.3.0</maven>
</prerequisites>

<parent>
<groupId>com.github.jcustenborder.kafka.connect</groupId>
<artifactId>kafka-connect-parent</artifactId>
<version>0.10.2.0-cp1</version>
<version>0.10.2.1-cp2</version>
</parent>

<artifactId>kafka-connect-spooldir</artifactId>
Expand Down Expand Up @@ -79,4 +75,4 @@
<version>3.8</version>
</dependency>
</dependencies>
</project>
</project>
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@

import java.util.Map;

@Description("The `SpoolDirCsvSourceConnector` will monitor the directory specified in `input.path` for files and read them as a CSV " +
"converting each of the records to the strongly typed equavalent specified in `key.schema` and `value.schema`.")
@Description("The SpoolDirCsvSourceConnector will monitor the directory specified in `input.path` for files and read them as a CSV " +
"converting each of the records to the strongly typed equivalent specified in `key.schema` and `value.schema`.")
public class SpoolDirCsvSourceConnector extends SpoolDirSourceConnector<SpoolDirCsvSourceConnectorConfig> {
@Override
protected SpoolDirCsvSourceConnectorConfig config(Map<String, String> settings) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@

import java.util.Map;

@Description("This connector is used to [stream](https://en.wikipedia.org/wiki/JSON_Streaming) JSON files from a directory " +
@Description("This connector is used to `stream <https://en.wikipedia.org/wiki/JSON_Streaming>` JSON files from a directory " +
"while converting the data based on the schema supplied in the configuration.")
public class SpoolDirJsonSourceConnector extends SpoolDirSourceConnector<SpoolDirJsonSourceConnectorConfig> {

Expand Down

0 comments on commit 9c753e6

Please sign in to comment.