spark-sql-streaming-MicroBatchReadSupport.adoc

MicroBatchReadSupport Contract — Data Sources with MicroBatchReaders

MicroBatchReadSupport is the abstraction of data sources with a MicroBatchReader in Micro-Batch Stream Processing.

ContinuousReadSupport defines a single createMicroBatchReader method to create a MicroBatchReader.

MicroBatchReader createMicroBatchReader(
  Optional<StructType> schema,
  String checkpointLocation,
  DataSourceOptions options)

createMicroBatchReader is used when:

• MicroBatchExecution is requested for the analyzed logical plan (and creates a StreamingExecutionRelation for a StreamingRelationV2 with a MicroBatchReadSupport data source)

• DataStreamReader is requested to create a streaming query for a MicroBatchReadSupport data source

Table 1. MicroBatchReadSupports

MicroBatchReadSupport	Description
KafkaSourceProvider	Data source provider for kafka format
RateStreamProvider
TextSocketSourceProvider