spark-sql-streaming-SymmetricHashJoinStateManager.adoc

SymmetricHashJoinStateManager

SymmetricHashJoinStateManager is created exclusively for OneSideHashJoiner (for StreamingSymmetricHashJoinExec physical operator to execute).

Creating SymmetricHashJoinStateManager Instance

SymmetricHashJoinStateManager takes the following to be created:

• JoinSide

• Input value attributes

• Join keys (Seq[Expression])

• StatefulOperatorStateInfo

• StateStoreConf

• Hadoop Configuration

SymmetricHashJoinStateManager initializes the internal properties.

Performance Metrics — metrics Method

metrics: StateStoreMetrics

metrics returns the combined StateStoreMetrics of the KeyToNumValuesStore and the KeyWithIndexToValueStore.

Note	metrics is used exclusively when OneSideHashJoiner is requested to commitStateAndGetMetrics.

allStateStoreNames Object Method

allStateStoreNames(joinSides: JoinSide*): Seq[String]

allStateStoreNames…​FIXME

Note	allStateStoreNames is used when…​FIXME

Internal Properties

Name	Description
keyAttributes	Key attributes Used when…​FIXME
keySchema	Key schema Used when…​FIXME
keyToNumValues	KeyToNumValuesStore Used when…​FIXME
keyWithIndexToValue	KeyWithIndexToValueStore Used when…​FIXME