Test and Review Timestamp Behavior

[slominskir]

We need to do some testing to ensure timestamps found in Kafka messages make sense. I believe we're using producer provided timestamps (EPICS IOCs), which we may want to re-think. Broker (Kafka server) is the other option. We could include both if we add one to the message payload. Not sure what should happen if they're very different. Sounds like an IOC that's drifted time-wise could cause havoc on Kafka Stream joins.

There has been questions about providing timestamps for both actual (direct-from-producer) and computed (effective) messages, and at this point I believe they're identical but testing is needed. The docs here suggest that a Kafka Stream join of a KTable result in max(left.ts, right.ts).

https://docs.confluent.io/platform/current/streams/concepts.html#time

[slominskir]

See Also: JeffersonLab/epics2kafka#1

I think I came to the wrong conclusion on that issue. The safest option is likely to override the default and configure global LogAppendTime.

See: d0de43b