Timestamp parsing breaks on different format

[matt-porter]

https://github.com/SalesforceFoundation/salesforce-streaming-client/blob/master/src/salesforce_streaming_client/__init__.py#L97-L101

Hi, I'm seeing a ValueError at this line, very rarely, when I receive a timestamp without the fractional seconds included. Just wondered if this was a known issue or something that I can work around.

This output is with the following modification:

def string_to_datetime(date_string):
    print('Parsing string:', date_string)
    try:
        return datetime.strptime(
            date_string,
            '%Y-%m-%dT%H:%M:%S.%fZ'
        ).replace(tzinfo=pytz.utc)
    except ValueError:
        return datetime.strptime(
            date_string,
            '%Y-%m-%dT%H:%M:%SZ'
        ).replace(tzinfo=pytz.utc)

Parsing string: 2019-09-11T11:31:03.855000Z
Parsing string: 2019-09-11T11:31:03.877000Z
Parsing string: 2019-09-11T11:31:04.020000Z
Parsing string: 2019-09-11T11:31:29.272000Z
Parsing string: 2019-09-11T11:31:29.396000Z
Parsing string: 2019-09-11T11:31:29.539000Z
Parsing string: 2019-09-11T09:11:26Z

Traceback (most recent call last):
  File "salesforce_to_kafka.py", line 224, in <module>
    run(None, None)
  File "salesforce_to_kafka.py", line 211, in run
    gevent.with_timeout(60*7, client.block)
  File "/Users/matt/.pyenv/versions/salesforce_to_kafka/lib/python3.6/site-packages/gevent/timeout.py", line 367, in with_timeout
    return function(*args, **kwds)
  File "/Users/matt/.pyenv/versions/salesforce_to_kafka/lib/python3.6/site-packages/python_bayeux/__init__.py", line 443, in block
    self._execute_greenlet()
  File "/Users/matt/.pyenv/versions/salesforce_to_kafka/lib/python3.6/site-packages/python_bayeux/__init__.py", line 268, in _execute_greenlet
    getattr(self, callback)(message_queue_message)
  File "/Users/matt/.pyenv/versions/salesforce_to_kafka/lib/python3.6/site-packages/salesforce_streaming_client/__init__.py", line 511, in generic_callback
    'created_date': string_to_datetime(created_date),
  File "/Users/matt/.pyenv/versions/salesforce_to_kafka/lib/python3.6/site-packages/salesforce_streaming_client/__init__.py", line 101, in string_to_datetime
    '%Y-%m-%dT%H:%M:%S.%fZ'
  File "/Users/matt/.pyenv/versions/3.6.4/lib/python3.6/_strptime.py", line 565, in _strptime_datetime
    tt, fraction = _strptime(data_string, format)
  File "/Users/matt/.pyenv/versions/3.6.4/lib/python3.6/_strptime.py", line 362, in _strptime
    (data_string, format))
ValueError: time data '2019-09-11T09:11:26Z' does not match format '%Y-%m-%dT%H:%M:%S.%fZ'

Thanks,

Matt

[adamlincoln]

Thanks @matt-porter - maybe it's a one-in-a-thousand thing when milliseconds is zero? It was not a known issue and there isn't a workaround (other than patching your copy like you have done), so I really appreciate the report.

I'll fix and release an update soon.