[BUG] strict_date_time should accept timestamps without a trailing Z or timezone offset

[wrigleyDan]

Describe the bug

When working with a mapping like the following it is impossible to index documents with timestamp values that do not contain a trailing Z or a timezone offset:

"timestamp": {
  "type": "date",
  "format": "strict_date_time",
  "ignore_malformed": true
}

Possible values:

2024-06-01T13:38:18.280Z - trailing Z
2024-06-01T13:38:18.280+04:00 - timezone offset

Invalid values:

2024-06-01T13:38:18.280

It seems like it enforces the ISO 8601 specification according to which the referenced invalid value would also be a valid value.

When using Python and isoformat() you can create an invalid timestamp:

• https://docs.python.org/3/library/datetime.html#datetime.date.isoformat
• https://www.geeksforgeeks.org/isoformat-method-of-datetime-class-in-python/

That may lead to confusion for OpenSearch users.

Related component

Search

To Reproduce

Field mapping:

"timestamp": {
  "type": "date",
  "format": "strict_date_time",
  "ignore_malformed": true
}

Index command:

POST ubi_queries/_bulk
{"index": {"_index": "ubi_queries", "_id": "83043e11-3d7d-4a17-a0cf-a108e6735ad5"}}
{"application": "esci_ubi_sample", "query_id": "0c6d8f76-449c-4acc-8e49-492be1606f9a", "client_id": "85ea3183-2264-47fc-ace1-2b1ed37ba168", "user_query": "abc", "query_attributes": {}, "timestamp": "2024-06-01T13:38:18.280"}

Expected behavior

Indexing values without a trailing Z or timezone offset should be accepted and default to either Z or 00:00

Additional Details

Plugins
Please list all plugins currently enabled.

Screenshots
If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

• OS: [e.g. iOS]
• Version [e.g. 22]

Additional context
Add any other context about the problem here.

[peteralfonsi]

I've raised a PR making this change. Not sure which, if any, other formats should also have optional timezones. Anybody have thoughts on this?