Add additional documentation for UTC representation of timestamps #6994
Conversation
Abdullahsab3
changed the title
There was a problem hiding this comment.
Thank you for the contribution @Abdullahsab3 . This change Makes sense to me.
I spetnt some time messing searching around in this repo and it does appear to use "+00:00" for UTC
https://github.com/search?q=repo%3Aapache%2Farrow-rs%20%22UTC%22&type=code
It also seems to be what is used by DataFusion mostly as well: https://github.com/search?q=repo%3Aapache%2Fdatafusion+%22UTC%22&type=code
💯
|
Seems as if there is one small CI failure |
|
I plan to merge this in the next few days -- I will leave it open a bit longer to give anyone else a chance to comment who might be interested. Thanks again @Abdullahsab3 |
| /// For UTC time | ||
| /// ---------------------------- | ||
| /// For UTC time, it is possible to use either the timezone representation, such as "UTC", or the absolute time zone offset "+00:00". | ||
| /// However, it is better to use the offset representation, as it is more explicit and less ambiguous. | ||
| /// This also ensures that other arrow-rs functionalities can interpret the UTC timestamps correctly | ||
| /// For example, the `with_timezone_utc` method that is applied on timestamp arrays to add the UTC timezone. |
There was a problem hiding this comment.
If I'm understanding the original issue correctly, this isn't limited to UTC and +00:00, right? If we substitute JST and +09:00 we'd run into the same issue; in this case, it might be worth generalizing this documentation such that it isn't specific to UTC but more around offsets vs named timezones.
There was a problem hiding this comment.
This part is specific to UTC, since arrow-rs functionalities regarding UTC time (such as with_timezone_utc) uses +00:00 to indicate UTC. So if you create a timestamp array with the UTC timezone, and call with_timezone_utc, the call would fail.
I don't believe it's possible to substitute named timezones with fixed offsets. There is daylight savings that needs to be taken into account. for example Europe/Brussels is either +01:00 or +02:00 depending on DST
There was a problem hiding this comment.
True, but some timezones have fixed offsets where DST doesn't factor in (hence why I gave example of JST). Just thought it might be beneficial to make it clear that even if the named timezone has only 1 fixed offset ever, they still aren't equivalent in the eyes of arrow-rs (and can still highlight UTC as the main example for this).
There was a problem hiding this comment.
I agree we can try and make the statement a bit more general. However, as written I think this is an improvement over main so I will merge it and make a follow on PR with proposed improvement
There was a problem hiding this comment.
|
Thank you @Abdullahsab3 and @Jefffrey 🙏 |
Which issue does this PR close?
closes #6993
Rationale for this change
Additional documentation of what representation should be used for UTC
What changes are included in this PR?
Are there any user-facing changes?