Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(expr): handle invalid local time during daylight saving forward #15670

Merged
merged 4 commits into from
Mar 14, 2024

Conversation

wangrunji0408
Copy link
Contributor

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

Recently US timezone started Daylight Saving Time since 2024-03-10 02:00. RisingWave could not correctly convert timestamps to local time during 2024-03-10 02:00-03:00.

dev=> select '2024-03-10 02:00'::timestamp AT TIME ZONE 'us/pacific';
ERROR:  Failed to run the query

Caused by these errors (recent errors listed first):
  1: Expr error
  2: Invalid parameter local timestamp: fail to interpret local timestamp "2024-03-10 02:00:00" in time zone "US/Pacific"

This PR fixes this error and makes its behavior consistent with postgres.

dev=> select '2024-03-10 02:00'::timestamp AT TIME ZONE 'us/pacific';
         ?column?          
---------------------------
 2024-03-10 10:00:00+00:00
(1 row)

Checklist

  • I have written necessary rustdoc comments
  • I have added necessary unit tests and integration tests
  • I have added test labels as necessary. See details.
  • I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features Sqlsmith: Sql feature generation #7934).
  • My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
  • All checks passed in ./risedev check (or alias, ./risedev c)
  • My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)
  • My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details)

Documentation

  • My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note

If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.

@github-actions github-actions bot added the type/fix Bug fix label Mar 13, 2024
Signed-off-by: Runji Wang <[email protected]>
e2e_test/batch/functions/issue_12072.slt.part Outdated Show resolved Hide resolved
// * use UTC offset after the transition
let instant_local = match input.0.and_local_timezone(time_zone) {
LocalResult::Single(t) => t,
LocalResult::None => (input.0 + chrono::Duration::hours(1))
Copy link
Contributor

@xiangjinwu xiangjinwu Mar 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI although the difference of 1-hour is very common, it is not always the case. As mentioned in issue_12072.slt.part, it could be half hour. Additionally, there may be one-off changes other than regular daylight saving, for example Singapore jumped from 1981-12-31 23:29:59 to 1982-01-01 00:00:00. ('1981-12-31 23:40'::timestamp at time zone 'Asia/Singapore' in PostgreSQL corresponds to 1981-12-31T16:10:00Z) The Singapore example is also half hour but such one-off change may be more arbitrary.

A real fix is to update the binary search logic inside chrono locating time zone offsets. If a naive time does not match any entry, it shall return its preceding and following entry rather than None. So that we can pick the preceding entry to match PostgreSQL.

A quick hack may be to minus 3 hours in naive time first, do the timezone conversion, and add 3 hours back in the UTC timeline. This assumes jump forwards are less than 3 hours (IIRC there was 2-hour change) and there is a single change within this 3-hour window.

To clarify my word usage in this comment: Asia/Singapore is a timezone, while UTC+07:30 and UTC+08:00 are two offsets used by this timezone.

Copy link
Contributor

@xiangjinwu xiangjinwu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we include the Singapore test case as well? Just as a reminder there are arbitrary one-off changes in addition to regular daylight saving.

src/expr/impl/src/scalar/timestamptz.rs Outdated Show resolved Hide resolved
@wangrunji0408
Copy link
Contributor Author

wangrunji0408 commented Mar 14, 2024

Can we include the Singapore test case as well? Just as a reminder there are arbitrary one-off changes in addition to regular daylight saving.

Test failed in this case. 🥵

test("1981-12-31 23:30:00", "Asia/Singapore", "1981-12-31 16:00:00+00:00");

assertion `left == right` failed
  left: "1981-12-31 15:30:00+00:00"
 right: "1981-12-31 16:00:00+00:00"

Observations:

  • the jump happens at 1981-12-31 16:00 -> 16:30 in our current implementation.
  • this is the UTC time of 23:30 -> 00:00 Singapore local time.
  • Not sure if it is a bug in chrono-tz

I decide to add the test case but just ignore it for now. After all it is very unlikely to encounter this problem in production.

@xiangjinwu
Copy link
Contributor

  • the jump happens at 1981-12-31 16:00 -> 16:30 in our current implementation.

I tried chrono-tz and got the same problem. Seems to be bug of its embedded tzdb (not reading from OS as PostgreSQL)

@wangrunji0408 wangrunji0408 enabled auto-merge March 14, 2024 08:39
@wangrunji0408 wangrunji0408 added this pull request to the merge queue Mar 14, 2024
Merged via the queue into main with commit 9d504ab Mar 14, 2024
26 of 27 checks passed
@wangrunji0408 wangrunji0408 deleted the wrj/fix-timestamp-at-time-zone branch March 14, 2024 09:31
github-actions bot pushed a commit that referenced this pull request Mar 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/fix Bug fix
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants