Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support set timezone in db #2992

Merged
merged 4 commits into from
Dec 27, 2023
Merged

feat: support set timezone in db #2992

merged 4 commits into from
Dec 27, 2023

Conversation

Taylor-lagrange
Copy link
Contributor

@Taylor-lagrange Taylor-lagrange commented Dec 25, 2023

I hereby agree to the terms of the GreptimeDB CLA

What's changed and what's your intention?

support set timezone in db

Please explain IN DETAIL what the changes are in this PR and why they are needed:

  • Add default_time_zone as a option in standalone / frontend, enable user config system time zone, default system time zone is UTC
  • Attach system time zone in QueryContext, when a new session build, the system time zone will be default user time zone, user can change time zone by SET time_zone = 'UTC';

Checklist

  • I have written the necessary rustdoc comments.
  • I have added the necessary unit tests and integration tests.

Refer to a related PR or issue link (optional)

#2907
#2893

Copy link

codecov bot commented Dec 25, 2023

Codecov Report

Attention: 11 lines in your changes are missing coverage. Please review.

Comparison is base (06fd7fd) 85.87% compared to head (22d7413) 85.19%.
Report is 7 commits behind head on develop.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #2992      +/-   ##
===========================================
- Coverage    85.87%   85.19%   -0.68%     
===========================================
  Files          780      783       +3     
  Lines       126184   126656     +472     
===========================================
- Hits        108356   107909     -447     
- Misses       17828    18747     +919     

@fengys1996 fengys1996 self-requested a review December 25, 2023 08:45
@waynexia waynexia mentioned this pull request Dec 25, 2023
8 tasks
src/cmd/src/frontend.rs Outdated Show resolved Hide resolved
src/common/time/src/time.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@killme2008 killme2008 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think timezone is better than time_zone. And it's a standard term. So can we rename all these funcitons, variables from time_zone to timezone?

Copy link
Member

@waynexia waynexia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Maybe also add some cases use time zone other than GMT and Shanghai?

src/common/time/src/timestamp.rs Outdated Show resolved Hide resolved
@Taylor-lagrange
Copy link
Contributor Author

I think timezone is better than time_zone. And it's a standard term. So can we rename all these funcitons, variables from time_zone to timezone?

time_zone mainly from SQL SET time_zone = xxxx; time_ zone or timezone both seems fine. I'll change it

@killme2008
Copy link
Contributor

I think timezone is better than time_zone. And it's a standard term. So can we rename all these funcitons, variables from time_zone to timezone?

time_zone mainly from SQL SET time_zone = xxxx; time_ zone or timezone both seems fine. I'll change it

In Chrono API, it use timezone https://docs.rs/chrono/latest/chrono/struct.DateTime.html#method.with_timezone

@Taylor-lagrange
Copy link
Contributor Author

But most of word used in our codebase is time_zone, even the main time zone related struct TimeZone in src/common/time/src/timezone.rs is call TimeZone, we should change it all?

@killme2008
Copy link
Contributor

killme2008 commented Dec 25, 2023

But most of word used in our codebase is time_zone, even the main time zone related struct TimeZone in src/common/time/src/timezone.rs is call TimeZone, we should change it all?

Yes, let's do a refactor. It looks like not so much!

tests-integration/tests/sql.rs
221:    let _ = conn.execute("SET time_zone = 'UTC'").await.unwrap();
222:    let time_zone = conn.fetch_all("SELECT @@time_zone").await.unwrap();
223:    assert_eq!(time_zone[0].get::<String, usize>(0), "UTC");
242:    let _ = conn.execute("SET time_zone = '+08:00'").await.unwrap();

src/common/time/src/timezone.rs
90:pub fn system_time_zone_name() -> String {

src/common/time/src/timestamp.rs
1057:    fn test_parse_in_time_zone() {

src/servers/src/mysql/writer.rs
197:                        .write_col(v.to_chrono_datetime_with_timezone(query_context.time_zone()))?,
199:                        .write_col(v.to_chrono_datetime_with_timezone(query_context.time_zone()))?,
211:                        .write_col(v.to_timezone_aware_string(query_context.time_zone()))?,

src/servers/src/mysql/federated.rs
24:use common_time::timezone::system_time_zone_name;
59:static SET_TIME_ZONE_PATTERN: Lazy<Regex> =
60:    Lazy::new(|| Regex::new(r"(?i)^SET TIME_ZONE\s*=\s*'(\S+)'").unwrap());
203:            "time_zone" => query_context
204:                .time_zone()
207:            "system_time_zone" => system_time_zone_name(),
271:    if let Some(captures) = SET_TIME_ZONE_PATTERN.captures(query) {
275:            session.set_time_zone(timezone);
396:        let query = "/* mysql-connector-java-8.0.17 (Revision: 16a712ddb3f826a1933ab42b0039f7fb9eebc6ec) */SELECT  @@session.auto_increment_increment AS auto_increment_increment, @@character_set_client AS character_set_client, @@character_set_connection AS character_set_connection, @@character_set_results AS character_set_results, @@character_set_server AS character_set_server, @@collation_server AS collation_server, @@collation_connection AS collation_connection, @@init_connect AS init_connect, @@interactive_timeout AS interactive_timeout, @@license AS license, @@lower_case_table_names AS lower_case_table_names, @@max_allowed_packet AS max_allowed_packet, @@net_write_timeout AS net_write_timeout, @@performance_schema AS performance_schema, @@sql_mode AS sql_mode, @@system_time_zone AS system_time_zone, @@time_zone AS time_zone, @@transaction_isolation AS transaction_isolation, @@wait_timeout AS wait_timeout;";
399:| auto_increment_increment | character_set_client | character_set_connection | character_set_results | character_set_server | collation_server | collation_connection | init_connect | interactive_timeout | license | lower_case_table_names | max_allowed_packet | net_write_timeout | performance_schema | sql_mode | system_time_zone | time_zone | transaction_isolation | wait_timeout; |
440:    fn test_set_time_zone() {
443:            "set time_zone = 'UTC'",
454:        assert_eq!("UTC", query_context.time_zone().unwrap().to_string());
456:        let output = check("select @@time_zone", query_context.clone(), session.clone());
461:| @@time_zone |

src/session/src/context.rs
38:    time_zone: Option<TimeZone>,
60:            time_zone: Default::default(),
118:    pub fn time_zone(&self) -> Option<TimeZone> {
119:        self.time_zone.clone()
145:            time_zone: self.time_zone.unwrap_or(None),

src/session/src/lib.rs
36:    time_zone: ArcSwap<Option<TimeZone>>,
48:            time_zone: ArcSwap::new(Arc::new(None)),
61:            .time_zone((**self.time_zone.load()).clone())
76:    pub fn time_zone(&self) -> Option<TimeZone> {
77:        self.time_zone.load().as_ref().clone()
81:    pub fn set_time_zone(&self, tz: Option<TimeZone>) {
82:        let _ = self.time_zone.swap(Arc::new(tz));

@killme2008
Copy link
Contributor

Do we need to process these convertions?

TimeUnit::Second => ScalarValue::TimestampSecond(val, None),

TimeUnit::Second => ScalarValue::TimestampSecond(Some(timestamp.value()), None),

Attach the timezone to the scalar value of datafusion.

@Taylor-lagrange
Copy link
Contributor Author

According to my observation, these case are used in calculation(all timestamp are convert to UTC to calculate and store, so no timezone info is fine), or the input is a i64 timestamp without timezone info ( such as prepare stmt's variable). We may need not dispose these situation, just leave it empty.

Copy link
Contributor

@killme2008 killme2008 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@killme2008 killme2008 added this pull request to the merge queue Dec 27, 2023
Merged via the queue into GreptimeTeam:develop with commit d1ee1ba Dec 27, 2023
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs-required This change requires docs update.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants