Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Implement CONCAT_WS #2427

Closed
wants to merge 6 commits into from
Closed

Conversation

kwannoel
Copy link
Contributor

@kwannoel kwannoel commented May 11, 2022

What's changed and what's your intention?

WIP, will fill these up later. #2405 , #112

  • Parsing
  • Display
  • parse - display test
  • parse test (fail, success, subexprs)
  • Update parser, should use Function instead
  • Typechecking seperator must be text, rest is any. Does it cast first arg if non-text?
  • Check modules for src/expr/expr_concat.ws.
  • Binding
  • Binding tests
  • Explicitly declare return type? Leave as comment
  • Evaluation how to do datum -> string?
  • linter, update description
  • Leave comment: What number is appropriate to assign for concat_ws in protobuf spec?
  • File header remarks for new files

PLEASE DO NOT LEAVE THIS EMPTY !!!

Please explain IN DETAIL what the changes are in this PR and why they are needed:

  • Summarize your change (mandatory)
  • How does this PR work? Need a brief introduction for the changed logic (optional)
  • Describe clearly one logical change and avoid lazy messages (optional)
  • Describe any limitations of the current code (optional)

Checklist

  • I have written necessary docs and comments
  • I have added necessary unit tests and integration tests

Refer to a related PR or issue link (optional)

@CLAassistant
Copy link

CLAassistant commented May 11, 2022

CLA assistant check
All committers have signed the CLA.

@kwannoel kwannoel changed the title Implement concat_ws parsing feat: Support CONCAT_WS May 11, 2022
@kwannoel kwannoel changed the title feat: Support CONCAT_WS feat: Implement CONCAT_WS May 11, 2022
@@ -0,0 +1,97 @@
// Copyright 2022 Singularity Data
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WIP, this needs to be updated

Copy link
Contributor

@xiangjinwu xiangjinwu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some early suggestions. Hope they are helpful.

Comment on lines 257 to 261
/// CONCAT_WS(SEP TEXT, STR "ANY" [, STR "ANY" [, ...] ])
ConcatWs {
sep_expr: Box<Expr>,
string_exprs: Vec<Expr>,
},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI we do not need to update the parser and add new variant / keyword for each new function. concat_ws(a, b, c) can be parsed as the general function call syntax. The variants here are for those with special syntax.

For example, substring(v1 from 2) needs special treatment because of the from, but substr(v1, 2) does not.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To check my understanding: I should be parsing it into Expr::Function variant instead?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes

@@ -107,6 +107,7 @@ impl FunctionCall {
Ok(DataType::Boolean)
}
ExprType::Coalesce => Ok(inputs[0].return_type()),
ExprType::ConcatWs => Ok(DataType::Varchar), // TODO: Validate input types, see below on how to recursively do so.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should have left some comments here... The infer_type can only handle functions with a limited number of parameters, while the ones listed here (case-when, in-list, coalesce, concat) are all variadic.

To insert cast expr over all input expr to string/varchar, call cast_explicit on them.

  • It can be a little bit confusing as the cast is actually implicit. But casting to varchar implicitly is disallowed in general cases and concat is just special here.
  • As you may noticed in Tracking: Basic Built-in Functions #112, casting from another type to string has not been implemented in backend except for boolean.

@kwannoel
Copy link
Contributor Author

Migrated over to #2470 for CI to run correctly.

@kwannoel kwannoel closed this May 12, 2022
@kwannoel kwannoel mentioned this pull request May 12, 2022
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants