Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add map type to scalar/array/expr #17690

Merged
merged 11 commits into from
Aug 12, 2024
Merged

feat: add map type to scalar/array/expr #17690

merged 11 commits into from
Aug 12, 2024

Conversation

xxchan
Copy link
Member

@xxchan xxchan commented Jul 16, 2024

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

First part of #17743.

Design/background doc: https://www.notion.so/risingwave-labs/Map-type-788a6cb277de46669052683493a5d170

Only the data structures are implemented. (So it's not ready for users yet.) Other things, which should be done in the frontend, like literal syntax etc are left to next PRs.

Basically Map = List<Struct<"key":K, "value":V>>:

/// `MapArray` is physically just a `List<Struct<key: K, value: V>>` array, but with some additional restrictions
/// on the `key`.
#[derive(Debug, Clone, Eq)]
pub struct MapArray {
    pub(super) inner: ListArray,
}

#[derive(Clone, Eq, EstimateSize)]
pub struct MapValue(pub(crate) ListValue);

A minimal working example:

dev=> select map_from_entries(array['k1','k2'],array[1,2]);
┌──────────────────┐
│ map_from_entries │
├──────────────────┤
│ {"k1":1,"k2":2}  │
└──────────────────┘

Checklist

  • I have written necessary rustdoc comments
  • I have added necessary unit tests and integration tests
  • I have added test labels as necessary. See details.
  • I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features Sqlsmith: Sql feature generation #7934).
  • My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
  • All checks passed in ./risedev check (or alias, ./risedev c)
  • My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)
  • My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details)

Documentation

  • My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note

If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.

Copy link
Member Author

xxchan commented Jul 16, 2024

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @xxchan and the rest of your teammates on Graphite Graphite

@xxchan xxchan changed the base branch from main to xxchan/a July 16, 2024 04:03
@xxchan xxchan force-pushed the xxchan/map branch 2 times, most recently from c00c228 to 39a86ad Compare July 16, 2024 06:36
Base automatically changed from xxchan/a to main July 16, 2024 07:27
@xxchan xxchan changed the base branch from main to 07-16-refactor_types_doc_refine_tobinary_text July 16, 2024 09:01
@xxchan xxchan force-pushed the 07-16-refactor_types_doc_refine_tobinary_text branch from a31225a to 45df1da Compare July 16, 2024 20:44
Base automatically changed from 07-16-refactor_types_doc_refine_tobinary_text to main July 16, 2024 21:11
@xxchan xxchan changed the title feat: map type feat: map type [WIP] Jul 16, 2024
@xxchan xxchan force-pushed the xxchan/map branch 2 times, most recently from 42b515a to 6fd081f Compare July 17, 2024 01:09
@xxchan xxchan force-pushed the xxchan/map branch 2 times, most recently from 185dac7 to dcb565e Compare July 22, 2024 00:59
@xxchan xxchan changed the base branch from main to 07-20-refactor_types_simplify_arrayimpl_from_protobuf July 22, 2024 00:59
Base automatically changed from 07-20-refactor_types_simplify_arrayimpl_from_protobuf to main July 22, 2024 07:06
@xxchan xxchan mentioned this pull request Jul 24, 2024
19 tasks
@xxchan xxchan changed the title feat: map type [WIP] feat: add map type to scalar/array/expr Jul 24, 2024
@xxchan xxchan marked this pull request as ready for review July 24, 2024 02:32
Copy link
Contributor

@wangrunji0408 wangrunji0408 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM!

src/common/src/types/map_type.rs Show resolved Hide resolved
src/frontend/src/binder/expr/function.rs Outdated Show resolved Hide resolved
src/expr/impl/src/scalar/array.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@xiangjinwu xiangjinwu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(posting some incomplete review comments first)

Comment on lines 926 to 928
// XXX: can we unify this with FUNC_SIG_MAP?
// For raw_call here, it seems unnecessary to declare it again here.
// For some functions, we have validation logic here. Is it still useful now?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Short answer: we could rely more on FUNC_SIG_MAP when no rewrite needed but it wont unify all special rewrites. Actually PostgreSQL does some rewrites in the sqlparser yacc/bison file, which also feels pretty dirty.
https://github.com/postgres/postgres/blob/REL_16_3/src/backend/parser/gram.y#L14527-L14535

Copy link
Member

@BugenZhao BugenZhao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

L🥰TM

src/common/src/types/mod.rs Outdated Show resolved Hide resolved
@xxchan xxchan enabled auto-merge August 12, 2024 08:51
@xxchan xxchan added this pull request to the merge queue Aug 12, 2024
Merged via the queue into main with commit b2b5a33 Aug 12, 2024
32 of 33 checks passed
@xxchan xxchan deleted the xxchan/map branch August 12, 2024 09:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants