Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(common): introduce JsonbArrayBuilder and JsonbArray #7986

Merged
merged 4 commits into from
Feb 17, 2023

Conversation

xiangjinwu
Copy link
Contributor

@xiangjinwu xiangjinwu commented Feb 16, 2023

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

Part of the jsonb type support (preview all on this branch):

In this PR:

  • The in-memory layout of JsonbArray is simply a Vec or roots of json trees. This is easier to operate but do have space for optimization.
  • The protobuf layout is the same as bytea variable length array, with each element storing its value encoding. In case we switch to a newer protobuf layout, a new ArrayType enum value can be introduced without affecting parts other than to_protobuf and from_protobuf.
  • Refactored VarSizedValueReader from returning RefItem into accepting &mut ArrayBuilder. This is to use JsonbArrayBuilder::append_move on the deserialized OwnedItem. It is impossible to get a JsonbArray::RefItem from &[u8].
  • Blanket implementation for arrow / HashKeySerDe / RandValue.

Checklist

  • I have written necessary rustdoc comments
    - [ ] I have added necessary unit tests and integration tests (tested together in later part)
  • All checks passed in ./risedev check (or alias, ./risedev c)

Documentation

  • My PR DOES NOT contain user-facing changes (yet).

@codecov
Copy link

codecov bot commented Feb 16, 2023

Codecov Report

Merging #7986 (9b64a68) into main (cdb5d9b) will decrease coverage by 0.03%.
The diff coverage is 32.41%.

@@            Coverage Diff             @@
##             main    #7986      +/-   ##
==========================================
- Coverage   71.41%   71.38%   -0.03%     
==========================================
  Files        1119     1119              
  Lines      180622   180760     +138     
==========================================
+ Hits       128984   129033      +49     
- Misses      51638    51727      +89     
Flag Coverage Δ
rust 71.38% <32.41%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/common/src/array/arrow.rs 63.88% <0.00%> (-0.14%) ⬇️
src/common/src/hash/key.rs 83.24% <0.00%> (-0.90%) ⬇️
src/common/src/array/jsonb_array.rs 19.89% <30.35%> (+19.89%) ⬆️
src/common/src/array/value_reader.rs 52.00% <35.71%> (-16.75%) ⬇️
src/common/src/array/mod.rs 73.06% <50.00%> (-0.73%) ⬇️
src/common/src/array/column_proto_readers.rs 80.80% <100.00%> (-0.16%) ⬇️
src/common/src/test_utils/rand_array.rs 100.00% <100.00%> (ø)
src/batch/src/task/channel.rs 61.53% <0.00%> (-3.85%) ⬇️
src/object_store/src/object/mem.rs 86.48% <0.00%> (-0.78%) ⬇️
src/storage/src/hummock/sstable_store.rs 65.00% <0.00%> (-0.32%) ⬇️
... and 7 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Member

@BugenZhao BugenZhao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -117,6 +117,12 @@ impl RandValue for bool {
}
}

impl RandValue for JsonbVal {
fn rand_value<R: rand::Rng>(_rand: &mut R) -> Self {
JsonbVal::dummy()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this by design? What about adding a TODO here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

List / Struct do not have this either. Opened #8001 for it.

@mergify mergify bot merged commit 7f9588a into main Feb 17, 2023
@mergify mergify bot deleted the jsonb-r3-p1-array branch February 17, 2023 07:03
mergify bot pushed a commit that referenced this pull request Feb 17, 2023
Part of the `jsonb` type support:
* #7977
* #7986
* Introduce `DataType::Jsonb` **(this PR)**
* Add more expressions #7714

In this PR:
* Add `DataType::Jsonb`.
* Support constructing it from string and displaying it as string.
* Add e2e tests for `insert` and `select`.

Also tested for the following but not added in CI:
* prepared statement with BINARY format
* kafka+json source with a `jsonb` field

Approved-By: BugenZhao
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants