Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(mito): Implement operations like concat and sort for Batch #2203

Merged
merged 12 commits into from
Aug 22, 2023

Conversation

evenyag
Copy link
Contributor

@evenyag evenyag commented Aug 18, 2023

I hereby agree to the terms of the GreptimeDB CLA

What's changed and what's your intention?

This PR implements some useful operations for Batch

  • concat(), concat multiple batches into one
  • sort_and_dedup(), sort and dedup the Batch in place
  • slice(), slice the Batch
  • filter_deleted(), remove rows with OpType::Delete

Checklist

  • I have written the necessary rustdoc comments.
  • I have added the necessary unit tests and integration tests.

Refer to a related PR or issue link (optional)

@evenyag evenyag force-pushed the feat/mito2-batch-op branch from 438002e to 719a98f Compare August 18, 2023 06:47
@evenyag evenyag marked this pull request as ready for review August 19, 2023 04:58
@evenyag evenyag requested review from v0y4g3r and waynexia August 19, 2023 04:58
@codecov
Copy link

codecov bot commented Aug 19, 2023

Codecov Report

Merging #2203 (e0fb2cf) into develop (be1e13c) will increase coverage by 0.01%.
Report is 1 commits behind head on develop.
The diff coverage is 98.25%.

@@             Coverage Diff             @@
##           develop    #2203      +/-   ##
===========================================
+ Coverage    84.64%   84.65%   +0.01%     
===========================================
  Files          702      703       +1     
  Lines       113824   114394     +570     
===========================================
+ Hits         96341    96839     +498     
- Misses       17483    17555      +72     

@evenyag evenyag mentioned this pull request Aug 21, 2023
2 tasks
src/datatypes/src/vectors/primitive.rs Show resolved Hide resolved
src/mito2/src/read.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@v0y4g3r v0y4g3r left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@evenyag evenyag requested a review from waynexia August 22, 2023 07:35
@evenyag evenyag self-assigned this Aug 22, 2023
src/mito2/src/read.rs Outdated Show resolved Hide resolved
Don't return Option in get_timestamp()/get_sequence()
@evenyag evenyag requested a review from waynexia August 22, 2023 11:42
@waynexia waynexia added this pull request to the merge queue Aug 22, 2023
Merged via the queue into GreptimeTeam:develop with commit cc3e198 Aug 22, 2023
paomian pushed a commit to paomian/greptimedb that referenced this pull request Oct 19, 2023
…timeTeam#2203)

* feat: Implement slice and first/last timestamp for Batch

* feat(mito): implements sort/concat for Batch

* chore: fix typo

* chore: remove comments

* feat: sort and dedup

* test: test batch operations

* chore: cast enum to test op type

* test: test filter related api

* sytle: fix clippy

* docs: comment for slice

* chore: address CR comment

Don't return Option in get_timestamp()/get_sequence()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants