Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the choice to dictionary-encode data results explicit #7405

Merged
merged 4 commits into from
Sep 18, 2024

Conversation

jleibs
Copy link
Member

@jleibs jleibs commented Sep 11, 2024

What

  • When requesting a Column explicitly, you can now use the new ColumnDescriptor::DictionaryEncoded(descriptor) to indicate that you want a dictionary-encoded view of a column.
  • Components that are neither part of the chunk, nor dictionary encoded, result in row-slices instead.
  • If there are no row-slices, we preserve the old full-chunk based join mechanism, giving large record-batches.
  • If there are row-slices, you get single rows.

Future work:

Checklist

  • I have read and agree to Contributor Guide and the Code of Conduct
  • I've included a screenshot or gif (if applicable)
  • I have tested the web demo (if applicable):
  • The PR title and labels are set such as to maximize their usefulness for the next release's CHANGELOG
  • If applicable, add a new check to the release checklist!
  • If have noted any breaking changes to the log API in CHANGELOG.md and the migration guide

To run all checks from main, comment on the PR with @rerun-bot full-check.

@jleibs jleibs added ⛃ re_datastore affects the datastore itself exclude from changelog PRs with this won't show up in CHANGELOG.md labels Sep 11, 2024
@jleibs jleibs marked this pull request as ready for review September 11, 2024 23:59
@jleibs jleibs force-pushed the jleibs/slice_based_queries branch from 27036c6 to abbe65b Compare September 18, 2024 15:54
@jleibs jleibs force-pushed the jleibs/slice_based_queries branch from abbe65b to 85c100c Compare September 18, 2024 16:38
@jleibs jleibs merged commit 7449e38 into main Sep 18, 2024
33 checks passed
@jleibs jleibs deleted the jleibs/slice_based_queries branch September 18, 2024 19:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
exclude from changelog PRs with this won't show up in CHANGELOG.md ⛃ re_datastore affects the datastore itself
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants