Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Supports Range-partitioning #575

Open
richox opened this issue Sep 18, 2024 · 0 comments
Open

Supports Range-partitioning #575

richox opened this issue Sep 18, 2024 · 0 comments
Labels
feature required Functionalities must have

Comments

@richox
Copy link
Collaborator

richox commented Sep 18, 2024

Is your feature request related to a problem? Please describe.
blaze has no range-partitioning at this moment. any shuffle exchanges with range partitioning will fallback to spark.

Describe the solution you'd like
implement a RangeRepartitioner in blaze, which does the same thing with spark:

  1. first run to collect data distribution histogram and compute ranges.
  2. second run to sort and split data into ranges.

Describe alternatives you've considered

Additional context

@richox richox pinned this issue Sep 18, 2024
@richox richox added the feature required Functionalities must have label Sep 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature required Functionalities must have
Projects
None yet
Development

No branches or pull requests

1 participant