Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Executor for polars logical plans #15504

Merged
merged 58 commits into from
May 30, 2024

Conversation

wence-
Copy link
Contributor

@wence- wence- commented Apr 10, 2024

Description

This builds out the infrastructure for executing polars logical plans using pylibcudf. See docs/overview.md in the cudf_polars subdirectory for some installation guidance.

Deliberately not fully fleshing out packaging and so forth yet.

Test coverage is incomplete but growing. I'd like to get this in so other people can build on top of it.

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@github-actions github-actions bot added the Python Affects Python cuDF API. label Apr 10, 2024
@wence- wence- force-pushed the wence/fea/cudf-polars branch from 0f00a4c to b31b7b0 Compare April 11, 2024 09:45
@wence- wence- added cudf.polars Issues specific to cudf.polars and removed Python Affects Python cuDF API. labels Apr 16, 2024
@wence- wence- force-pushed the wence/fea/cudf-polars branch from b31b7b0 to b10c74d Compare April 16, 2024 16:08
@github-actions github-actions bot added the Python Affects Python cuDF API. label Apr 16, 2024
@wence- wence- force-pushed the wence/fea/cudf-polars branch 3 times, most recently from c0336b3 to d86957c Compare April 22, 2024 13:24
@rapidsai rapidsai deleted a comment from copy-pr-bot bot Apr 22, 2024
@wence-
Copy link
Contributor Author

wence- commented Apr 22, 2024

/ok to test

@wence- wence- added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change and removed Python Affects Python cuDF API. labels Apr 22, 2024
@github-actions github-actions bot added the Python Affects Python cuDF API. label Apr 23, 2024
@wence- wence- force-pushed the wence/fea/cudf-polars branch from 11170d7 to b066d41 Compare April 23, 2024 12:52
@wence- wence- force-pushed the wence/fea/cudf-polars branch from 7be8f78 to 76b644c Compare April 30, 2024 14:43
@wence- wence- changed the title WIP: executor for polars logical plans Executor for polars logical plans Apr 30, 2024
@wence- wence- marked this pull request as ready for review April 30, 2024 16:21
@wence- wence- requested a review from a team as a code owner April 30, 2024 16:21
@wence- wence- requested review from isVoid, brandon-b-miller, quasiben and vyasr and removed request for isVoid April 30, 2024 16:21
@wence- wence- force-pushed the wence/fea/cudf-polars branch from 78929a1 to 7c2e8e6 Compare May 1, 2024 10:10
@wence- wence- marked this pull request as draft May 13, 2024 14:53
@vyasr
Copy link
Contributor

vyasr commented May 21, 2024

@wence- once you merge in the latest 24.06 I'm happy to take a pass at reviewing. It's fine if you want to keep in draft and you don't think it's quite ready for full review yet, but I figured I'd take a pass once you're reasonably happy so that we can iron out as many issues as possible immediately.

@wence- wence- changed the base branch from branch-24.06 to branch-24.08 May 21, 2024 15:44
@wence- wence- force-pushed the wence/fea/cudf-polars branch from 184ca04 to 0f82d0f Compare May 23, 2024 15:57
Copy link
Contributor

@vyasr vyasr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Took a pass through the tests. I don't have much to say on them content-wise, they all look fine to me and it seems like you have good coverage. Organization-wise, though, I'm confused at what went into test_basic.py and what got its own module. Can we establish some sort of organizational principle from the get-go?

@wence-
Copy link
Contributor Author

wence- commented May 24, 2024

Took a pass through the tests. I don't have much to say on them content-wise, they all look fine to me and it seems like you have good coverage. Organization-wise, though, I'm confused at what went into test_basic.py and what got its own module. Can we establish some sort of organizational principle from the get-go?

No good reason for this split (other than test_basic was written earlier), will split them out appropriately.

Copy link
Contributor

@vyasr vyasr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few small comments, but I think this is in more than good enough shape to merge and iterate.

python/cudf_polars/cudf_polars/containers/column.py Outdated Show resolved Hide resolved
python/cudf_polars/cudf_polars/containers/column.py Outdated Show resolved Hide resolved
python/cudf_polars/cudf_polars/containers/dataframe.py Outdated Show resolved Hide resolved
python/cudf_polars/cudf_polars/dsl/expr.py Show resolved Hide resolved
python/cudf_polars/cudf_polars/dsl/expr.py Show resolved Hide resolved
@wence- wence- force-pushed the wence/fea/cudf-polars branch from de62efe to 1240b62 Compare May 30, 2024 11:50
@wence-
Copy link
Contributor Author

wence- commented May 30, 2024

/merge

@rapids-bot rapids-bot bot merged commit e95894f into rapidsai:branch-24.08 May 30, 2024
70 checks passed
@wence- wence- deleted the wence/fea/cudf-polars branch May 30, 2024 14:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cudf.polars Issues specific to cudf.polars improvement Improvement / enhancement to an existing function non-breaking Non-breaking change Python Affects Python cuDF API.
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

3 participants