Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Expose Ballista Scheduler and Executor in Python #1148

Merged
merged 10 commits into from
Dec 9, 2024
Merged

feat: Expose Ballista Scheduler and Executor in Python #1148

merged 10 commits into from
Dec 9, 2024

Conversation

milenkovicm
Copy link
Contributor

@milenkovicm milenkovicm commented Dec 7, 2024 •
edited
Loading

Which issue does this PR close?

Closes #1107.

Rationale for this change

What changes are included in this PR?

  • added basic classes for scheduler & executor
  • added draft logical and physical codec with cloud pickle serialisation for udfs (not wired)
  • builder update to use ballista configuration
  • provided examples

limitations:

  • not all configuration options are supported, will be added at the later date

Are there any user-facing changes?

No

Note

@milenkovicm milenkovicm marked this pull request as draft December 7, 2024 21:19
@github-actions github-actions bot added the python label Dec 7, 2024
@milenkovicm milenkovicm marked this pull request as ready for review December 8, 2024 11:46
andygrove
andygrove approved these changes Dec 9, 2024
View reviewed changes
Copy link
Member

@andygrove andygrove left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice. Thanks @milenkovicm

@andygrove andygrove merged commit b3cf8d1 into apache:main Dec 9, 2024
24 checks passed
@milenkovicm milenkovicm deleted the feat_py_bin branch December 9, 2024 18:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andygrove andygrove andygrove approved these changes

Assignees
No one assigned
Labels
python
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Expose Ballista Scheduler and Executor in Python
2 participants
@milenkovicm @andygrove