Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Followup issue for new features to PackedSelection #848

Closed
3 tasks
lgray opened this issue Jun 27, 2023 · 6 comments
Closed
3 tasks

Followup issue for new features to PackedSelection #848

lgray opened this issue Jun 27, 2023 · 6 comments
Labels
enhancement New feature or request

Comments

@lgray
Copy link
Collaborator

lgray commented Jun 27, 2023

Things to finish later for #797

  • understand why compatible_parititions doesn't work through slices
  • double check for reducing the total number of calls to .compute()
  • ensure no unnecessary eager len() calls on dak.Arrays
@lgray lgray added the enhancement New feature or request label Jun 27, 2023
@ikrommyd
Copy link
Collaborator

I would also add "do another check for unnecessary len calls on dask_awkward collections".

@lgray
Copy link
Collaborator Author

lgray commented Jun 27, 2023

I gave it a once-over before I merged it but sure.

@ikrommyd
Copy link
Collaborator

@lgray In #895, I've partially adressed the 2nd point, meaning I've reduced the number of compute calls in PackedSelection by 2 but there are compute calls that can be reduced in tests to make them run quicker.

I've also checked again for unnecessary eager len() and didn't find any. I do see a couple in Weights but I don't know if those are supposed to only hit eager arrays. If those are fine you can mark the 3rd point as done.

I'm planning to fully address the first 2 points in a separate PR. If tests are good, I'm fine with merging #895

@lgray
Copy link
Collaborator Author

lgray commented Sep 16, 2023

We may wanna think about that a bit more. I'm still somewhat against calling computes for people unless it's really necessary, it introduce unexpected slowness.

Is there any way we can reorganize things to return futures to the user for them to compute?

@ikrommyd
Copy link
Collaborator

The only computes are while saving arrays (where the user has the option to call compute later) and while printing statistics of the cutflow and N-1 selections. I guess we can also add the option to print later, or warn that this will compute the arrays on the spot. For saving, I just made the default to be compute=False.

@lgray
Copy link
Collaborator Author

lgray commented Dec 7, 2023

I think this is done now. Reopen if not.

@lgray lgray closed this as completed Dec 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants