Skip to content

Commit

Permalink
feat: chunk specification in uproot.dask (#898)
Browse files Browse the repository at this point in the history
* delayed-open example of specifying chunks of files

* fix typo in uproot.dask docstring

* no need to change pyproject.toml since awkward is in dev

* Add both open_files states to the test.

* uproot._util.regularize_files is internal; I prefer non-default arguments in internal functions.

* Nontrivial number of divisions.

* true_val check hadn't been checking Dask arrays **with chunks**.

* Protect against multiple methods of specifying chunks/partitions/steps.

* Made it work for library='np'.

* Got the open_files=False, library="np" case really, actually working.

* Got the open_files=True, library="np" case working.

* Remove unnecessary nested function.

* Got the open_files=True, library="ak" case working.

* Assigning divisions in the open_files=True, library="ak" case.

* Assigning divisions in the open_files=False, library="ak" case.

* Explicit steps in 'files' argument now errors, rather than clips, when out of range.

* Make the test cases asymmetric.

* Rename 'chunk' -> 'step' everywhere OUTSIDE of uproot.dask.

* Toughened up 'regularize_steps'. Now I'm done.

* require that steps is a two-tuple, use `.tolist()` to regularize steps

* Move `out.shape[1] == 2` test earlier.

---------

Co-authored-by: Jim Pivarski <[email protected]>
Co-authored-by: Jim Pivarski <[email protected]>
  • Loading branch information
3 people authored Jun 9, 2023
1 parent 2d24309 commit 04f4199
Show file tree
Hide file tree
Showing 5 changed files with 332 additions and 70 deletions.
Loading

0 comments on commit 04f4199

Please sign in to comment.