Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync with zarr 3.0 rc-2 #544

Merged
merged 22 commits into from
Jan 7, 2025
Merged

Sync with zarr 3.0 rc-2 #544

merged 22 commits into from
Jan 7, 2025

Conversation

mpiannucci
Copy link
Contributor

@dcherian not sure if you already started on some of the xarray parts

Closes #541

@mpiannucci
Copy link
Contributor Author

I have no idea how this figures:

UserWarning: The codec `vlen-utf8` is currently not part in the Zarr version 3 specification. It may not be supported by other zarr imple...

@mpiannucci
Copy link
Contributor Author

mpiannucci commented Jan 6, 2025

Failures, that im currently not sure how to handle

FAILED tests/test_dask.py::test_distributed - TypeError: Expected an integer or an iterable of integers. Got None instead.
FAILED tests/test_dask.py::test_threaded - TypeError: Expected an integer or an iterable of integers. Got None instead.
FAILED tests/test_stateful_repo_ops.py::VersionControlTest::runTest - icechunk.IcechunkError: session error: cannot commit, no changes made to the session
FAILED tests/test_xarray.py::test_xarray_to_icechunk - TypeError: Expected an integer or an iterable of integers. Got None instead.
FAILED tests/test_zarr/test_stateful.py::test_zarr_hierarchy - hypothesis.errors.FlakyFailure: Inconsistent results: An example failed on the first run but now succeeds (or fails with another error, o...

@dcherian
Copy link
Contributor

dcherian commented Jan 6, 2025

TypeError: Expected an integer or an iterable of integers. Got None instead.

This is some Xarray-Zarr incompatibility with the latest Zarr candidate

@mpiannucci
Copy link
Contributor Author

mpiannucci commented Jan 6, 2025

Yeah think it possibly the change from chunk_shape to chunks in create_array

@mpiannucci
Copy link
Contributor Author

@dcherian
Copy link
Contributor

dcherian commented Jan 6, 2025

Is that relevant to the tuple thing too? Xarray doesn't use chunk_shape AFAICT

@mpiannucci
Copy link
Contributor Author

hmmm i have no idea

@dcherian
Copy link
Contributor

dcherian commented Jan 7, 2025

pydata/xarray#9927 is in so we can proceed here against Xarray main.

@mpiannucci
Copy link
Contributor Author

I ran this against xarray main branch and now just get these warnings that are reported as errors:

FAILED tests/test_dask.py::test_distributed - UserWarning: The codec `vlen-utf8` is currently not part in the Zarr version 3 specification. It may not be supported by other zarr implementations and may change in the future.
FAILED tests/test_dask.py::test_threaded - UserWarning: The codec `vlen-utf8` is currently not part in the Zarr version 3 specification. It may not be supported by other zarr implementations and may change in the future.
FAILED tests/test_xarray.py::test_xarray_to_icechunk - UserWarning: The codec `vlen-utf8` is currently not part in the Zarr version 3 specification. It may not be supported by other zarr implementations and may change in the future.

@dcherian
Copy link
Contributor

dcherian commented Jan 7, 2025

Then we need to copy https://github.com/pydata/xarray/blob/1e5045a8c56a93d83d9347abcc020be2bb3cf622/pyproject.toml#L339-L340

@mpiannucci
Copy link
Contributor Author

For some reason, that filter isnt working what the heck

@mpiannucci
Copy link
Contributor Author

Whyyyy isnt filterwarnings working what the heck

@dcherian dcherian mentioned this pull request Jan 7, 2025
@mpiannucci
Copy link
Contributor Author

The distributed failure is back! WTH

@mpiannucci mpiannucci marked this pull request as ready for review January 7, 2025 19:46
@mpiannucci mpiannucci changed the title Sync with zarr 3.0 rc-1 Sync with zarr 3.0 rc-2 Jan 7, 2025
@mpiannucci mpiannucci merged commit 480108f into main Jan 7, 2025
3 checks passed
@mpiannucci mpiannucci deleted the zarr-rc1 branch January 7, 2025 19:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Sync with zarr 3.0.0-rc1
3 participants