Can't access values of Measurement as property when dask>=2024.11.0

[emmaai]

Expected behaviour

Measurement values can be access as property e.g., by measurement.dtype.
The symptom is that somehow (and somewhere) Measurement is casted back to parent class dict instead of datacube.model.Measurement. Logging parameter type(measurement) in fuse_lazy shows the different classes information between dask==2024.10.0 and dask==2024.11.0

Actual behaviour

Emit error when compute

Exception: 'AttributeError("\'dict\' object has no attribute \'nodata\'")'
Traceback: '  File "/env/lib/python3.10/site-packages/datacube/api/core.py", line 946, in fuse_lazy\n    print(f\'measurement nodata {measurement.nodata}\')\n'

Steps to reproduce the behaviour

Loading multiple scenes with groupby="solar_day"

Environment information

• Which datacube --version are you using?
  1.8.19
• What datacube deployment/enviornment are you running against?
  sandbox

Note: Stale issues will be automatically closed after a period of six months with no activity.
To ensure critical issues are not closed, tag them with the Github pinned tag.
If you are a community member and not a maintainer please escalate this issue to maintainers via
GIS StackExchange or Discord.

[emmaai]

tag the issue for tracking dask/distributed#8932

[omad]

Thanks Emma. Love the reproducible example you posted in the dask issue!

[omad]

And then I promptly forgot, and spent far too much time today wondering why datacube-core storage test cases were failing for me! 😢

[SpacemanPaul]

It's particularly bad with numpy2.

2024.11.1 works with numpy1 - but not numpy2, and 2024.11.2 breaks both again.

[SpacemanPaul]

There are a bunch of other more or less subtle upstream library issues with numpy2.

I just got develop-1.9 CI running with numpy 2 in #1661 - required pinning the MOST recent versions of shapely, netCDF4, and cftime; installing netcdf4 with --no-binary; and pinning fairly specific recent versions of dask and pandas.

Dask seems to be the only one affecting numpy 1 though.