UTF-8 unusable on windows with 1.14.4 - 1.14.5

[hmaarrfk]

Solution to issue cannot be found in the documentation.

• I checked the documentation.

Issue

See HDFGroup/hdf5#5037 for more details but basically:

1. A patch in 1.14.4 broke UTF-8 filename support on windows.
2. This is blocking h5py from being compiled on windows.

I personally think UTF-8 is pretty important. I've seen users assume they can put any character they want (commas, quotes, colons, you name it.....) in their file names.

However, today HDF5 is splitting the conda ecosystem making it more difficult to co-install packages.

I think there are a few options for conda-forge:

1. Downgrade to 1.14.3
2. Dual build: 1.14.3 + 1.14.4 (heavy maintenance downstream)
3. Ship 1.14.4 and say it is a known issue (h5py's pypi package might do this)
4. Revert the offending patch on our version 1.14.4 Address code page issues w/ Windows file paths HDFGroup/hdf5#4172

I'm not sure what other maintainers have in mind, but it would be good to get to a decision and to unblock this by the end of the week.

Installed packages

hdf5 1.14.4 and 1.14.5

Environment info

Windows.

[hmaarrfk]

I would like to move forward with Option 2:

• Dual build 1.14.3 and 1.14.4/5/6

while this gets resolved upstream.

This isn't too different than what we've done in the past with HDF5 updates. HDF5 updates often take time, and the libraries that consume them can be quite old (due to the strong history of HDF5) and makes it difficult for packages to migrate.

I feel like 1.14.4/5 is actually quite disruptive to the ecosystem as a whole, while the UTF-8 issue isn't resolved.

Thoughts are welcome.

I've rebuilt 1.14.3 with the new features that have been contributed.

If an other maintainer agrees I have a checklist of the next steps outlined in #241 Perhaps the person who agrees can start by creating the new branch.

[hmaarrfk]

Ok i've waited two weeks with little response so I'm going to make use of my merge powers to push the idea of dual builds through until somebody else can propose a better alternative.

However, given the importance of HDF5 beyond just the conda(-forge) ecosystem, I think we should avoid patching this bug out.

[hmaarrfk]

@conda-forge-admin please rerender

so as to add a sacred branch

[conda-forge-webservices]

Hi! This is the friendly automated conda-forge-webservice.

I just wanted to let you know that I started rerendering the recipe in #242.

[minrk]

Following from conda-forge/conda-forge-pinning-feedstock#6900

As a maintainer of several of the heavy matrix feedstocks, the root of most of them is petsc, which I merged yesterday, launching the several really heavy ones (which add a Python matrix on top of the mpi + petsc scalar). I'm happy to close those PRs and revert the petsc PR, if that's what we should do. But it's not quite clear to me what action to take, other than don't merge the PRs the closed migrator already issued.

A bit tangential, but I'm also totally open to input to trim the matrix on several of the heavy petsc dependents, but I don't know how to make those decisions. ppc is by far the most costly platform to support (both in terms of compute and maintainer time), and I am not aware of a single user. It appears to be ~entirely unused in this stack of packages, except for similarly unused ppc builds of downstream packages, at least from anaconda.org download counts. I don't know who the conda-forge ppc builds are for, so I don't know if/when it's appropriate to drop ppc support.

[h-vetinari]

I'm happy to close those PRs and revert the petsc PR, if that's what we should do. But it's not quite clear to me what action to take, other than don't merge the PRs the closed migrator already issued.

You don't need to revert PRs, just ensure that the next respective PR rerenders the feedstock correctly (which will remove half the matrix; the ones for 1.14.4).

ppc is by far the most costly platform to support (both in terms of compute and maintainer time), and I am not aware of a single user.

You're free to drop it from your feedstocks if it causes maintenance issues, but for the larger picture, this is a discussion for https://github.com/conda-forge/conda-forge.github.io I think. I agree that whenever I looked at download counts, PPC is at least an order of magnitude behind the second-least-used platform, so I can see where you're coming from.

example

openssl 3.4.0

artefact	uptime	downloads
linux-64/openssl-3.4.0-hb9d3cd8_0.conda	1 month and 26 days ago	4'039'394
linux-aarch64/openssl-3.4.0-h86ecc28_0.conda	1 month and 26 days ago	134'917
linux-ppc64le/openssl-3.4.0-h190368a_0.conda	1 month and 26 days ago	10'885
osx-64/openssl-3.4.0-hd471939_0.conda	1 month and 26 days ago	186'830
osx-arm64/openssl-3.4.0-h39f12f2_0.conda	1 month and 26 days ago	215'090
win-64/openssl-3.4.0-h2466b09_0.conda	1 month and 26 days ago	507'144

OTOH, 10k users isn't nothing either 🤷