🛠️ [TASK] : Make HDF5 able to read data directly from IPFS rather than an on-disk copy of the Pinned HDF5 App package.

[stevenj]

Summary

Enable HDF5 to access IPFS content directly.

Description

We aim to have applications distributed over IPFS.
This will mean the app packages in-use by a Hermes node will be pinned and shared by that node.
However, currently IPFS needs a plain copy of the file, and a pinned IPFS file is not plain.
It can be read from, however.

So this task is to modify the core HDF5 C++ library to use rust provided file system functions so that we can direct read operations from within IPFS library directly to pinned Apps.

Local file storage will still be on-disk, so these functions need to be smart about how they direct the traffic, IPFS for app packages, and disk for the Root filesystem HDF5 file, etc.

See: https://github.com/HDFGroup/hdf5/blob/a8dcba252e4f58c85c681864f48371073e44d558/src/H5private.h#L728C9-L728C16
Which shows HDF5 library file system abstraction by way of #defined filesystem functions.

The work here would be to create #defines to cause the HDF5 library to link to internal functions that implement these functions in the way we require.