Helping Python developers understand the structure and 'cloud-friendliness' of an HDF5 file. To best keep on top of updates, star/watch this repository.
A weekend project inspired by the h5cloud project at the 2023 ICESat-2 Hackweek.
Don't want to deal with code? Try out the web app!
To get started with H5XRay, you can easily install it using pip.
pip install git+https://github.com/jonm3d/h5xray.gitpip install --upgrade git+https://github.com/jonm3d/h5xray.gitH5xray provides visualizations and reports of an HDF5 file structure. See the examples directory for more detailed usage, inluding reading local files or from S3.
Report for data/atl03_4.h5:
- Elapsed time (s): 1.373
- Total datasets: 1020
- Total requests: 1276.0
- Request byte size: 2097152 bytes
- Assumed cost per 1000 GET requests: $0.0004
- Total cost for file: $0.0005104
Top 5 datasets with most requests:
- /gt3r/heights/lat_ph - 14.0 requests | Chunking: (10000,) | Number of Chunks: [616.0]
- /gt3r/heights/lon_ph - 14.0 requests | Chunking: (10000,) | Number of Chunks: [616.0]
- /gt1r/heights/lat_ph - 13.0 requests | Chunking: (10000,) | Number of Chunks: [583.0]
- /gt1r/heights/lon_ph - 13.0 requests | Chunking: (10000,) | Number of Chunks: [583.0]
- /gt1r/heights/h_ph - 11.0 requests | Chunking: (10000,) | Number of Chunks: [583.0]
System Info:
- OS: posix
- Platform: Linux
- Platform Version: #1 SMP Tue Feb 14 21:50:23 UTC 2023
- Python Version: 3.10.12
- Machine: x86_64
- Processor: x86_64
- Current Working Directory: /home/jovyan/h5xray
- Host Name: jupyter-jonm3d
- Number of CPUs: 4
The package also includes an interactive plot for inspecting HDF5 files in a jupyter notebook.
Made with ❤️ and ☕️ by:
Jonathan Markel
PhD Student
3D Geospatial Laboratory
The University of Texas at Austin
[email protected]
Twitter | GitHub | Website | GoogleScholar | LinkedIn
This work was supported by NASA FINESST Award 80NSSC23K0272.