Example HDF4 files for developing readers and parsers
Small HDF4 files for testing HDF4 readers: 86 files, 64 MB. Each file has a
SHA-256 checksum, its provenance, tags describing how it is stored, and an answer key
(<file>.expected.json) recording what the HDF4 C library reports for it: every scientific
dataset's type, shape, dimensions, attributes, a hash of its decoded values and the byte
ranges holding its stored bytes, plus Vdata tables, Vgroups and HDF-EOS2 structure.
The tools that build this catalogue, the schemas and the issue tracker are at https://github.com/virtual-zarr/hdf4-examples.
The synthetic files, every answer key and all catalogue metadata are generated by this project and dedicated to the public domain under CC0 1.0. Files from GDAL, OPeNDAP and NASA keep their own terms; describing a file here does not change them. They are set out in THIRD-PARTY.md.
Each collection's license links point at the terms quoted here. NASA items carry their
collection's terms exactly as NASA records them, in the cmr:terms property, together
with the dataset DOI and the CMR record ids. A few NASA collections record no terms in
CMR; for those cmr:terms is empty and only NASA's general data-use guidance applies.
A version folder is frozen once published. Corrections and additions go in a new version.
Everything is readable without credentials:
https://data.source.coop/pangeo/example-hdf4/v1/catalog.jsonhttps://data.source.coop/pangeo/example-hdf4/v1/items.jsonpangeo, prefix example-hdf4/v1/, endpoint
https://data.source.coop, unsigned requestsitems.json is the simplest entry point. Each row gives an item's collection and id,
paths (relative to v1/) to the data file and its answer key, the file's size and
SHA-256, and its tags. Select by tag, download, verify the checksum, read the file with
your HDF4 reader, and compare against the answer key.
chunk_multi_range, chunked, compressed, contiguous, linked_scattered, linked_single, unwrittendeflate, none, rle, skphuffannotation, eos_grid, eos_swath, raster_image, sd, vdataThe tags are defined in the hdf4 STAC extension schema and the answer key in the answer-key schema.
By the HDF4 C library, through pyhdf and the library's own data-map function
(SDgetdatainfo) for byte ranges. Where OPeNDAP publishes a DMR++ description of a file,
its byte ranges were compared with ours and agree. Two parts are derived by this
project's tools instead of the library, and say so in a derived_by field: the HDF-EOS2
structure (parsed from the StructMetadata text) and the descriptor counts.
opendap| 13 |
| 0.1 MB |
| HDF4 files from OPeNDAP's DMR++ tests |
| From the OPeNDAP BES repository, whose COPYING file is the GNU LGPL version 3 |
synthetic | 22 | 0.2 MB | Synthetic HDF4 files, one storage feature each | CC0-1.0, public domain dedication |