Skip to content

fix, nested std::array reads fail against flat multi-dim H5T_ARRAY datasets #279

@steven-varga

Description

@steven-varga

Symptom

Reading an HDF5 dataset whose on-disk datatype is a multi-dimensional H5T_ARRAY (e.g. H5T_ARRAY { [3][5] H5T_STD_I64LE }) into the canonical-mapped C++ type fails with HDF5 type-conversion errors:

H5T__path_find_real(): no appropriate function for conversion path
unable to convert between src and dest datatype

Reproducer

Against the HDFGroup canonical h5ex_t_array.h5 (publicly hosted at https://rhdf5-public.s3.eu-central-1.amazonaws.com/h5ex_t_array.h5):

auto ds = h5::open(fd, "/DS1");
using row_t = std::array<std::array<int64_t, 5>, 3>;
auto data = h5::read<std::vector<row_t>>(ds);  // throws

On-disk type per h5dump:

DATATYPE  H5T_ARRAY { [3][5] H5T_STD_I64LE }
DATASPACE  SIMPLE { ( 4 ) / ( 4 ) }

Workaround used in examples/s3/s3.cpp: drop to raw H5Dread with H5Dget_type(ds) and a flat std::vector<int64_t>.

Hypothesis

The Winston canonical mapping spec at h5cpp/H5Tmeta.hpp:371 produces H5T_ARRAY{[3] H5T_ARRAY{[5] i64}} (nested) for std::array<std::array<i64,5>,3>, but the file stores H5T_ARRAY{[3][5] i64} (flat multi-dim). These have identical memory layouts but HDF5's H5T_path_find treats them as distinct types and refuses conversion.

Suggested fix direction

Collapse contiguous nested std::array<std::array<T,M>,N> chains into a single multi-dim H5T_ARRAY call: H5Tarray_create2(dt_t<T>, 2, {N,M}) instead of nested 1-D H5Tarray_create2 calls. This matches what h5ex_t_array.c (HDFGroup canonical example) emits and what any third-party writer using the documented multi-dim form will produce.

Out of scope

  • ROS3-specific concerns — the bug reproduces on any local file with multi-dim H5T_ARRAY; S3 is just the convenient public reproducer.
  • Forward-compat writes (already correct? worth verifying separately).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions