Fix HDF5 2.0 compatibility and gate DAOS-specific gaps on connector name#67
Merged
brtnfld merged 3 commits intoJul 2, 2026
Merged
Conversation
…st-effort cleanup errors - h5vl_test_driver.cxx: don't treat NOTICE-level server log lines as fatal test failures. The driver's OutputStringHasError() was matching on any line containing "FAILED"-adjacent substrings without regard to log severity, so benign DAOS NOTICE-level messages (e.g. pool-svc lock retry notices under load) were being treated as fatal test errors. - vol_test_util.c: suppress the error stack print in remove_test_file()'s best-effort H5Fdelete() call, since it may target a file that was never created and the resulting benign error was cluttering test output.
Several sub-tests in this suite exercise DAOS VOL connector features that are either unimplemented or have confirmed, tracked correctness gaps. Following the project's existing convention (already used in test_dataset_set_extent_invalid_params and vol_link_test.c for other DAOS-specific quirks), gate these on `H5VLget_connector_name() + strcmp(vol_name, "daos") == 0` rather than skipping unconditionally, so the original test logic still runs for any other VOL connector exercising this shared suite. Restores each skip site's original (pre-skip) logic in an else branch. Covers: - 56 decreasing-order (H5_INDEX_NAME + H5_ITER_DEC) iteration parts across vol_group_test.c, vol_link_test.c, vol_attribute_test.c, and vol_object_test.c - not implemented by the connector. - 7 multi-dataset I/O functions, 2 set_extent shrink/expand data-correctness parts, and the double-handle extent-cache-coherency test in vol_dataset_test.c. - The two H5Fget_obj_count(H5F_OBJ_ALL, ...) parts in vol_file_test.c that count the connector's internally-cached datatype hid_t's as if they were user-visible open objects. - test_resurrect_datatype in vol_datatype_test.c (H5VL_OBJECT_GET_NAME unconditionally unsupported). - The H5Iget_name()-dependent portions of test_create_hard_link_many and test_create_soft_link_many, and the H5Ovisit_by_name-on-an-attribute part in vol_object_test.c (same H5VL_OBJECT_GET_NAME gap, reached via a different code path). - The group max_corder-resets-to-0-after-emptying assertions in test_delete_link_reset_grp_max_crt_order and test_move_link_reset_grp_max_crt_order (max_corder is a monotonic counter in the connector with no reset-on-empty path). - The named-datatype reference-count assertions in test_attr_shared_dtype (the connector never increments a referenced committed datatype's on-disk refcount for attributes/datasets, unlike its hard-link target handling). All are known, confirmed, tracked connector-side gaps rather than test bugs; each site documents the specific root cause inline.
HDF5 2.0 (tracked by the develop branch) changes several API defaults and behaviors this suite relied on: - hdf5_test/tid.c: the bare H5Iregister_type() macro no longer defaults to the 3-argument H5Iregister_type1(hash_size, reserved, free_func); it now maps to the 2-argument H5Iregister_type2(reserved, free_func) instead (which doesn't exist as an explicit name pre-2.0). Add a H5_VERSION_GE-gated wrapper macro so this vendored copy of upstream HDF5's test/tid.c keeps building against both old and new HDF5, unlike upstream's own test suite (which only targets its own current version). - vol_test.c / vol_test_parallel.c: the startup check that verifies the default FAPL's VOL connector matches the one requested on the command line compared H5Pget_vol_id() and H5VLget_connector_id_by_name() results by raw ID equality. As of HDF5 2.0, H5VLget_connector_id_by_name() always registers a fresh ID wrapping the underlying connector object (see H5VLget_connector_id_by_name() in HDF5's src/H5VL.c), rather than incrementing the refcount on an existing, already-registered ID the way it did pre-2.0 - so two different, valid IDs can now legitimately refer to the same connector. This caused every run against HDF5 develop to fail immediately with "VOL connector set on default FAPL didn't match specified VOL connector", before any actual sub-tests could run. H5VLcmp_connector_cls() compares the underlying connector classes directly instead of the IDs, and works correctly on both HDF5 versions.
brtnfld
added a commit
to brtnfld/vol-daos
that referenced
this pull request
Jul 2, 2026
Lift the temporary --tests-regex restriction now that the DAOS engine-join issues are resolved and h5vl_test passes cleanly - run the complete ctest suite (h5daos_test_map, h5daos_test_recovery, parallel tests, etc.), not just h5vl_test. Also bump the test/vol submodule pointer to the vol-tests fork's newly squashed history (see HDFGroup/vol-tests#67) after force-pushing that branch's cleaned-up commits.
brtnfld
added a commit
to brtnfld/vol-daos
that referenced
this pull request
Jul 2, 2026
… test suite) Add a daos-test job to CI that installs DAOS 2.6.5, builds HDF5 from source (matrixed over hdf5_1.14.6 and develop, the latter marked continue-on-error since it tracks a moving upstream target), builds the connector against it, and runs the full ctest suite (h5vl_test plus HDF5's own native/parallel test binaries ported to run against a VOL connector, plus the connector's own h5daos_test_* targets) against a live, ephemeral, single-node DAOS instance. Key pieces: - Rocky Linux 8 container running --privileged (required to disable transparent hugepages, which DAOS's SPDK housekeeping needs even with a pure-RAM storage tier). - test/scripts/daos_server.yml.in: RAM-emulated SCM storage tier sized and configured for small/CI hosts (system_ram_reserved lowered, scm_hugepages_disabled, nr_hugepages fixed). - test/scripts/daos_pool.sh.in: rewritten for the modern dmg CLI syntax, polling for engine join and pool queryability before signaling readiness to the test driver, to fix races where the harness would start the client before the pool was actually usable. - Excludes 3 known, tracked failures (h5_test_testhdf5, h5_partest_t_bigio, h5vl_ext_h5daos_test_metadata_parallel) so CI stays green on everything that's actually been verified so far - see the PR description for root-cause details on each. - .gitmodules: test/vol temporarily points at the brtnfld/vol-tests fork branch pending HDFGroup/vol-tests#67 merging; revert to HDFGroup/vol-tests once that lands.
brtnfld
added a commit
to brtnfld/vol-daos
that referenced
this pull request
Jul 2, 2026
Points at the merged HDFGroup/vol-tests#67, which fixes test-driver robustness against benign DAOS log noise, gates known DAOS-specific test gaps on connector name rather than skipping unconditionally, and fixes HDF5 2.0 compatibility issues in the vendored test suite. .gitmodules is reverted to the upstream vol-tests URL now that the fix has landed there.
brtnfld
added a commit
to brtnfld/vol-daos
that referenced
this pull request
Jul 2, 2026
Points at the merged HDFGroup/vol-tests#67, which fixes test-driver robustness against benign DAOS log noise, gates known DAOS-specific test gaps on connector name rather than skipping unconditionally, and fixes HDF5 2.0 compatibility issues in the vendored test suite; and HDFGroup#68, which fixes an unrelated pre-existing CI workflow typo (ctest --build isn't a valid flag) discovered while investigating that suite's own CI status. .gitmodules is reverted to the upstream vol-tests URL now that both fixes have landed there.
brtnfld
added a commit
to brtnfld/vol-daos
that referenced
this pull request
Jul 7, 2026
Points at the merged HDFGroup/vol-tests#67, which fixes test-driver robustness against benign DAOS log noise, gates known DAOS-specific test gaps on connector name rather than skipping unconditionally, and fixes HDF5 2.0 compatibility issues in the vendored test suite; and HDFGroup#68, which fixes an unrelated pre-existing CI workflow typo (ctest --build isn't a valid flag) discovered while investigating that suite's own CI status. .gitmodules is reverted to the upstream vol-tests URL now that both fixes have landed there.
brtnfld
added a commit
to brtnfld/vol-daos
that referenced
this pull request
Jul 7, 2026
Points at the merged HDFGroup/vol-tests#67, which fixes test-driver robustness against benign DAOS log noise, gates known DAOS-specific test gaps on connector name rather than skipping unconditionally, and fixes HDF5 2.0 compatibility issues in the vendored test suite; and HDFGroup#68, which fixes an unrelated pre-existing CI workflow typo (ctest --build isn't a valid flag) discovered while investigating that suite's own CI status. .gitmodules is reverted to the upstream vol-tests URL now that both fixes have landed there.
brtnfld
added a commit
to brtnfld/vol-daos
that referenced
this pull request
Jul 7, 2026
Points at the merged HDFGroup/vol-tests#67, which fixes test-driver robustness against benign DAOS log noise, gates known DAOS-specific test gaps on connector name rather than skipping unconditionally, and fixes HDF5 2.0 compatibility issues in the vendored test suite; and HDFGroup#68, which fixes an unrelated pre-existing CI workflow typo (ctest --build isn't a valid flag) discovered while investigating that suite's own CI status. .gitmodules is reverted to the upstream vol-tests URL now that both fixes have landed there.
brtnfld
added a commit
to brtnfld/vol-daos
that referenced
this pull request
Jul 7, 2026
Points at the merged HDFGroup/vol-tests#67, which fixes test-driver robustness against benign DAOS log noise, gates known DAOS-specific test gaps on connector name rather than skipping unconditionally, and fixes HDF5 2.0 compatibility issues in the vendored test suite; and HDFGroup#68, which fixes an unrelated pre-existing CI workflow typo (ctest --build isn't a valid flag) discovered while investigating that suite's own CI status. .gitmodules is reverted to the upstream vol-tests URL now that both fixes have landed there.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Found and fixed while standing up real DAOS-backed CI for the HDFGroup/vol-daos connector (installing DAOS, building HDF5 1.14.6 and
develop, and running this suite against it end-to-end for the first time).h5vl_test_driver.cxxwas treating benign DAOSNOTICE-level server log lines as fatal test failures;vol_test_util.c's best-effort cleanup was printing a spurious error stack.H5VLget_connector_name() + strcmp(vol_name, "daos") == 0, following this suite's own existing convention (already used intest_dataset_set_extent_invalid_paramsand elsewhere) - the original test logic still runs for any other VOL connector exercising this shared suite.developbranch (tracking an upcoming 2.0 release) - a removed default API mapping forH5Iregister_type()in the vendoredhdf5_test/tid.c, and a VOL-connector-ID identity check invol_test.c/vol_test_parallel.cthat's no longer valid under HDF5 2.0's connector-ID semantics.Each commit has a detailed explanation; see individual commit messages for full root-cause writeups.