Test-suite improvement audit
A review of the tests/ suite (~30k LOC) following the move to pytest 9.x, focused on (a) adopting newer/idiomatic pytest features and (b) reducing redundancy. The suite is already fairly modern (uses RaisesGroup, getfixturevalue, caplog/monkeypatch, no nose-style/yield_fixture). Findings below are grouped into three actionable sections plus a "leave as-is" list. Each section is being addressed by a linked draft PR.
Effort = S/M/L, Risk = Low/Med/High.
Section A — Correctness / bugs
- Downstream marker registration is misplaced (real bug).
src/zarr/testing/conftest.py defines pytest_configure, but the pytest11 plugin entry point is zarr.testing (src/zarr/testing/__init__.py), which has no such hook. The stated goal — letting downstream projects that use StoreTests run under --strict-markers — is not met. Move the hook into __init__.py; drop the now-empty conftest and its coverage-omit line. (S / Low)
- Bare
pytest.raises(...) without match= on broad builtins (~46 of ~151 are broad ValueError/TypeError/KeyError on multi-line bodies, so they can pass on the wrong error path). Hotspots: test_api.py (test_save_errors), test_group.py, test_store/test_core.py. Add match=. (M / Low)
hasattr-guarded assertions that silently no-op in tests/test_experimental/test_cache_store.py (real TTL assertion vanishes if _is_key_fresh is renamed). (S / Low)
xfail/skip hygiene in tests/test_indexing.py: xfails with no reason=; an unconditional skip for an env-specific failure that should be skipif. (S / Low)
Section B — pytest-feature modernization
tmpdir/LEGACY_PATH → tmp_path (~22 sites, mostly tests/conftest.py); removes the private from _pytest.compat import LEGACY_PATH import and fixes an inverted annotation. (M / Low)
- Drop redundant markers now that
asyncio_mode = "auto" is set: remove the ~10 @pytest.mark.asyncio decorators and the asyncio marker registration; remove the unused s3 marker. (M / Low)
pytest.param(..., id=...) for unstable ids in test_api/test_synchronous.py::test_docstring_consistent_parameters (currently produces <function ... at 0x...> ids). (S / Low)
monkeypatch.setenv over hand-rolled env save/restore in tests/test_docs.py. (S / Low)
Section C — Redundancy / simplification
- Centralize store parametrization.
@pytest.mark.parametrize("store", [...]) appears ~110× with recurring literals and zero shared constants. Add ALL_STORES/LOCAL_MEMORY/MEMORY to conftest; remove the test_group.py shadowing store fixture (the pattern that collided with pytest 9.1.0); normalize indirect=True. (M / Low–Med)
- Drop ~30 redundant
@pytest.mark.parametrize("zarr_format", [2, 3]) decorators — a zarr_format fixture already exists in conftest. (S / Low)
- Collapse identical codec tests —
test_*_codec_sync_roundtrip/supports_sync are byte-identical across test_codecs/test_{crc32c,gzip,zstd,blosc,endian}.py. (S / Low)
- Shared array-construction helpers/fixtures (repeated
np.arange(...).reshape(...) + zarr_array_from_numpy_array, the uint16 16×16 array copy-pasted across 7 files). (M / Low)
Validated — leave as-is (so we don't churn)
tests/test_dtype/conftest.py pytest_generate_tests legitimately enables subclass parameter override.
filterwarnings ignores are minimal; the s3fs/aiobotocore one genuinely can't be a per-test marker (pytest#14096).
- pytest 9.1
--doctest-modules double-autouse (#14533) is not triggered — no module-scoped inline autouse fixtures.
- The ~69
sync() bridges are mostly intentional sync-API tests.
Audit performed with assistance from Claude Code.
Test-suite improvement audit
A review of the
tests/suite (~30k LOC) following the move to pytest 9.x, focused on (a) adopting newer/idiomatic pytest features and (b) reducing redundancy. The suite is already fairly modern (usesRaisesGroup,getfixturevalue,caplog/monkeypatch, no nose-style/yield_fixture). Findings below are grouped into three actionable sections plus a "leave as-is" list. Each section is being addressed by a linked draft PR.Effort = S/M/L, Risk = Low/Med/High.
Section A — Correctness / bugs
src/zarr/testing/conftest.pydefinespytest_configure, but thepytest11plugin entry point iszarr.testing(src/zarr/testing/__init__.py), which has no such hook. The stated goal — letting downstream projects that useStoreTestsrun under--strict-markers— is not met. Move the hook into__init__.py; drop the now-empty conftest and its coverage-omit line. (S / Low)pytest.raises(...)withoutmatch=on broad builtins (~46 of ~151 are broadValueError/TypeError/KeyErroron multi-line bodies, so they can pass on the wrong error path). Hotspots:test_api.py(test_save_errors),test_group.py,test_store/test_core.py. Addmatch=. (M / Low)hasattr-guarded assertions that silently no-op intests/test_experimental/test_cache_store.py(real TTL assertion vanishes if_is_key_freshis renamed). (S / Low)xfail/skiphygiene intests/test_indexing.py: xfails with noreason=; an unconditionalskipfor an env-specific failure that should beskipif. (S / Low)Section B — pytest-feature modernization
tmpdir/LEGACY_PATH→tmp_path(~22 sites, mostlytests/conftest.py); removes the privatefrom _pytest.compat import LEGACY_PATHimport and fixes an inverted annotation. (M / Low)asyncio_mode = "auto"is set: remove the ~10@pytest.mark.asynciodecorators and theasynciomarker registration; remove the unuseds3marker. (M / Low)pytest.param(..., id=...)for unstable ids intest_api/test_synchronous.py::test_docstring_consistent_parameters(currently produces<function ... at 0x...>ids). (S / Low)monkeypatch.setenvover hand-rolled env save/restore intests/test_docs.py. (S / Low)Section C — Redundancy / simplification
@pytest.mark.parametrize("store", [...])appears ~110× with recurring literals and zero shared constants. AddALL_STORES/LOCAL_MEMORY/MEMORYto conftest; remove thetest_group.pyshadowingstorefixture (the pattern that collided with pytest 9.1.0); normalizeindirect=True. (M / Low–Med)@pytest.mark.parametrize("zarr_format", [2, 3])decorators — azarr_formatfixture already exists in conftest. (S / Low)test_*_codec_sync_roundtrip/supports_syncare byte-identical acrosstest_codecs/test_{crc32c,gzip,zstd,blosc,endian}.py. (S / Low)np.arange(...).reshape(...)+zarr_array_from_numpy_array, theuint1616×16 array copy-pasted across 7 files). (M / Low)Validated — leave as-is (so we don't churn)
tests/test_dtype/conftest.pypytest_generate_testslegitimately enables subclass parameter override.filterwarningsignores are minimal; the s3fs/aiobotocore one genuinely can't be a per-test marker (pytest#14096).--doctest-modulesdouble-autouse (#14533) is not triggered — no module-scoped inline autouse fixtures.sync()bridges are mostly intentional sync-API tests.Audit performed with assistance from Claude Code.