Skip to content

Test-suite improvements: pytest 9.x feature adoption + redundancy cleanup #195

Description

@d-v-b

Test-suite improvement audit

A review of the tests/ suite (~30k LOC) following the move to pytest 9.x, focused on (a) adopting newer/idiomatic pytest features and (b) reducing redundancy. The suite is already fairly modern (uses RaisesGroup, getfixturevalue, caplog/monkeypatch, no nose-style/yield_fixture). Findings below are grouped into three actionable sections plus a "leave as-is" list. Each section is being addressed by a linked draft PR.

Effort = S/M/L, Risk = Low/Med/High.

Section A — Correctness / bugs

  • Downstream marker registration is misplaced (real bug). src/zarr/testing/conftest.py defines pytest_configure, but the pytest11 plugin entry point is zarr.testing (src/zarr/testing/__init__.py), which has no such hook. The stated goal — letting downstream projects that use StoreTests run under --strict-markers — is not met. Move the hook into __init__.py; drop the now-empty conftest and its coverage-omit line. (S / Low)
  • Bare pytest.raises(...) without match= on broad builtins (~46 of ~151 are broad ValueError/TypeError/KeyError on multi-line bodies, so they can pass on the wrong error path). Hotspots: test_api.py (test_save_errors), test_group.py, test_store/test_core.py. Add match=. (M / Low)
  • hasattr-guarded assertions that silently no-op in tests/test_experimental/test_cache_store.py (real TTL assertion vanishes if _is_key_fresh is renamed). (S / Low)
  • xfail/skip hygiene in tests/test_indexing.py: xfails with no reason=; an unconditional skip for an env-specific failure that should be skipif. (S / Low)

Section B — pytest-feature modernization

  • tmpdir/LEGACY_PATHtmp_path (~22 sites, mostly tests/conftest.py); removes the private from _pytest.compat import LEGACY_PATH import and fixes an inverted annotation. (M / Low)
  • Drop redundant markers now that asyncio_mode = "auto" is set: remove the ~10 @pytest.mark.asyncio decorators and the asyncio marker registration; remove the unused s3 marker. (M / Low)
  • pytest.param(..., id=...) for unstable ids in test_api/test_synchronous.py::test_docstring_consistent_parameters (currently produces <function ... at 0x...> ids). (S / Low)
  • monkeypatch.setenv over hand-rolled env save/restore in tests/test_docs.py. (S / Low)

Section C — Redundancy / simplification

  • Centralize store parametrization. @pytest.mark.parametrize("store", [...]) appears ~110× with recurring literals and zero shared constants. Add ALL_STORES/LOCAL_MEMORY/MEMORY to conftest; remove the test_group.py shadowing store fixture (the pattern that collided with pytest 9.1.0); normalize indirect=True. (M / Low–Med)
  • Drop ~30 redundant @pytest.mark.parametrize("zarr_format", [2, 3]) decorators — a zarr_format fixture already exists in conftest. (S / Low)
  • Collapse identical codec teststest_*_codec_sync_roundtrip/supports_sync are byte-identical across test_codecs/test_{crc32c,gzip,zstd,blosc,endian}.py. (S / Low)
  • Shared array-construction helpers/fixtures (repeated np.arange(...).reshape(...) + zarr_array_from_numpy_array, the uint16 16×16 array copy-pasted across 7 files). (M / Low)

Validated — leave as-is (so we don't churn)

  • tests/test_dtype/conftest.py pytest_generate_tests legitimately enables subclass parameter override.
  • filterwarnings ignores are minimal; the s3fs/aiobotocore one genuinely can't be a per-test marker (pytest#14096).
  • pytest 9.1 --doctest-modules double-autouse (#14533) is not triggered — no module-scoped inline autouse fixtures.
  • The ~69 sync() bridges are mostly intentional sync-API tests.

Audit performed with assistance from Claude Code.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions