From be22268148652dca5b4e1e0dbc49a0a6770088b7 Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 22 May 2026 17:55:17 +0200 Subject: [PATCH 01/27] docs: design spec for indexing test cleanup Plan to consolidate and speed up tests/test_indexing.py: shrink oversized arrays (>=3 chunks/axis, partial edge), replace np.random selection loops with hand-picked parametrized cases via the Expect/ExpectFail dataclasses, one-behavior-per-test isolation, and docstrings throughout. Includes a prerequisite step deduplicating the two divergent Expect dataclass pairs (tests/conftest.py vs tests/test_codecs/conftest.py) onto one canonical pair. Co-Authored-By: Claude Opus 4.7 (1M context) --- ...2026-05-22-indexing-test-cleanup-design.md | 220 ++++++++++++++++++ 1 file changed, 220 insertions(+) create mode 100644 docs/superpowers/specs/2026-05-22-indexing-test-cleanup-design.md diff --git a/docs/superpowers/specs/2026-05-22-indexing-test-cleanup-design.md b/docs/superpowers/specs/2026-05-22-indexing-test-cleanup-design.md new file mode 100644 index 0000000000..5ffdd31978 --- /dev/null +++ b/docs/superpowers/specs/2026-05-22-indexing-test-cleanup-design.md @@ -0,0 +1,220 @@ +# Indexing test suite cleanup — design + +Date: 2026-05-22 +Branch: `indexing-test-cleanup` +Target file: `tests/test_indexing.py` (2177 lines, 50 test functions) + +## Goals + +Three goals, in priority order as confirmed with the user: + +1. **Per-test isolation** — each test verifies one behavior. A failure should + point at a named case, not a 20-iteration loop inside a helper. +2. **Faster** — shrink the oversized arrays and cut redundant loop iterations. + Speed comes purely from smaller work per test (see "Speed model" below), + not from any matrix change. +3. **Clarity / consolidation** — remove duplicate coverage, replace loop + + helper-function patterns with `@pytest.mark.parametrize`, give every test a + docstring stating the behavior it verifies. + +Scope: **whole file**. Already-parametrized tests get swept for consistency +(docstrings, shared helpers, array sizes) alongside the loop-based ones. + +## Background / current state + +The file is a mix of two eras: + +- **Ported-from-v2 tests** (the slow ones): build a large array, set + `np.random.seed(42)`, then loop `for p in 0.5, 0.1, 0.01` generating random + boolean/integer index arrays at several sparsity levels, and call a private + helper (`_test_get_orthogonal_selection_3d`, etc.) that itself loops over + 7–20 hardcoded selection tuples. One test function exercises dozens of + selections with bundled assertions. + - Example arrays: `np.arange(32400).reshape(120, 30, 9)` (3d orthogonal), + `np.arange(5400).reshape(600, 9)` (2d), `np.arange(1050)` (1d bool). +- **Modern tests**: `@pytest.mark.parametrize` with explicit selection lists, + e.g. `test_indexing_equals_numpy`, `test_orthogonal_bool_indexing_like_numpy_ix`. + +### Key facts that constrain the design + +- `tests/test_indexing.py` defines its **own** `store` fixture (line 42–44): + `StorePath(await MemoryStore.open())`. It does **not** use the conftest + matrix-parametrized `store` fixture. Therefore every test runs **once** — + there is no store/format/codec amplification for this file. +- Default `addopts` (pyproject.toml:428) does not include `-n auto`; xdist is + available but opt-in. Tests run single-process by default. +- Local full-file runtime: ~3.5 s in pytest, ~4 s wall. Slowest tests are + exactly the loop-heavy orthogonal ones (`test_set_orthogonal_selection_3d` + 0.92 s, `test_get_orthogonal_selection_1d_bool` 0.79 s, + `test_set_orthogonal_selection_2d` 0.54 s, `test_set_orthogonal_selection_1d` + 0.41 s). + +### Speed model + +Runtime per test ≈ array construction + encode (`z[()] = a`) + per-selection +decode/encode. The large arrays dominate. Halving each dimension of +`(120, 30, 9)` and keeping a chunk grid of (≈3 chunks per axis with a partial +edge chunk) cuts the encoded volume by ~8× while preserving every structural +property the tests rely on (multi-chunk spans, partial edge chunks, +cross-chunk selections, negative/wraparound indices). + +## Approach (selected: A — surgical consolidation, applied file-wide) + +For each test family: + +1. **Shrink arrays** under a hard rule (confirmed with user): every indexed + axis spans **≥3 chunks**, and at least one axis has a **partial (non-full) + edge chunk**. Smallest shapes meeting that: + - 1d: `arange(30)`, chunks `(7,)` → 5 chunks, last (size 2) partial. + - 2d: `arange(60).reshape(12, 5)`, chunks `(5, 2)` → 3×3 chunks; axis-0 + edge (size 2) and axis-1 edge (size 1) partial. + - 3d: `arange(420).reshape(7, 6, 10)`, chunks `(3, 2, 4)` → 3×3×3 chunks; + partial edges on every axis. + Exact numbers may shift slightly during implementation so the hand-picked + selections stay in-bounds and meaningful, but the ≥3-chunks / partial-edge + rule is fixed. + +2. **Replace `np.random` selections with hand-picked deterministic ones** + (user decision). For each axis kind we keep explicit cases that name what + they cover: + - boolean mask: empty (all False), full (all True), alternating, single-True, + a sparse hand-chosen mask. + - integer array: sorted, unsorted, with-duplicates, with-wraparound + (negative), single-element. + The 0.5 / 0.1 / 0.01 sparsity loop collapses into the "alternating" and + "sparse" cases — the density sweep was fuzzing, not targeted coverage, and + the user de-prioritized RNG breadth. + +3. **Convert loop-over-selections helpers to `@pytest.mark.parametrize`.** + The private `_test_*` helpers that loop over selection lists become a + `selection` parameter on the public test. The shared + `_test_get_orthogonal_selection` / `_test_set_orthogonal_selection` + oracle (compare zarr result to numpy via `oindex`/`oindex_set`) is **kept** + as a small assertion helper — it is the right abstraction, just called once + per parametrized case instead of in a loop. + +4. **Docstrings** on every test stating the behavior verified (per project + convention / memory `feedback_test_docstrings`). + +5. **Preserve error-path tests as their own named tests** (per memory + `feedback_test_structure`: one test per failure mode). The `IndexError` + blocks currently bundled at the end of the get/set tests (too-short mask, + too-long mask, too-many-dims, out-of-bounds) become a `test_*_raises` + parametrized over `ExpectFail` cases, using `pytest.raises(case.exception, + match=case.msg)`. + +## Prerequisite: deduplicate the two `Expect` dataclass pairs + +The repo currently has **two divergent** test-case dataclass pairs. The new +indexing tests should use one canonical pair, so this PR unifies them first. + +| | `tests/conftest.py` (canonical) | `tests/test_codecs/conftest.py` (to delete) | +|---|---|---| +| success | `Expect[TIn, TOut]`: `input`, `output`, `id`; not frozen | `Expect[TIn, TOut]`: `input`, `expected`; frozen | +| failure | `ExpectFail[TIn]`: `input`, `exception`, `id`, `msg` | `ExpectErr[TIn]`: `input`, `msg`, `exception_cls` | + +**Decision:** keep the `tests/conftest.py` pair (`Expect` + `ExpectFail`) as +the single source of truth, because it carries `id` — the id lives with the +case, cannot drift out of sync with a separate `ids=[...]` list, and survives +reordering. Add `frozen=True` to both (the codecs version's one good idea; +these are value objects). Delete the definitions in +`tests/test_codecs/conftest.py` (that file contains only these two classes). + +**Migration of the three codecs/chunk-grid consumers** +(`tests/test_chunk_grids.py`, `tests/test_codecs/test_cast_value.py`, +`tests/test_codecs/test_scale_offset.py`): + +1. Change imports to `from tests.conftest import Expect, ExpectFail`. +2. Rename `expected=` → `output=` at every `Expect(...)` call site. +3. Replace `ExpectErr` → `ExpectFail`, renaming `exception_cls=` → `exception=`. +4. Add `id=` to every case, sourced from the existing `ids=[...]` list that the + parametrize call passes positionally, then replace `ids=[...]` with + `ids=lambda c: c.id`. Where a parametrize block has no `ids=` list today, + synthesize concise ids. +5. Run the affected suites green before touching indexing tests. + +This is a mechanical but real change (~40 call sites). It is a true +prerequisite: doing it first means the indexing rewrite imports the final, +stable `Expect`/`ExpectFail` and we never write against a soon-to-change shape. + +### Shared helpers introduced + +- **Use the conftest `Expect[TIn, TOut]` / `ExpectFail[TIn]` dataclasses** as + the selection-table mechanism, matching the established idiom in + `tests/test_metadata/test_v3.py`: + - Selection cases: `Expect[Selection, None]` where `input` is the selection + and `id` names the case (e.g. `"alternating-mask"`, `"wraparound-int"`). + `output` is unused for the oracle-style tests (the numpy result is computed, + not hardcoded) and set to `None`; the oracle helper does the comparison. + - Error cases: `ExpectFail[Selection]` carrying `input` (the bad selection), + `exception` (`IndexError`), `id`, and `msg` (regex for `pytest.raises`). + - Parametrize with `ids=lambda c: c.id` so `pytest -k ` selects a + single named case — this is the readable-id payoff that plain tuple + parametrize lacks. + - Module-level case lists (`_ORTHO_1D_CASES`, `_ORTHO_BAD_1D_CASES`, etc.) + are shared across get/set tests of the same dimensionality so 1d/2d/3d + don't each redefine "alternating mask". + - Field is `Expect.output` (the canonical `tests/conftest.py` name). The + `expected=` spelling came from the now-deleted codecs copy (see the dedup + section above). +- Keep `zarr_array_from_numpy_array`; it is already the right builder. +- Keep the `_test_get/set_orthogonal_selection` zarr-vs-numpy oracle helpers; + they consume one `Expect.input` per parametrized case instead of looping. + +### Things explicitly NOT changed + +- The local `store` fixture stays MemoryStore-only. Widening it to the conftest + matrix would multiply runtime against goal 2; out of scope. +- `xfail`/`skip` markers (structured-field tests, repeated-index test) stay as-is. +- Regression tests tied to specific GH issues (`test_set_item_1d_last_two_chunks`, + `test_indexing_with_zarr_array`, `test_vectorized_indexing_incompatible_shape`, + `test_zero_sized_chunks`) keep their concrete reproducing values — shrinking + them would weaken the regression. Only docstrings added. +- `CountingDict` / `test_accessed_chunks` logic (verifies which chunks are + touched) keeps shapes that produce a meaningful access pattern. + +## Rewrite inventory + +Loop/helper-based (primary rewrite targets): +`test_get_basic_selection_1d`, `test_get_basic_selection_2d`, +`test_get_orthogonal_selection_1d_bool`, `test_get_orthogonal_selection_1d_int`, +`test_get_orthogonal_selection_2d`, `test_get_orthogonal_selection_3d`, +`test_set_orthogonal_selection_1d/2d/3d`, +`test_get_coordinate_selection_1d/2d`, `test_set_coordinate_selection_1d/2d`, +`test_get_block_selection_1d/2d`, `test_set_block_selection_1d/2d`, +`test_get_mask_selection_1d/2d`, `test_set_mask_selection_1d/2d`, +`test_get_selection_out`. + +Already parametrized (sweep for consistency + array size only): +`test_get_basic_selection_0d`, `test_set_basic_selection_0d`, +`test_indexing_equals_numpy`, `test_orthogonal_bool_indexing_like_numpy_ix`, +the `*_fallback_*` family, `test_iter_grid`, `test_iter_regions`. + +Leave essentially as-is (docstring only): +the GH-regression tests and `xfail`/`skip` tests listed above, +`test_normalize_integer_selection`, `test_replace_ellipsis`, +`test_iter_grid_invalid`, `test_iter_chunk_regions`, `TestAsync`. + +## Testing / verification + +- After the `Expect` dedup (step 0): the migrated suites must stay green — + `uv run --frozen pytest tests/test_chunk_grids.py tests/test_codecs/test_cast_value.py tests/test_codecs/test_scale_offset.py tests/test_metadata/test_v3.py -q`. +- After each indexing family rewrite: + `uv run --frozen pytest tests/test_indexing.py -q` must stay green (same + pass/skip/xfail counts modulo intentional restructure). +- Coverage sanity: the rewritten cases must still exercise, for each selection + type, at least one cross-chunk selection, one partial-edge-chunk selection, + one negative/wraparound index, and the documented error paths. +- Final: `--durations=25` before/after to record the speedup, and a mypy pass + via `prek --all-files` (per memory) since type annotations on the parametrize + lists change and the `Expect` shape moves. + +## Success criteria + +- All loop-based selection tests replaced by parametrized, docstring'd tests + with one behavior per case. +- Error paths are individually named tests. +- Full-file wall time measurably lower (target: the four >0.4 s tests each + drop below ~0.15 s; expect total well under the current ~3.5 s). +- No loss of structural coverage per the checklist above. +- `git diff` is reviewable as a per-family sequence of commits. From 6c50c9e047952018cb61a82dacbdb7860a6f82a5 Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 22 May 2026 18:06:02 +0200 Subject: [PATCH 02/27] docs: implementation plan for indexing test cleanup Task-by-task plan: Expect dataclass dedup (Part 0) then per-family rewrites of tests/test_indexing.py to parametrized Expect/ExpectFail cases on smaller arrays (Part 1), with final verification and speed measurement (Part 2). Co-Authored-By: Claude Opus 4.7 (1M context) --- .../plans/2026-05-22-indexing-test-cleanup.md | 661 ++++++++++++++++++ 1 file changed, 661 insertions(+) create mode 100644 docs/superpowers/plans/2026-05-22-indexing-test-cleanup.md diff --git a/docs/superpowers/plans/2026-05-22-indexing-test-cleanup.md b/docs/superpowers/plans/2026-05-22-indexing-test-cleanup.md new file mode 100644 index 0000000000..50b0d1adc0 --- /dev/null +++ b/docs/superpowers/plans/2026-05-22-indexing-test-cleanup.md @@ -0,0 +1,661 @@ +# Indexing Test Cleanup Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Consolidate and speed up `tests/test_indexing.py` by shrinking oversized arrays, replacing `np.random` selection loops with hand-picked parametrized cases, isolating one behavior per test, and unifying the duplicated `Expect` test-case dataclasses onto one canonical pair. + +**Architecture:** First deduplicate the two divergent `Expect`/error dataclass pairs (`tests/conftest.py` vs `tests/test_codecs/conftest.py`) onto the richer `tests/conftest.py` pair, migrating the three codec/chunk-grid consumers. Then rewrite each indexing test family: smaller arrays (≥3 chunks/axis, partial edge), explicit `Expect[Selection, None]` / `ExpectFail[Selection]` case tables parametrized with `ids=lambda c: c.id`, comparing zarr against a numpy oracle. Error paths become their own named parametrized tests. + +**Tech Stack:** Python 3.12, pytest, numpy, zarr; tests run via `uv run --frozen pytest`. Lint/type via `prek`. + +**Spec:** `docs/superpowers/specs/2026-05-22-indexing-test-cleanup-design.md` (gist: https://gist.github.com/d-v-b/508a7294cba8bc702f36a4b85f4a8a90) + +**Conventions (from project memory — apply throughout):** +- Every new/rewritten test gets a docstring stating the *behavior* it verifies. +- Compact happy-path via parametrization; each exception path gets its own named test. +- Single backticks in docstrings; no RST roles, no double-backticks. +- No falsy conditionals — test `is None` etc. explicitly. +- Run mypy via `prek --all-files`, never ad-hoc `uvx mypy`. +- Commit messages end with the `Co-Authored-By: Claude Opus 4.7 (1M context)` trailer. + +--- + +## Part 0 — Deduplicate the `Expect` dataclasses + +The canonical pair lives in `tests/conftest.py`: + +```python +@dataclass +class Expect[TIn, TOut]: + """A test case with explicit input, expected output, and a human-readable id.""" + input: TIn + output: TOut + id: str + +@dataclass +class ExpectFail[TIn]: + """A test case that should raise an exception.""" + input: TIn + exception: type[Exception] + id: str + msg: str +``` + +The duplicate to delete lives in `tests/test_codecs/conftest.py`: + +```python +@dataclass(frozen=True) +class Expect[TIn, TOut]: + input: TIn + expected: TOut + +@dataclass(frozen=True) +class ExpectErr[TIn]: + input: TIn + msg: str + exception_cls: type[Exception] +``` + +**Migration rule** (applied at every call site in the three consumer files): +- `Expect(input=X, expected=Y)` → `Expect(input=X, output=Y, id="")` +- `ExpectErr(input=X, msg=M, exception_cls=E)` → `ExpectFail(input=X, exception=E, id="", msg=M)` +- The `` comes from the existing positional `ids=[...]` list on the `@pytest.mark.parametrize` for that block (1:1 by order). After moving ids onto cases, replace `ids=[...]` with `ids=lambda c: c.id`. +- Where a parametrize block has no `ids=` today, synthesize concise kebab-case ids. +- Import line in each consumer becomes `from tests.conftest import Expect, ExpectFail`. + +Consumers (all import both classes from `tests.test_codecs.conftest`): +`tests/test_chunk_grids.py`, `tests/test_codecs/test_cast_value.py`, `tests/test_codecs/test_scale_offset.py`. + +### Task 0.1: Make the canonical `Expect`/`ExpectFail` frozen + +**Files:** +- Modify: `tests/conftest.py:67-83` + +- [ ] **Step 1: Add `frozen=True` to both canonical dataclasses** + +In `tests/conftest.py`, change the two decorators from `@dataclass` to `@dataclass(frozen=True)`: + +```python +@dataclass(frozen=True) +class Expect[TIn, TOut]: + """A test case with explicit input, expected output, and a human-readable id.""" + + input: TIn + output: TOut + id: str + + +@dataclass(frozen=True) +class ExpectFail[TIn]: + """A test case that should raise an exception.""" + + input: TIn + exception: type[Exception] + id: str + msg: str +``` + +- [ ] **Step 2: Verify the existing canonical consumer still passes** + +Run: `uv run --frozen pytest tests/test_metadata/test_v3.py -q` +Expected: PASS (no field names changed for this consumer; only `frozen` added). + +- [ ] **Step 3: Commit** + +```bash +git add tests/conftest.py +git commit -m "$(cat <<'EOF' +test: make canonical Expect/ExpectFail frozen + +Prepares for deduplicating the second Expect pair in test_codecs/conftest.py. + +Co-Authored-By: Claude Opus 4.7 (1M context) +EOF +)" +``` + +### Task 0.2: Migrate `tests/test_chunk_grids.py` to canonical pair + +**Files:** +- Modify: `tests/test_chunk_grids.py` (import line 7; all `Expect(...)`, `ExpectErr(...)`, and `ids=[...]` blocks) + +- [ ] **Step 1: Change the import** + +Replace `from tests.test_codecs.conftest import Expect, ExpectErr` with `from tests.conftest import Expect, ExpectFail`. + +- [ ] **Step 2: Migrate every `Expect(...)` and `ExpectErr(...)` call per the rule above** + +Worked example — the `test_normalize_chunks_1d_errors` block (currently lines 131-175). The existing `ids=[...]` list maps 1:1 to the cases. After migration: + +```python +@pytest.mark.parametrize( + "case", + [ + ExpectFail(input=(0, 100), exception=ValueError, id="zero-uniform", msg="Chunk size must be positive"), + ExpectFail(input=(-2, 100), exception=ValueError, id="negative-uniform", msg="Chunk size must be positive"), + ExpectFail(input=([], 100), exception=ValueError, id="empty-list", msg="must not be empty"), + ExpectFail(input=([10, -1, 10], 100), exception=ValueError, id="negative-element", msg="must be positive"), + ExpectFail(input=([10, 0, 10], 20), exception=ValueError, id="zero-element", msg="must be positive"), + ExpectFail(input=([10, 20], 100), exception=ValueError, id="wrong-sum", msg="do not sum to span"), + ExpectFail(input=([[3, 3], 1], 7), exception=TypeError, id="rle-single-dim", msg="non-integer element(s) ([3, 3],) at indices (0,)"), + ExpectFail(input=([1, [2, 2], 1, [3]], 9), exception=TypeError, id="multiple-non-ints", msg="non-integer element(s) ([2, 2], [3]) at indices (1, 3)"), + ExpectFail(input=([2, "3", 5], 10), exception=TypeError, id="string-element", msg="non-integer element(s) ('3',) at indices (1,)"), + ], + ids=lambda c: c.id, +) +def test_normalize_chunks_1d_errors(case: ExpectFail[tuple[Any, int]]) -> None: + """Invalid 1D chunk specifications are rejected with informative error messages.""" + chunks, span = case.input + with pytest.raises(case.exception, match=re.escape(case.msg)): + normalize_chunks_1d(chunks, span=span) +``` + +Apply the same transformation to the remaining parametrize blocks in this file: +the `test_normalize_chunks_nd_errors` block (ids `["none", "true", "string", "too-many-dims", "too-few-dims", "rle-inner-dim"]`), and the success-case `Expect(...)` blocks (rename `expected=`→`output=`, add `id=` from the block's `ids=[...]`, switch `ids=` to `lambda c: c.id`). Update each function's `case:` type annotation: `ExpectErr[...]` → `ExpectFail[...]`, and references `case.exception_cls` → `case.exception`. + +- [ ] **Step 3: Run the file to verify green** + +Run: `uv run --frozen pytest tests/test_chunk_grids.py -q` +Expected: PASS, same test count as before (ids change but cases don't). + +- [ ] **Step 4: Commit** + +```bash +git add tests/test_chunk_grids.py +git commit -m "$(cat <<'EOF' +test: migrate test_chunk_grids to canonical Expect/ExpectFail + +Co-Authored-By: Claude Opus 4.7 (1M context) +EOF +)" +``` + +### Task 0.3: Migrate `tests/test_codecs/test_cast_value.py` to canonical pair + +**Files:** +- Modify: `tests/test_codecs/test_cast_value.py` (import line 9; all `Expect(...)`/`ExpectErr(...)`/`ids=[...]` blocks) + +- [ ] **Step 1: Change the import** to `from tests.conftest import Expect, ExpectFail`. + +- [ ] **Step 2: Migrate every call site per the rule.** For success cases, `expected=`→`output=`, add `id=` taken 1:1 from the block's `ids=[...]` list (e.g. `["minimal", "full"]`, `["defaults", "explicit"]`, `["no-scalar-map", "with-scalar-map"]`, `["complex-source", "wrap-float-target"]`, `["f64→f32", "f32→f64", "i32→i64", "i64→i16", "f64→i32", "i32→f64"]`, `["towards-zero"]`, `["int32→int8"]`), then set `ids=lambda c: c.id`. For error blocks, `ExpectErr`→`ExpectFail`, `exception_cls=`→`exception=`, add `id=`, and update `case.exception_cls`→`case.exception` plus the `case:` type annotations. + +- [ ] **Step 3: Run the file** + +Run: `uv run --frozen pytest tests/test_codecs/test_cast_value.py -q` +Expected: PASS, same test count. + +- [ ] **Step 4: Commit** + +```bash +git add tests/test_codecs/test_cast_value.py +git commit -m "$(cat <<'EOF' +test: migrate test_cast_value to canonical Expect/ExpectFail + +Co-Authored-By: Claude Opus 4.7 (1M context) +EOF +)" +``` + +### Task 0.4: Migrate `tests/test_codecs/test_scale_offset.py` to canonical pair + +**Files:** +- Modify: `tests/test_codecs/test_scale_offset.py` (import line 9; all call sites) + +- [ ] **Step 1: Change the import** to `from tests.conftest import Expect, ExpectFail`. + +- [ ] **Step 2: Migrate every call site per the rule** (same mechanical transformation as Task 0.2/0.3). This file's success cases at lines 27-38 have no `ids=` list today — synthesize ids: `id="default"`, `id="offset-only"`, `id="scale-only"`, `id="offset-and-scale"` matching the four `Expect(...)` cases in order. Error blocks (`ExpectErr` at ~82-87 and ~262-272): convert to `ExpectFail`, add ids (`"non-numeric-offset"`, `"non-numeric-scale"` and `"unrepresentable-1"`, `"unrepresentable-2"`, `"unrepresentable-3"` respectively — pick descriptive names from each case's input). Update annotations and `.exception_cls`→`.exception`. + +- [ ] **Step 3: Run the file** + +Run: `uv run --frozen pytest tests/test_codecs/test_scale_offset.py -q` +Expected: PASS, same test count. + +- [ ] **Step 4: Commit** + +```bash +git add tests/test_codecs/test_scale_offset.py +git commit -m "$(cat <<'EOF' +test: migrate test_scale_offset to canonical Expect/ExpectFail + +Co-Authored-By: Claude Opus 4.7 (1M context) +EOF +)" +``` + +### Task 0.5: Delete the duplicate dataclasses and verify nothing imports them + +**Files:** +- Delete contents of: `tests/test_codecs/conftest.py` (file is only these two classes) + +- [ ] **Step 1: Confirm no remaining importers** + +Run: `grep -rn "from tests.test_codecs.conftest import\|test_codecs.conftest import" tests/` +Expected: no output (all three consumers migrated in 0.2–0.4). + +- [ ] **Step 2: Delete the file** + +`tests/test_codecs/conftest.py` contains only the two duplicate dataclasses, so remove it entirely: + +```bash +git rm tests/test_codecs/conftest.py +``` + +- [ ] **Step 3: Run the full codecs + chunk-grids + v3 suites** + +Run: `uv run --frozen pytest tests/test_chunk_grids.py tests/test_codecs/ tests/test_metadata/test_v3.py -q` +Expected: PASS, no collection errors. + +- [ ] **Step 4: Run prek on touched files** + +Run: `prek run --all-files` +Expected: all hooks pass (mypy in particular — the `case:` annotations now reference `ExpectFail`). + +- [ ] **Step 5: Commit** + +```bash +git add -A +git commit -m "$(cat <<'EOF' +test: delete duplicate Expect dataclasses in test_codecs/conftest.py + +All consumers now use the canonical Expect/ExpectFail from tests/conftest.py. + +Co-Authored-By: Claude Opus 4.7 (1M context) +EOF +)" +``` + +--- + +## Part 1 — Rewrite the indexing test families + +All indexing tests use the file-local `store` fixture (MemoryStore, runs once +per test) and the `zarr_array_from_numpy_array(store, a, chunk_shape=...)` +helper. Both stay. The numpy-oracle assertion helpers +(`_test_get_basic_selection`, `_test_get_orthogonal_selection`, +`_test_set_orthogonal_selection`, `_test_get_coordinate_selection`, +`_test_set_coordinate_selection`, `_test_get_block_selection`, +`_test_set_block_selection`, `_test_get_mask_selection`, +`_test_set_mask_selection`) are **kept** — they encapsulate the +zarr-vs-numpy comparison and are now called once per parametrized case. + +**Array-size rule (fixed):** every indexed axis spans ≥3 chunks; at least one +axis has a partial (non-full) edge chunk. Canonical shapes: +- 1d: `np.arange(30)`, chunks `(7,)` → 5 chunks, last size 2. +- 2d: `np.arange(60).reshape(12, 5)`, chunks `(5, 2)` → 3×3 chunks, partial edges. +- 3d: `np.arange(420).reshape(7, 6, 10)`, chunks `(3, 2, 4)` → 3×3×3 chunks, partial edges everywhere. + +Add the import for the case dataclasses at the top of `tests/test_indexing.py`: +`from tests.conftest import Expect, ExpectFail`. + +### Task 1.1 (EXEMPLAR): Rewrite `test_get_orthogonal_selection_1d_bool` and split its error paths + +This task is the worked template. Tasks 1.2+ apply the same recipe and reference it. + +**Recipe (apply to every family task):** +1. Shrink the array to the canonical shape for its dimensionality. +2. Replace the `np.random.seed` + `for p in ...` loop with an explicit + module-level `Expect[Selection, None]` list named `__CASES`, each + case carrying a hand-picked selection in `input`, `output=None`, and a + descriptive `id`. Cover: empty (all-False mask / size-0 slice), full, + alternating, single-element, sparse, and (for int arrays) sorted, unsorted, + duplicate, negative/wraparound. +3. Make the test parametrized over that list with `ids=lambda c: c.id`, calling + the kept oracle helper with `case.input`. +4. Extract the bundled `pytest.raises(IndexError)` block into a separate + `test__raises` parametrized over an `ExpectFail[Selection]` list + `__BAD_CASES` (each with `exception=IndexError`, an `id`, and a `msg` + regex — use `msg=""` to match any `IndexError` message where the original + only asserted the type). +5. Give both tests docstrings stating the behavior. + +**Files:** +- Modify: `tests/test_indexing.py` (replace `test_get_orthogonal_selection_1d_bool`, lines 615-633) + +- [ ] **Step 1: Add the case tables and rewritten tests** + +Replace the existing `test_get_orthogonal_selection_1d_bool` with: + +```python +_ORTHO_1D_BOOL_CASES: list[Expect[OrthogonalSelection, None]] = [ + Expect(input=np.zeros(30, dtype=bool), output=None, id="empty-mask"), + Expect(input=np.ones(30, dtype=bool), output=None, id="full-mask"), + Expect(input=np.arange(30) % 2 == 0, output=None, id="alternating-mask"), + Expect(input=np.eye(1, 30, 7, dtype=bool)[0], output=None, id="single-true"), + Expect( + input=np.isin(np.arange(30), [0, 1, 8, 15, 29]), + output=None, + id="sparse-cross-chunk", + ), +] + +_ORTHO_1D_BOOL_BAD_CASES: list[ExpectFail[Any]] = [ + ExpectFail(input=np.zeros(5, dtype=bool), exception=IndexError, id="mask-too-short", msg=""), + ExpectFail(input=np.zeros(50, dtype=bool), exception=IndexError, id="mask-too-long", msg=""), + ExpectFail( + input=[[True, False], [False, True]], + exception=IndexError, + id="mask-too-many-dims", + msg="", + ), +] + + +@pytest.mark.parametrize("case", _ORTHO_1D_BOOL_CASES, ids=lambda c: c.id) +def test_get_orthogonal_selection_1d_bool(store: StorePath, case: Expect[OrthogonalSelection, None]) -> None: + """oindex with a 1D boolean mask matches numpy across chunk boundaries.""" + a = np.arange(30, dtype=int) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(7,)) + _test_get_orthogonal_selection(a, z, case.input) + + +@pytest.mark.parametrize("case", _ORTHO_1D_BOOL_BAD_CASES, ids=lambda c: c.id) +def test_get_orthogonal_selection_1d_bool_raises( + store: StorePath, case: ExpectFail[Any] +) -> None: + """oindex rejects masks of the wrong length or dimensionality with IndexError.""" + a = np.arange(30, dtype=int) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(7,)) + with pytest.raises(case.exception, match=case.msg): + z.oindex[case.input] +``` + +- [ ] **Step 2: Run the rewritten tests to verify they pass** + +Run: `uv run --frozen pytest "tests/test_indexing.py::test_get_orthogonal_selection_1d_bool" "tests/test_indexing.py::test_get_orthogonal_selection_1d_bool_raises" -v` +Expected: PASS — 5 parametrized happy-path cases (by id) + 3 raises cases. + +- [ ] **Step 3: Run the whole file to confirm no breakage** + +Run: `uv run --frozen pytest tests/test_indexing.py -q` +Expected: PASS (one fewer monolithic test, several new parametrized cases). + +- [ ] **Step 4: Commit** + +```bash +git add tests/test_indexing.py +git commit -m "$(cat <<'EOF' +test: rewrite orthogonal 1d bool indexing as parametrized cases + +Smaller array (30 elems, chunks of 7), hand-picked deterministic masks +replacing np.random sparsity sweep, error paths split into their own test. + +Co-Authored-By: Claude Opus 4.7 (1M context) +EOF +)" +``` + +### Task 1.2: Rewrite `test_get_orthogonal_selection_1d_int` + +**Files:** Modify `tests/test_indexing.py` (lines 637-668). + +- [ ] **Step 1: Apply the recipe.** Array `np.arange(30)`, chunks `(7,)`. Happy-path `_ORTHO_1D_INT_CASES` covering: `id="sorted"` (`[0, 8, 15, 29]`), `id="unsorted"` (`[3, 29, 1, 16]`), `id="duplicates"` (`[2, 2, 8, 8]`), `id="wraparound"` (`[0, 3, 10, -23, -12, -1]`), `id="single"` (`[15]`). Error `_ORTHO_1D_INT_BAD_CASES` (each `ExpectFail(exception=IndexError, msg="")`): `id="out-of-bounds-high"` (`[31]`), `id="out-of-bounds-low"` (`[-31]`), `id="too-many-dims"` (`[[2, 4], [6, 8]]`). The raises test asserts both `z.get_orthogonal_selection(case.input)` and `z.oindex[case.input]` raise (two `pytest.raises` blocks in the test body, as the original did). Docstrings on both. + +- [ ] **Step 2: Run the two tests** + +Run: `uv run --frozen pytest "tests/test_indexing.py::test_get_orthogonal_selection_1d_int" "tests/test_indexing.py::test_get_orthogonal_selection_1d_int_raises" -v` +Expected: PASS. + +- [ ] **Step 3: Commit** + +```bash +git add tests/test_indexing.py +git commit -m "test: rewrite orthogonal 1d int indexing as parametrized cases + +Co-Authored-By: Claude Opus 4.7 (1M context) " +``` + +### Task 1.3: Rewrite `test_get_orthogonal_selection_2d` and `_test_get_orthogonal_selection_2d` + +**Files:** Modify `tests/test_indexing.py` (lines 671-727). + +- [ ] **Step 1: Apply the recipe.** Array `np.arange(60).reshape(12, 5)`, chunks `(5, 2)`. The private `_test_get_orthogonal_selection_2d` helper currently loops over 7 selection shapes built from two index arrays — instead build a `_ORTHO_2D_CASES: list[Expect[OrthogonalSelection, None]]` table directly with concrete hand-picked `ix0`/`ix1` (defined once at module scope), covering the same shapes: both-axes-array, array×slice, array×strided-slice, slice×array, array×int, int×array, plus the mixed int-array/bool-array pair. Use deterministic indices, e.g. `ix0_bool = np.isin(np.arange(12), [0, 5, 11])`, `ix1_bool = np.array([True, False, True, False, True])`, `ix0_int = np.array([0, 5, 11])`, `ix1_int = np.array([0, 2, 4])`. Fold the `basic_selections_2d` coverage of orthogonal into existing parametrize ids. Error cases from `basic_selections_2d_bad` → `_ORTHO_2D_BAD_CASES` (`ExpectFail`, `IndexError`, `msg=""`), raises test asserting both `get_orthogonal_selection` and `oindex`. Delete the now-unused `_test_get_orthogonal_selection_2d` helper. Docstrings on both tests. + +- [ ] **Step 2: Run** + +Run: `uv run --frozen pytest "tests/test_indexing.py::test_get_orthogonal_selection_2d" "tests/test_indexing.py::test_get_orthogonal_selection_2d_raises" -v` +Expected: PASS. + +- [ ] **Step 3: Commit** + +```bash +git add tests/test_indexing.py +git commit -m "test: rewrite orthogonal 2d indexing as parametrized cases + +Co-Authored-By: Claude Opus 4.7 (1M context) " +``` + +### Task 1.4: Rewrite `test_get_orthogonal_selection_3d` and `_test_get_orthogonal_selection_3d` + +**Files:** Modify `tests/test_indexing.py` (lines 729-792). + +- [ ] **Step 1: Apply the recipe.** Array `np.arange(420).reshape(7, 6, 10)`, chunks `(3, 2, 4)`. The private helper enumerates 20 selection tuples — turn each into an `Expect` case in `_ORTHO_3D_CASES` with a descriptive id (e.g. `"single-value"`, `"all-negative"`, `"three-arrays"`, `"array-slice-slice"`, ... matching the comment groupings in the original). Define the hand-picked index arrays once at module scope: `ix0 = np.isin(np.arange(7), [0, 3, 6])` (bool) and integer variants per axis. Keep both bool-array and sorted-int-array variants by including both kinds as separate ids rather than a sparsity loop. Adjust the literal int indices in the tuples (e.g. `60, 15, 4`) to be in-bounds for the new `(7, 6, 10)` shape (e.g. `5, 3, 8`). Delete the unused helper. Docstring. + +- [ ] **Step 2: Run** + +Run: `uv run --frozen pytest "tests/test_indexing.py::test_get_orthogonal_selection_3d" -v` +Expected: PASS. + +- [ ] **Step 3: Commit** + +```bash +git add tests/test_indexing.py +git commit -m "test: rewrite orthogonal 3d indexing as parametrized cases + +Co-Authored-By: Claude Opus 4.7 (1M context) " +``` + +### Task 1.5: Rewrite the set-orthogonal family (1d/2d/3d) and helpers + +**Files:** Modify `tests/test_indexing.py` (`test_set_orthogonal_selection_1d/2d/3d` and their `_test_set_orthogonal_selection_2d/3d` helpers, lines 807-969). + +- [ ] **Step 1: Apply the recipe** to all three set tests, reusing the same `_ORTHO_*_CASES` selection tables defined in Tasks 1.1-1.4 where the selections are valid for set (most are). The kept `_test_set_orthogonal_selection` helper already iterates over the three value forms (scalar / array / list) per selection and skips selections that produce an empty result (the existing `value == []` guard), so each parametrized case still exercises all applicable value forms. If a get-only case turns out not to round-trip through set, give it its own filtered list (e.g. `_ORTHO_3D_SET_CASES`) rather than forcing it. Use the same canonical shapes/chunks. For 1d, parametrize over `_ORTHO_1D_BOOL_CASES + _ORTHO_1D_INT_CASES` minus any that can't be set (e.g. empty selections that can't preserve dimensions — keep the existing skip-empty guard inside `_test_set_orthogonal_selection`). Delete the now-unused `_test_set_orthogonal_selection_2d` and `_test_set_orthogonal_selection_3d` helpers. Docstrings on all three. + +- [ ] **Step 2: Run** + +Run: `uv run --frozen pytest "tests/test_indexing.py::test_set_orthogonal_selection_1d" "tests/test_indexing.py::test_set_orthogonal_selection_2d" "tests/test_indexing.py::test_set_orthogonal_selection_3d" -v` +Expected: PASS. + +- [ ] **Step 3: Commit** + +```bash +git add tests/test_indexing.py +git commit -m "test: rewrite set-orthogonal indexing family as parametrized cases + +Co-Authored-By: Claude Opus 4.7 (1M context) " +``` + +### Task 1.6: Rewrite the basic-selection 1d/2d families + +**Files:** Modify `tests/test_indexing.py` (`basic_selections_1d`, `basic_selections_1d_bad`, `test_get_basic_selection_1d`, `basic_selections_2d`, `basic_selections_2d_bad`, `test_get_basic_selection_2d`, lines 204-385). + +- [ ] **Step 1: Shrink the selection tables to the new array sizes and parametrize.** + - 1d array `np.arange(30)`, chunks `(7,)`. Rewrite `basic_selections_1d` as a trimmed `Expect[BasicSelection, None]` list: keep one representative of each kind — single value (`5`, `-1`), bounded slice (`slice(3, 18)`), over-bounds slice (`slice(0, 100)`), negative slice (`slice(-18, -3)`), empty (`slice(0, 0)`, `slice(-1, 0)`), full (`slice(None)`, `Ellipsis`, `()`), and a few stepped slices (`slice(None, None, 3)`, `slice(3, 27, 5)`). Drop the giant step-sweep (steps 10..10000 on a 30-element array are redundant). Each gets an id. + - `basic_selections_1d_bad` → `_BASIC_1D_BAD_CASES` (`ExpectFail`, `IndexError`, `msg=""`): keep one negative-step slice (`slice(None, None, -1)`), the type errors (`2.3`, `"foo"`, `b"xxx"`, `None`), the shape errors (`(0, 0)`, `(slice(None), slice(None))`), and the integer-list case `[1, 0]`. raises test asserts both `get_basic_selection` and `z[...]`. + - Same treatment for 2d (`np.arange(60).reshape(12, 5)`, chunks `(5, 2)`): trim `basic_selections_2d` to representatives, adjust literal indices to be in-bounds (e.g. `42`→`5`, `slice(250, 350)`→`slice(2, 9)`). Keep the fancy-indexing fallback assertion (`z[([0, 1], [0, 1])]`) as a small standalone test `test_basic_2d_fancy_fallback` with a docstring. + - Keep the kept `_test_get_basic_selection` oracle helper (it also checks the `out=` param). Docstrings on all. + +- [ ] **Step 2: Run** + +Run: `uv run --frozen pytest "tests/test_indexing.py::test_get_basic_selection_1d" "tests/test_indexing.py::test_get_basic_selection_2d" -v` +Expected: PASS. + +- [ ] **Step 3: Commit** + +```bash +git add tests/test_indexing.py +git commit -m "test: rewrite basic 1d/2d selection families as parametrized cases + +Co-Authored-By: Claude Opus 4.7 (1M context) " +``` + +### Task 1.7: Rewrite the coordinate-selection family (get/set, 1d/2d) + +**Files:** Modify `tests/test_indexing.py` (`test_get_coordinate_selection_1d/2d`, `test_set_coordinate_selection_1d/2d`, `_test_set_coordinate_selection`, and `coordinate_selections_1d_bad`, lines 1019-1186). + +- [ ] **Step 1: Apply the recipe.** 1d array `np.arange(30)`, chunks `(7,)`. `_COORD_1D_CASES` (`Expect[CoordinateSelection, None]`): single (`5`, `-1`), wraparound (`[0, 3, 10, -23, -12, -1]` → adjust to in-bounds for 30: `[0, 3, 10, -23, -12, -1]` is valid since -23..−1 map into 0..29), out-of-order (`[3, 25, 8, 17]`), multi-dim (`np.array([[2, 4], [6, 8]])`), sorted (`[1, 8, 15, 29]`), reversed (`[29, 15, 8, 1]`). Replace the `for p in 2, 0.5, 0.1, 0.01` random sweep entirely. Error `_COORD_1D_BAD_CASES` from `coordinate_selections_1d_bad` + out-of-bounds (`[31]`, `[-31]`). 2d array `np.arange(60).reshape(12, 5)`, chunks `(5, 2)`: `_COORD_2D_CASES` covering single `(5, 4)`, `(-1, -1)`, both-axes-array `(ix0, ix1)` with deterministic `ix0=[0,5,11,2,8]`, `ix1=[1,3,4,0,2]`, mixed array/int, not-monotonic cases, multi-dim arrays. 2d error cases (slice mixed with array, Ellipsis) → `_COORD_2D_BAD_CASES`. Keep `_test_get_coordinate_selection`/`_test_set_coordinate_selection` helpers. Docstrings throughout. + +- [ ] **Step 2: Run** + +Run: `uv run --frozen pytest tests/test_indexing.py -k coordinate -v` +Expected: PASS. + +- [ ] **Step 3: Commit** + +```bash +git add tests/test_indexing.py +git commit -m "test: rewrite coordinate selection family as parametrized cases + +Co-Authored-By: Claude Opus 4.7 (1M context) " +``` + +### Task 1.8: Rewrite the block-selection family (get/set, 1d/2d) + +**Files:** Modify `tests/test_indexing.py` (`block_selections_*`, `test_get_block_selection_1d/2d`, `test_set_block_selection_1d/2d`, lines 1189-1378). + +- [ ] **Step 1: Apply the recipe, preserving the selection↔projection pairing.** The block tests pair a block selection with the array slice it projects to. Keep that by making each case carry both, using `Expect[BasicSelection, slice | tuple[slice, ...]]` where `input` is the block selection and `output` is the expected array-index projection (this is the one family where `output` is meaningfully used). 1d array `np.arange(30)`, chunks `(7,)` → 5 blocks. Recompute the projections for the new chunking: block `0`→`slice(0,7)`, block `4`→`slice(28,30)`, `-1`→`slice(28,30)`, `slice(None,3)`→`slice(0,21)`, etc. Build `_BLOCK_1D_CASES` with ids. Error `_BLOCK_1D_BAD_CASES` from `block_selections_1d_bad` + out-of-bounds (`n_blocks+1`, `-(n_blocks+1)` computed from `z._chunk_grid.get_nchunks()`). 2d array `np.arange(60).reshape(12, 5)`, chunks `(5, 2)` → 3×3 blocks; recompute the 2d projections. Update `_test_get_block_selection`/`_test_set_block_selection` to take the projection from `case.output`. Docstrings. + +- [ ] **Step 2: Run** + +Run: `uv run --frozen pytest tests/test_indexing.py -k block -v` +Expected: PASS. + +- [ ] **Step 3: Commit** + +```bash +git add tests/test_indexing.py +git commit -m "test: rewrite block selection family as parametrized cases + +Co-Authored-By: Claude Opus 4.7 (1M context) " +``` + +### Task 1.9: Rewrite the mask-selection family (get/set, 1d/2d) + +**Files:** Modify `tests/test_indexing.py` (`mask_selections_1d_bad`, `test_get_mask_selection_1d/2d`, `test_set_mask_selection_1d/2d`, lines 1391-1497). + +- [ ] **Step 1: Apply the recipe.** 1d array `np.arange(30)`, chunks `(7,)`. `_MASK_1D_CASES` (`Expect[Any, None]`): all-false, all-true, alternating (`np.arange(30) % 2 == 0`), sparse (`np.isin(np.arange(30), [0, 7, 14, 29])`). 2d array `np.arange(60).reshape(12, 5)`, chunks `(5, 2)`: `_MASK_2D_CASES`: all-false, all-true, checkerboard (`(np.add.outer(np.arange(12), np.arange(5)) % 2).astype(bool)`), sparse. Replace both `for p in 0.5, 0.1, 0.01` sweeps. Error `_MASK_1D_BAD_CASES` from `mask_selections_1d_bad` + too-short (`np.zeros(5, bool)`), too-long (`np.zeros(50, bool)`), too-many-dims (`[[True, False], [False, True]]`); and a `_MASK_2D_BAD_CASES` for the 2d wrong-shape/wrong-ndim cases. Keep `_test_get_mask_selection`/`_test_set_mask_selection`. Docstrings. + +- [ ] **Step 2: Run** + +Run: `uv run --frozen pytest tests/test_indexing.py -k mask -v` +Expected: PASS. + +- [ ] **Step 3: Commit** + +```bash +git add tests/test_indexing.py +git commit -m "test: rewrite mask selection family as parametrized cases + +Co-Authored-By: Claude Opus 4.7 (1M context) " +``` + +### Task 1.10: Shrink `test_get_selection_out` and the remaining large-array tests + +**Files:** Modify `tests/test_indexing.py` (`test_get_selection_out` lines 1500-1568; `test_indexing_equals_numpy` lines 1853-1880; `test_orthogonal_bool_indexing_like_numpy_ix` lines 1883-1900). + +- [ ] **Step 1: Shrink arrays without changing test logic.** + - `test_get_selection_out`: read the full body (lines 1500-1568) first; reduce `np.arange(1050)`/`(100,)` to `np.arange(30)`/`(7,)` and adjust the literal selection bounds proportionally, keeping the same `out=` buffer assertions. + - `test_indexing_equals_numpy` and `test_orthogonal_bool_indexing_like_numpy_ix`: these already parametrize but on a `(1000, 10)` array. Shrink to `(12, 5)` chunks `(5, 2)` and rescale the selections (e.g. `np.arange(1000)`→`np.arange(12)`, `np.tile([True, False], (1000, 5))`→`np.tile([True, False], (12, ...))` sized to the new shape; `[100, 200, 300]`→in-bounds coords). Add/keep docstrings. + +- [ ] **Step 2: Run** + +Run: `uv run --frozen pytest "tests/test_indexing.py::test_get_selection_out" "tests/test_indexing.py::test_indexing_equals_numpy" "tests/test_indexing.py::test_orthogonal_bool_indexing_like_numpy_ix" -v` +Expected: PASS. + +- [ ] **Step 3: Commit** + +```bash +git add tests/test_indexing.py +git commit -m "test: shrink arrays in selection_out and numpy-equivalence tests + +Co-Authored-By: Claude Opus 4.7 (1M context) " +``` + +### Task 1.11: Add docstrings to the leave-as-is tests and sweep already-parametrized ones + +**Files:** Modify `tests/test_indexing.py` (the GH-regression, xfail/skip, helper, fallback, and iter tests that keep their logic). + +- [ ] **Step 1: Add a one-line behavior docstring** to every test function that lacks one and was not rewritten above: `test_normalize_integer_selection`, `test_replace_ellipsis`, `test_fancy_indexing_fallback_on_get_setitem`, the `*_fallback_*` family, `test_setitem_zarr_array_as_value`, `test_set_basic_selection_0d`, `test_orthogonal_indexing_edge_cases`, `test_set_item_1d_last_two_chunks`, `test_orthogonal_indexing_fallback_on_get_setitem`, `test_slice_selection_uints`, `test_numpy_int_indexing`, `test_accessed_chunks`, `test_iter_grid_invalid`, `test_indexing_with_zarr_array`, `test_zero_sized_chunks`, `test_vectorized_indexing_incompatible_shape`, `test_iter_chunk_regions`, `test_iter_regions`, and the `TestAsync` methods. Do not change their logic or array sizes (regression tests keep their reproducing values per the spec). + +- [ ] **Step 2: Run the whole file** + +Run: `uv run --frozen pytest tests/test_indexing.py -q` +Expected: PASS, same skip/xfail counts as the baseline (1 skipped, 5 xfailed). + +- [ ] **Step 3: Commit** + +```bash +git add tests/test_indexing.py +git commit -m "test: add behavior docstrings to remaining indexing tests + +Co-Authored-By: Claude Opus 4.7 (1M context) " +``` + +--- + +## Part 2 — Final verification + +**Baseline (measured 2026-05-22, before any changes):** full file +`150 passed, 1 skipped, 5 xfailed` in ~3.5 s (pytest) / ~4 s wall. Slowest: +`test_set_orthogonal_selection_3d` 0.92 s, `test_get_orthogonal_selection_1d_bool` +0.79 s, `test_set_orthogonal_selection_2d` 0.54 s, +`test_set_orthogonal_selection_1d` 0.41 s. The post-rewrite test *count* will +differ (monolithic tests split into many parametrized cases); skip/xfail counts +must stay `1`/`5`. + +### Task 2.1: Full-suite verification and speed measurement + +- [ ] **Step 1: Run the whole indexing file with durations** + +Run: `uv run --frozen pytest tests/test_indexing.py --durations=25 -q` +Expected: PASS; the four tests that were >0.4 s at baseline +(`test_set_orthogonal_selection_3d`, `test_get_orthogonal_selection_1d_bool`, +`test_set_orthogonal_selection_2d`, `test_set_orthogonal_selection_1d`) each +now under ~0.15 s. Record the new total in the commit message. + +- [ ] **Step 2: Run the dedup-affected suites once more** + +Run: `uv run --frozen pytest tests/test_chunk_grids.py tests/test_codecs/ tests/test_metadata/test_v3.py -q` +Expected: PASS. + +- [ ] **Step 3: Full lint + type pass** + +Run: `prek run --all-files` +Expected: all hooks pass (ruff, mypy, codespell, etc.). + +- [ ] **Step 4: Add a changelog fragment if the repo requires one** + +Check `changes/` (towncrier). If indexing test-only changes need a fragment per `towncrier-check`, add one; otherwise skip. Run `prek run towncrier-check --all-files` to confirm. + +- [ ] **Step 5: Final commit** + +```bash +git add -A +git commit -m "$(cat <<'EOF' +test: finalize indexing test cleanup + +Full suite green; orthogonal tests dropped from ~0.5-0.9s to <0.15s each. + +Co-Authored-By: Claude Opus 4.7 (1M context) +EOF +)" +``` + +--- + +## Notes for the implementer + +- **Read before you rewrite.** Several tasks reference line ranges that shift + as earlier tasks edit the file. Re-locate each function by name before + editing; don't trust stale line numbers. +- **`Selection` types** (`BasicSelection`, `OrthogonalSelection`, + `CoordinateSelection`, `Selection`) are already imported at the top of + `tests/test_indexing.py` — use them in the `Expect[...]` annotations. +- **`msg=""`** in `ExpectFail` matches any message (the original error tests + only asserted `IndexError`, not text). Don't invent message regexes that the + code doesn't actually emit. +- **Keep the oracle helpers.** Do not inline `_test_get_orthogonal_selection` + etc. into the parametrized tests; calling them once per case is the design. +- If a rewritten happy-path case turns out to disagree with numpy (the oracle + fails), that is a real finding — stop and investigate per + systematic-debugging, don't tweak the case to make it pass. +``` From 72bc014fd01fda4c0038ee5288970ab1db4f327b Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 22 May 2026 18:07:18 +0200 Subject: [PATCH 03/27] test: make canonical Expect/ExpectFail frozen Prepares for deduplicating the second Expect pair in test_codecs/conftest.py. Co-Authored-By: Claude Opus 4.7 (1M context) --- tests/conftest.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/conftest.py b/tests/conftest.py index 3515acace0..5af4aa21de 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -64,7 +64,7 @@ from zarr.core.dtype.wrapper import ZDType -@dataclass +@dataclass(frozen=True) class Expect[TIn, TOut]: """A test case with explicit input, expected output, and a human-readable id.""" @@ -73,7 +73,7 @@ class Expect[TIn, TOut]: id: str -@dataclass +@dataclass(frozen=True) class ExpectFail[TIn]: """A test case that should raise an exception.""" From 1d03c1ff9192fbe123908e4ad7437bf719f57f66 Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 22 May 2026 18:29:24 +0200 Subject: [PATCH 04/27] test: migrate test_chunk_grids to canonical Expect/ExpectFail Co-Authored-By: Claude Opus 4.7 (1M context) --- tests/test_chunk_grids.py | 105 ++++++++++++++++++++++---------------- 1 file changed, 62 insertions(+), 43 deletions(-) diff --git a/tests/test_chunk_grids.py b/tests/test_chunk_grids.py index 681a599130..cbdbc50766 100644 --- a/tests/test_chunk_grids.py +++ b/tests/test_chunk_grids.py @@ -4,7 +4,7 @@ import numpy as np import pytest -from tests.test_codecs.conftest import Expect, ExpectErr +from tests.conftest import Expect, ExpectFail from zarr.core.chunk_grids import ( ChunkLayout, _guess_regular_chunks, @@ -131,81 +131,100 @@ def test_chunk_layout_nested() -> None: @pytest.mark.parametrize( "case", [ - ExpectErr(input=(0, 100), msg="Chunk size must be positive", exception_cls=ValueError), - ExpectErr(input=(-2, 100), msg="Chunk size must be positive", exception_cls=ValueError), - ExpectErr(input=([], 100), msg="must not be empty", exception_cls=ValueError), - ExpectErr(input=([10, -1, 10], 100), msg="must be positive", exception_cls=ValueError), - ExpectErr(input=([10, 0, 10], 20), msg="must be positive", exception_cls=ValueError), - ExpectErr(input=([10, 20], 100), msg="do not sum to span", exception_cls=ValueError), + ExpectFail( + input=(0, 100), + exception=ValueError, + id="zero-uniform", + msg="Chunk size must be positive", + ), + ExpectFail( + input=(-2, 100), + exception=ValueError, + id="negative-uniform", + msg="Chunk size must be positive", + ), + ExpectFail(input=([], 100), exception=ValueError, id="empty-list", msg="must not be empty"), + ExpectFail( + input=([10, -1, 10], 100), + exception=ValueError, + id="negative-element", + msg="must be positive", + ), + ExpectFail( + input=([10, 0, 10], 20), exception=ValueError, id="zero-element", msg="must be positive" + ), + ExpectFail( + input=([10, 20], 100), exception=ValueError, id="wrong-sum", msg="do not sum to span" + ), # Nested/RLE form for a single dim is rejected with offending indices. - ExpectErr( + ExpectFail( input=([[3, 3], 1], 7), + exception=TypeError, + id="rle-single-dim", msg="non-integer element(s) ([3, 3],) at indices (0,)", - exception_cls=TypeError, ), # Multiple non-int elements: all offending indices reported. - ExpectErr( + ExpectFail( input=([1, [2, 2], 1, [3]], 9), + exception=TypeError, + id="multiple-non-ints", msg="non-integer element(s) ([2, 2], [3]) at indices (1, 3)", - exception_cls=TypeError, ), # Strings are non-integers and should be reported the same way. - ExpectErr( + ExpectFail( input=([2, "3", 5], 10), + exception=TypeError, + id="string-element", msg="non-integer element(s) ('3',) at indices (1,)", - exception_cls=TypeError, ), ], - ids=[ - "zero-uniform", - "negative-uniform", - "empty-list", - "negative-element", - "zero-element", - "wrong-sum", - "rle-single-dim", - "multiple-non-ints", - "string-element", - ], + ids=lambda c: c.id, ) -def test_normalize_chunks_1d_errors(case: ExpectErr[tuple[Any, int]]) -> None: +def test_normalize_chunks_1d_errors(case: ExpectFail[tuple[Any, int]]) -> None: """Invalid 1D chunk specifications are rejected with informative error messages.""" chunks, span = case.input - with pytest.raises(case.exception_cls, match=re.escape(case.msg)): + with pytest.raises(case.exception, match=re.escape(case.msg)): normalize_chunks_1d(chunks, span=span) @pytest.mark.parametrize( "case", [ - ExpectErr( + ExpectFail( input=(None, (100,)), + exception=ValueError, + id="none", msg="None is not a valid chunk input", - exception_cls=ValueError, ), # `True` is rejected explicitly because bool is a subclass of int — without # this guard, `chunks=True` would silently produce size-1 chunks. - ExpectErr( + ExpectFail( input=(True, (100,)), + exception=ValueError, + id="true", msg="True is not a valid chunk input", - exception_cls=ValueError, ), - ExpectErr(input=("foo", (100,)), msg="dimensions", exception_cls=ValueError), - ExpectErr(input=((100, 10), (100,)), msg="dimensions", exception_cls=ValueError), - ExpectErr(input=((10,), (100, 100)), msg="dimensions", exception_cls=ValueError), + ExpectFail(input=("foo", (100,)), exception=ValueError, id="string", msg="dimensions"), + ExpectFail( + input=((100, 10), (100,)), exception=ValueError, id="too-many-dims", msg="dimensions" + ), + ExpectFail( + input=((10,), (100, 100)), exception=ValueError, id="too-few-dims", msg="dimensions" + ), # End-to-end: per-dim RLE surfaces through normalize_chunks_nd. - ExpectErr( + ExpectFail( input=([[6, 4], [[3, 3], 1]], (10, 10)), + exception=TypeError, + id="rle-inner-dim", msg="non-integer element(s) ([3, 3],) at indices (0,)", - exception_cls=TypeError, ), ], - ids=["none", "true", "string", "too-many-dims", "too-few-dims", "rle-inner-dim"], + ids=lambda c: c.id, ) -def test_normalize_chunks_nd_errors(case: ExpectErr[tuple[Any, tuple[int, ...]]]) -> None: +def test_normalize_chunks_nd_errors(case: ExpectFail[tuple[Any, tuple[int, ...]]]) -> None: """Invalid N-D chunk specifications are rejected with informative error messages.""" chunks, shape = case.input - with pytest.raises(case.exception_cls, match=re.escape(case.msg)): + with pytest.raises(case.exception, match=re.escape(case.msg)): normalize_chunks_nd(chunks, shape) @@ -213,13 +232,13 @@ def test_normalize_chunks_nd_errors(case: ExpectErr[tuple[Any, tuple[int, ...]]] "case", [ # uniform-chunks branch: one int → broadcast across span via np.full. - Expect(input=(1000, 100_000), expected=[1000] * 100), + Expect(input=(1000, 100_000), output=[1000] * 100, id="uniform"), # explicit-per-chunk branch. - Expect(input=([10, 20, 30, 40], 100), expected=[10, 20, 30, 40]), + Expect(input=([10, 20, 30, 40], 100), output=[10, 20, 30, 40], id="explicit-list"), # -1 sentinel branch: one chunk covering the full span. - Expect(input=(-1, 100), expected=[100]), + Expect(input=(-1, 100), output=[100], id="full-span-sentinel"), ], - ids=["uniform", "explicit-list", "full-span-sentinel"], + ids=lambda c: c.id, ) def test_normalize_chunks_1d_returns_int64_array( case: Expect[tuple[Any, int], list[int]], @@ -230,4 +249,4 @@ def test_normalize_chunks_1d_returns_int64_array( assert isinstance(result, np.ndarray) assert result.dtype == np.int64 assert result.ndim == 1 - assert result.tolist() == case.expected + assert result.tolist() == case.output From 6918872cfef500cfac66eb9b040a3818b0dd6fac Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 22 May 2026 18:39:27 +0200 Subject: [PATCH 05/27] test: migrate test_cast_value to canonical Expect/ExpectFail Co-Authored-By: Claude Opus 4.7 (1M context) --- tests/test_codecs/test_cast_value.py | 114 +++++++++++++++------------ 1 file changed, 62 insertions(+), 52 deletions(-) diff --git a/tests/test_codecs/test_cast_value.py b/tests/test_codecs/test_cast_value.py index 361073f96b..2dd7b2088b 100644 --- a/tests/test_codecs/test_cast_value.py +++ b/tests/test_codecs/test_cast_value.py @@ -6,7 +6,7 @@ import pytest import zarr -from tests.test_codecs.conftest import Expect, ExpectErr +from tests.conftest import Expect, ExpectFail from zarr.codecs.cast_value import CastValue try: @@ -31,7 +31,8 @@ [ Expect( input=CastValue(data_type="uint8"), - expected={"name": "cast_value", "configuration": {"data_type": "uint8"}}, + output={"name": "cast_value", "configuration": {"data_type": "uint8"}}, + id="minimal", ), Expect( input=CastValue( @@ -40,7 +41,7 @@ out_of_range="clamp", scalar_map={"encode": [("NaN", 0)]}, ), - expected={ + output={ "name": "cast_value", "configuration": { "data_type": "uint8", @@ -49,13 +50,14 @@ "scalar_map": {"encode": [("NaN", 0)]}, }, }, + id="full", ), ], - ids=["minimal", "full"], + ids=lambda c: c.id, ) def test_to_dict(case: Expect[CastValue, dict[str, Any]]) -> None: """to_dict produces the expected JSON structure.""" - assert case.input.to_dict() == case.expected + assert case.input.to_dict() == case.output @pytest.mark.parametrize( @@ -63,7 +65,8 @@ def test_to_dict(case: Expect[CastValue, dict[str, Any]]) -> None: [ Expect( input={"name": "cast_value", "configuration": {"data_type": "float32"}}, - expected=("float32", "nearest-even", None), + output=("float32", "nearest-even", None), + id="defaults", ), Expect( input={ @@ -74,15 +77,16 @@ def test_to_dict(case: Expect[CastValue, dict[str, Any]]) -> None: "out_of_range": "clamp", }, }, - expected=("int16", "towards-zero", "clamp"), + output=("int16", "towards-zero", "clamp"), + id="explicit", ), ], - ids=["defaults", "explicit"], + ids=lambda c: c.id, ) def test_from_dict(case: Expect[dict[str, Any], tuple[str, str, str | None]]) -> None: """from_dict deserializes configuration with correct values and defaults.""" codec = CastValue.from_dict(case.input) - dtype_name, rounding, out_of_range = case.expected + dtype_name, rounding, out_of_range = case.output assert codec.dtype.to_native_dtype() == np.dtype(dtype_name) assert codec.rounding == rounding assert codec.out_of_range == out_of_range @@ -133,22 +137,24 @@ def test_construction_rejects_invalid_target_dtype() -> None: @pytest.mark.parametrize( "case", [ - ExpectErr( + ExpectFail( input={"dtype": "complex128", "target": "float64"}, msg="only supports integer and floating-point", - exception_cls=ValueError, + exception=ValueError, + id="complex-source", ), - ExpectErr( + ExpectFail( input={"dtype": "int32", "target": "float64", "out_of_range": "wrap"}, msg="only valid for integer", - exception_cls=ValueError, + exception=ValueError, + id="wrap-float-target", ), ], - ids=["complex-source", "wrap-float-target"], + ids=lambda c: c.id, ) -def test_validation_rejects_invalid(case: ExpectErr[dict[str, Any]]) -> None: +def test_validation_rejects_invalid(case: ExpectFail[dict[str, Any]]) -> None: """Invalid dtype or out_of_range combinations are rejected at array creation.""" - with pytest.raises(case.exception_cls, match=case.msg): + with pytest.raises(case.exception, match=case.msg): zarr.create_array( store={}, shape=(10,), @@ -216,14 +222,14 @@ def test_zero_itemsize_raises() -> None: @pytest.mark.parametrize( "case", [ - Expect(input=("float64", "float32"), expected=np.arange(50, dtype="float64")), - Expect(input=("float32", "float64"), expected=np.arange(50, dtype="float32")), - Expect(input=("int32", "int64"), expected=np.arange(50, dtype="int32")), - Expect(input=("int64", "int16"), expected=np.arange(50, dtype="int64")), - Expect(input=("float64", "int32"), expected=np.arange(50, dtype="float64")), - Expect(input=("int32", "float64"), expected=np.arange(50, dtype="int32")), + Expect(input=("float64", "float32"), output=np.arange(50, dtype="float64"), id="f64→f32"), + Expect(input=("float32", "float64"), output=np.arange(50, dtype="float32"), id="f32→f64"), + Expect(input=("int32", "int64"), output=np.arange(50, dtype="int32"), id="i32→i64"), + Expect(input=("int64", "int16"), output=np.arange(50, dtype="int64"), id="i64→i16"), + Expect(input=("float64", "int32"), output=np.arange(50, dtype="float64"), id="f64→i32"), + Expect(input=("int32", "float64"), output=np.arange(50, dtype="int32"), id="i32→f64"), ], - ids=["f64→f32", "f32→f64", "i32→i64", "i64→i16", "f64→i32", "i32→f64"], + ids=lambda c: c.id, ) def test_encode_decode_roundtrip( case: Expect[tuple[str, str], np.ndarray[Any, np.dtype[Any]]], @@ -241,8 +247,8 @@ def test_encode_decode_roundtrip( compressors=None, fill_value=0, ) - arr[:] = case.expected - np.testing.assert_array_equal(arr[:], case.expected) + arr[:] = case.output + np.testing.assert_array_equal(arr[:], case.output) @requires_cast_value_rs @@ -251,10 +257,11 @@ def test_encode_decode_roundtrip( [ Expect( input=np.array([1.7, -1.7, 2.5, -2.5], dtype="float64"), - expected=np.array([1, -1, 2, -2], dtype="float64"), + output=np.array([1, -1, 2, -2], dtype="float64"), + id="towards-zero", ), ], - ids=["towards-zero"], + ids=lambda c: c.id, ) def test_float_to_int_rounding( case: Expect[np.ndarray[Any, np.dtype[Any]], np.ndarray[Any, np.dtype[Any]]], @@ -272,7 +279,7 @@ def test_float_to_int_rounding( fill_value=0, ) arr[:] = case.input - np.testing.assert_array_equal(arr[:], case.expected) + np.testing.assert_array_equal(arr[:], case.output) @requires_cast_value_rs @@ -281,10 +288,11 @@ def test_float_to_int_rounding( [ Expect( input=np.array([0, 200, -200], dtype="int32"), - expected=np.array([0, 127, -128], dtype="int32"), + output=np.array([0, 127, -128], dtype="int32"), + id="int32→int8", ), ], - ids=["int32→int8"], + ids=lambda c: c.id, ) def test_out_of_range_clamp( case: Expect[np.ndarray[Any, np.dtype[Any]], np.ndarray[Any, np.dtype[Any]]], @@ -302,7 +310,7 @@ def test_out_of_range_clamp( fill_value=0, ) arr[:] = case.input - np.testing.assert_array_equal(arr[:], case.expected) + np.testing.assert_array_equal(arr[:], case.output) def test_compute_encoded_size() -> None: @@ -355,55 +363,54 @@ def test_scalar_map_encode_decode_roundtrip() -> None: @pytest.mark.parametrize( "case", [ - ExpectErr( + ExpectFail( input={ "dtype": "int32", "target": "int8", "scalar_map": {"encode": [("NaN", 0)]}, }, msg="not representable in dtype int32", - exception_cls=ValueError, + exception=ValueError, + id="nan-key-for-int-source", ), - ExpectErr( + ExpectFail( input={ "dtype": "int32", "target": "float64", "scalar_map": {"decode": [(0, "NaN")]}, }, msg="not representable in dtype int32", - exception_cls=ValueError, + exception=ValueError, + id="nan-value-for-int-decode-target", ), - ExpectErr( + ExpectFail( input={ "dtype": "float64", "target": "int8", "scalar_map": {"encode": [("NaN", 999)]}, }, msg="not representable in dtype int8", - exception_cls=ValueError, + exception=ValueError, + id="encode-value-out-of-range", ), - ExpectErr( + ExpectFail( input={ "dtype": "float64", "target": "int8", "scalar_map": {"encode": [("NaN", 1.5)]}, }, msg="not representable in dtype int8", - exception_cls=ValueError, + exception=ValueError, + id="encode-value-not-integer", ), ], - ids=[ - "nan-key-for-int-source", - "nan-value-for-int-decode-target", - "encode-value-out-of-range", - "encode-value-not-integer", - ], + ids=lambda c: c.id, ) -def test_scalar_map_validation_rejects_invalid(case: ExpectErr[dict[str, Any]]) -> None: +def test_scalar_map_validation_rejects_invalid(case: ExpectFail[dict[str, Any]]) -> None: """Invalid scalar_map entries are rejected at array creation.""" import zarr - with pytest.raises(case.exception_cls, match=case.msg): + with pytest.raises(case.exception, match=case.msg): zarr.create_array( store={}, shape=(10,), @@ -450,20 +457,23 @@ def test_combined_with_scale_offset() -> None: [ Expect( input={"encode": [("NaN", 0)]}, - expected={"encode": {"NaN": 0}}, + output={"encode": {"NaN": 0}}, + id="encode-only", ), Expect( input={"encode": [("NaN", 0)], "decode": [(0, "NaN")]}, - expected={"encode": {"NaN": 0}, "decode": {0: "NaN"}}, + output={"encode": {"NaN": 0}, "decode": {0: "NaN"}}, + id="both-directions", ), Expect( input={"encode": {"NaN": 0}}, - expected={"encode": {"NaN": 0}}, + output={"encode": {"NaN": 0}}, + id="already-normalized", ), ], - ids=["encode-only", "both-directions", "already-normalized"], + ids=lambda c: c.id, ) def test_parse_scalar_map(case: Expect[Any, Any]) -> None: from zarr.codecs.cast_value import parse_scalar_map - assert parse_scalar_map(case.input) == case.expected + assert parse_scalar_map(case.input) == case.output From 97dbb4c2dd0f25bd050de06ce13beecb6174fa6c Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 22 May 2026 19:35:03 +0200 Subject: [PATCH 06/27] test: migrate test_scale_offset to canonical Expect/ExpectFail Co-Authored-By: Claude Opus 4.7 (1M context) --- tests/test_codecs/test_scale_offset.py | 75 +++++++++++++++----------- 1 file changed, 43 insertions(+), 32 deletions(-) diff --git a/tests/test_codecs/test_scale_offset.py b/tests/test_codecs/test_scale_offset.py index 99a5e3b99d..486f6b3d36 100644 --- a/tests/test_codecs/test_scale_offset.py +++ b/tests/test_codecs/test_scale_offset.py @@ -6,7 +6,7 @@ import pytest import zarr -from tests.test_codecs.conftest import Expect, ExpectErr +from tests.conftest import Expect, ExpectFail from zarr.codecs.scale_offset import ( ScaleOffset, _decode, @@ -24,42 +24,46 @@ @pytest.mark.parametrize( "case", [ - Expect(input=ScaleOffset(), expected={"name": "scale_offset"}), + Expect(input=ScaleOffset(), output={"name": "scale_offset"}, id="default"), Expect( input=ScaleOffset(offset=5), - expected={"name": "scale_offset", "configuration": {"offset": 5}}, + output={"name": "scale_offset", "configuration": {"offset": 5}}, + id="offset-only", ), Expect( input=ScaleOffset(scale=0.1), - expected={"name": "scale_offset", "configuration": {"scale": 0.1}}, + output={"name": "scale_offset", "configuration": {"scale": 0.1}}, + id="scale-only", ), Expect( input=ScaleOffset(offset=5, scale=0.1), - expected={"name": "scale_offset", "configuration": {"offset": 5, "scale": 0.1}}, + output={"name": "scale_offset", "configuration": {"offset": 5, "scale": 0.1}}, + id="both", ), ], - ids=["default", "offset-only", "scale-only", "both"], + ids=lambda c: c.id, ) def test_to_dict(case: Expect[ScaleOffset, dict[str, Any]]) -> None: """to_dict produces the expected JSON structure.""" - assert case.input.to_dict() == case.expected + assert case.input.to_dict() == case.output @pytest.mark.parametrize( "case", [ - Expect(input={"name": "scale_offset"}, expected=(0, 1)), + Expect(input={"name": "scale_offset"}, output=(0, 1), id="no-config"), Expect( input={"name": "scale_offset", "configuration": {"offset": 3, "scale": 2}}, - expected=(3, 2), + output=(3, 2), + id="with-config", ), ], - ids=["no-config", "with-config"], + ids=lambda c: c.id, ) def test_from_dict(case: Expect[dict[str, Any], tuple[int | float, int | float]]) -> None: """from_dict deserializes configuration with correct values and defaults.""" codec = ScaleOffset.from_dict(case.input) - expected_offset, expected_scale = case.expected + expected_offset, expected_scale = case.output assert codec.offset == expected_offset assert codec.scale == expected_scale @@ -79,38 +83,42 @@ def test_serialization_roundtrip() -> None: @pytest.mark.parametrize( "case", [ - ExpectErr( + ExpectFail( input={"offset": [1, 2]}, + exception=TypeError, + id="list-offset", msg="offset must be a number or string", - exception_cls=TypeError, ), - ExpectErr( - input={"scale": [1, 2]}, msg="scale must be a number or string", exception_cls=TypeError + ExpectFail( + input={"scale": [1, 2]}, + exception=TypeError, + id="list-scale", + msg="scale must be a number or string", ), ], - ids=["list-offset", "list-scale"], + ids=lambda c: c.id, ) -def test_construction_rejects_non_numeric(case: ExpectErr[dict[str, Any]]) -> None: +def test_construction_rejects_non_numeric(case: ExpectFail[dict[str, Any]]) -> None: """Non-numeric offset or scale is rejected at construction time.""" - with pytest.raises(case.exception_cls, match=case.msg): + with pytest.raises(case.exception, match=case.msg): ScaleOffset(**case.input) @pytest.mark.parametrize( "case", [ - Expect(input={"offset": 5, "scale": 2}, expected=(5, 2)), - Expect(input={"offset": 0.5, "scale": 0.1}, expected=(0.5, 0.1)), + Expect(input={"offset": 5, "scale": 2}, output=(5, 2), id="int"), + Expect(input={"offset": 0.5, "scale": 0.1}, output=(0.5, 0.1), id="float"), ], - ids=["int", "float"], + ids=lambda c: c.id, ) def test_construction_accepts_numeric( case: Expect[dict[str, Any], tuple[int | float, int | float]], ) -> None: """Integer and float values are accepted for both parameters.""" codec = ScaleOffset(**case.input) - assert codec.offset == case.expected[0] - assert codec.scale == case.expected[1] + assert codec.offset == case.output[0] + assert codec.scale == case.output[1] # --------------------------------------------------------------------------- @@ -259,28 +267,31 @@ def test_rejects_zero_scale() -> None: @pytest.mark.parametrize( "case", [ - ExpectErr( + ExpectFail( input={"dtype": "int32", "offset": 1.5, "scale": 1}, + exception=ValueError, + id="float-offset-for-int", msg="offset value 1.5 is not representable", - exception_cls=ValueError, ), - ExpectErr( + ExpectFail( input={"dtype": "int32", "offset": 0, "scale": 0.5}, + exception=ValueError, + id="float-scale-for-int", msg="scale value 0.5 is not representable", - exception_cls=ValueError, ), - ExpectErr( + ExpectFail( input={"dtype": "int16", "offset": "NaN", "scale": 1}, + exception=ValueError, + id="nan-offset-for-int", msg="offset value 'NaN' is not representable", - exception_cls=ValueError, ), ], - ids=["float-offset-for-int", "float-scale-for-int", "nan-offset-for-int"], + ids=lambda c: c.id, ) -def test_rejects_unrepresentable_scale_offset(case: ExpectErr[dict[str, Any]]) -> None: +def test_rejects_unrepresentable_scale_offset(case: ExpectFail[dict[str, Any]]) -> None: """Scale/offset values that can't be represented in the array dtype are rejected.""" - with pytest.raises(case.exception_cls, match=case.msg): + with pytest.raises(case.exception, match=case.msg): zarr.create_array( store={}, shape=(10,), From e3975982a6608d143c75e11354ec831baef0eda4 Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 22 May 2026 19:40:01 +0200 Subject: [PATCH 07/27] test: delete duplicate Expect dataclasses in test_codecs/conftest.py All consumers now use the canonical Expect/ExpectFail from tests/conftest.py. Co-Authored-By: Claude Opus 4.7 (1M context) --- tests/test_codecs/conftest.py | 20 -------------------- 1 file changed, 20 deletions(-) delete mode 100644 tests/test_codecs/conftest.py diff --git a/tests/test_codecs/conftest.py b/tests/test_codecs/conftest.py deleted file mode 100644 index b654ab1ec0..0000000000 --- a/tests/test_codecs/conftest.py +++ /dev/null @@ -1,20 +0,0 @@ -from __future__ import annotations - -from dataclasses import dataclass - - -@dataclass(frozen=True) -class Expect[TIn, TOut]: - """Model an input and an expected output value for a test case.""" - - input: TIn - expected: TOut - - -@dataclass(frozen=True) -class ExpectErr[TIn]: - """Model an input and an expected error message for a test case.""" - - input: TIn - msg: str - exception_cls: type[Exception] From af8c302ff336fe3f1c6c09f5d259d5fa92389417 Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 22 May 2026 19:41:53 +0200 Subject: [PATCH 08/27] test: rewrite orthogonal 1d bool indexing as parametrized cases Smaller array (30 elems, chunks of 7), hand-picked deterministic masks replacing np.random sparsity sweep, error paths split into their own test. Co-Authored-By: Claude Opus 4.7 (1M context) --- tests/test_indexing.py | 68 +++++++++++++++++++++++++++++++----------- 1 file changed, 50 insertions(+), 18 deletions(-) diff --git a/tests/test_indexing.py b/tests/test_indexing.py index c45942eee7..915d681689 100644 --- a/tests/test_indexing.py +++ b/tests/test_indexing.py @@ -11,6 +11,7 @@ from numpy.testing import assert_array_equal import zarr +from tests.conftest import Expect, ExpectFail from zarr import Array from zarr.core.buffer import default_buffer_prototype from zarr.core.indexing import ( @@ -611,26 +612,57 @@ def _test_get_orthogonal_selection( assert_array_equal(expect, actual) -# noinspection PyStatementEffect -def test_get_orthogonal_selection_1d_bool(store: StorePath) -> None: - # setup - a = np.arange(1050, dtype=int) - z = zarr_array_from_numpy_array(store, a, chunk_shape=(100,)) +_ORTHO_1D_BOOL_CASES: list[Expect[OrthogonalSelection, None]] = [ + Expect(input=np.zeros(30, dtype=bool), output=None, id="empty-mask"), + Expect(input=np.ones(30, dtype=bool), output=None, id="full-mask"), + Expect(input=np.arange(30) % 2 == 0, output=None, id="alternating-mask"), + Expect(input=np.eye(1, 30, 7, dtype=bool)[0], output=None, id="single-true"), + Expect( + input=np.isin(np.arange(30), [0, 1, 8, 15, 29]), + output=None, + id="sparse-cross-chunk", + ), +] - np.random.seed(42) - # test with different degrees of sparseness - for p in 0.5, 0.1, 0.01: - ix = np.random.binomial(1, p, size=a.shape[0]).astype(bool) - _test_get_orthogonal_selection(a, z, ix) +_ORTHO_1D_BOOL_BAD_CASES: list[ExpectFail[Any]] = [ + ExpectFail( + input=np.zeros(5, dtype=bool), + exception=IndexError, + id="mask-too-short", + msg="wrong length for dimension; expected 30, got 5", + ), + ExpectFail( + input=np.zeros(50, dtype=bool), + exception=IndexError, + id="mask-too-long", + msg="wrong length for dimension; expected 30, got 50", + ), + ExpectFail( + input=[[True, False], [False, True]], + exception=IndexError, + id="mask-too-many-dims", + msg="must be 1-dimensional only", + ), +] - # test errors - with pytest.raises(IndexError): - z.oindex[np.zeros(50, dtype=bool)] # too short - with pytest.raises(IndexError): - z.oindex[np.zeros(2000, dtype=bool)] # too long - with pytest.raises(IndexError): - # too many dimensions - z.oindex[[[True, False], [False, True]]] # type: ignore[index] + +@pytest.mark.parametrize("case", _ORTHO_1D_BOOL_CASES, ids=lambda c: c.id) +def test_get_orthogonal_selection_1d_bool( + store: StorePath, case: Expect[OrthogonalSelection, None] +) -> None: + """oindex with a 1D boolean mask matches numpy across chunk boundaries.""" + a = np.arange(30, dtype=int) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(7,)) + _test_get_orthogonal_selection(a, z, case.input) + + +@pytest.mark.parametrize("case", _ORTHO_1D_BOOL_BAD_CASES, ids=lambda c: c.id) +def test_get_orthogonal_selection_1d_bool_raises(store: StorePath, case: ExpectFail[Any]) -> None: + """oindex rejects masks of the wrong length or dimensionality with IndexError.""" + a = np.arange(30, dtype=int) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(7,)) + with pytest.raises(case.exception, match=case.msg): + z.oindex[case.input] # noinspection PyStatementEffect From db5588587f1f736b7dd4747996c18e98462b0b7d Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 22 May 2026 19:46:28 +0200 Subject: [PATCH 09/27] docs: add Task 0.6 (optional ExpectFail.msg + raises helper) Structural fix for the msg="" footgun discovered during Task 1.1: msg becomes optional and regex-matched via a case.raises() helper, with an escape flag. Co-Authored-By: Claude Opus 4.7 (1M context) --- .../plans/2026-05-22-indexing-test-cleanup.md | 128 ++++++++++++++++-- 1 file changed, 118 insertions(+), 10 deletions(-) diff --git a/docs/superpowers/plans/2026-05-22-indexing-test-cleanup.md b/docs/superpowers/plans/2026-05-22-indexing-test-cleanup.md index 50b0d1adc0..662a759027 100644 --- a/docs/superpowers/plans/2026-05-22-indexing-test-cleanup.md +++ b/docs/superpowers/plans/2026-05-22-indexing-test-cleanup.md @@ -264,6 +264,102 @@ EOF )" ``` +### Task 0.6: Make `ExpectFail.msg` optional with a `raises()` helper + +**Motivation:** `ExpectFail` requires a `msg: str`, and consumers do +`pytest.raises(case.exception, match=case.msg)`. Passing `msg=""` to express +"any message" triggers pytest's "matching against an empty string will always +pass" warning, which this repo escalates to an error via +`filterwarnings=["error"]`. Make "don't care about the message" expressible and +move the `match`/escape logic into one place. + +**Files:** +- Modify: `tests/conftest.py` (the `ExpectFail` dataclass) +- Modify all 9 consumer sites: `tests/test_chunk_grids.py` (2), + `tests/test_metadata/test_v3.py` (2), `tests/test_codecs/test_cast_value.py` + (2), `tests/test_codecs/test_scale_offset.py` (2), `tests/test_indexing.py` (1) + +- [ ] **Step 1: Add optional `msg`/`escape` and a `raises()` method to `ExpectFail`** + +```python +import re +from contextlib import AbstractContextManager + +import pytest + + +@dataclass(frozen=True) +class ExpectFail[TIn]: + """A test case that should raise an exception. + + `msg` is treated as a regex matched against the exception text (pytest's + native `match=` semantics). Leave it `None` to assert only the exception + type. Set `escape=True` when `msg` is a literal containing regex + metacharacters. + """ + + input: TIn + exception: type[Exception] + id: str + msg: str | None = None + escape: bool = False + + def raises(self) -> AbstractContextManager[pytest.ExceptionInfo[Exception]]: + if self.msg is None: + return pytest.raises(self.exception) + pattern = re.escape(self.msg) if self.escape else self.msg + return pytest.raises(self.exception, match=pattern) +``` + +(Place `import re` and `import pytest` with the existing imports if not already +present; `pytest` is certainly already imported in conftest.) + +- [ ] **Step 2: Update all 9 consumer sites** to use the helper. Replace each + `with pytest.raises(case.exception, match=re.escape(case.msg)):` and + `with pytest.raises(case.exception, match=case.msg):` with: + +```python + with case.raises(): +``` + +For the two `tests/test_chunk_grids.py` sites that used `re.escape(case.msg)`: +those three cases whose `msg` contains regex metacharacters +(`non-integer element(s) ([3, 3],) at indices (0,)` and the two like it) must +set `escape=True` on the `ExpectFail(...)` so the helper escapes them. All +other chunk_grids cases stay `escape=False` (default). Remove the now-unused +`import re` from `test_chunk_grids.py` if nothing else uses it. + +For `tests/test_metadata/test_v3.py`: the `msg=".*"` case (match-anything) may +become `msg=None` (drop the field) — behavior-preserving. The +`msg="dimension_names.*shape"` case keeps its regex `msg` with `escape=False`. + +- [ ] **Step 3: Run all affected suites** + +Run: `uv run --frozen pytest tests/test_chunk_grids.py tests/test_codecs/ tests/test_metadata/test_v3.py tests/test_indexing.py -q` +Expected: PASS, same counts as before this task (the matching behavior is +unchanged for every existing case; only the empty-string footgun is removed). + +- [ ] **Step 4: prek** + +Run: `prek run --all-files` +Expected: all hooks pass. + +- [ ] **Step 5: Commit** + +```bash +git add -A +git commit -m "$(cat <<'EOF' +test: make ExpectFail.msg optional with a raises() helper + +msg defaults to None (assert exception type only) and is treated as a regex, +with an escape flag for literal messages. Removes the msg="" footgun that the +repo's filterwarnings=["error"] config turned into a failure. + +Co-Authored-By: Claude Opus 4.7 (1M context) +EOF +)" +``` + --- ## Part 1 — Rewrite the indexing test families @@ -303,9 +399,17 @@ This task is the worked template. Tasks 1.2+ apply the same recipe and reference the kept oracle helper with `case.input`. 4. Extract the bundled `pytest.raises(IndexError)` block into a separate `test__raises` parametrized over an `ExpectFail[Selection]` list - `__BAD_CASES` (each with `exception=IndexError`, an `id`, and a `msg` - regex — use `msg=""` to match any `IndexError` message where the original - only asserted the type). + `__BAD_CASES` (each with `exception=IndexError`, an `id`, and an + optional `msg`). The raises test body uses the `case.raises()` helper + (added in Task 0.6): + ```python + with case.raises(): + z.oindex[case.input] + ``` + `msg` is a regex matched against the error text; prefer a stable substring + of the real message (run the bad selection once to capture it). Omit `msg` + (leave it `None`) to assert only the exception type. Set `escape=True` if a + literal `msg` contains regex metacharacters. Never pass `msg=""`. 5. Give both tests docstrings stating the behavior. **Files:** @@ -329,15 +433,17 @@ _ORTHO_1D_BOOL_CASES: list[Expect[OrthogonalSelection, None]] = [ ] _ORTHO_1D_BOOL_BAD_CASES: list[ExpectFail[Any]] = [ - ExpectFail(input=np.zeros(5, dtype=bool), exception=IndexError, id="mask-too-short", msg=""), - ExpectFail(input=np.zeros(50, dtype=bool), exception=IndexError, id="mask-too-long", msg=""), + ExpectFail(input=np.zeros(5, dtype=bool), exception=IndexError, id="mask-too-short", msg="wrong length for dimension; expected 30, got 5"), + ExpectFail(input=np.zeros(50, dtype=bool), exception=IndexError, id="mask-too-long", msg="wrong length for dimension; expected 30, got 50"), ExpectFail( input=[[True, False], [False, True]], exception=IndexError, id="mask-too-many-dims", - msg="", + msg="must be 1-dimensional only", ), ] +# msg is an optional regex matched against the real error text; omit it to +# assert only the exception type. The raises test body uses `with case.raises():`. @pytest.mark.parametrize("case", _ORTHO_1D_BOOL_CASES, ids=lambda c: c.id) @@ -355,7 +461,7 @@ def test_get_orthogonal_selection_1d_bool_raises( """oindex rejects masks of the wrong length or dimensionality with IndexError.""" a = np.arange(30, dtype=int) z = zarr_array_from_numpy_array(store, a, chunk_shape=(7,)) - with pytest.raises(case.exception, match=case.msg): + with case.raises(): z.oindex[case.input] ``` @@ -650,9 +756,11 @@ EOF - **`Selection` types** (`BasicSelection`, `OrthogonalSelection`, `CoordinateSelection`, `Selection`) are already imported at the top of `tests/test_indexing.py` — use them in the `Expect[...]` annotations. -- **`msg=""`** in `ExpectFail` matches any message (the original error tests - only asserted `IndexError`, not text). Don't invent message regexes that the - code doesn't actually emit. +- **Error paths use `case.raises()`** (the `ExpectFail` helper from Task 0.6), + not a bare `pytest.raises`. `msg` is an optional regex; use a stable + substring of the *actual* error (run the bad selection once to capture it), + or omit it to assert only the exception type. Never pass `msg=""`. Use + `escape=True` for literal messages containing regex metacharacters. - **Keep the oracle helpers.** Do not inline `_test_get_orthogonal_selection` etc. into the parametrized tests; calling them once per case is the design. - If a rewritten happy-path case turns out to disagree with numpy (the oracle From 967ff216d58a41c3485f1627cb76b4da834ece14 Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 22 May 2026 19:49:55 +0200 Subject: [PATCH 10/27] test: make ExpectFail.msg optional with a raises() helper msg defaults to None (assert exception type only) and is treated as a regex, with an escape flag for literal messages. Removes the msg="" footgun that the repo's filterwarnings=["error"] config turned into a failure. Co-Authored-By: Claude Opus 4.7 (1M context) --- tests/conftest.py | 19 +++++++++++++++++-- tests/test_chunk_grids.py | 9 ++++++--- tests/test_codecs/test_cast_value.py | 4 ++-- tests/test_codecs/test_scale_offset.py | 4 ++-- tests/test_indexing.py | 2 +- tests/test_metadata/test_v3.py | 5 ++--- 6 files changed, 30 insertions(+), 13 deletions(-) diff --git a/tests/conftest.py b/tests/conftest.py index 5af4aa21de..9a3819814c 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -3,6 +3,7 @@ import math import os import pathlib +import re import sys from collections.abc import Mapping, Sequence from dataclasses import dataclass, field @@ -50,6 +51,7 @@ if TYPE_CHECKING: from collections.abc import Generator + from contextlib import AbstractContextManager from typing import Any, Literal from _pytest.compat import LEGACY_PATH @@ -75,12 +77,25 @@ class Expect[TIn, TOut]: @dataclass(frozen=True) class ExpectFail[TIn]: - """A test case that should raise an exception.""" + """A test case that should raise an exception. + + `msg` is a regex matched against the exception text (pytest's native + `match=` semantics). Leave it `None` to assert only the exception type. Set + `escape=True` when `msg` is a literal that contains regex metacharacters + such as `(`, `[`, or `.`; `escape` has no effect when `msg` is `None`. + """ input: TIn exception: type[Exception] id: str - msg: str + msg: str | None = None + escape: bool = False + + def raises(self) -> AbstractContextManager[pytest.ExceptionInfo[Exception]]: + if self.msg is None: + return pytest.raises(self.exception) + pattern = re.escape(self.msg) if self.escape else self.msg + return pytest.raises(self.exception, match=pattern) async def parse_store( diff --git a/tests/test_chunk_grids.py b/tests/test_chunk_grids.py index cbdbc50766..b730a43901 100644 --- a/tests/test_chunk_grids.py +++ b/tests/test_chunk_grids.py @@ -1,4 +1,3 @@ -import re from typing import Any import numpy as np @@ -162,6 +161,7 @@ def test_chunk_layout_nested() -> None: exception=TypeError, id="rle-single-dim", msg="non-integer element(s) ([3, 3],) at indices (0,)", + escape=True, ), # Multiple non-int elements: all offending indices reported. ExpectFail( @@ -169,6 +169,7 @@ def test_chunk_layout_nested() -> None: exception=TypeError, id="multiple-non-ints", msg="non-integer element(s) ([2, 2], [3]) at indices (1, 3)", + escape=True, ), # Strings are non-integers and should be reported the same way. ExpectFail( @@ -176,6 +177,7 @@ def test_chunk_layout_nested() -> None: exception=TypeError, id="string-element", msg="non-integer element(s) ('3',) at indices (1,)", + escape=True, ), ], ids=lambda c: c.id, @@ -183,7 +185,7 @@ def test_chunk_layout_nested() -> None: def test_normalize_chunks_1d_errors(case: ExpectFail[tuple[Any, int]]) -> None: """Invalid 1D chunk specifications are rejected with informative error messages.""" chunks, span = case.input - with pytest.raises(case.exception, match=re.escape(case.msg)): + with case.raises(): normalize_chunks_1d(chunks, span=span) @@ -217,6 +219,7 @@ def test_normalize_chunks_1d_errors(case: ExpectFail[tuple[Any, int]]) -> None: exception=TypeError, id="rle-inner-dim", msg="non-integer element(s) ([3, 3],) at indices (0,)", + escape=True, ), ], ids=lambda c: c.id, @@ -224,7 +227,7 @@ def test_normalize_chunks_1d_errors(case: ExpectFail[tuple[Any, int]]) -> None: def test_normalize_chunks_nd_errors(case: ExpectFail[tuple[Any, tuple[int, ...]]]) -> None: """Invalid N-D chunk specifications are rejected with informative error messages.""" chunks, shape = case.input - with pytest.raises(case.exception, match=re.escape(case.msg)): + with case.raises(): normalize_chunks_nd(chunks, shape) diff --git a/tests/test_codecs/test_cast_value.py b/tests/test_codecs/test_cast_value.py index 2dd7b2088b..c43edb76e8 100644 --- a/tests/test_codecs/test_cast_value.py +++ b/tests/test_codecs/test_cast_value.py @@ -154,7 +154,7 @@ def test_construction_rejects_invalid_target_dtype() -> None: ) def test_validation_rejects_invalid(case: ExpectFail[dict[str, Any]]) -> None: """Invalid dtype or out_of_range combinations are rejected at array creation.""" - with pytest.raises(case.exception, match=case.msg): + with case.raises(): zarr.create_array( store={}, shape=(10,), @@ -410,7 +410,7 @@ def test_scalar_map_validation_rejects_invalid(case: ExpectFail[dict[str, Any]]) """Invalid scalar_map entries are rejected at array creation.""" import zarr - with pytest.raises(case.exception, match=case.msg): + with case.raises(): zarr.create_array( store={}, shape=(10,), diff --git a/tests/test_codecs/test_scale_offset.py b/tests/test_codecs/test_scale_offset.py index 486f6b3d36..513caf463a 100644 --- a/tests/test_codecs/test_scale_offset.py +++ b/tests/test_codecs/test_scale_offset.py @@ -100,7 +100,7 @@ def test_serialization_roundtrip() -> None: ) def test_construction_rejects_non_numeric(case: ExpectFail[dict[str, Any]]) -> None: """Non-numeric offset or scale is rejected at construction time.""" - with pytest.raises(case.exception, match=case.msg): + with case.raises(): ScaleOffset(**case.input) @@ -291,7 +291,7 @@ def test_rejects_zero_scale() -> None: def test_rejects_unrepresentable_scale_offset(case: ExpectFail[dict[str, Any]]) -> None: """Scale/offset values that can't be represented in the array dtype are rejected.""" - with pytest.raises(case.exception, match=case.msg): + with case.raises(): zarr.create_array( store={}, shape=(10,), diff --git a/tests/test_indexing.py b/tests/test_indexing.py index 915d681689..20a71ff4e3 100644 --- a/tests/test_indexing.py +++ b/tests/test_indexing.py @@ -661,7 +661,7 @@ def test_get_orthogonal_selection_1d_bool_raises(store: StorePath, case: ExpectF """oindex rejects masks of the wrong length or dimensionality with IndexError.""" a = np.arange(30, dtype=int) z = zarr_array_from_numpy_array(store, a, chunk_shape=(7,)) - with pytest.raises(case.exception, match=case.msg): + with case.raises(): z.oindex[case.input] diff --git a/tests/test_metadata/test_v3.py b/tests/test_metadata/test_v3.py index 07e06cd3fe..5c30fde76a 100644 --- a/tests/test_metadata/test_v3.py +++ b/tests/test_metadata/test_v3.py @@ -281,7 +281,6 @@ def test_array_metadata_roundtrip(case: Expect[dict[str, Any], dict[str, Any]]) ExpectFail( input={"data_type": "uint8", "fill_value": {}}, exception=TypeError, - msg=".*", id="invalid_fill_value_type", ), ], @@ -290,7 +289,7 @@ def test_array_metadata_roundtrip(case: Expect[dict[str, Any], dict[str, Any]]) def test_array_metadata_from_dict_fails(case: ExpectFail[dict[str, Any]]) -> None: """from_dict rejects invalid metadata documents.""" d = minimal_metadata_dict_v3(**case.input) - with pytest.raises(case.exception, match=case.msg): + with case.raises(): ArrayV3Metadata.from_dict(d) # type: ignore[arg-type] @@ -314,7 +313,7 @@ def test_array_metadata_from_dict_fails(case: ExpectFail[dict[str, Any]]) -> Non ) def test_array_metadata_extra_fields_rejected(case: ExpectFail[dict[str, Any]]) -> None: """from_dict rejects extra fields that don't conform to the spec.""" - with pytest.raises(case.exception, match=case.msg): + with case.raises(): ArrayV3Metadata.from_dict(case.input) From f51b0ec83f8520ddc11f08bde109e7bc62985edf Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 22 May 2026 20:03:30 +0200 Subject: [PATCH 11/27] test: clearer single-true mask idiom in orthogonal 1d exemplar np.arange(30) == 7 reads better than np.eye(1, 30, 7)[0]; matters because the other indexing families copy this exemplar. Co-Authored-By: Claude Opus 4.7 (1M context) --- docs/superpowers/plans/2026-05-22-indexing-test-cleanup.md | 2 +- tests/test_indexing.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/superpowers/plans/2026-05-22-indexing-test-cleanup.md b/docs/superpowers/plans/2026-05-22-indexing-test-cleanup.md index 662a759027..88ad160c7c 100644 --- a/docs/superpowers/plans/2026-05-22-indexing-test-cleanup.md +++ b/docs/superpowers/plans/2026-05-22-indexing-test-cleanup.md @@ -424,7 +424,7 @@ _ORTHO_1D_BOOL_CASES: list[Expect[OrthogonalSelection, None]] = [ Expect(input=np.zeros(30, dtype=bool), output=None, id="empty-mask"), Expect(input=np.ones(30, dtype=bool), output=None, id="full-mask"), Expect(input=np.arange(30) % 2 == 0, output=None, id="alternating-mask"), - Expect(input=np.eye(1, 30, 7, dtype=bool)[0], output=None, id="single-true"), + Expect(input=np.arange(30) == 7, output=None, id="single-true"), Expect( input=np.isin(np.arange(30), [0, 1, 8, 15, 29]), output=None, diff --git a/tests/test_indexing.py b/tests/test_indexing.py index 20a71ff4e3..37e6cef3ce 100644 --- a/tests/test_indexing.py +++ b/tests/test_indexing.py @@ -616,7 +616,7 @@ def _test_get_orthogonal_selection( Expect(input=np.zeros(30, dtype=bool), output=None, id="empty-mask"), Expect(input=np.ones(30, dtype=bool), output=None, id="full-mask"), Expect(input=np.arange(30) % 2 == 0, output=None, id="alternating-mask"), - Expect(input=np.eye(1, 30, 7, dtype=bool)[0], output=None, id="single-true"), + Expect(input=np.arange(30) == 7, output=None, id="single-true"), Expect( input=np.isin(np.arange(30), [0, 1, 8, 15, 29]), output=None, From 524e88e460db47ad18dec4ca92be70b91ad39312 Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 22 May 2026 20:07:39 +0200 Subject: [PATCH 12/27] test: rewrite orthogonal 1d int indexing as parametrized cases Co-Authored-By: Claude Opus 4.7 (1M context) --- tests/test_indexing.py | 76 +++++++++++++++++++++++++----------------- 1 file changed, 46 insertions(+), 30 deletions(-) diff --git a/tests/test_indexing.py b/tests/test_indexing.py index 37e6cef3ce..ee20e04201 100644 --- a/tests/test_indexing.py +++ b/tests/test_indexing.py @@ -665,39 +665,55 @@ def test_get_orthogonal_selection_1d_bool_raises(store: StorePath, case: ExpectF z.oindex[case.input] -# noinspection PyStatementEffect -def test_get_orthogonal_selection_1d_int(store: StorePath) -> None: - # setup - a = np.arange(550, dtype=int) - z = zarr_array_from_numpy_array(store, a, chunk_shape=(100,)) +_ORTHO_1D_INT_CASES: list[Expect[OrthogonalSelection, None]] = [ + Expect(input=[0, 8, 15, 29], output=None, id="sorted"), + Expect(input=[3, 29, 1, 16], output=None, id="unsorted"), + Expect(input=[2, 2, 8, 8], output=None, id="duplicates"), + Expect(input=[0, 3, 10, -23, -12, -1], output=None, id="wraparound"), + Expect(input=[15], output=None, id="single"), +] - np.random.seed(42) - # test with different degrees of sparseness - for p in 0.5, 0.01: - # sorted integer arrays - ix = np.random.choice(a.shape[0], size=int(a.shape[0] * p), replace=True) - ix.sort() - _test_get_orthogonal_selection(a, z, ix) +_ORTHO_1D_INT_BAD_CASES: list[ExpectFail[Any]] = [ + ExpectFail( + input=[31], + exception=IndexError, + id="out-of-bounds-high", + msg="index out of bounds for dimension with length 30", + ), + ExpectFail( + input=[-31], + exception=IndexError, + id="out-of-bounds-low", + msg="index out of bounds for dimension with length 30", + ), + ExpectFail( + input=[[2, 4], [6, 8]], + exception=IndexError, + id="too-many-dims", + msg="integer arrays in an orthogonal selection must be 1-dimensional only", + ), +] - selections = basic_selections_1d + [ - # test wraparound - [0, 3, 10, -23, -12, -1], - # explicit test not sorted - [3, 105, 23, 127], - ] - for selection in selections: - _test_get_orthogonal_selection(a, z, selection) - bad_selections = basic_selections_1d_bad + [ - [a.shape[0] + 1], # out of bounds - [-(a.shape[0] + 1)], # out of bounds - [[2, 4], [6, 8]], # too many dimensions - ] - for bad_selection in bad_selections: - with pytest.raises(IndexError): - z.get_orthogonal_selection(bad_selection) # type: ignore[arg-type] - with pytest.raises(IndexError): - z.oindex[bad_selection] # type: ignore[index] +@pytest.mark.parametrize("case", _ORTHO_1D_INT_CASES, ids=lambda c: c.id) +def test_get_orthogonal_selection_1d_int( + store: StorePath, case: Expect[OrthogonalSelection, None] +) -> None: + """oindex with a 1D integer array matches numpy, including wraparound and duplicates.""" + a = np.arange(30, dtype=int) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(7,)) + _test_get_orthogonal_selection(a, z, case.input) + + +@pytest.mark.parametrize("case", _ORTHO_1D_INT_BAD_CASES, ids=lambda c: c.id) +def test_get_orthogonal_selection_1d_int_raises(store: StorePath, case: ExpectFail[Any]) -> None: + """oindex rejects out-of-bounds or multi-dimensional integer selections with IndexError.""" + a = np.arange(30, dtype=int) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(7,)) + with case.raises(): + z.get_orthogonal_selection(case.input) + with case.raises(): + z.oindex[case.input] def _test_get_orthogonal_selection_2d( From 33cbf0a26adc606b55f51ada46ed4b79d857c4ae Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 22 May 2026 20:12:51 +0200 Subject: [PATCH 13/27] test: rewrite orthogonal 2d indexing as parametrized cases Co-Authored-By: Claude Opus 4.7 (1M context) --- tests/test_indexing.py | 123 ++++++++++++++++++++++++----------------- 1 file changed, 72 insertions(+), 51 deletions(-) diff --git a/tests/test_indexing.py b/tests/test_indexing.py index ee20e04201..a3bd37f2cb 100644 --- a/tests/test_indexing.py +++ b/tests/test_indexing.py @@ -716,62 +716,83 @@ def test_get_orthogonal_selection_1d_int_raises(store: StorePath, case: ExpectFa z.oindex[case.input] -def _test_get_orthogonal_selection_2d( - a: npt.NDArray[Any], z: Array, ix0: npt.NDArray[np.bool], ix1: npt.NDArray[np.bool] -) -> None: - selections = [ - # index both axes with array - (ix0, ix1), - # mixed indexing with array / slice - (ix0, slice(1, 5)), - (ix0, slice(1, 5, 2)), - (slice(250, 350), ix1), - (slice(250, 350, 10), ix1), - # mixed indexing with array / int - (ix0, 4), - (42, ix1), - ] - for selection in selections: - _test_get_orthogonal_selection(a, z, selection) - - -# noinspection PyStatementEffect -def test_get_orthogonal_selection_2d(store: StorePath) -> None: - # setup - a = np.arange(5400, dtype=int).reshape(600, 9) - z = zarr_array_from_numpy_array(store, a, chunk_shape=(300, 3)) +_ORTHO_2D_IX0_BOOL = np.isin(np.arange(12), [0, 5, 11]) # rows 0, 5, 11 +_ORTHO_2D_IX1_BOOL = np.array([True, False, True, False, True]) # cols 0, 2, 4 +_ORTHO_2D_IX0_INT = np.array([0, 5, 11]) +_ORTHO_2D_IX1_INT = np.array([0, 2, 4]) + +_ORTHO_2D_CASES: list[Expect[OrthogonalSelection, None]] = [ + Expect(input=(_ORTHO_2D_IX0_BOOL, _ORTHO_2D_IX1_BOOL), output=None, id="both-bool"), + Expect(input=(_ORTHO_2D_IX0_BOOL, slice(1, 4)), output=None, id="bool-slice"), + Expect(input=(_ORTHO_2D_IX0_BOOL, slice(0, 5, 2)), output=None, id="bool-strided-slice"), + Expect(input=(slice(2, 9), _ORTHO_2D_IX1_BOOL), output=None, id="slice-bool"), + Expect(input=(slice(0, 12, 4), _ORTHO_2D_IX1_BOOL), output=None, id="strided-slice-bool"), + Expect(input=(_ORTHO_2D_IX0_BOOL, 3), output=None, id="bool-int"), + Expect(input=(7, _ORTHO_2D_IX1_BOOL), output=None, id="int-bool"), + Expect(input=(_ORTHO_2D_IX0_INT, _ORTHO_2D_IX1_INT), output=None, id="both-int"), + Expect(input=(_ORTHO_2D_IX0_INT, _ORTHO_2D_IX1_BOOL), output=None, id="int-array-bool-array"), + Expect(input=(_ORTHO_2D_IX0_BOOL, _ORTHO_2D_IX1_INT), output=None, id="bool-array-int-array"), + Expect(input=7, output=None, id="single-row"), + Expect(input=(slice(None), 3), output=None, id="single-col"), + Expect(input=(slice(None), slice(None)), output=None, id="full"), + Expect(input=slice(2, 9), output=None, id="row-slice"), +] - np.random.seed(42) - # test with different degrees of sparseness - for p in 0.5, 0.01: - # boolean arrays - ix0 = np.random.binomial(1, p, size=a.shape[0]).astype(bool) - ix1 = np.random.binomial(1, 0.5, size=a.shape[1]).astype(bool) - _test_get_orthogonal_selection_2d(a, z, ix0, ix1) +_ORTHO_2D_BAD_CASES: list[ExpectFail[Any]] = [ + ExpectFail( + input=2.3, + exception=IndexError, + id="float-index", + msg="unsupported selection item for orthogonal indexing", + ), + # get_orthogonal_selection and oindex raise different messages for a string + # selection, so assert only the exception type. + ExpectFail( + input="foo", + exception=IndexError, + id="string-index", + msg=None, + ), + ExpectFail( + input=None, + exception=IndexError, + id="none-index", + msg="unsupported selection item for orthogonal indexing", + ), + ExpectFail( + input=slice(None, None, -1), + exception=IndexError, + id="negative-step", + msg="only slices with step >= 1 are supported", + ), + ExpectFail( + input=(0, 0, 0), + exception=IndexError, + id="too-many-dims", + msg="too many indices for array", + ), +] - # mixed int array / bool array - selections = ( - (ix0, np.nonzero(ix1)[0]), - (np.nonzero(ix0)[0], ix1), - ) - for selection in selections: - _test_get_orthogonal_selection(a, z, selection) - # sorted integer arrays - ix0 = np.random.choice(a.shape[0], size=int(a.shape[0] * p), replace=True) - ix1 = np.random.choice(a.shape[1], size=int(a.shape[1] * 0.5), replace=True) - ix0.sort() - ix1.sort() - _test_get_orthogonal_selection_2d(a, z, ix0, ix1) +@pytest.mark.parametrize("case", _ORTHO_2D_CASES, ids=lambda c: c.id) +def test_get_orthogonal_selection_2d( + store: StorePath, case: Expect[OrthogonalSelection, None] +) -> None: + """oindex on a 2D array matches numpy for array/slice/int combinations per axis.""" + a = np.arange(60, dtype=int).reshape(12, 5) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(5, 2)) + _test_get_orthogonal_selection(a, z, case.input) - for selection_2d in basic_selections_2d: - _test_get_orthogonal_selection(a, z, selection_2d) - for selection_2d_bad in basic_selections_2d_bad: - with pytest.raises(IndexError): - z.get_orthogonal_selection(selection_2d_bad) # type: ignore[arg-type] - with pytest.raises(IndexError): - z.oindex[selection_2d_bad] # type: ignore[index] +@pytest.mark.parametrize("case", _ORTHO_2D_BAD_CASES, ids=lambda c: c.id) +def test_get_orthogonal_selection_2d_raises(store: StorePath, case: ExpectFail[Any]) -> None: + """oindex on a 2D array rejects malformed selections with IndexError.""" + a = np.arange(60, dtype=int).reshape(12, 5) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(5, 2)) + with case.raises(): + z.get_orthogonal_selection(case.input) + with case.raises(): + z.oindex[case.input] def _test_get_orthogonal_selection_3d( From 9bf028ba0ef3700b50bc5ab00335c14bf6b6990e Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 22 May 2026 20:19:39 +0200 Subject: [PATCH 14/27] test: rewrite orthogonal 3d indexing as parametrized cases Co-Authored-By: Claude Opus 4.7 (1M context) --- tests/test_indexing.py | 145 ++++++++++++++++++++++++----------------- 1 file changed, 84 insertions(+), 61 deletions(-) diff --git a/tests/test_indexing.py b/tests/test_indexing.py index a3bd37f2cb..520c60ce78 100644 --- a/tests/test_indexing.py +++ b/tests/test_indexing.py @@ -795,69 +795,92 @@ def test_get_orthogonal_selection_2d_raises(store: StorePath, case: ExpectFail[A z.oindex[case.input] -def _test_get_orthogonal_selection_3d( - a: npt.NDArray, - z: Array, - ix0: npt.NDArray[np.bool], - ix1: npt.NDArray[np.bool], - ix2: npt.NDArray[np.bool], -) -> None: - selections = [ - # single value - (60, 15, 4), - (-1, -1, -1), - # index all axes with array - (ix0, ix1, ix2), - # mixed indexing with single array / slices - (ix0, slice(10, 20), slice(1, 5)), - (slice(30, 50), ix1, slice(1, 5)), - (slice(30, 50), slice(10, 20), ix2), - (ix0, slice(10, 20, 5), slice(1, 5, 2)), - (slice(30, 50, 3), ix1, slice(1, 5, 2)), - (slice(30, 50, 3), slice(10, 20, 5), ix2), - # mixed indexing with single array / ints - (ix0, 15, 4), - (60, ix1, 4), - (60, 15, ix2), - # mixed indexing with single array / slice / int - (ix0, slice(10, 20), 4), - (15, ix1, slice(1, 5)), - (slice(30, 50), 15, ix2), - # mixed indexing with two array / slice - (ix0, ix1, slice(1, 5)), - (slice(30, 50), ix1, ix2), - (ix0, slice(10, 20), ix2), - # mixed indexing with two array / integer - (ix0, ix1, 4), - (15, ix1, ix2), - (ix0, 15, ix2), - ] - for selection in selections: - _test_get_orthogonal_selection(a, z, selection) - - -def test_get_orthogonal_selection_3d(store: StorePath) -> None: - # setup - a = np.arange(32400, dtype=int).reshape(120, 30, 9) - z = zarr_array_from_numpy_array(store, a, chunk_shape=(60, 20, 3)) +_ORTHO_3D_IX0_BOOL = np.isin(np.arange(7), [0, 3, 6]) # axis 0 +_ORTHO_3D_IX1_BOOL = np.isin(np.arange(6), [0, 2, 5]) # axis 1 +_ORTHO_3D_IX2_BOOL = np.isin(np.arange(10), [0, 4, 9]) # axis 2 +_ORTHO_3D_IX0_INT = np.array([0, 3, 6]) +_ORTHO_3D_IX1_INT = np.array([0, 2, 5]) +_ORTHO_3D_IX2_INT = np.array([0, 4, 9]) + +_ORTHO_3D_CASES: list[Expect[OrthogonalSelection, None]] = [ + # single value + Expect(input=(5, 3, 8), output=None, id="single-value"), + Expect(input=(-1, -1, -1), output=None, id="all-negative"), + # index all axes with arrays + Expect( + input=(_ORTHO_3D_IX0_BOOL, _ORTHO_3D_IX1_BOOL, _ORTHO_3D_IX2_BOOL), + output=None, + id="three-bool-arrays", + ), + Expect( + input=(_ORTHO_3D_IX0_INT, _ORTHO_3D_IX1_INT, _ORTHO_3D_IX2_INT), + output=None, + id="three-int-arrays", + ), + # mixed indexing with single array / slices + Expect( + input=(_ORTHO_3D_IX0_BOOL, slice(1, 5), slice(2, 9)), output=None, id="array-slice-slice" + ), + Expect( + input=(slice(1, 6), _ORTHO_3D_IX1_BOOL, slice(2, 9)), output=None, id="slice-array-slice" + ), + Expect( + input=(slice(1, 6), slice(1, 5), _ORTHO_3D_IX2_BOOL), output=None, id="slice-slice-array" + ), + Expect( + input=(_ORTHO_3D_IX0_BOOL, slice(0, 6, 2), slice(0, 10, 3)), + output=None, + id="array-strided-strided", + ), + Expect( + input=(slice(0, 7, 2), _ORTHO_3D_IX1_BOOL, slice(0, 10, 3)), + output=None, + id="strided-array-strided", + ), + Expect( + input=(slice(0, 7, 2), slice(0, 6, 2), _ORTHO_3D_IX2_BOOL), + output=None, + id="strided-strided-array", + ), + # mixed indexing with single array / ints + Expect(input=(_ORTHO_3D_IX0_BOOL, 3, 8), output=None, id="array-int-int"), + Expect(input=(5, _ORTHO_3D_IX1_BOOL, 8), output=None, id="int-array-int"), + Expect(input=(5, 3, _ORTHO_3D_IX2_BOOL), output=None, id="int-int-array"), + # mixed indexing with single array / slice / int + Expect(input=(_ORTHO_3D_IX0_BOOL, slice(1, 5), 8), output=None, id="array-slice-int"), + Expect(input=(5, _ORTHO_3D_IX1_BOOL, slice(2, 9)), output=None, id="int-array-slice"), + Expect(input=(slice(1, 6), 3, _ORTHO_3D_IX2_BOOL), output=None, id="slice-int-array"), + # mixed indexing with two arrays / slice + Expect( + input=(_ORTHO_3D_IX0_BOOL, _ORTHO_3D_IX1_BOOL, slice(2, 9)), + output=None, + id="two-arrays-slice", + ), + Expect( + input=(slice(1, 6), _ORTHO_3D_IX1_BOOL, _ORTHO_3D_IX2_BOOL), + output=None, + id="slice-two-arrays", + ), + Expect( + input=(_ORTHO_3D_IX0_BOOL, slice(1, 5), _ORTHO_3D_IX2_BOOL), + output=None, + id="array-slice-array", + ), + # mixed indexing with two arrays / integer + Expect(input=(_ORTHO_3D_IX0_BOOL, _ORTHO_3D_IX1_BOOL, 8), output=None, id="two-arrays-int"), + Expect(input=(5, _ORTHO_3D_IX1_BOOL, _ORTHO_3D_IX2_BOOL), output=None, id="int-two-arrays"), + Expect(input=(_ORTHO_3D_IX0_BOOL, 3, _ORTHO_3D_IX2_BOOL), output=None, id="array-int-array"), +] - np.random.seed(42) - # test with different degrees of sparseness - for p in 0.5, 0.01: - # boolean arrays - ix0 = np.random.binomial(1, p, size=a.shape[0]).astype(bool) - ix1 = np.random.binomial(1, 0.5, size=a.shape[1]).astype(bool) - ix2 = np.random.binomial(1, 0.5, size=a.shape[2]).astype(bool) - _test_get_orthogonal_selection_3d(a, z, ix0, ix1, ix2) - # sorted integer arrays - ix0 = np.random.choice(a.shape[0], size=int(a.shape[0] * p), replace=True) - ix1 = np.random.choice(a.shape[1], size=int(a.shape[1] * 0.5), replace=True) - ix2 = np.random.choice(a.shape[2], size=int(a.shape[2] * 0.5), replace=True) - ix0.sort() - ix1.sort() - ix2.sort() - _test_get_orthogonal_selection_3d(a, z, ix0, ix1, ix2) +@pytest.mark.parametrize("case", _ORTHO_3D_CASES, ids=lambda c: c.id) +def test_get_orthogonal_selection_3d( + store: StorePath, case: Expect[OrthogonalSelection, None] +) -> None: + """oindex on a 3D array matches numpy for array/slice/int combinations per axis.""" + a = np.arange(420, dtype=int).reshape(7, 6, 10) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(3, 2, 4)) + _test_get_orthogonal_selection(a, z, case.input) def test_orthogonal_indexing_edge_cases(store: StorePath) -> None: From 482f7b40b56f5fa3b26c375f77cf7f93e467ace7 Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 22 May 2026 20:32:52 +0200 Subject: [PATCH 15/27] test: rewrite set-orthogonal indexing family as parametrized cases Reuses the get-orthogonal case tables; deletes the per-dimensionality set helpers. Co-Authored-By: Claude Opus 4.7 (1M context) --- tests/test_indexing.py | 139 +++++++---------------------------------- 1 file changed, 23 insertions(+), 116 deletions(-) diff --git a/tests/test_indexing.py b/tests/test_indexing.py index 520c60ce78..fbea817cab 100644 --- a/tests/test_indexing.py +++ b/tests/test_indexing.py @@ -916,27 +916,15 @@ def _test_set_orthogonal_selection( assert_array_equal(a, z[:]) -def test_set_orthogonal_selection_1d(store: StorePath) -> None: - # setup - v = np.arange(550, dtype=int) - a = np.empty(v.shape, dtype=int) - z = zarr_array_from_numpy_array(store, a, chunk_shape=(100,)) - - # test with different degrees of sparseness - np.random.seed(42) - for p in 0.5, 0.01: - # boolean arrays - ix = np.random.binomial(1, p, size=a.shape[0]).astype(bool) - _test_set_orthogonal_selection(v, a, z, ix) - - # sorted integer arrays - ix = np.random.choice(a.shape[0], size=int(a.shape[0] * p), replace=True) - ix.sort() - _test_set_orthogonal_selection(v, a, z, ix) - - # basic selections - for selection in basic_selections_1d: - _test_set_orthogonal_selection(v, a, z, selection) +@pytest.mark.parametrize("case", _ORTHO_1D_BOOL_CASES + _ORTHO_1D_INT_CASES, ids=lambda c: c.id) +def test_set_orthogonal_selection_1d( + store: StorePath, case: Expect[OrthogonalSelection, None] +) -> None: + """set_orthogonal_selection on a 1D array round-trips through numpy for masks and int arrays.""" + v = np.arange(30, dtype=int) + a = np.empty_like(v) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(7,)) + _test_set_orthogonal_selection(v, a, z, case.input) def test_set_item_1d_last_two_chunks(store: StorePath): @@ -958,107 +946,26 @@ def test_set_item_1d_last_two_chunks(store: StorePath): np.testing.assert_equal(z["zoo"][()], np.array(1)) -def _test_set_orthogonal_selection_2d( - v: npt.NDArray[np.int_], - a: npt.NDArray[np.int_], - z: Array, - ix0: npt.NDArray[np.bool], - ix1: npt.NDArray[np.bool], +@pytest.mark.parametrize("case", _ORTHO_2D_CASES, ids=lambda c: c.id) +def test_set_orthogonal_selection_2d( + store: StorePath, case: Expect[OrthogonalSelection, None] ) -> None: - selections = [ - # index both axes with array - (ix0, ix1), - # mixed indexing with array / slice or int - (ix0, slice(1, 5)), - (slice(250, 350), ix1), - (ix0, 4), - (42, ix1), - ] - for selection in selections: - _test_set_orthogonal_selection(v, a, z, selection) - - -def test_set_orthogonal_selection_2d(store: StorePath) -> None: - # setup - v = np.arange(5400, dtype=int).reshape(600, 9) + """set_orthogonal_selection on a 2D array round-trips through numpy.""" + v = np.arange(60, dtype=int).reshape(12, 5) a = np.empty_like(v) - z = zarr_array_from_numpy_array(store, a, chunk_shape=(300, 3)) - - np.random.seed(42) - # test with different degrees of sparseness - for p in 0.5, 0.01: - # boolean arrays - ix0 = np.random.binomial(1, p, size=a.shape[0]).astype(bool) - ix1 = np.random.binomial(1, 0.5, size=a.shape[1]).astype(bool) - _test_set_orthogonal_selection_2d(v, a, z, ix0, ix1) - - # sorted integer arrays - ix0 = np.random.choice(a.shape[0], size=int(a.shape[0] * p), replace=True) - ix1 = np.random.choice(a.shape[1], size=int(a.shape[1] * 0.5), replace=True) - ix0.sort() - ix1.sort() - _test_set_orthogonal_selection_2d(v, a, z, ix0, ix1) - - for selection in basic_selections_2d: - _test_set_orthogonal_selection(v, a, z, selection) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(5, 2)) + _test_set_orthogonal_selection(v, a, z, case.input) -def _test_set_orthogonal_selection_3d( - v: npt.NDArray[np.int_], - a: npt.NDArray[np.int_], - z: Array, - ix0: npt.NDArray[np.bool], - ix1: npt.NDArray[np.bool], - ix2: npt.NDArray[np.bool], +@pytest.mark.parametrize("case", _ORTHO_3D_CASES, ids=lambda c: c.id) +def test_set_orthogonal_selection_3d( + store: StorePath, case: Expect[OrthogonalSelection, None] ) -> None: - selections = ( - # single value - (60, 15, 4), - (-1, -1, -1), - # index all axes with bool array - (ix0, ix1, ix2), - # mixed indexing with single bool array / slice or int - (ix0, slice(10, 20), slice(1, 5)), - (slice(30, 50), ix1, slice(1, 5)), - (slice(30, 50), slice(10, 20), ix2), - (ix0, 15, 4), - (60, ix1, 4), - (60, 15, ix2), - (ix0, slice(10, 20), 4), - (slice(30, 50), ix1, 4), - (slice(30, 50), 15, ix2), - # indexing with two arrays / slice - (ix0, ix1, slice(1, 5)), - # indexing with two arrays / integer - (ix0, ix1, 4), - ) - for selection in selections: - _test_set_orthogonal_selection(v, a, z, selection) - - -def test_set_orthogonal_selection_3d(store: StorePath) -> None: - # setup - v = np.arange(32400, dtype=int).reshape(120, 30, 9) + """set_orthogonal_selection on a 3D array round-trips through numpy.""" + v = np.arange(420, dtype=int).reshape(7, 6, 10) a = np.empty_like(v) - z = zarr_array_from_numpy_array(store, a, chunk_shape=(60, 20, 3)) - - np.random.seed(42) - # test with different degrees of sparseness - for p in 0.5, 0.01: - # boolean arrays - ix0 = np.random.binomial(1, p, size=a.shape[0]).astype(bool) - ix1 = np.random.binomial(1, 0.5, size=a.shape[1]).astype(bool) - ix2 = np.random.binomial(1, 0.5, size=a.shape[2]).astype(bool) - _test_set_orthogonal_selection_3d(v, a, z, ix0, ix1, ix2) - - # sorted integer arrays - ix0 = np.random.choice(a.shape[0], size=int(a.shape[0] * p), replace=True) - ix1 = np.random.choice(a.shape[1], size=int(a.shape[1] * 0.5), replace=True) - ix2 = np.random.choice(a.shape[2], size=int(a.shape[2] * 0.5), replace=True) - ix0.sort() - ix1.sort() - ix2.sort() - _test_set_orthogonal_selection_3d(v, a, z, ix0, ix1, ix2) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(3, 2, 4)) + _test_set_orthogonal_selection(v, a, z, case.input) def test_orthogonal_indexing_fallback_on_get_setitem(store: StorePath) -> None: From 0cec6a84809e58a6938e8765d8165d1b1350e160 Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 22 May 2026 20:45:59 +0200 Subject: [PATCH 16/27] test: rewrite basic 1d/2d selection families as parametrized cases Co-Authored-By: Claude Opus 4.7 (1M context) --- tests/test_indexing.py | 304 +++++++++++++++++++++-------------------- 1 file changed, 156 insertions(+), 148 deletions(-) diff --git a/tests/test_indexing.py b/tests/test_indexing.py index fbea817cab..ea5487afe3 100644 --- a/tests/test_indexing.py +++ b/tests/test_indexing.py @@ -202,77 +202,68 @@ def test_get_basic_selection_0d(store: StorePath, use_out: bool, value: Any, dty # assert_array_equal(a[["foo", "bar"]], c) -basic_selections_1d: list[BasicSelection] = [ - # single value - 42, - -1, - # slices - slice(0, 1050), - slice(50, 150), - slice(0, 2000), - slice(-150, -50), - slice(-2000, 2000), - slice(0, 0), # empty result - slice(-1, 0), # empty result - # total selections - slice(None), - Ellipsis, - (), - (Ellipsis, slice(None)), - # slice with step - slice(None), - slice(None, None), - slice(None, None, 1), - slice(None, None, 10), - slice(None, None, 100), - slice(None, None, 1000), - slice(None, None, 10000), - slice(0, 1050), - slice(0, 1050, 1), - slice(0, 1050, 10), - slice(0, 1050, 100), - slice(0, 1050, 1000), - slice(0, 1050, 10000), - slice(1, 31, 3), - slice(1, 31, 30), - slice(1, 31, 300), - slice(81, 121, 3), - slice(81, 121, 30), - slice(81, 121, 300), - slice(50, 150), - slice(50, 150, 1), - slice(50, 150, 10), +_BASIC_1D_CASES: list[Expect[BasicSelection, None]] = [ + Expect(input=5, output=None, id="single-positive"), + Expect(input=-1, output=None, id="single-negative"), + Expect(input=slice(3, 18), output=None, id="bounded-slice"), + Expect(input=slice(0, 100), output=None, id="over-bounds-slice"), + Expect(input=slice(-18, -3), output=None, id="negative-slice"), + Expect(input=slice(0, 0), output=None, id="empty-slice"), + Expect(input=slice(-1, 0), output=None, id="empty-negative-slice"), + Expect(input=slice(None), output=None, id="full-slice"), + Expect(input=Ellipsis, output=None, id="ellipsis"), + Expect(input=(), output=None, id="empty-tuple"), + Expect(input=(Ellipsis, slice(None)), output=None, id="ellipsis-slice"), + Expect(input=slice(None, None, 3), output=None, id="stride-3"), + Expect(input=slice(3, 27, 5), output=None, id="bounded-stride"), ] -basic_selections_1d_bad = [ - # only positive step supported - slice(None, None, -1), - slice(None, None, -10), - slice(None, None, -100), - slice(None, None, -1000), - slice(None, None, -10000), - slice(1050, -1, -1), - slice(1050, -1, -10), - slice(1050, -1, -100), - slice(1050, -1, -1000), - slice(1050, -1, -10000), - slice(1050, 0, -1), - slice(1050, 0, -10), - slice(1050, 0, -100), - slice(1050, 0, -1000), - slice(1050, 0, -10000), - slice(150, 50, -1), - slice(150, 50, -10), - slice(31, 1, -3), - slice(121, 81, -3), - slice(-1, 0, -1), - # bad stuff - 2.3, - "foo", - b"xxx", - None, - (0, 0), - (slice(None), slice(None)), +_BASIC_1D_BAD_CASES: list[ExpectFail[Any]] = [ + ExpectFail( + input=slice(None, None, -1), + exception=IndexError, + id="negative-step", + msg="only slices with step >= 1 are supported", + ), + ExpectFail( + input=2.3, + exception=IndexError, + id="float", + msg="unsupported selection item for basic indexing; expected integer or slice, got ", + escape=True, + ), + ExpectFail( + input="foo", + exception=IndexError, + id="string", + msg=None, + ), + ExpectFail( + input=b"xxx", + exception=IndexError, + id="bytes", + msg="unsupported selection item for basic indexing; expected integer or slice, got ", + escape=True, + ), + ExpectFail( + input=None, + exception=IndexError, + id="none", + msg="unsupported selection item for basic indexing; expected integer or slice, got ", + escape=True, + ), + ExpectFail( + input=(0, 0), + exception=IndexError, + id="tuple-too-many", + msg="too many indices for array; expected 1, got 2", + ), + ExpectFail( + input=(slice(None), slice(None)), + exception=IndexError, + id="two-slices", + msg="too many indices for array; expected 1, got 2", + ), ] @@ -293,96 +284,113 @@ def _test_get_basic_selection( assert_array_equal(expect, b.as_numpy_array()) -# noinspection PyStatementEffect -def test_get_basic_selection_1d(store: StorePath) -> None: - # setup - a = np.arange(1050, dtype=int) - z = zarr_array_from_numpy_array(store, a, chunk_shape=(100,)) - - for selection in basic_selections_1d: - _test_get_basic_selection(a, z, selection) +@pytest.mark.parametrize("case", _BASIC_1D_CASES, ids=lambda c: c.id) +def test_get_basic_selection_1d(store: StorePath, case: Expect[BasicSelection, None]) -> None: + """Basic getitem on a 1D array matches numpy for ints, slices, strides, and full selections.""" + a = np.arange(30, dtype=int) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(7,)) + _test_get_basic_selection(a, z, case.input) - for selection_bad in basic_selections_1d_bad: - with pytest.raises(IndexError): - z.get_basic_selection(selection_bad) # type: ignore[arg-type] - with pytest.raises(IndexError): - z[selection_bad] # type: ignore[index] - with pytest.raises(IndexError): - z.get_basic_selection([1, 0]) # type: ignore[arg-type] +@pytest.mark.parametrize("case", _BASIC_1D_BAD_CASES, ids=lambda c: c.id) +def test_get_basic_selection_1d_raises(store: StorePath, case: ExpectFail[Any]) -> None: + """Basic getitem on a 1D array rejects negative steps and invalid index types with IndexError.""" + a = np.arange(30, dtype=int) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(7,)) + with case.raises(): + z.get_basic_selection(case.input) + with case.raises(): + z[case.input] -basic_selections_2d: list[BasicSelection] = [ - # single row - 42, - -1, - (42, slice(None)), - (-1, slice(None)), - # single col - (slice(None), 4), - (slice(None), -1), - # row slices - slice(None), - slice(0, 1000), - slice(250, 350), - slice(0, 2000), - slice(-350, -250), - slice(0, 0), # empty result - slice(-1, 0), # empty result - slice(-2000, 0), - slice(-2000, 2000), - # 2D slices - (slice(None), slice(1, 5)), - (slice(250, 350), slice(None)), - (slice(250, 350), slice(1, 5)), - (slice(250, 350), slice(-5, -1)), - (slice(250, 350), slice(-50, 50)), - (slice(250, 350, 10), slice(1, 5)), - (slice(250, 350), slice(1, 5, 2)), - (slice(250, 350, 33), slice(1, 5, 3)), - # total selections - (slice(None), slice(None)), - Ellipsis, - (), - (Ellipsis, slice(None)), - (Ellipsis, slice(None), slice(None)), +_BASIC_2D_CASES: list[Expect[BasicSelection, None]] = [ + Expect(input=5, output=None, id="single-row"), + Expect(input=-1, output=None, id="single-row-neg"), + Expect(input=(5, slice(None)), output=None, id="row-and-full-col"), + Expect(input=(slice(None), 3), output=None, id="single-col"), + Expect(input=(slice(None), -1), output=None, id="single-col-neg"), + Expect(input=slice(None), output=None, id="full"), + Expect(input=slice(2, 9), output=None, id="row-slice"), + Expect(input=slice(0, 0), output=None, id="empty-row-slice"), + Expect(input=(slice(2, 9), slice(1, 4)), output=None, id="2d-slice"), + Expect(input=(slice(0, 12, 3), slice(0, 5, 2)), output=None, id="strided-2d-slice"), + Expect(input=Ellipsis, output=None, id="ellipsis"), + Expect(input=(), output=None, id="empty-tuple"), ] -basic_selections_2d_bad = [ - # bad stuff - 2.3, - "foo", - b"xxx", - None, - (2.3, slice(None)), - # only positive step supported - slice(None, None, -1), - (slice(None, None, -1), slice(None)), - (0, 0, 0), - (slice(None), slice(None), slice(None)), +_BASIC_2D_BAD_CASES: list[ExpectFail[Any]] = [ + ExpectFail( + input=2.3, + exception=IndexError, + id="float", + msg="unsupported selection item for basic indexing; expected integer or slice, got ", + escape=True, + ), + ExpectFail( + input="foo", + exception=IndexError, + id="string", + msg="unsupported selection item for basic indexing; expected integer or slice, got ", + escape=True, + ), + ExpectFail( + input=None, + exception=IndexError, + id="none", + msg="unsupported selection item for basic indexing; expected integer or slice, got ", + escape=True, + ), + ExpectFail( + input=(2.3, slice(None)), + exception=IndexError, + id="float-in-tuple", + msg="unsupported selection item for basic indexing; expected integer or slice, got ", + escape=True, + ), + ExpectFail( + input=slice(None, None, -1), + exception=IndexError, + id="negative-step", + msg="only slices with step >= 1 are supported", + ), + ExpectFail( + input=(slice(None), slice(None), slice(None)), + exception=IndexError, + id="too-many-dims", + msg="too many indices for array; expected 2, got 3", + ), + ExpectFail( + input=[0, 1], + exception=IndexError, + id="integer-list", + msg="unsupported selection item for basic indexing; expected integer or slice, got ", + escape=True, + ), ] -# noinspection PyStatementEffect -def test_get_basic_selection_2d(store: StorePath) -> None: - # setup - a = np.arange(10000, dtype=int).reshape(1000, 10) - z = zarr_array_from_numpy_array(store, a, chunk_shape=(300, 3)) +@pytest.mark.parametrize("case", _BASIC_2D_CASES, ids=lambda c: c.id) +def test_get_basic_selection_2d(store: StorePath, case: Expect[BasicSelection, None]) -> None: + """Basic getitem on a 2D array matches numpy for rows, cols, slices, and strides.""" + a = np.arange(60, dtype=int).reshape(12, 5) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(5, 2)) + _test_get_basic_selection(a, z, case.input) - for selection in basic_selections_2d: - _test_get_basic_selection(a, z, selection) - bad_selections = basic_selections_2d_bad + [ - # integer arrays - [0, 1], - (slice(None), [0, 1]), - ] - for selection_bad in bad_selections: - with pytest.raises(IndexError): - z.get_basic_selection(selection_bad) # type: ignore[arg-type] - # check fallback on fancy indexing - fancy_selection = ([0, 1], [0, 1]) - np.testing.assert_array_equal(z[fancy_selection], [0, 11]) +@pytest.mark.parametrize("case", _BASIC_2D_BAD_CASES, ids=lambda c: c.id) +def test_get_basic_selection_2d_raises(store: StorePath, case: ExpectFail[Any]) -> None: + """Basic getitem on a 2D array rejects malformed selections with IndexError.""" + a = np.arange(60, dtype=int).reshape(12, 5) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(5, 2)) + with case.raises(): + z.get_basic_selection(case.input) + + +def test_basic_2d_fancy_fallback(store: StorePath) -> None: + """Indexing a 2D array with paired integer lists falls back to fancy (vectorized) indexing.""" + a = np.arange(60, dtype=int).reshape(12, 5) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(5, 2)) + np.testing.assert_array_equal(z[([0, 1], [0, 1])], a[([0, 1], [0, 1])]) def test_fancy_indexing_fallback_on_get_setitem(store: StorePath) -> None: From 3539816de307b41cc87c4140763afd67df5e8607 Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 22 May 2026 21:07:45 +0200 Subject: [PATCH 17/27] test: restore basic-indexing integer-list rejection coverage The basic-selection rewrite dropped two assertions that get_basic_selection rejects integer-list selections (1D direct, 2D nested in a tuple); restore them as dedicated get-only tests, since z[...] falls back to fancy indexing. Co-Authored-By: Claude Opus 4.7 (1M context) --- tests/test_indexing.py | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/tests/test_indexing.py b/tests/test_indexing.py index ea5487afe3..59a09fd35e 100644 --- a/tests/test_indexing.py +++ b/tests/test_indexing.py @@ -393,6 +393,22 @@ def test_basic_2d_fancy_fallback(store: StorePath) -> None: np.testing.assert_array_equal(z[([0, 1], [0, 1])], a[([0, 1], [0, 1])]) +def test_get_basic_selection_1d_rejects_integer_list(store: StorePath) -> None: + """get_basic_selection on a 1D array rejects an integer list (basic indexing is int/slice only).""" + a = np.arange(30, dtype=int) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(7,)) + with pytest.raises(IndexError, match="unsupported selection item for basic indexing"): + z.get_basic_selection([1, 0]) + + +def test_get_basic_selection_2d_rejects_list_in_tuple(store: StorePath) -> None: + """get_basic_selection on a 2D array rejects a list nested in an index tuple.""" + a = np.arange(60, dtype=int).reshape(12, 5) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(5, 2)) + with pytest.raises(IndexError, match="unsupported selection item for basic indexing"): + z.get_basic_selection((slice(None), [0, 1])) + + def test_fancy_indexing_fallback_on_get_setitem(store: StorePath) -> None: z = zarr_array_from_numpy_array(store, np.zeros((20, 20))) z[[1, 2, 3], [1, 2, 3]] = 1 From b96b289920bc197a9243a48b75156c00c1eae1b8 Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 22 May 2026 21:16:01 +0200 Subject: [PATCH 18/27] test: rewrite coordinate selection family as parametrized cases Co-Authored-By: Claude Opus 4.7 (1M context) --- tests/test_indexing.py | 284 +++++++++++++++++++---------------------- 1 file changed, 134 insertions(+), 150 deletions(-) diff --git a/tests/test_indexing.py b/tests/test_indexing.py index 59a09fd35e..b2c7d45d79 100644 --- a/tests/test_indexing.py +++ b/tests/test_indexing.py @@ -1023,118 +1023,135 @@ def _test_get_coordinate_selection( assert_array_equal(expect, actual) -coordinate_selections_1d_bad = [ - # slice not supported - slice(5, 15), - slice(None), - Ellipsis, - # bad stuff - 2.3, - "foo", - b"xxx", - None, - (0, 0), - (slice(None), slice(None)), +_COORD_1D_CASES: list[Expect[CoordinateSelection, None]] = [ + Expect(input=5, output=None, id="single"), + Expect(input=-1, output=None, id="single-negative"), + Expect(input=[0, 3, 10, -23, -12, -1], output=None, id="wraparound"), + Expect(input=[3, 25, 8, 17], output=None, id="out-of-order"), + Expect(input=[1, 8, 15, 29], output=None, id="sorted"), + Expect(input=[29, 15, 8, 1], output=None, id="reversed"), + Expect(input=[2, 2, 8, 8], output=None, id="duplicates"), + Expect(input=np.array([[2, 4], [6, 8]]), output=None, id="multi-dim"), ] +# get_coordinate_selection and vindex word their errors differently for these +# invalid-type inputs, so these cases assert only the exception type (msg=None). +_COORD_1D_BAD_CASES: list[ExpectFail[Any]] = [ + ExpectFail(input=slice(5, 15), exception=IndexError, id="slice", msg=None), + ExpectFail(input=slice(None), exception=IndexError, id="full-slice", msg=None), + ExpectFail(input=Ellipsis, exception=IndexError, id="ellipsis", msg=None), + ExpectFail(input=2.3, exception=IndexError, id="float", msg=None), + ExpectFail(input="foo", exception=IndexError, id="string", msg=None), + ExpectFail(input=b"xxx", exception=IndexError, id="bytes", msg=None), + ExpectFail(input=None, exception=IndexError, id="none", msg=None), + ExpectFail(input=(0, 0), exception=IndexError, id="tuple-pair", msg=None), + ExpectFail(input=(slice(None), slice(None)), exception=IndexError, id="two-slices", msg=None), + ExpectFail( + input=[31], + exception=IndexError, + id="out-of-bounds-high", + msg="index out of bounds for dimension with length 30", + ), + ExpectFail( + input=[-31], + exception=IndexError, + id="out-of-bounds-low", + msg="index out of bounds for dimension with length 30", + ), +] -# noinspection PyStatementEffect -def test_get_coordinate_selection_1d(store: StorePath) -> None: - # setup - a = np.arange(1050, dtype=int) - z = zarr_array_from_numpy_array(store, a, chunk_shape=(100,)) - - np.random.seed(42) - # test with different degrees of sparseness - for p in 2, 0.5, 0.1, 0.01: - n = int(a.size * p) - ix = np.random.choice(a.shape[0], size=n, replace=True) - _test_get_coordinate_selection(a, z, ix) - ix.sort() - _test_get_coordinate_selection(a, z, ix) - ix = ix[::-1] - _test_get_coordinate_selection(a, z, ix) +_COORD_2D_IX0 = np.array([0, 5, 11, 2, 8]) +_COORD_2D_IX1 = np.array([1, 3, 4, 0, 2]) + +_COORD_2D_CASES: list[Expect[CoordinateSelection, None]] = [ + Expect(input=(5, 4), output=None, id="single"), + Expect(input=(-1, -1), output=None, id="single-negative"), + Expect(input=(_COORD_2D_IX0, _COORD_2D_IX1), output=None, id="both-arrays"), + # scalar broadcasts in coordinate indexing (numpy and zarr agree) + Expect(input=(np.array([0, 5, 11]), 4), output=None, id="array-int"), + Expect(input=(7, np.array([0, 2, 4])), output=None, id="int-array"), + Expect(input=([3, 3, 4, 2, 5], [1, 3, 4, 0, 2]), output=None, id="not-monotonic-first"), + Expect(input=([1, 1, 2, 2, 5], [1, 3, 2, 1, 0]), output=None, id="not-monotonic-second"), + Expect( + input=(np.array([[1, 1, 2], [2, 2, 5]]), np.array([[1, 3, 2], [1, 0, 0]])), + output=None, + id="multi-dim", + ), +] - selections = [ - # test single item - 42, - -1, - # test wraparound - [0, 3, 10, -23, -12, -1], - # test out of order - [3, 105, 23, 127], # not monotonically increasing - # test multi-dimensional selection - np.array([[2, 4], [6, 8]]), - ] - for selection in selections: - _test_get_coordinate_selection(a, z, selection) +_COORD_2D_BAD_CASES: list[ExpectFail[Any]] = [ + ExpectFail( + input=(slice(5, 15), [1, 2, 3]), + exception=IndexError, + id="slice-with-array", + msg=None, + ), + ExpectFail( + input=([1, 2, 3], slice(5, 15)), + exception=IndexError, + id="array-with-slice", + msg=None, + ), + ExpectFail( + input=(Ellipsis, [1, 2, 3]), + exception=IndexError, + id="ellipsis-with-array", + msg=None, + ), + ExpectFail(input=Ellipsis, exception=IndexError, id="ellipsis", msg=None), + ExpectFail( + input=(np.array([12]), np.array([0])), + exception=IndexError, + id="out-of-bounds-axis0", + msg="index out of bounds for dimension with length 12", + ), + ExpectFail( + input=(np.array([0]), np.array([5])), + exception=IndexError, + id="out-of-bounds-axis1", + msg="index out of bounds for dimension with length 5", + ), +] - # test errors - bad_selections = coordinate_selections_1d_bad + [ - [a.shape[0] + 1], # out of bounds - [-(a.shape[0] + 1)], # out of bounds - ] - for selection in bad_selections: - with pytest.raises(IndexError): - z.get_coordinate_selection(selection) # type: ignore[arg-type] - with pytest.raises(IndexError): - z.vindex[selection] # type: ignore[index] +@pytest.mark.parametrize("case", _COORD_1D_CASES, ids=lambda c: c.id) +def test_get_coordinate_selection_1d( + store: StorePath, case: Expect[CoordinateSelection, None] +) -> None: + """vindex and get_coordinate_selection on a 1D array match numpy for int, list, and multi-dim selections.""" + a = np.arange(30, dtype=int) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(7,)) + _test_get_coordinate_selection(a, z, case.input) -def test_get_coordinate_selection_2d(store: StorePath) -> None: - # setup - a = np.arange(10000, dtype=int).reshape(1000, 10) - z = zarr_array_from_numpy_array(store, a, chunk_shape=(300, 3)) - np.random.seed(42) - ix0: npt.ArrayLike - ix1: npt.ArrayLike - # test with different degrees of sparseness - for p in 2, 0.5, 0.1, 0.01: - n = int(a.size * p) - ix0 = np.random.choice(a.shape[0], size=n, replace=True) - ix1 = np.random.choice(a.shape[1], size=n, replace=True) - selections = [ - # single value - (42, 4), - (-1, -1), - # index both axes with array - (ix0, ix1), - # mixed indexing with array / int - (ix0, 4), - (42, ix1), - (42, 4), - ] - for selection in selections: - _test_get_coordinate_selection(a, z, selection) +@pytest.mark.parametrize("case", _COORD_1D_BAD_CASES, ids=lambda c: c.id) +def test_get_coordinate_selection_1d_raises(store: StorePath, case: ExpectFail[Any]) -> None: + """get_coordinate_selection and vindex both raise IndexError for invalid 1D selections.""" + a = np.arange(30, dtype=int) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(7,)) + with case.raises(): + z.get_coordinate_selection(case.input) # type: ignore[arg-type] + with case.raises(): + z.vindex[case.input] # type: ignore[index] - # not monotonically increasing (first dim) - ix0 = [3, 3, 4, 2, 5] - ix1 = [1, 3, 5, 7, 9] - _test_get_coordinate_selection(a, z, (ix0, ix1)) - # not monotonically increasing (second dim) - ix0 = [1, 1, 2, 2, 5] - ix1 = [1, 3, 2, 1, 0] - _test_get_coordinate_selection(a, z, (ix0, ix1)) +@pytest.mark.parametrize("case", _COORD_2D_CASES, ids=lambda c: c.id) +def test_get_coordinate_selection_2d( + store: StorePath, case: Expect[CoordinateSelection, None] +) -> None: + """vindex and get_coordinate_selection on a 2D array match numpy for coordinate selections.""" + a = np.arange(60, dtype=int).reshape(12, 5) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(5, 2)) + _test_get_coordinate_selection(a, z, case.input) - # multi-dimensional selection - ix0 = np.array([[1, 1, 2], [2, 2, 5]]) - ix1 = np.array([[1, 3, 2], [1, 0, 0]]) - _test_get_coordinate_selection(a, z, (ix0, ix1)) - selection = slice(5, 15), [1, 2, 3] - with pytest.raises(IndexError): - z.get_coordinate_selection(selection) # type:ignore[arg-type] - selection = [1, 2, 3], slice(5, 15) - with pytest.raises(IndexError): - z.get_coordinate_selection(selection) # type:ignore[arg-type] - selection = Ellipsis, [1, 2, 3] - with pytest.raises(IndexError): - z.get_coordinate_selection(selection) # type:ignore[arg-type] - selection = Ellipsis - with pytest.raises(IndexError): - z.get_coordinate_selection(selection) # type:ignore[arg-type] +@pytest.mark.parametrize("case", _COORD_2D_BAD_CASES, ids=lambda c: c.id) +def test_get_coordinate_selection_2d_raises(store: StorePath, case: ExpectFail[Any]) -> None: + """get_coordinate_selection raises IndexError when slices or Ellipsis appear in a 2D coordinate selection.""" + a = np.arange(60, dtype=int).reshape(12, 5) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(5, 2)) + with case.raises(): + z.get_coordinate_selection(case.input) # type: ignore[arg-type] def _test_set_coordinate_selection( @@ -1154,59 +1171,26 @@ def _test_set_coordinate_selection( assert_array_equal(a, z[:]) -def test_set_coordinate_selection_1d(store: StorePath) -> None: - # setup - v = np.arange(550, dtype=int) - a = np.empty(v.shape, dtype=v.dtype) - z = zarr_array_from_numpy_array(store, a, chunk_shape=(100,)) - - np.random.seed(42) - # test with different degrees of sparseness - for p in 0.5, 0.01: - n = int(a.size * p) - ix = np.random.choice(a.shape[0], size=n, replace=True) - _test_set_coordinate_selection(v, a, z, ix) - - # multi-dimensional selection - ix = np.array([[2, 4], [6, 8]]) - _test_set_coordinate_selection(v, a, z, ix) - - for selection in coordinate_selections_1d_bad: - with pytest.raises(IndexError): - z.set_coordinate_selection(selection, 42) # type:ignore[arg-type] - with pytest.raises(IndexError): - z.vindex[selection] = 42 # type:ignore[index] - - -def test_set_coordinate_selection_2d(store: StorePath) -> None: - # setup - v = np.arange(5400, dtype=int).reshape(600, 9) +@pytest.mark.parametrize("case", _COORD_1D_CASES, ids=lambda c: c.id) +def test_set_coordinate_selection_1d( + store: StorePath, case: Expect[CoordinateSelection, None] +) -> None: + """set_coordinate_selection and vindex assignment on a 1D array round-trip through numpy.""" + v = np.arange(30, dtype=int) a = np.empty_like(v) - z = zarr_array_from_numpy_array(store, a, chunk_shape=(300, 3)) - - np.random.seed(42) - # test with different degrees of sparseness - for p in 0.5, 0.01: - n = int(a.size * p) - ix0 = np.random.choice(a.shape[0], size=n, replace=True) - ix1 = np.random.choice(a.shape[1], size=n, replace=True) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(7,)) + _test_set_coordinate_selection(v, a, z, case.input) - selections = ( - (42, 4), - (-1, -1), - # index both axes with array - (ix0, ix1), - # mixed indexing with array / int - (ix0, 4), - (42, ix1), - ) - for selection in selections: - _test_set_coordinate_selection(v, a, z, selection) - # multi-dimensional selection - ix0 = np.array([[1, 2, 3], [4, 5, 6]]) - ix1 = np.array([[1, 3, 2], [2, 0, 5]]) - _test_set_coordinate_selection(v, a, z, (ix0, ix1)) +@pytest.mark.parametrize("case", _COORD_2D_CASES, ids=lambda c: c.id) +def test_set_coordinate_selection_2d( + store: StorePath, case: Expect[CoordinateSelection, None] +) -> None: + """set_coordinate_selection and vindex assignment on a 2D array round-trip through numpy.""" + v = np.arange(60, dtype=int).reshape(12, 5) + a = np.empty_like(v) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(5, 2)) + _test_set_coordinate_selection(v, a, z, case.input) def _test_get_block_selection( From dad6aca102708d85302300fcf187f4c935cbd164 Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 22 May 2026 21:26:19 +0200 Subject: [PATCH 19/27] test: rewrite block selection family as parametrized cases Co-Authored-By: Claude Opus 4.7 (1M context) --- tests/test_indexing.py | 232 +++++++++++++++++------------------------ 1 file changed, 97 insertions(+), 135 deletions(-) diff --git a/tests/test_indexing.py b/tests/test_indexing.py index b2c7d45d79..074c59c287 100644 --- a/tests/test_indexing.py +++ b/tests/test_indexing.py @@ -1206,122 +1206,84 @@ def _test_get_block_selection( assert_array_equal(expect, actual) -block_selections_1d: list[BasicSelection] = [ - # test single item - 0, - 5, - # test wraparound - -1, - -4, - # test slice - slice(5), - slice(None, 3), - slice(5, 6), - slice(-3, -1), - slice(None), # Full slice +_BLOCK_1D_CASES: list[Expect[BasicSelection, slice]] = [ + Expect(input=0, output=slice(0, 7), id="block-0"), + Expect(input=2, output=slice(14, 21), id="block-mid"), + Expect(input=4, output=slice(28, 30), id="block-last"), + Expect(input=-1, output=slice(28, 30), id="block-neg-1"), + Expect(input=-2, output=slice(21, 28), id="block-neg-2"), + Expect(input=slice(3), output=slice(0, 21), id="slice-to-3"), + Expect(input=slice(None, 2), output=slice(0, 14), id="slice-none-2"), + Expect(input=slice(1, 2), output=slice(7, 14), id="slice-1-2"), + Expect(input=slice(-2, -1), output=slice(21, 28), id="slice-neg"), + Expect(input=slice(None), output=slice(0, 30), id="full"), ] -block_selections_1d_array_projection: list[slice] = [ - # test single item - slice(100), - slice(500, 600), - # test wraparound - slice(1000, None), - slice(700, 800), - # test slice - slice(500), - slice(None, 300), - slice(500, 600), - slice(800, 1000), - slice(None), +_BLOCK_1D_BAD_CASES: list[ExpectFail[Any]] = [ + ExpectFail(input=slice(3, 8, 2), exception=IndexError, id="strided-slice"), + ExpectFail(input=2.3, exception=IndexError, id="float"), + ExpectFail(input=b"xxx", exception=IndexError, id="bytes"), + ExpectFail(input=None, exception=IndexError, id="none"), + ExpectFail(input=(0, 0), exception=IndexError, id="tuple-pair"), + ExpectFail(input=(slice(None), slice(None)), exception=IndexError, id="two-slices"), + ExpectFail(input=[0, 5, 3], exception=IndexError, id="int-list"), + ExpectFail(input=5, exception=IndexError, id="out-of-bounds-high"), + ExpectFail(input=-6, exception=IndexError, id="out-of-bounds-low"), ] -block_selections_1d_bad = [ - # slice not supported - slice(3, 8, 2), - # bad stuff - 2.3, - # "foo", # TODO - b"xxx", - None, - (0, 0), - (slice(None), slice(None)), - [0, 5, 3], +_BLOCK_2D_CASES: list[Expect[BasicSelection, tuple[slice, slice]]] = [ + Expect(input=(0, 0), output=(slice(0, 5), slice(0, 2)), id="single-00"), + Expect(input=(1, 1), output=(slice(5, 10), slice(2, 4)), id="single-mid"), + Expect(input=(-1, -1), output=(slice(10, 12), slice(4, 5)), id="neg"), + Expect(input=(slice(0, 2), 0), output=(slice(0, 10), slice(0, 2)), id="slice-rows"), + Expect(input=(2, slice(1, 3)), output=(slice(10, 12), slice(2, 5)), id="slice-cols"), + Expect(input=(slice(0, 2), slice(0, 2)), output=(slice(0, 10), slice(0, 4)), id="both-slices"), + Expect(input=(slice(None), slice(None)), output=(slice(0, 12), slice(0, 5)), id="full"), ] +_BLOCK_2D_BAD_CASES: list[ExpectFail[Any]] = [ + ExpectFail(input=(slice(5, 15), [1, 2, 3]), exception=IndexError, id="slice-with-array"), + ExpectFail(input=(Ellipsis, [1, 2, 3]), exception=IndexError, id="ellipsis-with-array"), + ExpectFail(input=(slice(15, 20), slice(None)), exception=IndexError, id="out-of-bounds"), +] -def test_get_block_selection_1d(store: StorePath) -> None: - # setup - a = np.arange(1050, dtype=int) - z = zarr_array_from_numpy_array(store, a, chunk_shape=(100,)) - - for selection, expected_idx in zip( - block_selections_1d, block_selections_1d_array_projection, strict=True - ): - _test_get_block_selection(a, z, selection, expected_idx) - - bad_selections = block_selections_1d_bad + [ - z._chunk_grid.get_nchunks() + 1, # out of bounds - -(z._chunk_grid.get_nchunks() + 1), # out of bounds - ] - for selection_bad in bad_selections: - with pytest.raises(IndexError): - z.get_block_selection(selection_bad) # type:ignore[arg-type] - with pytest.raises(IndexError): - z.blocks[selection_bad] # type:ignore[index] +@pytest.mark.parametrize("case", _BLOCK_1D_CASES, ids=lambda c: c.id) +def test_get_block_selection_1d(store: StorePath, case: Expect[BasicSelection, slice]) -> None: + """get_block_selection / .blocks on a 1D array selects whole chunks matching the array slice.""" + a = np.arange(30, dtype=int) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(7,)) + _test_get_block_selection(a, z, case.input, case.output) -block_selections_2d: list[BasicSelection] = [ - # test single item - (0, 0), - (1, 2), - # test wraparound - (-1, -1), - (-3, -2), - # test slice - (slice(1), slice(2)), - (slice(None, 2), slice(-2, -1)), - (slice(2, 3), slice(-2, None)), - (slice(-3, -1), slice(-3, -2)), - (slice(None), slice(None)), # Full slice -] +@pytest.mark.parametrize("case", _BLOCK_1D_BAD_CASES, ids=lambda c: c.id) +def test_get_block_selection_1d_raises(store: StorePath, case: ExpectFail[Any]) -> None: + """get_block_selection / .blocks on a 1D array rejects invalid block selections with IndexError.""" + a = np.arange(30, dtype=int) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(7,)) + with case.raises(): + z.get_block_selection(case.input) + with case.raises(): + z.blocks[case.input] -block_selections_2d_array_projection: list[tuple[slice, slice]] = [ - # test single item - (slice(300), slice(3)), - (slice(300, 600), slice(6, 9)), - # test wraparound - (slice(900, None), slice(9, None)), - (slice(300, 600), slice(6, 9)), - # test slice - (slice(300), slice(6)), - (slice(None, 600), slice(6, 9)), - (slice(600, 900), slice(6, None)), - (slice(300, 900), slice(3, 6)), - (slice(None), slice(None)), # Full slice -] +@pytest.mark.parametrize("case", _BLOCK_2D_CASES, ids=lambda c: c.id) +def test_get_block_selection_2d( + store: StorePath, case: Expect[BasicSelection, tuple[slice, slice]] +) -> None: + """get_block_selection / .blocks on a 2D array selects whole chunk regions matching the array slices.""" + a = np.arange(60, dtype=int).reshape(12, 5) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(5, 2)) + _test_get_block_selection(a, z, case.input, case.output) -def test_get_block_selection_2d(store: StorePath) -> None: - # setup - a = np.arange(10000, dtype=int).reshape(1000, 10) - z = zarr_array_from_numpy_array(store, a, chunk_shape=(300, 3)) - for selection, expected_idx in zip( - block_selections_2d, block_selections_2d_array_projection, strict=True - ): - _test_get_block_selection(a, z, selection, expected_idx) - - selection = slice(5, 15), [1, 2, 3] - with pytest.raises(IndexError): - z.get_block_selection(selection) - selection = Ellipsis, [1, 2, 3] - with pytest.raises(IndexError): - z.get_block_selection(selection) - selection = slice(15, 20), slice(None) - with pytest.raises(IndexError): # out of bounds - z.get_block_selection(selection) +@pytest.mark.parametrize("case", _BLOCK_2D_BAD_CASES, ids=lambda c: c.id) +def test_get_block_selection_2d_raises(store: StorePath, case: ExpectFail[Any]) -> None: + """get_block_selection on a 2D array rejects invalid or out-of-bounds block selections with IndexError.""" + a = np.arange(60, dtype=int).reshape(12, 5) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(5, 2)) + with case.raises(): + z.get_block_selection(case.input) def _test_set_block_selection( @@ -1329,7 +1291,7 @@ def _test_set_block_selection( a: npt.NDArray[Any], z: zarr.Array, selection: BasicSelection, - expected_idx: slice, + expected_idx: slice | tuple[slice, ...], ) -> None: for value in 42, v[expected_idx], v[expected_idx].tolist(): # setup expectation @@ -1345,44 +1307,44 @@ def _test_set_block_selection( assert_array_equal(a, z[:]) -def test_set_block_selection_1d(store: StorePath) -> None: - # setup - v = np.arange(1050, dtype=int) - a = np.empty(v.shape, dtype=v.dtype) - z = zarr_array_from_numpy_array(store, a, chunk_shape=(100,)) +@pytest.mark.parametrize("case", _BLOCK_1D_CASES, ids=lambda c: c.id) +def test_set_block_selection_1d(store: StorePath, case: Expect[BasicSelection, slice]) -> None: + """set_block_selection / .blocks assignment on a 1D array round-trips through numpy for each block selection.""" + v = np.arange(30, dtype=int) + a = np.empty_like(v) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(7,)) + _test_set_block_selection(v, a, z, case.input, case.output) - for selection, expected_idx in zip( - block_selections_1d, block_selections_1d_array_projection, strict=True - ): - _test_set_block_selection(v, a, z, selection, expected_idx) - for selection_bad in block_selections_1d_bad: - with pytest.raises(IndexError): - z.set_block_selection(selection_bad, 42) # type:ignore[arg-type] - with pytest.raises(IndexError): - z.blocks[selection_bad] = 42 # type:ignore[index] +@pytest.mark.parametrize("case", _BLOCK_1D_BAD_CASES, ids=lambda c: c.id) +def test_set_block_selection_1d_raises(store: StorePath, case: ExpectFail[Any]) -> None: + """set_block_selection / .blocks assignment on a 1D array rejects invalid block selections with IndexError.""" + a = np.arange(30, dtype=int) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(7,)) + with case.raises(): + z.set_block_selection(case.input, 42) + with case.raises(): + z.blocks[case.input] = 42 -def test_set_block_selection_2d(store: StorePath) -> None: - # setup - v = np.arange(10000, dtype=int).reshape(1000, 10) - a = np.empty(v.shape, dtype=v.dtype) - z = zarr_array_from_numpy_array(store, a, chunk_shape=(300, 3)) +@pytest.mark.parametrize("case", _BLOCK_2D_CASES, ids=lambda c: c.id) +def test_set_block_selection_2d( + store: StorePath, case: Expect[BasicSelection, tuple[slice, slice]] +) -> None: + """set_block_selection / .blocks assignment on a 2D array round-trips through numpy for each block selection.""" + v = np.arange(60, dtype=int).reshape(12, 5) + a = np.empty_like(v) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(5, 2)) + _test_set_block_selection(v, a, z, case.input, case.output) - for selection, expected_idx in zip( - block_selections_2d, block_selections_2d_array_projection, strict=True - ): - _test_set_block_selection(v, a, z, selection, expected_idx) - selection = slice(5, 15), [1, 2, 3] - with pytest.raises(IndexError): - z.set_block_selection(selection, 42) - selection = Ellipsis, [1, 2, 3] - with pytest.raises(IndexError): - z.set_block_selection(selection, 42) - selection = slice(15, 20), slice(None) - with pytest.raises(IndexError): # out of bounds - z.set_block_selection(selection, 42) +@pytest.mark.parametrize("case", _BLOCK_2D_BAD_CASES, ids=lambda c: c.id) +def test_set_block_selection_2d_raises(store: StorePath, case: ExpectFail[Any]) -> None: + """set_block_selection on a 2D array rejects invalid or out-of-bounds block selections with IndexError.""" + a = np.arange(60, dtype=int).reshape(12, 5) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(5, 2)) + with case.raises(): + z.set_block_selection(case.input, 42) def _test_get_mask_selection(a: npt.NDArray[Any], z: Array, selection: npt.NDArray) -> None: From 88e574b45ff7784f056443a9ac375acb7531cac6 Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 22 May 2026 21:35:32 +0200 Subject: [PATCH 20/27] test: rewrite mask selection family as parametrized cases Co-Authored-By: Claude Opus 4.7 (1M context) --- tests/test_indexing.py | 181 ++++++++++++++++++++++++----------------- 1 file changed, 106 insertions(+), 75 deletions(-) diff --git a/tests/test_indexing.py b/tests/test_indexing.py index 074c59c287..23630bd424 100644 --- a/tests/test_indexing.py +++ b/tests/test_indexing.py @@ -1357,65 +1357,99 @@ def _test_get_mask_selection(a: npt.NDArray[Any], z: Array, selection: npt.NDArr assert_array_equal(expect, actual) -mask_selections_1d_bad = [ - # slice not supported - slice(5, 15), - slice(None), - Ellipsis, - # bad stuff - 2.3, - "foo", - b"xxx", - None, - (0, 0), - (slice(None), slice(None)), +_MASK_1D_CASES: list[Expect[Any, None]] = [ + Expect(input=np.zeros(30, dtype=bool), output=None, id="all-false"), + Expect(input=np.ones(30, dtype=bool), output=None, id="all-true"), + Expect(input=np.arange(30) % 2 == 0, output=None, id="alternating"), + Expect( + input=np.isin(np.arange(30), [0, 7, 14, 29]), + output=None, + id="sparse-cross-chunk", + ), ] +# msg=None for all 1d bad cases: get_mask_selection and vindex raise different +# messages for the same input, so no single substring satisfies both assertions. +_MASK_1D_BAD_CASES: list[ExpectFail[Any]] = [ + ExpectFail(input=slice(5, 15), exception=IndexError, id="slice"), + ExpectFail(input=slice(None), exception=IndexError, id="full-slice"), + ExpectFail(input=Ellipsis, exception=IndexError, id="ellipsis"), + ExpectFail(input=2.3, exception=IndexError, id="float"), + ExpectFail(input="foo", exception=IndexError, id="string"), + ExpectFail(input=b"xxx", exception=IndexError, id="bytes"), + ExpectFail(input=None, exception=IndexError, id="none"), + ExpectFail(input=(0, 0), exception=IndexError, id="tuple-pair"), + ExpectFail(input=(slice(None), slice(None)), exception=IndexError, id="two-slices"), + ExpectFail(input=np.zeros(5, dtype=bool), exception=IndexError, id="mask-too-short"), + ExpectFail(input=np.zeros(50, dtype=bool), exception=IndexError, id="mask-too-long"), + ExpectFail(input=[[True, False], [False, True]], exception=IndexError, id="too-many-dims"), +] -# noinspection PyStatementEffect -def test_get_mask_selection_1d(store: StorePath) -> None: - # setup - a = np.arange(1050, dtype=int) - z = zarr_array_from_numpy_array(store, a, chunk_shape=(100,)) - np.random.seed(42) - # test with different degrees of sparseness - for p in 0.5, 0.1, 0.01: - ix = np.random.binomial(1, p, size=a.shape[0]).astype(bool) - _test_get_mask_selection(a, z, ix) - - # test errors - bad_selections = mask_selections_1d_bad + [ - np.zeros(50, dtype=bool), # too short - np.zeros(2000, dtype=bool), # too long - [[True, False], [False, True]], # too many dimensions - ] - for selection in bad_selections: - with pytest.raises(IndexError): - z.get_mask_selection(selection) # type: ignore[arg-type] - with pytest.raises(IndexError): - z.vindex[selection] # type:ignore[index] +def _make_sparse_2d_mask() -> npt.NDArray[np.bool_]: + """Build a deterministic sparse (12, 5) boolean mask with Trues at (0,0), (5,2), (11,4), (2,3).""" + mask = np.zeros((12, 5), dtype=bool) + for r, c in [(0, 0), (5, 2), (11, 4), (2, 3)]: + mask[r, c] = True + return mask -# noinspection PyStatementEffect -def test_get_mask_selection_2d(store: StorePath) -> None: - # setup - a = np.arange(10000, dtype=int).reshape(1000, 10) - z = zarr_array_from_numpy_array(store, a, chunk_shape=(300, 3)) +_MASK_2D_CASES: list[Expect[Any, None]] = [ + Expect(input=np.zeros((12, 5), dtype=bool), output=None, id="all-false"), + Expect(input=np.ones((12, 5), dtype=bool), output=None, id="all-true"), + Expect( + input=(np.add.outer(np.arange(12), np.arange(5)) % 2).astype(bool), + output=None, + id="checkerboard", + ), + Expect( + input=_make_sparse_2d_mask(), + output=None, + id="sparse", + ), +] - np.random.seed(42) - # test with different degrees of sparseness - for p in 0.5, 0.1, 0.01: - ix = np.random.binomial(1, p, size=a.size).astype(bool).reshape(a.shape) - _test_get_mask_selection(a, z, ix) +_MASK_2D_BAD_CASES: list[ExpectFail[Any]] = [ + ExpectFail(input=np.zeros((12, 3), dtype=bool), exception=IndexError, id="too-few-cols"), + ExpectFail(input=np.zeros((20, 5), dtype=bool), exception=IndexError, id="too-many-rows"), + ExpectFail(input=[True, False], exception=IndexError, id="wrong-ndim"), +] - # test errors - with pytest.raises(IndexError): - z.vindex[np.zeros((1000, 5), dtype=bool)] # too short - with pytest.raises(IndexError): - z.vindex[np.zeros((2000, 10), dtype=bool)] # too long - with pytest.raises(IndexError): - z.vindex[[True, False]] # wrong no. dimensions + +@pytest.mark.parametrize("case", _MASK_1D_CASES, ids=lambda c: c.id) +def test_get_mask_selection_1d(store: StorePath, case: Expect[Any, None]) -> None: + """get_mask_selection / vindex / getitem on a 1D array match numpy for boolean masks.""" + a = np.arange(30, dtype=int) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(7,)) + _test_get_mask_selection(a, z, case.input) + + +@pytest.mark.parametrize("case", _MASK_1D_BAD_CASES, ids=lambda c: c.id) +def test_get_mask_selection_1d_raises(store: StorePath, case: ExpectFail[Any]) -> None: + """get_mask_selection / vindex on a 1D array reject non-boolean-mask and mis-shaped selections.""" + a = np.arange(30, dtype=int) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(7,)) + with case.raises(): + z.get_mask_selection(case.input) # type: ignore[arg-type] + with case.raises(): + z.vindex[case.input] # type: ignore[index] + + +@pytest.mark.parametrize("case", _MASK_2D_CASES, ids=lambda c: c.id) +def test_get_mask_selection_2d(store: StorePath, case: Expect[Any, None]) -> None: + """get_mask_selection / vindex / getitem on a 2D array match numpy for boolean masks.""" + a = np.arange(60, dtype=int).reshape(12, 5) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(5, 2)) + _test_get_mask_selection(a, z, case.input) + + +@pytest.mark.parametrize("case", _MASK_2D_BAD_CASES, ids=lambda c: c.id) +def test_get_mask_selection_2d_raises(store: StorePath, case: ExpectFail[Any]) -> None: + """vindex on a 2D array rejects masks of the wrong shape or dimensionality.""" + a = np.arange(60, dtype=int).reshape(12, 5) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(5, 2)) + with case.raises(): + z.vindex[case.input] # type: ignore[index] def _test_set_mask_selection( @@ -1434,36 +1468,33 @@ def _test_set_mask_selection( assert_array_equal(a, z[:]) -def test_set_mask_selection_1d(store: StorePath) -> None: - # setup - v = np.arange(1050, dtype=int) +@pytest.mark.parametrize("case", _MASK_1D_CASES, ids=lambda c: c.id) +def test_set_mask_selection_1d(store: StorePath, case: Expect[Any, None]) -> None: + """set_mask_selection / vindex / setitem on a 1D array match numpy for boolean masks.""" + v = np.arange(30, dtype=int) a = np.empty_like(v) - z = zarr_array_from_numpy_array(store, a, chunk_shape=(100,)) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(7,)) + _test_set_mask_selection(v, a, z, case.input) - np.random.seed(42) - # test with different degrees of sparseness - for p in 0.5, 0.1, 0.01: - ix = np.random.binomial(1, p, size=a.shape[0]).astype(bool) - _test_set_mask_selection(v, a, z, ix) - for selection in mask_selections_1d_bad: - with pytest.raises(IndexError): - z.set_mask_selection(selection, 42) # type: ignore[arg-type] - with pytest.raises(IndexError): - z.vindex[selection] = 42 # type: ignore[index] +@pytest.mark.parametrize("case", _MASK_1D_BAD_CASES, ids=lambda c: c.id) +def test_set_mask_selection_1d_raises(store: StorePath, case: ExpectFail[Any]) -> None: + """set_mask_selection / vindex on a 1D array reject non-boolean-mask and mis-shaped selections.""" + a = np.arange(30, dtype=int) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(7,)) + with case.raises(): + z.set_mask_selection(case.input, 42) # type: ignore[arg-type] + with case.raises(): + z.vindex[case.input] = 42 # type: ignore[index] -def test_set_mask_selection_2d(store: StorePath) -> None: - # setup - v = np.arange(10000, dtype=int).reshape(1000, 10) +@pytest.mark.parametrize("case", _MASK_2D_CASES, ids=lambda c: c.id) +def test_set_mask_selection_2d(store: StorePath, case: Expect[Any, None]) -> None: + """set_mask_selection / vindex / setitem on a 2D array match numpy for boolean masks.""" + v = np.arange(60, dtype=int).reshape(12, 5) a = np.empty_like(v) - z = zarr_array_from_numpy_array(store, a, chunk_shape=(300, 3)) - - np.random.seed(42) - # test with different degrees of sparseness - for p in 0.5, 0.1, 0.01: - ix = np.random.binomial(1, p, size=a.size).astype(bool).reshape(a.shape) - _test_set_mask_selection(v, a, z, ix) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(5, 2)) + _test_set_mask_selection(v, a, z, case.input) def test_get_selection_out(store: StorePath) -> None: From 3c7026ebd771891fec4c497efba758e8947c7b47 Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 22 May 2026 21:45:14 +0200 Subject: [PATCH 21/27] test: shrink arrays in selection_out and numpy-equivalence tests Co-Authored-By: Claude Opus 4.7 (1M context) --- tests/test_indexing.py | 122 +++++++++++++++++++---------------------- 1 file changed, 57 insertions(+), 65 deletions(-) diff --git a/tests/test_indexing.py b/tests/test_indexing.py index 23630bd424..8eef7d4045 100644 --- a/tests/test_indexing.py +++ b/tests/test_indexing.py @@ -1498,13 +1498,14 @@ def test_set_mask_selection_2d(store: StorePath, case: Expect[Any, None]) -> Non def test_get_selection_out(store: StorePath) -> None: + """get_*_selection writes results into a provided out buffer, matching numpy.""" # basic selections - a = np.arange(1050) - z = zarr_array_from_numpy_array(store, a, chunk_shape=(100,)) + a = np.arange(30) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(7,)) selections = [ - slice(50, 150), - slice(0, 1050), + slice(5, 15), + slice(0, 30), slice(1, 2), ] for selection in selections: @@ -1517,53 +1518,42 @@ def test_get_selection_out(store: StorePath) -> None: z.get_basic_selection(Ellipsis, out=[]) # type: ignore[arg-type] # orthogonal selections - a = np.arange(10000, dtype=int).reshape(1000, 10) - z = zarr_array_from_numpy_array(store, a, chunk_shape=(300, 3)) - np.random.seed(42) - # test with different degrees of sparseness - for p in 0.5, 0.1, 0.01: - ix0 = np.random.binomial(1, p, size=a.shape[0]).astype(bool) - ix1 = np.random.binomial(1, 0.5, size=a.shape[1]).astype(bool) - selections = [ - # index both axes with array - (ix0, ix1), - # mixed indexing with array / slice - (ix0, slice(1, 5)), - (slice(250, 350), ix1), - # mixed indexing with array / int - (ix0, 4), - (42, ix1), - # mixed int array / bool array - (ix0, np.nonzero(ix1)[0]), - (np.nonzero(ix0)[0], ix1), - ] - for selection in selections: - expect = oindex(a, selection) - out = get_ndbuffer_class().from_numpy_array(np.zeros(expect.shape, dtype=expect.dtype)) - z.get_orthogonal_selection(selection, out=out) - assert_array_equal(expect, out.as_numpy_array()[:]) + a = np.arange(60, dtype=int).reshape(12, 5) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(5, 2)) + selections = [ + # index both axes with bool array + (_ORTHO_2D_IX0_BOOL, _ORTHO_2D_IX1_BOOL), + # mixed indexing with bool array / slice + (_ORTHO_2D_IX0_BOOL, slice(1, 4)), + (slice(2, 9), _ORTHO_2D_IX1_BOOL), + # mixed indexing with bool array / int + (_ORTHO_2D_IX0_BOOL, 3), + (7, _ORTHO_2D_IX1_BOOL), + # mixed int array / bool array + (_ORTHO_2D_IX0_BOOL, _ORTHO_2D_IX1_INT), + (_ORTHO_2D_IX0_INT, _ORTHO_2D_IX1_BOOL), + ] + for selection in selections: + expect = oindex(a, selection) + out = get_ndbuffer_class().from_numpy_array(np.zeros(expect.shape, dtype=expect.dtype)) + z.get_orthogonal_selection(selection, out=out) + assert_array_equal(expect, out.as_numpy_array()[:]) # coordinate selections - a = np.arange(10000, dtype=int).reshape(1000, 10) - z = zarr_array_from_numpy_array(store, a, chunk_shape=(300, 3)) - np.random.seed(42) - # test with different degrees of sparseness - for p in 0.5, 0.1, 0.01: - n = int(a.size * p) - ix0 = np.random.choice(a.shape[0], size=n, replace=True) - ix1 = np.random.choice(a.shape[1], size=n, replace=True) - selections = [ - # index both axes with array - (ix0, ix1), - # mixed indexing with array / int - (ix0, 4), - (42, ix1), - ] - for selection in selections: - expect = a[selection] - out = get_ndbuffer_class().from_numpy_array(np.zeros(expect.shape, dtype=expect.dtype)) - z.get_coordinate_selection(selection, out=out) - assert_array_equal(expect, out.as_numpy_array()[:]) + a = np.arange(60, dtype=int).reshape(12, 5) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(5, 2)) + selections = [ + # index both axes with array + (np.array([0, 5, 11]), np.array([0, 2, 4])), + # mixed indexing with array / int + (np.array([0, 5, 11]), 3), + (7, np.array([0, 2, 4])), + ] + for selection in selections: + expect = a[selection] + out = get_ndbuffer_class().from_numpy_array(np.zeros(expect.shape, dtype=expect.dtype)) + z.get_coordinate_selection(selection, out=out) + assert_array_equal(expect, out.as_numpy_array()[:]) @pytest.mark.xfail(reason="fields are not supported in v3") @@ -1858,22 +1848,23 @@ async def test_accessed_chunks( [1, ...], [slice(None)], [1, 3], - [[1, 2, 3], 9], - [np.arange(1000)], - [slice(5, 15)], - [slice(2, 4), 4], + [[1, 2, 3], 4], + [np.arange(12)], + [slice(2, 9)], + [slice(1, 3), 3], [[1, 3]], # mask selection - [np.tile([True, False], (1000, 5))], - [np.full((1000, 10), False)], + [np.tile([True, False, True, False, True], (12, 1))], + [np.full((12, 5), False)], # coordinate selection - [[1, 2, 3, 4], [5, 6, 7, 8]], - [[100, 200, 300], [4, 5, 6]], + [[1, 2, 3, 4], [0, 1, 2, 3]], + [[10, 11, 5], [4, 0, 2]], ], ) def test_indexing_equals_numpy(store: StorePath, selection: Selection) -> None: - a = np.arange(10000, dtype=int).reshape(1000, 10) - z = zarr_array_from_numpy_array(store, a, chunk_shape=(300, 3)) + """Indexing a zarr array with assorted basic/mask/coordinate selections matches numpy.""" + a = np.arange(60, dtype=int).reshape(12, 5) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(5, 2)) # note: in python 3.10 a[*selection] is not valid unpacking syntax expected = a[*selection,] actual = z[*selection,] @@ -1883,17 +1874,18 @@ def test_indexing_equals_numpy(store: StorePath, selection: Selection) -> None: @pytest.mark.parametrize( "selection", [ - [np.tile([True, False], 500), np.tile([True, False], 5)], - [np.full(1000, False), np.tile([True, False], 5)], - [np.full(1000, True), np.full(10, True)], - [np.full(1000, True), [True, False] * 5], + [np.tile([True, False], 6), np.tile([True, False, True, False, True], 1)], + [np.full(12, False), np.array([True, False, True, False, True])], + [np.full(12, True), np.full(5, True)], + [np.full(12, True), [True, False, True, False, True]], ], ) def test_orthogonal_bool_indexing_like_numpy_ix( store: StorePath, selection: list[npt.ArrayLike] ) -> None: - a = np.arange(10000, dtype=int).reshape(1000, 10) - z = zarr_array_from_numpy_array(store, a, chunk_shape=(300, 3)) + """Orthogonal boolean indexing on each axis matches numpy's np.ix_ semantics.""" + a = np.arange(60, dtype=int).reshape(12, 5) + z = zarr_array_from_numpy_array(store, a, chunk_shape=(5, 2)) expected = a[np.ix_(*selection)] # note: in python 3.10 z[*selection] is not valid unpacking syntax actual = z[*selection,] From ed346c526435ffad462a59a7fe1bd6d26493737a Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 22 May 2026 21:54:20 +0200 Subject: [PATCH 22/27] test: add behavior docstrings to remaining indexing tests Co-Authored-By: Claude Opus 4.7 (1M context) --- tests/test_indexing.py | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/tests/test_indexing.py b/tests/test_indexing.py index 8eef7d4045..310fc8a8e3 100644 --- a/tests/test_indexing.py +++ b/tests/test_indexing.py @@ -103,6 +103,7 @@ def set_sync(self, key: str, value: Buffer) -> None: def test_normalize_integer_selection() -> None: + """normalize_integer_selection handles positive/negative indices and raises IndexError for out-of-bounds values.""" assert 1 == normalize_integer_selection(1, 100) assert 99 == normalize_integer_selection(-1, 100) with pytest.raises(IndexError): @@ -114,6 +115,7 @@ def test_normalize_integer_selection() -> None: def test_replace_ellipsis() -> None: + """replace_ellipsis expands Ellipsis to full slice(None) selections for 1D and 2D shapes.""" # 1D, single item assert (0,) == replace_ellipsis(0, (100,)) @@ -162,6 +164,7 @@ def test_replace_ellipsis() -> None: ) @pytest.mark.parametrize("use_out", [True, False]) def test_get_basic_selection_0d(store: StorePath, use_out: bool, value: Any, dtype: Any) -> None: + """get_basic_selection on a 0-dimensional array returns the scalar value via Ellipsis and (), including the `out` buffer path.""" # setup arr_np = np.array(value, dtype=dtype) arr_z = zarr_array_from_numpy_array(store, arr_np) @@ -410,6 +413,7 @@ def test_get_basic_selection_2d_rejects_list_in_tuple(store: StorePath) -> None: def test_fancy_indexing_fallback_on_get_setitem(store: StorePath) -> None: + """Paired integer-list indexing falls back to vectorized (fancy) get and set via `__getitem__`/`__setitem__`.""" z = zarr_array_from_numpy_array(store, np.zeros((20, 20))) z[[1, 2, 3], [1, 2, 3]] = 1 np.testing.assert_array_equal( @@ -469,6 +473,7 @@ def test_orthogonal_indexing_fallback_on_getitem_2d( def test_setitem_zarr_array_as_value() -> None: + """Assigning a zarr array as a value in `__setitem__` does not raise a SyncError (regression for GH3611).""" # Regression test for https://github.com/zarr-developers/zarr-python/issues/3611 # Assigning a zarr Array as the value used to raise # SyncError("Calling sync() from within a running loop") because the codec @@ -488,6 +493,7 @@ def test_setitem_zarr_array_as_value() -> None: @pytest.mark.skip(reason="fails on ubuntu, windows; numpy=2.2; in CI") def test_setitem_repeated_index(): + """oindex assignment with repeated indices writes the last value for each duplicated index position.""" array = zarr.array(data=np.zeros((4,)), chunks=(1,)) indexer = np.array([-1, -1, 0, 0]) array.oindex[(indexer,)] = [0, 1, 2, 3] @@ -573,6 +579,7 @@ def test_orthogonal_indexing_fallback_on_setitem_2d( def test_fancy_indexing_doesnt_mix_with_implicit_slicing(store: StorePath) -> None: + """Fancy indexing that would require implicit slicing over an unspecified axis raises IndexError on a 3D array.""" z2 = zarr_array_from_numpy_array(store, np.zeros((5, 5, 5))) with pytest.raises(IndexError): z2[[1, 2, 3], [1, 2, 3]] = 2 @@ -596,6 +603,7 @@ def test_fancy_indexing_doesnt_mix_with_implicit_slicing(store: StorePath) -> No def test_set_basic_selection_0d( store: StorePath, value: Any, dtype: str | list[tuple[str, str]] ) -> None: + """set_basic_selection and `__setitem__` write scalar values correctly to a 0-dimensional array.""" arr_np = np.array(value, dtype=dtype) arr_np_zeros = np.zeros_like(arr_np, dtype=dtype) arr_z = zarr_array_from_numpy_array(store, arr_np_zeros) @@ -908,6 +916,7 @@ def test_get_orthogonal_selection_3d( def test_orthogonal_indexing_edge_cases(store: StorePath) -> None: + """oindex on a shape-(1, 2, 3) array correctly handles mixing integer, slice, int-list, and bool-list indexers per axis.""" a = np.arange(6).reshape(1, 2, 3) z = zarr_array_from_numpy_array(store, a, chunk_shape=(1, 2, 3)) @@ -952,6 +961,7 @@ def test_set_orthogonal_selection_1d( def test_set_item_1d_last_two_chunks(store: StorePath): + """Regression for GH2849: `__setitem__` correctly writes to the last two chunks of a 1D array and to 0-dimensional scalar arrays.""" # regression test for GH2849 g = zarr.open_group(store=store, zarr_format=3, mode="w") a = g.create_array("bar", shape=(10,), chunks=(3,), dtype=int) @@ -993,6 +1003,7 @@ def test_set_orthogonal_selection_3d( def test_orthogonal_indexing_fallback_on_get_setitem(store: StorePath) -> None: + """Paired integer-list indexing on a 2D array falls back to orthogonal get and set via `__getitem__`/`__setitem__`.""" z = zarr_array_from_numpy_array(store, np.zeros((20, 20))) z[[1, 2, 3], [1, 2, 3]] = 1 np.testing.assert_array_equal( @@ -1558,6 +1569,7 @@ def test_get_selection_out(store: StorePath) -> None: @pytest.mark.xfail(reason="fields are not supported in v3") def test_get_selections_with_fields(store: StorePath) -> None: + """Would verify that basic, orthogonal, coordinate, and mask selections with structured-array `fields` arguments return the correct sub-fields (xfail: fields unsupported in v3).""" a = np.array( [("aaa", 1, 4.2), ("bbb", 2, 8.4), ("ccc", 3, 12.6)], dtype=[("foo", "S3"), ("bar", "i4"), ("baz", "f8")], @@ -1666,6 +1678,7 @@ def test_get_selections_with_fields(store: StorePath) -> None: @pytest.mark.xfail(reason="fields are not supported in v3") def test_set_selections_with_fields(store: StorePath) -> None: + """Would verify that basic, orthogonal, coordinate, and mask set-selections with structured-array `fields` correctly write individual fields and reject multi-field assignment (xfail: fields unsupported in v3).""" v = np.array( [("aaa", 1, 4.2), ("bbb", 2, 8.4), ("ccc", 3, 12.6)], dtype=[("foo", "S3"), ("bar", "i4"), ("baz", "f8")], @@ -1751,6 +1764,7 @@ def test_set_selections_with_fields(store: StorePath) -> None: def test_slice_selection_uints() -> None: + """make_slice_selection accepts unsigned integer indices without error and produces correct shape.""" arr = np.arange(24).reshape((4, 6)) idx = np.uint64(3) slice_sel = make_slice_selection((idx,)) @@ -1758,6 +1772,7 @@ def test_slice_selection_uints() -> None: def test_numpy_int_indexing(store: StorePath) -> None: + """Indexing with a plain Python int and with `np.int64` both return the correct scalar element.""" a = np.arange(1050) z = zarr_array_from_numpy_array(store, a, chunk_shape=(100,)) assert a[42] == z[42] @@ -1792,6 +1807,7 @@ def test_numpy_int_indexing(store: StorePath) -> None: async def test_accessed_chunks( shape: tuple[int, ...], chunks: tuple[int, ...], ops: list[tuple[str, tuple[slice, ...]]] ) -> None: + """Only the chunks intersected by a slice selection are read or written, verified via a `CountingDict` store.""" # Test that only the required chunks are accessed during basic selection operations # shape: array shape # chunks: chunk size @@ -1943,6 +1959,7 @@ def test_iter_grid_invalid() -> None: def test_indexing_with_zarr_array(store: StorePath) -> None: + """Regression for GH2133: indexing a zarr array with another zarr array (boolean or integer) as the indexer produces the same result as indexing with the equivalent numpy array.""" # regression test for https://github.com/zarr-developers/zarr-python/issues/2133 a = np.arange(10) za = zarr.array(a, chunks=2, store=store, path="a") @@ -1962,6 +1979,7 @@ def test_indexing_with_zarr_array(store: StorePath) -> None: @pytest.mark.parametrize("store", ["local", "memory"], indirect=["store"]) @pytest.mark.parametrize("shape", [(0, 2, 3), (0,), (3, 0)]) def test_zero_sized_chunks(store: StorePath, shape: list[int]) -> None: + """Arrays with zero-extent dimensions can be created and indexed without error; reading back returns the fill value.""" # Chunk sizes must be >= 1 per spec; use 1 for zero-extent dimensions. chunks = tuple(max(1, s) for s in shape) z = zarr.create_array(store=store, shape=shape, chunks=chunks, zarr_format=3, dtype="f8") @@ -1971,6 +1989,7 @@ def test_zero_sized_chunks(store: StorePath, shape: list[int]) -> None: @pytest.mark.parametrize("store", ["memory"], indirect=["store"]) def test_vectorized_indexing_incompatible_shape(store) -> None: + """Regression for GH2469: vectorized set-indexing raises ValueError when the value shape is incompatible with the indexer shape.""" # GH2469 shape = (4, 4) chunks = (2, 2) @@ -1988,6 +2007,7 @@ def test_vectorized_indexing_incompatible_shape(store) -> None: def test_iter_chunk_regions(): + """_iter_chunk_regions yields slices that exactly cover each chunk, and reading/writing each region round-trips correctly.""" chunks = (2, 3) a = zarr.create((10, 10), chunks=chunks) a[:] = 1 @@ -2100,6 +2120,7 @@ class TestAsync: ) @pytest.mark.asyncio async def test_async_oindex(self, store, indexer, expected): + """The async `oindex.getitem` interface returns the correct orthogonally-indexed result for int, slice, ellipsis, array, and boolean indexers.""" z = zarr.create_array(store=store, shape=(2, 2), chunks=(1, 1), zarr_format=3, dtype="i8") z[...] = np.array([[1, 2], [3, 4]]) async_zarr = z._async_array @@ -2109,6 +2130,7 @@ async def test_async_oindex(self, store, indexer, expected): @pytest.mark.asyncio async def test_async_oindex_with_zarr_array(self, store): + """The async `oindex.getitem` interface accepts a zarr boolean array as the indexer and returns the correct rows.""" group = zarr.create_group(store=store, zarr_format=3) z1 = group.create_array(name="z1", shape=(2, 2), chunks=(1, 1), dtype="i8") @@ -2133,6 +2155,7 @@ async def test_async_oindex_with_zarr_array(self, store): ) @pytest.mark.asyncio async def test_async_vindex(self, store, indexer, expected): + """The async `vindex.getitem` interface returns the correct vectorized-indexed result for coordinate and boolean indexers.""" z = zarr.create_array(store=store, shape=(2, 2), chunks=(1, 1), zarr_format=3, dtype="i8") z[...] = np.array([[1, 2], [3, 4]]) async_zarr = z._async_array @@ -2142,6 +2165,7 @@ async def test_async_vindex(self, store, indexer, expected): @pytest.mark.asyncio async def test_async_vindex_with_zarr_array(self, store): + """The async `vindex.getitem` interface accepts a zarr 2D boolean array as the indexer and returns the correct elements.""" group = zarr.create_group(store=store, zarr_format=3) z1 = group.create_array(name="z1", shape=(2, 2), chunks=(1, 1), dtype="i8") @@ -2158,6 +2182,7 @@ async def test_async_vindex_with_zarr_array(self, store): @pytest.mark.asyncio async def test_async_invalid_indexer(self, store): + """The async `vindex.getitem` and `oindex.getitem` interfaces raise IndexError when given an unsupported indexer type.""" z = zarr.create_array(store=store, shape=(2, 2), chunks=(1, 1), zarr_format=3, dtype="i8") z[...] = np.array([[1, 2], [3, 4]]) async_zarr = z._async_array From 1b7611dd47763e1703132f441db109b132340cd8 Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 22 May 2026 21:55:45 +0200 Subject: [PATCH 23/27] test: drop redundant pytest.mark.asyncio decorators asyncio_mode = "auto" already collects async test functions; the explicit marks were no-ops. Co-Authored-By: Claude Opus 4.7 (1M context) --- tests/test_indexing.py | 5 ----- 1 file changed, 5 deletions(-) diff --git a/tests/test_indexing.py b/tests/test_indexing.py index 310fc8a8e3..e033fa812c 100644 --- a/tests/test_indexing.py +++ b/tests/test_indexing.py @@ -2118,7 +2118,6 @@ class TestAsync: (np.array([False, False]), np.empty(shape=(0, 2), dtype="i8")), ], ) - @pytest.mark.asyncio async def test_async_oindex(self, store, indexer, expected): """The async `oindex.getitem` interface returns the correct orthogonally-indexed result for int, slice, ellipsis, array, and boolean indexers.""" z = zarr.create_array(store=store, shape=(2, 2), chunks=(1, 1), zarr_format=3, dtype="i8") @@ -2128,7 +2127,6 @@ async def test_async_oindex(self, store, indexer, expected): result = await async_zarr.oindex.getitem(indexer) assert_array_equal(result, expected) - @pytest.mark.asyncio async def test_async_oindex_with_zarr_array(self, store): """The async `oindex.getitem` interface accepts a zarr boolean array as the indexer and returns the correct rows.""" group = zarr.create_group(store=store, zarr_format=3) @@ -2153,7 +2151,6 @@ async def test_async_oindex_with_zarr_array(self, store): (np.array([[False, True], [False, True]]), np.array([2, 4])), ], ) - @pytest.mark.asyncio async def test_async_vindex(self, store, indexer, expected): """The async `vindex.getitem` interface returns the correct vectorized-indexed result for coordinate and boolean indexers.""" z = zarr.create_array(store=store, shape=(2, 2), chunks=(1, 1), zarr_format=3, dtype="i8") @@ -2163,7 +2160,6 @@ async def test_async_vindex(self, store, indexer, expected): result = await async_zarr.vindex.getitem(indexer) assert_array_equal(result, expected) - @pytest.mark.asyncio async def test_async_vindex_with_zarr_array(self, store): """The async `vindex.getitem` interface accepts a zarr 2D boolean array as the indexer and returns the correct elements.""" group = zarr.create_group(store=store, zarr_format=3) @@ -2180,7 +2176,6 @@ async def test_async_vindex_with_zarr_array(self, store): expected = np.array([2, 4]) assert_array_equal(result, expected) - @pytest.mark.asyncio async def test_async_invalid_indexer(self, store): """The async `vindex.getitem` and `oindex.getitem` interfaces raise IndexError when given an unsupported indexer type.""" z = zarr.create_array(store=store, shape=(2, 2), chunks=(1, 1), zarr_format=3, dtype="i8") From 5e2ff9a43f7d83b00b0ad8bf1a28fafcf25d2e94 Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 22 May 2026 22:05:51 +0200 Subject: [PATCH 24/27] docs: add changelog fragment for indexing test cleanup Rename XXXX.misc.md to the PR number when the PR is opened. Co-Authored-By: Claude Opus 4.7 (1M context) --- changes/XXXX.misc.md | 1 + 1 file changed, 1 insertion(+) create mode 100644 changes/XXXX.misc.md diff --git a/changes/XXXX.misc.md b/changes/XXXX.misc.md new file mode 100644 index 0000000000..1f44b551ed --- /dev/null +++ b/changes/XXXX.misc.md @@ -0,0 +1 @@ +Consolidated the array indexing test suite (`tests/test_indexing.py`): the loop-and-`np.random` based selection tests were rewritten as deterministic, parametrized `Expect`/`ExpectFail` cases on small arrays, error paths were split into their own named tests, and the two divergent `Expect` test-case dataclass pairs were unified onto the canonical one in `tests/conftest.py` (whose `ExpectFail` now has an optional regex `msg` and a `raises()` helper). Test-only change with no effect on the public API. From ac0989e84d43ca64f2f4808e7df1e56597e91700 Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 22 May 2026 22:08:50 +0200 Subject: [PATCH 25/27] test: explain msg=None on the basic-1d string bad case Co-Authored-By: Claude Opus 4.7 (1M context) --- tests/test_indexing.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/test_indexing.py b/tests/test_indexing.py index e033fa812c..a9358e4fcf 100644 --- a/tests/test_indexing.py +++ b/tests/test_indexing.py @@ -235,6 +235,8 @@ def test_get_basic_selection_0d(store: StorePath, use_out: bool, value: Any, dty msg="unsupported selection item for basic indexing; expected integer or slice, got ", escape=True, ), + # get_basic_selection and z[...] word their errors differently for a string + # selection, so this case asserts only the exception type (msg=None). ExpectFail( input="foo", exception=IndexError, From e271a5ade908c707fc1597f7140e278660cf2105 Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 22 May 2026 22:18:25 +0200 Subject: [PATCH 26/27] docs: remove LLM plans --- .../plans/2026-05-22-indexing-test-cleanup.md | 769 ------------------ ...2026-05-22-indexing-test-cleanup-design.md | 220 ----- 2 files changed, 989 deletions(-) delete mode 100644 docs/superpowers/plans/2026-05-22-indexing-test-cleanup.md delete mode 100644 docs/superpowers/specs/2026-05-22-indexing-test-cleanup-design.md diff --git a/docs/superpowers/plans/2026-05-22-indexing-test-cleanup.md b/docs/superpowers/plans/2026-05-22-indexing-test-cleanup.md deleted file mode 100644 index 88ad160c7c..0000000000 --- a/docs/superpowers/plans/2026-05-22-indexing-test-cleanup.md +++ /dev/null @@ -1,769 +0,0 @@ -# Indexing Test Cleanup Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Consolidate and speed up `tests/test_indexing.py` by shrinking oversized arrays, replacing `np.random` selection loops with hand-picked parametrized cases, isolating one behavior per test, and unifying the duplicated `Expect` test-case dataclasses onto one canonical pair. - -**Architecture:** First deduplicate the two divergent `Expect`/error dataclass pairs (`tests/conftest.py` vs `tests/test_codecs/conftest.py`) onto the richer `tests/conftest.py` pair, migrating the three codec/chunk-grid consumers. Then rewrite each indexing test family: smaller arrays (≥3 chunks/axis, partial edge), explicit `Expect[Selection, None]` / `ExpectFail[Selection]` case tables parametrized with `ids=lambda c: c.id`, comparing zarr against a numpy oracle. Error paths become their own named parametrized tests. - -**Tech Stack:** Python 3.12, pytest, numpy, zarr; tests run via `uv run --frozen pytest`. Lint/type via `prek`. - -**Spec:** `docs/superpowers/specs/2026-05-22-indexing-test-cleanup-design.md` (gist: https://gist.github.com/d-v-b/508a7294cba8bc702f36a4b85f4a8a90) - -**Conventions (from project memory — apply throughout):** -- Every new/rewritten test gets a docstring stating the *behavior* it verifies. -- Compact happy-path via parametrization; each exception path gets its own named test. -- Single backticks in docstrings; no RST roles, no double-backticks. -- No falsy conditionals — test `is None` etc. explicitly. -- Run mypy via `prek --all-files`, never ad-hoc `uvx mypy`. -- Commit messages end with the `Co-Authored-By: Claude Opus 4.7 (1M context)` trailer. - ---- - -## Part 0 — Deduplicate the `Expect` dataclasses - -The canonical pair lives in `tests/conftest.py`: - -```python -@dataclass -class Expect[TIn, TOut]: - """A test case with explicit input, expected output, and a human-readable id.""" - input: TIn - output: TOut - id: str - -@dataclass -class ExpectFail[TIn]: - """A test case that should raise an exception.""" - input: TIn - exception: type[Exception] - id: str - msg: str -``` - -The duplicate to delete lives in `tests/test_codecs/conftest.py`: - -```python -@dataclass(frozen=True) -class Expect[TIn, TOut]: - input: TIn - expected: TOut - -@dataclass(frozen=True) -class ExpectErr[TIn]: - input: TIn - msg: str - exception_cls: type[Exception] -``` - -**Migration rule** (applied at every call site in the three consumer files): -- `Expect(input=X, expected=Y)` → `Expect(input=X, output=Y, id="")` -- `ExpectErr(input=X, msg=M, exception_cls=E)` → `ExpectFail(input=X, exception=E, id="", msg=M)` -- The `` comes from the existing positional `ids=[...]` list on the `@pytest.mark.parametrize` for that block (1:1 by order). After moving ids onto cases, replace `ids=[...]` with `ids=lambda c: c.id`. -- Where a parametrize block has no `ids=` today, synthesize concise kebab-case ids. -- Import line in each consumer becomes `from tests.conftest import Expect, ExpectFail`. - -Consumers (all import both classes from `tests.test_codecs.conftest`): -`tests/test_chunk_grids.py`, `tests/test_codecs/test_cast_value.py`, `tests/test_codecs/test_scale_offset.py`. - -### Task 0.1: Make the canonical `Expect`/`ExpectFail` frozen - -**Files:** -- Modify: `tests/conftest.py:67-83` - -- [ ] **Step 1: Add `frozen=True` to both canonical dataclasses** - -In `tests/conftest.py`, change the two decorators from `@dataclass` to `@dataclass(frozen=True)`: - -```python -@dataclass(frozen=True) -class Expect[TIn, TOut]: - """A test case with explicit input, expected output, and a human-readable id.""" - - input: TIn - output: TOut - id: str - - -@dataclass(frozen=True) -class ExpectFail[TIn]: - """A test case that should raise an exception.""" - - input: TIn - exception: type[Exception] - id: str - msg: str -``` - -- [ ] **Step 2: Verify the existing canonical consumer still passes** - -Run: `uv run --frozen pytest tests/test_metadata/test_v3.py -q` -Expected: PASS (no field names changed for this consumer; only `frozen` added). - -- [ ] **Step 3: Commit** - -```bash -git add tests/conftest.py -git commit -m "$(cat <<'EOF' -test: make canonical Expect/ExpectFail frozen - -Prepares for deduplicating the second Expect pair in test_codecs/conftest.py. - -Co-Authored-By: Claude Opus 4.7 (1M context) -EOF -)" -``` - -### Task 0.2: Migrate `tests/test_chunk_grids.py` to canonical pair - -**Files:** -- Modify: `tests/test_chunk_grids.py` (import line 7; all `Expect(...)`, `ExpectErr(...)`, and `ids=[...]` blocks) - -- [ ] **Step 1: Change the import** - -Replace `from tests.test_codecs.conftest import Expect, ExpectErr` with `from tests.conftest import Expect, ExpectFail`. - -- [ ] **Step 2: Migrate every `Expect(...)` and `ExpectErr(...)` call per the rule above** - -Worked example — the `test_normalize_chunks_1d_errors` block (currently lines 131-175). The existing `ids=[...]` list maps 1:1 to the cases. After migration: - -```python -@pytest.mark.parametrize( - "case", - [ - ExpectFail(input=(0, 100), exception=ValueError, id="zero-uniform", msg="Chunk size must be positive"), - ExpectFail(input=(-2, 100), exception=ValueError, id="negative-uniform", msg="Chunk size must be positive"), - ExpectFail(input=([], 100), exception=ValueError, id="empty-list", msg="must not be empty"), - ExpectFail(input=([10, -1, 10], 100), exception=ValueError, id="negative-element", msg="must be positive"), - ExpectFail(input=([10, 0, 10], 20), exception=ValueError, id="zero-element", msg="must be positive"), - ExpectFail(input=([10, 20], 100), exception=ValueError, id="wrong-sum", msg="do not sum to span"), - ExpectFail(input=([[3, 3], 1], 7), exception=TypeError, id="rle-single-dim", msg="non-integer element(s) ([3, 3],) at indices (0,)"), - ExpectFail(input=([1, [2, 2], 1, [3]], 9), exception=TypeError, id="multiple-non-ints", msg="non-integer element(s) ([2, 2], [3]) at indices (1, 3)"), - ExpectFail(input=([2, "3", 5], 10), exception=TypeError, id="string-element", msg="non-integer element(s) ('3',) at indices (1,)"), - ], - ids=lambda c: c.id, -) -def test_normalize_chunks_1d_errors(case: ExpectFail[tuple[Any, int]]) -> None: - """Invalid 1D chunk specifications are rejected with informative error messages.""" - chunks, span = case.input - with pytest.raises(case.exception, match=re.escape(case.msg)): - normalize_chunks_1d(chunks, span=span) -``` - -Apply the same transformation to the remaining parametrize blocks in this file: -the `test_normalize_chunks_nd_errors` block (ids `["none", "true", "string", "too-many-dims", "too-few-dims", "rle-inner-dim"]`), and the success-case `Expect(...)` blocks (rename `expected=`→`output=`, add `id=` from the block's `ids=[...]`, switch `ids=` to `lambda c: c.id`). Update each function's `case:` type annotation: `ExpectErr[...]` → `ExpectFail[...]`, and references `case.exception_cls` → `case.exception`. - -- [ ] **Step 3: Run the file to verify green** - -Run: `uv run --frozen pytest tests/test_chunk_grids.py -q` -Expected: PASS, same test count as before (ids change but cases don't). - -- [ ] **Step 4: Commit** - -```bash -git add tests/test_chunk_grids.py -git commit -m "$(cat <<'EOF' -test: migrate test_chunk_grids to canonical Expect/ExpectFail - -Co-Authored-By: Claude Opus 4.7 (1M context) -EOF -)" -``` - -### Task 0.3: Migrate `tests/test_codecs/test_cast_value.py` to canonical pair - -**Files:** -- Modify: `tests/test_codecs/test_cast_value.py` (import line 9; all `Expect(...)`/`ExpectErr(...)`/`ids=[...]` blocks) - -- [ ] **Step 1: Change the import** to `from tests.conftest import Expect, ExpectFail`. - -- [ ] **Step 2: Migrate every call site per the rule.** For success cases, `expected=`→`output=`, add `id=` taken 1:1 from the block's `ids=[...]` list (e.g. `["minimal", "full"]`, `["defaults", "explicit"]`, `["no-scalar-map", "with-scalar-map"]`, `["complex-source", "wrap-float-target"]`, `["f64→f32", "f32→f64", "i32→i64", "i64→i16", "f64→i32", "i32→f64"]`, `["towards-zero"]`, `["int32→int8"]`), then set `ids=lambda c: c.id`. For error blocks, `ExpectErr`→`ExpectFail`, `exception_cls=`→`exception=`, add `id=`, and update `case.exception_cls`→`case.exception` plus the `case:` type annotations. - -- [ ] **Step 3: Run the file** - -Run: `uv run --frozen pytest tests/test_codecs/test_cast_value.py -q` -Expected: PASS, same test count. - -- [ ] **Step 4: Commit** - -```bash -git add tests/test_codecs/test_cast_value.py -git commit -m "$(cat <<'EOF' -test: migrate test_cast_value to canonical Expect/ExpectFail - -Co-Authored-By: Claude Opus 4.7 (1M context) -EOF -)" -``` - -### Task 0.4: Migrate `tests/test_codecs/test_scale_offset.py` to canonical pair - -**Files:** -- Modify: `tests/test_codecs/test_scale_offset.py` (import line 9; all call sites) - -- [ ] **Step 1: Change the import** to `from tests.conftest import Expect, ExpectFail`. - -- [ ] **Step 2: Migrate every call site per the rule** (same mechanical transformation as Task 0.2/0.3). This file's success cases at lines 27-38 have no `ids=` list today — synthesize ids: `id="default"`, `id="offset-only"`, `id="scale-only"`, `id="offset-and-scale"` matching the four `Expect(...)` cases in order. Error blocks (`ExpectErr` at ~82-87 and ~262-272): convert to `ExpectFail`, add ids (`"non-numeric-offset"`, `"non-numeric-scale"` and `"unrepresentable-1"`, `"unrepresentable-2"`, `"unrepresentable-3"` respectively — pick descriptive names from each case's input). Update annotations and `.exception_cls`→`.exception`. - -- [ ] **Step 3: Run the file** - -Run: `uv run --frozen pytest tests/test_codecs/test_scale_offset.py -q` -Expected: PASS, same test count. - -- [ ] **Step 4: Commit** - -```bash -git add tests/test_codecs/test_scale_offset.py -git commit -m "$(cat <<'EOF' -test: migrate test_scale_offset to canonical Expect/ExpectFail - -Co-Authored-By: Claude Opus 4.7 (1M context) -EOF -)" -``` - -### Task 0.5: Delete the duplicate dataclasses and verify nothing imports them - -**Files:** -- Delete contents of: `tests/test_codecs/conftest.py` (file is only these two classes) - -- [ ] **Step 1: Confirm no remaining importers** - -Run: `grep -rn "from tests.test_codecs.conftest import\|test_codecs.conftest import" tests/` -Expected: no output (all three consumers migrated in 0.2–0.4). - -- [ ] **Step 2: Delete the file** - -`tests/test_codecs/conftest.py` contains only the two duplicate dataclasses, so remove it entirely: - -```bash -git rm tests/test_codecs/conftest.py -``` - -- [ ] **Step 3: Run the full codecs + chunk-grids + v3 suites** - -Run: `uv run --frozen pytest tests/test_chunk_grids.py tests/test_codecs/ tests/test_metadata/test_v3.py -q` -Expected: PASS, no collection errors. - -- [ ] **Step 4: Run prek on touched files** - -Run: `prek run --all-files` -Expected: all hooks pass (mypy in particular — the `case:` annotations now reference `ExpectFail`). - -- [ ] **Step 5: Commit** - -```bash -git add -A -git commit -m "$(cat <<'EOF' -test: delete duplicate Expect dataclasses in test_codecs/conftest.py - -All consumers now use the canonical Expect/ExpectFail from tests/conftest.py. - -Co-Authored-By: Claude Opus 4.7 (1M context) -EOF -)" -``` - -### Task 0.6: Make `ExpectFail.msg` optional with a `raises()` helper - -**Motivation:** `ExpectFail` requires a `msg: str`, and consumers do -`pytest.raises(case.exception, match=case.msg)`. Passing `msg=""` to express -"any message" triggers pytest's "matching against an empty string will always -pass" warning, which this repo escalates to an error via -`filterwarnings=["error"]`. Make "don't care about the message" expressible and -move the `match`/escape logic into one place. - -**Files:** -- Modify: `tests/conftest.py` (the `ExpectFail` dataclass) -- Modify all 9 consumer sites: `tests/test_chunk_grids.py` (2), - `tests/test_metadata/test_v3.py` (2), `tests/test_codecs/test_cast_value.py` - (2), `tests/test_codecs/test_scale_offset.py` (2), `tests/test_indexing.py` (1) - -- [ ] **Step 1: Add optional `msg`/`escape` and a `raises()` method to `ExpectFail`** - -```python -import re -from contextlib import AbstractContextManager - -import pytest - - -@dataclass(frozen=True) -class ExpectFail[TIn]: - """A test case that should raise an exception. - - `msg` is treated as a regex matched against the exception text (pytest's - native `match=` semantics). Leave it `None` to assert only the exception - type. Set `escape=True` when `msg` is a literal containing regex - metacharacters. - """ - - input: TIn - exception: type[Exception] - id: str - msg: str | None = None - escape: bool = False - - def raises(self) -> AbstractContextManager[pytest.ExceptionInfo[Exception]]: - if self.msg is None: - return pytest.raises(self.exception) - pattern = re.escape(self.msg) if self.escape else self.msg - return pytest.raises(self.exception, match=pattern) -``` - -(Place `import re` and `import pytest` with the existing imports if not already -present; `pytest` is certainly already imported in conftest.) - -- [ ] **Step 2: Update all 9 consumer sites** to use the helper. Replace each - `with pytest.raises(case.exception, match=re.escape(case.msg)):` and - `with pytest.raises(case.exception, match=case.msg):` with: - -```python - with case.raises(): -``` - -For the two `tests/test_chunk_grids.py` sites that used `re.escape(case.msg)`: -those three cases whose `msg` contains regex metacharacters -(`non-integer element(s) ([3, 3],) at indices (0,)` and the two like it) must -set `escape=True` on the `ExpectFail(...)` so the helper escapes them. All -other chunk_grids cases stay `escape=False` (default). Remove the now-unused -`import re` from `test_chunk_grids.py` if nothing else uses it. - -For `tests/test_metadata/test_v3.py`: the `msg=".*"` case (match-anything) may -become `msg=None` (drop the field) — behavior-preserving. The -`msg="dimension_names.*shape"` case keeps its regex `msg` with `escape=False`. - -- [ ] **Step 3: Run all affected suites** - -Run: `uv run --frozen pytest tests/test_chunk_grids.py tests/test_codecs/ tests/test_metadata/test_v3.py tests/test_indexing.py -q` -Expected: PASS, same counts as before this task (the matching behavior is -unchanged for every existing case; only the empty-string footgun is removed). - -- [ ] **Step 4: prek** - -Run: `prek run --all-files` -Expected: all hooks pass. - -- [ ] **Step 5: Commit** - -```bash -git add -A -git commit -m "$(cat <<'EOF' -test: make ExpectFail.msg optional with a raises() helper - -msg defaults to None (assert exception type only) and is treated as a regex, -with an escape flag for literal messages. Removes the msg="" footgun that the -repo's filterwarnings=["error"] config turned into a failure. - -Co-Authored-By: Claude Opus 4.7 (1M context) -EOF -)" -``` - ---- - -## Part 1 — Rewrite the indexing test families - -All indexing tests use the file-local `store` fixture (MemoryStore, runs once -per test) and the `zarr_array_from_numpy_array(store, a, chunk_shape=...)` -helper. Both stay. The numpy-oracle assertion helpers -(`_test_get_basic_selection`, `_test_get_orthogonal_selection`, -`_test_set_orthogonal_selection`, `_test_get_coordinate_selection`, -`_test_set_coordinate_selection`, `_test_get_block_selection`, -`_test_set_block_selection`, `_test_get_mask_selection`, -`_test_set_mask_selection`) are **kept** — they encapsulate the -zarr-vs-numpy comparison and are now called once per parametrized case. - -**Array-size rule (fixed):** every indexed axis spans ≥3 chunks; at least one -axis has a partial (non-full) edge chunk. Canonical shapes: -- 1d: `np.arange(30)`, chunks `(7,)` → 5 chunks, last size 2. -- 2d: `np.arange(60).reshape(12, 5)`, chunks `(5, 2)` → 3×3 chunks, partial edges. -- 3d: `np.arange(420).reshape(7, 6, 10)`, chunks `(3, 2, 4)` → 3×3×3 chunks, partial edges everywhere. - -Add the import for the case dataclasses at the top of `tests/test_indexing.py`: -`from tests.conftest import Expect, ExpectFail`. - -### Task 1.1 (EXEMPLAR): Rewrite `test_get_orthogonal_selection_1d_bool` and split its error paths - -This task is the worked template. Tasks 1.2+ apply the same recipe and reference it. - -**Recipe (apply to every family task):** -1. Shrink the array to the canonical shape for its dimensionality. -2. Replace the `np.random.seed` + `for p in ...` loop with an explicit - module-level `Expect[Selection, None]` list named `__CASES`, each - case carrying a hand-picked selection in `input`, `output=None`, and a - descriptive `id`. Cover: empty (all-False mask / size-0 slice), full, - alternating, single-element, sparse, and (for int arrays) sorted, unsorted, - duplicate, negative/wraparound. -3. Make the test parametrized over that list with `ids=lambda c: c.id`, calling - the kept oracle helper with `case.input`. -4. Extract the bundled `pytest.raises(IndexError)` block into a separate - `test__raises` parametrized over an `ExpectFail[Selection]` list - `__BAD_CASES` (each with `exception=IndexError`, an `id`, and an - optional `msg`). The raises test body uses the `case.raises()` helper - (added in Task 0.6): - ```python - with case.raises(): - z.oindex[case.input] - ``` - `msg` is a regex matched against the error text; prefer a stable substring - of the real message (run the bad selection once to capture it). Omit `msg` - (leave it `None`) to assert only the exception type. Set `escape=True` if a - literal `msg` contains regex metacharacters. Never pass `msg=""`. -5. Give both tests docstrings stating the behavior. - -**Files:** -- Modify: `tests/test_indexing.py` (replace `test_get_orthogonal_selection_1d_bool`, lines 615-633) - -- [ ] **Step 1: Add the case tables and rewritten tests** - -Replace the existing `test_get_orthogonal_selection_1d_bool` with: - -```python -_ORTHO_1D_BOOL_CASES: list[Expect[OrthogonalSelection, None]] = [ - Expect(input=np.zeros(30, dtype=bool), output=None, id="empty-mask"), - Expect(input=np.ones(30, dtype=bool), output=None, id="full-mask"), - Expect(input=np.arange(30) % 2 == 0, output=None, id="alternating-mask"), - Expect(input=np.arange(30) == 7, output=None, id="single-true"), - Expect( - input=np.isin(np.arange(30), [0, 1, 8, 15, 29]), - output=None, - id="sparse-cross-chunk", - ), -] - -_ORTHO_1D_BOOL_BAD_CASES: list[ExpectFail[Any]] = [ - ExpectFail(input=np.zeros(5, dtype=bool), exception=IndexError, id="mask-too-short", msg="wrong length for dimension; expected 30, got 5"), - ExpectFail(input=np.zeros(50, dtype=bool), exception=IndexError, id="mask-too-long", msg="wrong length for dimension; expected 30, got 50"), - ExpectFail( - input=[[True, False], [False, True]], - exception=IndexError, - id="mask-too-many-dims", - msg="must be 1-dimensional only", - ), -] -# msg is an optional regex matched against the real error text; omit it to -# assert only the exception type. The raises test body uses `with case.raises():`. - - -@pytest.mark.parametrize("case", _ORTHO_1D_BOOL_CASES, ids=lambda c: c.id) -def test_get_orthogonal_selection_1d_bool(store: StorePath, case: Expect[OrthogonalSelection, None]) -> None: - """oindex with a 1D boolean mask matches numpy across chunk boundaries.""" - a = np.arange(30, dtype=int) - z = zarr_array_from_numpy_array(store, a, chunk_shape=(7,)) - _test_get_orthogonal_selection(a, z, case.input) - - -@pytest.mark.parametrize("case", _ORTHO_1D_BOOL_BAD_CASES, ids=lambda c: c.id) -def test_get_orthogonal_selection_1d_bool_raises( - store: StorePath, case: ExpectFail[Any] -) -> None: - """oindex rejects masks of the wrong length or dimensionality with IndexError.""" - a = np.arange(30, dtype=int) - z = zarr_array_from_numpy_array(store, a, chunk_shape=(7,)) - with case.raises(): - z.oindex[case.input] -``` - -- [ ] **Step 2: Run the rewritten tests to verify they pass** - -Run: `uv run --frozen pytest "tests/test_indexing.py::test_get_orthogonal_selection_1d_bool" "tests/test_indexing.py::test_get_orthogonal_selection_1d_bool_raises" -v` -Expected: PASS — 5 parametrized happy-path cases (by id) + 3 raises cases. - -- [ ] **Step 3: Run the whole file to confirm no breakage** - -Run: `uv run --frozen pytest tests/test_indexing.py -q` -Expected: PASS (one fewer monolithic test, several new parametrized cases). - -- [ ] **Step 4: Commit** - -```bash -git add tests/test_indexing.py -git commit -m "$(cat <<'EOF' -test: rewrite orthogonal 1d bool indexing as parametrized cases - -Smaller array (30 elems, chunks of 7), hand-picked deterministic masks -replacing np.random sparsity sweep, error paths split into their own test. - -Co-Authored-By: Claude Opus 4.7 (1M context) -EOF -)" -``` - -### Task 1.2: Rewrite `test_get_orthogonal_selection_1d_int` - -**Files:** Modify `tests/test_indexing.py` (lines 637-668). - -- [ ] **Step 1: Apply the recipe.** Array `np.arange(30)`, chunks `(7,)`. Happy-path `_ORTHO_1D_INT_CASES` covering: `id="sorted"` (`[0, 8, 15, 29]`), `id="unsorted"` (`[3, 29, 1, 16]`), `id="duplicates"` (`[2, 2, 8, 8]`), `id="wraparound"` (`[0, 3, 10, -23, -12, -1]`), `id="single"` (`[15]`). Error `_ORTHO_1D_INT_BAD_CASES` (each `ExpectFail(exception=IndexError, msg="")`): `id="out-of-bounds-high"` (`[31]`), `id="out-of-bounds-low"` (`[-31]`), `id="too-many-dims"` (`[[2, 4], [6, 8]]`). The raises test asserts both `z.get_orthogonal_selection(case.input)` and `z.oindex[case.input]` raise (two `pytest.raises` blocks in the test body, as the original did). Docstrings on both. - -- [ ] **Step 2: Run the two tests** - -Run: `uv run --frozen pytest "tests/test_indexing.py::test_get_orthogonal_selection_1d_int" "tests/test_indexing.py::test_get_orthogonal_selection_1d_int_raises" -v` -Expected: PASS. - -- [ ] **Step 3: Commit** - -```bash -git add tests/test_indexing.py -git commit -m "test: rewrite orthogonal 1d int indexing as parametrized cases - -Co-Authored-By: Claude Opus 4.7 (1M context) " -``` - -### Task 1.3: Rewrite `test_get_orthogonal_selection_2d` and `_test_get_orthogonal_selection_2d` - -**Files:** Modify `tests/test_indexing.py` (lines 671-727). - -- [ ] **Step 1: Apply the recipe.** Array `np.arange(60).reshape(12, 5)`, chunks `(5, 2)`. The private `_test_get_orthogonal_selection_2d` helper currently loops over 7 selection shapes built from two index arrays — instead build a `_ORTHO_2D_CASES: list[Expect[OrthogonalSelection, None]]` table directly with concrete hand-picked `ix0`/`ix1` (defined once at module scope), covering the same shapes: both-axes-array, array×slice, array×strided-slice, slice×array, array×int, int×array, plus the mixed int-array/bool-array pair. Use deterministic indices, e.g. `ix0_bool = np.isin(np.arange(12), [0, 5, 11])`, `ix1_bool = np.array([True, False, True, False, True])`, `ix0_int = np.array([0, 5, 11])`, `ix1_int = np.array([0, 2, 4])`. Fold the `basic_selections_2d` coverage of orthogonal into existing parametrize ids. Error cases from `basic_selections_2d_bad` → `_ORTHO_2D_BAD_CASES` (`ExpectFail`, `IndexError`, `msg=""`), raises test asserting both `get_orthogonal_selection` and `oindex`. Delete the now-unused `_test_get_orthogonal_selection_2d` helper. Docstrings on both tests. - -- [ ] **Step 2: Run** - -Run: `uv run --frozen pytest "tests/test_indexing.py::test_get_orthogonal_selection_2d" "tests/test_indexing.py::test_get_orthogonal_selection_2d_raises" -v` -Expected: PASS. - -- [ ] **Step 3: Commit** - -```bash -git add tests/test_indexing.py -git commit -m "test: rewrite orthogonal 2d indexing as parametrized cases - -Co-Authored-By: Claude Opus 4.7 (1M context) " -``` - -### Task 1.4: Rewrite `test_get_orthogonal_selection_3d` and `_test_get_orthogonal_selection_3d` - -**Files:** Modify `tests/test_indexing.py` (lines 729-792). - -- [ ] **Step 1: Apply the recipe.** Array `np.arange(420).reshape(7, 6, 10)`, chunks `(3, 2, 4)`. The private helper enumerates 20 selection tuples — turn each into an `Expect` case in `_ORTHO_3D_CASES` with a descriptive id (e.g. `"single-value"`, `"all-negative"`, `"three-arrays"`, `"array-slice-slice"`, ... matching the comment groupings in the original). Define the hand-picked index arrays once at module scope: `ix0 = np.isin(np.arange(7), [0, 3, 6])` (bool) and integer variants per axis. Keep both bool-array and sorted-int-array variants by including both kinds as separate ids rather than a sparsity loop. Adjust the literal int indices in the tuples (e.g. `60, 15, 4`) to be in-bounds for the new `(7, 6, 10)` shape (e.g. `5, 3, 8`). Delete the unused helper. Docstring. - -- [ ] **Step 2: Run** - -Run: `uv run --frozen pytest "tests/test_indexing.py::test_get_orthogonal_selection_3d" -v` -Expected: PASS. - -- [ ] **Step 3: Commit** - -```bash -git add tests/test_indexing.py -git commit -m "test: rewrite orthogonal 3d indexing as parametrized cases - -Co-Authored-By: Claude Opus 4.7 (1M context) " -``` - -### Task 1.5: Rewrite the set-orthogonal family (1d/2d/3d) and helpers - -**Files:** Modify `tests/test_indexing.py` (`test_set_orthogonal_selection_1d/2d/3d` and their `_test_set_orthogonal_selection_2d/3d` helpers, lines 807-969). - -- [ ] **Step 1: Apply the recipe** to all three set tests, reusing the same `_ORTHO_*_CASES` selection tables defined in Tasks 1.1-1.4 where the selections are valid for set (most are). The kept `_test_set_orthogonal_selection` helper already iterates over the three value forms (scalar / array / list) per selection and skips selections that produce an empty result (the existing `value == []` guard), so each parametrized case still exercises all applicable value forms. If a get-only case turns out not to round-trip through set, give it its own filtered list (e.g. `_ORTHO_3D_SET_CASES`) rather than forcing it. Use the same canonical shapes/chunks. For 1d, parametrize over `_ORTHO_1D_BOOL_CASES + _ORTHO_1D_INT_CASES` minus any that can't be set (e.g. empty selections that can't preserve dimensions — keep the existing skip-empty guard inside `_test_set_orthogonal_selection`). Delete the now-unused `_test_set_orthogonal_selection_2d` and `_test_set_orthogonal_selection_3d` helpers. Docstrings on all three. - -- [ ] **Step 2: Run** - -Run: `uv run --frozen pytest "tests/test_indexing.py::test_set_orthogonal_selection_1d" "tests/test_indexing.py::test_set_orthogonal_selection_2d" "tests/test_indexing.py::test_set_orthogonal_selection_3d" -v` -Expected: PASS. - -- [ ] **Step 3: Commit** - -```bash -git add tests/test_indexing.py -git commit -m "test: rewrite set-orthogonal indexing family as parametrized cases - -Co-Authored-By: Claude Opus 4.7 (1M context) " -``` - -### Task 1.6: Rewrite the basic-selection 1d/2d families - -**Files:** Modify `tests/test_indexing.py` (`basic_selections_1d`, `basic_selections_1d_bad`, `test_get_basic_selection_1d`, `basic_selections_2d`, `basic_selections_2d_bad`, `test_get_basic_selection_2d`, lines 204-385). - -- [ ] **Step 1: Shrink the selection tables to the new array sizes and parametrize.** - - 1d array `np.arange(30)`, chunks `(7,)`. Rewrite `basic_selections_1d` as a trimmed `Expect[BasicSelection, None]` list: keep one representative of each kind — single value (`5`, `-1`), bounded slice (`slice(3, 18)`), over-bounds slice (`slice(0, 100)`), negative slice (`slice(-18, -3)`), empty (`slice(0, 0)`, `slice(-1, 0)`), full (`slice(None)`, `Ellipsis`, `()`), and a few stepped slices (`slice(None, None, 3)`, `slice(3, 27, 5)`). Drop the giant step-sweep (steps 10..10000 on a 30-element array are redundant). Each gets an id. - - `basic_selections_1d_bad` → `_BASIC_1D_BAD_CASES` (`ExpectFail`, `IndexError`, `msg=""`): keep one negative-step slice (`slice(None, None, -1)`), the type errors (`2.3`, `"foo"`, `b"xxx"`, `None`), the shape errors (`(0, 0)`, `(slice(None), slice(None))`), and the integer-list case `[1, 0]`. raises test asserts both `get_basic_selection` and `z[...]`. - - Same treatment for 2d (`np.arange(60).reshape(12, 5)`, chunks `(5, 2)`): trim `basic_selections_2d` to representatives, adjust literal indices to be in-bounds (e.g. `42`→`5`, `slice(250, 350)`→`slice(2, 9)`). Keep the fancy-indexing fallback assertion (`z[([0, 1], [0, 1])]`) as a small standalone test `test_basic_2d_fancy_fallback` with a docstring. - - Keep the kept `_test_get_basic_selection` oracle helper (it also checks the `out=` param). Docstrings on all. - -- [ ] **Step 2: Run** - -Run: `uv run --frozen pytest "tests/test_indexing.py::test_get_basic_selection_1d" "tests/test_indexing.py::test_get_basic_selection_2d" -v` -Expected: PASS. - -- [ ] **Step 3: Commit** - -```bash -git add tests/test_indexing.py -git commit -m "test: rewrite basic 1d/2d selection families as parametrized cases - -Co-Authored-By: Claude Opus 4.7 (1M context) " -``` - -### Task 1.7: Rewrite the coordinate-selection family (get/set, 1d/2d) - -**Files:** Modify `tests/test_indexing.py` (`test_get_coordinate_selection_1d/2d`, `test_set_coordinate_selection_1d/2d`, `_test_set_coordinate_selection`, and `coordinate_selections_1d_bad`, lines 1019-1186). - -- [ ] **Step 1: Apply the recipe.** 1d array `np.arange(30)`, chunks `(7,)`. `_COORD_1D_CASES` (`Expect[CoordinateSelection, None]`): single (`5`, `-1`), wraparound (`[0, 3, 10, -23, -12, -1]` → adjust to in-bounds for 30: `[0, 3, 10, -23, -12, -1]` is valid since -23..−1 map into 0..29), out-of-order (`[3, 25, 8, 17]`), multi-dim (`np.array([[2, 4], [6, 8]])`), sorted (`[1, 8, 15, 29]`), reversed (`[29, 15, 8, 1]`). Replace the `for p in 2, 0.5, 0.1, 0.01` random sweep entirely. Error `_COORD_1D_BAD_CASES` from `coordinate_selections_1d_bad` + out-of-bounds (`[31]`, `[-31]`). 2d array `np.arange(60).reshape(12, 5)`, chunks `(5, 2)`: `_COORD_2D_CASES` covering single `(5, 4)`, `(-1, -1)`, both-axes-array `(ix0, ix1)` with deterministic `ix0=[0,5,11,2,8]`, `ix1=[1,3,4,0,2]`, mixed array/int, not-monotonic cases, multi-dim arrays. 2d error cases (slice mixed with array, Ellipsis) → `_COORD_2D_BAD_CASES`. Keep `_test_get_coordinate_selection`/`_test_set_coordinate_selection` helpers. Docstrings throughout. - -- [ ] **Step 2: Run** - -Run: `uv run --frozen pytest tests/test_indexing.py -k coordinate -v` -Expected: PASS. - -- [ ] **Step 3: Commit** - -```bash -git add tests/test_indexing.py -git commit -m "test: rewrite coordinate selection family as parametrized cases - -Co-Authored-By: Claude Opus 4.7 (1M context) " -``` - -### Task 1.8: Rewrite the block-selection family (get/set, 1d/2d) - -**Files:** Modify `tests/test_indexing.py` (`block_selections_*`, `test_get_block_selection_1d/2d`, `test_set_block_selection_1d/2d`, lines 1189-1378). - -- [ ] **Step 1: Apply the recipe, preserving the selection↔projection pairing.** The block tests pair a block selection with the array slice it projects to. Keep that by making each case carry both, using `Expect[BasicSelection, slice | tuple[slice, ...]]` where `input` is the block selection and `output` is the expected array-index projection (this is the one family where `output` is meaningfully used). 1d array `np.arange(30)`, chunks `(7,)` → 5 blocks. Recompute the projections for the new chunking: block `0`→`slice(0,7)`, block `4`→`slice(28,30)`, `-1`→`slice(28,30)`, `slice(None,3)`→`slice(0,21)`, etc. Build `_BLOCK_1D_CASES` with ids. Error `_BLOCK_1D_BAD_CASES` from `block_selections_1d_bad` + out-of-bounds (`n_blocks+1`, `-(n_blocks+1)` computed from `z._chunk_grid.get_nchunks()`). 2d array `np.arange(60).reshape(12, 5)`, chunks `(5, 2)` → 3×3 blocks; recompute the 2d projections. Update `_test_get_block_selection`/`_test_set_block_selection` to take the projection from `case.output`. Docstrings. - -- [ ] **Step 2: Run** - -Run: `uv run --frozen pytest tests/test_indexing.py -k block -v` -Expected: PASS. - -- [ ] **Step 3: Commit** - -```bash -git add tests/test_indexing.py -git commit -m "test: rewrite block selection family as parametrized cases - -Co-Authored-By: Claude Opus 4.7 (1M context) " -``` - -### Task 1.9: Rewrite the mask-selection family (get/set, 1d/2d) - -**Files:** Modify `tests/test_indexing.py` (`mask_selections_1d_bad`, `test_get_mask_selection_1d/2d`, `test_set_mask_selection_1d/2d`, lines 1391-1497). - -- [ ] **Step 1: Apply the recipe.** 1d array `np.arange(30)`, chunks `(7,)`. `_MASK_1D_CASES` (`Expect[Any, None]`): all-false, all-true, alternating (`np.arange(30) % 2 == 0`), sparse (`np.isin(np.arange(30), [0, 7, 14, 29])`). 2d array `np.arange(60).reshape(12, 5)`, chunks `(5, 2)`: `_MASK_2D_CASES`: all-false, all-true, checkerboard (`(np.add.outer(np.arange(12), np.arange(5)) % 2).astype(bool)`), sparse. Replace both `for p in 0.5, 0.1, 0.01` sweeps. Error `_MASK_1D_BAD_CASES` from `mask_selections_1d_bad` + too-short (`np.zeros(5, bool)`), too-long (`np.zeros(50, bool)`), too-many-dims (`[[True, False], [False, True]]`); and a `_MASK_2D_BAD_CASES` for the 2d wrong-shape/wrong-ndim cases. Keep `_test_get_mask_selection`/`_test_set_mask_selection`. Docstrings. - -- [ ] **Step 2: Run** - -Run: `uv run --frozen pytest tests/test_indexing.py -k mask -v` -Expected: PASS. - -- [ ] **Step 3: Commit** - -```bash -git add tests/test_indexing.py -git commit -m "test: rewrite mask selection family as parametrized cases - -Co-Authored-By: Claude Opus 4.7 (1M context) " -``` - -### Task 1.10: Shrink `test_get_selection_out` and the remaining large-array tests - -**Files:** Modify `tests/test_indexing.py` (`test_get_selection_out` lines 1500-1568; `test_indexing_equals_numpy` lines 1853-1880; `test_orthogonal_bool_indexing_like_numpy_ix` lines 1883-1900). - -- [ ] **Step 1: Shrink arrays without changing test logic.** - - `test_get_selection_out`: read the full body (lines 1500-1568) first; reduce `np.arange(1050)`/`(100,)` to `np.arange(30)`/`(7,)` and adjust the literal selection bounds proportionally, keeping the same `out=` buffer assertions. - - `test_indexing_equals_numpy` and `test_orthogonal_bool_indexing_like_numpy_ix`: these already parametrize but on a `(1000, 10)` array. Shrink to `(12, 5)` chunks `(5, 2)` and rescale the selections (e.g. `np.arange(1000)`→`np.arange(12)`, `np.tile([True, False], (1000, 5))`→`np.tile([True, False], (12, ...))` sized to the new shape; `[100, 200, 300]`→in-bounds coords). Add/keep docstrings. - -- [ ] **Step 2: Run** - -Run: `uv run --frozen pytest "tests/test_indexing.py::test_get_selection_out" "tests/test_indexing.py::test_indexing_equals_numpy" "tests/test_indexing.py::test_orthogonal_bool_indexing_like_numpy_ix" -v` -Expected: PASS. - -- [ ] **Step 3: Commit** - -```bash -git add tests/test_indexing.py -git commit -m "test: shrink arrays in selection_out and numpy-equivalence tests - -Co-Authored-By: Claude Opus 4.7 (1M context) " -``` - -### Task 1.11: Add docstrings to the leave-as-is tests and sweep already-parametrized ones - -**Files:** Modify `tests/test_indexing.py` (the GH-regression, xfail/skip, helper, fallback, and iter tests that keep their logic). - -- [ ] **Step 1: Add a one-line behavior docstring** to every test function that lacks one and was not rewritten above: `test_normalize_integer_selection`, `test_replace_ellipsis`, `test_fancy_indexing_fallback_on_get_setitem`, the `*_fallback_*` family, `test_setitem_zarr_array_as_value`, `test_set_basic_selection_0d`, `test_orthogonal_indexing_edge_cases`, `test_set_item_1d_last_two_chunks`, `test_orthogonal_indexing_fallback_on_get_setitem`, `test_slice_selection_uints`, `test_numpy_int_indexing`, `test_accessed_chunks`, `test_iter_grid_invalid`, `test_indexing_with_zarr_array`, `test_zero_sized_chunks`, `test_vectorized_indexing_incompatible_shape`, `test_iter_chunk_regions`, `test_iter_regions`, and the `TestAsync` methods. Do not change their logic or array sizes (regression tests keep their reproducing values per the spec). - -- [ ] **Step 2: Run the whole file** - -Run: `uv run --frozen pytest tests/test_indexing.py -q` -Expected: PASS, same skip/xfail counts as the baseline (1 skipped, 5 xfailed). - -- [ ] **Step 3: Commit** - -```bash -git add tests/test_indexing.py -git commit -m "test: add behavior docstrings to remaining indexing tests - -Co-Authored-By: Claude Opus 4.7 (1M context) " -``` - ---- - -## Part 2 — Final verification - -**Baseline (measured 2026-05-22, before any changes):** full file -`150 passed, 1 skipped, 5 xfailed` in ~3.5 s (pytest) / ~4 s wall. Slowest: -`test_set_orthogonal_selection_3d` 0.92 s, `test_get_orthogonal_selection_1d_bool` -0.79 s, `test_set_orthogonal_selection_2d` 0.54 s, -`test_set_orthogonal_selection_1d` 0.41 s. The post-rewrite test *count* will -differ (monolithic tests split into many parametrized cases); skip/xfail counts -must stay `1`/`5`. - -### Task 2.1: Full-suite verification and speed measurement - -- [ ] **Step 1: Run the whole indexing file with durations** - -Run: `uv run --frozen pytest tests/test_indexing.py --durations=25 -q` -Expected: PASS; the four tests that were >0.4 s at baseline -(`test_set_orthogonal_selection_3d`, `test_get_orthogonal_selection_1d_bool`, -`test_set_orthogonal_selection_2d`, `test_set_orthogonal_selection_1d`) each -now under ~0.15 s. Record the new total in the commit message. - -- [ ] **Step 2: Run the dedup-affected suites once more** - -Run: `uv run --frozen pytest tests/test_chunk_grids.py tests/test_codecs/ tests/test_metadata/test_v3.py -q` -Expected: PASS. - -- [ ] **Step 3: Full lint + type pass** - -Run: `prek run --all-files` -Expected: all hooks pass (ruff, mypy, codespell, etc.). - -- [ ] **Step 4: Add a changelog fragment if the repo requires one** - -Check `changes/` (towncrier). If indexing test-only changes need a fragment per `towncrier-check`, add one; otherwise skip. Run `prek run towncrier-check --all-files` to confirm. - -- [ ] **Step 5: Final commit** - -```bash -git add -A -git commit -m "$(cat <<'EOF' -test: finalize indexing test cleanup - -Full suite green; orthogonal tests dropped from ~0.5-0.9s to <0.15s each. - -Co-Authored-By: Claude Opus 4.7 (1M context) -EOF -)" -``` - ---- - -## Notes for the implementer - -- **Read before you rewrite.** Several tasks reference line ranges that shift - as earlier tasks edit the file. Re-locate each function by name before - editing; don't trust stale line numbers. -- **`Selection` types** (`BasicSelection`, `OrthogonalSelection`, - `CoordinateSelection`, `Selection`) are already imported at the top of - `tests/test_indexing.py` — use them in the `Expect[...]` annotations. -- **Error paths use `case.raises()`** (the `ExpectFail` helper from Task 0.6), - not a bare `pytest.raises`. `msg` is an optional regex; use a stable - substring of the *actual* error (run the bad selection once to capture it), - or omit it to assert only the exception type. Never pass `msg=""`. Use - `escape=True` for literal messages containing regex metacharacters. -- **Keep the oracle helpers.** Do not inline `_test_get_orthogonal_selection` - etc. into the parametrized tests; calling them once per case is the design. -- If a rewritten happy-path case turns out to disagree with numpy (the oracle - fails), that is a real finding — stop and investigate per - systematic-debugging, don't tweak the case to make it pass. -``` diff --git a/docs/superpowers/specs/2026-05-22-indexing-test-cleanup-design.md b/docs/superpowers/specs/2026-05-22-indexing-test-cleanup-design.md deleted file mode 100644 index 5ffdd31978..0000000000 --- a/docs/superpowers/specs/2026-05-22-indexing-test-cleanup-design.md +++ /dev/null @@ -1,220 +0,0 @@ -# Indexing test suite cleanup — design - -Date: 2026-05-22 -Branch: `indexing-test-cleanup` -Target file: `tests/test_indexing.py` (2177 lines, 50 test functions) - -## Goals - -Three goals, in priority order as confirmed with the user: - -1. **Per-test isolation** — each test verifies one behavior. A failure should - point at a named case, not a 20-iteration loop inside a helper. -2. **Faster** — shrink the oversized arrays and cut redundant loop iterations. - Speed comes purely from smaller work per test (see "Speed model" below), - not from any matrix change. -3. **Clarity / consolidation** — remove duplicate coverage, replace loop + - helper-function patterns with `@pytest.mark.parametrize`, give every test a - docstring stating the behavior it verifies. - -Scope: **whole file**. Already-parametrized tests get swept for consistency -(docstrings, shared helpers, array sizes) alongside the loop-based ones. - -## Background / current state - -The file is a mix of two eras: - -- **Ported-from-v2 tests** (the slow ones): build a large array, set - `np.random.seed(42)`, then loop `for p in 0.5, 0.1, 0.01` generating random - boolean/integer index arrays at several sparsity levels, and call a private - helper (`_test_get_orthogonal_selection_3d`, etc.) that itself loops over - 7–20 hardcoded selection tuples. One test function exercises dozens of - selections with bundled assertions. - - Example arrays: `np.arange(32400).reshape(120, 30, 9)` (3d orthogonal), - `np.arange(5400).reshape(600, 9)` (2d), `np.arange(1050)` (1d bool). -- **Modern tests**: `@pytest.mark.parametrize` with explicit selection lists, - e.g. `test_indexing_equals_numpy`, `test_orthogonal_bool_indexing_like_numpy_ix`. - -### Key facts that constrain the design - -- `tests/test_indexing.py` defines its **own** `store` fixture (line 42–44): - `StorePath(await MemoryStore.open())`. It does **not** use the conftest - matrix-parametrized `store` fixture. Therefore every test runs **once** — - there is no store/format/codec amplification for this file. -- Default `addopts` (pyproject.toml:428) does not include `-n auto`; xdist is - available but opt-in. Tests run single-process by default. -- Local full-file runtime: ~3.5 s in pytest, ~4 s wall. Slowest tests are - exactly the loop-heavy orthogonal ones (`test_set_orthogonal_selection_3d` - 0.92 s, `test_get_orthogonal_selection_1d_bool` 0.79 s, - `test_set_orthogonal_selection_2d` 0.54 s, `test_set_orthogonal_selection_1d` - 0.41 s). - -### Speed model - -Runtime per test ≈ array construction + encode (`z[()] = a`) + per-selection -decode/encode. The large arrays dominate. Halving each dimension of -`(120, 30, 9)` and keeping a chunk grid of (≈3 chunks per axis with a partial -edge chunk) cuts the encoded volume by ~8× while preserving every structural -property the tests rely on (multi-chunk spans, partial edge chunks, -cross-chunk selections, negative/wraparound indices). - -## Approach (selected: A — surgical consolidation, applied file-wide) - -For each test family: - -1. **Shrink arrays** under a hard rule (confirmed with user): every indexed - axis spans **≥3 chunks**, and at least one axis has a **partial (non-full) - edge chunk**. Smallest shapes meeting that: - - 1d: `arange(30)`, chunks `(7,)` → 5 chunks, last (size 2) partial. - - 2d: `arange(60).reshape(12, 5)`, chunks `(5, 2)` → 3×3 chunks; axis-0 - edge (size 2) and axis-1 edge (size 1) partial. - - 3d: `arange(420).reshape(7, 6, 10)`, chunks `(3, 2, 4)` → 3×3×3 chunks; - partial edges on every axis. - Exact numbers may shift slightly during implementation so the hand-picked - selections stay in-bounds and meaningful, but the ≥3-chunks / partial-edge - rule is fixed. - -2. **Replace `np.random` selections with hand-picked deterministic ones** - (user decision). For each axis kind we keep explicit cases that name what - they cover: - - boolean mask: empty (all False), full (all True), alternating, single-True, - a sparse hand-chosen mask. - - integer array: sorted, unsorted, with-duplicates, with-wraparound - (negative), single-element. - The 0.5 / 0.1 / 0.01 sparsity loop collapses into the "alternating" and - "sparse" cases — the density sweep was fuzzing, not targeted coverage, and - the user de-prioritized RNG breadth. - -3. **Convert loop-over-selections helpers to `@pytest.mark.parametrize`.** - The private `_test_*` helpers that loop over selection lists become a - `selection` parameter on the public test. The shared - `_test_get_orthogonal_selection` / `_test_set_orthogonal_selection` - oracle (compare zarr result to numpy via `oindex`/`oindex_set`) is **kept** - as a small assertion helper — it is the right abstraction, just called once - per parametrized case instead of in a loop. - -4. **Docstrings** on every test stating the behavior verified (per project - convention / memory `feedback_test_docstrings`). - -5. **Preserve error-path tests as their own named tests** (per memory - `feedback_test_structure`: one test per failure mode). The `IndexError` - blocks currently bundled at the end of the get/set tests (too-short mask, - too-long mask, too-many-dims, out-of-bounds) become a `test_*_raises` - parametrized over `ExpectFail` cases, using `pytest.raises(case.exception, - match=case.msg)`. - -## Prerequisite: deduplicate the two `Expect` dataclass pairs - -The repo currently has **two divergent** test-case dataclass pairs. The new -indexing tests should use one canonical pair, so this PR unifies them first. - -| | `tests/conftest.py` (canonical) | `tests/test_codecs/conftest.py` (to delete) | -|---|---|---| -| success | `Expect[TIn, TOut]`: `input`, `output`, `id`; not frozen | `Expect[TIn, TOut]`: `input`, `expected`; frozen | -| failure | `ExpectFail[TIn]`: `input`, `exception`, `id`, `msg` | `ExpectErr[TIn]`: `input`, `msg`, `exception_cls` | - -**Decision:** keep the `tests/conftest.py` pair (`Expect` + `ExpectFail`) as -the single source of truth, because it carries `id` — the id lives with the -case, cannot drift out of sync with a separate `ids=[...]` list, and survives -reordering. Add `frozen=True` to both (the codecs version's one good idea; -these are value objects). Delete the definitions in -`tests/test_codecs/conftest.py` (that file contains only these two classes). - -**Migration of the three codecs/chunk-grid consumers** -(`tests/test_chunk_grids.py`, `tests/test_codecs/test_cast_value.py`, -`tests/test_codecs/test_scale_offset.py`): - -1. Change imports to `from tests.conftest import Expect, ExpectFail`. -2. Rename `expected=` → `output=` at every `Expect(...)` call site. -3. Replace `ExpectErr` → `ExpectFail`, renaming `exception_cls=` → `exception=`. -4. Add `id=` to every case, sourced from the existing `ids=[...]` list that the - parametrize call passes positionally, then replace `ids=[...]` with - `ids=lambda c: c.id`. Where a parametrize block has no `ids=` list today, - synthesize concise ids. -5. Run the affected suites green before touching indexing tests. - -This is a mechanical but real change (~40 call sites). It is a true -prerequisite: doing it first means the indexing rewrite imports the final, -stable `Expect`/`ExpectFail` and we never write against a soon-to-change shape. - -### Shared helpers introduced - -- **Use the conftest `Expect[TIn, TOut]` / `ExpectFail[TIn]` dataclasses** as - the selection-table mechanism, matching the established idiom in - `tests/test_metadata/test_v3.py`: - - Selection cases: `Expect[Selection, None]` where `input` is the selection - and `id` names the case (e.g. `"alternating-mask"`, `"wraparound-int"`). - `output` is unused for the oracle-style tests (the numpy result is computed, - not hardcoded) and set to `None`; the oracle helper does the comparison. - - Error cases: `ExpectFail[Selection]` carrying `input` (the bad selection), - `exception` (`IndexError`), `id`, and `msg` (regex for `pytest.raises`). - - Parametrize with `ids=lambda c: c.id` so `pytest -k ` selects a - single named case — this is the readable-id payoff that plain tuple - parametrize lacks. - - Module-level case lists (`_ORTHO_1D_CASES`, `_ORTHO_BAD_1D_CASES`, etc.) - are shared across get/set tests of the same dimensionality so 1d/2d/3d - don't each redefine "alternating mask". - - Field is `Expect.output` (the canonical `tests/conftest.py` name). The - `expected=` spelling came from the now-deleted codecs copy (see the dedup - section above). -- Keep `zarr_array_from_numpy_array`; it is already the right builder. -- Keep the `_test_get/set_orthogonal_selection` zarr-vs-numpy oracle helpers; - they consume one `Expect.input` per parametrized case instead of looping. - -### Things explicitly NOT changed - -- The local `store` fixture stays MemoryStore-only. Widening it to the conftest - matrix would multiply runtime against goal 2; out of scope. -- `xfail`/`skip` markers (structured-field tests, repeated-index test) stay as-is. -- Regression tests tied to specific GH issues (`test_set_item_1d_last_two_chunks`, - `test_indexing_with_zarr_array`, `test_vectorized_indexing_incompatible_shape`, - `test_zero_sized_chunks`) keep their concrete reproducing values — shrinking - them would weaken the regression. Only docstrings added. -- `CountingDict` / `test_accessed_chunks` logic (verifies which chunks are - touched) keeps shapes that produce a meaningful access pattern. - -## Rewrite inventory - -Loop/helper-based (primary rewrite targets): -`test_get_basic_selection_1d`, `test_get_basic_selection_2d`, -`test_get_orthogonal_selection_1d_bool`, `test_get_orthogonal_selection_1d_int`, -`test_get_orthogonal_selection_2d`, `test_get_orthogonal_selection_3d`, -`test_set_orthogonal_selection_1d/2d/3d`, -`test_get_coordinate_selection_1d/2d`, `test_set_coordinate_selection_1d/2d`, -`test_get_block_selection_1d/2d`, `test_set_block_selection_1d/2d`, -`test_get_mask_selection_1d/2d`, `test_set_mask_selection_1d/2d`, -`test_get_selection_out`. - -Already parametrized (sweep for consistency + array size only): -`test_get_basic_selection_0d`, `test_set_basic_selection_0d`, -`test_indexing_equals_numpy`, `test_orthogonal_bool_indexing_like_numpy_ix`, -the `*_fallback_*` family, `test_iter_grid`, `test_iter_regions`. - -Leave essentially as-is (docstring only): -the GH-regression tests and `xfail`/`skip` tests listed above, -`test_normalize_integer_selection`, `test_replace_ellipsis`, -`test_iter_grid_invalid`, `test_iter_chunk_regions`, `TestAsync`. - -## Testing / verification - -- After the `Expect` dedup (step 0): the migrated suites must stay green — - `uv run --frozen pytest tests/test_chunk_grids.py tests/test_codecs/test_cast_value.py tests/test_codecs/test_scale_offset.py tests/test_metadata/test_v3.py -q`. -- After each indexing family rewrite: - `uv run --frozen pytest tests/test_indexing.py -q` must stay green (same - pass/skip/xfail counts modulo intentional restructure). -- Coverage sanity: the rewritten cases must still exercise, for each selection - type, at least one cross-chunk selection, one partial-edge-chunk selection, - one negative/wraparound index, and the documented error paths. -- Final: `--durations=25` before/after to record the speedup, and a mypy pass - via `prek --all-files` (per memory) since type annotations on the parametrize - lists change and the `Expect` shape moves. - -## Success criteria - -- All loop-based selection tests replaced by parametrized, docstring'd tests - with one behavior per case. -- Error paths are individually named tests. -- Full-file wall time measurably lower (target: the four >0.4 s tests each - drop below ~0.15 s; expect total well under the current ~3.5 s). -- No loss of structural coverage per the checklist above. -- `git diff` is reviewable as a per-family sequence of commits. From da7ce66d48194f5ecb5f070f2f85002f151a00a9 Mon Sep 17 00:00:00 2001 From: Davis Vann Bennett Date: Fri, 22 May 2026 22:20:31 +0200 Subject: [PATCH 27/27] docs: rename changelog --- changes/{XXXX.misc.md => 4001.misc.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename changes/{XXXX.misc.md => 4001.misc.md} (100%) diff --git a/changes/XXXX.misc.md b/changes/4001.misc.md similarity index 100% rename from changes/XXXX.misc.md rename to changes/4001.misc.md