Skip to content

Add free-threaded tests with pytest-run-parallel #70

Description

@kylebarron

Follow up from #69


Claude:

Follow-up: free-threaded (3.14t) CI testing

PR #69 adds the wheel builds for free-threaded CPython 3.14 (cp314-cp314t)
and scaffolds the abi3t (3.15+) wheel. It intentionally does not add CI that
tests zarrista on a free-threaded interpreter — this issue tracks that work,
which turned out to need a test-suite restructure rather than a CI one-liner.

Why it was deferred

Spiking the 3.14t test job surfaced three blockers:

  1. numcodecs re-enables the GIL. numcodecs.blosc hasn't declared that it
    can run without the GIL, so importing it on 3.14t silently turns the GIL back
    on:

    RuntimeWarning: The global interpreter lock (GIL) has been enabled to load module 'numcodecs.blosc' ...

    Since nearly all tests pull in zarr-python (→ numcodecs), running the current
    suite on 3.14t gives a false free-threaded signal — the GIL is on.

  2. Missing free-threaded wheels → slow / degraded. icechunk and
    numcodecs compile from source on 3.14t (no cp314t wheels yet), and
    google-crc32c (via icechunk) falls back to its pure-Python implementation:

    As the c extension couldn't be imported, google-crc32c is using a pure python implementation that is significantly slower.

  3. Tests use zarr-python as the fixture writer. 8 of 10 test files call
    zarr.create_array / zarr.open_group to produce fixtures, then read them
    back with zarrista. These are really interop/integration tests. We have
    almost no coverage that exercises zarrista's own read+write round-trip
    (ArrayBuilder.create / store_chunk / store_metadataArray read).

Proposed plan

  • Split the suite into own tests (zarrista-only read/write) vs
    integration tests (zarr-python / icechunk / arro3 interop), via markers
    or a tests/integration/ directory.
  • Add zarrista-only round-trip tests (create + store chunks + read back,
    compared against numpy) so there's real own-code coverage.
  • Split dependency groups so the free-threaded job installs only GIL-safe
    deps (avoid zarr/numcodecs/icechunk).
  • Add a 3.14t CI job that runs only the own tests on the free-threaded
    interpreter.
  • Revisit pytest-run-parallel for genuine thread-safety stress — note it
    runs the same test in N threads sharing one fixture, so any test that
    mutates shared store state flakes; concurrency tests must use per-thread
    isolated stores.
  • Re-enable integration tests on 3.14t once numcodecs/icechunk ship
    free-threaded wheels and declare GIL-safety.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions