release: v10.12.0 — JP3D batched bridge extension to partial-res + ROI

README.md

@@ -8,13 +8,15 @@
 
 A pure Swift 6.2 implementation of JPEG 2000 (ISO/IEC 15444) encoding and decoding with strict concurrency support.
 
-**Current Version**: 10.11.0
-**Status**: Apple Silicon-first JPEG 2000 / HTJ2K (Part-15) implementation. v10.11.0 ships **JP3D batched GPU iDWT**: a new single-dispatch Metal kernel amortises per-slice GPU overhead across the whole volume. `JP3DSliceStackCodec.decode` now runs ONE batched iDWT dispatch across all slices of a tile instead of N per-slice dispatches. M2 release JP3D full decode wins **−5 ms on small CT to −115 ms on 16M-voxel CT** (1.07–1.17× faster, 5/6 fixtures cross the 3 ms acceptance threshold). Decoder-only; codestream bytes byte-identical to v10.10.0.
-**Previous Release**: 10.10.0 (JP3D true partial-resolution + ROI footprint-skip + Z-narrow)
+**Current Version**: 10.12.0
+**Status**: Apple Silicon-first JPEG 2000 / HTJ2K (Part-15) implementation. v10.12.0 extends v10.11.0's JP3D batched bridge to the **partial-resolution and ROI lanes** — `JP3DDecoder(cfg).decode` with `cfg.resolutionLevel > 0` and `JP3DROIDecoder().decode(data, region:)` now both run through ONE batched iDWT dispatch instead of N per-slice dispatches. M2 release JP3D wins **−10 ms to −83 ms** on production CT volumes across full / partial-res / ROI lanes (1.07–1.33× faster). Decoder-only; codestream bytes byte-identical to v10.11.0.
+**Previous Release**: 10.11.0 (JP3D batched GPU iDWT — full-volume)
 **Release process**: see [RELEASING.md](RELEASING.md). Every release MUST update this README (Current Version line + new Release Status paragraph) — see the Release artefacts checklist for the full requirements.
 
 ## 📦 Release Status
 
+**v10.12.0** extends v10.11.0's JP3D batched bridge SPI to the **partial-resolution and ROI lanes** — closes the explicit "Partial-resolution + ROI paths not yet batched" known limitation. A new public `JP3DBridgeOptions(partialResolutionLevel:, regionOfInterest:)` struct + `J2KDecoder._jp3dDecodeToCoefficients(_:options:)` overload thread the options into the bridge SPI; the batched orchestrator truncates the chain to `effectiveLevels = N − K` for partial-res (output dims = `levelSizes[K]`), runs the full chain for ROI and crops the per-slice output via `cropImage`, and composes both (`K>0 + ROI` inside the reduced grid). The eligibility gate now requires uniform options across the batch (which JP3D's per-volume options trivially satisfy). `JP3DSliceStackCodec` removes the per-slice fallback on both branches and now builds a `JP3DBridgeOptions` from `K` + `regionOfInterest`, decodes all slices to coefficients in parallel via the new overload, then runs ONE batched iDWT + finalize. **M2 release, J2KBenchMac --jp3d, in-process, 7 timed runs / 2 warmups, median**: `full` lane ct_3d_large **−82.89 ms** (1.13×), `res1` lane ct_3d_large **−25.82 ms** (1.12×), `ROIq` lane ct_3d_large **−47.53 ms** (1.33×); ct_3d_mid full **−38.85 / res1 −10.44 / ROIq −21.18**. Per acceptance discipline (≥3 ms wins): full 4/6, res1 2/6, ROIq 3/6 fixtures clear; smaller fixtures wash on the lighter lanes (reduced workload leaves less to amortise). K=0 thumbnail (no iDWT chain) falls back to per-slice serial. The only `K > 0 AND ROI` composition still throws (the 2D codec doesn't compose those — separate arc). Opt-out via `J2K_JP3D_BATCHED_BRIDGE=0` (introduced v10.11.0; still works for all three lanes). **`V10_21_BatchedBridgeOptionsParityTests` 7/7 PASS** (K=0/2/4/N, ROI 128² and 64², ROI+K composition), `V10_20_BatchedBridgeParityTests` 5/5 PASS, `V10_20_BatchedInverseInt32ParityTests` 12/12 PASS, `V10_20_JP3DBridgeParityTests` 5/5 PASS, full `swift test --filter JP3D` regression sweep **519/519 PASS**, mandatory commit gate 7/7 PASS. Codestream bytes byte-identical to v10.11.0; encoder unchanged. See [RELEASE_NOTES_v10.12.0.md](RELEASE_NOTES_v10.12.0.md) and the bench JSONs at [`Documentation/Benchmarks/data/jp3d-bench-arm64-v10_21-{batched,serial}-20260524.json`](Documentation/Benchmarks/data/).
+
 **v10.11.0** ships **JP3D batched GPU iDWT** — the production landing of a multi-week research arc that splits the JP3D per-tile decode pipeline at the dequant↔iDWT boundary and submits one batched Metal dispatch across the whole z-range instead of N per-slice dispatches. A new opaque-payload bridge SPI on `J2KDecoder` (`_jp3dDecodeToCoefficients` / `_jp3dIDWTAndFinalize` / `_jp3dIDWTAndFinalizeBatched`) is the architectural surface; two new Metal kernels (`j2k_dwt_inverse_53_horizontal_int_tiled_batched`, `..._vertical_..._batched`) extend the v10.3 tiled threadgroup-memory kernels with a Z-dim grid axis, processing N slices in one dispatch. `JP3DSliceStackCodec` now runs **parallel `_jp3dDecodeToCoefficients` across `[zStart, zUpper)` via a TaskGroup, then ONE batched iDWT call** before the existing sequential Z-delta residual chain. Per-slice GPU dispatch overhead amortises across the volume; the gain scales with slice count × per-slice work. **M2 release, J2KBenchMac --jp3d, in-process, 7 timed runs / 2 warmups**: ct_3d_small **−5.12 ms** (1.13×), us_3d_small −4.15 ms (1.07×), mr_3d_mid **−9.50 ms** (1.12×), ct_3d_mid **−53.25 ms** (1.17×), **ct_3d_large 16M-voxel CT −114.57 ms** (1.17×). 5/6 fixtures clear the 3 ms acceptance threshold; the wash (mr_3d_small @ 13 ms wall) is the smallest fixture where per-slice overhead dominates anyway. Kernel-level bench (16 slices × 256×256 × 3 levels): **5.93 → 2.06 ms = 2.4× faster GPU dispatch**. Eligibility gate: only `K=0 + no ROI` routes through the batched path; `K>0` keeps the per-slice `decodeResolution` loop and ROI keeps `decodeRegion` (bridging those is future work). Opt-out via `J2K_JP3D_BATCHED_BRIDGE=0`. **`V10_20_BatchedBridgeParityTests` 5/5 PASS, `V10_20_BatchedInverseInt32ParityTests` 12/12 PASS, `V10_20_JP3DBridgeParityTests` 5/5 PASS, full `swift test --filter JP3D` regression sweep 519/519 PASS, mandatory commit gate 7/7 PASS**. Codestream bytes byte-identical to v10.10.0; encoder unchanged. See [RELEASE_NOTES_v10.11.0.md](RELEASE_NOTES_v10.11.0.md) and the bench JSONs in [Documentation/Benchmarks/data/](Documentation/Benchmarks/data/).
 
 **v10.10.0** ships **JP3D true partial-resolution + ROI footprint-skip + Z-narrow ROI skip** — three coordinated decoder-side changes inside `Sources/J2K3D/` that close the long-standing follow-up on JP3D's selective-decode story. **`JP3DDecoderConfiguration.resolutionLevel`** has been wired into the struct since v5.x but the decoder ignored it (silently returned full resolution); v10.10.0 routes each per-slice 2D codestream inside `JP3DSliceStackCodec` through the existing v10.5.0 `J2KDecoder.decodeResolution(_:options:)`, producing a volume sized `⌈W / 2^K⌉ × ⌈H / 2^K⌉ × D` as documented. **`JP3DROIDecoder`** swaps decode-then-crop for true ROI footprint-skip — the per-tile in-tile XY sub-region is passed through to the slice-stack codec, which routes the per-slice 2D decode through v10.6.0 `decodeRegion(_:options:)` with `.direct` strategy (code-blocks whose inverse-DWT cone-of-influence misses the region skip entropy decode entirely). **Z-narrow ROI skip** pre-scans slice headers (no decode), finds the latest non-residual slice ≤ `zRange.lowerBound`, and starts decoding from there — completely skipping the unused Z-prefix while keeping Z-delta residual chain integrity via the restart anchor. M2 release: **`resolutionLevel = 1` is 2.2–3.1× faster than full decode** (mr_3d_small 2.22×, ct_3d_small 3.05×, mr_3d_mid 3.01×); **ROI 1/4 is 3.4–4.1× faster than full decode** (mr_3d_small 3.37×, ct_3d_small 4.14×, mr_3d_mid 3.96×). Combined `resolutionLevel + ROI` throws loud (was silently ignored previously). `V10_18_TrueSelectiveParityTests` 9/9 PASS, existing `JP3DDecoderTests` 61/61 PASS, mandatory commit gate clean. Encoder unchanged. Codestream bytes byte-identical to v10.9.3. See [RELEASE_NOTES_v10.10.0.md](RELEASE_NOTES_v10.10.0.md) and [V10_18_JP3D_TRUE_SELECTIVE_DECODE.md](Documentation/research/V10_18_JP3D_TRUE_SELECTIVE_DECODE.md).

RELEASE_NOTES_v10.12.0.md

@@ -0,0 +1,196 @@
+# J2KSwift v10.12.0
+
+**JP3D batched bridge extension — partial-resolution + ROI.** Closes
+the v10.11.0 known limitation: the batched bridge SPI now handles
+the `JP3DDecoderConfiguration.resolutionLevel > 0` and
+`JP3DROIDecoder(_:region:)` cases inside `JP3DSliceStackCodec` via a
+new `JP3DBridgeOptions` overload. Same single-dispatch architecture
+as v10.11.0; the orchestrator truncates the chain for partial-res
+and crops after iDWT for ROI. M2 release JP3D decode wins
+**−10 ms to −83 ms** on production-relevant CT volumes across all
+three lanes (full / partial-res-1 / ROI 1/4).
+
+Decoder-only release; codestream bytes are byte-identical to v10.11.0.
+Encoder unchanged. MINOR per RELEASING.md — additive public surface
+(`JP3DBridgeOptions` + new `_jp3dDecodeToCoefficients(_:options:)`
+overload), no signature removal, no default flip that affects bytes.
+
+## Summary
+
+Three coordinated changes that extend v10.11.0's batched bridge SPI
+to the two cases it previously dropped to per-slice serial:
+
+1. **`JP3DBridgeOptions` struct + new bridge SPI overload.**
+   `JP3DBridgeOptions(partialResolutionLevel:, regionOfInterest:)`
+   carries the partial-resolution K and/or ROI rectangle. The new
+   `J2KDecoder._jp3dDecodeToCoefficients(_:options:)` overload
+   forwards them to the underlying `DecoderPipeline`, where the
+   entropy-stage Stage B.1 (partial-res) / Stage 1 (ROI footprint-
+   skip) wins fire just like in the standalone
+   `decodeResolution` / `decodeRegion`. The options ride along
+   inside the returned bundle so the matching finalize call sizes /
+   crops the output without re-plumbing.
+
+2. **Batched orchestrator handles uniform-options batches.** The
+   eligibility gate now accepts partial-res K and ROI when all
+   slices in the batch carry the same options (JP3D slice-stack
+   always does — these are per-volume not per-slice). For partial-
+   res the orchestrator truncates the chain to `effectiveLevels =
+   N − K` levels (output dims = `levelSizes[K]`); for ROI the chain
+   runs full and `cropImage` slices the per-slice output to the
+   region rectangle. K=0 (thumbnail) falls back to per-slice serial
+   because the orchestrator can't dispatch an empty iDWT chain.
+
+3. **`JP3DSliceStackCodec` consumes both new cases.** The K>0
+   (partial-resolution) and ROI branches now build a
+   `JP3DBridgeOptions` and route through the bulk parallel
+   `_jp3dDecodeToCoefficients` + ONE batched iDWT, instead of
+   per-slice `decoder.decodeResolution` / `decodeRegion`. The
+   only combination still routed per-slice is `K > 0 AND ROI` —
+   the 2D codec doesn't compose partial-res with ROI; the
+   pre-existing throw still fires for that case.
+
+## What's New — production-default
+
+| Public API | v10.11.0 behaviour | v10.12.0 behaviour |
+|---|---|---|
+| `JP3DDecoder(configuration: cfg).decode(data)` with `cfg.resolutionLevel > 0` | Per-slice serial `decodeResolution` loop | Parallel `_jp3dDecodeToCoefficients(options:)` + ONE batched iDWT (truncated chain). **Same output bytes**; 10–26 ms faster wall on the JP3D corpus' larger CT volumes |
+| `JP3DROIDecoder().decode(data, region:)` | Per-slice serial `decodeRegion` loop | Parallel `_jp3dDecodeToCoefficients(options:)` + ONE batched iDWT + per-slice ROI crop. **Same output bytes**; 4–48 ms faster wall on the JP3D corpus |
+| `JP3DDecoder().decode(data)` full-volume decode | v10.11.0 batched bridge (unchanged) | Unchanged |
+
+The new bridge SPI surface (`JP3DBridgeOptions` + the options
+overload) is public but `JP3DBridgeOptions` ships in J2KCodec
+alongside the existing underscored `_jp3d*` methods. Consumers
+outside `J2K3D` should keep calling the normal
+`J2KDecoder.decode(_:)` / `decodeResolution(_:options:)` /
+`decodeRegion(_:options:)`.
+
+## What's New — opt-in / opt-out
+
+`J2K_JP3D_BATCHED_BRIDGE=0` env var (introduced v10.11.0) continues
+to disable the batched path on ALL three lanes — full, partial-res,
+ROI — forcing per-slice serial. Diagnostic-A/B only; production
+should leave it unset.
+
+## Backward compatibility
+
+- **Codestream bytes**: byte-identical to v10.11.0 on every input.
+  Encoder unchanged.
+- **`JP3DDecoder(cfg).decode(data)` with `cfg.resolutionLevel > 0`**
+  is byte-identical to v10.11.0 — validated by
+  `V10_21_BatchedBridgeOptionsParityTests` (per-slice serial vs
+  batched-with-options bit-exact on 4-slice K=2 / K=4 / K=N=full /
+  K=0 batches).
+- **`JP3DROIDecoder().decode(data, region:)`** is byte-identical to
+  v10.11.0 — validated by `V10_21_BatchedBridgeOptionsParityTests`
+  ROI-only and ROI+K composition tests.
+- **Behaviour change**: none.
+
+## Measured wins — JP3D corpus
+
+M2 release, J2KBenchMac --jp3d, in-process, 7 timed runs / 2 warmups,
+median per fixture per lane (`full` = full-volume decode, `res1` =
+JP3D `resolutionLevel = 1`, `ROIq` = centre quarter ROI):
+
+| Fixture (modality WxHxD) | voxels | full Δ ms (ratio) | res1 Δ ms (ratio) | ROIq Δ ms (ratio) |
+|---|---:|---:|---:|---:|
+| mr_3d_small  MR 128×128×16 | 262K   | −0.05 (1.00×) | +1.86 (0.75×) | −0.02 (1.00×) |
+| ct_3d_small  CT 256×256×16 | 1.05M  | **−4.19** (1.11×) | −0.32 (1.02×) | −2.19 (1.24×) |
+| us_3d_small  US 320×240×24 | 1.84M  | **−4.69** (1.08×) | +1.25 (0.95×) | −2.73 (1.24×) |
+| mr_3d_mid    MR 256×256×32 | 2.10M  | **−9.35** (1.12×) | −1.63 (1.06×) | **−4.22** (1.23×) |
+| ct_3d_mid    CT 512×512×32 | 8.39M  | **−38.85** (1.12×) | **−10.44** (1.09×) | **−21.18** (1.30×) |
+| ct_3d_large  CT 512×512×64 | 16.78M | **−82.89** (1.13×) | **−25.82** (1.12×) | **−47.53** (1.33×) |
+
+Per acceptance discipline (≥3 ms wins): **full lane 4/6, res1 lane
+2/6, ROIq lane 3/6** clear. Smaller fixtures wash on res1/ROIq
+because the reduced workload (fewer iDWT levels or smaller region)
+leaves less for the batched dispatch to amortise. The 8M-voxel
+ct_3d_mid and 16M-voxel ct_3d_large CT volumes (radiologist
+scrolling case) win on every lane.
+
+Raw bench JSONs:
+- `Documentation/Benchmarks/data/jp3d-bench-arm64-v10_21-batched-20260524.json`
+- `Documentation/Benchmarks/data/jp3d-bench-arm64-v10_21-serial-20260524.json`
+
+## Cross-codec parity (2D codec unchanged)
+
+The 2D codec path (entropy, dequant, iDWT, colour, DC, reconstruct)
+is touched only at the bridge SPI splitting point — the same per-
+stage code paths execute, same `SubbandInfo` flows, same partial-res
+truncation and ROI crop the standalone `decodeResolution` /
+`decodeRegion` apply. Cross-codec parity vs OpenJPH 0.27.0 /
+Grok 20.3.0 / Kakadu 8.4.1 is inherited from v10.11.0 (no change
+measured or asserted in this release).
+
+## Test Suite Results
+
+| Suite | Tests | Result | Coverage |
+|---|---:|---|---|
+| `V10_21_BatchedBridgeOptionsParityTests` | 7/7 | PASS | New options-bridge parity (K=0/2/4/N, ROI, K+ROI) |
+| `V10_20_JP3DBridgeParityTests` | 5/5 | PASS | Phase 1 bridge SPI bit-exact composition |
+| `V10_20_BatchedInverseInt32ParityTests` | 12/12 | PASS | Batched Metal kernel + multi-level orchestrator bit-exact vs serial GPU |
+| `V10_20_BatchedBridgeParityTests` | 5/5 | PASS | Batched bridge SPI bit-exact vs per-slice serial (full-volume) |
+| `swift test --filter JP3D` (regression sweep) | 519/519 | PASS | Full JP3D test suite green with the K>0 + ROI batched wiring |
+| Mandatory commit gate (release mode) | 7/7 | PASS | Encode-perf + decode-perf + cross-codec strict validation |
+
+## API surface — additions only
+
+```swift
+/// v10.12.0 — JP3D bridge SPI decode options.
+public struct JP3DBridgeOptions: Sendable, Equatable {
+    public let partialResolutionLevel: Int?
+    public let regionOfInterest: J2KRegion?
+    public init(partialResolutionLevel: Int? = nil,
+                regionOfInterest: J2KRegion? = nil)
+    public static let `default`: JP3DBridgeOptions
+}
+
+extension J2KDecoder {
+    /// v10.12.0 — options overload.
+    public func _jp3dDecodeToCoefficients(
+        _ data: Data,
+        options: JP3DBridgeOptions
+    ) async throws -> JP3DSliceCoefficients
+}
+```
+
+No removals. No existing signatures changed.
+
+## Known limitations
+
+- **`K > 0 AND ROI` composition** still throws. The 2D codec doesn't
+  support "footprint-skip at a downsampled resolution" — that's a
+  v10.5.0 + v10.6.0 follow-up that wasn't included in this arc.
+- **K=0 thumbnail batched dispatch.** The orchestrator can't
+  dispatch an empty iDWT chain, so K=0 (which produces the deepest
+  LL only) falls back to per-slice serial. Per-slice K=0 is already
+  cheap (one LL Int32→pixel copy per slice), so the impact is minor.
+
+## Reproducing the headline numbers
+
+```bash
+# Build the JP3D bench (research tool; not in main's Package.swift
+# but checked out on v10.21-research):
+git fetch origin v10.21-research
+git checkout v10.21-research -- Sources/J2KBenchMac/ Package.swift
+swift build -c release --product J2KBenchMac
+
+# Batched (default)
+.build/release/J2KBenchMac --jp3d --output /tmp/jp3d_batched.json
+
+# Serial baseline (opt-out)
+J2K_JP3D_BATCHED_BRIDGE=0 .build/release/J2KBenchMac --jp3d --output /tmp/jp3d_serial.json
+```
+
+## Companion documents
+
+- `Documentation/Benchmarks/data/jp3d-bench-arm64-v10_21-batched-20260524.json` — raw bench (batched)
+- `Documentation/Benchmarks/data/jp3d-bench-arm64-v10_21-serial-20260524.json` — raw bench (serial baseline)
+
+## Backward upgrade
+
+`swift package update` will not auto-pick this release if your
+`Package.swift` pins an exact version; bump the requirement to
+`from: "10.12.0"` (or accept the next `.upToNextMinor` per your
+policy). Consumers of `JP3DDecoder` / `JP3DROIDecoder` see only a
+perf improvement; no source changes required.