From 9203cca6a15c381e82335712a7267b96bf1f6416 Mon Sep 17 00:00:00 2001 From: raster Date: Sat, 23 May 2026 23:33:14 +0530 Subject: [PATCH] =?UTF-8?q?release:=20v10.10.0=20=E2=80=94=20JP3D=20true?= =?UTF-8?q?=20partial-resolution=20+=20ROI=20footprint-skip=20+=20Z-narrow?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes the long-standing JP3D follow-up: the resolution-level field on JP3DDecoderConfiguration was wired but never consulted; the ROI decoder was decode-then-crop. Both are now true-selective. Decoder-only release; codestream bytes byte-identical to v10.9.3. Encoder unchanged. MINOR bump for new functional behaviour on existing public APIs (JP3DDecoderConfiguration.resolutionLevel and JP3DROIDecoder.decode(_:region:)). Three coordinated changes inside Sources/J2K3D/: 1. JP3DDecoderConfiguration.resolutionLevel becomes functional. The field has shipped since v5.x but the decoder ignored it. v10.10.0 routes each per-slice 2D codestream inside JP3DSliceStackCodec through the 2D codec's existing decodeResolution(_:options:) (v10.5.0); returned volume sized ⌈W / 2^K⌉ × ⌈H / 2^K⌉ × D as documented. M2 release: 2.2-3.1x faster decode at resolutionLevel = 1. 2. JP3DROIDecoder swaps decode-then-crop for true ROI footprint-skip. Per-tile in-tile sub-region passed into JP3DSliceStackCodec, which routes the per-slice 2D decode through decodeRegion(_:options:) .direct strategy (v10.6.0 footprint-skip). Returned buffers sized exactly for the region, no second intersection-crop pass needed. 3. Z-narrow ROI skip. JP3DSliceStackCodec.decode pre-scans slice headers (no decode), finds the latest non-residual slice ≤ zRange.lowerBound, decodes from there. Slices in [z_start, zLower) keep the Z-delta residual chain intact; out-of-Z-range slices that precede the request are entirely skipped. M2 release ROI 1/4 (XY footprint-skip + Z-narrow combined): mr_3d_small 18.78 → 5.57 ms (3.37x faster than full) ct_3d_small 64.45 → 15.55 ms (4.14x) mr_3d_mid 130.13 → 32.84 ms (3.96x) Combined resolutionLevel + ROI throws loud (the 2D codec doesn't yet implement footprint-skip at a downsampled resolution — separate arc). Previously the combination silently ignored resolutionLevel. Files (Sources/J2K3D + Tests/JP3DTests): - JP3DDecoder.swift Phase 2 resolutionLevel - JP3DSliceStackCodec.swift Phase 2 + Phase 3 + Phase 6 (zRange) + 20-line inline COD-marker peek (peekDecompositionLevels) for per-slice N - JP3DROIDecoder.swift Phase 3 (true ROI) + Phase 5 guard + Phase 6 (zRange plumbing) - V10_18_TrueSelectiveParityTests.swift (NEW) 9 tests: testFullDecodeLosslessRoundTripBaseline (anchor) testROIDecodeMatchesFullDecodeCropped (Phase 3 guard) testROIDecodeMatchesFullDecodeCroppedAtCorner testResolutionLevel0IsBitExactFullDecode (Phase 2 backward-compat) testPartialResolutionShape (Phase 2 dim contract) testPartialResolutionDeterministic testPartialResolutionLevel2Shape (Phase 2 K=2) testZNarrowROIBitExactWithFullCropped (Phase 6 guard) testCombinedResolutionLevelAndROIIsPhase5Future (Phase 5 tripwire) Tests (release mode): - V10_18_TrueSelectiveParityTests: 9/9 PASS - JP3DDecoderTests: 61/61 PASS - Mandatory commit gate (release mode): * J2KMedicalCorpusEncodePerformanceTests: 2/2 PASS * J2KMedicalCorpusPerformanceTests: 2/2 PASS * J2KStrictCrossCodecValidationTests: 3/3 PASS Total: 7 tests, 0 failures, 0 unexpected (32.26 s) README updated with v10.10.0 Current Version + Release Status paragraph + v10.9.3 catch-up paragraph (was missing in the v10.9.3 release). Companion doc: Documentation/research/V10_18_JP3D_TRUE_SELECTIVE_DECODE.md. The bench-app scaffolding from v10.18-research (J2KBenchApp Volumes tab, 3-plane MPR viewer, 6 anatomical phantoms, J2KBenchMac --jp3d) stays on the research branch per feedback_research_no_main_merge.md — only the codec changes ship in this release. Co-Authored-By: Claude Opus 4.7 (1M context) --- .../V10_18_JP3D_TRUE_SELECTIVE_DECODE.md | 193 +++++++++ README.md | 10 +- RELEASE_NOTES_v10.10.0.md | 197 +++++++++ Sources/J2K3D/JP3DDecoder.swift | 63 ++- Sources/J2K3D/JP3DROIDecoder.swift | 191 +++++++-- Sources/J2K3D/JP3DSliceStackCodec.swift | 284 +++++++++++-- .../V10_18_TrueSelectiveParityTests.swift | 384 ++++++++++++++++++ 7 files changed, 1251 insertions(+), 71 deletions(-) create mode 100644 Documentation/research/V10_18_JP3D_TRUE_SELECTIVE_DECODE.md create mode 100644 RELEASE_NOTES_v10.10.0.md create mode 100644 Tests/JP3DTests/V10_18_TrueSelectiveParityTests.swift diff --git a/Documentation/research/V10_18_JP3D_TRUE_SELECTIVE_DECODE.md b/Documentation/research/V10_18_JP3D_TRUE_SELECTIVE_DECODE.md new file mode 100644 index 00000000..14eba26f --- /dev/null +++ b/Documentation/research/V10_18_JP3D_TRUE_SELECTIVE_DECODE.md @@ -0,0 +1,193 @@ +# v10.18-research — JP3D true partial-resolution / ROI + +**Branch:** `v10.18-research` · **Status:** Phases 0-5 landed on +branch · **Started:** 2026-05-23 + +Port the v10.4–v10.7 2D true-selective decode arc (per-resolution +code-block filter; ROI footprint-skip; tile-skip) to the JP3D slice- +stack decoder. Closes the **`project_jp3d_beat_openjpeg.md` open +follow-up** ("ROI decoder decodes-then-crops, no true per-resolution +selective decode"). Concrete, scoped, with a clear test surface (3D +fixtures decode-then-crop vs true-selective parity, plus per-fixture +wall-clock A/B). + +Research stays on `v10.18-research` per `feedback_research_no_main_merge.md`. + +## Why now + +Three things lined up: + +1. **The 2D arc shipped** — v10.5.0 (partial-res), v10.6.0 (ROI + `.direct`), v10.7.0 (tile-skip) all landed on `main` and proved + 3-8× thumbnail / 1.4-1.7× ROI speedup on the 2D codec. +2. **JP3D's architecture maps directly** — JP3DSliceStackCodec.swift + already runs each volume slice as an independent 2D JPEG 2000 + codestream, so the JP3D win = the 2D win applied per-slice. +3. **JP3DDecoderConfiguration.resolutionLevel was a half-finished + field** — wired into the struct since v5.x but ignored by the + decoder. Documentation said one thing, behaviour did another. Bad + smell for downstream consumers; the v10.18 port closes it. + +## Deliverable shape + +A multi-week JP3D arc spanning four code areas: + +| Phase | Scope | Status | +|---|---|---| +| **Phase 0** | Bench scaffolding: J2KBenchApp Volumes tab + 3-plane MPR viewer + J2KBenchMac `--jp3d` mode + JP3D corpus + LCG synthesis | **Done** | +| **Phase 1** | Parity oracle (`V10_18_TrueSelectiveParityTests`) — establishes the specification BEFORE any codec change | **Done** | +| **Phase 2** | Port v10.5.0 `maxResolutionLevel` into JP3DSliceStackCodec per-slice 2D decode | **Done — 2.4-3.2× speedup** | +| **Phase 3** | Port v10.6.0 ROI footprint-skip into JP3DROIDecoder (replaces decode-then-crop) | **Done — 1.5-2.2× speedup** | +| **Phase 4** | v10.7.0 tile-skip — already covered by `JP3DTilingConfiguration.tilesIntersecting` since pre-v10.18 | **Done (documentation-only)** | +| **Phase 5** | Wire `resolutionLevel` + ROI through `JP3DDecoderConfiguration`; throw loud on the combined case (Phase 5 future for the 2D codec) | **Done** | +| **Phase 6** | Z-narrow ROI skip — `JP3DSliceStackCodec.decode` gains `zRange:` param; pre-scans slice headers, decodes from latest non-residual ≤ `zLower`, skips out-of-Z-range slices entirely | **Done — additional 17-31% on top of Phase 3 ROI** | + +## Measured wins (M2 release, J2KBenchMac --jp3d --quick) + +| Fixture | full ms | res1 ms (P2) | Phase 2 vs full | ROI 1/4 ms (P6) | Phase 6 vs full | +|---|---:|---:|---:|---:|---:| +| mr_3d_small (128×128×16, 262K vox) | 18.78 | 8.45 | **2.22×** | **5.57** | **3.37×** | +| ct_3d_small (256×256×16, 1 M vox) | 64.45 | 21.10 | **3.05×** | **15.55** | **4.14×** | +| mr_3d_mid (256×256×32, 2 M vox) | 130.13 | 43.25 | **3.01×** | **32.84** | **3.96×** | + +The Phase 2 ratios (~2.2–3.1×) match the 2D v10.5.0 arc's 3-8× +thumbnail finding — JP3D's slice-stack arch means the 3D win is +exactly the 2D win applied per-slice, including the Z-delta +residual path which decodes residual + base at the same downsampled +resolution. + +Phase 3 alone (XY footprint-skip) brought centre-1/4 ROI to ~1.7-2.2× +faster than full. **Phase 6 (Z-narrow skip) brings the combined +total to 3.4-4.1× faster** — the per-tile Z range is now passed +through to the codec, which pre-scans slice headers and starts +decoding from the latest non-residual ≤ `zLower`, completely +skipping the unused Z-prefix slices. Z-delta residual correctness +is preserved by the non-residual restart anchor. + +The combined case (`resolutionLevel > 0 + sub-region ROI`) throws +LOUDLY rather than silently doing decode-at-full-res. The 2D codec +itself doesn't yet implement "footprint-skip at a downsampled +resolution" — that's a separate arc; the JP3D port surfaces the +limitation rather than hiding it. + +## Architecture port + +JP3D's slice-stack design (each volume tile encoded as N independent +2D codestreams + Z-delta residual chaining) made the port surgical: + +``` +JP3DDecoder.decode + └ for each tile: + JP3DSliceStackCodec.decode( + payload, expectedTile, + resolutionLevel: K, ← v10.18 Phase 2 + regionOfInterest: (xRange, yRange), ← v10.18 Phase 3 + zRange: inTileZRange) ← v10.18 Phase 6 + └ pre-scan slice headers (O(N), no decode) + └ z_start = latest non-residual ≤ zRange.lowerBound + └ for each slice in [z_start, zRange.upperBound): + K == 0 + ROI nil → J2KDecoder.decode (current) + K == 0 + ROI set → J2KDecoder.decodeRegion(.direct) (P3) + K > 0 + ROI nil → J2KDecoder.decodeResolution (P2) + K > 0 + ROI set → THROW: Phase 5 wiring future + place into output buffer only if z ∈ [zLower, zUpper) ← P6 +``` + +Each of the four cells composes the existing 2D API surface — no +duplication of footprint-skip / partial-res implementation, no new +codec logic invented for the 3D path. + +### Phase 2 (resolutionLevel) + +Volume-level scaling per JPEG 2000 spec rule: +- `outW = ⌈siz.width / 2^K⌉`, `outH = ⌈siz.height / 2^K⌉`, `outD = depth` +- Tile origins / dims scale by `⌈ref / 2^K⌉` (no gaps or overlaps; + testROIDecodeMatchesFullDecodeCroppedAtCorner + testPartialResolution­ + Level2Shape exercise the corner / interior cases) +- Z is per-slice (slice-stack), so `resolutionLevel` never downsamples Z + +Per-slice 2D level mapping: 2D codec uses `level=N` for full, +`level=0` for thumbnail; JP3D's K is halvings-from-full. So +slice 2D `level = max(0, N - K)`. + +`N` is read once per JP3D tile via an inline 20-line COD-marker peek +(`peekDecompositionLevels`, `static` on `JP3DSliceStackCodec`) — no +new module dependency on J2KCodec internals or J2KFileFormat. + +### Phase 3 (ROI footprint-skip) + +`regionOfInterest: (xRange: Range, yRange: Range)?` +parameter on `JP3DSliceStackCodec.decode`: +- `nil` → whole-tile path (unchanged) +- non-nil → per-slice 2D decode via `J2KDecoder.decodeRegion(_:options:)` + with `.direct` strategy and the in-tile XY sub-region; returned + Float buffers sized exactly for the region (`regionW × regionH × + sliceCount`) + +JP3DROIDecoder.decode reshape: +- Per-tile in-tile XY range computed from intersection of clamped + ROI and tile spatial extent (tile-local coords) +- `useRegionPath = strict inset on either axis` triggers the new + fast path; otherwise (tile entirely inside ROI) the legacy + whole-tile + intersection-crop runs as before +- Z range filtering at the per-component placement loop, unchanged + +Z-delta residual correctness: residual + prior slice both decoded +at the SAME region parameters, per-voxel add operates on matched +dims. + +## Test surface + +`Tests/JP3DTests/V10_18_TrueSelectiveParityTests.swift` — 8 tests: + +| Test | Phase | Purpose | +|---|---|---| +| testFullDecodeLosslessRoundTripBaseline | baseline | Sanity anchor for the LCG-synthesised volume + JP3D round-trip | +| testROIDecodeMatchesFullDecodeCropped | 3 guard | Specification: ROI output ≡ full-decode-then-crop, bit-exact | +| testROIDecodeMatchesFullDecodeCroppedAtCorner | 3 guard | Edge-region (0,0,0) variant | +| testResolutionLevel0IsBitExactFullDecode | 2 guard | Backward-compat: K=0 must match default decode | +| testPartialResolutionShape | 2 | Dim contract: `⌈W/2⌉ × ⌈H/2⌉ × D` at K=1 | +| testPartialResolutionDeterministic | 2 | Deterministic per-fixture output | +| testPartialResolutionLevel2Shape | 2 | Higher-K validity at K=2 (off-by-one guard) | +| testCombinedResolutionLevelAndROIIsPhase5Future | 5 tripwire | Throws today; unskip when 2D codec gains the combined path | + +Result: **8/8 pass** + **61/61 existing JP3DDecoderTests pass** = no +regression in the existing JP3D surface. + +## App surface + +`Sources/J2KBenchApp/`: + +- **`Volumes` tab** (new) on `J2KBenchApp` — root list of saved JP3D + bench runs, share-as-JSON, per-fixture drill-down. +- **JP3DBenchRunner** exercises all three decode lanes per fixture: + full / res-1 / ROI 1/4 with warm 2+7 timing methodology. +- **3-plane MPR viewer** — pushes from a fixture detail row; + axial + sagittal + coronal slices with tap-to-sync crosshair, a + decode-mode picker (Full / Res-1 / ROI 1/4) that re-decodes and + shows the wall-time for each mode, and per-plane sliders for + navigating Z (axial) / X (sagittal) / Y (coronal). + +`Sources/J2KBenchMac/`: + +- **`--jp3d` flag** runs the macOS JP3D bench (same corpus, same + three lanes, canonical JSON schema `warm-inproc-jp3d-v1`). + +Apples-to-apples cross-silicon: when an iPhone-side JP3D bench JSON +and a J2KBenchMac --jp3d JSON have the same schema, the same +cross_silicon_compare.py harness reads them. + +## Open follow-ups + +- **Combined `resolutionLevel + ROI`** — needs the 2D codec to support + "footprint-skip at a downsampled resolution" first; not a JP3D + problem. +- ~~**Z-narrow ROI skip**~~ — **CLOSED** by Phase 6 above. ROI 1/4 + is now 3.4-4.1× faster than full decode (vs Phase 3's 1.7-2.2×). +- **GPU iDWT for JP3D** — JP3DMetalDWT.swift exists but isn't wired + into JP3DDecoder. Orthogonal to v10.18; would be its own arc. +- **Real DICOM fixtures in the iOS app** — `project.yml` bundles 2D + PGM medical images today; the Volumes tab uses synthetic LCG + volumes only. Real DICOM-derived JP3D fixtures (via + `Scripts/prep_jp3d_volume.py`) would let testers run the bench on + data closer to their workload. diff --git a/README.md b/README.md index 73725514..04288c64 100644 --- a/README.md +++ b/README.md @@ -8,13 +8,17 @@ A pure Swift 6.2 implementation of JPEG 2000 (ISO/IEC 15444) encoding and decoding with strict concurrency support. -**Current Version**: 10.9.2 -**Status**: Apple Silicon-first JPEG 2000 / HTJ2K (Part-15) implementation. v10.9.2 is a **stability + packaging patch** — it fixes two latent `SIGSEGV` crashes (HTJ2K encoder; performance-validation report generator) and makes J2KSwift resolvable as a SwiftPM URL dependency. Codestream bytes byte-identical to v10.9.1. -**Previous Release**: 10.9.1 (decoder correctness hotfix — GPU multi-tile inverse 5/3 DWT) +**Current Version**: 10.10.0 +**Status**: Apple Silicon-first JPEG 2000 / HTJ2K (Part-15) implementation. v10.10.0 ships **JP3D true partial-resolution + ROI footprint-skip**: `JP3DDecoderConfiguration.resolutionLevel` becomes functional (was wired but ignored — 2.2–3.1× faster decode at `resolutionLevel = 1` on M2 release) and `JP3DROIDecoder` replaces decode-then-crop with true per-tile XY footprint-skip + Z-narrow non-residual scan (ROI 1/4 is 3.4–4.1× faster than full decode on M2 release). Decoder-only; codestream bytes byte-identical to v10.9.3. +**Previous Release**: 10.9.3 (always-URL CompressionFamily dependency, fixes #438) **Release process**: see [RELEASING.md](RELEASING.md). Every release MUST update this README (Current Version line + new Release Status paragraph) — see the Release artefacts checklist for the full requirements. ## 📦 Release Status +**v10.10.0** ships **JP3D true partial-resolution + ROI footprint-skip + Z-narrow ROI skip** — three coordinated decoder-side changes inside `Sources/J2K3D/` that close the long-standing follow-up on JP3D's selective-decode story. **`JP3DDecoderConfiguration.resolutionLevel`** has been wired into the struct since v5.x but the decoder ignored it (silently returned full resolution); v10.10.0 routes each per-slice 2D codestream inside `JP3DSliceStackCodec` through the existing v10.5.0 `J2KDecoder.decodeResolution(_:options:)`, producing a volume sized `⌈W / 2^K⌉ × ⌈H / 2^K⌉ × D` as documented. **`JP3DROIDecoder`** swaps decode-then-crop for true ROI footprint-skip — the per-tile in-tile XY sub-region is passed through to the slice-stack codec, which routes the per-slice 2D decode through v10.6.0 `decodeRegion(_:options:)` with `.direct` strategy (code-blocks whose inverse-DWT cone-of-influence misses the region skip entropy decode entirely). **Z-narrow ROI skip** pre-scans slice headers (no decode), finds the latest non-residual slice ≤ `zRange.lowerBound`, and starts decoding from there — completely skipping the unused Z-prefix while keeping Z-delta residual chain integrity via the restart anchor. M2 release: **`resolutionLevel = 1` is 2.2–3.1× faster than full decode** (mr_3d_small 2.22×, ct_3d_small 3.05×, mr_3d_mid 3.01×); **ROI 1/4 is 3.4–4.1× faster than full decode** (mr_3d_small 3.37×, ct_3d_small 4.14×, mr_3d_mid 3.96×). Combined `resolutionLevel + ROI` throws loud (was silently ignored previously). `V10_18_TrueSelectiveParityTests` 9/9 PASS, existing `JP3DDecoderTests` 61/61 PASS, mandatory commit gate clean. Encoder unchanged. Codestream bytes byte-identical to v10.9.3. See [RELEASE_NOTES_v10.10.0.md](RELEASE_NOTES_v10.10.0.md) and [V10_18_JP3D_TRUE_SELECTIVE_DECODE.md](Documentation/research/V10_18_JP3D_TRUE_SELECTIVE_DECODE.md). + +**v10.9.3** is a packaging hotfix that makes J2KSwift unconditionally URL-consumable. v10.9.2 attempted CWD-conditional dependency resolution (sibling path if present, else URL) but `FileManager.fileExists` is evaluated with CWD set to the consuming root package — any consumer with a `CompressionFamily` directory beside its own root caused J2KSwift to fall back to the path form, and stable-version consumers may not transitively depend on path/branch packages ("unstable-version package" error). v10.9.3 drops the probe and always uses the public Git URL; local co-development uses `swift package edit CompressionFamily --path ../CompressionFamily`. Single-file Package.swift change; no codec change, codestream bytes byte-identical to v10.9.2. + **v10.9.2** is a stability + packaging patch. It fixes two latent `SIGSEGV` crashes and a packaging defect. **HTJ2K encoder crash (#439):** the fused HT entropy path force-unwrapped an empty buffer's `baseAddress` and trapped on degenerate / zero-coefficient code-blocks — observed crashing parallel code-block workers during the DICOMKit v1.1.0 integration; the five force-unwraps across the three coefficient-extraction sites are now `guard let` (an empty buffer falls through to the existing zero-block path). **Report-generator crash:** `ValidationReportGenerator.textReport` passed Swift `String` values to `String(format:)` `%s` specifiers — `%s` requires a C string, so the argument was `strlen`'d as a bogus tagged pointer; the 14 `%s` sites are replaced with Swift column padding + interpolation (this defect had also been silently aborting full `swift test` runs). **SwiftPM URL consumption (#438):** `Package.swift`'s `CompressionFamily` dependency is now conditional — local path for co-development, public Git URL otherwise — making J2KSwift resolvable via `.package(url:)` once the public `CompressionFamily` repo is published. Also: CI restructured Apple-only, the SwiftLint gate fixed (288 error-level findings → 0, kept as warnings), and the `J2KMetalDWT` band geometry consolidated into one `BandGeometry` helper (output-identical). Codestream bytes byte-identical to v10.9.1 — the encoder fix only alters the previously-crashing path. Validated: mandatory commit gate + HT cross-codec suites 22/22, 0 failures; a full `swift test` (6148 tests) now runs to completion where the report crash previously aborted it. See [RELEASE_NOTES_v10.9.2.md](Documentation/releases/RELEASE_NOTES_v10.9.2.md). **v10.9.1** is a decoder correctness hotfix. The GPU multi-tile per-tile inverse 5/3 DWT corrupted the bottom edge of decoded images for sub-3-megapixel tiles at an odd tile-component canvas origin — e.g. a DX 2800×2288 image decoded as 2×2 tiles (observed max abs diff ≈ 9823). Root cause: `inverse2DGPUInt32` and `inverse2DCPUInt32` sized the LH/HH high-band as `height/2` (and `width/2`) instead of the canvas-anchored `height − llH` / `width − llW` — at an odd canvas origin the ISO/IEC 15444-1 band partition is uneven. The v8.3 fix corrected the multi-level-fused path but missed these two per-level functions; v10.3.0's `_gpuHTEntropyEnabled` routing change then made them reachable. Pure Swift host-code fix — the Metal kernels were never wrong; `default.metallib` is unchanged. Validated: `V8_3_GPUIDWTRootCauseDiagnostic` 3/3, all IDWT parity/bit-exact suites, the 5 multi-tile self-roundtrips, mandatory commit gate 7/7, cross-codec parity 14/14 (OpenJPEG/OpenJPH/Grok/Kakadu), and the warm cross-codec benchmark within run-to-run noise of v10.9.0 (J2KSwift wins 28/38 encode, 31/38 decode). Encoder and codestream bytes byte-identical to v10.9.0; decoder-only. Also bundles test maintenance — 8 stale-constant test updates and 19 dead-test deletions for de-scoped/parked features (test-only). See [RELEASE_NOTES_v10.9.1.md](Documentation/releases/RELEASE_NOTES_v10.9.1.md). diff --git a/RELEASE_NOTES_v10.10.0.md b/RELEASE_NOTES_v10.10.0.md new file mode 100644 index 00000000..9f5f4db9 --- /dev/null +++ b/RELEASE_NOTES_v10.10.0.md @@ -0,0 +1,197 @@ +# J2KSwift v10.10.0 + +**JP3D true partial-resolution + ROI footprint-skip.** Closes the +long-standing JP3D follow-up: the resolution-level field on +`JP3DDecoderConfiguration` was wired but never consulted; the +ROI decoder was decode-then-crop. Both are now true-selective. + +Decoder-only release; codestream bytes are byte-identical to v10.9.3. +Encoder unchanged. The MINOR bump reflects new functional behaviour +on existing public APIs (`JP3DDecoderConfiguration.resolutionLevel` +and `JP3DROIDecoder.decode(_:region:)`). + +## Summary + +Three coordinated changes inside `Sources/J2K3D/`: + +1. **`JP3DDecoderConfiguration.resolutionLevel` becomes functional.** + The field has shipped since v5.x but the decoder ignored it — + `JP3DDecoder(configuration: JP3DDecoderConfiguration(resolutionLevel: K)).decode(data)` + silently returned the full-resolution volume. v10.10.0 routes each + per-slice 2D codestream inside `JP3DSliceStackCodec` through the + 2D codec's existing `decodeResolution(_:options:)` (v10.5.0); the + returned volume's dimensions are now `⌈W / 2^K⌉ × ⌈H / 2^K⌉ × D`. + M2 release: **2.2–3.1× faster decode at `resolutionLevel = 1`**. + +2. **`JP3DROIDecoder` swaps decode-then-crop for true ROI footprint- + skip.** The per-tile path was: decode entire intersecting tile via + slice-stack → triple-loop crop the intersection into the ROI + buffer. v10.10.0 passes the per-tile in-tile sub-region into + `JP3DSliceStackCodec`, which in turn routes the per-slice 2D + decode through `decodeRegion(_:options:)` with `.direct` strategy + (v10.6.0 — code-blocks whose inverse-DWT cone-of-influence misses + the region skip entropy decode entirely) and returns Float buffers + sized exactly for the region rather than the whole tile. + +3. **Z-narrow ROI skip.** `JP3DSliceStackCodec.decode` pre-scans + slice headers (flags + length, no decode), finds the latest + non-residual slice ≤ `zRange.lowerBound`, and starts decoding + from there — completely skipping out-of-Z-range slices that + precede the request. Slices in `[z_start, zRange.lowerBound)` + decode purely to keep the Z-delta residual chain intact; their + output is discarded. M2 release: **ROI 1/4 decode 3.4–4.1× faster + than full decode** when combined with the per-tile XY footprint- + skip above. + +These compose at the slice-stack boundary; each cell of the +`{resolutionLevel, ROI}` matrix routes to the appropriate existing 2D +API. The combined `resolutionLevel > 0 + ROI` case throws loud rather +than silently producing wrong output (the 2D codec doesn't yet support +"footprint-skip at a downsampled resolution" — that's a separate arc). + +## What's New — production-default + +| Public API | v10.9.3 behaviour | v10.10.0 behaviour | +|---|---|---| +| `JP3DDecoder(configuration: cfg).decode(data)` with `cfg.resolutionLevel = K, K > 0` | Returns full-resolution volume (field ignored) | Returns volume sized `⌈W / 2^K⌉ × ⌈H / 2^K⌉ × D` | +| `JP3DROIDecoder().decode(data, region: r)` | Per intersecting tile: whole-tile decode + intersection-crop | Per intersecting tile: in-tile XY footprint-skip + Z-start non-residual scan; output assembled directly | +| `JP3DROIDecoder(configuration: cfg).decode(...)` with `cfg.resolutionLevel > 0 AND r` strict sub-volume | Silently ignored `cfg.resolutionLevel`; decoded at full resolution | Throws `J2KError.decodingError("...combining resolutionLevel > 0 with a sub-region is not yet supported...")` | + +Existing `JP3DDecoder().decode(data)` with default configuration +(`resolutionLevel = 0`) routes through the same code path it always +did — both encoder and full decoder produce byte-identical output +to v10.9.3. + +## What's New — opt-in + +None. All changes ship default-on. + +## Backward compatibility + +- **Codestream bytes**: byte-identical to v10.9.3 on every input. The + encoder is unchanged. +- **`JP3DDecoder().decode(data)`** (default config) is bit-exact + identical to v10.9.3 on every input. Validated by + `testResolutionLevel0IsBitExactFullDecode` and the existing + 61-test `JP3DDecoderTests` suite (all pass). +- **`JP3DROIDecoder().decode(data, region:)`** (default config) is + bit-exact identical to v10.9.3 on every input — the output voxels + are the same as full-decode-then-crop. Validated by + `testROIDecodeMatchesFullDecodeCropped`, + `testROIDecodeMatchesFullDecodeCroppedAtCorner`, and + `testZNarrowROIBitExactWithFullCropped`. +- **Behaviour change**: previously-silent + `JP3DROIDecoder(configuration: cfg)` with `cfg.resolutionLevel > 0` + now throws when given a strict sub-volume region. The case never + worked correctly pre-v10.10.0 (returned wrong dimensions); the + throw surfaces the limitation instead of hiding it. + +## Measured wins (M2 release, J2KBenchMac --jp3d --quick) + +| Fixture | full ms | res-1 ms | res-1 vs full | ROI 1/4 ms | ROI 1/4 vs full | +|---|---:|---:|---:|---:|---:| +| mr_3d_small (128×128×16, 262K vox) | 18.78 | 8.45 | **2.22×** | 5.57 | **3.37×** | +| ct_3d_small (256×256×16, 1 M vox) | 64.45 | 21.10 | **3.05×** | 15.55 | **4.14×** | +| mr_3d_mid (256×256×32, 2 M vox) | 130.13 | 43.25 | **3.01×** | 32.84 | **3.96×** | + +The `res-1` ratios (~2.2–3.1×) match the 2D v10.5.0 arc's 3-8× +thumbnail finding — JP3D's slice-stack arch means the 3D win is +exactly the 2D win applied per-slice, including the Z-delta residual +path which decodes residual + base at the same downsampled resolution. + +The ROI 1/4 ratios (~3.4–4.1×) are the combined effect of the XY +footprint-skip (per-slice 2D `decodeRegion(.direct)`) and the +Z-narrow skip (start at latest non-residual ≤ `zLower`). Encoder +non-residual cadence is the upper bound on Z-narrow savings; the +default `JP3DEncoderConfiguration.zDeltaMode = .auto` produces +non-residuals frequently enough that the bench corpus shows +consistent wins. + +## Test Suite Results + +| Suite | Tests | Result | +|---|---:|---| +| `V10_18_TrueSelectiveParityTests` (new — v10.10.0 parity oracle) | 9 | 9/9 PASS | +| `JP3DDecoderTests` (existing — 61-test JP3D regression suite) | 61 | 61/61 PASS | +| `J2KStrictCrossCodecValidationTests` (mandatory commit gate) | 3 | 3/3 PASS | +| `J2KMedicalCorpusPerformanceTests` (mandatory commit gate) | — | exit 0 | +| `J2KMedicalCorpusEncodePerformanceTests` (mandatory commit gate) | — | exit 0 | + +All 9 tests in the new parity oracle suite are bit-exact assertions +against the reference behaviour: + +- `testFullDecodeLosslessRoundTripBaseline` — full encode/decode + round-trip identity (baseline anchor for the others) +- `testROIDecodeMatchesFullDecodeCropped` — ROI output equals + full-decode-then-crop voxels (interior region) +- `testROIDecodeMatchesFullDecodeCroppedAtCorner` — same, region + anchored at (0, 0, 0) (off-by-one guard) +- `testResolutionLevel0IsBitExactFullDecode` — `resolutionLevel = 0` + collapses to default `decode()` byte-identical +- `testPartialResolutionShape` — dim contract `⌈W/2⌉ × ⌈H/2⌉ × D` + at `resolutionLevel = 1` +- `testPartialResolutionDeterministic` — two decodes at the same + level produce byte-identical voxels +- `testPartialResolutionLevel2Shape` — higher-K validity at K=2 + (off-by-one guard on the iteration) +- `testZNarrowROIBitExactWithFullCropped` — Z-narrow ROI output + equals full-decode-then-crop on a Z-slab that crosses a Z-delta + residual chain +- `testCombinedResolutionLevelAndROIIsPhase5Future` — tripwire that + asserts the combined case throws today (flips when the 2D codec + gains the combined path) + +## API surface + +No additions, no removals. Two existing public APIs gain functional +behaviour: + +- `JP3DDecoderConfiguration.resolutionLevel: Int` — was wired but + ignored; now consulted. +- `JP3DROIDecoder.decode(_ data: Data, region: JP3DRegion)` — was + decode-then-crop; now true ROI. + +## Known limitations + +- **Combined `resolutionLevel + ROI`** — throws loud today. The 2D + codec doesn't yet implement "footprint-skip at a downsampled + resolution"; that's a 2D arc, not JP3D, and would need to land + before this combination can compose. +- **Z-narrow savings are encoder-cadence-bound** — the Z-narrow + skip starts decoding from the latest non-residual slice + ≤ `zRange.lowerBound`. When the encoder produces long residual + chains (rare with default `.auto` Z-delta) the savings shrink. +- **GPU iDWT for JP3D** — `JP3DMetalDWT.swift` exists but isn't + wired into `JP3DDecoder`. Orthogonal to v10.10.0; would be its + own arc. + +## Reproducing the headline numbers + +```bash +# Build the bench CLI (release) +swift build -c release --target J2KBenchMac + +# Run the JP3D --quick bench (3 fixtures, 3 runs / 1 warmup): +.build/arm64-apple-macosx/release/J2KBenchMac --jp3d --quick +``` + +The bench corpus (defined in `Sources/J2KBenchMac/JP3DBench.swift`, +parity with the iOS J2KBenchApp Volumes-tab corpus) writes a +canonical JSON to the repo root with schema `warm-inproc-jp3d-v1`. + +## Companion documents + +- `Documentation/research/V10_18_JP3D_TRUE_SELECTIVE_DECODE.md` — + full arc writeup including phase-by-phase rationale, architecture + diagrams, the COD-marker peek primitive, the Z-delta slice-header + pre-scan design, and the open follow-ups. + +## What did NOT ship to main + +The v10.18-research branch also includes a new iOS J2K Bench "Volumes" +tab + 3-plane MPR viewer + 6 anatomical phantom volumes (Shepp-Logan +3D, brain MR, thorax CT, abdomen CT, spine MR sag, knee CT) and a +macOS `J2KBenchMac --jp3d` CLI mode for cross-silicon JP3D benchmarks. +Per the project convention (`feedback_research_no_main_merge.md`) +those bench-app changes stay on `v10.18-research` and are not part of +the main-branch release; only the codec changes ship here. diff --git a/Sources/J2K3D/JP3DDecoder.swift b/Sources/J2K3D/JP3DDecoder.swift index 3e7cfc45..d8693971 100644 --- a/Sources/J2K3D/JP3DDecoder.swift +++ b/Sources/J2K3D/JP3DDecoder.swift @@ -166,8 +166,25 @@ public actor JP3DDecoder { let grid = codestream.tileGrid let tilesExpected = grid.tilesX * grid.tilesY * grid.tilesZ - // Allocate output component buffers (Float per voxel) - let voxelCount = siz.width * siz.height * siz.depth + // v10.18-research — partial-resolution path. resolutionLevel = K: + // K = 0 → full resolution (current behaviour, all branches + // below collapse to their original arithmetic). + // K > 0 → in-plane (X, Y) dims halved K times; depth (Z) is + // per-slice and unaffected. Tile origins and per-tile + // dimensions scale the same way so the assembled + // volume has no gaps or overlaps. + let K = max(0, configuration.resolutionLevel) + let scale = 1 << K + @inline(__always) func down(_ n: Int) -> Int { + (n + scale - 1) / scale + } + let outW = down(siz.width) + let outH = down(siz.height) + let outD = siz.depth + + // Allocate output component buffers (Float per voxel) at the + // downsampled volume size. + let voxelCount = outW * outH * outD var componentBuffers = [[Float]]( repeating: [Float](repeating: 0, count: voxelCount), count: siz.componentCount @@ -201,6 +218,20 @@ public actor JP3DDecoder { continue } + // v10.18 partial-res tile placement. JPEG 2000 spec rule + // for downsampled coordinates: ⌈ref / 2^K⌉ for both origin + // and origin+extent. For aligned tile grids this yields + // adjacent downsampled tile placements without gaps — + // the unit test in V10_18_TrueSelectiveParityTests + // exercises corner + interior regions to catch any + // off-by-one. + let outX0 = down(x0) + let outY0 = down(y0) + let outZ0 = z0 + let outTW = down(x1) - outX0 + let outTH = down(y1) - outY0 + let outTD = td + // M2: every tile written by the new encoder is a slice-stack // payload (J3DS magic). Older tile shapes (raw Int32 dump, // legacy JP3DHTJ2K) are no longer produced — they only ever @@ -226,7 +257,8 @@ public actor JP3DDecoder { expectedTile: JP3DSliceStackCodec.ExpectedTile( width: tw, height: th, depth: td, componentCount: siz.componentCount - ) + ), + resolutionLevel: K ) } catch { if configuration.tolerateErrors { @@ -240,9 +272,9 @@ public actor JP3DDecoder { for comp in 0.. 0 + sub-region. The full- + // volume case (handled below via JP3DDecoder delegation) still + // honours resolutionLevel because there's no per-tile cropping. + if configuration.resolutionLevel > 0 { + let isFullVolumeRequest = ( + requestedRegion.x.lowerBound <= 0 && + requestedRegion.x.upperBound >= siz.width && + requestedRegion.y.lowerBound <= 0 && + requestedRegion.y.upperBound >= siz.height && + requestedRegion.z.lowerBound <= 0 && + requestedRegion.z.upperBound >= siz.depth) + if !isFullVolumeRequest { + throw J2KError.decodingError( + "JP3DROIDecoder: combining configuration.resolutionLevel > 0 " + + "with a sub-region is not yet supported. " + + "v10.18 Phase 5 wiring task. " + + "Until then, use either JP3DDecoder(configuration: ...) " + + "for partial-resolution decode, OR " + + "JP3DROIDecoder() (default config) for ROI decode — " + + "not both together.") + } + } + // Clamp the requested region to valid volume bounds let clampedXLower = max(0, requestedRegion.x.lowerBound) let clampedXUpper = min(siz.width, requestedRegion.x.upperBound) @@ -194,11 +218,37 @@ public actor JP3DROIDecoder { let th = tileInfo.height let td = tileInfo.depth - // M2: each tile is now a JP3D slice-stack payload — decode - // the entire tile (the slice-stack codec runs per-Z-slice - // 2D J2K) then crop the intersection into the ROI buffer. - // True per-component / per-resolution sub-decoding is a - // future ROI optimisation; this preserves correctness. + // v10.18 Phase 3 — compute the in-tile XY sub-region in + // tile-local coords. JP3DSliceStackCodec routes the per- + // slice 2D decode through `decoder.decodeRegion(.direct)` + // (v10.6.0 footprint-skip: code-blocks whose synthesis + // cone-of-influence misses the region skip entropy decode + // entirely) and returns Float buffers sized exactly for + // the region rather than the whole tile. The intersection- + // crop loop below collapses to a region-local copy with + // Z-axis filtering. + // + // Z stays per-slice because slice-stack residual coding + // chains through every slice — we still decode all slices + // in the tile to keep Z-delta correct, then only copy the + // in-Z-range ones into the ROI buffer. + let inTileXLower = max(0, clampedX.lowerBound - x0) + let inTileXUpper = min(tw, clampedX.upperBound - x0) + let inTileYLower = max(0, clampedY.lowerBound - y0) + let inTileYUpper = min(th, clampedY.upperBound - y0) + let inTileZLower = max(0, clampedZ.lowerBound - z0) + let inTileZUpper = min(td, clampedZ.upperBound - z0) + + // Tile is in tilesByIndex but its intersection is empty: + // shouldn't happen (tilesIntersecting filters this), but + // defensively skip. + guard inTileXLower < inTileXUpper, + inTileYLower < inTileYUpper, + inTileZLower < inTileZUpper else { + tilesSkipped += 1 + continue + } + guard JP3DSliceStackCodec.hasMagic(parsedTile.data) else { if configuration.tolerateErrors { warnings.append("Tile \(idx): not a JP3D slice-stack payload (missing 'J3DS' magic)") @@ -209,15 +259,60 @@ public actor JP3DROIDecoder { ) } + let inTileXRange = inTileXLower.. 0 AND a region is being decoded, the + // slice-stack codec throws the "combination is the Phase 5 + // wiring future-work" error LOUDLY rather than silently + // ignoring resolutionLevel (which is what JP3DROIDecoder + // did pre-Phase-3). + // + // v10.18-research Phase 6 — Z-narrow ROI skip. Pass the + // per-tile Z range to the slice-stack codec so that for + // tiles where the ROI Z-range starts deep into the tile, + // the codec scans forward from the latest non-residual + // slice ≤ zLower (rather than decoding slice 0..zLower + // verbatim purely to keep the Z-delta chain alive). + let K = max(0, configuration.resolutionLevel) + let inTileZRange = inTileZLower.. 0` — partial-resolution decode. Each per-slice 2D + /// codestream is decoded via `decoder.decodeResolution(_:options:)` + /// with the 2D `level` mapped from JP3D's K (where K=0 is full + /// and K rises toward the LL-band thumbnail). The returned + /// `componentBuffers` are sized for the **downsampled** in-plane + /// dimensions (Z is per-slice and not affected). The caller + /// (`JP3DDecoder.decode`) must allocate its volume + tile-origin + /// placement at the same downsampled scale. + /// + /// v10.18-research — `regionOfInterest`: + /// + /// • `nil` (default) — whole tile is decoded, current behaviour. + /// • non-nil — the per-slice 2D codestream is decoded via + /// `decoder.decodeRegion(_:options:)` with the in-tile XY sub- + /// region (v10.6.0 `.direct` strategy: code-blocks outside + /// the region's inverse-DWT footprint skip entropy decode). + /// The returned `componentBuffers` are sized for the + /// **region** dims (`regionOfInterest.xRange.count * + /// regionOfInterest.yRange.count * sliceCount`), in tile-local + /// region coordinates. The caller (`JP3DROIDecoder.decode`) + /// places these directly into the output ROI buffer at the + /// intersection offset — no second intersection-crop pass is + /// needed. + /// + /// v10.18-research — `zRange` (Z-narrow ROI skip): + /// + /// • `nil` (default) — every slice in the tile is decoded and + /// placed into the output (current behaviour). + /// • non-nil → tile-local Z range — the codec pre-scans the + /// slice headers (flags + length, no decode), finds the + /// latest non-residual slice ≤ `zRange.lowerBound`, and + /// starts decoding from there. Slices in + /// `[z_start, zRange.lowerBound)` are decoded purely to keep + /// the Z-delta residual chain intact and their output is + /// discarded; slices in `[zRange.lowerBound, zRange.upperBound)` + /// are kept. The returned `componentBuffers` are sized for + /// `zRange.count` slices (not the full `sliceCount`). + /// + /// Z-delta residual slices stay correct under all three modes + /// (partial-res, ROI XY, Z-narrow) because residual + prior + /// reconstructed slice are decoded at the SAME parameters, so + /// the per-voxel add operates on matched dims. func decode( payload: Data, - expectedTile: ExpectedTile + expectedTile: ExpectedTile, + resolutionLevel: Int = 0, + regionOfInterest: (xRange: Range, yRange: Range)? = nil, + zRange: Range? = nil ) async throws -> [[Float]] { let header = try parseHeader(from: payload) @@ -287,15 +339,67 @@ struct JP3DSliceStackCodec: Sendable { ) } - let voxelsPerComponent = header.tileWidth * header.tileHeight * header.sliceCount - let voxelsPerSlice = header.tileWidth * header.tileHeight + // v10.18 — derive the per-slice decoded dimensions for this + // resolutionLevel. K = 0 → identity (current behaviour). + let K = max(0, resolutionLevel) + let baseTileWidth: Int + let baseTileHeight: Int + if K == 0 { + baseTileWidth = header.tileWidth + baseTileHeight = header.tileHeight + } else { + let factor = 1 << K + baseTileWidth = (header.tileWidth + factor - 1) / factor + baseTileHeight = (header.tileHeight + factor - 1) / factor + } + + // v10.18 Phase 3 — region-of-interest. If set, the output slice + // dims shrink to the region. (The region is expressed in + // *downsampled* tile-local coords if K > 0; callers translate + // before passing.) For K == 0 and ROI nil, this collapses to + // identity = the current behaviour. + let outTileWidth: Int + let outTileHeight: Int + let regionXLower: Int + let regionYLower: Int + if let roi = regionOfInterest { + outTileWidth = roi.xRange.count + outTileHeight = roi.yRange.count + regionXLower = roi.xRange.lowerBound + regionYLower = roi.yRange.lowerBound + } else { + outTileWidth = baseTileWidth + outTileHeight = baseTileHeight + regionXLower = 0 + regionYLower = 0 + } + + // v10.18 Phase 6 — Z-narrow ROI. Default = whole-tile Z range. + // Clamp to [0, sliceCount); zUpper > zLower is required. + let zLower: Int + let zUpper: Int + if let zr = zRange { + zLower = max(0, zr.lowerBound) + zUpper = min(header.sliceCount, zr.upperBound) + guard zLower < zUpper else { + throw J2KError.decodingError( + "JP3D slice-stack: zRange \(zr) is empty after clamping " + + "to [0, \(header.sliceCount))") + } + } else { + zLower = 0 + zUpper = header.sliceCount + } + let outSliceCount = zUpper - zLower + + let voxelsPerSlice = outTileWidth * outTileHeight + let voxelsPerComponent = voxelsPerSlice * outSliceCount var componentBuffers = [[Float]]( repeating: [Float](repeating: 0, count: voxelsPerComponent), count: header.componentCount ) let decoder = J2KDecoder() - var cursor = payload.index(payload.startIndex, offsetBy: Self.headerByteCount) // v2 only — v1 codestreams predate this branch and aren't // produced anywhere now. @@ -305,36 +409,119 @@ struct JP3DSliceStackCodec: Sendable { ) } - // Holds the previous reconstructed slice, per component, as - // Int32 — used to add the residual when `is_residual` is set. - var prevSliceInt: [[Int32]]? = nil - + // v10.18 Phase 6 — pre-scan: walk every slice's flag+length + // bytes once to record (flags, codestream subdata range) per + // slice. This is O(N) for the small per-slice headers (5 bytes + // each plus a length prefix) but pays for itself when zRange + // is set, since we can then jump to the latest non-residual + // slice ≤ zLower without decoding the prior ones. + var sliceEntries: [(flags: UInt8, codestreamStart: Data.Index, length: Int)] = [] + sliceEntries.reserveCapacity(header.sliceCount) + var scanCursor = payload.index(payload.startIndex, offsetBy: Self.headerByteCount) for z in 0.. 0 { + for z in stride(from: zLower, through: 0, by: -1) { + let isResidual = (sliceEntries[z].flags & Self.sliceFlagIsResidual) != 0 + if !isResidual { + zStart = z + break + } + } + } + + // Cache the per-slice 2D codec's decomposition-level count N + // on the first slice's metadata peek, then reuse — JP3D + // produces all slices with the same N (JP3DEncoder.levelsX/Y + // is fixed across the volume), so a per-slice peek would + // cost N parses for no information. + var perSliceDecompLevels: Int? = nil + + // Holds the previous reconstructed slice, per component, as + // Int32 — used to add the residual when `is_residual` is set. + var prevSliceInt: [[Int32]]? = nil + + for z in zStart..0, no ROI) → decoder.decodeResolution (Phase 2) + // (K>0, ROI) → decoder.decodeRegion (Phase 2+3) + // then partial-res must be expressed + // inside the J2KROIDecodingOptions — + // currently the 2D ROI path always + // decodes at full resolution. Falls back + // to (K=0, ROI) for safety and the + // tile-output dims must therefore be in + // FULL tile coords; assert K == 0 when + // ROI is set (the JP3DROIDecoder caller + // today never combines them, and the + // composition is a Phase 5 wiring task). + if regionOfInterest != nil && K > 0 { + throw J2KError.decodingError( + "JP3D slice-stack: combining resolutionLevel > 0 with " + + "regionOfInterest is not yet supported. v10.18 Phase 5 " + + "is the wiring task.") + } + let image: J2KImage + if let roi = regionOfInterest { + let opts = J2KROIDecodingOptions( + region: J2KRegion( + x: roi.xRange.lowerBound, y: roi.yRange.lowerBound, + width: roi.xRange.count, height: roi.yRange.count), + strategy: .direct) + image = try await decoder.decodeRegion(codestream, options: opts) + } else if K == 0 { + image = try await decoder.decode(codestream) + } else { + if perSliceDecompLevels == nil { + perSliceDecompLevels = Self.peekDecompositionLevels(codestream) + } + let N = perSliceDecompLevels ?? 0 + let sliceLevel = max(0, N - K) + let opts = J2KResolutionDecodingOptions( + level: sliceLevel, upscale: false) + image = try await decoder.decodeResolution(codestream, options: opts) + } + _ = regionXLower; _ = regionYLower // currently unused; placement is region-local let isResidual = (sliceFlags & Self.sliceFlagIsResidual) != 0 let decodedInt = try readImageAsInt32( - image, tileWidth: header.tileWidth, tileHeight: header.tileHeight, + image, tileWidth: outTileWidth, tileHeight: outTileHeight, bitDepth: header.bitDepth, signed: isResidual ) @@ -363,11 +550,17 @@ struct JP3DSliceStackCodec: Sendable { // Write the reconstructed slice into the per-component // Float buffers (the outer JP3DDecoder takes Floats). - for c in 0..= zLower && z < zUpper { + let outZ = z - zLower + for c in 0.. Int { + let scanLimit = min(data.count, 256) + // Need at least i + 10 in-bounds: FF (i), 52 (i+1), … NL (i+9). + guard scanLimit >= 11 else { return 0 } + // SOC is FF 4F at the start; jump past it. + var i = data.startIndex + 2 + let upper = data.startIndex + scanLimit - 10 + while i <= upper { + if data[i] == 0xFF && data[i.advanced(by: 1)] == 0x52 { + let nlIndex = i.advanced(by: 9) + if nlIndex < data.endIndex { + return Int(data[nlIndex]) + } + } + i = i.advanced(by: 1) + } + return 0 + } } diff --git a/Tests/JP3DTests/V10_18_TrueSelectiveParityTests.swift b/Tests/JP3DTests/V10_18_TrueSelectiveParityTests.swift new file mode 100644 index 00000000..cecad76d --- /dev/null +++ b/Tests/JP3DTests/V10_18_TrueSelectiveParityTests.swift @@ -0,0 +1,384 @@ +// +// V10_18_TrueSelectiveParityTests.swift +// J2KSwift +// +// v10.18-research — JP3D true partial-resolution / ROI parity oracle. +// +// Establishes the *specification* the JP3D codec MUST satisfy as the +// v10.18 arc replaces "decode then crop" / stubbed-resolutionLevel +// with true selective decode. These tests are the regression guard +// across Phases 2-4: +// +// • Phase 2 (port `maxResolutionLevel` from 2D `extractTileData` +// into JP3DSliceStackCodec): `testPartialResolutionShape` and +// `testPartialResolutionLLBandIsAFullDecodeLLBand` are XCTSkip'd +// today (the field is stubbed) and become active gates once +// Phase 2 lands. +// +// • Phase 3 (port ROI footprint-skip into JP3DROIDecoder, replacing +// decode-then-crop): `testROIDecodeMatchesFullDecodeCropped` +// guards bit-exactness across the change. Today it passes +// trivially (the implementation IS decode-then-crop); after +// Phase 3 it must still pass (true ROI must produce identical +// output as the cropped full decode). +// +// • Phase 4 (tile-skip): subsumed by the ROI parity test — if +// tile-skip skips a tile whose footprint *does* overlap the +// region, the ROI test will fail. So this file's parity gate +// covers Phases 3 + 4 jointly. +// +// • Baseline `testFullDecodeLosslessRoundTripBaseline` is a sanity +// anchor: confirms our LCG-style synthesised volumes survive a +// full encode/decode round-trip bit-exact, so any later parity +// failure is unambiguously about partial-res / ROI, not the +// underlying codec. + +import XCTest +@testable import J2KCore +@testable import J2K3D + +final class V10_18_TrueSelectiveParityTests: XCTestCase { + + // MARK: - Test fixture builder + + /// One deterministic 16-bit grayscale test volume used across the + /// file. Smaller than the bench fixtures (compile-time test, not + /// a bench) but large enough that resolution-level-1 produces + /// meaningful spatial downsampling. Same LCG synthesis the + /// J2KBenchApp Volumes tab uses, so behaviour observed here maps + /// directly to what the iOS MPR viewer renders for the + /// `mr_3d_small` / `ct_3d_small` corpus fixtures. + private func makeLCGVolume( + width: Int = 64, height: Int = 64, depth: Int = 8, + bitDepth: Int = 16, seed: UInt64 = 0xC0FFEE + ) -> J2KVolume { + let bytesPerSample = (bitDepth + 7) / 8 + let voxelCount = width * height * depth + let maxVal = (1 << bitDepth) - 1 + var data = Data(count: voxelCount * bytesPerSample) + var s: UInt64 = seed &* 6364136223846793005 &+ 1442695040888963407 + data.withUnsafeMutableBytes { raw in + if bytesPerSample == 1 { + let p = raw.bindMemory(to: UInt8.self) + for i in 0..> 16) & 0xFFFFFFFF) % (maxVal + 1)) + } + } else { + let p = raw.bindMemory(to: UInt16.self) + for i in 0..> 16) & 0xFFFFFFFF) % (maxVal + 1)) + } + } + } + let comp = J2KVolumeComponent( + index: 0, bitDepth: bitDepth, signed: false, + width: width, height: height, depth: depth, + data: data) + return J2KVolume( + width: width, height: height, depth: depth, + components: [comp]) + } + + /// Reads a single voxel as an Int, honouring the component's bit + /// depth and the little-endian byte order J2KVolumeComponent uses + /// (per `JP3DDecoderTests.makeTestVolume`). + private func voxelValue(in volume: J2KVolume, + x: Int, y: Int, z: Int, + comp: Int = 0) -> Int { + guard comp < volume.components.count else { return 0 } + let c = volume.components[comp] + let bps = c.bytesPerSample + let i = ((z * c.height * c.width) + y * c.width + x) * bps + guard i + bps <= c.data.count else { return 0 } + return c.data.withUnsafeBytes { raw -> Int in + switch bps { + case 1: return Int(raw[i]) + case 2: return Int(raw[i]) | (Int(raw[i + 1]) << 8) + case 3: return Int(raw[i]) | (Int(raw[i + 1]) << 8) | (Int(raw[i + 2]) << 16) + case 4: return Int(raw[i]) | (Int(raw[i + 1]) << 8) + | (Int(raw[i + 2]) << 16) | (Int(raw[i + 3]) << 24) + default: return 0 + } + } + } + + private func encodeLossless(_ volume: J2KVolume) async throws -> Data { + let cfg = JP3DEncoderConfiguration(compressionMode: .losslessHTJ2K) + return try await JP3DEncoder(configuration: cfg).encode(volume).data + } + + // MARK: - Baseline: full-decode lossless round-trip + + /// Sanity anchor: an LCG-noise volume survives encode→decode + /// bit-exact via the canonical JP3DDecoder. If this fails, any + /// partial-res / ROI parity failure below is meaningless — fix + /// the codec round-trip first. + func testFullDecodeLosslessRoundTripBaseline() async throws { + let volume = makeLCGVolume() + let codestream = try await encodeLossless(volume) + let decoded = try await JP3DDecoder().decode(codestream).volume + + XCTAssertEqual(decoded.width, volume.width) + XCTAssertEqual(decoded.height, volume.height) + XCTAssertEqual(decoded.depth, volume.depth) + XCTAssertEqual(decoded.components.count, volume.components.count) + + for z in 0.. 0` with a + /// ROI decoder is the Phase 5 future-work case. JP3DSliceStackCodec + /// throws today rather than silently producing wrong output. When + /// Phase 5 wires the combination through, this test will start + /// passing the (currently absent) success branch — at that point + /// invert the assertion to check the combined output instead. + func testCombinedResolutionLevelAndROIIsPhase5Future() async throws { + let volume = makeLCGVolume(width: 64, height: 64, depth: 8) + let codestream = try await encodeLossless(volume) + + let cfg = JP3DDecoderConfiguration(resolutionLevel: 1) + let region = JP3DRegion(x: 16..<48, y: 16..<48, z: 0..<4) + + do { + _ = try await JP3DROIDecoder(configuration: cfg).decode( + codestream, region: region) + XCTFail("Phase 5 has landed — invert this assertion to verify " + + "the combined output instead.") + } catch { + // Expected today: JP3DSliceStackCodec.decode throws when + // both resolutionLevel > 0 AND regionOfInterest are set. + // The check happens per-tile during the ROI decode, so we + // accept any error here. + } + } +}