diff --git a/docs/adr/0137-denoise-forward-lookahead-cadence.md b/docs/adr/0137-denoise-forward-lookahead-cadence.md new file mode 100644 index 0000000..829f95f --- /dev/null +++ b/docs/adr/0137-denoise-forward-lookahead-cadence.md @@ -0,0 +1,102 @@ + +# ADR-0137: Forward-lookahead (bidirectional) temporal denoise — cadence-aware + +- **Status**: Proposed (premise validated; build pending) +- **Date**: 2026-06-20 +- **Deciders**: Lusoris + +## Context + +`vf_pelorus_denoise_vulkan`'s temporal term is **causal**: it averages the +current frame against a ring of *previous* frames, gated by `tcut` (a tap whose +delta exceeds `tcut` breaks the walk, so motion / scene-cuts / drawing-changes +cannot ghost). This is correct but asymmetric, and animation exposes the gap. + +Limited animation holds each drawing for 2–3 video frames (2s/3s cadence). With +independent per-frame noise (grain, capture, prior compression) on a held +drawing `{A0, A1}`, averaging the duplicates recovers the clean drawing — but +the causal walk only helps the **trailing** frame: denoising `A1` finds `A0` +(prev, same drawing, low delta → averaged), while the **leading** frame `A0` +sees only the *previous, different* drawing (`tcut` breaks) → it gets no temporal +help. Half the frames of every held run are under-denoised. + +This is the inverse of the encoder-steering negatives: it is **preprocessing** +(cleaning the source), which is where Pelorus wins. + +## Premise validation (measured, 2026-06-20) + +Synthetic 2s-cadence clip: 12 distinct BBB drawings, each held ×2 (24 frames), +with independent per-frame noise (`noise=alls=20:allf=t+u`). Denoised with the +current causal filter (`sigmat=0.08 strength=0.9 prev=4 blend=0.85`), PSNR vs the +clean held cadence: + +| | PSNR vs clean | +|---|---| +| noisy (baseline) | 32.89 dB | +| causal denoise | 34.42 dB (**+1.53 dB**) | + +Fully averaging the two held duplicates would give ≈ +3 dB (noise ÷ √2). The +causal filter captures only ~half of that — exactly the leading-frame gap. The +headroom (≈ +1.5 dB more, ~doubling the recovery) is real, unlike the encoder-RC +bets. (This is the premise check the fp16/subgroup/per-shot-CRF negatives taught +us to run *before* building.) + +## Decision (proposed) + +Add a **forward lookahead**: a small ring of *next* frames so the temporal walk +is bidirectional. `A0` then finds `A1` (next, same drawing) and both held frames +recover ≈ +3 dB. Cadence-awareness falls out of the existing `tcut` gate (held +duplicates are low-delta in *both* directions; a drawing change breaks the walk +either way) — no explicit cadence classifier needed for v1. + +Scope for v1 (minimal, same-coordinate forward taps; forward-MC is a follow-up +since the cadence case is static-held, no motion): + +1. **Filter lifecycle**: convert from the immediate `filter_frame` model to a + 1-frame-delay buffered model (`activate()`): hold the incoming frame, emit the + previous one processed with the new frame as its forward tap; flush the last + held frame at EOF. (The filter is currently causal — "no 1-frame latency, no + EOF flush" — so this is the load-bearing change.) `lookahead` AVOption, + default 1, range 0..MAX (0 = today's causal behaviour, bit-identical). +2. **Shader**: one `next0_images[]` binding (mirror `prev0_images`); a forward + same-coord temporal tap in the temporal loop, `tcut`-gated like the prev taps. +3. **Lockstep** `pelorus_denoise.comp` (AGENTS rule 4). + +## Validation plan + +- **Cadence oracle** (above): the forward-lookahead recovery must approach +3 dB + (vs the causal +1.5 dB) on the synthetic 2s-cadence clip. +- **Correctness**: output frame count == input frame count (no drop/dup at EOF); + `lookahead=0` is bit-identical to the current causal filter (SSIM 1.0). +- **Live-action**: SSIM-vs-clean on a noisy real clip (bidirectional same-coord + taps also help low-motion live-action); confirm no regression vs causal. +- **Kill-criterion**: if the cadence recovery does not improve materially over + causal, or `lookahead=0` is not bit-identical, do not ship. + +## Alternatives considered + +- **Causal-only, boost spatial strength on the leading frame.** Rejected — + spatial denoising blurs detail; it cannot recover the held-duplicate noise the + way lossless temporal averaging of the same drawing does. +- **Explicit cadence classifier (detect the 2s/3s pattern).** Deferred — the + `tcut` gate already discriminates same-drawing from drawing-change in both + directions; an explicit classifier is only needed for cadence-driven *frame + decimation* (a separate, encoder-facing feature), not for the denoise. +- **Forward motion-compensated taps.** Deferred to a follow-up — needs forward + MV from `pelorus_mc`; the cadence win is the static-held case (no motion). + +## Consequences + +- Adds a 1-frame latency + an EOF flush (acceptable for offline pre-encode; the + header invariant "no latency / no flush" is updated). `lookahead=0` preserves + today's behaviour exactly. +- A genuine preprocessing improvement (Pelorus's proven lane), validated in + premise before the build — the first positive direction after the v0.11–v0.13 + encoder-RC negatives. + +## References + +- ADR-0112 (the denoise filter), ADR-0131 (the MC→denoise warp — the causal + motion path this complements), ADR-0125 (the anime tune pipeline this serves). +- Premise-check data: `docs/development/bench-results.md` (v0.14, to accompany + the build). diff --git a/docs/adr/README.md b/docs/adr/README.md index db9bb24..f87624f 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -49,3 +49,4 @@ sibling keep vmafx's number (e.g. 0100, 0108) for an easy cross-walk. | [0134](0134-denoise-shared-mem-tile.md) | Shared-memory tiling of the denoise NLM spatial window (opt-in `tile`, default off): the kernel is fetch-bound (~441 image loads/px re-reading an ~81-px window 9×), so tile=1 cooperatively loads the window into shared memory once per workgroup. Bit-identical (SSIM 1.0). Validated +65.7% (2.9×) on Arc A380, neutral on AMD, −1–4% on the cache-rich 4090 → default off, flagship-first. Supersedes the reverted fp16 attempt (memory-bound, not ALU-bound) | Accepted | | [0135](0135-analyze-perceptual-aq-map.md) | Source-side perceptual bit-allocation map (`aq` perceptual-AQ on the clean source) for analyze, aiming to beat the encoder's in-loop variance-AQ. Implemented (Chou-Li luma + NAMM masking + edge/banding floors, signed mean-centered qoffsets) + measured: strongly cuts banding (CAMBI −14–25%) but at a net fidelity cost (SSIMULACRA2 BD-rate +13–20%, PSNR +7–14%), and a pure loss on real non-banding content. Fails the kill-criterion; `roi=1` already does banding without the texture-starving harm. Code not merged | Rejected | | [0136](0136-analyze-frame-metadata.md) | analyze emits its per-frame scalars (complexity/texture/motion/variance/edge/banding/scene_cut) as `lavfi.pelorus.*` frame metadata (the vf_scdet idiom), readable via ffprobe / `metadata=print`. The interop blob isn't muxed to file, so this is the host-side extraction path for per-shot CRF / autotune orchestration + shell debugging. Additive, host-side; no interop-ABI or shader change | Accepted | +| [0137](0137-denoise-forward-lookahead-cadence.md) | Forward-lookahead (bidirectional) temporal denoise — cadence-aware. The causal walk only denoises the trailing frame of each held animation drawing (2s/3s cadence); the leading frame sees a different drawing (tcut-breaks) → under-denoised. Premise validated: causal recovers +1.5 dB on a synthetic 2s-cadence clip vs ~+3 dB potential. Design: a 1-frame-delay buffered model (activate) + a next-frame shader tap, tcut-gated. First positive direction after the encoder-RC negatives. Build pending | Proposed |