VMAFx · lusoris · Jun 27, 2026 · Jun 27, 2026
@@ -62,6 +62,7 @@ All notable changes to Pelorus are documented here. The format is
 - **`vf_pelorus_denoise_vulkan`** gains an opt-in `lookahead` option (default 0) — forward-lookahead (bidirectional) temporal denoise ([ADR-0137](docs/adr/0137-denoise-forward-lookahead-cadence.md)). `lookahead=1` delays output one frame so the temporal walk also samples the **next** frame (same-coordinate, `tcut`-gated), recovering the leading frame of a held animation drawing (2s/3s cadence) that the causal walk under-denoises. Validated +0.37 dB on a 2s-cadence clip; `lookahead=0` is bit-identical/no-latency. Implemented as an `activate()` delay-line + a 6th `next0_images` shader tap, `.comp` kept in lockstep. Docs: `docs/metrics/denoise.md`.
 - **`vf_pelorus_dehalo_vulkan`** gains an opt-in `tile` option (default 0) — shared-memory tiling of the box-blur window ([ADR-0139](docs/adr/0139-dehalo-shared-mem-tile.md)). `box_blur` re-reads an overlapping 17×17 window ~5× per pixel (fetch-bound; an ALU-strip cut Arc rtime −70%), so `tile=1` cooperatively loads the window into shared memory once per plane. **Bit-identical** (SSIM 1.000000); validated **−38% (1.6×) on an Arc A380**, ~neutral on the cache-rich 4090 → default off, opt-in for weak/integrated/mobile GPUs + `tune=anime`. The ADR-0134 denoise-tiling idiom applied to its fetch-bound twin (the `aa` sobel kernel was ALU-bound and refuted). Docs: `docs/metrics/dehalo.md`.
 - **`vf_pelorus_aa_vulkan`** gains an opt-in `fast` option (default 0) — hoists the redundant sobel-mag into shared memory ([ADR-0140](docs/adr/0140-aa-sobel-mag-hoist.md)). aa is ALU-bound (a premise-check found stripping the sobel collapsed rtime −86%, so tiling was refuted), and `sobel_mag` is recomputed ~1156×/px across the overlapping `emask` windows; `fast=1` computes each cell's sobel **once** per workgroup into shared memory and `emask` reduces from the cache. **Bit-identical** (SSIM 1.000000, `cmp` 0 bytes both GPUs); validated **12.6× (−92%) on Arc A380, 2.6× (−62%) on the RTX 4090** — being an ALU win it helps every GPU (unlike fetch-tiling). Docs: `docs/metrics/aa.md`.
+- **`vf_pelorus_grain_estimate_vulkan`** now emits `lavfi.pelorus.grain_sigma` and `lavfi.pelorus.grain_flat` as per-frame metadata (the [ADR-0136](docs/adr/0136-analyze-frame-metadata.md) `av_dict_set` pattern — no interop ABI or shader change, the value was already computed). `grain_sigma` is the peak per-band RMS residual measured over edge-gated locally-flat pixels (structure excluded → what survives is grain stddev); `grain_flat` is the flat fraction it was measured over (confidence). This is the first detection enabler for the `tune=auto` content-adaptive router ([ADR-0142](docs/adr/0142-tune-auto-content-router.md)) — it unblocks routing grainy content (the proven −34% denoise lever) without parsing the `PEL_SEC_FILMGRAIN` side-data blob. Verified discriminating: heavy-grain 0.019 vs clean-ish Bluray 0.012. Docs: `docs/metrics/grain_estimate.md`.
 
 ### Changed
 

@@ -0,0 +1 @@
+- **`vf_pelorus_grain_estimate_vulkan`** now emits `lavfi.pelorus.grain_sigma` and `lavfi.pelorus.grain_flat` as per-frame metadata (the [ADR-0136](docs/adr/0136-analyze-frame-metadata.md) `av_dict_set` pattern — no interop ABI or shader change, the value was already computed). `grain_sigma` is the peak per-band RMS residual measured over edge-gated locally-flat pixels (structure excluded → what survives is grain stddev); `grain_flat` is the flat fraction it was measured over (confidence). This is the first detection enabler for the `tune=auto` content-adaptive router ([ADR-0142](docs/adr/0142-tune-auto-content-router.md)) — it unblocks routing grainy content (the proven −34% denoise lever) without parsing the `PEL_SEC_FILMGRAIN` side-data blob. Verified discriminating: heavy-grain 0.019 vs clean-ish Bluray 0.012. Docs: `docs/metrics/grain_estimate.md`.
@@ -0,0 +1,109 @@
+<!-- markdownlint-disable MD013 -->
+# ADR-0142: `tune=auto` — content-adaptive pre-encode router (design) + the grain-sigma detection enabler
+
+- **Status**: Proposed (2026-06-27) — design accepted; first enabler (grain-sigma metadata) shipped in this PR; per-content legs validated + the router built incrementally
+- **Date**: 2026-06-27
+- **Deciders**: Lusoris
+
+## Context
+
+The session's measurement campaign (real `vmaf` v1.0.16 + v0.6.1-NEG, an x265-enabled
+ffmpeg, on-hardware NVENC) established a **law** that reframes the whole product:
+
+- Hardware encoders (NVENC/QSV/AMF) **are** worse than software (x265 beats NVENC ~0.005
+  SSIM iso-bitrate — no psy-RD/RDOQ/trellis, weak temporal filter, weightp-off-with-B).
+- **But the pre-encode gain that closes the gap is REDUCTIVE and CONTENT-DEPENDENT:**
+  - **Additive** (pre-sharpen/CAS) is a **wash** on VMAF-NEG and costs bits — you cannot
+    add your way past the deadzone (the HF-energy "recovery" is a false proxy).
+  - **Reductive** (denoise/dehalo/demosquito/deblock) is real, but the magnitude scales
+    with *how much removable impairment exists*: −34% BD-rate on the real GPU denoise at
+    heavy grain, ~5–7% at moderate grain, **~0 on clean** (clean Bluray barely bands →
+    deband marginal; same law).
+  - **Source-side rate-control** (perceptual-AQ map, per-shot CRF) **loses** structurally
+    (ADR-0135/0132) — the encoder's own RC wins. Never emit a QP map.
+- And it is **mostly tuning**: the same filter is a false-negative with the wrong params.
+  Measured: the default denoise leans spatial and *over-smoothed* (tied at high bitrate);
+  retuned temporal-dominant (`blend=0.85 prev=4`) it recovers a gain (+0.7 NEG) at the
+  same point — grain is temporally *incoherent*, so the temporal walk is the lever, not
+  spatial strength.
+
+There is therefore **no single best filter** — there is the **right tuned boost per
+content class**, and a router that detects the class and applies it. This is `tune=auto`.
+
+## Decision
+
+Build a content-adaptive router (`tune=auto`) that reads `vf_pelorus_analyze`'s per-frame
+metadata, classifies the content per shot (segmented by `scene_cut`, with hysteresis),
+and applies the matching **reductive** boost with params **scaled to the measured
+impairment**. The routing table (designed by an 8-class boost panel, grounded in the law):
+
+| content class | detect (analyze.*) | boost | key params (scale with) | expected |
+|---|---|---|---|---|
+| **grainy/noisy** | `grain_sigma` over flat bands (NEW metadata) | denoise (temporal-dominant) + grain_estimate→FGS resynth | `strength,blend,sigmat ∝ grain_sigma` | **−34%** proven |
+| **anime/2D** | high `edge` AND high `banding`/flat co-occurrence | dehalo + aa + deband, co-gated by one edge mask | per the anime tune | ~3–8% SSIMu |
+| **textured live-action** | high `variance`/`edge`, low `motion` | denoise spatial-dominant (strip micro-noise only) | `blend 0.3–0.4` | medium |
+| **re-encode/compressed** | edge-band temporal excess (NEW) | **new `demosquito`** (edge-band temporal, the band denoise refuses) | premise-gated | single-digit % |
+| **dark/low-light/HDR** | `dark_frac` (NEW) | denoise luma-masked to dark tiles | dark-gated | ~5–15% sub-frame |
+| **screen/text** | very-high `edge`, very-high flat fraction, low `motion` | deblock (flat-snap + ring-strip) | edge-preserving | medium |
+| **high-motion/sports** | `motion` ≥ 0.6 (needs upstream `mc`) | denoise + mc-warp (ADR-0131, on cleaner pixels) | tcut tight | large |
+| **clean/pristine** | all impairment signals below floor | **NO-OP** (emit zero filters) | — | ~0 by design (avoided loss) |
+
+**Critical constraints the router must respect** (verified against the analyze source):
+`analyze.texture` is a *derived* `0.5·var + 0.5·edge` blend (not orthogonal) — classify on
+`variance`+`edge` directly; `analyze.motion` is 0 unless an upstream `pelorus_mc` attached
+`PEL_SEC_MOTION` — every motion-gated route needs `mc` in the graph.
+
+**The detection gap (this PR's first enabler).** Three detectors need host-side scalars
+that did not exist (only `analyze` emitted metadata; `grain_estimate`/`denoise`/`mc` emitted
+none): `grain_sigma`, `noise_sigma`, `dark_frac`. All are addable via the ADR-0136
+`av_dict_set` pattern — **no interop ABI change, no shader change, the value is already
+computed**. This PR ships the first and highest-leverage one: `vf_pelorus_grain_estimate`
+now emits `lavfi.pelorus.grain_sigma` (peak per-band RMS residual over the edge-gated flat
+bands — structure is excluded by construction, so what survives is grain stddev) and
+`lavfi.pelorus.grain_flat` (the flat fraction it was measured over = confidence). Verified
+discriminating: heavy-grain clip 0.0191 vs clean-ish Bluray 0.0116. The grainy leg is the
+proven `−34%` lever, so unblocking its detection is the right first step.
+
+## Validation priority (each leg iso-bitrate-proven before it enters the router)
+
+1. **grainy** `strength ∝ grain_sigma` scaling (ablate vs a flat preset) — proven lever, just the law.
+2. **clean** NO-OP guard (default ties/beats the full chain on pristine clips — the avoided-loss).
+3. **high-motion** denoise (the denoise half is proven on BBB; validate on real sports).
+4. **textured** micro-noise strip (stand-in shows −19…−23% bits; confirm it survives the gate).
+5. **anime** stack (dehalo/aa/deband ablations on real impaired anime).
+6. **re-encode** demosquito **premise-check first** (edge-band temporal excess ≫ flat, and denoise leaves it untouched).
+7. **dark/HDR** dark-masked denoise (rides the proven lever; the survival-dither leg is the ADR-0141 risk).
+
+Every leg: tune on one clip, **confirm on a held-out clip**, iso-bitrate, VMAF-NEG vs clean —
+so we find *real* tuned gains, not params overfit to one clip's metric.
+
+## A parallel track — encoder-integration patches
+
+Distinct from the pre-encode filter lane: patch FFmpeg's `nvenc`/`qsv`/`amf` wrappers to
+expose/tune hardware features they underuse (temporal-AQ, deeper lookahead, weighted
+prediction for the fade weakness, `b_ref_mode`, the native emphasis-level map our analyze
+maps could drive; oneVPL per-block QP / low-power AQ for Intel). Tracked as a parallel
+research+validate track — **check the current SDKs** (not memory) for what FFmpeg actually
+fails to expose, and prove each patch iso-bitrate on real hardware (the NVENC ME-hints patch
+was a zero-gain reminder that encoder patches can also be no-gains).
+
+## Consequences
+
+- This PR: `grain_estimate` emits `grain_sigma`/`grain_flat` (filter-only, no ABI/shader
+  change, side-data path untouched). Foundation for the grainy route.
+- Follow-ups (separate PRs, in priority order): the remaining detection enablers
+  (`noise_sigma`, `dark_frac`, `edge_temporal_excess`), the per-leg validations, the
+  `tune=auto` router filter/preset, and the encoder-patch track.
+- Reinforces the project lane: reductive preprocessing tuned per content, never RC-competing,
+  never additive-past-the-deadzone.
+
+## References
+
+- bench-results v0.2/v0.3 (denoise BD-rate), v0.16/ADR-0141 (additive wash + over-reach), the
+  session campaign (NVENC<x265, content-dependence, tuning-recovers-false-negatives).
+- ADR-0136 (analyze frame metadata — the av_dict_set pattern), ADR-0115 (grain_estimate),
+  ADR-0112/0131/0137 (denoise + MC-warp + lookahead), ADR-0123/0124/0125 (anime tune),
+  ADR-0127 (deblock), ADR-0133 (coarse banding), ADR-0135/0132 (RC-competing negatives).
+- User direction: build a per-content tuner because the gains are per-content and mostly a
+  matter of finding the right tuned boost for each content type; re-check apparent negatives
+  for wrong-params false-negatives; and investigate encoder-side integration patches.
@@ -54,3 +54,4 @@ sibling keep vmafx's number (e.g. 0100, 0108) for an easy cross-walk.
 | [0139](0139-dehalo-shared-mem-tile.md) | Shared-memory tiling of the dehalo box-blur window (opt-in `tile`, default off): box_blur is fetch-bound (~1445 loads/px from a 5×-called 17×17 window, trivial ALU — an ALU-strip cut Arc rtime −70%), so the ADR-0134 denoise-tiling clone applies. Bit-identical (SSIM 1.0). Validated −38% (1.6×) on Arc A380, neutral on the cache-rich 4090 → default off. Twin of the refuted aa tiling (ALU-bound) | Accepted |
 | [0140](0140-aa-sobel-mag-hoist.md) | Hoist the redundant aa sobel-mag into shared memory (opt-in `fast`, default off). aa is ALU-bound (premise-check: stripping sobel collapsed rtime −86%; tiling refuted), and `sobel_mag` is recomputed ~1156×/px across overlapping emask windows. fast=1 computes each cell's sobel once into shared mem; emask reduces from the cache. Bit-identical (SSIM 1.0). Validated 12.6× (−92%) on Arc, 2.6× (−62%) on the 4090 — an ALU win, so it helps every GPU. ADR-0134 idiom applied to a computed result | Accepted |
 | [0141](0141-deband-wide-band-reach.md) | Wide-band reach for deband (multi-radius / extended `range`) to span bands wider than `2·range`. Designed (5-agent panel) + built behind opt-in `reach` + measured: the pre-encode CAMBI gain (v0.15) is real (larger range monotonic 4.30→3.55, detail-safe SSIM ~0.998) but **inverts post-encode** — wider range is monotonically *worse* after `hevc_nvenc` at every quality (cq10/18/28), because the encoder re-bands the over-smoothed gentle ramp into wider bands. Completes v0.15's deferred durability gate with a negative. Code reverted | Rejected |
+| [0142](0142-tune-auto-content-router.md) | `tune=auto` content-adaptive pre-encode router (design) — the session's law (reductive wins / additive washes / content-dependent / mostly-tuning) means there's no single best filter but a tuned reductive boost per content class, routed from analyze.* per-shot. Ships the foundational detection enabler: `grain_estimate` now emits `lavfi.pelorus.grain_sigma`/`grain_flat` (ADR-0136 pattern, no ABI/shader change), unblocking the proven grainy route. Routing table + validation priority + the encoder-patch parallel track recorded | Proposed |
@@ -135,3 +135,24 @@ SEI bitstream filter ([ADR-0117](../adr/0117-grain-fgs-bsf.md), static model),
 and `av1_nvenc` via `-pelorus_film_grain` into NVENC's hardware AV1 film-grain
 synthesis ([ADR-0118](../adr/0118-nvenc-av1-filmgrain.md)). The H.264 and VVC
 legs and a per-frame H.274 model remain deferred.
+
+## Frame metadata (the `tune=auto` grain discriminator)
+
+Alongside the side-data, the filter emits two per-frame scalars as `lavfi.pelorus.*`
+frame metadata (the [ADR-0136](../adr/0136-analyze-frame-metadata.md) `av_dict_set`
+pattern — no interop ABI or shader change), so the content-adaptive router
+([ADR-0142](../adr/0142-tune-auto-content-router.md)) can read the grain estimate
+without parsing the `PEL_SEC_FILMGRAIN` blob:
+
+| key | meaning |
+|---|---|
+| `lavfi.pelorus.grain_sigma` | peak per-band RMS residual (grain stddev) over the populated intensity bands, normalized `[0,1]`. Measured **only** on edge-gated locally-flat pixels, so real structure is excluded by construction — what survives is grain. The router's grain discriminator (≈ `<0.004` clean · `0.012–0.05` moderate · `>0.05` heavy). |
+| `lavfi.pelorus.grain_flat` | fraction of the frame the estimate was measured over (flat pixels / total). The estimate's confidence — heavy-edge frames give a small flat fraction and an unreliable sigma. |
+
+Read them with `metadata=print` or `ffprobe -show_frames`:
+
+```bash
+ffmpeg -init_hw_device vulkan=vk:0 -i in.mkv \
+  -vf "hwupload,pelorus_grain_estimate_vulkan,hwdownload,format=yuv420p,metadata=print:key=lavfi.pelorus.grain_sigma" \
+  -f null -
+```
@@ -1,7 +1,7 @@
 From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
 From: Lusoris <lusoris@pm.me>
-Date: Fri, 19 Jun 2026 19:50:06 +0200
-Subject: [PATCH 06/11] feat(grain): add vf_pelorus_grain_estimate_vulkan (FGS
+Date: Sat, 27 Jun 2026 21:26:02 +0200
+Subject: [PATCH 06/18] feat(grain): add vf_pelorus_grain_estimate_vulkan (FGS
  param estimator)
 MIME-Version: 1.0
 Content-Type: text/plain; charset=UTF-8
@@ -43,8 +43,8 @@ ADR: docs/adr/0115-grain-estimate.md
  configure                                     |   2 +
  libavfilter/Makefile                          |   1 +
  libavfilter/allfilters.c                      |   1 +
- .../vf_pelorus_grain_estimate_vulkan.c        | 654 ++++++++++++++++++
- 4 files changed, 658 insertions(+)
+ .../vf_pelorus_grain_estimate_vulkan.c        | 682 ++++++++++++++++++
+ 4 files changed, 686 insertions(+)
  create mode 100644 libavfilter/vf_pelorus_grain_estimate_vulkan.c
 
 diff --git a/configure b/configure
@@ -93,10 +93,10 @@ index 8f49f54808..5a5f777074 100644
  extern const FFFilter ff_vf_phase;
 diff --git a/libavfilter/vf_pelorus_grain_estimate_vulkan.c b/libavfilter/vf_pelorus_grain_estimate_vulkan.c
 new file mode 100644
-index 0000000000..3d80ab0f44
+index 0000000000..68891c58ab
 --- /dev/null
 +++ b/libavfilter/vf_pelorus_grain_estimate_vulkan.c
-@@ -0,0 +1,654 @@
+@@ -0,0 +1,682 @@
 +/*
 + * Copyright 2026 Lusoris
 + *
@@ -356,6 +356,16 @@ index 0000000000..3d80ab0f44
 +    uint64_t flat_total; /* total flat pixels measured                          */
 +} PelorusGrainStats;
 +
++/* Emit a scalar as lavfi.pelorus.* frame metadata (ADR-0136 pattern), so the
++ * tune=auto content router can read the grain estimate without parsing the
++ * PEL_SEC_FILMGRAIN side-data blob. */
++static void pel_set_meta_f(AVFrame *frame, const char *key, float v)
++{
++    char buf[32];
++    snprintf(buf, sizeof(buf), "%.6f", v);
++    av_dict_set(&frame->metadata, key, buf, 0);
++}
++
 +static void reduce_stats(const PelorusGrainBuf *acc, PelorusGrainStats *st)
 +{
 +    double corr_sum = 0.0, var_sum = 0.0;
@@ -663,6 +673,24 @@ index 0000000000..3d80ab0f44
 +
 +    reduce_stats(acc, &st);
 +
++    /* Grain discriminator as frame metadata for the tune=auto router: peak grain
++     * sigma over populated bands (the residual is measured ONLY on edge-gated
++     * locally-flat pixels, so structure is excluded by construction and what
++     * survives is grain stddev), plus the flat fraction it was measured over
++     * (the estimate's confidence). No interop ABI / shader change; the
++     * PEL_SEC_FILMGRAIN side-data path is untouched. */
++    {
++        float gs = 0.0f;
++        int b;
++        for (b = 0; b < PEL_GRAIN_BANDS; b++)
++            if (st.band_cnt[b] > 0 && st.band_rms[b] > gs)
++                gs = st.band_rms[b];
++        pel_set_meta_f(in, "lavfi.pelorus.grain_sigma", gs);
++        pel_set_meta_f(in, "lavfi.pelorus.grain_flat",
++                       (float)((double)st.flat_total /
++                               ((double)in->width * (double)in->height)));
++    }
++
 +    memset(&g, 0, sizeof(g));
 +    map_aom(s, &st, &g);
 +
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		- `vf_pelorus_grain_estimate_vulkan` now emits `lavfi.pelorus.grain_sigma` and `lavfi.pelorus.grain_flat` as per-frame metadata (the [ADR-0136](docs/adr/0136-analyze-frame-metadata.md) `av_dict_set` pattern — no interop ABI or shader change, the value was already computed). `grain_sigma` is the peak per-band RMS residual measured over edge-gated locally-flat pixels (structure excluded → what survives is grain stddev); `grain_flat` is the flat fraction it was measured over (confidence). This is the first detection enabler for the `tune=auto` content-adaptive router ([ADR-0142](docs/adr/0142-tune-auto-content-router.md)) — it unblocks routing grainy content (the proven −34% denoise lever) without parsing the `PEL_SEC_FILMGRAIN` side-data blob. Verified discriminating: heavy-grain 0.019 vs clean-ish Bluray 0.012. Docs: `docs/metrics/grain_estimate.md`.