Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 82 additions & 0 deletions docs/adr/1122-vmaf-v1-model-port.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
<!-- markdownlint-disable MD013 MD041 MD060 -->

# ADR-1122: Adopt and port VMAF v1 models (opt-in, v0.6.1 stays default)

- **Status**: Proposed
- **Date**: 2026-07-01
- **Deciders**: Lusoris Dev (dsp@mvdnet.org)
- **Tags**: `vmaf`, `models`, `upstream-port`, `cambi`, `chroma`, `sycl`, `golden-gate`

## Context

Netflix shipped **VMAF v1** in libvmaf `v3.2.0` (2026-06-20) — a new generation
of models (`model/vmaf_v1.0.16/`). It is still a libsvm ν-SVR fusion, but the
**feature set changed**: VIF is removed, and CAMBI (banding) plus a chroma
feature (`speed_chroma_uv`) become core features; ADM/motion move to the
`adm3`/`motion3` variants. v1 adds viewing-condition-calibrated models (1080p@3H,
phone@5H, 4K@1.5H, 4K@3H), an `_hfr` high-frame-rate variant, a **[0, 110]**
score range for 4K@3H, CAMBI encode-side parameters, and a recommendation to
measure at **10-bit** even for 8-bit SDR. Details and citations are in the
[research digest](../research/2026-07-01-vmaf-v1-models.md).

The fork tracks Netflix/vmaf and already implements almost all v1 features on
GPU/SIMD (`adm3`, `motion3`, CAMBI, `speed_chroma`). Two of them — `cambi_sycl`
and `speed_chroma_sycl` — currently **fail on SYCL** (CAMBI parity failure;
speed_chroma needs fp64 and SIGABRTs on Arc A380). Under v1 these are no longer
optional: they are evaluated for **every** score. The fork must decide how far
to go, and whether v1 replaces or coexists with v0.

## Decision

We will **adopt VMAF v1 as a supported, opt-in model generation, keeping
`vmaf_v0.6.1` as the default.** Concretely:

1. **Port and build in the six v1 JSONs** (`vmaf_v1.0.16_{3d0h,5d0h,1d5h_2160,
3d0h_2160}` + the two `_hfr` variants) via `/add-model`, matching upstream's
built-in table. Selectable via `--model version=vmaf_v1.0.16_3d0h` etc.
2. **Fix the SYCL core features v1 depends on** — `cambi_sycl` parity and
`speed_chroma_sycl` (fp64-free per ADR-0220) — as a hard prerequisite, since
both are core v1 features on every backend.
3. **Support the [0, 110] range** end-to-end (CLI, `--precision`, the FFmpeg
`libvmaf*` filters and `vf_libvmaf` patches) — no clamp-to-100.
4. **Plumb the CAMBI encode-side params** (`cambi.enc_width/enc_height/
enc_bitdepth`) through the model-option parser and the FFmpeg filter AVOptions.
5. **Add v1 correctness tests against new v1 reference values** in *separate*
test files. The three Netflix v0.6.1 CPU golden pairs and their assertions
remain **untouched** (CLAUDE.md rule 1).
6. **Keep VIF** for v0 compatibility; the v1 dispatch simply does not register
it (off the v1 critical path).

The default model stays `vmaf_v0.6.1` for one release cycle to preserve the
golden gate and score-scale stability; v1 is opt-in until its GPU features and
tests are green across CPU/CUDA/SYCL/HIP.

## Alternatives considered

| Option | Pros | Cons | Why not chosen |
|---|---|---|---|
| **Port v1 opt-in, v0 stays default (chosen)** | Tracks upstream; no golden-gate/score-scale disruption; ships when GPU features are green | Two model generations to maintain for a while | Selected — correctness-preserving, incremental |
| Adopt v1 as the new default (replace v0.6.1) | Single generation; latest accuracy by default | Breaks the v0 golden gate; changes the score scale ([0,110]); silently shifts every downstream number | Rejected — violates the golden-data contract and surprises users |
| Don't port v1 (stay on v0) | No work | Falls behind Netflix; misses banding/chroma/phone accuracy; the fork's raison d'être is tracking upstream | Rejected — regressive |
| Port the JSONs only, defer SYCL CAMBI/chroma fixes | Fast to land models | v1 on SYCL would be broken (CAMBI/chroma are core, currently failing) — a shipped-but-broken backend | Rejected — v1 must work on the fork's flagship SYCL backend |

## Consequences

- **Positive**: the fork gains Netflix's latest, more-accurate, faster metric
(VIF removed), with banding/chroma awareness and viewing-condition models; the
10-bit recommendation aligns with the zero-copy P010 work (ADR-1121, PR #1081).
- **Negative**: two model generations coexist; the SYCL CAMBI/`speed_chroma`
fixes are now on the critical path (they were tolerated as failing because v0
doesn't fuse them). The `[0,110]` range touches user-visible output surfaces.
- **Neutral / follow-ups**: a `/sync-upstream` (or targeted `/port-upstream-commit`)
against `Netflix/vmaf v3.2.0` for the fusion/VIF-removal changes; new v1 golden
references; CAMBI enc-param AVOptions imply an ffmpeg-patch update (CLAUDE.md
rule 14); HDR v1 (Netflix "planned") is deferred/scaffolded.

## References

- req: user request (2026-07-01) — "исследуй новые модели v1 … заведи ADR на порт v1".
- Research digest: [docs/research/2026-07-01-vmaf-v1-models.md](../research/2026-07-01-vmaf-v1-models.md).
- Upstream: Netflix/vmaf `v3.2.0`; `resource/doc/models_v1.md`; `model/vmaf_v1.0.16/*.json`.
- Related: [ADR-1121](1121-sycl-qsv-zerocopy-p010-normalization.md) (10-bit zero-copy),
[ADR-0220](0220-sycl-fp64-fallback.md) (fp64-free SYCL kernels — blocks `speed_chroma_sycl`).
1 change: 1 addition & 0 deletions docs/adr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -966,6 +966,7 @@ ADRs may exist there for local session continuity, but the tracked
| [ADR-1118](1118-perceptual-sidedata-weighting.md) | Pelorus side-data perceptually re-weights VMAF **spatial pooling** (opt-in, golden-isolated). The vendored interop parser (ADR-1113) reads each frame's per-cell banding-risk / variance maps; a normalized `[0,1]` salience becomes a per-frame pooling weight `w = 1 + strength·salience`, turning pooled MEAN/HARMONIC_MEAN into their weighted forms (MIN/MAX unaffected). **Golden-gate isolation (load-bearing):** inert unless BOTH the `perceptual_weight` opt-in is set AND a valid Pelorus blob is present for the frame; otherwise `w ≡ 1.0` and the pooling runs the *literal* upstream expression, so the no-side-data path (and the Netflix golden pairs, which carry no side-data) is byte-identical — proven by `test_perceptual_weight.c`. New C-API `vmaf_set_perceptual_{weight_enabled,weight_strength,sidedata}` (`core/include/libvmaf/perceptual_weight.h`); weight module `core/src/feature/perceptual_weight.c`; `vf_libvmaf` reader + `perceptual_weight` AVOption (ffmpeg-patch 0017). R1–R6 compat: `min(known_size,dir.size)` reads, unknown bits ignored, `grid==0` degrades to frame-level scalar, ABI-major mismatch rejected (unweighted + log). CPU-only, no GPU. Alternatives (auxiliary-feature / both) rejected — maintainer chose spatial-pooling weighting. | Accepted | 2026-06-14 | scoring, pooling, pelorus, interop, ffmpeg, golden-gate, fork-local |
| [ADR-1119](1119-golusoris-go-framework-adoption.md) | Adopt the `github.com/golusoris/golusoris` fx framework (pinned `v0.4.0`) across all six vmafx Go binaries + `pkg/`. Every binary becomes an `fx.New(...).Run()` over golusoris modules (Core/otel/HTTP/grpc/k8s-operator/clikit), sharing a vmafx-local `internal/app/bootstrap` stanza (`Base` + `FxLogger`). RC-blocking, phased services-first (`vmafx-server` → `controller` → `node` → `operator`, then `mcp`/`tune` + `pkg` sweep). Keeps the `VMAFX_` config env-prefix via `fx.Replace(config.Options{...})` (not golusoris's default `APP_`) and keeps the controller's embedded SQLite queue (not `golusoris.Jobs`/Postgres). Dependency closures already align byte-for-byte, so the `go.mod` merge is low-risk (`go build ./...` clean post-`go get`). Four framework gaps filed upstream were all integrated by the maintainer (#225 gRPC interceptor injection, #226 version module, #227 operator SetLogger + webhook, #234 log reads `log.level` from config); the pin was bumped v0.3.1→v0.4.0. Only #226 + the `k8s/operator` module are in the v0.4.0 tag; #225/#227/#234 are merged to golusoris main but untagged, so the service binaries carry small interim shims (the #234 log-env bridge; the #227 operator SetLogger + webhook gate) and the controller (needs #225) follows the next golusoris tag. | Accepted | 2026-06-14 | go, framework, fx, golusoris, server, controller, node, operator, mcp, vmaf-tune, rc-blocking, fork-local |
| [ADR-1120](1120-pelorus-abi-minor3-resync-complexity.md) | Re-pin the vendored Pelorus interop ABI (ADR-1113) from `pelorus@835e097` (ABI 1.0) to `pelorus@818d844` (ABI 1.3) and consume the new `PEL_SEC_COMPLEXITY` per-frame section in perceptual weighting (ADR-1118). Minor-3 appends three sections (`QPREPORT` 1.1, `MOTION_CONF` 1.2, `COMPLEXITY` 1.3) and three files; the mirror gains `denoise.h`, `denoise_params.c`, `qp_report_csv.c` (the last is required for the minor-3 conformance fixture to link `pel_x265_csv_parse`). The `PelorusSideData` header layout is unchanged (append-only R1) and the parser keys on `abi_major`, so it is a deliberate re-pin, not a break. Fixes the sync-script `--update` bug that re-vendored the six manifest files but not the conformance-fixture body (the immediately-following drift check then failed). **Complexity modulation:** `perceptual_weight.c` attenuates the banding salience by `(1 − 0.5·complexity)` floored at `0.25` — banding is more visible on flat/simple frames and masked on busy/textured frames. **Golden-isolated (load-bearing):** engages only when the opt-in is set AND a complexity section is present; absent section → factor `1.0` → byte-identical pooling, so the Netflix golden 576×324 pair scores `76.667831` unchanged (verified). Proven by `test_complexity_modulates_weight` / `_grid_zero` (toggle-proven). New vendored files inherit the existing prefix-glob lint/format exclusions. CPU-only, no GPU. Alternatives (trim-fixture, threshold/step modulation, replace-banding) rejected. | Accepted | 2026-06-27 | interop, abi, vendoring, pelorus, scoring, pooling, golden-gate, build, fork-local |
| [ADR-1122](1122-vmaf-v1-model-port.md) | Adopt & port **VMAF v1** models (Netflix, libvmaf v3.2.0, June 2026) as an **opt-in** generation while **`vmaf_v0.6.1` stays default**. v1 is still a libsvm ν-SVR fusion but the feature set changed: **VIF removed**, **CAMBI (banding) + `speed_chroma` (chroma) promoted to core**, ADM/motion → `adm3`/`motion3`. Six built-in JSONs (`vmaf_v1.0.16_{3d0h,5d0h,1d5h_2160,3d0h_2160}` + two `_hfr`) for 1080p@3H / phone@5H / 4K@1.5H / 4K@3H (the last on a **[0, 110]** range). Netflix recommends **10-bit** measurement even for 8-bit SDR (banding) — aligns with the zero-copy P010 path (ADR-1121). Port scope: build in the JSONs (`/add-model`); **fix SYCL `cambi`/`speed_chroma`** (now core, currently failing on Arc A380 — fp64 SIGABRT / parity fail); handle `[0,110]` end-to-end (CLI/`--precision`/ffmpeg filters, no clamp-to-100); plumb CAMBI `enc_width/enc_height/enc_bitdepth` (ffmpeg-patch impact); add **new** v1 golden refs in separate tests (the 3 Netflix v0.6.1 golden pairs stay untouched). Alternatives (v1-as-default / don't-port / models-only) rejected — golden-gate + score-scale + broken-SYCL respectively. | Proposed | 2026-07-01 | vmaf, models, upstream-port, cambi, chroma, sycl, golden-gate, fork-local |
| [ADR-0452](0452-vif-scratch-buf-hoist-to-vifstate.md) | Hoist VIF 10-plane scratch buffer from per-frame allocation to VifState init/close lifecycle; eliminates ~79 MB/frame allocator traffic at 1080p. | Accepted | perf, vif, cpu, build, fork-local |
| [ADR-0460](0460-dispatch-registry-audit-2026-05-15.md) | Dispatch-strategy registry audit: deduplicate SYCL/Vulkan feature registry rows and align HIP/Metal dispatch-support tables with registered extractor names. | Accepted | dispatch, hip, metal, sycl, vulkan, correctness |
| [ADR-0539](0539-hip-ssimulacra2-blur-fp-contract-off.md) | Per-kernel `hip_cu_extra_flags` keeps SSIMULACRA2 recursive blur FP contraction disabled on HIP so parity stays inside the ADR-0214 gate. | Accepted | hip, build, ssimulacra2, numerics, fork-local |
Expand Down
1 change: 1 addition & 0 deletions docs/adr/_index_fragments/1122-vmaf-v1-model-port.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
| [ADR-1122](1122-vmaf-v1-model-port.md) | Adopt & port **VMAF v1** models (Netflix, libvmaf v3.2.0, June 2026) as an **opt-in** generation while **`vmaf_v0.6.1` stays default**. v1 is still a libsvm ν-SVR fusion but the feature set changed: **VIF removed**, **CAMBI (banding) + `speed_chroma` (chroma) promoted to core**, ADM/motion → `adm3`/`motion3`. Six built-in JSONs (`vmaf_v1.0.16_{3d0h,5d0h,1d5h_2160,3d0h_2160}` + two `_hfr`) for 1080p@3H / phone@5H / 4K@1.5H / 4K@3H (the last on a **[0, 110]** range). Netflix recommends **10-bit** measurement even for 8-bit SDR (banding) — aligns with the zero-copy P010 path (ADR-1121). Port scope: build in the JSONs (`/add-model`); **fix SYCL `cambi`/`speed_chroma`** (now core, currently failing on Arc A380 — fp64 SIGABRT / parity fail); handle `[0,110]` end-to-end (CLI/`--precision`/ffmpeg filters, no clamp-to-100); plumb CAMBI `enc_width/enc_height/enc_bitdepth` (ffmpeg-patch impact); add **new** v1 golden refs in separate tests (the 3 Netflix v0.6.1 golden pairs stay untouched). Alternatives (v1-as-default / don't-port / models-only) rejected — golden-gate + score-scale + broken-SYCL respectively. | Proposed | 2026-07-01 | vmaf, models, upstream-port, cambi, chroma, sycl, golden-gate, fork-local |
1 change: 1 addition & 0 deletions docs/adr/_index_fragments/_order.txt
Original file line number Diff line number Diff line change
Expand Up @@ -875,3 +875,4 @@
1118-perceptual-sidedata-weighting
1119-golusoris-go-framework-adoption
1120-pelorus-abi-minor3-resync-complexity
1122-vmaf-v1-model-port
Loading