docs(bench): v0.17 — SSIMULACRA2 metric + tuned NVENC settings close most of the GPU->CPU gap#45
Merged
Merged
Conversation
…close most of the GPU->CPU gap The session's payoff + a methodology correction. Pelorus' goal is smaller + higher-quality GPU encodes approaching CPU at GPU speed; VMAF/VMAF-NEG hid the signal. VMAF-NEG zeroes the enhancement credit (it was the filter-lane anti-gaming guard) and saturates, so it was blind to the biggest lever: tuned NVENC settings (p7 + spatial/temporal-AQ + lookahead + multipass + b_ref) beat the conservative defaults by ~+5..+10 SS2 at iso-bitrate (the default is garbage), invisible to VMAF (+/-0.5). SSIMULACRA2 (no saturation, robust to sharpening) reveals it AND correctly refutes additive sharpen. With tuned settings the real GPU->CPU gap (clean, CQ-locked) is ~2 SS2 (tuned hevc_nvenc vs x265-slow; av1_nvenc vs SVT-AV1 similar) -- down from the garbage- default's ~16; pushing AQ/lookahead harder doesn't help (lever tapped). That residual is what the Pelorus Vulkan filters close on top, at GPU speed (NVENC is far faster than x265-slow). Adopts SS2-primary + VMAF-secondary + CQ-locked ladders as the methodology; retires VMAF-NEG. Docs-only; feeds tune=auto (ADR-0142). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Records the session's payoff + a methodology correction. Pelorus' mission: smaller + higher-quality GPU encodes approaching CPU quality, at GPU speed. We were measuring it with the wrong metric.
The findings
SSIMULACRA2 is the primary metric; VMAF-NEG is retired. NEG zeroes the enhancement credit (it was the anti-gaming guard for the pre-encode filter lane) and VMAF saturates at 100 on clean content — together they hid the biggest lever. On the same encodes where NEG said "no gain," SS2 (no saturation, robust to sharpening-gaming) shows large differences. Set: SS2 primary + regular VMAF secondary (VMAF-up/SS2-down = gaming flag).
The NVENC default is garbage; tuned settings are the big free lever.
p7 -tune hq -spatial-aq 1 -temporal-aq 1 -aq-strength 8 -rc-lookahead 32 -multipass fullres -b_ref_mode middlebeats default by +5..+10 SS2 at iso-bitrate — invisible to VMAF (+/-0.5; it can't see the flat-region AQ win). Pushing AQ15/lookahead-53 doesn't help (lever tapped).The real GPU->CPU gap is small (~2 SS2) — clean, CQ-locked: tuned
hevc_nvencvsx265 -preset slow, andav1_nvencvsSVT-AV1, both ~2-2.5 SS2. The earlier "huge gap" was the garbage default + noisy content + VBR overshoot. The residual is what the Pelorus Vulkan filters close on top, at GPU speed (NVENC >> x265-slow, so it stays faster than CPU).Methodology (benchmarking.md)
CQ-locked ladders (VBR overshoots ~50% with multipass), SS2 primary, VMAF secondary, NEG retired, CPU encode as the gold-standard gap reference.
Deliverables
bench-results v0.17 + benchmarking.md + changelog. Docs-only; feeds tune=auto (ADR-0142).
🤖 Generated with Claude Code