L9a: the first-party NVFP4 DiT loads -- it was SWIZZLED and HIGH-nibble-first, not linear (#435) - #571
Merged
localai-bot merged 6 commits intoAug 13, 2026
Conversation
…rs disagree (#435) `DequantNvfp4ToBf16` is low-nibble-first, which is right for torchao and ModelOpt and wrong for Lightricks' `nvfp4-prequant` and for the H3 community converter. Read the wrong way round, every adjacent fp4 pair is transposed: finite, correctly shaped, and wrong. The header is shared with the MiniMax-H3, Laguna, DeepSeek-V4 and Qwen3-32B NVFP4 arms, so this is not LTX-2.5 loader work and does not ride inside that row. Spec committed before implementation. Records four things the implementation then has to honour: * the convention per producer, each with the line that packs or unpacks it (torchao `kernels.py:160`, vLLM `nvfp4_emulation_utils.py:321-324`, ltx-kernels `docs/NVFP4.md:27-29`); * that ltx-2-5.md section 4.2's open question is ANSWERED -- torchao is low-first on three independent witnesses, so the shipped text-encoder arm is correct and there is no defect to chase; * why H3's existing nibble-swap-at-load is NOT reused here, and the exact condition under which that decision must be revisited -- an fp4-resident LTX path, which a host-side dequant parameter could not fix; * a correlation gate against the vonkaiser FP8 DiT as an independent oracle, WITH a control arm, because a gate that cannot separate a correct read from a wrong one is section 7.0(c) over again. The parameter defaults to today's behaviour, so existing callers are untouched by construction; the proof obligation is that all ten NVFP4 gates keep their exact case and assertion counts, and those baselines are recorded here. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…le-first (#435) The file was refused, and the refusal carried a WRONG diagnosis: it said the group scale was stored LINEAR [N, K/16]. It is not. The bytes are SWIZZLED -- the permutation Ltx2UnswizzleNvfp4BlockScale already inverts -- merely declared in the cuBLAS-padded framing [round_up(N,128), round_up(G,4)] instead of the to_blocked [32*ceil(N/128), 16*ceil(G/4)] the loader knew. For every layer in that file N % 128 == 0 and G % 4 == 0, so the padded framing is NUMERICALLY IDENTICAL to the linear shape. That coincidence is why the wrong diagnosis looked confirmed, and why no shape test could ever have settled it. The file also packs element 2j in the HIGH nibble, which no shape encodes at all. Both facts came from correlating the dequantized weights against the vonkaiser FP8 DiT of the same base weights -- an oracle that is not ours -- and were then confirmed in Lightricks' own runtime. ltx-kernels/docs/NVFP4.md:27-29 hi_first=True is the default ltx-kernels/csrc/nvfp4/quantize.cu:26-31 swizzled_offset, padded framing ltx-core/quantization/nvfp4/linear.py:6-7 both statements together torchao kernels.py:160 low-first, the other convention vllm nvfp4_emulation_utils.py:321-324 vLLM reads low-first, agreeing What lands: * Nvfp4NibbleOrder on DequantNvfp4ToBf16, DEFAULTING to low-first, so the MiniMax-H3, Laguna, DeepSeek-V4 and Qwen3-32B callers are untouched by construction rather than by inspection; * Ltx2ResolveNvfp4Producer -- the torchao marker decides, the framing corroborates, and every other combination REFUSES BY NAME. torchao always writes that marker, so its absence excludes torchao; that inference is stated at the code site with the evidence behind it, because it is the one step here not read directly off the file; * a CORRELATION GATE against the FP8 checkpoint, with a CONTROL arm and a deliberately-wrong-order arm, both asserted to collapse. corr 0.995, rms 10.1%, control 0.004, wrong-order 0.005. It also asserts absmax is IDENTICAL under both nibble orders -- which is why "the output was finite and correctly scaled" was never evidence of anything here. The synthetic NVFP4 fixture declared the to_blocked framing with NO marker: a combination neither producer emits. The new discriminator refused it, correctly, and the fixture was what was wrong -- it now mirrors the shipped artifact. Ten NVFP4 gates re-run. Eight byte-identical to baseline; the two that rose are the two this change adds cases to (test_nvfp4_dequant 4/47 -> 5/69, test_ltx2_loader 20/2363 -> 23/2426). All nine LTX/video baselines unchanged. Also settles ltx-2-5.md section 4.2: torchao is low-first on three independent witnesses, so the shipped text-encoder arm is CORRECT and there is no defect to chase. Its gate is now source-anchored instead of comparing our helper to itself. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…435) The loader refuses in single quotes, and users grep the checkpoint with what those quotes contain. An apostrophe in ordinary prose opens a quote that closes on the next one, so the message starts naming a fragment of its own explanation instead of a tensor. The marker-present branch shipped one: "under the other's assumption" made the refusal quote 'transformer_blocks.0.attn1.to_q.weight_scale' ... under the other's with an ODD number of delimiters. The whole-file refusal path already asserted this invariant; the resolver's own two refusals did not, which is how it got in. The discriminator case now scans both of them: quote count must be EVEN, at least one pair, and every quoted span must equal the tensor the refusal is about. Reinstating the apostrophe as a mutation turns that case red on "single-quote count = 3", so the assertion is gated rather than asserted. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
… modules (#435) The correlation gate proves ONE module decodes correctly. It does not prove the discriminator resolves the same way for the other 1175, and "I sampled a few and they agreed" is precisely how the linear-layout diagnosis got written. So the shipped NVFP4 DiT's own header goes in as a fixture (7876 tensors, names and shapes only, no weight bytes) and every U8 weight in it is pushed through Ltx2ResolveNvfp4Producer. Measured: packed=1176 markers=0 weight_scale_2=1176 input_scale=1176 resolved kNvfp4Prequant=1176 padded framing=1176 to_blocked framing=0 ambiguous_with_linear=1176 The last number is the one worth keeping. For EVERY module in this file the cuBLAS-padded framing is numerically identical to the linear [N, K/16] shape, so a shape test could not have separated swizzled from linear for a single one of them. That is now a counted, gated fact instead of an argument in a comment. markers=0 is the other load-bearing one: torchao always writes its sidecar, so zero of them across 7876 tensors is what excludes torchao and licenses the marker-less arm. input_scale=1176 records the export as W4A4-shaped, whose activation scales this weight-only path discards exactly as vLLM's own W4A16 method does (modelopt.py:1264-1268). test_ltx2_loader 23/2432 -> 24/4793. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…ejected (#435) Records what neither the code nor git does: the four-way correlation table, the three designs rejected and why, why the default is low-first, and the GB10 run where BOTH shipped DiTs stage and forward. The FP8 arm's absmax reproduces the previously recorded GB10 figures to the digit. That is what makes the NVFP4 numbers beside it evidence rather than merely new -- the harness is shown to be the same one that produced the accepted result. Also records what was NOT obtained: the device case's MESSAGE text was cut by a tail in the run harness, and the capture re-run timed out on the GPU lock (FLOCK_EXIT=1) against a box saturated by two other rows. The facts it would have printed are asserted among the 5626 passing assertions and gated from the committed manifest, so this is cosmetic -- said plainly rather than quietly omitted. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
…d the sweep proving nothing (#435, #598) Six review findings on the L9a nibble-order change. The implementation is sound and the correctness argument holds; every one of these is an accuracy or coverage defect in what the change CLAIMS. F1 -- a safety guarantee that is false, stated to users. Three places said any other marker/shape combination is "refused by name". It is not. A marker-less NVFP4 checkpoint whose weight_scale is stored LINEAR [N, K/16] has, for every geometry with N % 128 == 0 and G % 4 == 0, a shape numerically IDENTICAL to the cuBLAS-padded framing. Ltx2ResolveNvfp4Producer returns kNvfp4Prequant for it, the loader then unswizzles scales that were never swizzled AND reads high-first, and the result is finite, correctly shaped, correctly scaled, WRONG. The refusal branch is unreachable for that entire class. That is not an exotic file. LINEAR [N, K/16] is what ModelOpt, llm-compressor and compressed-tensors all write -- vLLM's own readers allocate exactly that shape at modelopt.py:1335-1345 and compressed_tensors/schemes/compressed_tensors_w4a4_nvfp4.py:73-76 (pin 555967922) -- and none of the three emits a .torchao_nvfp4 sidecar. So the marker's absence excludes torchao and NOTHING ELSE. The decision is still right, because there is no better evidence in the file, and that was checked rather than assumed: the shipped DiT's __metadata__ carries exactly config, gemma_source_checkpoint, model_version and license -- no quantization_config, no producer key, no nvfp4/torchao/quant substring anywhere in the config, and no tensor name mentioning the quantizer over all 7876 tensors. Unlike MiniMax-H3, whose community checkpoint DID name its converter, there is nothing here to key on. So the gate is NOT weakened; the sentence is corrected, in ltx2_loader.h, docs/USAGE.md and the spec's risk table, and the residual hazard becomes a tracked condition at spec section 3.1.1 beside the existing H3 reversal condition. F2 -- the ten-gate "byte-identical" evidence is half vacuous, and two live gates are blind. Instrumented DequantNvfp4ToBf16 with a call counter and re-ran mutation M3 (flip the shared default to kHighFirst) across all ten: RED test_nvfp4_dequant 6 calls, test_gguf_nvfp4 6, test_ltx2_loader 294 GREEN test_qwen3_forward 30 calls, test_minimax_h3 44 calls / 57,395 asserts GREEN the other five make ZERO calls on a default run Three of ten can see a nibble-order change. Five never execute the function, so their identical counts are evidence of nothing about it. Two execute it live and still pass -- and one of those is the project's OTHER high-first family, gated by exactly the statistic this spec proves is blind. If H3 is ever routed through the new kHighFirst while MiniMaxH3Nvfp4HighNibbleFirst() stays default-ON, the two compose into a double flip and nothing in the tree fires. Opened #598 for a nibble-sensitive H3 gate, linked from the roadmap issue table and the spec. F3 -- a recorded number the gate does not produce. Section 7 recorded the wrong-order arm at 0.00514, which matches no row of its own table and no line of the gate. The committed gate reports corr -0.00239115 / rel 1.41856. The control on the same sentence, 0.00362, was exact. Section 5.1 also named "the loader refuses the shape" as THE red-before for a change with two independent halves; a throw can only ever demonstrate the layout half, so both reds are now stated separately with the value each produces. F4 -- a comment contradicted by four call sites. nvfp4_dequant.h said the order is "never inferred and never defaulted per call site". It IS defaulted, and deliberately: minimax_h3_nvfp4.cpp:112, minimax_h3_device.cpp:1311, qwen3_5.cpp:1298 and dense_nvfp4_gemm.h all rely on it. The seam that genuinely has no default is Ltx2DequantNvfp4ToBf16. Comment corrected to say which is which and why. F5 -- the correlation tolerance was loose in one direction and unbounded in the other, RE-BOUNDED strictly stronger. rel_rms is a PREDICTED QUANTITY (the disagreement two different quantizations of the same base weights must show, measured 0.100672), not an error budget. `rel <= 0.15` alone admitted a +10% uniform group-scale error, and Pearson correlation is scale-INVARIANT -- corr reads 0.994968 to every printed digit for every multiplier -- so nothing else in the gate could catch it. It also had no floor, so an arm that reproduced the FP8 oracle exactly scored rel 0.0 / corr 1.0 and passed every correlation assertion. Replaced by a band: 0.085 <= rel <= 0.115, plus corr <= 0.998. Proven strictly stronger by A/B on the SAME binary path, only the bound differing: x1.09 group-scale error inside DequantNvfp4ToBf16, rel 0.1411 old `rel <= 0.15` GREEN 45/45 Status: SUCCESS! band RED CHECK( 0.1411 <= 0.115 ) is NOT correct! the FP8 oracle handed to the gate as its own answer old bounds every CORRELATION assertion passes; only the incidental absmax equality fires band RED on the result itself: rel >= 0.085 AND corr <= 0.998 Both mutations are now permanent arms of the gate rather than one-off logs, in the same style as the wrong-nibble arm: the gate asserts that a x1.10 scale error moves corr by less than 1e-6 (measured 7.35e-11) while pushing rel out of the band, and that the oracle-against-itself lands outside it too. F6 -- record the limit honestly. Section 1.1 claimed "three mutually consistent and independent witnesses". torchao is neither installed nor vendored on this box (import fails; no mx_formats/kernels.py on the filesystem), so its two lines are upstream source only, not executable here. And vLLM's break_fp4_bytes is reached only from dequantize_to_dtype, which serves the ModelOpt and compressed-tensors paths -- vLLM's torchao path (torchao.py:290-318) delegates to torchao.quantization.quantize_ and never calls it. So vLLM witnesses ModelOpt, not torchao's writer. Section 1.1 now tabulates what each witness proves and how it was verified. The default is unchanged; nothing moves either way. Gates (Release, CUDA=OFF, BUILD_EXIT=0, no ENOSPC/BFD in any build log, df 84% throughout). ctest -N 409, full ctest 409/409 passed, 0 failed, 1 skipped (test_voxtral_e2e). The ten NVFP4 gates, nine unchanged and one moved: test_nvfp4_dequant 5 / 69 test_ops_nvfp4_matmul 4 / 1 test_gguf_nvfp4 14 / 2352 test_ops_moe_grouped 6 / 3 test_qwen36_weights 7 / 45 test_ops_nvfp4_fp4 22 / 919 test_qwen3_forward 7 / 1557 test_ltx2_device 13 / 498 test_minimax_h3 79 / 57395 test_ltx2_loader 24 / 4809 test_ltx2_loader 4793 -> 4809 is the +16 from the re-banding and its two built-in mutation arms; it is the suite this change adds assertions to. Tree restored byte-for-byte after every mutation. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
localai-bot
pushed a commit
that referenced
this pull request
Aug 13, 2026
…survive measurement (#435) Eight findings from the L10 fresh review. Six close as written; two close differently, because the thing they prescribed does not work and the oracle says so. Both are recorded here rather than quietly worked around, since an implementer who accepts a prescription that fails is the failure this protocol exists to catch. ──────────────────────────────────────────────────────────────────────────────── F3 — THE PRESCRIBED FIX IS REFUTED; THE ROPE TABLE IS THE INSTRUMENT The finding is real: `g.rope_partial_full = 1.0` (full rotary on the full- attention layers instead of the config's 0.25) left the suite green at 23 cases / 3583 assertions. Reproduced here before touching anything. The prescription was "a fixture whose global_head_dim/seq keep the rope contribution above the floor". MEASURED against the oracle, per state, as signal = max|bf16 @0.25 - bf16 @1.00| over floor = max|f32 - bf16|: committed head_dim 8/16 seq 8 worst signal/floor 1.095 wider full head head_dim 16/32 seq 8 0.258 longer seq head_dim 8/16 seq 24 0.764 wider + longer head_dim 16/32 seq 24 0.523 wider + longer head_dim 16/32 seq 32 0.650 Enlarging the fixture makes it WORSE. bf16 accumulation noise grows at least as fast as the rope contribution, so no reachable fixture size separates them: the hidden states are the wrong instrument, not a badly sized one. The right instrument is the table. `BuildProportionalRopeCache`'s host computation is split out and exported as `Gemma4ProportionalRopeCosSin`, the generator emits the oracle's own `Gemma4UnifiedTextRotaryEmbedding` cos|sin for the full-attention layer type (section 6), and the new case compares them in f32 with nothing accumulating. Measured agreement 1.267e-07 against a 1e-6 bound, plus an EXACT structural check that pairs at and beyond `int(partial*head_dim//2)` are cos=1, sin=0 and that the pairs below it really rotate. RED under the mutation: 202 assertions fail, worst 0.56533 vs 1e-6, exit 1 GREEN restored: 26 cases / 4115 assertions, exit 0 ──────────────────────────────────────────────────────────────────────────────── F2 — THE VALUE GATE AND THE LOADER, IN CI; AND M6 IS NOT WHAT IT LOOKED LIKE `Ltx2EncodePromptToConditioning`, `Ltx2LoadGemmaTowerFromSafetensors` and all four documented refusals were reachable from exactly ONE place — the opt-in 24 GB case — so none had CI coverage. They do now. The reduced tower fixture, which is already held to a running oracle, is written out as a real .safetensors under the CHECKPOINT's tensor names, loaded back, and compared to the fixture byte for byte; then a tokenizer whose added tokens spell the oracle's own token ids drives the whole prompt path and its conditioning is held to the committed LEFT-PADDED oracle run at a floor propagated through the identical projection. concat order q,k,v -> q,v,k RED 14 assertions, exit 1 PLE refusal disabled RED 1 assertion, exit 1 missing-v_proj refusal disabled RED 1 assertion, exit 1 module dtype F16 read as bf16 RED 1 assertion, exit 1 norm vector F16 read as bf16 RED 1 assertion, exit 1 conditioning at 0.565x / 0.688x of its propagated floor Now the part that did not survive. The review called `positions[i] = i` "the exact defect the surrounding comment warns about". It is not a defect at all. Rotary embedding is RELATIVE and the pads are masked out of attention, so shifting every position by the pad count cancels. MEASURED in the oracle, same 8 tokens told 12..19 and told 0..7: f32 max|diff| 5.11e-05 over max|value| 14.35 3.6e-06 relative bf16 max|diff| 1.09 0.65-1.70x the dtype floor f32 round-off, and at bf16 the rounding of different absolute angles. So the comment at ltx2_text_encoder.cpp — "a port that renumbers from zero rotates every query by the wrong angle" — was an overclaim, and so was its twin in the left-pad case. Both now carry the measurement. The numbering is still mirrored, because transformers derives positions from `cache_position` and fidelity is the reason; arithmetic is not. The new case does red under the renumbering, at 1.10x the audio floor, and says in its own comment that the narrow margin is a property of the defect rather than of the gate. ──────────────────────────────────────────────────────────────────────────────── F4 — THE GENERATOR'S LEGS WERE COUPLED, AND NOT WHERE IT LOOKED `run_tower` did `inner.to(dtype)`, which converts IN PLACE, so every leg after the bf16 one ran downstream of it. Each leg now runs on a `copy.deepcopy`. The mechanism is narrower than "bf16-rounded weights": MEASURED on this transformers build, the parameters round-trip unchanged, and it is the rotary embedding's non-persistent `inv_freq` BUFFERS that do not — 9.36e-05 on the full-attention table, 9.77e-05 on the sliding one. Regenerating moved 12 of the 13 padded f32 states (state 12 by 1.03e-02 on values of 2.705) and every entry of the pad-equivalence vector, while sections 2, 3 and 3b are byte-identical — which is exactly the partition the in-place conversion predicts. The header's claims about sections 4 and 5 are corrected to match. ──────────────────────────────────────────────────────────────────────────────── F5, F7, F8, F9 F5 337 -> 329, and pinned exactly rather than `> 300`. The count is arithmetic and now says so: 40*7 + 8*6 + 1. A lower bound is satisfied by a loader that quietly took a third of the layers from a bf16 fallback. F7 The rationale was inverted. Upstream calls `self.tokenizer(text, ...)` — `__call__` with its default `add_special_tokens=True` — so upstream DOES run the post-processor and we do not. Identical on this checkpoint because its `special_tokens` map is empty, measured. "THE TWO REFERENCES DISAGREE" also overstated it: both let the post-processor run, and only the explicit BOS prepend differs. Both corrected. F8 `kv.dtype` on the production prompt path was f32 where upstream resolves bf16 and where the attention itself runs bf16. Narrowed, not annotated. It was never wrong — it was WIDER, which is why nothing could see it: with the cache back at f32 the new conditioning gate reproduces 0.0617859 / 0.0394362 byte for byte. Half the cache bytes, and it stops taking `Gemma4AttnBlock`'s two-cast-buffers-plus-CastF32 arm on every layer. F9 The dropped `docs/FEATURES.md` edit is reapplied onto the campaign's rewritten row, truthfully: "Text tower RUNS" alongside the `encoder_path` refusal, which another change owns and this one does not touch. Row 533 chars, cells 60/28/217/215. `docs/USAGE.md` is owed too, and the debt turned out to be real rather than procedural: line 372 still said "the Gemma-4 12B text tower is not ported", which is false and which the paragraph 85 lines below it already contradicted. One document, two answers about one model. Corrected to state that the tower runs and to defer to that paragraph for what still blocks a prompt. The `encoder_path` passage itself is untouched — it belongs to the change that owns F1. ──────────────────────────────────────────────────────────────────────────────── NOT CLOSED HERE, AND OWED F1 and F6 are out of scope by instruction — another implementer owns whether the `encoder_path` refusal can be lifted now that L9c landed the connector weights. The spec is operator-owned, so this reports rather than edits it. Two things it needs. First, §4.2's open question stands and should not be narrowed by anything above: the shipped NVFP4 tower's output is still never compared to an independent oracle, so "all 49 hidden states within the oracle's bf16 noise floor" is proven for the SYNTHETIC reduced tower with bf16 weights and not for the quantized one that ships. PR #571 built the independent-oracle correlation technique that would close it. Second, the position-numbering claim appears in the spec in the same overclaiming form corrected here. ──────────────────────────────────────────────────────────────────────────────── GATE cmake -S . -B bld -G Ninja -DCMAKE_BUILD_TYPE=Release -DVLLM_CPP_BUILD_EXAMPLES=OFF cmake --build bld -j 12 BUILD_EXIT=0, no ENOSPC/BFD, df 88% ctest -N 409 tests ctest -j 8 --output-on-failure 409/409 passed, exit 0 ./bld/tests/test_ltx2_text_encoder 26 cases / 4115 assertions, exit 0 (baseline was 23 / 3583) `scripts/agent-preflight.sh` fails `doc-checkpoint range` on commits b0aa475, d67f812 and aa6aa0e. All three are ancestors of the branch head this work started from and none is touched here. Issue: #435 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the L9a gap in #435.
Also opens #598 (H3's NVFP4 gate is nibble-blind).
Specs:
.agents/specs/nvfp4-nibble-order.md(new, committed BEFORE implementation),.agents/specs/ltx-2-5.md§4.1/§4.2.The premise was wrong, and measuring it first is why nothing shipped broken
L9a was briefed to add a LINEAR
[N, K/16]scale arm. The file is not linear. It isSWIZZLED — the permutation
Ltx2UnswizzleNvfp4BlockScalealready inverts — merely declared inthe cuBLAS-padded framing instead of torchao's
to_blockedone. For every layer in that fileN % 128 == 0andG % 4 == 0, so the padded framing is numerically identical to thelinear shape. That coincidence is why the wrong diagnosis looked confirmed, and why no shape
test could ever have settled it. It also packs element
2jin the HIGH nibble.Building the briefed arm would have made the file load and render silently wrong. L9a returned
NEEDS_DECISIONinstead; the operator retracted the diagnosis and ratified the design here.How it was established
The
vonkaiserFP8 DiT quantizes the same base weights, so it is an oracle that is not ours:Control: the same read against four other modules' FP8 weights gives 0.021, 0.005, 0.002,
0.001. Confirmed independently in Lightricks' own runtime (
ltx-kernels/docs/NVFP4.md:27-29,csrc/nvfp4/quantize.cu:26-31,ltx-core/quantization/nvfp4/linear.py:6-7).What lands
Nvfp4NibbleOrderonDequantNvfp4ToBf16, defaulting to low-first, so the H3, Laguna,DeepSeek-V4 and Qwen3-32B callers are untouched by construction.
Ltx2ResolveNvfp4Producer: thetorchao_nvfp4marker decides, the framing corroborates,and every combination the shape can separate refuses by name. torchao always writes that
marker, so its absence excludes torchao — an inference stated at the code site with its
evidence, and with the limit of that refusal stated beside it (see F1 below).
deliberately-wrong-order arm, and two scale/oracle-integrity arms, all asserted to collapse.
It also asserts absmax is identical under both nibble orders, which is why "finite and
correctly scaled" was never evidence here.
markers=0,resolved kNvfp4Prequant=1176,ambiguous_with_linear=1176.Review repair (fresh review returned FAIL; six findings, all closed)
F1 — a "refuses by name" guarantee that was FALSE, stated to users. A marker-less NVFP4
checkpoint whose
weight_scaleis stored LINEAR[N, K/16]is shape-identical to the paddedframing for every geometry with
N % 128 == 0andG % 4 == 0, so it RESOLVES asnvfp4-prequantand is read swizzled and high-first: finite, correctly shaped, correctlyscaled, wrong. That is the normal ModelOpt / llm-compressor / compressed-tensors layout —
vLLM allocates exactly it at
modelopt.py:1335-1345andcompressed_tensors_w4a4_nvfp4.py:73-76— and none of the three emits a.torchao_nvfp4sidecar, so the marker's absence excludes torchao and nothing else.
The decision is still right: the shipped DiT's
__metadata__carries onlyconfig,gemma_source_checkpoint,model_version,license— noquantization_config, no producerkey, no quantizer-named tensor over all 7876 — so there is nothing better to key on. The
gate was NOT weakened; the sentence was corrected in
ltx2_loader.h,docs/USAGE.mdand thespec's risk table, and the residual hazard is now a tracked condition at spec §3.1.1 beside the
existing H3 reversal condition.
F2 — the ten-gate "byte-identical" evidence was half vacuous. Instrumented
DequantNvfp4ToBf16with a call counter and re-ran mutation M3 (flip the shared default tokHighFirst) across all ten gates:test_nvfp4_dequanttest_gguf_nvfp4test_ltx2_loadertest_qwen3_forwardtest_minimax_h3Three of ten can see a nibble-order change. If H3 is ever routed through the new
kHighFirstwhile
MiniMaxH3Nvfp4HighNibbleFirst()stays default-ON, the two compose into a double flipand nothing fires — #598, linked from the
roadmap issue table and spec §5.5, which now carries the per-gate call counts.
F3 — a recorded number the gate does not produce. §7 recorded the wrong-order arm at
0.00514, matching no row of its own table. The gate reports corr -0.00239115 / rel1.41856. §5.1 also named a throw as the red-before for a change with two independent
halves; both reds are now stated separately with the value each produces.
F4 — a comment contradicted by four call sites.
nvfp4_dequant.hclaimed the order is"never defaulted per call site". It is, deliberately, and
minimax_h3_nvfp4.cpp:112,minimax_h3_device.cpp:1311,qwen3_5.cpp:1298anddense_nvfp4_gemm.hall rely on it.F5 — the correlation tolerance, RE-BOUNDED strictly stronger.
rel_rmsis a predictedquantity (0.100672), not an error budget.
rel <= 0.15admitted a +10% uniform group-scaleerror, and Pearson correlation is scale-invariant — corr reads 0.994968 to every printed
digit for every multiplier — so nothing else could catch it. It also had no floor, so an arm
reproducing the FP8 oracle scored rel 0.0 / corr 1.0 and passed every correlation assertion.
Replaced by
0.085 <= rel <= 0.115pluscorr <= 0.998, proven strictly stronger by A/B onthe same binary path with only the bound differing:
Status: SUCCESS!Both are now permanent arms of the gate, not one-off logs.
F6 — the witnesses, recorded honestly. torchao is neither installed nor vendored here, so
kernels.py:160/:137-139are upstream source only. vLLM'sbreak_fp4_bytesis reachedonly from
dequantize_to_dtype(ModelOpt / compressed-tensors); vLLM's torchao path(
torchao.py:290-318) delegates totorchao.quantization.quantize_and never calls it. §1.1now tabulates what each witness proves and how it was verified. The default is unchanged.
Evidence
corr = -0.0024,rel_rms = 1.419), GREEN after (0.994968/0.100672).marker-less-padded as torchao, an apostrophe in a refusal, M3 (shared default flip), M4
(x1.09 and x1.10 group scale), and the too-good oracle arm.
ctest -N409; fullctest409/409 passed, 0 failed, 1 skipped (test_voxtral_e2e).Release, CUDA=OFF,
BUILD_EXIT=0, noNo space left/BFD assertionin any build log.test_nvfp4_dequant5/69,test_gguf_nvfp414/2352,test_qwen36_weights7/45,test_qwen3_forward7/1557,test_minimax_h379/57395,test_ops_nvfp4_matmul4/1,test_ops_moe_grouped6/3,test_ops_nvfp4_fp422/919,test_ltx2_device13/498,test_ltx2_loader24/4809(4793 -> 4809, the +16 from the re-banding and its two built-in mutation arms).
flockhold 02:31:03-02:37:15: both shipped DiTs stage device-resident andforward. NVFP4 (18,721,432,024 B, 7876 tensors) 5626 assertions, absmax video 0.275391 /
audio 1.02344. FP8 5624 assertions, absmax video 0.300781 / audio 2.14062 — reproducing the
previously recorded GB10 figures to the digit.
ltx-2-5.md§4.2 is closed with no defect: torchao is low-first, so the shippedtext-encoder arm was already correct and its gate is now source-anchored instead of comparing
our helper to itself. (The earlier "three independent witnesses" phrasing is corrected in
spec §1.1 per F6.)
Known pre-existing:
check-doc-checkpointflags three commits inherited by this branch(
b0aa475a3,d67f8125e,7d08e220a) for not updatingdocs/FEATURES.md. Identical beforeand after this repair, and
docs/FEATURES.mdis the lock recorded in#595. The staged-change checkpoint passes.
🤖 Generated with Claude Code