L9B: the LTX-2.5 render driver, --video-family, and the first shipped-weights render - #548
Conversation
…t shipped-weights render FOLLOWING_AGENTS_PROTOCOL A way to drive an LTX-2.5 render, the server flag a second video family makes necessary, and the plain record of what the shipped weights actually produced. `examples/ltx2_gen` (ltx2-gen) is a THIN CLIENT of the public C ABI and nothing else. It exists rather than a flag on minimax-h3-gen because LTX-2.5 cannot LOAD without three of the v18 family-specific extras -- the audio-stream prompt embeds, the DiT config the shipped FP8 checkpoint does not carry, and the latent spatial upsampler the distilled recipe's second phase needs -- and that example drives neither `family` nor the extras arrays. It DECLARES the family rather than letting detection infer it, because an FP8-vs-NVFP4 comparison is only a statement about the two FILES if the family is declared. `--video-family` on the server. The code there said the flag was deliberately not invented "until a second family exists to disambiguate"; a second family now exists. Detection stays the default when the flag is absent, so every pre-existing invocation is byte-identical. `--video-extra KEY=VALUE` lands with it, because pinning a family is useless if that family's required load knobs are unreachable: LTX-2.5 needs `dit_config_path` and `audio_prompt_embeds_path`, and neither has -- or should have -- a dedicated flag on a family-generic server. A bare KEY is refused rather than read as KEY="", and a `--video-extra partition=X` contradicting `--video-partition Y` is refused rather than resolved by assignment order. The declared name is validated in ParseArgs against RegisteredVideoFamilies(), before the TEXT model loads, for the same reason --tool-call-parser is; the REGISTRY is the authority, never a literal list, so a family added in its own file needs no edit here. RED first, black-box, because the flag table lives in an anonymous namespace inside server_main.cpp and the binary IS the surface under test. Three new ctests, all three failing with "unknown argument '--video-family'" before the change and passing after. The registered-name case proves the check consults the REGISTRY by getting PAST it to the next validation, so a hardcoded list omitting ltx-2.5 REDs. WHAT THE RENDER PRODUCED, stated as measured. The shipped 21.00B FP8 DiT under LTX-2.5's DECLARED config renders end to end on the GB10 at 128x128 / 9 frames through BOTH distilled phases: 9 PPM frames, a 0.33 s 48 kHz stereo WAV that is not silent, and a valid h264+AAC MP4. The frames are NOT a scene. They are smooth low-frequency colour fields -- neighbour |dx| / whole-image sd is 0.03 to 0.06 where white noise gives ~1.13, so they are not noise either, and the block-mean ratio near 1.0 says they are not H3's patch grid. That is the expected outcome, not a defect in the DiT: the Gemma-4 tower is unported AND so is the embeddings connector between it and cross-attention, so the conditioning never passes through two of its links. AND THE SIZE CEILING IS MEMORY. 320x192 / 25 frames staged fine and then grew device usage by a further ~58 GB during the loop; unified memory makes those host bytes and this box reboots instead of OOM-killing -- it did, twice, before the run went behind a watchdog that stops it at a floor we choose. USAGE documents the command WITH both facts, because a recipe printed without them reads as working. Issue: #435 Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
|
CI note: the This PR touches neither file. Both lines are present verbatim at Filed as #553 rather than fixed here: it is a phase-L8 device-wiring question |
…out one model Only conflict was docs/FEATURES.md and docs/USAGE.md, and neither was a union: both sides describe the SAME keyed row with facts gathered at different points in the campaign, so an append would have left the table asserting a model both renders and does not. FEATURES: the branch knew the render facts (128x128/9f e2e, valid MP4+WAV, frames NOT a scene, 320x192 exhausts unified memory) and the campaign knew the coverage facts (both VAE encoders + mel front-end, the conditioning items, the 6.0e-07 encode bound, FP8 + torchao-NVFP4 loading). Each was accurate about itself and wrong about the other. Resolved to the facts true of the MERGED tree, keeping both halves and the "first-party NVFP4 does NOT load" limit, which L9A has not landed yet. USAGE: the campaign text said no CLI flag, server endpoint or C ABI call reaches LTX-2.5 and that there is "nothing to run here yet". That stopped being true on this branch, which added ltx2-gen and the family flag. Took the branch's reachable-path opening and kept the campaign's encoder-conditioning caveat, which is still exactly true: the encoders existing does not make image, keyframe, reference-video or reference-audio conditioning usable, because the request-side work is unported and the engine refuses each by name. Worth recording that this conflict was structural rather than accidental. doc-checkpoint REQUIRES a FEATURES.md edit from every phase, so every phase collides there -- which is the lock AGENTS.md warns about, created by the gate rather than by the people touching it. Filed separately. Issue: #435 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
|
Landed on This branch had gone Resolving it needed judgement rather than a union: both sides described the SAME keyed row with facts gathered at different points, so appending them would have produced a table asserting the model both renders end-to-end and has no reachable entry point. Same for That contention was structural, not carelessness — Closing rather than merging; it reaches |
Row
MODEL-DIFFUSION-LTX25, phase L9B. Issue #435. Spec.agents/specs/ltx-2-5.md.Base is
row/MODEL-DIFFUSION-LTX25(the campaign branch), notmain.What lands
examples/ltx2_gen(ltx2-gen) — a thin client of the public C ABI andnothing else. It exists rather than a flag on
minimax-h3-genbecause LTX-2.5cannot load without three of the v18 family-specific extras (the audio-stream
prompt embeds, the DiT config the shipped FP8 checkpoint does not carry, and the
latent spatial upsampler the distilled recipe's second phase needs), and that
example drives neither
familynor the extras arrays. It declares thefamily rather than letting detection infer it, because an FP8-vs-NVFP4
comparison is only a statement about the two files if the family is declared.
--video-familyon the server. The code said the flag was deliberately notinvented "until a second family exists to disambiguate"; a second family now
exists. Detection stays the default when the flag is absent, so every
pre-existing invocation is byte-identical.
--video-extra KEY=VALUElands withit, because pinning a family is useless if that family's required load knobs are
unreachable. The declared name is validated in
ParseArgsagainstRegisteredVideoFamilies(), before the text model loads, mirroring the--tool-call-parserprecedent; the registry is the authority, never a literallist.
Evidence
RED first, black-box, because the flag table lives in an anonymous namespace
inside
server_main.cppand the binary is the surface under test. Three newctests; all three failed with
unknown argument '--video-family'before thechange and pass after.
The render, on dgx.casa GB10 under one
flock $HOME/gpu.lockhold. Theshipped
vonkaiserFP8 DiT (21.00B, 6124 tensors) under LTX-2.5's declaredconfig (extracted from the first-party NVFP4 sibling's
__metadata__:frequencies_precision=float64,av_ca_timestep_scale_multiplier=1000) rendersend to end at 128x128 / 9 frames through both distilled phases: 9 PPM
frames, a 0.33 s 48 kHz stereo WAV that is not silent, and a valid h264+AAC MP4.
The frames are not a scene. They are smooth low-frequency colour fields.
Measured: neighbour
|dx|/ whole-image sd is 0.03-0.06 where white noise gives~1.13, so they are not noise either, and the block-mean aligned/offset ratio sits
at 0.95-1.19, so they are not MiniMax-H3's patch grid. That is the expected
outcome rather than a defect in the DiT: the Gemma-4 tower is unported and so
is the embeddings connector between it and cross-attention, so the conditioning
never passes through two of its links.
The size ceiling is memory. 320x192 / 25 frames staged fine (~44 GB) and then
grew device usage by a further ~58 GB during the denoise loop. GB10's memory is
unified and this box reboots instead of OOM-killing — it did, twice, before the
run went behind a watchdog that stops it at a floor we choose.
docs/USAGE.mddocuments the command with both facts.
Not in this PR
row/LTX25-L9A-NVFP4-LINEARretracted its premise ratherthan implementing (see
f400413ef), so the first-party NVFP4 DiT still doesnot load and the second arm is blocked upstream of this work.
is no production-configuration denominator. Wall clocks in the logs are sizing
only.
🤖 Generated with Claude Code