LTX-2.5 FULL PORT: close every refused arm, prove the prompted path on real weights, and lift the resolution ceiling

[localai-bot]

The core port landed in cefacd2d0 (#435): DiT, both VAEs and their encoders, pipeline, quantized loaders (FP8 / torchao-NVFP4 / first-party NVFP4), the Gemma-4 tower, and a typed prompt wired to cross-attention. It renders a coherent scene.

It is not a complete port. This issue is the campaign to make it one. Directive: complete the LTX port, with all features, complete e2e, landed in main.

The gaps, from the code rather than from the summary

A — the prompted path is WIRED but never run on real weights (highest value)

--encoder + --prompt is reachable, has_encoder() is true, and prompt → tokenizer → Gemma-4 tower → caption projections → connector → cross-attention closes. But every gate on it is a reduced-dimension synthetic fixture. No real-checkpoint prompted render has been run in either direction, so we can say neither that it works nor that it fails. Until this is measured, "you can prompt it" is unproven.

B — image / keyframe / reference-video / reference-audio conditioning is REFUSED

The MiniMax-parity gap. H3 serves fl2va (keyframes) and ref2va (references); LTX-2.5 refuses all four kinds by name at ltx2_video.cpp:1122. Two distinct causes, both real:

1. The VAE encoder landed (L11, Ltx2ConvVideoEncode) but nothing reaches it — the engine materializes the decoder key filter only, so no VAE_ENCODER_COMFY_KEYS_FILTER / VideoEncoderConfigurator path puts encoder weights in memory.
2. The request-side work is unported: image decode, aspect-fill resize, and the H.264 CRF round trip upstream performs before encoding (ImageConditioner.resolve_crf → preprocess, skipped only at crf == 0).

C — the resolution ceiling is memory, and it is in the HOST VAE DECODE

Measured: 128x128/9f and 320x192/25f complete; 448x256/25f stops. MemAvailable is flat at 75.2 GiB through both denoise phases, then falls 73.0 → 13.8 GiB in 24 s with process RSS flat at 4.9 GB — on the decode side of the last drain, so no drain can shrink it. Named next steps: instrument Ltx2ConvVideoDecode's own allocations, then temporal chunking + spatial tiling (H3 needed both; either alone was worse than neither), then a device decode. No ceiling is declared — this is an unresolved implementation gap, not a limit of the hardware.

D–I — arms refused by name, each recorded rather than silently dropped

• DiffVAE / NADiffusionDecoder — the higher-quality decoder, built on neighborhood attention. Needs a new kernel.
• Temporal x2 upsampler (the spatial one ships).
• LoRA fusion.
• Multishot.
• int8-convrot (ComfyUI-only quantization).
• Multi-GPU / CFG parallelism.

J–N — correctness and completeness debt

• No value oracle for the conditioning composition. Scaling it x1.5 or reversing its rows passes every assertion; last_conditioning() is a change detector. Closure path exists: gen-ltx2-pipeline-goldens.py already executes upstream embeddings_connector.py under a pinned SHA, so a section executing EmbeddingsProcessor.process_hidden_states end to end would give the join a real oracle.
• The shipped NVFP4 tower is never compared to an independent oracle — its goldens come from our own low-first helper. PR L9a: the first-party NVFP4 DiT loads -- it was SWIZZLED and HIGH-nibble-first, not linear (#435) #571 built exactly the correlation technique that would close this.
• The text encoder runs on a CPU queue even on the device arm, because it is f32 by declaration.
• duration_head_path is accepted and never read (LTX-2.5: duration_head_path is accepted and never read — a caller who supplies it silently gets the recipe default #611) — a caller supplying it silently gets the recipe default.
• Speed is structurally PENDING: vLLM-Omni carries no native 2.5 and its diffusers adapter is a black box (supports_step_execution=False), so no production-configuration denominator exists. Needs an explicitly ratified alternative denominator or a documented refusal.

Order

A first — it is cheap, it is the headline claim, and its result changes what B and C are worth. Then B (parity with our own H3 lane), then C (the ceiling). D–I are independent and parallelisable. J–N ride along with whichever row touches them.

Each gets a row and a committed spec before code, per AGENTS.md.

[localai-bot]

Row C's premise is refuted, measured. The resolution ceiling is NOT the host VAE decode.

row/LTX25-TILED-DECODE (PR #656) instrumented the decode with an exact live/peak allocation counter (global operator new/delete) and decoded a real 448x256/25f latent to completion on the shipped conv VAE:

	64x64/9f	448x256/25f
largest single allocation	1.74 MiB (predicted 1.74)	99.20 MiB (predicted 99.3)
heap peak live, exact	5.13 MiB	361.72 MiB
decode-attributed RSS	6.13 MiB	362.23 MiB

362 MiB against the reported 60 GiB — a factor of 170. The analytic ladder model predicted the largest allocation to three significant figures at two different scales, so this is not one run's artefact. And the original signature — process RSS flat at 4.9 GB while MemAvailable fell 60 GiB — is consistent with the decode not being the consumer, which is what it now measures as.

Two further corrections to the row's brief, both mine:

1. Upstream's AUTO tiling is a NO-OP at 448x256/25f. split_by_size returns one interval when dim <= size (tiling.py:199-200): the latent is 8x14 against a 14x24 grid tile, and 4 frames against a 10-latent-frame temporal tile. Executed at the pin and emitted as a golden: 1 tile, 1 group. Tiling first binds at 896x512 spatially and 121 frames temporally. So "the win at our resolution is temporal chunking (80/24)" was false.
2. That decode took 2681 s (44.7 min) of single-threaded double-precision convolution. The reported 24-second collapse window cannot have been a completed decode.

Consequences.

• The 60 GiB is unattributed and the resolution axis stays OPEN. Named next hypotheses: which PID the sampler read and at what interval; other processes on the box (dgx has rebooted five times and runs concurrent multi-hour jobs); a CUDA or mmap allocation that never lands in VmRSS. Nobody should treat "448x256 dies in the host VAE decode" as established — it is now disproved.
• The tiling port still lands and is still right, because it is upstream's own mechanism and binds at larger geometries; it simply does not explain this ceiling.
• The row also declined to invent the tiled-vs-untiled equivalence bound the brief asked for: upstream's OWN max|tiled - untiled| is 0.67-1.29x the full output range and does not converge, because the receptive field is wider than the 2-latent-cell overlap. What is exact is the one-tile control — a config whose splits all short-circuit reproduces forward bit for bit (upstream measures 0; the port is held to == 0.0).

This is the third premise in this campaign refuted by measuring before building, and the second where the refutation came from the implementer rather than a reviewer.

[localai-bot]

Campaign status — two rows landed, and both were CORRECTNESS fixes rather than features

On main:

• cefacd2d0 — the core port (DiT, both VAEs, the VideoEngine seam, pipeline, quantized loaders)
• 65e79eee5 — feat(ltx-2.5): port the prompt-side AdaLN, and stop the opt-in from clearing it #654, the prompt-side AdaLN. use_prompt_adaln_single was disabled unconditionally while both references default it TRUE, and allow_unported=1 silently cleared it. Measured on the shipped checkpoint by the reviewer, not the synthetic fixture: the timestep term is 1439% (video) / 1593% (audio) of the static table by RMS. Every render before this applied roughly a twentieth of the prompt conditioning upstream applies, and nothing in any output announced it.

What the grounding pass changed about this campaign's SCOPE

Three of the six "refused arms" I scoped as work do not exist upstream:

• multishot — our refusal cites a fabricated anchor; no such entry point, symbol or string in either reference
• int8-convrot — no upstream counterpart; our "ComfyUI-only" message is correct
• multi-GPU / CFG parallelism — exists, but is "a latency tool, not a memory tool" (docs/multigpu/README.md:5-16), and the distilled recipe we run uses no guidance at all (SimpleDenoiser at distilled.py:266,295, documented "single transformer call, no guidance" at denoisers.py:3, guider cfg_scale=1.0, stg_scale=0.0 at :25-28). Zero win.

And DiffVAE is a port, not a new kernel: diffusers' default processor is a portable FlexAttention path whose own docstring says it stands in for NATTEN, Lightricks ships a pure-torch masked-SDPA fallback that runs on CPU, and ltx-kernels/csrc contains no neighborhood-attention kernel at all.

Two premises this campaign DISPROVED by measuring

The resolution ceiling is not the host VAE decode. Instrumented with an exact allocation counter: 361.72 MiB heap peak at 448x256/25f against a reported 60 GiB — a factor of 170. Upstream's tiling is a no-op at that geometry, and the decode takes 44.7 minutes, so a 24-second collapse cannot have been a completed decode. The axis is REOPENED with named next hypotheses. A reviewer independently reproduced the probe at two scales plus an out-of-sample third it predicted analytically first (6.21 MiB predicted, 6.21 MiB measured), and observed the confound in situ: during a decode whose true peak was 19.72 MiB, MemAvailable fell 2795 MiB — a 140x overstatement by the instrument behind the original figure.

Temporal chunking binds at 81 frames, not 121 — wrong in five places including two self-contradicting bullets in docs/USAGE.md. This matters: the recipe default is 1024x1536 at 121 frames, so a user at the default IS in the tiled regime while the record says otherwise. Demonstrated on the shipped VAE at 81 frames: 985,849 of 995,328 floats change, max|diff| 0.716 in a signal of scale ~1. Root cause is a sweep that skipped its own binding point — probe_ltx2_tiling_layout.py:56-63 samples 9, 25, 25, 25, 121, 241.

In flight

Image conditioning (repair: a keyframe refusal whose reason is false at the pin, plus a phase-0 survivor), tiled decode (repair: the 81-frame correction and an untiled-axis arm where a blackout mutation passes 9/9 and 830/830 green), the AdaLN claims cleanup, the temporal x2 upsampler, the dead-arm retirement (+#611), and a prompted render on dgx — the first with a complete conditioning chain.

Needs a decision, not code

• Every LTX-2.5 gate runs against an oracle the new policy does not admit (Lightricks ltx_core is not in the registry) #655 — every LTX gate runs against Lightricks ltx_core, which the new closed oracle registry does not admit. No checker catches it; AGENTS.md says prose and checker are deliberately unverified against each other.
• LTX-2.5: upstream adds a TRAINED per-token bias to the first latent frame and we do not — the served first-frame arm is not upstream-equivalent #658 — upstream adds a trained per-token bias to the first latent frame on every forward; we never do. May be inert on these two checkpoints, but that is a property of the files, not of the port.

---

Edited: two file:line anchors in the multi-GPU bullet were eaten by shell command substitution when this comment was posted, and are restored above. Worth noting as its own small instance of the campaign's theme — a citation that silently becomes () reads as "no anchor was ever offered" rather than as "an anchor was lost", and the reader cannot tell the difference.

[localai-bot]

Campaign status — 2026-08-13 21:15Z

Recorded by the operator. Every number here is either from a gate run or from a review that re-derived it; where something is unverified it says so.

Landed on main

• 65e79eee5 — the prompt-AdaLN defect. ltx2_loader.cpp force-cleared use_prompt_adaln_single, which defaults TRUE in both references (transformer_ltx2.py:1185, model_configurator.py:76,138). Three assignments replaced by a guard that fails when the resolved flag disagrees with the file.
• 11cc1d589 — the device seam (row/MODEL-DIFFUSION-LTX25 is RED on the device-leakage DSR ratchet: ltx2_video.cpp adds two kcuda sites to the shared layer #553, windows-msvc CI is RED on main: test_backend_cross_device.cpp uses POSIX setenv/unsetenv, which MSVC does not provide #603, test_agent_record: the MODEL-ratchet docstring is two contradictory paragraphs spliced together, and the surviving half is factually wrong #651, model-matrix.md prose counters drifted: LTX-2.5 was added to the rows and the rollup but to none of the four sentences that count them #652). Landed by another session; the red it cleared was this lane's.
• cefacd2d0 — the generalized VideoEngine seam.

In review or repair

PR	Row	State
#661	retire the invented refusals	reviewed FAIL, repair in flight
#662	temporal x2 latent upsampler	awaiting review; merging main to clear an inherited red
#663	prompt-AdaLN claims	reviewed FAIL on one claim, repair in flight
#666	image conditioning + its repair	under review; contains #657, which closes as superseded
#656	tiled + streaming VAE decode	reviewed FAIL, repair in flight
#671	device-seam sibling (#659, #660)	spec only, open

Findings worth reading even if you skip the PRs

A scoped row was built on a number nobody re-derived. "448x256 dies in the host VAE decode — 73.0 → 13.8 GiB in 24 seconds" passed through a spec, a campaign issue, three briefs and several summaries. Measured with an exact allocation counter: heap peak 361.72 MiB, largest single allocation 99.20 MiB. 362 MiB against 60 GiB — a factor of 170. Two independent corroborations: upstream's own tiling is a no-op at that geometry, and the decode takes 44.7 minutes, so a 24-second collapse was never a completed decode. The resolution axis is reopened; the tiling row survives on its own merits, not as "the fix".

The prompt-AdaLN defect was understated, not overstated. Re-measured on the real checkpoint through upstream's own AdaLayerNormSingle: max|term|/max|table| = 7119% video / 2817% audio, against a synthetic fixture that had said 51.7%. A reviewer re-ran the script and reproduced every digit independently.

Temporal chunking binds at 81 frames, not 121 — the probe that established 121 swept 9, 25, 25, 25, 121, 241 and skipped its own binding point. Recorded in five places, contradicted by the row's own golden.

The FP8 DiT cannot be rendered inside the contract. It carries keyframes_abs_pos_embedding (F8_E4M3 [1,4096] + F32 scale) and declares **no __metadata__ at all; the first-party NVFP4 DiT declares its config and carries no keyframe tensor. Both header facts read directly. The refusal is correct and stays; the NVFP4 arm is the one that loads, and it is the preferred arm anyway.

#664 — this lane reddened the whole repository. src/vllm/multimodal/video_engine.cpp:21,57-65 reaches Windows with unguarded POSIX stat, failing windows-msvc-cpu/windows-msvc-vulkan on every open PR, across five unrelated lanes. It hid because those jobs are skipped on main pushes, so a green main carried zero information about them. Fix in flight.

Owed, and honest about it

• LTX2_CHECKPOINT_ROOT appears in no CI workflow, so CI runs 502 of 8734 assertions — 5.7% — of test_ltx2_video. The checkpoint-derived evidence in this campaign is manual and host-local.
• Decisions still owed to the developer: Every LTX-2.5 gate runs against an oracle the new policy does not admit (Lightricks ltx_core is not in the registry) #655 (Lightricks ltx_core is not in the closed oracle registry) and LTX-2.5: upstream adds a TRAINED per-token bias to the first latent frame and we do not — the served first-frame arm is not upstream-equivalent #658 (a trained first-frame bias that is never applied).
• dgx.casa rebooted three times today; the e2e render is queued behind another session's job on flock $HOME/gpu.lock.