feat(ltx-2.5): image conditioning at crf=0, and the VAE encoder load path that did not exist (#644)

[localai-bot]

Row 1 of the #644 LTX-2.5 full-port campaign: closes the image conditioning arm and builds the thing every arm was blocked on.

Spec 0c1390f8e, committed before any code: .agents/specs/ltx25-image-conditioning.md.

What was actually missing

The refusal at ltx2_video.cpp:1122 was honestly written and its anchors verify, but it led with the CRF round trip, and that is not what blocked anything. Re-verified against the current tree:

claim	check	result
the encoder MATH landed	Ltx2ConvVideoEncode	present, gated since phase L11
the placement landed	Ltx2ConditionVideoByLatentIndex	present, gated
the engine loads the DECODER filter only	ltx2_video.cpp:752	confirmed
an encoder key filter exists anywhere	grep -rn Ltx2VideoVaeEncoderKeyRules src include tests	no match
an encoder CONFIG parser exists anywhere	grep -rn ParseConvVideoEncoder src include tests	no match

So the encoder was a brick with no delivery route. preprocess short-circuits at if crf == 0: return image (media_io/decode.py:425-426), so a crf=0 path is upstream-legal and needs no codec — this box has no av module at all, which is the demonstration rather than the obstacle.

What this adds

• Ltx2VideoVaeEncoderKeyRules / Ltx2ParseConvVideoEncoderConfig / Ltx2CheckpointHasVideoEncoder — VAE_ENCODER_COMFY_KEYS_FILTER (video_vae/model_configurator.py:267-276) and VideoEncoderConfigurator over _prepare_video_encoder_kwargs (:37-69, 72-78).
• The pixel front-end — Ltx2DecodePpmRgb, Ltx2ResizeAndCenterCrop, Ltx2LoadImageAndPreprocess, Ltx2ResolveDefaultImageCrf (media_io/decode.py:46-79, 139-170, 413-435, resize.py:41-73, range_map.py:8-9, constants.py:36-37, 124, 126-133).
• The engine path — encoder loaded beside the decoder, and the image decoded, resized, encoded and placed per phase, before the noiser, because the two-stage recipe renders its stages at different resolutions.
• image_crf, the family's first per-generation extra, reachable through the existing ABI arrays with no ABI change, and through ltx2-gen --first-frame / --image-crf.

Both new TUs are additive files rather than additions to ltx2_loader.cpp, which a concurrent row of this campaign also needs to edit.

CRF 0 is out of distribution, and says so

An LTX-2.5 checkpoint resolves default_image_crf = 18 (constants.py:37/124/130-133), and the H.264 round trip is unported. The default therefore refuses — a caller has to ask for image_crf=0 knowingly and gets a render conditioned on uncompressed pixels. Both halves are stated rather than one of them.

The other three refusals now name different missing pieces

This campaign has had five refusals whose stated reason went stale, including the one replaced here. Each surviving message names the exact symbol a later reader can go and check:

• keyframe → the DiT's unported keyframes_abs_pos_embedding (the encoder and Ltx2ConditionVideoByKeyframe are both here now);
• reference image / video → the IC-LoRA's downscale_factor / temporal_scale_factor, which live in LoRA metadata this project does not read;
• reference audio → the AUDIO VAE's encoder key filter, which this row did not build.

Evidence

scripts/gen-ltx2-image-cond-goldens.py executes upstream ltx_core and ltx_pipelines at the pinned SHA fd4ded7f. The package __init__ files are bypassed and av / OpenImageIO are stubbed as tripwires, not shortcuts: preprocess(image, 0) is image is asserted by object identity (no stub can manufacture that) and preprocess(image, 18) is asserted to reach the stubbed codec and raise, so the branch is measured rather than read.

Bands derived from measurement, not picked (both tolerances set to 1e-12 and the reported worst read back):

section	space	worst	band
resize	0..255	6.10352e-05	2e-4
preprocess	[-1, 1]	4.76837e-07	2e-6
encoded latent / conditioned clean	latent	2.68221e-07	2e-6
noised latent	latent	1.78814e-07	2e-6

Six source mutations RED, each rebuilt and restored byte-for-byte (md5 verified). Green baseline test_ltx2_image_cond 15 / 198 and test_ltx2_video 32 / 550:

mutation	result
ceil → llround in the resize	image_cond 14/15, 197/198
encoder key rules → the DECODER's	image_cond 14/15, 195/198; video 31/32 (THREW; assertion COUNT moved to 517)
mask 1 - strength → strength	image_cond 13/15, 193/198
latent width from vae.out_channels	image_cond 14/15, 192/198; video 9/32, 36/53
CRF refusal removed	image_cond 14/15, 193/198; video 31/32
encode the image, never PLACE it	video 31/32, 548/550

Encoder weights are loaded AND used, proven separately: one tensor of the encoder half is perturbed in a second fixture (offset read from the safetensors header, not guessed) with the request byte-identical, and the conditioning digest must move. The first version of that check searched for the tensor NAME and flipped a byte a fixed distance past it — which lands in a neighbouring tensor's JSON — and it failed, which is the only reason that is a footnote rather than a false green.

Three of this row's own claims were REFUTED

Recorded in the spec's §8 rather than quietly fixed:

1. The ceil does not guard what upstream's comment says it guards. Swept every source/target pair in 3..40 → {16, 24}: no pair makes src * scale land just above an integer. It is still load-bearing, for the ordinary reason that it disagrees with round and floor, and the generator asserts that.
2. No golden here can see a resize/normalize ORDER swap. Resize is a convex combination and normalize is affine, so the two orders are equal in exact arithmetic; the f32 gap is 1.94e-07, below every band and not amplifiable. The order is mirrored because it is upstream's, and that is written down in three places instead of assumed covered.
3. Torch's bilinear cannot be matched bit for bit portably. Index map and lambdas were probed with basis images and match exactly; the residual is FMA contraction in torch's kernel, and this tree builds -ffp-contract=off.

One mutation SURVIVED and is analysed rather than hidden (spec §8.4): phase 0 runs at noise_scale = 1.0, where the mutated value is genuinely discarded — and separately, test_ltx2_video gates no value of the composed latent, which test_ltx2_image_cond does against executed upstream. That survivor is why the trace digest is taken over the tokens as written rather than over the encoder's output.

Gate

• BUILD_EXIT=0, 0 ENOSPC/BFD hits, CPU Release, gcc.
• ctest -N → 424; ctest -j 4 → 424/424 passed, 0 failed, exit 0 — denominator asserted against the listing.
• Focused: test_ltx2_image_cond 15 cases / 198 assertions, test_ltx2_video 32 / 550, both exit 0.
• scripts/agent-preflight.sh --staged → all gates green, including the new check-oracle-pins.py from policy(oracle): admit a pinned secondary oracle where vLLM implements nothing (#647) #650.
• Generator is byte-reproducible: md5 da4f7423…a439 before and after a regeneration.

Closes part of #644.

🤖 Generated with Claude Code

[localai-bot]

Superseded by #786 — please do not merge this

This PR's head branch, row/LTX25-IMAGE-COND, now carries 247eba52b: the
result of #666 having been squash-merged into this branch instead of into
main. That squash landed on top of a7154bd1e, which was itself a
merge: origin/main, and squashing on top of a merge flattens the merge away.

Measured consequences on this branch as it stands:

git merge-base 247eba52b origin/main	b81a5e413 — 67 commits back
this branch's apparent diff	252 files, 42,426 insertions
git merge origin/main	19 conflicts, incl. add/add on files purely main's

The add/add set includes gpt2.*, talker.*, w2vbert.*, test_talker.cpp,
test_w2vbert.cpp, .agents/specs/indextts-2-5.md and
.agents/specs/minimax-music3.md — none of which this row ever touched. Merging
or hand-resolving from here is how main's newest work gets silently reverted.

The content is not lost. The tree at 247eba52b is byte-identical
(d276e5194) to the last gated head, and the whole row — this feature plus the
#666 repair — has been reconstructed onto current main as #786, verified by
file list: exactly 19 LTX-2.5 files, zero of main's, registration delta +1, and
git merge-base back to main's tip.

Close this once #786 lands.

🤖 Generated with Claude Code

[localai-bot]

Closing as superseded by #786, which landed on main as c629b5d0f.

Why this one could not land

This branch's head is 247eba52b — the squash of #666, whose base branch was this branch rather than main. My error as operator: I read ancestry (#657 is an ancestor of #666) and concluded supersession without checking baseRefName.

That squash flattened away the merge: origin/main commit beneath it, so git no longer knew this row had ever seen main. Measured:

• git merge-base 247eba52b origin/main → 67 commits back
• git diff --stat against that base → 252 files, 42,426 insertions — git believed this row authored main's oracle scripts and website/
• git merge origin/main → 19 conflicts, including add/add on files that are purely main's (gpt2.*, talker.*, w2vbert.*)

Merging it would have dragged that diff onto main. Hand-resolving add/add conflicts on files this row never authored is precisely how main's newest work gets silently reverted.

Nothing was lost

#786 was rebuilt from a pinned main SHA and carries this row's content provably intact: the substantive +/- payload of this branch against the main it was last gated on is byte-identical to #786's — 4471 lines each, sha256 8e6ba77f… both, differing only in 16 hunk-header lines that are main's own drift.

Both directions were verified, because for a reconstruction "nothing leaked in" is only half the claim: the complete 19-file list contains zero of main's files (positive control ltx2 = 13, negative controls all 0), and every distinctive string the repair introduced is present at the new head and absent from the pre-repair commit.

The transferable part

A PR's base is not implied by its ancestry, and the only reliable proof a merge landed is that its commit is an ancestor of origin/main — gh pr view --json state reporting MERGED answers a different question. Both are now recorded.

Continue at #786 (merged) and #644.