Skip to content

Compute the illustration camera deterministically, raise clip resolution - #28

Merged
upgradedev merged 2 commits into
mainfrom
feat/deterministic-camera
Aug 2, 2026
Merged

Compute the illustration camera deterministically, raise clip resolution#28
upgradedev merged 2 commits into
mainfrom
feat/deterministic-camera

Conversation

@upgradedev

@upgradedev upgradedev commented Aug 2, 2026

Copy link
Copy Markdown
Owner

Summary

Two live renders proved pixverse-v6-i2v cannot be steered on camera motion or resolution by prompting alone (#26, #27): it exposes no motion/camera parameter at all (verified against the SDK's own param_allowlist), and asking for "gentle camera parallax and a slow push" plus a negative_prompt naming "zoom out" still drifted into an unusable extreme close-up on a follow-up render. This PR stops asking the model to move the camera and computes the move ourselves, deterministically, from the case's own factual Timeline — the same sealed record the schematic itself is drawn from.

  • New src/claimscene/camera.py: compute_camera_path derives a slow push toward the reconstructed contact point purely from Timeline geometry, sized and positioned (via a clamped-centering construction, proven non-empty by construction) so every impact participant's full footprint stays inside the frame for the whole clip. Applied via an ffmpeg zoompan filter built from the case's own numbers — spiked and verified against real ffmpeg locally — expressed in terms of ffmpeg's own iw/ih so it's resolution-agnostic, with the actual clip's dimensions/frame-rate probed via ffprobe (JSON output, not positional CSV — verified empirically that ffprobe's CSV writer does not preserve request field order). Runs on the model's raw output, before the disclosure caption is burned in, so the caption is never cropped or distorted by the push. Fails safe to the unmoved clip on any error (no contact point, a computed push that would be a visual no-op, ffmpeg/ffprobe absent, undecodable input, subprocess failure) — never raises, never fails the render.
  • report.py: the clip prompt now asks for a locked-off, static shot with no camera movement of any kind, replacing "gentle camera parallax and a slow push." The negative prompt now suppresses camera motion in both directions (it previously only named the pull-out failure mode from feat(illustration): suppress the runaway camera, pin a reproducible seed #27's render).
  • pipeline.py: quality raised from "360p" to "720p" (module constant, ILLUSTRATION_CLIP_QUALITY); aspect_ratio pinned to "4:3" to match the seed raster's own canvas, so the camera math's coordinate assumption holds. See the constant's docstring for exactly how "720p" was determined (the SDK performs no value validation for this field — param_schemas is empty — so this is the standard next tier in the SDK's own portable resolution vocabulary, not a live-verified value).
  • provenance.py / readiness.py: seal and verify the camera move's own provenance (camera_move_source, camera_move_note, camera_move_applied, camera_move_error) alongside the existing watermark-burn honesty checks. Unlike the watermark check, success is never required — only that the outcome is honestly recorded (the camera push is an intentional best-effort, not a second hard requirement).
  • schematic.py: the pixel-projection formula (_View.px) is extracted into a public world_to_pixel() — a pure, behavior-preserving refactor (golden SVG test unchanged) — so camera.py reuses the exact mapping the seed image was drawn with instead of duplicating it.

Follow-up hardening (commit 6598937, after self-review)

Two behavioral fixes on top of the initial version above — both change what a real render can honestly seal, so calling them out explicitly:

  • _probe_video_params now prefers avg_frame_rate over r_frame_rate. r_frame_rate is ffprobe's own BASE rate — the lowest rate every frame timestamp can be expressed at exactly — which only equals the real playback rate for constant-frame-rate content. A live provider's clip is not guaranteed to be CFR: for variable-frame-rate or odd-timestamp content, r_frame_rate can read back as something unrelated to the actual rate (e.g. a raw stream timebase like 90000/1), which would tell zoompan's fps= to emit tens of thousands of frames per second — the fail-safe would still catch the resulting timeout/error, but with nothing pointing at why. avg_frame_rate (total frames / duration — "what frame rate does this clip actually play at") is now probed alongside r_frame_rate in the same call and preferred whenever it's itself usable; r_frame_rate remains the fallback for the rarer case where avg_frame_rate is ffprobe's own "0/0" unknown sentinel. Confirmed empirically that this project's own CFR test fixtures report identical values for both fields, so no existing assertion needed to change.
  • A computed push that would be a visual no-op now fails safe instead of sealing an empty applied=True. compute_camera_path's end_zoom is clamped to at most 1.0 (only reachable when containment needs the entire frame just to hold both impact participants — a pathologically large or spread-out footprint). At exactly 1.0 the zoompan filter is a constant z=1 for the whole clip: nothing would actually move. apply_camera_push now detects this before touching ffmpeg and returns applied=False with an explicit reason, the same as "no contact point at all" — instead of re-encoding for zero visual change and recording a technically-truthful-sounding manifest note ("a slow zoom toward...") that would not correspond to anything visible in the pixels.
  • A doc-comment correction: report.py's ILLUSTRATION_NEGATIVE_PROMPT docstring previously described the camera push's containment padding as "letterboxed framing" — there is no letterboxing anywhere in this design (no black bars; it's a crop-and-scale in the seed image's own coordinate space). Corrected to describe what the padding actually does: tolerance for minor drift in exactly where the model draws the vehicles, not a guarantee against a changed field of view.
  • A new test that proves the push is visually meaningful, not just "bytes changed." The original real-success test (test_apply_camera_push_succeeds_on_a_real_tiny_clip) passes duration_s=5.0 against a clip that only plays for 1 real second, so the zoom ramp barely leaves its starting point (u≈0.2) — any re-encode changes bytes regardless of whether the filter did anything visible. The new test_apply_camera_push_visibly_moves_the_frame_by_the_end_of_the_clip builds a clip whose real duration matches duration_s, decodes the pushed clip's last frame and the original's last frame via a real ffmpeg call, and requires the pixel-level difference to be well beyond that same pair's own first-frame re-encode noise floor (self-calibrated, not an arbitrary constant). Measured empirically on this exact fixture: noise floor ≈1.6, moved diff ≈43 (≈27x) — the assertion's threshold (3x the floor, minimum 5.0) has wide margin either direction.

Determining "720p" was valid

GMICloudVideoProvider.create_registry().get("pixverse-v6-i2v").param_schemas is an empty dict — the SDK performs no value validation for quality or resolution offline, so the accepted set genuinely cannot be confirmed without a live render. What is confirmed: the Pixverse family's own docstring states quality is "required by the upstream API," and genblaze_core.providers.canonical_params.RESOLUTIONS_TIERED (the SDK's own portable vocabulary) lists "720p" alongside 480p/1080p/1440p/4k — notably, "360p" is not in that set despite working today. "720p" is the standard next tier and the sensible target; it is a module constant, trivial to change.

Constraints honored

  • SceneGraph schema, layout.py's geometry maths, the DISCLOSURE/WATERMARK constants, and everything in tests/security/ are untouched (verified via diff).
  • Offline/fake path still works end to end; the honest-degrade/fail-safe behavior is unit- and integration-tested.
  • No em-dashes in added strings (verified via git diff filtered to added lines, re-checked after the follow-up commit).

Tests touched (existing tests edited, with why)

  • tests/integration/test_genblaze_contract.py::test_pixverse_v6_i2v_routes_image_to_native_slot — updated the example quality value from "360p" to "720p" (and added aspect_ratio to the allowlist/payload assertions) purely for consistency with the pipeline's new default; this test exercises SDK passthrough plumbing with its own literal params, not pipeline.py's actual value, so it would have passed unmodified either way.

No other existing test was modified — the new prompt wording, negative-prompt terms, and manifest fields are all covered by new tests instead (test_camera.py, new tests appended to test_pipeline.py/test_report.py/test_review_provenance.py/test_schematic.py, and the new test_illustration_camera_move.py).

Test plan

  • pytest tests/ (497 tests, excluding 3 files broken by a pre-existing local pytest-9.0.2 environment quirk unrelated to this change — confirmed via git stash that they fail identically on unmodified main) — all green, 96% combined coverage (gate: 90%), camera.py itself at 99% (the one uncovered line mirrors watermark.py's own identical uncovered branch).
  • ruff check src tests scripts — clean.
  • pip-audit -r requirements.txt --strict — clean.
  • python scripts/readiness.py — 100% automatable (gate: 95%), including the honest_media.camera_move_sealed check.
  • pytest tests/security (65 tests) — clean, including test_provenance_integrity.py.
  • Camera math unit-tested against the factual layer: hand-verified exact pixel mapping, containment property across all 4 scene fixtures, determinism, clamping-engaged and containment-widening edge cases, and (new) the full-frame-containment no-op case.
  • Fail-safe tested at both the camera.py unit level and end-to-end through the real pipeline (camera fails → watermark still burns onto the pre-camera bytes → render still succeeds).
  • Real ffmpeg success path tested (tiny synthetic clips) — dimensions and duration preserved, bytes genuinely changed, and (new) the framing genuinely moves by the end of the clip, not just the bytes.
  • Frame-rate field selection tested against mocked ffprobe output: prefers avg_frame_rate, falls back to r_frame_rate when avg_frame_rate is unusable, returns None when both are, tolerates a missing field.

Efthimios Fousekis added 2 commits August 2, 2026 11:38
…resolution

Two live renders proved pixverse-v6-i2v cannot be steered on camera motion
or resolution by prompting alone (#26, #27): it exposes no motion/camera
param at all (verified against the SDK's param_allowlist), and asking for
"gentle parallax" with a negative_prompt naming "zoom out" still drifted
into an unusable extreme close-up. So we stop asking the model to move the
camera and compute the move ourselves from the case's own factual Timeline.

- claimscene/camera.py (new): compute_camera_path derives a slow push
  toward the reconstructed contact point from Timeline geometry alone,
  sized and positioned so every impact participant's full footprint stays
  inside the frame for the whole clip; applied via an ffmpeg zoompan filter
  built from the case's own numbers, resolution-agnostic (targets ffmpeg's
  iw/ih, probed from the actual clip via ffprobe). Runs on the raw model
  output, before the disclosure caption is burned in, so the caption is
  never cropped or distorted; fails safe to the unmoved clip on any error,
  never failing the render.
- report.py: the clip prompt now asks for a locked-off, static shot (no
  camera movement of any kind) instead of "gentle parallax"; the negative
  prompt now suppresses camera motion in both directions, not just the
  pull-out direction measured in #27.
- pipeline.py: quality raised from 360p to 720p (the SDK performs no value
  validation for this field; 720p is the standard next tier in the SDK's
  own portable resolution vocabulary), aspect_ratio pinned to 4:3 to match
  the seed raster so the camera math's coordinate mapping holds.
- provenance.py / readiness.py: seal + verify the camera move's own
  provenance (source, note, applied/error) alongside the existing
  watermark-burn honesty checks; success is never required, only that the
  outcome is honestly recorded.

schematic.py's pixel-projection formula is extracted into a public
world_to_pixel() (pure refactor, golden SVG unchanged) so camera.py reuses
the exact same mapping the seed image was drawn with, instead of
duplicating it.
_probe_video_params now prefers ffprobe's avg_frame_rate (frames over
duration -- the actual playback rate) over r_frame_rate (the lowest
rate every timestamp divides into exactly, which only equals the real
rate for constant-frame-rate content). A variable-frame-rate or
odd-timestamp clip from a live provider could otherwise hand zoompan's
fps= a stream timebase like 90000/1 and silently blow up the encode.
Falls back to r_frame_rate when avg_frame_rate is ffprobe's own
"unknown" sentinel.

apply_camera_push now treats an end_zoom clamped all the way to 1.0
(containment needed the whole frame -- only reachable for a
pathologically large or spread-out footprint) as a fail-safe no-op
instead of re-encoding a filter that would visually do nothing and
sealing a truthful-sounding but empty applied=True.

Also corrects a comment in report.py that described the camera push's
containment padding as "letterboxed framing" -- it is a crop-and-scale
in the seed image's own coordinate space, not letterboxing.

Adds a real-ffmpeg test proving the push is visibly meaningful by the
end of the clip (not just "bytes changed", which a re-encode alone
would already satisfy), calibrated against that same clip's own
first-frame re-encode noise floor.
@upgradedev
upgradedev merged commit ea89d39 into main Aug 2, 2026
11 checks passed
@upgradedev
upgradedev deleted the feat/deterministic-camera branch August 2, 2026 09:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant