Skip to content

fix(illustration): burn the disclosure into pixels, not just the prompt - #24

Merged
upgradedev merged 1 commit into
mainfrom
fix/illustration-watermark-burn-in
Jul 30, 2026
Merged

fix(illustration): burn the disclosure into pixels, not just the prompt#24
upgradedev merged 1 commit into
mainfrom
fix/illustration-watermark-burn-in

Conversation

@upgradedev

Copy link
Copy Markdown
Owner

Summary

The illustration clip's "not evidence" disclosure was only requested via the generation prompt (illustration_prompt's Overlay text: '{DISCLOSURE}' instruction), never guaranteed. A live render proved the model can simply ignore it: case live-forensic-retry-0bb32eeb (provider genblaze, degraded: false) produced a clip with the disclosure absent from every one of four sampled frames.

This adds a deterministic, post-generation burn-in step so the sealed bytes always carry the disclosure, independent of the model's compliance.

How it works

  • New module src/claimscene/watermark.py:
    • render_caption_png(text) renders the disclosure as a small RGBA caption image with Pillow (legible white text on a semi-transparent dark backing bar, auto-sized to the text).
    • burn_still_watermark(data) composites that caption onto the still with Pillow (no ffmpeg needed).
    • burn_clip_watermark(data) overlays the same pre-rendered caption image onto every frame of the clip via ffmpeg's overlay filter.
    • ffmpeg's drawtext filter was deliberately not used: it needs a resolvable font (fontfile= or fontconfig), and the production image installs ffmpeg with --no-install-recommends and no font package. drawtext would very likely fail there with "Cannot find a valid font", i.e. reintroduce the exact class of silent, looks-right-in-code failure this PR exists to close. Rendering the caption once with Pillow (which ships its own bundled font) and compositing it as a plain image sidesteps that entirely.
  • pipeline.py now generates the still and clip as before, burns the watermark into each, and stores/seals the post-overlay bytes. The raw (pre-burn) still is still what feeds the image-to-video step, so the Genblaze provider's chained-output optimisation (URL reuse keyed by the still's original sha256) is unaffected.
  • The prompt's Overlay text: instruction in report.py is unchanged (belt and braces).

Fail-safe (chosen deliberately: ship + honest flag, not fail-loud)

Both burn functions never raise. On any failure (undecodable input, ffmpeg absent, a subprocess error, empty output) they return the original, untouched bytes plus burned=False and a short error. The pipeline always stores and seals whichever bytes come back.

I considered "fail the render loudly" (raise, no case produced) but rejected it:

  • api.py::_run_render already catches an illustration failure and re-seals the whole case with the offline fake provider. If burn-in raised for a live render, a real, successfully-generated clip would be discarded and replaced by a generic fake one over what is, structurally, a cosmetic post-processing failure — worse for the user, not better.
  • Every offline/CI code path uses FakeMediaProvider, whose synthetic bytes are not real media and can never be burned. "Fail loudly" as the default would make the entire offline pipeline (and most of the test suite) unable to produce a case at all.
  • The aggregate verification receipt (see below) already makes a live burn-in failure loud in the sense that matters: it flips verify_all(...).success to False, without destroying the case or its other, perfectly good artifacts.

What the manifest now records

manifest["illustration"] gains:

  • watermark_text — always the sealed DISCLOSURE constant.
  • watermark_burned / watermark_error — for the clip.
  • still_watermark_burned / still_watermark_error — for the still.

Verification / readiness

  • provenance.verify_all gains two new structural checks (mirroring the existing structural.schematic_watermark check): structural.illustration_watermark_burned and structural.illustration_still_watermark_burned. Each requires watermark_text == DISCLOSURE unconditionally, and requires burned is True whenever degraded is False (a live illustration). An offline/degraded illustration is exempt from requiring True (burning synthetic fake bytes is not meaningful) but must still carry an explicit boolean, never a silent omission. This is what keeps every existing FakeMediaProvider-backed test green with zero special-casing, while making the exact reported production gap fail loudly in the receipt.
  • scripts/readiness.py gains honest_media.illustration_watermark_burned: drives the real burn functions (not mocks) end to end, a real Pillow still burn that visibly changes bytes, and the fail-safe path on genuinely undecodable input for both burners, plus structural sealing via the standard offline pipeline run. Consistent with the gate's existing convention, it does not hard-depend on ffmpeg (every other check there uses animate=False for the same reason); the clip's real-success path is proven in the pytest integration suite instead.

Tests

  • tests/unit/test_watermark.py (new, 15 tests): caption rendering determinism and the exact-DISCLOSURE-by-default property, pure command/filter construction, still burn success + fail-safe, clip fail-safe (undecodable input, ffmpeg-absent via monkeypatch, a vanishing ffmpeg binary), and a real @needs_ffmpeg-gated success test mirroring tests/integration/test_video.py's own style.
  • tests/integration/test_illustration_watermark_burn.py (new, 2 tests): a real CasePipeline run with a non-"fake"-prefixed media-provider double that returns genuinely decodable bytes, asserting the manifest records the burn-in, the stored/sealed hashes match the post-overlay bytes, the raw still still fed the video step, and verify_all passes; plus the fail-safe path through the full pipeline with undecodable provider output, asserting the case still seals with an honest False and verify_all fails loudly.
  • tests/unit/test_review_provenance.py (existing file, purely additive, 4 new tests): the aggregate receipt passes when degraded, fails when a live illustration lacks a burned watermark (the exact reported bug), and fails when the text/flag is silently absent.

No existing test was modified. tests/security/ is untouched. DISCLOSURE/WATERMARK constants, the SceneGraph schema, and the forensic-register prompts are untouched.

One deliberate presentation-only detail: Pillow's bundled default font has no glyph for the em dash in DISCLOSURE and silently draws a tofu box for it. The caption rendering substitutes a plain hyphen for legibility; the sealed watermark_text field and all hash/structural checks still use the literal, unmodified DISCLOSURE constant.

Test plan

  • ruff check clean on all new/changed files
  • New unit + integration tests pass locally (real ffmpeg available locally, so the real burn-in success paths were exercised for real, not skipped)
  • Full existing suite passes locally alongside the changes (410 collected across all files not affected by a pre-existing, unrelated local Windows namespace-package quirk in 3 untouched files; confirmed via git diff that those 3 files are byte-identical to main)
  • CI green on all jobs (polling after opening this PR)

illustration_prompt's "Overlay text" instruction is a request the
generative model is free to ignore, and a live render proved it does
(case live-forensic-retry-0bb32eeb, provider genblaze, degraded: false,
disclosure absent from every sampled frame).

Add a deterministic, post-generation burn-in step (watermark.py):
Pillow composites the disclosure caption onto the still; ffmpeg
overlays the same pre-rendered caption image onto the clip via the
overlay filter (drawtext was deliberately avoided -- it needs a
resolvable font, and the production image installs ffmpeg with none).
Fail-safe throughout: any decode/subprocess failure ships the original
bytes with an honest, sealed watermark_burned=false rather than a
silent or false-positive claim. The still still feeds the video step
with its raw bytes, preserving the Genblaze chained-output optimisation.

The manifest now records watermark_text / watermark_burned /
watermark_error for both the still and the clip, and
provenance.verify_all gains matching structural checks (mirroring the
existing schematic-watermark check) that fail whenever a live
(non-degraded) illustration shipped without a burned-in disclosure.
The readiness gate exercises the real burn-in and its fail-safe path
with real evidence, offline and without a hard ffmpeg dependency.
@upgradedev
upgradedev merged commit 5b94ad3 into main Jul 30, 2026
11 checks passed
@upgradedev
upgradedev deleted the fix/illustration-watermark-burn-in branch July 30, 2026 07:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant