fix: line-wrap-split typing masquerades as an extra prompt submission#23
Merged
Conversation
…detect_turns A long, multi-line-wrapping prompt was mis-detected as TWO (or more) separate prompt submissions, corrupting neighbouring turns' hard-segment boundaries. Root cause (confirmed against a real --robust capture, not guessed): the auto-located "input band" in signals() is a FIXED, ~5-row window near the bottom of the frame. When a long prompt wraps onto a new line, the input box grows a row — the just-typed line's content shifts out of that fixed band and the freshly wrapped (still-blank) line takes its place. That produces a sharp brightness drop followed by a climb as the new line fills in — pixel-for-pixel the same "typing fills the box, Enter clears it" shape detect_turns uses to mean a real submission, even though the user never stopped typing. Measured on the repro (27-word/3-line prompt): two "submits" 1.52s apart (frames 386 and 424), both >=0.92 of the frame's peak brightness — real typed text, not the weaker spinner/response leakage the existing GUIDED-selection logic already knew to filter out. With 4 raw groups detected for 3 scripted turns, "keep the 3 strongest peaks" kept BOTH wrap pieces and discarded the turn actually mapped to the discarded group (`/theme`'s own, genuinely weaker, native-menu peak) — corrupting that turn's submit AND starving the preceding turn's `done` search window (bounded by the second wrap piece's start, ~1.5s after its real submit instead of after the real ~8s think+response). The resulting hard segment was too short to contain even the scripted typing duration, and the freeze frame used for the next turn's "pre" segment sourced from mid-sentence — a half-typed prompt held statically for over 20s while unrelated narration played over it. Fix: fuse adjacent groups into one BEFORE any n-vs-count reasoning when they are (a) short apart (<=2s, generous vs. the ~1.5s dip measured — a real turn boundary always has a multi-second settle/think/PAD gap), (b) both genuinely near-max (not moderate spinner leakage), AND (c) the gap between them is RISING rather than idle (a real Enter leaves the box idle; a wrap dip keeps climbing because the same burst is still typing). All three conditions matter — gap-length or peak-strength alone also fire on two short, fast, genuinely distinct real turns. Also clamp `done` for the video's LAST turn to never exceed the raw capture's own length: fixing the above correctly re-surfaced a turn (in this same demo, `/theme`) whose group runs off the very end of the analyzed video because the recording stopped before a real Enter-clear was ever observed (an open native menu at capture end) — previously masked by the wrap-split bug assigning it a different, wrong group instead. Leaves one frame of slack so a degenerate zero-length tail freeze still has a real frame to source from. Per CONTRIBUTING.md's testing ethos, this failure mode is reproduced as synthetic pixel-array unit tests (matching the measured real values) before the fix, in tests/test_detect_anchors.py. The real repro artifacts (a genuine --robust capture exhibiting this bug, plus the ledger it produced and the hook-timestamped ground truth) were used to diagnose and verify the fix end-to-end (author.py + splice.py rebuilt successfully; the previously frozen half-typed frame is gone) but are not committed — the synthetic tests already reproduce the exact pixel shape, and the repo's stated test ethos keeps tests/ free of external tools or large binary fixtures (*.mp4 is git-ignored repo-wide save one existing hero asset). 134 pre-existing tests still pass; 5 new tests added (139 total). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012EwptawkV4dcGhoGefzxgV
4 tasks
htlin222
added a commit
that referenced
this pull request
Jul 5, 2026
#23 postmortem) (#24) PR #23 fixed a real bug where a long wrapping prompt fooled the pixel-based turn-boundary detector into producing a turn whose "hard" segment was mathematically impossible (shorter than the deterministic scripted typing time alone) — silently corrupting the ledger. It was only caught by a human eyeballing video frames, not by the pipeline itself. This adds two independent, complementary guards so the SAME class of bug (and others shaped like it) gets caught automatically next time, instead of waiting for another reactive patch. Layer 1 (detect_anchors.py, detect_turns()): a deterministic self-consistency floor requiring no external data. submit - typing_start must equal exactly type_dur + pre_enter (the scripted time to type the prompt and reach Enter) — a real submit can never be faster than that. The existing `max(0.0, ...)` clamp used to silently absorb a violation of this; now it raises immediately, naming the turn and the impossible numbers. Layer 2 (author.py, main()): a best-effort cross-check of the pixel-detected per-turn response window (done - submit) against session-timeline.jsonl's real UserPromptSubmit->Stop wall-clock delta for the same turn, when that file is available and its turns can be unambiguously correlated (mirrors panel.py's existing instant-turn-aware matching). This catches a DIFFERENT class of error Layer 1 cannot: a wrong-but-internally-self-consistent group selection that still satisfies Layer 1's floor. It degrades silently on a missing/malformed/uncorrelated timeline — never a hard requirement — and only raises on a wild disagreement (tolerance: max(5.0s, 50% of the real delta), chosen generously against detect_turns()'s own documented "many seconds" video/wall-clock drift risk during heavy output, while still comfortably catching a mismatch of the #23 bug's scale; validated against one real 110s-recording data point showing ~7% pixel-vs-wallclock agreement). Layer 2 intentionally lives in author.py, not detect_anchors.py: detect_turns already argues wall-clock timing can't be the PRIMARY source of truth (video drift), and author.py is the orchestration layer that already knows the demo path, per CONTRIBUTING.md's pure-logic/thin-I/O split. 12 new regression tests (3 Layer 1, 9 Layer 2); full suite green (151 passed, up from the pre-existing 139), including every #16-#23 recovery-path regression test unmodified. Claude-Session: https://claude.ai/code/session_012EwptawkV4dcGhoGefzxgV Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
A real
record-session --robustdemo (3 turns: a short Q&A, then a long27-word/3-line prompt, then
/theme) produced a badly desynced final video:turn 1's (the long-prompt turn's) hard segment came out only 6.16s long — too
short to even contain its own scripted typing duration (6.74s), let alone the
real ~8.1s of Claude thinking+response that followed. Downstream, the frozen
freeze-frame chosen for the next turn's "pre" segment sourced from mid
sentence, so a half-typed prompt sat frozen on screen for ~23s while
unrelated narration ("turn 1 done" / "now watch
/theme") played over it.Root cause (confirmed against the real capture's pixels, not guessed)
detect_anchors.signals()auto-locates the "input band" it tracks as aFIXED, ~5-row-tall window near the bottom of the frame — deliberately tight so
response content doesn't leak in. That's fine for a short, single-line
prompt. But when a prompt is long enough to wrap onto a new line, the
input box grows a row: the just-typed line's content shifts out of that fixed
band and the freshly wrapped (still-blank) line takes its place. That's a
sharp brightness drop followed by a climb as the new line fills in —
pixel-for-pixel the same "type-then-Enter-clears" shape
detect_turnsusesto mean a real submission, even though the user never stopped typing.
Measured on the real repro: one continuous typing burst produced two
"submits" 1.52s apart (raw frames 386 and 424 @ 12.5fps), with peaks at 100%
and 93% of the frame's global max brightness — i.e. genuinely strong, not
the weaker spinner/response leakage the existing "keep the N strongest peaks"
disambiguation already knew to filter out. With 4 raw groups detected for 3
scripted turns, that logic kept both wrap pieces and discarded the turn
that actually owned the 4th (weaker, correctly-detected) group — corrupting
that turn's own submit and starving the preceding turn's
donesearchwindow (which is bounded by the next detected group's start).
Confirmed via frame-by-frame extraction (
ffmpeg -ss ... -frames:v 1): atthe exact raw timestamps the old code treated as "Enter pressed" for the long
prompt, the terminal screenshot still shows the sentence being actively
typed, with a live cursor — no Enter has happened yet.
The fix
In
detect_anchors.detect_turns(), fuse adjacent detected groups into onebefore any n-vs-count reasoning, when all three hold:
boundary always has a multi-second settle/think/PAD gap; back-to-back
instant turns are handled by the separate, existing
_recover_merged_instant_groupspath and never show a partial dip likethis);
spinner leakage);
at its post-clear floor for the whole gap; a wrap dip instead climbs
almost immediately because the same typing burst is still going. This
is what actually distinguishes a wrap dip from two short-but-genuinely-
distinct back-to-back turns (which can also have a short gap with
near-max peaks either side) — gap-length or peak-strength alone aren't
enough on their own.
Fixing this correctly re-surfaced a second, separate, pre-existing edge case
in this same demo: the final turn (
/theme, a native menu) never shows areal Enter-clear at all before the raw capture ends (the recording stops
while the menu is still open), so its detected group runs to the raw video's
very last analyzed frame.
submitin that case just means "recordingended here" — the existing
+0.3sfloor ondonewas pushing the segmentpast the video's own reported length, which broke the freeze-frame splice
downstream (no frame exists past the file's own duration). Added a matching
clamp (with one frame of slack, mirroring
BootStrategy's existing "at end"convention in
strategies.py) sodonefor the last turn never exceeds theraw capture's own length.
Testing
Per
CONTRIBUTING.md's testing ethos, the exact failure mode is reproducedas synthetic pixel-array unit tests (using the real measured peak
values/gap/timings) in
tests/test_detect_anchors.py, verified to failbefore the fix and pass after:
test_merge_same_burst_groups_fuses_wrap_split_typingtest_merge_same_burst_groups_leaves_distinct_turns_alonetest_merge_same_burst_groups_leaves_weak_spinner_leak_alonetest_detect_turns_wrap_split_no_longer_starves_neighbouring_turnstest_detect_turns_last_turn_done_never_exceeds_raw_video_length134 pre-existing tests still pass unmodified; 139 total, all green.
I additionally used a real
--robustcapture exhibiting this exact bug (notcommitted — see below) to diagnose the root cause via direct pixel/frame
inspection, and to verify end-to-end: re-running
detect_anchors+author.pysplice.pyagainst it now produces a properly-sized hard segment for thelong-prompt turn (18.44s vs. the previous 6.16s, correctly containing the real
submit and enough room for the actual response) and the full splice pipeline,
which previously crashed on the newly-surfaced
/themeedge case, nowcompletes successfully.
On the real repro fixture
I did not commit the real capture used to diagnose this (
terminal_raw.mp4+its capture/script/ledger/timeline siblings). The repo's
.gitignoreexcludes
*.mp4repo-wide save one existing hero asset, andCONTRIBUTING.mdstates the unit test suite intentionally needs no externaltools (
ffmpeg/vhs/tmux) to run. The synthetic tests above alreadyreproduce the precise pixel shape (constructed from the real measured
peak/gap/climb values), so I judged a large committed binary fixture wasn't
worth the added repo weight/precedent. Happy to reconsider if maintainers
would rather have it as a
-f-added fixture.Scope note: is this robust/tmux-specific?
Inconclusive either way from what I could test in the time available — I
did not get to build a comparable standard (non-
--robust, non-tmux)capture with a similarly long, wrapping second-turn prompt to compare
against. The mechanism itself (a fixed-row input-band tracker losing a line
to box growth on wrap) is a property of
signals()'s pixel-band tracking,not of
--robust/tmux capture specifically, so I'd expect it to reproduceunder standard VHS capture too given a long enough wrapping prompt — but
that's inference, not something I verified against a second real capture.
Worth confirming with a standard-mode repro if it matters for prioritization.
Issue-filing for this bug is left to the orchestrator/maintainer if still
wanted — no existing issue number is referenced here.
Co-Authored-By: Claude Sonnet 5 noreply@anthropic.com