Skip to content

fix: detect_anchors recovers consecutive instant-turn submissions via scripted timing#20

Merged
htlin222 merged 2 commits into
mainfrom
fix/detect-anchors-consecutive-instant
Jul 5, 2026
Merged

fix: detect_anchors recovers consecutive instant-turn submissions via scripted timing#20
htlin222 merged 2 commits into
mainfrom
fix/detect-anchors-consecutive-instant

Conversation

@htlin222

@htlin222 htlin222 commented Jul 5, 2026

Copy link
Copy Markdown
Owner

Closes #18.

The bug

Two client-side-only turns back-to-back (e.g. /context then /fast on,
with no real model-invoking turn between them) have no "thinking gap" for
detect_anchors.py's pixel-based turn detector to split on. The merged
input-band signal reads as ONE continuous group instead of two, so the
detector undercounts prompt submissions and author.py fails with:

detected 3 prompt submissions, expected 4. Tune the input band / thresholds, or the recording differs.

The fix

detect_turns() already receives each turn's instant flag (written by
gen_capture_tape.py's render() into capture.json's per-turn plan, per
PR #16 / #19) via the existing turns parameter — no call-site signature
change was needed.

When len(groups) < n, a new recovery path checks whether the shortfall is
exactly explained by one or more maximal runs of consecutive instant
turns collapsing into a single detected group. If so, it splits the merged
group back apart using the known scripted timing — each instant turn's
settle Sleep + pre_enter + the next turn's type_dur — instead of trying
to re-detect the split points from pixels (there's no pixel signal left to
re-detect from). If the shortfall isn't fully explained this way, it still
raises: this is a targeted recovery for a known deterministic failure mode,
not a blanket excuse to swallow a genuine detection miss. The existing
len(groups) > n "strongest peaks" fallback is untouched.

While rebasing onto main I picked up the just-merged #17/#19 fix (native
CLI menu turns, e.g. /theme), which gives those turns the same
instant flag/shape but can use a longer settle (NATIVE_MENU_SETTLE
under --tmux) than the no-menu INSTANT_SETTLE. My recovery originally
assumed a single hardcoded settle constant, which would have silently
mis-timed (not raised, just produced wrong offsets for) a merged run
containing a native-menu turn. Fixed by having gen_capture_tape.py record
the settle it actually used per-turn (turn_plan["settle"]) and having the
recovery read that value (falling back to INSTANT_SETTLE for older
capture.json plans that predate the field).

Tests

  • tests/test_detect_anchors.py::test_detect_turns_recovers_merged_consecutive_instant_group
    — reproduces the exact 3-vs-4 mismatch from the issue with a synthetic
    merged input-band signal; confirmed it fails with the original SystemExit
    before the fix.
  • tests/test_detect_anchors.py::test_detect_turns_recovers_merged_group_using_per_turn_settle
    — proves the recovery uses each turn's own settle value, not a hardcoded
    constant (regression test for the native-menu interaction above).
  • tests/test_detect_anchors.py::test_detect_turns_raises_when_instant_shortfall_not_fully_explained
    — confirms a shortfall NOT fully explained by instant merges still raises.
  • All pre-existing tests (including the len(groups) > n guided-selection
    path and the plain under-detection raise) still pass unmodified.

Local suite: 131 passed, 0 failed (.venv/bin/python -m pytest tests/ -q).

Not done

Did not run a real end-to-end record-session (capture → author → splice →
overlay → panel → lint) against a live claude session — that needs several
minutes plus real API usage. vhs/tmux/ffmpeg/edge-tts/claude are
all present in this environment, so it's possible to run later if desired,
but per the task's own guidance the unit tests (with a faithful synthetic
repro of the exact reported mismatch) are the primary bar here.

🤖 Generated with Claude Code

htlin222 added 2 commits July 5, 2026 15:06
…ons using known scripted timing

Two client-side-only turns back-to-back (e.g. "/context" then "/fast on") have
no real thinking-gap between them, so the input-band pixel signal never dips
long enough to split them -- they read as ONE continuous group instead of two,
undercounting detected prompt submissions and raising in author.py (#18).

The fix adds a targeted recovery for the len(groups) < n case: when the
turns.json plan already flags a run of >=2 consecutive turns as "instant"
(written by gen_capture_tape.py's render(), consumed via plan["turns"]) and the
shortfall is EXACTLY explained by that run collapsing into one detected group,
split the merged group back apart using the deterministic scripted timing
(INSTANT_SETTLE + pre_enter + each turn's type_dur) instead of re-detecting
split points from pixels -- there is no pixel signal left to re-detect from.
If the shortfall isn't fully explained this way, it still raises: this is a
recovery for a known, deterministic failure mode, not a blanket excuse to
swallow a genuine detection miss.

Regression tests added in tests/test_detect_anchors.py reproduce the exact
3-vs-4 mismatch from the issue with a synthetic merged input-band signal, and
confirm the existing "shortfall not explained" case still raises.

Closes #18.
…tion, not a hardcoded constant

The just-landed #17/#19 fix (native CLI menu turns, e.g. "/theme") gives
those turns the SAME turn_plan["instant"] = True flag/shape as the no-menu
INSTANT_COMMANDS from #16, but they can use a LONGER settle
(NATIVE_MENU_SETTLE under --tmux) than the no-menu INSTANT_SETTLE. My
detect_anchors.py recovery (previous commit) assumed every "instant"-flagged
turn used the module's INSTANT_SETTLE constant -- correct for #18's own
"/context" + "/fast on" repro, but silently wrong (not raising, just
mis-timed) for a merged run that includes a native-menu turn.

gen_capture_tape.py now records the settle it actually used per-turn in
turn_plan["settle"]; detect_anchors.py's split reads that value
(turns[idx].get("settle", INSTANT_SETTLE)) instead of assuming one constant,
falling back to the old default for capture.json plans predating this field.

Added a regression test with a mixed run (a native-menu-settle turn followed
by a plain-instant turn) proving the recovery uses the per-turn value.
@htlin222
htlin222 merged commit 02b47ae into main Jul 5, 2026
5 checks passed
@htlin222
htlin222 deleted the fix/detect-anchors-consecutive-instant branch July 5, 2026 07:11
htlin222 added a commit that referenced this pull request Jul 5, 2026
… same settle duration (#22)

#20's recovery treated a whole maximal run of consecutive `instant`-flagged
turns as one atomic collapse candidate (either all merge into one detected
group, or none do). Real pixel behavior isn't atomic once settle durations
differ within a run: a NATIVE_MENU_SETTLE (10s) turn is long enough to stay
its own visually-distinct group even sandwiched between short INSTANT_SETTLE
(1.8s) turns, so only the short-settle sub-run actually merges (issue #21).

Fix: if the whole-run collapse doesn't exactly explain the shortfall, retry
using maximal SAME-settle sub-runs within it as the collapse candidates
instead (e.g. "/theme" stays isolated, "/context"+"/fast on" merge). The
original whole-run attempt runs first and unchanged, so every existing
recovery path (including the per-turn-settle test where a run already fully
collapses) is untouched; the sub-run fallback only kicks in for shortfalls
the whole-run math can't already account for.

Closes #21.


Claude-Session: https://claude.ai/code/session_012EwptawkV4dcGhoGefzxgV

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Back-to-back instant-command turns collapse into fewer detected prompt-submissions in detect_anchors.py, failing the author stage

1 participant