operator_event is never emitted by any framework code path

[jeqcho]

Problem

transcript.py defines operator_event (and TrialRecord.operator_judgement exists), but grep shows no framework code path ever emits/sets them — so operator_scorer() always returns False for framework-run rollouts. Operator judgement currently only works when an embodiment adapter sets termination_reason/metadata itself (as the so101/yam adapters do via their own operator loops).

(approval_event has the same gap on main today, but the review-fixes forward-port adds its emission from the rollout — this issue is about the operator loop.)

Decision needed

Design the operator-in-the-loop path: where does the framework solicit an operator verdict (end of trial? via an OperatorIO protocol?), who records operator_judgement, and how does it reach the scorer. Until then, docs should state that operator_scorer requires adapter-side support.

🤖 Generated with Claude Code

[Tushar2604]

@jeqcho Investigated this against PR #66, which is in flight and already emits operator_event for the ad-hoc (--instruction) path when an embodiment self-reports termination_reason (_reuse_embodiment_verdict). Once that merges, the remaining gap is narrower than it looked when this issue was filed:

What stays broken after #66: a registered --task (or the new eval-set) run with the operator scorer still never captures a verdict — not even from a self-confirming embodiment that already knows the answer. That's silent data loss for exactly the human-supervised real-hardware benchmark sessions the field reports (#50, #53) describe. It's not an oversight: plan 0001 R6 / plan 0005 §3.4 deliberately keep registered-task runs unprompted by default so a 50-scene × 5-epoch benchmark can't produce 250 modal prompts. So this isn't "the framework never emits operator_event" — it's "there's no opt-in for the one case (registered task + real hardware + a human actually watching) where prompting is wanted."

Proposed fix, scoped to that gap:

Add an explicit opt-in flag — --prompt-operator — to run (and eval-set), off by default. When set and the resolved scorers include operator, wire the same dispatch #66 already builds: reuse an embodiment-reported verdict if one exists, else fall back to the interactive TTY prompt (respecting --no-prompt as the override). This reuses _reuse_embodiment_verdict/_prompt_operator as-is rather than inventing a second mechanism.
No changes to eval()/rollout() core, and no new OperatorIO protocol — before_scoring was already built in plan 0005 as the seam for exactly this; registered-task runs just never had a caller wire it. Direct Python API callers (KitchenBench scripts, etc.) keep full control by passing their own before_scoring, unaffected.
Docs fix regardless of the above: operator_scorer's docstring and the CLI guide should say plainly that a registered --task run needs --prompt-operator (or adapter-side self-confirmation) to ever produce a judgement — today it silently scores "no operator judgement recorded" forever, which is the confusing part of what this issue reported.
This is blocked on #66 landing first, since it reuses the helpers that PR introduces rather than re-deriving them. Want me to go ahead on this once #66 merges, or would you rather keep registered-task runs permanently unprompted and just fix the docs?