Import CLI operator runbooks from the lash repo

CONTEXT.md

@@ -16,7 +16,7 @@
 
 ## Autonomous CLI Testing
 
-- Agent-driven e2e runbooks live in `e2e/` (start with `e2e/RULES.md`): scenario documents an agent executes through `scripts/lash-operator.py`, judging CLI semantics against this file's Operator UI contracts. Scripted harnesses (`e2e/restate-postgres-workers`) remain deterministic gate evidence; runbooks are the judged layer on top.
+- Agent-driven e2e runbooks live in `runbooks/` (start with `runbooks/RULES.md`): scenario documents an agent executes through `scripts/lash-operator.py`, judging CLI semantics against this file's Operator UI contracts. The scripted CLI E2E harness remains deterministic gate evidence; runbooks are the judged layer on top.
 - Run `cargo test -p lash-cli --features test-provider --test cli_e2e` to exercise the real `lash` binary without live provider credentials.
 - The PTY smoke test launches interactive mode with a deterministic `test` provider, types a prompt, waits for rendered output, exits with `/exit`, and validates the generated UI trace/snapshot.
 - Run `scripts/lash-operator.py --provider test` for an agent-operated PTY session. It builds/launches the real `lash` binary with an isolated deterministic provider, then accepts commands such as `expect 15 Idle`, `type hello`, `key enter`, `screen`, and `lash-exit`.

runbooks/RULES.md

@@ -0,0 +1,154 @@
+# E2E Runbook Rules
+
+Read this before running any scenario in `runbooks/`. Each runbook links here and does
+**not** repeat these rules — it only adds its scenario-specific purpose, golden rules,
+phases, and scorecard.
+
+CLI E2E has **two layers**. The scripted deterministic harness (`cargo test -p lash-cli
+--features test-provider --test cli_e2e`) is gate **evidence**: it drives the real binary
+and asserts exact outcomes, and it stays code. `runbooks/` is the **agent-judged semantic
+layer** on top: you (the agent) drive the real `lash` binary through the PTY operator and
+judge the result with your own reasoning, gating on what the CLI actually renders. Keep
+the layers separate — a runbook never re-implements a scripted harness, and a scripted
+harness never asks for judgement.
+
+These are **agent-driven runbooks**, not scripts. Use judgement freely — but never skip a
+scenario's verification gates or the Abort rule below.
+
+## What you're testing
+
+You are testing **lash's operator surface**, not the model and not your own operator
+script. The scenario is only valid if the **CLI** — its input handling, runtime, and
+render — produces the observed result, not the literal text you typed. When a runbook
+asserts a rendered label (`◆ Will send in this turn`), the gate is that lash's queue
+projector drew it in response to the ingress you drove, not that the string appears
+because you typed it into the draft. When a scenario calls for isolation (a fresh session
+that must not see another session's transcript), a leak **voids the run**.
+
+The contract source for every operator behavior a runbook asserts is **CONTEXT.md →
+"Operator UI"**. Cross-check a claim against CONTEXT.md before gating on it. **Do not edit
+CONTEXT.md** — a divergence between it and the CLI is a finding to report, never to
+reconcile by editing the doc.
+
+## The operator surface
+
+CLI scenarios are driven through `scripts/lash-operator.py`, which launches the real `lash`
+binary in a child PTY and reads line commands from stdin or `--script FILE`.
+
+**Launch lines**
+
+- `scripts/lash-operator.py --provider test` — build + launch with an **isolated
+  deterministic** provider: a temp `LASH_HOME`, `config.json` pinned to
+  `{"active_provider":"test"}`, model `test/cli-e2e-model`, scenario `standard-echo`.
+- `--scenario <name>` selects the deterministic test-provider scenario. Available:
+  `standard-echo` (echoes `test-provider echo: <prompt>`; a bare prompt echoes
+  `interactive prompt`, the literal `hello from pty` echoes itself), `standard-slow-echo`
+  (as echo, but the prompt `slow initial prompt` sleeps ~2s before replying — a bounded
+  active-turn window), `standard-gated-escape` (the prompt `gated initial prompt` blocks
+  the turn **indefinitely** until a `gated-initial-release` marker file appears in
+  `LASH_HOME` — an unbounded active-turn window), and the RLM scenarios
+  `rlm-subagent-smoke` / `rlm-workspace-smoke` / `rlm-nonzero-exit-smoke` (need `-- -em rlm`).
+- `--no-build` launches the existing `target/debug/lash` instead of rebuilding.
+- `--lash-home DIR` fixes `LASH_HOME` so sessions and artifacts **persist across runs**
+  (sessions live in `DIR/sessions/*.db`). Omit it for a throwaway temp home.
+- `--trace PATH` passes `--debug-ui-trace PATH` to lash: a replayable UI-trace JSON plus a
+  final snapshot — the same on-disk artifact the PTY smoke test asserts on.
+- `-- <lash args>` — everything after `--` is forwarded to `lash` (e.g. `-- -em rlm`).
+- `scripts/lash-operator.py --provider real -- --model <provider/model>` drives the user's
+  configured provider/credentials. **Deliberate only** — it spends real tokens.
+
+**Commands** (one per line; `#` and blank lines are ignored)
+
+- `type TEXT` — type literal text into the draft (no newline).
+- `send ESCAPED` — write raw bytes; `\xNN`/`\t`/`\r` escapes are decoded (e.g. `send \x03`).
+- `key NAME [COUNT]` — send a named key `COUNT` times. Known: `enter`/`return`, `tab`,
+  `backtab`, `esc`/`escape`, `ctrl-c`, `ctrl-d`, `backspace`, `delete`, `up`, `down`,
+  `left`, `right`, `alt-up`, `alt-down`, `pageup`, `pagedown`, `home`, `end`. An unknown
+  key name is an operator error → Abort.
+- `expect [SECS] TEXT` — poll the rendered screen until `TEXT` appears (substring, matched
+  against raw or ANSI-stripped output). Default 10s. **This is a gate.** A timeout, or
+  lash exiting before the text appears, raises and stops the run.
+- `expect-re [SECS] REGEX` — as `expect`, but a multiline regex over the stripped screen.
+- `wait SECS` — sleep. **Never a gate** (see below).
+- `screen [LINES]` / `raw [LINES]` — print the last N stripped / raw lines for inspection.
+- `clear` — drop the capture buffer (so the next `screen`/`expect` only sees new frames).
+- `status` — print the child exit code (`None` while running).
+- `lash-exit [SECS]` — send `/exit`, then wait up to SECS for a clean process exit.
+- `kill` — SIGTERM/SIGKILL the child process group. `quit` — stop the driver.
+
+**Pre-flight** (every runbook's Phase 0): launch, and confirm the launched binary carries
+the deterministic provider. A startup failure `provider 'test' is not supported by this
+CLI build` means the binary was compiled **without** `--features test-provider` — rebuild
+(`cargo build -p lash-cli --features test-provider`) and make sure the launched
+`target/debug/lash` is that fresh artifact, then Abort/RCA the run and note it as a
+harness gap. Confirm the idle prompt renders (`expect 20 Message · / for commands`) before
+driving anything. If a scenario needs a key, scenario, or surface the operator cannot
+reach, that missing capability → Abort/RCA, noted as a harness gap (do not pretend around
+it).
+
+## Poll, don't sleep
+
+Turns, tool calls, subagent spawns, and queued-turn dispatch are all async and render over
+several frames. Gate on the **rendered outcome** with `expect <secs> <text>` — never `wait
+N`. Submit a turn, then `expect <secs> <assistant-marker-text>`; the footer status walks
+`Idle → Working → Thinking → Responding → Idle` (and `Running tool · <name>` for tool
+calls), so gate on the terminal render, not a fixed sleep. `wait` is only for letting the
+render settle before a `screen`, or spacing session-file writes so timestamp filenames
+don't collide — **never** to decide that async work finished. An `expect` timeout at a gate
+is a hard failure → Abort/RCA. `expect` also fails fast if lash exits first (`lash exited
+with N before '<text>' appeared`) — that is itself an Abort trigger.
+
+## Gate objectively before you judge
+
+Prefer an objective signal over eyeballing. In order of authority:
+
+1. **Rendered screen** — the `expect`/`screen` gate: footer status labels, the user `●` /
+   assistant `■` markers, queue previews (`◆ Will send in this turn` / `◇ Queued for next
+   turn`), overlay boxes (`Resume Session (n/m)`, the suggestion popup, `/help` / `/info`
+   document overlays). This is what the operator surface actually shows a user.
+2. **UI trace / snapshot** (`--trace PATH`) — the durable record of what was *submitted*,
+   independent of what rendered: ops like `user_turn`, `queue_current_turn_input` (Early
+   Injection), `queue_turn` (Next Full Turn). Use it to prove intent-vs-render (e.g. Enter
+   mid-turn recorded exactly one `queue_current_turn_input` and zero `queue_turn`).
+3. **On-disk state** — `LASH_HOME/sessions/*.db` (durable session evidence; message counts
+   drive the resume picker), and `LASH_HOME/test-provider-requests.jsonl` (the visible user
+   texts each provider call actually saw — objective proof of what reached the model, and
+   how many times).
+
+Run the structural gate **before** judging behavior. If the objective signal is missing
+(the label never rendered, the trace op is absent), the failure is upstream of anything you
+would judge — Abort/RCA, don't score the vibe.
+
+## When to STOP (Abort triggers)
+
+Stop immediately on **any** of:
+
+- an operator command error (an `ERROR ...` line / non-zero driver exit);
+- an `expect` / `expect-re` timeout at a gate;
+- lash exiting unexpectedly before a gate (`lash exited with N before ...`);
+- a **contract violation** — an isolation or invariant break: a supposedly-hidden empty
+  session surfacing in the resume picker, a rendered queue label that contradicts the
+  ingress you drove, a Runtime Process ending merely because its session was deleted, or a
+  document-overlay that won't close on `Esc`/`Ctrl+C`;
+- an assertion that contradicts the scenario's answer key.
+
+Do not push through, do not paper over, do not attempt a fix as part of the run.
+
+## How to REPORT
+
+**On abort — RCA, then stop:**
+1. **Stop.** Do not continue the scenario.
+2. **Capture evidence** — the failing operator command and its `ERROR` tail; the last
+   `screen`; the child `status`; the `LASH_HOME` path and any `--trace` artifact / session
+   db involved; and the exact gate string that failed.
+3. **RCA** — symptom → the CLI stage it broke at (input handling / suggestion+modal
+   dismissal / turn submit / runtime execution / render / session persistence) → root
+   cause → the evidence that proves it. Never stop at "the expect timed out."
+4. **Report and stop.** This is a diagnosis, not a repair. A divergence between an observed
+   behavior and CONTEXT.md or the docs is reported as a finding — **do not** edit the doc
+   or the code to make the run pass.
+
+**On success — score, don't vibe:** for each scored item, name the **specific rendered
+string** (or trace op / on-disk fact) the gate matched — no credit for vibes. Mark the
+objective gate (screen / trace / disk) separately from any judged behavior. Fill the
+scenario's scorecard verbatim.

runbooks/cancel-semantics/runbook.md

@@ -0,0 +1,138 @@
+# E2E Scenario: Cancel Semantics — The Ctrl+C Ladder
+
+> **Read [../RULES.md](../RULES.md) first** — operator surface, poll-don't-sleep, stop
+> triggers, and reporting/RCA conventions. This runbook only adds the scenario-specific
+> parts.
+
+**Purpose.** Prove the inline operator's cooperative cancel surface walks its documented
+`Ctrl+C` ladder, one rung per press, in order:
+**close a suggestion/overlay → cancel the active turn → clear a non-empty draft → quit only
+from an idle empty prompt.** Each rung is gated objectively, and a single escalating
+sequence of four presses peels all four layers so the *ordering* — not just each behavior in
+isolation — is what's under test.
+
+**Why this matters.** `Ctrl+C` is overloaded on purpose (CONTEXT.md → "Operator UI":
+"reserved for cancel/dismiss/quit semantics"). If the rungs fired out of order — quitting
+while a turn ran, or cancelling a turn when the user only meant to close a popup — the
+operator would destroy work on a keystroke people press reflexively. Copy is `Ctrl+Shift+C`
+precisely so `Ctrl+C` can own this ladder.
+
+## Scenario-specific golden rules
+
+1. **One rung per press, in order.** With a suggestion popup open, an active turn, and a
+   non-empty draft all present, the four presses must fire suggestion-dismiss → turn-cancel
+   → draft-clear → quit — never skip a rung, never quit early.
+2. **Cancel must reach a terminal state.** Rung 2 is not "the turn looked interrupted" — it
+   is the turn committing to `Manually interrupted.` and the footer returning to `Idle`.
+3. **Quit only from idle + empty.** The final press exits the process (clean status 0) only
+   because the prompt is idle and empty; any earlier press that exits is a violation.
+4. **Interrupted means a committed Lash terminal.** The rendered interruption is never
+   inferred from killing an engine invocation. Engine break-glass cancellation is an
+   operational failure and must not be presented as `Cancelled`. Exact-address durable
+   request and evidence gates live in the Lash repository's scripted Restate harness and
+   companion `runbooks/workbench-durable-stop/runbook.md` browser runbook.
+
+## Working material
+
+- Scenario `standard-gated-escape`, holding prompt `gated initial prompt` (an unbounded
+  active turn, so rung 2 can't win a race by accident). Typing `/` opens the slash-command
+  suggestion popup for rung 1; that `/` draft is the non-empty draft rung 3 clears.
+
+## Phase 0 — Pre-flight
+
+Per [../RULES.md](../RULES.md). Launch `scripts/lash-operator.py --provider test --scenario
+standard-gated-escape`, confirm the deterministic provider, gate the idle prompt
+(`expect 20 Message · / for commands`).
+
+## Phase 1 — Arm all four layers
+
+Start the holding turn, confirm it is active, then open the suggestion popup on top of it:
+
+```
+type gated initial prompt
+key enter
+expect 20 gated initial prompt
+expect 15 Thinking
+type /
+expect 8 Reset conversation
+```
+
+State now: active turn (`Thinking`), a non-empty draft (`/`), and the suggestion popup open
+(the bordered list showing `/clear  Reset conversation`, `/controls`, …). All four ladder
+layers are armed.
+
+## Phase 2 — Walk the ladder, one `Ctrl+C` at a time
+
+**Rung 1 — close the suggestion overlay.** The popup closes; the `/` draft and the active
+turn survive.
+
+```
+key ctrl-c
+wait 1
+screen 20
+```
+
+Gate: the suggestion border is gone, the input still shows `❯ /`, and the footer still reads
+`Thinking`. (Absence of the popup — inspect the `screen`; the border box and `Reset
+conversation` must be gone.)
+
+**Rung 2 — cancel the active turn.** The turn commits to its interrupted terminal state.
+
+```
+key ctrl-c
+expect 20 Manually interrupted
+expect 10 Idle
+```
+
+Gate: `Manually interrupted.` renders and the footer returns to `Idle`. A turn that keeps
+running, or errors instead of interrupting, → Abort/RCA (turn cancel). The `/` draft still
+survives.
+
+**Rung 3 — clear the non-empty draft.** The `/` draft is discarded; the idle placeholder
+returns.
+
+```
+clear
+key ctrl-c
+wait 1
+expect 5 Message · / for commands
+```
+
+Gate: the input returns to the `❯ Message · / for commands` placeholder (empty draft). The
+process is **still running** — confirm with `status` (`None`).
+
+**Rung 4 — quit from idle + empty.** Only now does `Ctrl+C` exit.
+
+```
+key ctrl-c
+wait 1
+status
+```
+
+Gate: the child has exited — `status` reports `STATUS 0`. (The `wait` only lets the process
+reap so `status` doesn't poll a still-shutting-down child; the exit — not the sleep — is the
+gate. A *clean* exit here is the expected outcome of this rung, not an Abort trigger.)
+
+## Phase 3 — Score
+
+| Item | Objective gate | Verdict | Notes |
+|------|----------------|---------|-------|
+| Rung 1: suggestion closes | popup + `Reset conversation` gone; `/` draft + `Thinking` survive |  |  |
+| Rung 2: active turn cancels | `Manually interrupted.` + footer `Idle` |  |  |
+| Rung 3: draft clears | `❯ Message · / for commands` returns; process still alive |  |  |
+| Rung 4: quit from idle empty | child exits `0` |  |  |
+| Order held | exactly four presses, one rung each, none out of order |  |  |
+| Semantic layer | interruption is a committed cooperative turn result, not engine break-glass |  | rendered terminal + companion gate references |
+
+**Aggregate:** did four presses peel four layers in the documented order, with the turn
+reaching a real interrupted terminal state and the quit firing only from idle + empty.
+
+This CLI scenario deliberately does not invent an external cancellation API. The Lash
+repository additionally gates the first-party primitive with `just
+restate-postgres-workers-e2e` (exact request, replay, race, terminal evidence, and
+break-glass distinction) and `runbooks/workbench-durable-stop/runbook.md` (browser/API
+agreement across web-process restart).
+
+---
+
+_Stop triggers and the Abort/RCA + reporting protocol are in [../RULES.md](../RULES.md)._