cancellation: decide who reaps the process groups, worktrees, map slots, and claim locks a dropped future leaves

[macanderson]

What decision this asks for

Pick the ownership rule for resources created mid-flight when a future is dropped — Ctrl-C, a caller select!, a session timeout, or Esc in the deck — and then apply that one shape everywhere. The blocker is the same at every site: Drop cannot await, and no binary in this workspace installs a SIGINT/SIGTERM handler today, so tokio::spawn-from-Drop silently does nothing during runtime shutdown — precisely the case being fixed. The three candidate shapes are (a) a synchronous Drop guard that blocks on a sync syscall, (b) prune-on-next-start, and (c) an explicit runner/finally that owns teardown. Choose one, decide whether cancellation is in scope for stella-media at all, and rule separately on the escape hatch for worktrees and file_locks rows that have already leaked into user workspaces.

Every item below was re-checked against origin/main (092b2dda) before filing: all eleven are still present.

Items

• verify_done leaks a registered git worktree — a cancelled verify_done never reaches cleanup_shadow, leaving an entry in .git/worktrees plus a /tmp directory that no later run removes. stella-tools/src/verify.rs:285. Deferred because the Fix text offers two remediations without choosing (an RAII guard that runs cleanup_shadow or git worktree prune on next start) — an owner call, not a mechanical port. cleanup_shadow is async (three awaited git subprocesses plus tokio::fs::remove_dir_all), and the prune-on-start variant reaches outside stella-tools into a startup path. A witness would have to abort a real VerifyDone::execute in the window between git worktree add and cleanup, which is racy by construction. (audit stella-tools: the bash tool orphans its detached process group on cancellation #550, deferred by fix(stella-tools): kill the bash and custom-tool process groups on cancellation #582)
• The other unguarded pre_exec(setsid) spawn site — stella-cli/src/agent/tools.rs:476 has the same orphaned-process-group shape as the bash/run_custom leak that fix(stella-tools): kill the bash and custom-tool process groups on cancellation #582 fixed, but no GroupKillGuard. Deferred because it was not cited by stella-tools: the bash tool orphans its detached process group on cancellation #550 and lives in a different crate; it deserves its own witness rather than a drive-by. (audit stella-tools: the bash tool orphans its detached process group on cancellation #550, deferred by fix(stella-tools): kill the bash and custom-tool process groups on cancellation #582)
• ContextStore's detached warm task is never aborted — add impl Drop for ContextStore that aborts the background warm JoinHandle, plus a cancellation check between warm batches (for chunk in pending.chunks(BATCH) has no deadline, budget, or abort check). stella-context/src/store.rs:384. Deferred because aborting on drop is a behavior change for any caller relying on detached completion, and it interacts with await_warm's documented join-once semantics. (audit Batched audit cleanup: area:context — 15 items #558, deferred by chore(stella-context): document the public surface and cache store_kinds #589)
• MediaError::Cancelled advertises a capability the crate does not have — documented as "the request was cancelled (e.g. Ctrl-C draining the turn)" but never constructed in stella-media or stella-tools: no cancellation token, no select!, no abort path on any media call. stella-media/src/error.rs:65. The remediation is a fork — thread a CancellationToken (or &mut oneshot::Receiver) through generate_image / poll_video / download_bytes and select! it, or delete the variant and the module-doc claim. Deferred because the Fix text ruled out the plumbing (an async/signature change across the MediaProvider trait) and then offered the opposite option; pure owner judgment on whether cancellation is in scope here. (audit Batched audit cleanup: area:media — 10 items #565, deferred by chore(stella-media): fix the live-smoke gate, preview labels, branding #591)
• MCP pending-map slot leaks per dropped request — reclaim the pending entry when a caller drops the request future; today every timed-out call leaks one map entry plus its oneshot sender until EOF or close(). stella-mcp/src/stdio.rs:135 (the module doc on origin/main now states the leak explicitly). Deferred because Drop cannot await the async mutex, so a correct fix either spawns from Drop or swaps the map to a std::sync::Mutex — structural transport changes, and spawn-from-Drop is exactly the shape that looks right and leaks under cancellation. (audit Batched audit cleanup: area:mcp — 14 items #564, deferred by fix(stella-mcp): bound the stdio reader's per-line read #592)
• No worktree reclamation in the fleet — add WorktreeManager::prune(&self, keep: usize) that lists worktrees under worktrees_root, removes those whose branch has no commits beyond base, and finishes with git worktree prune; wire it to a stella fleet clean verb and/or auto-prune run scopes older than N days. Without it, .stella/worktrees/ directories and fleet/* branches grow monotonically for the life of the repository — nothing prunes, nothing warns. stella-fleet/src/git.rs:334, stella-cli/src/fleet_cmd.rs. Deferred because the Fix asks for "a reclaim policy with an owner" (what to keep, how old is stale, whether pruning is automatic), it adds a new user-facing CLI verb, and destructive worktree/branch removal cannot be safely verified locally. (audit Batched audit cleanup: area:fleet — 14 items #562, deferred by chore(stella-fleet): apply the area:fleet audit batch #593)
• No escape hatch for claim locks already leaked in the field — add a claim-lock TTL and/or a stella fleet claims --release verb to clear file_locks rows left by earlier releases. stella-fleet/src/fleet.rs:407 against stella-store's file_locks table. The ClaimGuard landed by chore(stella-fleet): apply the area:fleet audit batch #593 stops new leaks but does nothing for locks already stranded, which still fail every later run in that workspace with a ClaimConflict naming a run id that no longer exists. Deferred because a new CLI verb plus a lock-expiry policy — what TTL, whether stealing is allowed — is an owner decision about file_locks semantics, and a wrong TTL lets two workers write the same file. (audit Batched audit cleanup: area:fleet — 14 items #562, deferred by chore(stella-fleet): apply the area:fleet audit batch #593)
• run_best_of_n leaks every candidate workspace on cancellation — the Box<dyn CandidateWorkspace> handles are never wrapped in a drop guard, so cancellation between port.create() and the post-loop cleanup pass leaks every workspace created so far — in the real CLI, up to n git worktrees per cancelled run, accumulating in git worktree list. stella-pipeline/src/pipeline.rs:1302. Deferred because CandidateWorkspace::remove is async, so a correct guard needs a new sync trait method or a runtime handle (cross-crate trait reshaping), and the leak is not observable from cargo test without a new cancellation harness. (audit Batched audit cleanup: area:pipeline — 15 items #567, deferred by test(pipeline): name the work-item test modules for what they pin #594)
• The deck's !-escape kills only the direct child — route it through stella-tools' shared runner and kill the whole process group. stella-tui/src/deck_shell.rs:124. chore(stella-tui): clear the area:tui audit batch (9 items) #602 took the one-line kill_on_drop, which by its own note "only reaps while the tokio runtime is alive, and signals the direct child, not its process group. It fixes deck-exit orphans, nothing more." Deferred because the real fix depends on the shared-runner decision above, or adds an unsafe pre_exec whose #[cfg(not(unix))] branch could not be compiled or tested on the audit machine. (audit Batched audit cleanup: area:tui — 26 items #571, deferred by chore(stella-tui): clear the area:tui audit batch (9 items) #602)
• grep/glob spawn rg outside the shared runner — route stella-tools/src/grep.rs:176 through exec::run_argv so the invocation gets a timeout, kill_on_drop, and bounded buffering instead of an unbounded rg.output().await. Deferred because exec::drive merges stderr into stdout before returning, which would destroy grep's exit-2 pattern-error discrimination and the (no matches) path — the literal fix silently changes tool semantics — and a timeout/cancellation witness is inherently flaky. (The column/byte-cap half of this item was applied at grep.rs:164.) (audit Batched audit cleanup: area:tools — 26 items #570, deferred by chore(stella-tools): bound tool output, harden sub-resource auth, prune the process table #603)
• No binary installs a SIGINT/SIGTERM handler — install signal handling in the CLI entry point so GroupKillGuard actually runs on a headless Ctrl-C and child process trees are reaped. stella-tools/src/exec.rs:174, stella-cli/src/main.rs. Deferred because the fix lands in stella-cli's main, outside the area:tools surface of its audit issue, and signal delivery plus child-tree reaping is not witness-testable locally without a flaky harness. This item gates the practical value of every Drop guard listed above. (audit Batched audit cleanup: area:tools — 26 items #570, deferred by chore(stella-tools): bound tool output, harden sub-resource auth, prune the process table #603)

Why these are together

These are not eleven independent leaks — they are one unanswered question asked at eleven call sites. Each is a resource created mid-flight (a process group, a git worktree, a map slot, a background task, a database row) whose teardown lives after an await that a dropped future never reaches, and each is blocked on the same two facts: Drop cannot await, and nothing in this workspace catches a signal. Fixing them one at a time invites eleven different guard shapes, and the wrong shape — tokio::spawn from Drop — looks correct in review while doing nothing in the exact scenario it claims to handle. Decide the rule once, then land it across all the sites, with the signal handler first so the guards can fire and the two reclaim hatches last so workspaces already broken in the field can recover.

_{Deferred during the #546 audit remediation. Put Closes #<this issue> in the PR description and as a commit trailer when it lands.}

[macanderson]

Triage 2026-07-26 — the gating item shipped; the other nine are now unblocked

PR #658 (63ba537d, commit 9ddb75d3, trailer Refs #613) landed the precondition this issue's "Why these are together" section is built around.

• Item 11 (SIGINT/SIGTERM) — BUILT. New stella-cli/src/signals.rs (200 lines), wired at stella-cli/src/main.rs:60 (mod signals), :1348 (signals::interrupted_exit_code()), :1502 and :1535 (signals::block_on_interruptible(…)). The design matches this issue's own analysis — a biased select! that drops the work future while the runtime is alive, rather than a Drop-spawn — plus 130/143 exit codes and a 2s runtime shutdown timeout.
• Item 7 (leaked claim locks) — HALF BUILT. Store::release_file_locks_of_dead_holders exists and is swept at run start (stella-cli/src/fleet_cmd.rs:190, with prune_stale_file_locks at :191) and reaped-and-retried inside stella-fleet/src/fleet.rs:597. No stella fleet claims --release verb and no TTL policy — fix: the nine P0s from the backlog triage #658's own message says that half was left out deliberately as "a genuine owner decision rather than a defect".

The other nine reproduce unchanged, e.g. stella-tools/src/verify.rs has no Drop and calls cleanup_shadow only at :268/:274/:286; stella-cli/src/agent/tools.rs:475-476 still pre_exec + setsid() with no GroupKillGuard; MediaError::Cancelled (stella-media/src/error.rs:64) is still never constructed; stella-tui/src/deck_shell.rs still .kill_on_drop(true) with the comment conceding it signals only the direct child.

Net 9.5 of 11 remain — but their framing changes. The caveat that nothing catches a signal is no longer true, so these nine guards are now worth building rather than theoretical. Suggested edits: tick item 11 citing stella-cli/src/signals.rs, rewrite item 7 to cover only the manual hatch + TTL policy, and drop the no-signal-handler caveat from the intro.

Line drift: item 3 → stella-context/src/store.rs:396; item 5's leak is in request() at stella-mcp/src/stdio.rs:160-176, not :135; item 8 → pipeline.rs:1382; item 10 → grep.rs:197/:221.

[macanderson]

Closing as completed.

PR #698 states the ruling this issue existed to obtain — a synchronous Drop guard wherever the teardown syscall is synchronous, and never tokio::spawn from Drop — and lands 4 rows against it. Three more landed elsewhere: #658 added the signal handler, and #703 added the rg kill_on_drop plus the claim-lock escape hatch.

The 5 rows that remain are not lost. PR #698 lists them verbatim in its "Deliberately NOT in scope, so a future audit stops re-reporting them" section, which is exactly the outcome this issue was asking for: a settled owner for process-group / worktree / slot / lock reaping, and a written record of what was consciously left out.

Contract decided, residual recorded — closing.