fix(hub): orphaned agents on a dead host no longer block its deletion

[physercoe]

The bug

Deleting an offline host 409s with host still has active agents — terminate them first, even though the host-runner is gone and the agents died with it. Two defects meet in that one refusal.

1. One query disagreed with the rest of the package about what "alive" means

handleDeleteHost counted live agents with status NOT IN ('terminated','failed'). Every other live-agent query in the package uses ('terminated','crashed','failed') — and so does migration 0024's handle-uniqueness index.

crashed is precisely what the host-runner's reconcile loop reports for an agent whose pane has vanished (hostrunner/reconcile.go:126). So the hub's own verdict that an agent was dead was not enough to let its host go.

Hoisted into one definition (agent_status.go) and used at all five sites, so the spellings cannot drift apart again.

2. A live agent on a dead host was a dead end

Even with the set corrected, an agent that never reached a terminal status blocked forever. The hub only reaches an agent through its host-runner; with the runner gone, nothing will ever report that agent terminal, so the refusal could not be satisfied by waiting. The fleet view showed agents "running" on a machine that no longer existed, and the only way out was stopping each agent by hand.

Now the host's own status decides:

host	live agents	outcome
online	yes	409 — the runner is there and can stop them properly
offline	yes	reaped to crashed, delete proceeds
either	none	delete proceeds

Why not a background reaper

An offline host is not proof of a dead agent. M4 agents launch into tmux windows (TmuxLauncher), and the tmux server outlives a host-runner restart — an agent can be perfectly healthy while its runner is being upgraded. Only M1/M2 agents are children of the runner (exec.CommandContext + cmd.Start() in launch_m2.go) and die with it.

A reaper on a clock would kill the healthy ones, and irreversibly: tickReconcile skips agents already in a terminal status, so a wrong crashed verdict never heals back to running — and the driver-teardown pass then stops its driver for real.

The director's DELETE is the judgement that the machine is gone. That is where the reap belongs.

Reaping is not just a status write

reapOrphanedAgent runs the same aftermath as any crash — session paused, bearer token revoked, run digest sealed, task status derived (ADR-029 D-3 reads crashed → blocked, distinct from a terminate-with-no-result → cancelled). That aftermath was inline in handlePatchAgent; extracted to applyAgentCrashEffects so a dead agent's token cannot outlive it through the new path.

Tests

The handler had none. Added: terminal agents never block (all three statuses), online still refuses and touches nothing, offline reaps, the aftermath runs, the reap is scoped to its own host, missing host is 404, and the SQL predicate agrees with the Go one.

Each verified by mutation:

• reintroduce the missing crashed → TestDeleteHost_TerminalAgentsNeverBlock/crashed fails with the exact 409 from the report
• refuse unconditionally → the three offline tests fail
• skip the aftermath → the session stays active

go test ./... green; full lint set green (openapi validated in a venv).

Not covered

No live-fleet reproduction — this was traced and fixed from the code and exercised against the real handler through the router in tests. Worth one manual delete of a genuinely dead host after deploy. Note the hub takes up to ~120s (HostStaleThreshold + sweep interval) to flip a host to offline; a delete attempted inside that window still 409s by design.

🤖 Generated with Claude Code