Proposal: Persistent Run State + list/reattach Actions

[ShinonKagura]

Proposal: Persistent Run State + list / reattach Actions

Feature proposal — looking for maintainer feedback on shape before writing any code.

Motivation

Detached runs today are session-scoped:

• Process-local runId (e.g. r1, r2) lives in detached-registry.ts Map
• Pi restart / crash / pi -p exit → all run state gone
• No way to discover or re-attach to a run started by a different Pi process
• No durable evidence beyond what was already mirrored to artifact paths

Concrete operator pain points this would solve:

1. Pi crash mid-run: long-running graph terminates mid-execution → operator can't see what already finished, what's in-flight, what's blocked
2. Multi-session ops: planner Pi spawns a run, hands off to reviewer Pi — no clean way for reviewer to see the run from a fresh Pi
3. Audit / forensics: after a session ends, no record of which graphs ran, with what config, when, by which session

Sketch (open to revision)

Persistent state (B1a)

<state-root>/pi-multiagent/runs/<runId>/
  manifest.json        — schemaVersion, runId, createdAt, invocationCwd, objective,
                         ownerPid, ownerSessionId, terminalRetentionSeconds
  status.json          — { runId, status, updatedAt, terminal, terminalAt }
  worktrees.json       — per-step isolation evidence (when applicable — see PI3)
  artifacts/           — mirrored step-final artifacts
  events.jsonl         — append-only event log

Writes are atomic (write-temp + rename). State dir auto-created at first start.

New actions (B1b)

// List all visible persistent runs (own session + orphaned)
{ action: "list" }
// Returns:
{
  ok: true,
  runs: [
    {
      runId: "r1",
      ownerClass: "this-session" | "foreign-session" | "orphan" | "unknown",
      status: "running" | "succeeded" | "...",
      objective: "...",
      createdAt: "ISO",
      runDir: "/abs/path/to/run/dir",
      pendingWorktrees: 0,
    },
    // ...
  ],
  scheduledRuns: [/* optional, see PI1 */]
}

// Read-only snapshot of an orphan or terminal run
{ action: "reattach", runId: "r1" }
// Returns:
{
  ok: true,
  manifest: { /* ... */ },
  status: { /* ... */ },
  worktrees: [ /* ... */ ],
  mirroredArtifacts: ["/abs/path/..."]
}

Mutation actions (message, cancel, cleanup) on a reattached foreign run are denied unless the calling session is the original owner.

Open questions for maintainer

1. ID generation strategy

This is the biggest open question. v0.8.7 moved runIds to short process-local handles like r1. If we persist state across processes, r1 from session A and r1 from session B will collide on disk.

Options:

• (a) Switch to UUIDs in persistent dir (runs/<uuid>/), keep r1 as display alias inside session
• (b) Namespace by sessionId+pid (runs/sid:abc-r1/), more readable, still collision-safe
• (c) Sequential atomic counter on disk (next-id file), simplest but introduces a global serializer

I'd lean (b) for readability. What's your preference?

2. Ownership policy

• A foreign Pi can list and reattach (read-only) — yes/no?
• A foreign Pi can cancel an orphan run — yes/no? (orphan = PID gone but state on disk)
• A foreign Pi can cleanup an orphan run — yes/no?

Proposed: read always allowed, mutation denied for foreign-running, allowed for orphan with explicit --force flag (or new take-ownership action).

3. Garbage collection

• Terminal runs should expire after terminalRetentionSeconds (already a runtime option)
• Background sweep timer (proposed 6h interval) deletes expired terminal run dirs
• Open: what about runs with ownerClass = orphan that never terminalized (Pi crashed mid-run)? Stay forever, or grace period (say 24h after last status update) before classification → expired?

4. State dir location

• $XDG_STATE_HOME/pi-multiagent/runs/ (per-user, persistent across reboots)
• $TMPDIR/pi-multiagent/runs/ (ephemeral, cleared on reboot — current behavior of run-artifact temp paths)
• .pi/multiagent-runs/ (project-local, follows the cwd)

Proposed: $XDG_STATE_HOME for cross-session/cross-reboot visibility, with an env override PI_MULTIAGENT_STATE_DIR. The cwd doesn't reliably stay constant across sessions, and $TMPDIR defeats the whole point.

5. Schema versioning

• manifest.json has schemaVersion. What to do when schema changes between Pi versions?
• Proposed: read-only fallback — old-schema runs are listable but reattach returns "schema-version-too-old" with the original schemaVersion echoed. No silent migration. Operator runs cleanup manually or upgrades Pi to a compatible read range.

6. Concurrency / locking

• Multiple Pi processes can write to the same state dir
• Proposed: per-run-dir lock file (flock-style), held by owner during write paths. Reads don't take the lock.

7. Privacy / multi-tenant

• State dir is per-user (XDG default), so other OS users can't see it directly
• But: a list action exposes objectives + status to anything that can call agent_team in that user's session
• Open: is there a use case where two distinct Pi-using personas share a UNIX account and want isolation? Probably not for v1.

Known concern from upstream review

r1 collision after reload — your PR #2 review explicitly called this out:

"persistent state is keyed by process-local run IDs like r1, which can collide after reload"

Question 1 above is my response. I want to fix this properly in the proposal — not punt it to "later".

What I'd write if you say go

• extensions/multiagent/src/persistent-run-state.ts — atomic writes, manifest/status read+write, ownership classification, list+reattach helpers
• Extend detached-state.ts to write through to persistent layer on lifecycle transitions
• New tool actions: list, reattach in AGENT_TEAM_ACTION_VALUES, schema, preflight-shape
• New result-format branches for formatList, formatReattach
• Background sweep timer for terminal-run garbage collection
• Tests covering: persistence durability, reload + collision-safety, ownership classification, foreign-mutation denial, schema-version compat, GC sweep

Roughly ~600-800 lines + tests. Larger than PI1 because it touches lifecycle in multiple places.

Status

I have an earlier implementation from PR #2 — but it had the r1 collision issue you flagged. Want to fix this in the proposal stage before writing code.

---

Decision needed from you

• Direction OK in principle?
• Preferred answers to the 7 open questions — especially Q1 (ID strategy) and Q4 (state dir location)?
• Any showstopper concerns I should know before writing code?

If green, I'll open a focused PR against current main with the chosen ID strategy + dir location baked in.

[tiziano-contorno]

This remains a valid lifecycle proposal, but it is not part of the current maintainer integration PR (#13).

Persistent run state is a contract replacement, not a small patch: durable run IDs, redacted/bounded list, read-only reattach semantics, foreign/orphan mutation denial, private state dir permissions, schema versioning, locking, GC, and cleanup/evidence-deletion semantics all need to be designed together.

Keeping this open as design backlog and as the likely prerequisite for any durable scheduling work in #6.

[tiziano-contorno]

Closing this backlog proposal after the 0.9.8 release. 0.9.8 deliberately shipped child-session audit metadata and recovery forensics, not persistent run IDs, list/reattach, crash resume, or cross-session run ownership. A future durable-run-state design would need a new proposal with collision-safe IDs, redaction, locking, GC, and mutation/cleanup ownership defined up front.

[tiziano-contorno]

Reopening as backlog / potential future PR. This was not a 0.9.8 release blocker and is not part of the shipped 0.9.8 contract; it remains a separately scoped future design candidate.