fix(daemon): own the session-metadata outbox per member

[zfy0701]

Summary

Fixes #1023. On a daemon pool the session-metadata outbox (session_metadata_outbox)
lives in the shared data-plane store, but an event/session-sync frame is scoped by the
agent's organization, and the daemon resolves that organization from its own agent
registry — which carries only the agents this member serves. The drain had no owner and
no agent filter, so every member picked up every row, and a row it could not scope was
raised locally as a non-retryable SCOPE_DENIED, deferred, and retried forever.

The outbox now gets the ownership model the hook-completion outbox (#1035) and the purge
receipts (#1032) already use — SHARED_OUTBOX_LEASE_MS, a CAS claim before every emit,
an owner-fenced ACK — plus one thing those two do not need: a row this member cannot
scope is parked for the member that can, instead of counting as a failure.

Failure scenario

1. Member A runs a turn for agent X and writes the terminal snapshot into the shared
   outbox. The rollout then moves X's duty to member B (or A is retired).
2. Every member's drain reads the outbox install-wide. A, B and C all pick up the same
   row, oldest first.
3. CpClient.scopedFrame needs an orgId for a non-install-wide frame and derives it
   from the payload's agent id. A member that does not serve X has no spec for X, so
   orgForAgent yields nothing and the client throws SCOPE_DENIED locally, before
   anything is sent.
4. retryable: false is read as "permanently rejected": the row is failure-counted and
   deferred, logging event/session snapshot deferred after N failures for session <id>
   on every member, with N climbing forever. The snapshot is never persisted by the CP,
   and because the drain is one row at a time it also delays every row behind it.

Fix

Ownership in the store (packages/daemon/src/store/local-store.ts)

• session_metadata_outbox gains ownerId / claimedAt (schema v10 plus its
  SCHEMA_MIGRATIONS step, so an existing store upgrades in place).
  saveSessionMetadataSnapshot stamps the writing member — the daemon that produced the
  snapshot is serving the agent, so it is the one that can scope the frame.
• nextSessionMetadataSnapshot, hasPendingSessionMetadata and
  nextSessionMetadataAttemptAt share one scope: this member's own rows, plus unowned or
  lease-lapsed rows for agents it currently serves. Unlike the hook outbox, an
  unowned row is deliberately not offered install-wide — the frame carries the agent's
  organization, so a parked snapshot must wait for the member that has it rather than
  circling the pool. The scheduler reads the same scope, so a foreign row can no longer
  keep re-arming the retry timer.
• claimSessionMetadataSnapshot is the CAS taken before every emit (revision-fenced, so
  a newer snapshot written mid-flight is not claimed by the older attempt).
• parkSessionMetadataSnapshot releases the claim and applies a short backoff without
  touching the body or the failure count; acknowledgeSessionMetadataSnapshot and
  recordSessionMetadataSnapshotFailure are fenced to the claim holder, so a member can
  never destroy or fail a peer's row.
• resumeSessionMetadataSnapshots re-arms the parked rows of a set of agents.

Drain and duty handover (packages/daemon/src/daemon.ts)

• runSessionMetadataDrain reads with the owner + served-agent scope, claims before it
  emits, and skips (never returns on) a row a peer took over between the read and the
  claim.
• Two paths park instead of failing: a row for an agent this member does not serve, and a
  SCOPE_DENIED raised while emitting. Neither increments failedAttempts or logs the
  defer warning; the row simply waits for its serving member.
• settleDutyChange re-arms the parked snapshots of agentsGained and re-runs the drain,
  next to the purge-receipt replay added in fix(daemon): scope the session TTL/GC sweeps to the duty holder and lease purge receipts per member #1065, so a takeover replays them immediately
  rather than after the backoff a departed holder wrote.

A local single-daemon store is unchanged: it lists and settles unfenced, the claim is a
no-op, and parking is refused so the pre-existing retry/defer path still applies.

Test plan

• packages/daemon/test/daemon-session-metadata-outbox-pool.test.ts (new) — two
  members over one shared store: B drains only its own row and leaves A's pending and
  unfailed; a row for an agent nobody here serves is parked (claim released, backoff
  set, failedAttempts still 0, no defer warning) and the peer leaves it alone; B
  gaining the duty replays that row exactly once, scoped to the agent's org, and a
  second drain emits nothing; a served row whose organization is momentarily
  unresolvable is parked, not failure-counted, and drains once the scope resolves; a
  single daemon on its own store drains every row unfenced. The first four fail on
  main.
• packages/daemon/test/local-store.test.ts — the v1/v5/v7 upgrade paths land on
  schema v10 with ownerId / claimedAt present on the outbox.
• pnpm --filter @agentconnect.md/daemon typecheck
• local-store, daemon-session-metadata-outbox-pool, daemon-smoke,
  durable-inbox, daemon-lifecycle, daemon-session-sweeps-pool,
  postgres-migrations, daemon-duty-*, orchestration suites
• pnpm lint, pnpm format:check

[agentconnect-md-test]

Requesting changes for one blocking liveness bug in the new ownership handoff. A fresh session-metadata row owned by a retiring member is excluded from the successor's drain scope; the gain replay only resumes ownerless rows, and the scheduler does not arm a wake for the future owner-lease expiry. That can leave a terminal session snapshot unsynced indefinitely during the graceful rollout path this patch is intended to fix. The ownership/CAS direction otherwise looks coherent.

I verified the trusted synthetic merge has exactly the supplied base and head as parents, and the changed worktree blobs match the PR diff. Focused daemon tests were not runnable because this isolated checkout has no installed dependencies.

_{sent by review-bot (Codex · gpt-5.6-sol) · open in session}

[agentconnect-md-test]

[P1] Keep a wake for fresh rows owned by the previous holder. On a graceful handoff, A can persist a final snapshot, enter shutdown (which sets draining and clears the metadata retry timer), and release the duty before it parks the row. B then gains the agent while ownerId = A and claimedAt is still fresh. replayGainedSessionMetadata() only resets ownerId IS NULL rows, this scope hides the row from B, and schedulePendingSessionMetadataDrain() uses the same scope, so it arms no timer for claimedAt + SHARED_OUTBOX_LEASE_MS. When the claim eventually lapses, nothing wakes B unless an unrelated snapshot or reconnect happens, leaving the metadata unsynced indefinitely. Please either transfer/release rows for newly gained agents under a safe fence or include the owner-lease expiry in the next-attempt scheduling.

[zfy0701]

Good catch on the graceful handoff — it was a real hole, and the first push had no way out of it. Fixed in 3d2792241, taking both suggested routes plus the shutdown release:

• Transfer on agentsGained — reclaimSessionMetadataSnapshots now releases a previous holder's claim for the gained agents (ownerId IS NULL OR ownerId <> @ownerId), not just the parked rows. The duty ledger has already proved that member no longer serves the agent. This member's own claims are deliberately untouched: only it knows whether one is in flight.
• Wake at the lease expiry — nextSessionMetadataAttemptAt and hasPendingSessionMetadata now read a wider work scope (own rows plus every row of a served agent, whoever holds the claim) and return max(nextAttemptAt, claimedAt + SHARED_OUTBOX_LEASE_MS) for a row a peer still holds. So a wake is armed even when no duty change fires.
• Release on shutdown — stop() hands back every claim this member still holds, after its drain is joined and before the store is closed, so a successor never waits out a lease nobody will renew.

Tests (all three fail on the previous commit):

• takes over a claim the departed holder never released when the duty is gained — row still owned by A with a fresh claim, B's settleDutyChange emits it immediately.
• arms a wake at the lease expiry of a peer's claim on a served agent — no duty change at all: B's drain emits nothing, asserts the armed fake-clock timer at claimedAt + lease, then advances onto it and emits.
• releases the claims it still holds when it stops, so a successor drains at once — B is blocked by A's live claim, a.daemon.stop(), B's next drain emits.

Re-ran typecheck, the touched suites plus daemon-lifecycle / daemon-session-sweeps-pool / durable-inbox / postgres-migrations / daemon-duty-* / orchestration, pnpm lint and pnpm format:check. Rebased on origin/main before pushing; SCHEMA_VERSION there is still 9, so this stays at 10.

[agentconnect-md-test]

The original handoff hole is fixed, but 3d2792241 has one remaining blocker:

nextSessionMetadataAttemptAt() uses two-argument MAX(a, b) as a scalar function. That works in SQLite, where the new tests run, but the real shared pool uses PostgreSQL. Its SQL worker does not rewrite this expression, and PostgreSQL’s aggregate MAX accepts only one argument.

Consequently, when a pool row is pending, the scheduler query fails instead of arming the lease-expiry/retry timer—the no-duty-change recovery path can still remain stuck. Please use a portable CASE expression or add an explicit PostgreSQL GREATEST rewrite, ideally exercising this query through PostgresSyncDatabase.

_{sent by review-bot (Codex · gpt-5.6-sol) · open in session}

[zfy0701]

@review-bot continue

[agentconnect-md-test]

Requesting changes for one blocking backend-portability defect on 3d2792241. The new duty-gain reclaim and shutdown release address the original graceful-handoff hole, but the lease-expiry scheduler uses SQLite's scalar two-argument MAX syntax. The production shared pool runs this LocalStore query through PostgreSQL, whose worker does not rewrite that expression, so the query fails instead of arming the fallback timer. The no-duty-change lease-expiry path can therefore still remain stuck.

The checkout ref matches the trusted head exactly. The new focused tests exercise DatabaseSync/SQLite and do not cover this PostgreSQL query path.

_{sent by review-bot (Codex · gpt-5.6-sol) · open in session}

[agentconnect-md-test]

[P1] Keep this lease-expiry expression portable to the real shared-store backend. SQLite accepts MAX(x, y) as a scalar maximum, but PostgreSQL's MAX is a one-argument aggregate, and postgres-store-worker.js#rewrite does not translate this expression. Consequently, once hasPendingSessionMetadata() finds a pool row, nextSessionMetadataAttemptAt() throws, the refill check only logs a warning, and no lease-expiry timer is armed—the exact no-duty-change recovery path added here stays stuck. Please use a portable CASE expression or explicitly rewrite this to PostgreSQL GREATEST, and exercise the query through PostgresSyncDatabase.

[zfy0701]

Right, and it reproduced exactly as described. Fixed in 3afa7cce5.

The bug — driving nextSessionMetadataAttemptAt through PostgresSyncDatabase raises:

error: function max(bigint, bigint) does not exist

Since the refill check only logs a failed read, the lease-expiry wake was never armed on a pool — the one recovery path the previous commit added.

The fix — the later-of is one CASE now, the same text on both engines:

CASE WHEN ownerId IS NOT NULL AND ownerId <> @ownerId
          AND COALESCE(claimedAt, 0) + @lease > COALESCE(nextAttemptAt, 0)
     THEN COALESCE(claimedAt, 0) + @lease ELSE COALESCE(nextAttemptAt, 0) END

I deliberately did not add a worker rewrite: MAX( → GREATEST( would also rewrite every legitimate one-argument aggregate MAX(col) in this file into a scalar, which is a much worse failure than the one it fixes. There is no established two-arg rewrite to follow — the worker's list is DDL and syntax shims (BEGIN IMMEDIATE, INTEGER, INSERT OR IGNORE, LIMIT -1 OFFSET), not function translation.

Exercised against a real engine — postgres-pool-store.int.test.ts gains leases session-metadata snapshots per member and wakes when the claim lapses, driving the lease, the claim CAS, the ACK fence, the park, the reclaim, the shutdown release and this query through PostgresSyncDatabase. Against a throwaway postgres:16-alpine:

• with the two-argument max(): fails with the error above;
• with the CASE: all 4 cases in the file pass.

I also exercised the in-place migration on PostgreSQL rather than only the fresh-schema path: dropped ownerid / claimedat, set _local_store_schema_version back to 9, reopened — the v9→v10 step applied and the suite passed, ending at version 10 with both columns present.

Rest of the diff audited for engine-specific SQL (max/min two-arg, IFNULL, ||, INSERT OR IGNORE, datetime(), AUTOINCREMENT, rowid): the only remaining MIN(...) is a one-argument aggregate, everything else added is COALESCE / IS NULL / <> / IN / ON CONFLICT … RETURNING, all portable. attemptAt was already in the worker's canonical column map, so the lowercase fold round-trips.

One drive-by in the same file: the first case called store.getCronLastRun(...), which was renamed to cronRun() (and setCronLastRun gained a definition argument). The suite only runs with DATA_PLANE_TEST_DATABASE_URL set, so nothing caught it; repaired so the file passes as a whole.

Re-ran typecheck, the daemon suites (touched plus daemon-lifecycle, daemon-session-sweeps-pool, durable-inbox, postgres-migrations, daemon-loop-guard-pool, daemon-duty-*, orchestration), pnpm lint, pnpm format:check. Rebased onto origin/main (59c04e700) before pushing; SCHEMA_VERSION there is still 9, so this stays at 10.

[agentconnect-md-test]

Approved on 3afa7cce. Both prior blocking findings are resolved:

• Graceful handoff now reclaims fresh peer-owned rows when duty is gained, keeps a wake armed for a live peer claim's lease expiry, and releases this member's remaining claims during shutdown.
• The lease-expiry calculation now uses a SQLite/PostgreSQL-portable CASE expression, with a real PostgreSQL pool-store integration test covering claim isolation, lapse, takeover, parking, reclaim, release, and settlement.

I inspected the complete rebased base-to-head diff. The checkout ref matches the trusted head exactly, and every changed worktree blob matches the PR diff. I could not rerun the suites locally because this isolated checkout has no installed dependencies.

_{sent by review-bot (Codex · gpt-5.6-sol) · open in session}

[agentconnect-md-test]

Confirmed—the CASE fix and real PostgreSQL coverage resolve the issue cleanly. I approved 3afa7cce; no further blockers found.

_{sent by review-bot (Codex · gpt-5.6-sol) · open in session}