Skip to content

feat(daemon): loop body wires executeIntent end-to-end (claim → dispatch → heartbeat)#107

Merged
chitcommit merged 2 commits into
feat/meta-executors-registryfrom
feat/daemon-loop-executes-intents
Jun 4, 2026
Merged

feat(daemon): loop body wires executeIntent end-to-end (claim → dispatch → heartbeat)#107
chitcommit merged 2 commits into
feat/meta-executors-registryfrom
feat/daemon-loop-executes-intents

Conversation

@chitcommit

Copy link
Copy Markdown
Contributor

Stacked on #106 (feat/meta-executors-registry). Merge #106 first.

Summary

Closes the meta-orchestrator end-to-end loop. The cluster daemon's persistent leader loop now drives claimed Intents through executeIntent (added in #106), which dispatches via the executor registry, applies the second sovereignty gate, and atomically updates cc_intents.status + writes cc_actions_log.

Removes the injected IntentExecutor abstraction from the foundation skeleton — it was a placeholder per ADR-001 out-of-scope, replaced now that the registry exists.

What changed

daemon/loop.ts

  • Drops the IntentExecutor interface and markIntentDispatched / completeIntent / failIntent calls from the loop body — those are now owned by executeIntent, which avoids double-driving the state machine.
  • Inner loop body: claimNextIntentexecuteIntent → classify ExecutorResult → heartbeat → continue.
  • Four outcomes are classified distinctly per the spec:
Outcome ok replayed Action
Fresh successful execution true false bump intentsProcessed, reset backoff
Replayed (terminal audit exists) true true bump intentsReplayed, no backoff, no double-count
Sovereignty refusal false bump intentsRefused, no backoff (steady-state)
Executor error false bump intentsErrored, bounded exp backoff (1s → 30s cap)
  • Sovereignty refusals are identified by the canonical error prefixes emitted by meta/executors/dispatch.ts (sovereignty re-reckon: and sovereignty snapshot stale ...). Refusals are NOT treated as transient faults.
  • Leadership lifecycle, lease heartbeats, reclaimStuckIntents, and AbortSignal-driven shutdown via releaseLeadership are preserved from the skeleton.
  • New maxIterations option provides a hard safety cap for tests when the queue might drain to empty before reaching maxIntents.

tests/daemon/loop.spec.ts (new)

Real-Neon integration test. Skips without DATABASE_URL. Seeds two real Goals/Plans/Intents against update_obligation_status plus two real cc_obligations rows, then runs runLeaderLoop with maxIntents=2.

Asserts:

  • both intents reach status='done'
  • each produces exactly one cc_actions_log row (attempt=1, 64-char hex idempotency_key, status='completed', action_type='status_change')
  • both cc_obligations rows actually moved to 'deferred'
  • cc_node_leases row shows leadership was released on clean exit (node_id NULL, session_id NULL)
  • log stream contains intent_heartbeat_before / intent_heartbeat_after pairs and at least one leader_acquired

End-to-end trace

Test ran on Neon project cool-bar-13270800, branch pr-daemon-loop-executes-intents (parent: pr-a-meta-executors-registry):

RUN  tests/daemon/loop.spec.ts
✓ drains two pending intents, writes audit rows, heartbeats, releases on clean exit (1918ms)
Test Files  1 passed (1)
Tests       1 passed (1)

Lease state after clean exit (post-test query):

role                       | node_id | session_id | claimed_at | heartbeat_at | lease_expires_at
meta-orchestrator-leader   | NULL    | NULL       | NULL       | NULL         | NULL

The loop log stream during the run shows the canonical sequence:

leader_acquired → intent_claimed → intent_heartbeat_before → intent_completed → intent_heartbeat_after
                → intent_claimed → intent_heartbeat_before → intent_completed → intent_heartbeat_after
                → (release on maxIntents)

What is NOT in this PR

Validation

  • npm run typecheck — clean
  • SKIP_INTEGRATION=1 npm test — 15 passed, 13 skipped
  • Integration test (above) — passed against real Neon

Test plan

  • Real-Neon integration covers the two-intent happy path
  • Replay-as-noop and sovereignty refusal paths are explicit branches in the loop with distinct log lines and counters (covered by unit-level reasoning; happy-path integration test exercises the executed branch)
  • Follow-up: extend the integration test with a refused intent (requires a stale-snapshot fixture and an actor with sub-autonomous trust) and an errored intent (e.g., obligation_id pointing at a deleted row)

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com

…tch → heartbeat)

Stacked on #106. Replaces the injected-executor abstraction in daemon/loop.ts
with a direct call to meta/intent.ts::executeIntent, closing the
meta-orchestrator loop. Status transitions, audit-row writes, and the second
sovereignty gate are all owned by executeIntent → dispatch; the loop's
responsibility is leader lifecycle, intent claiming, heartbeats, and outcome
classification.

Four outcomes are handled distinctly:

  - ok=true (executed)      → bump processed counter, reset error backoff
  - ok=true (replayed)      → bump replayed counter, no backoff, no double-count
  - ok=false (refused)      → bump refused counter, no backoff (steady-state)
  - ok=false (exec error)   → bump errored counter, bounded exp backoff

Sovereignty refusals are identified by canonical error prefixes emitted by
meta/executors/dispatch.ts ("sovereignty re-reckon:" /
"sovereignty snapshot stale ..."). Refusals are NOT treated as transient
faults — they are valid outcomes and do not trigger backoff.

The loop honors options.signal via AbortController throughout, including
inside the sleep helper, so SIGTERM from daemon/runtime/entrypoint.ts (PR #105)
unwinds cleanly through releaseLeadership.

tests/daemon/loop.spec.ts — real Neon integration. Seeds two pending intents
against the update_obligation_status executor, runs runLeaderLoop with
maxIntents=2, asserts:
  - both intents reach status='done'
  - each produces exactly one cc_actions_log row (attempt=1, key set, status='completed')
  - cc_obligations rows actually moved to 'deferred'
  - cc_node_leases shows leadership released on clean exit
  - log stream contains intent_heartbeat_before / intent_heartbeat_after pairs

Out of scope (not in this PR):
  - new executors (mercury, etc.)
  - production deploy
  - multi-node coordination beyond single-node leader
  - schema additions on cc_intents / cc_actions_log

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jun 4, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
chittycommand 56603bc Jun 04 2026, 12:58 PM

@github-actions

github-actions Bot commented Jun 4, 2026

Copy link
Copy Markdown
  1. @coderabbitai review
  2. @copilot review
  3. @codex review
  4. @claude review
    Adversarial review request: evaluate security, policy bypass paths, regression risk, and merge-gating bypass attempts.

@coderabbitai

coderabbitai Bot commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 3f8c5067-67ad-4c8d-88da-c47370abfb84

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/daemon-loop-executes-intents

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@chatgpt-codex-connector

Copy link
Copy Markdown

To use Codex here, create a Codex account and connect to github.

chitcommit pushed a commit that referenced this pull request Jun 4, 2026
P1 entrypoint stub executor → refusal (codex 3352439151):
  The stub returned a sentinel dispatchedTaskId, causing the leader loop
  to call markIntentDispatched + completeIntent and record real pending
  intents as 'done' without execution. Replace with an explicit throw so
  the loop's failure path runs (intent → failed with clear refusal
  message). The real executor lands in PR #107.

P2 entrypoint SIGTERM release sessionId (codex 3353069328):
  releaseLeadership gates on session ownership (codex-p2 PR#101); the
  fallback shutdown release passed no sessionId, so it always no-op'd
  against our own session-stamped lease. Pass the loop's sessionId.

P2 install-daemon-vm.sh devDeps prune (codex 3352439158):
  npm ci --omit=dev pruned typescript, breaking 'npm run build:daemon'.
  Install full deps for the repo build step; runtime image in
   still gets --omit=dev separately.

P2 systemd unit NODE_BIN (codex 3352439162):
  Hard-coded /usr/bin/node breaks nvm / /usr/local/bin installs. Ship
  unit with @@NODE_BIN@@ placeholder; install script substitutes the
  detected node path before installing.

P2 systemd MDWE+JIT (codex 3352439169):
  MemoryDenyWriteExecute=true is documented incompatible with V8 JIT
  and would abort node at startup. Remove the flag; document why.

P2 launchd plist missing env (codex 3352439167):
  launchd has no EnvironmentFile equivalent. Plist invoked node
  directly without DATABASE_URL/NODE_CHITTY_ID/NODE_DESCRIPTOR, hitting
  entrypoint.ts's fatal-missing-env branch on Mac Mini nodes. Add a
  launchd-shim.sh that sources /etc/chittycommand/env (same shape as
  systemd EnvironmentFile) then execs node.

P2 migration 0016 single-upload conflict (codex 3352439160):
  Unique partial index on r2_key broke single /upload's plain
  INSERT...RETURNING — re-uploads now 500'd on unique violation. Add
  ON CONFLICT (r2_key) WHERE (r2_key IS NOT NULL) DO NOTHING to match
  batch path; fall back to SELECT-existing returning 200.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@chitcommit

Copy link
Copy Markdown
Contributor Author

Code review findings (automated)

Critical

  • intent_heartbeat_before doesn't actually heartbeat (daemon/loop.ts:269) — log line is named _before but no DB heartbeat is forced before dispatch. Background setInterval(dispatchHeartbeat, ...) doesn't update lastHeartbeat closure variable, leading to misleading naming + an actual gap between claim and first interval tick (up to innerHeartbeatMs, half the lease). For 30s lease this is ~15s gap. Fix: update lastHeartbeat from inside the interval, rename or force synchronous heartbeat. Confidence 88.
  • Lease-loss not detected during dispatch (daemon/loop.ts) — dispatchHeartbeat interval logs heartbeat_lost_during_intent when heartbeat returns falsy, but does NOT abort the in-flight executeIntent, does NOT set a flag the outer loop checks, does NOT cancel via AbortController. If a newer leader takes over mid-dispatch, old leader keeps executing → split-brain writes to cc_actions_log under stale leadership. executeIntent's idempotency may save the audit row from duplication but the split-brain write happens. Fix: pass AbortSignal into executeIntent derived from heartbeat success. Confidence 85.

Important

  • Sovereignty refusal detection is string-prefix couplingisSovereigntyRefusal() matches prefixes from meta/executors/dispatch.ts. Any reword silently reclassifies refusals as errors → triggers exponential backoff on valid steady-state outcome. Replace with typed discriminator (result.refusal: true or errorCode: 'sovereignty_refusal') on ExecutorResult. Confidence 86.
  • maxIntents semantics changed — Now counts processed + replayed + refused + errored toward cap. Field name suggests "max successful" to readers. Document or rename to maxTerminalOutcomes. Confidence 82.

Suggestion

  • The loop branching paths (replayed / refused / errored) have no unit-level test coverage now that IntentExecutor injection was dropped. Add a second integration test seeding an intent with obligation_id pointing at a nonexistent UUID to exercise the error path + backoff.

Confirmed OK

  • Backoff bounds: 1s → 30s cap, reset on success/replay/refusal ✓
  • Test cleanup + SIGTERM releaseLeadership on maxIntents ✓

@chitcommit chitcommit merged commit a8eb86c into feat/meta-executors-registry Jun 4, 2026
9 of 11 checks passed
@github-actions

github-actions Bot commented Jun 4, 2026

Copy link
Copy Markdown
  1. @coderabbitai review
  2. @copilot review
  3. @codex review
  4. @claude review
    Adversarial review request: evaluate security, policy bypass paths, regression risk, and merge-gating bypass attempts.

@chitcommit chitcommit deleted the feat/daemon-loop-executes-intents branch June 4, 2026 12:58
@chatgpt-codex-connector

Copy link
Copy Markdown

To use Codex here, create a Codex account and connect to github.

chitcommit added a commit that referenced this pull request Jun 10, 2026
…tch → heartbeat) (#107)

Stacked on #106. Replaces the injected-executor abstraction in daemon/loop.ts
with a direct call to meta/intent.ts::executeIntent, closing the
meta-orchestrator loop. Status transitions, audit-row writes, and the second
sovereignty gate are all owned by executeIntent → dispatch; the loop's
responsibility is leader lifecycle, intent claiming, heartbeats, and outcome
classification.

Four outcomes are handled distinctly:

  - ok=true (executed)      → bump processed counter, reset error backoff
  - ok=true (replayed)      → bump replayed counter, no backoff, no double-count
  - ok=false (refused)      → bump refused counter, no backoff (steady-state)
  - ok=false (exec error)   → bump errored counter, bounded exp backoff

Sovereignty refusals are identified by canonical error prefixes emitted by
meta/executors/dispatch.ts ("sovereignty re-reckon:" /
"sovereignty snapshot stale ..."). Refusals are NOT treated as transient
faults — they are valid outcomes and do not trigger backoff.

The loop honors options.signal via AbortController throughout, including
inside the sleep helper, so SIGTERM from daemon/runtime/entrypoint.ts (PR #105)
unwinds cleanly through releaseLeadership.

tests/daemon/loop.spec.ts — real Neon integration. Seeds two pending intents
against the update_obligation_status executor, runs runLeaderLoop with
maxIntents=2, asserts:
  - both intents reach status='done'
  - each produces exactly one cc_actions_log row (attempt=1, key set, status='completed')
  - cc_obligations rows actually moved to 'deferred'
  - cc_node_leases shows leadership released on clean exit
  - log stream contains intent_heartbeat_before / intent_heartbeat_after pairs

Out of scope (not in this PR):
  - new executors (mercury, etc.)
  - production deploy
  - multi-node coordination beyond single-node leader
  - schema additions on cc_intents / cc_actions_log

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
chitcommit pushed a commit that referenced this pull request Jun 10, 2026
P1 entrypoint stub executor → refusal (codex 3352439151):
  The stub returned a sentinel dispatchedTaskId, causing the leader loop
  to call markIntentDispatched + completeIntent and record real pending
  intents as 'done' without execution. Replace with an explicit throw so
  the loop's failure path runs (intent → failed with clear refusal
  message). The real executor lands in PR #107.

P2 entrypoint SIGTERM release sessionId (codex 3353069328):
  releaseLeadership gates on session ownership (codex-p2 PR#101); the
  fallback shutdown release passed no sessionId, so it always no-op'd
  against our own session-stamped lease. Pass the loop's sessionId.

P2 install-daemon-vm.sh devDeps prune (codex 3352439158):
  npm ci --omit=dev pruned typescript, breaking 'npm run build:daemon'.
  Install full deps for the repo build step; runtime image in
   still gets --omit=dev separately.

P2 systemd unit NODE_BIN (codex 3352439162):
  Hard-coded /usr/bin/node breaks nvm / /usr/local/bin installs. Ship
  unit with @@NODE_BIN@@ placeholder; install script substitutes the
  detected node path before installing.

P2 systemd MDWE+JIT (codex 3352439169):
  MemoryDenyWriteExecute=true is documented incompatible with V8 JIT
  and would abort node at startup. Remove the flag; document why.

P2 launchd plist missing env (codex 3352439167):
  launchd has no EnvironmentFile equivalent. Plist invoked node
  directly without DATABASE_URL/NODE_CHITTY_ID/NODE_DESCRIPTOR, hitting
  entrypoint.ts's fatal-missing-env branch on Mac Mini nodes. Add a
  launchd-shim.sh that sources /etc/chittycommand/env (same shape as
  systemd EnvironmentFile) then execs node.

P2 migration 0016 single-upload conflict (codex 3352439160):
  Unique partial index on r2_key broke single /upload's plain
  INSERT...RETURNING — re-uploads now 500'd on unique violation. Add
  ON CONFLICT (r2_key) WHERE (r2_key IS NOT NULL) DO NOTHING to match
  batch path; fall back to SELECT-existing returning 200.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant