feat(agy): Phase A — Antigravity CLI backend for the gemini sibling#35
Closed
michaeljboscia wants to merge 17 commits into
Closed
feat(agy): Phase A — Antigravity CLI backend for the gemini sibling#35michaeljboscia wants to merge 17 commits into
michaeljboscia wants to merge 17 commits into
Conversation
Adds the `agy` (Antigravity CLI) backend for the public `gemini` agent, behind TRIUMVIRATE_GEMINI_BACKEND=agy. Default path is byte-for-byte unchanged (REQ-002); Codex untouched (C4); rollback is config-only (C5). mcp-bridge: GeminiBackend enum + gemini_backend() selector + agy_command() resolution (REQ-001/002/010). triumvirate/agy.rs (new module): - selector wiring at the run_agent_process_with_session seam (REQ-003/005) - pipe capture (verified default) + SIGKILL-process-group timeout + one retry on hang/empty — Go ignores soft signals (REQ-014/020/024/103) - sandbox-exec containment: per-dispatch rendered profile, workspace canonicalized for macOS symlinks; agy's own --sandbox is not a fs boundary, agy -p auto-writes (Issue #45), so this is load-bearing (REQ-016, verified probe4) - subscription-only: never an API key; single-turn, session_id ignored/None, no resume flags (C1, REQ-040/042) - prompt -p, --print-timeout, --log-file; never -o/--model/-r/-c (REQ-011/012/013) - ARG_MAX fail-loud guard (REQ-058) - --log-file model/auth/quota parse, quotaProject= false-positive suppressed (REQ-100) - auth/quota/exec failure classification with remediation text (REQ-034/051/052) - honest lifecycle only, no fabricated events (REQ-050) - TRIUMVIRATE_AGY_CAPTURE=pty fails loud (PTY path lands in Slice 2) Verified: workspace compiles, clippy clean, 6 unit tests (ANSI strip, log parse, quota detector, forbidden-flag guard); assembled command runs e2e against live agy 1.0.2. REQ-060-064 battery re-passed on the 1.0.2 upgrade (FINDINGS updated). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…054) When the agy backend hard-fails, route the gemini-sibling request to a fallback instead of an outage. This is the cutover-safety mechanism for 2026-06-18. - agy is single-attempt at the dispatch loop (REQ-013): no --model faildown chain (agy ignores --model and does its own internal retry), so a failure reaches the degraded route without wasting 5x quota. - degraded route (TRIUMVIRATE_GEMINI_DEGRADED_ROUTE, default gemini-cli,codex; "fail" disables): quota-class failures SKIP gemini-cli (shared Google quota pool) and go straight to codex; auth/exec failures try gemini-cli first. gemini-cli self-disables by failed exec when it retires — no date check (REQ-053 R2). - total wall-clock budget (TRIUMVIRATE_GEMINI_DEGRADED_TOTAL_TIMEOUT_SECS, 900s): each hop runs within min(remaining); budget exhaustion fails loud (REQ-054). - substitution honesty (REQ-053 R3): AskAgentResponse gains answered_by_agent / answered_by_backend / degraded_from_backend / degradation_reason (serde-default + skip-if-none → wire-compatible); a codex hop also prefixes the text. Public `agent` field stays `gemini`. gemini-cli bypasses the selector (else it re-selects agy and loops). Verified: compiles, clippy clean, 10 agy unit tests (classification, route planning quota-skips-gemini-cli, auth-tries-gemini-cli-first, fail-disables). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…(REQ-055/101/102/103) Provider-level resilience in mcp-bridge::agy_resilience (shared state, so the same limits apply to the ask path and, later, fleet — not duplicated per call site). - concurrency cap (REQ-055): process-global tokio Semaphore, default 3 (TRIUMVIRATE_AGY_MAX_CONCURRENT). agy shares one quota pool, so fan-out is bounded. - rate limit (REQ-102): token-bucket RPM ceiling, default 30 (TRIUMVIRATE_AGY_MAX_RPM); mutex never held across await. Throttles self-inflicted 429s. - circuit breaker (REQ-101): trips OPEN on repeated quota; half-open probe after an exponential cooldown (TRIUMVIRATE_AGY_BREAKER_*) capped at the ~5-hr Ultra window; ambiguous repeated failures bias toward OPEN (REQ-103). Modeled as a clock-injectable struct so it is unit-tested without flaky globals. Wiring: - agy.rs acquires a concurrency slot + rate-limit token before every spawn (held for the dispatch), so both ask-path and fleet callers share the cap. - execute_ask_agent: if the breaker is OPEN, skip the agy attempt and route around it with a quota-class reason (degraded route → codex, skips the shared-pool gemini-cli); on the agy attempt, record success / quota / other to the breaker. Verified: compiles, clippy clean, 5 resilience unit tests + 10 agy unit tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
agy has no honest headless token count (verified: /context is TUI-only, the model fabricates a number, no usageMetadata in stdout/log). Record agy dispatches as "unmetered" rather than a fake zero that reads as real spend. - schema: token_records gains usage_source (exact|estimated|unmetered), via an idempotent migration (PRAGMA table_info check + ALTER TABLE ADD COLUMN) so existing DBs upgrade in place; new DBs get it in CREATE TABLE. - TokenRecord/TokenSummaryRow gain usage_source (serde-default "exact" → legacy rows/JSON stay valid). All construction + SELECT/insert sites updated. - agy rows (keyed off parser_mode "agy-*") are written usage_source=unmetered, total_tokens=0, cost_usd=NULL. - excluded from cost/token sums: summary_query skips unmetered and surfaces an honest unmetered_records count; calculate_cost_usd never costs unmetered rows. Dispatch occurrence still counted (record_count). Verified: 19 token-economics tests pass (incl. migration round-trip + a new test asserting unmetered is counted-but-excluded), clippy clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A periodic probe runs `agy -p "2+2"` through the SAME production capture path and records capture vs backend health, surfaced on /health (not the request path). This catches a silent stdout-drop regression that real traffic cannot detect — an empty answer is legitimate on a request but a red flag for a known-non-empty probe. - agy_resilience: AgyHealthSnapshot + agy_record_health + agy_health_snapshot; capture-degraded (empty+exit0) and backend-failed (non-zero) are distinct signals; TRIUMVIRATE_AGY_HEALTH_PROBE_SECS (default 300s). - agy::health_probe(): reuses run_agy_cli_process_with_session; classifies the "empty output" retry-exhausted error as capture-degraded, others as backend-failed; WARN-logs degradation. - main.rs: spawns the probe loop at startup only when backend=agy; /health adds agy_capture_health / agy_backend_health / agy_health_detail / last_probe_unix_ms. Verified: compiles, clippy clean, 6 resilience unit tests (incl. health transitions: capture-degraded keeps backend ok). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…Q-080-083)
Mock-based subprocess reality tests that drive the REAL run_agy_cli_process_with_session
against a mock agy the live binary cannot impersonate on command, plus a real-binary
e2e test (#[ignore], run with --ignored). macOS-gated (the backend targets macOS and
runs under sandbox-exec).
BUG FOUND BY THESE TESTS: exit-0-empty returned Ok("") on the retry attempt — a silent
empty success (violates US-2 "NEVER an empty body reported as a successful answer").
Fixed: empty always retries once then fails loud via last_err.
Tests:
- mock_agy_pipe_captures_plain_text / strips_ansi — happy path + REQ-023/025/040.
- mock_agy_empty_exit0_fails_loud_never_silent — REQ-024 (the bug this caught).
- mock_agy_pipe_drop_is_caught_not_silent — REQ-081 trap (a pipe-only Command::output
refactor would silently pass empty; this fails it).
- mock_agy_quota_exit_classifies_as_quota — REQ-051/053 (feeds breaker + degraded route).
- mock_agy_concurrent_dispatches_carry_no_session_id — REQ-082 (no resume, no synth id).
- gemini_backend selector default/agy/rollback — REQ-001/002/083.
- build_result_is_single_turn_plain_text — REQ-025/040.
- real_agy_end_to_end_two_plus_two (#[ignore]) — live-binary happy path on demand.
Also moved strip_ansi above the test module (clippy items-after-test-module).
Verified: merge gate passes (cargo check + clippy -D warnings, workspace excl pantheon);
17 agy tests pass + 1 ignored.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…y-side) User-driven: instead of a hard cutover, run both Gemini backends side-by-side and compare their responses on real traffic, to build confidence in agy before relying on it. Opt-in (off by default). TRIUMVIRATE_GEMINI_SHADOW=on: every `gemini` request ALSO dispatches the OTHER Gemini backend (gemini-cli primary -> agy shadow, or vice-versa). The primary still answers (.response unchanged); the shadow's answer/latency/error ride along in new AskAgentResponse fields (shadow_backend/shadow_response/shadow_error/shadow_latency_ms, serde-default skip-if-none) and each comparison is appended to <triumvirate_home>/agy-shadow-compare.jsonl for offline review. - shadow is best-effort: agy errors are captured, the request still returns the primary's answer; shadow failures do NOT feed the circuit breaker (observation, not routing). - WARNING surfaced to the operator: doubles usage of the shared Google quota pool, so it's a validation tool, not steady-state. The agy rate limiter + concurrency cap still throttle the shadow side. Verified: merge gate passes (check + clippy -D warnings); 26 mcp-bridge tests incl. shadow_counterpart + gemini_shadow opt-in; shared-types 27 tests. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…059) - version pin: TRIUMVIRATE_AGY_EXPECTED_VERSION (default 1.0.2, the verified-good version) + TRIUMVIRATE_AGY_STRICT_VERSION. The dispatch path runs `agy --version` once (cached) and warns on drift, or refuses under strict mode. Bounds LOCAL drift only — Google's server-side harness updates can still change behavior with the binary pinned (noted). - triumvirate doctor gains an "AGY backend" section: binary runnable, version match, and a real `agy -p "2+2"` probe whose success proves OAuth + capture both work non-interactively (covers the spec's OAuth + probe checks in one call). Verified end-to-end: `triumvirate doctor` against live agy 1.0.2 prints version PASS (1.0.2=1.0.2) and probe PASS (oauth + capture ok); log confirms model=Gemini 3.1 Pro (High), authMethod=consumer. Merge gate passes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fleet had an independent Gemini launch site (orchestrator.rs) that spawned `gemini -p` directly, bypassing agent_exec — at cutover it would spawn the dead gemini binary. - Extracted the agy command assembly (sandbox-exec wrapper + flags + the security- critical sandbox profile) into a shared mcp_bridge::agy module so the ask path and fleet build the IDENTICAL invocation from one source — no duplicated profile (REQ-016/090). - triumvirate::agy now uses mcp_bridge::agy::build_agy_invocation; capture, retry, log parsing stay local. AgyRun carries the parsed log; run_agy_once owns profile+log lifecycle. Behavior identical — all 16 mock subprocess reality tests still pass. - fleet: when TRIUMVIRATE_GEMINI_BACKEND=agy, the gemini arm spawns the shared sandbox-exec invocation (single-turn, no resume flags — REQ-091) instead of `gemini`; pipe capture is fine (agy doesn't drop). Non-zero/empty agy → fleet task fails loud, never a silent success (REQ-092). Temp profile/log reaped by the OS. Note: fleet's Child-return launcher doesn't hold the shared concurrency permit across the child's lifetime, so fleet agy concurrency is bounded by fleet's own wave limits rather than the REQ-055 semaphore (a quota knob, not correctness). Verified: merge gate passes (check + clippy -D warnings); fleet 14 tests, mcp-bridge 28 tests (incl. new agy module), 16 agy subprocess tests. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
agy is single-turn. When a follow-up to a named Gemini session is served by the agy backend, prepend a one-line notice that prior turns did not carry — never silently fake continuity. Fires only on turn 2+ (turn 1 has no prior context to lose) and only when agent=gemini + backend=agy. Covers the MCP ask_session tool and the daemon HTTP proxy path (both route through inter_agent::ask_session). Verified: merge gate passes (check + clippy -D warnings); mcp-tools 29 tests. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… cascade
The bin test suite shared a `Mutex` env_lock via `.lock().expect("env lock poisoned")`.
When any test panicked while holding it (e.g. the pre-existing abe_red_team assertion
failure), the mutex poisoned and EVERY subsequent test that locked env panicked too —
turning 1 real failure into 40. Recover the guard from a poisoned lock with
`unwrap_or_else(PoisonError::into_inner)` at all 44 call sites.
Effect: full bin suite goes from 40 failed → 1 failed, exposing the true state (one
genuine pre-existing failure: abe_red_team_enforcement_blocks_non_compliant_worker,
an ABE/red-team test unrelated to the agy work — it fails identically on the branch
base 76fa522). Test-harness only; no production code changed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
One-time OAuth setup + stable binary path, the operating modes (default gemini-cli, agy cutover, shadow-compare), resilience behavior, version-drift handling, the full env-var table, and rollback. Closes the Phase A docs items. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Regression insurance behind TRIUMVIRATE_AGY_CAPTURE=pty: run agy under a pseudo- terminal so a future agy that regresses to dropping stdout when stdout is not a TTY still produces output. Pipe stays the verified default. - portable-pty declared as a dependency (REQ-021). - run_agy_once_pty: opens a PTY, spawns the shared sandbox-exec invocation into the slave, drops the slave in the parent so EOF is delivered, reads the master on a dedicated OS thread forwarded to async via a channel (REQ-022). Merged stdout+stderr, ANSI-stripped, parser_mode=agy-pty-plain-text. - Non-interactivity guard (PTY only): sets CI/NO_COLOR/TERM=dumb/PAGER/GIT_PAGER and trips on an interactive prompt at the TAIL of output (avoids matching answer text); SIGKILLs the process group on timeout (Go ignores soft signals). - Cargo.lock reconciled (also picks up the stress-tests/trace-capture member + the fleet->mcp-bridge edge from Slice 4b). Completes the REQ-081 trap: the drop-mock FAILS over pipe (caught, not silent) and is RECOVERED under PTY. Verified: clippy gate green; 19 agy tests incl. pty_captures_plain, pty_recovers_tty_only_output, tail_interactive_prompt; real-binary PTY path confirmed via `TRIUMVIRATE_AGY_CAPTURE=pty triumvirate doctor` → probe PASS. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…oute + cap REQ-083: a rollback test that, with TRIUMVIRATE_GEMINI_BACKEND=gemini-cli explicit, dispatches through a real (non-"mock-") gemini-cli script and asserts the path BOTH passes `-o stream-json` (recorded argv) AND parses the NDJSON stream via the real GeminiStreamParser — proving the selector rolled back to gemini-cli, not agy. REQ-055 (fleet): each fleet task holds a shared agy concurrency slot for the lifetime of its child, so fleet's agy fan-out is bounded by the SAME global semaphore as the ask path (not unbounded against the shared quota pool). REQ-092 (fleet): when an agy gemini fleet task fails, degrade to codex before failing loud. Fleet doesn't capture output to classify quota, so it skips the shared-pool gemini-cli and goes straight to codex (the safe cross-provider fallback); on codex success the task is marked completed (degraded_from=agy), else it fails loud as before. Verified: merge gate passes (check + clippy -D warnings); fleet 14 tests, rollback test passes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…L10) Codex (read-only review via ask_agent) found real bugs in the agy backend; fixed: HIGH: - H1 breaker half-open stampede: when cooldown elapsed, every concurrent caller saw HalfOpen and was allowed → stampede. Added a half_open_inflight flag so only ONE probe runs; others keep skipping until it resolves. - H2 PTY kill could hang the runtime: ignored libc::kill result + blocking child.wait(). Now SIGKILLs the group, falls back to portable-pty child.kill(), and reaps via a bounded try_wait poll loop (pty_force_kill_and_reap / pty_wait_bounded) — never a bare blocking wait. Reader thread is detached, not joined, so it can't hang either. - H3 TRIUMVIRATE_AGY_ARGS could smuggle forbidden flags (-c/--model/-o/--log-file/...) and defeat single-turn/output/containment. build_agy_invocation now rejects them (bare + --flag=value forms). - H4 consult sandbox allowed repo writes: dropped @workspace@ from the write-allowlist. Consults read the repo (reads default-allow) but can NEVER write it, even on an auto-approved tool call. Writes confined to ~/.gemini + temp. MEDIUM: - M5 pipe child.wait() now bounded (POST_STDIO_WAIT grace then SIGKILL). - M6 finish_spawn_error now reaps the child (no zombie). - M7 strict version mode re-checks agy --version every dispatch (catches mid-daemon upgrade). - M8 output cap (TRIUMVIRATE_AGY_MAX_OUTPUT_BYTES, default 8 MiB) on both pipe + PTY. - M9 periodic sweep of leaked agy temp files (fleet's child outlives the invocation, so RAII/Drop would race the launch; an age-based sweep is safe for both paths). LOW: - L10 non-interactive env (CI/TERM=dumb/PAGER/GIT_PAGER) now on the pipe path too. (Codex L11 — "empty-output fails diverges from REQ-024" — intentionally kept: US-2 says never report empty as success; for a consult an empty answer is a failure.) Also: build_agy_invocation no longer takes cwd (profile has no workspace write); callers updated. Verified: gate green (check + clippy -D warnings); 31 mcp-bridge + 19 agy tests pass; real agy 1.0.2 still PASS on both pipe and PTY via `triumvirate doctor`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… lease Codex re-reviewed the fixes (e4819a8): confirmed H1/H3/H4/M5/M6/M7/L10 fixed, found 3 new HIGH + 2 MED. Addressed: HIGH: - Fleet agy child waited with no timeout → a hang held the shared concurrency slot forever and the codex-degrade never ran. Bounded the agy wait (connector timeout + grace); on timeout the task completes (slot released), kill_on_drop kills sandbox-exec, and agy self-terminates via its own --print-timeout. (A non-zero agy EXIT still degrades to codex; a TIMEOUT fails loud — degrade-on-timeout is a follow-up.) - Pipe output cap only checked stdout; a runaway stderr slipped past. Now either stream over the cap kills + fails loud. - Half-open breaker could deadlock if the probing request was cancelled before recording a result (inflight flag never cleared). Added a HALF_OPEN_LEASE: a probe outstanding past the lease is assumed abandoned and a new probe may take over. MEDIUM: - PTY clone-reader error path still had a bare blocking child.wait() → now uses the bounded pty_force_kill_and_reap. - pty_force_kill_and_reap now WARN-logs if it can't confirm the reap within the window. Deferred (MED, noted): repos located UNDER a temp path (/tmp, /var/folders) remain writable since temp is broadly writable for agy scratch + our profile/log files; an unusual layout. A per-dispatch temp subdir would be the fix. Verified: gate green (check + clippy -D warnings); 32 mcp-bridge (incl. half-open lease) + 14 fleet + 19 agy tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…e sizing) Round 3 confirmed all round-2 fixes correct (no new HIGH). Addressed the MED/LOW hardening it surfaced: - Fleet timeout now explicitly start_kill()s + bounded-reaps the child before returning the error (no longer relies solely on kill_on_drop); agy's --print-timeout bounds any orphaned grandchild. - PTY "stdout EOF but couldn't confirm exit after force-kill" no longer reports clean success — it returns a failure (output kept as diagnostic) so it routes to the degraded path rather than masking an anomaly. - Half-open lease is now sized relative to the connector timeout (max(30m, 3x)), so a slow-but-alive probe can never trip the lease → no second concurrent probe / stale overwrite; the lease only fires for a genuinely abandoned (never-completing) probe. - Pipe kill paths drop the redundant trailing child.wait() (tokio kill().await already reaps), matching the bounded model. Verified: gate green (check + clippy -D warnings); 32 mcp-bridge + 14 fleet + 19 agy tests pass; real agy 1.0.2 PASS on both pipe and PTY. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase A of the Gemini→Antigravity (
agy) migration: a subscription-only, single-turnagy backend for the public
geminiagent, selectable by env, with the gemini-cli pathleft intact for rollback. Built from
daemon/docs/specs/agy-integration-spec.md(spec'd, goatrodeo-reviewed, verified against the live binary). Stacks on #34 (docs).
Constraints honored
gemini; agy is an internal backend behindTRIUMVIRATE_GEMINI_BACKEND(C3).sandbox-execprofile from day one (agy-pauto-writes; its own--sandboxisn't a fs boundary).What's in it (by slice)
run_agy_cli_process_with_session: pipe capture, SIGKILL-process-group timeout + 1 retry,--log-filemodel/auth/quota parse, ARG_MAX guard, single-turn, ANSI stripTRIUMVIRATE_AGY_CAPTURE=pty(regression insurance), non-interactivity guard, dedicated reader threadmcp_bridge::agy_resilience)/health(catches silent capture regression)triumvirate doctoragy readiness checks#[ignore]test — caught a silent empty-success bug, fixed; rollback test asserts gemini-cli still gets-o stream-json+ parses NDJSON (REQ-083)TRIUMVIRATE_GEMINI_SHADOW): run both backends side-by-side, compare answers, JSONL logPlus a test-harness fix:
env_lockmade poison-tolerant (one panicking pre-existingtest was cascading into 40 false failures; the suite now shows the true 1).
Verified against the live binary (agy 1.0.2)
4, exit 0, under sandbox, log parsed (Gemini 3.1 Pro (High),authMethod=consumer).triumvirate doctor: version PASS, probe PASS — over both pipe andTRIUMVIRATE_AGY_CAPTURE=pty.cargo check+clippy -D warnings, workspace excl. pantheon) green. ~50 new unit/reality tests pass.Operating it
See
daemon/docs/agy-operator-runbook.md. Side-by-side comparison today:TRIUMVIRATE_GEMINI_BACKEND=gemini-cli+TRIUMVIRATE_GEMINI_SHADOW=on.Known gaps (can't close here / Phase B)
--dangerously-skip-permissionstool-exec path (consult-only; conditional req).CONVERSATION_IDverified non-functional; 070 is an evaluation bet).abe_red_team_enforcement_blocks_non_compliant_workerfailure (ABE subsystem, fails identically on the branch base — unrelated to agy).🤖 Generated with Claude Code