A multi-AI coding agent, desktop cockpit, and personal companion that writes code, runs commands, searches the web, talks to you, sees opt-in camera context, and manages your projects -- from the terminal, Cowork, your phone, or a 24/7 background server.
Now with a Multi-AI Fleet Hub — multiple Code Buddy peers can observe each other's events live and invoke each other's LLMs via /fleet send peer.chat. Pilot local Ollama LLMs from any peer in your Tailscale network for free coding/reasoning. (Phase (d).1 → (d).16a, May 2026.)
Cowork + Buddy companion cockpit — buddy gui / buddy desktop opens the Electron workspace powered by the same Code Buddy core engine. Buddy has a companion panel for ChatGPT-backed identity, voice, camera snapshots, MediaPipe face/hand/pose perception, self-evaluation, competitive radar, missions, routines, safety, and proactive check-ins.
Auto-memory writeback — the agent persists what it learns (preferences, decisions, gotchas) to .codebuddy/CODEBUDDY_MEMORY.md across sessions. View with /memory recent. buddy --init also generates an AGENTS.md at the project root for cross-CLI compatibility (read by Claude Code, Gemini CLI, Cursor, Codex). (rc.2, May 2026.)
Quick Start | Cowork + Companion | Features | Documentation | Contributing
Code Buddy is an open-source multi-provider AI coding agent with a terminal UI, HTTP/WebSocket server, and Cowork desktop app. It supports 15 LLM providers with automatic failover and per-provider circuit breakers. It works as both a development tool (reads files, writes code, runs commands, creates PRs, plans complex tasks) and a personal companion (bidirectional voice conversation, durable memory, opt-in camera perception, screen/presence context, push notifications via 20+ messaging channels, and 24/7 background operation). With buddy login, a ChatGPT Plus / Pro subscription can become the flat-fee brain of the system without API-key metering.
ChatGPT Pro / Plus subscription login (Phase d.23+) — buddy login, sign in once with your ChatGPT account, then chat with gpt-5.5 directly from the terminal. No API key, cost reported as $0.0000 (flat-fee plan).
Interactive TUI + tool calling. The agent reads project context, calls tools in parallel (web_search ×2 for the screenshot below), and streams the synthesised answer.
Self-audit. Asked to find a bug in its own integration code, gpt-5.5 reads src/codebuddy/providers/provider-chatgpt-responses.ts, identifies a stale-variable issue (mutated body.model not propagated), and proposes the exact fix:
More captures + walk-through: docs/screenshots/.
# Install
npm install -g @phuetz/code-buddy
# Option A — bring your own API key
export GROK_API_KEY=your_api_key # or GEMINI_API_KEY / OPENAI_API_KEY / ANTHROPIC_API_KEY
buddy
# Option B — log in with your ChatGPT Plus / Pro subscription (no API key needed)
buddy login # opens browser for OAuth → tokens persisted
buddy whoami # ✅ connected · your.email@example.com · Plan: pro
buddy # auto-routes to gpt-5.5 via the Codex backend, cost $0.0000
# Or with a specific task
buddy --prompt "analyze the codebase structure"
# Full autonomy
buddy --yoloSee Getting Started for installation options, headless mode, session management, and typical workflows. The ChatGPT Codex OAuth flow is documented with screenshots in docs/screenshots/.
Cowork is the desktop cockpit for Code Buddy: chat, tools, traces, workflows, settings, permissions, models, MCP connectors, skills, artifacts, and companion controls all run against the same core agent as the CLI. The Code Buddy settings panel can probe the local backend, start it when needed, discover models, and route Cowork turns through the embedded engine or a configured server endpoint.
# First-time identity and flat-fee brain route
buddy login
buddy companion setup
# Local backend for Cowork, Fleet, and OpenAI-compatible clients
buddy server --port 3000
# Launch the desktop app from an installed build
buddy gui
# or
buddy desktop
# Source checkout dev loop
npm install
npm run build
npm run dev:guiBuddy companion commands are available in both the CLI and Cowork panel:
buddy companion status
buddy companion self
buddy companion evaluate
buddy companion radar
buddy companion impulses
buddy companion missions sync
buddy companion missions run-next
buddy companion safety recent
buddy companion camera status
buddy companion camera snapshot
buddy companion percepts recentThe camera bridge is explicit and local: snapshots are opt-in, percepts are append-only under .codebuddy/companion/, and Cowork uses MediaPipe Tasks Vision for face, hand, finger-tip, and pose signals. Face enrollment/presence recognition lives in Cowork's presence bridge and uses the local MediaPipe/Buffalo_S pipeline.
For low-latency voice experiments, Cowork can route STT/TTS through a local Kyutai DSM / moshi-server endpoint while keeping faster-whisper and Piper as fallbacks:
$env:COWORK_VOICE_PROVIDER='kyutai'
$env:COWORK_KYUTAI_URL='ws://127.0.0.1:8080'
npm run dev:guiUse Buddy companion's Inspect voice action to probe the active route, Kyutai STT/TTS websocket reachability, ffmpeg availability, and the Piper/faster-whisper fallback state from the Electron app.
From source, Cowork requires Node.js >=22 in cowork/; the root CLI still supports Node.js >=18.
| Category | Highlights | Docs |
|---|---|---|
| AI Providers | 15 providers (Grok, Claude, GPT, Gemini, Ollama, LM Studio, AWS Bedrock, Azure, Groq, Together, Fireworks, OpenRouter, vLLM, Copilot, Mistral), circuit breaker, model pairs | providers.md |
| Tools | ~110 tools with RAG selection, multi-strategy edit matching, Codex-style apply_patch, streaming, BM25 tool search, code exec sandbox | tools-reference.md |
| Commands | 190+ slash commands, CLI subcommands (/dev, /pr, /lint, /switch, /think, /batch, /watch, /conflicts, /vulns, /replace) |
commands.md |
| Cowork Desktop | Electron cockpit, embedded Code Buddy engine, backend health/start controls, model settings, permission rules, visual workflows, traces, artifacts, MCP/skills/plugin management | cowork/README.md, cowork/ARCHITECTURE.md |
| Buddy Companion | ChatGPT-backed identity, voice/TTS, proactive check-ins, self-evaluation, competitive radar, mission board, learned routines, safety ledger, local percept journal | commands.md |
| Vision & Presence | Opt-in webcam snapshots, MediaPipe face/hand/pose/finger-tip analysis, local face enrollment, presence state for the agent | cowork/ARCHITECTURE.md |
| Agents | Multi-agent orchestration (5-tool API), 8 specialized agents, SWE agent, planning flow, A2A protocol, batch decomposition, agent teams | agents.md |
| Reasoning | Tree-of-Thought + MCTS (4 depth levels), extended thinking, auto-escalation, /think command |
reasoning.md |
| Security | Guardian Agent (AI risk scoring), OS/Docker/OpenShell sandbox, SSRF guard, secrets vault, write policy, exec policy, loop detection, omission detection, output sanitizer | security.md |
| Channels | 20+ messaging channels (Telegram, Discord, Slack, WhatsApp, Signal, Teams, Matrix, IRC, and more), DM pairing, send policy | channels.md |
| Context Engine | Smart compression, tool output masking, image pruning, transcript repair, pre-compaction flush, restorable compression, JIT context, importance-weighted window | context-engine.md |
| Infrastructure | HTTP server (OpenAI-compatible), WebSocket gateway, daemon mode, cron, device nodes, canvas/A2UI, 6 cloud deploy configs, MCP, plugins | infrastructure.md |
| Configuration | Env vars, TOML config with profiles, model-aware limits, per-agent params, i18n (6 locales), personas | configuration.md |
| Development | TypeScript strict, Vitest (27,334 tests), ESM, middleware pipeline, facade architecture | development.md |
- Voice: 7 TTS providers, wake word detection, voice-to-code pipeline, hands-free companion conversation
- Companion Loop: readiness checks, self-state snapshots, self-evaluation, competitive radar, impulses, missions, routines, and safety review
- Vision: local camera snapshots, MediaPipe face/hand/pose/finger-tip percepts, face enrollment, and presence state for context-aware collaboration
- Memory: Persistent + semantic + prospective + decision + coding style memory, ICM cross-session memory
- Knowledge: Knowledge base injection, 40 bundled skills, self-authoring skills at runtime
- Git Workflow: Auto-commit (Aider-style),
/prcreation, merge conflict resolver, ghost snapshots - Code Intelligence: LSP rename/refactor, auto-import, bug finder (25+ patterns, 6 langs), OpenAPI generator, log analyzer
- IDE Integration: VS Code extension (diff view, inline edit, model switch), JetBrains plugin, LSP server
- Inline Context:
@web,@git,@terminalmentions for contextual references - Workflows: Lobster typed DAG engine with approval gates, pause/resume tokens
| Document | Description |
|---|---|
| Getting Started | Prerequisites, install, first run, headless mode, session management |
| Providers | All 15 providers, connection profiles, model pairs, circuit breaker |
| Tools Reference | Tool categories, RAG selection, edit matching, apply_patch, streaming |
| Commands | All slash commands, CLI subcommands, companion commands, global flags |
| Cowork README | Desktop installation, features, source build, sandbox modes |
| Cowork Architecture | Electron contexts, bridges, embedded engine, persistence, runner model |
| Agents | Multi-agent orchestration, roles, SWE agent, planning flow, A2A |
| Reasoning | Extended thinking, Tree-of-Thought, MCTS, /think command |
| Security | Permission modes, Guardian Agent, sandboxing, SSRF, secrets vault |
| Channels | 20+ messaging channels, DM pairing, send policy |
| Context Engine | Compression, tool output masking, JIT context, pre-compaction flush |
| Infrastructure | HTTP server, WebSocket gateway, daemon, cron, deploy, plugins |
| Fleet Guide | Multi-AI hub — /fleet listen + /fleet send peer.chat, env-driven multi-provider auto-detect, Tailscale lab examples |
| Configuration | Environment variables, TOML config, project settings, model limits |
| Development | Build, test, architecture, coding conventions, adding tools |
Latest local verification for the Cowork + Buddy companion loop (2026-05-24):
npm run typecheck
cd cowork && npm run typecheck
cd cowork && npm run build:e2e
cd cowork && npx vitest run tests/kyutai-bridge.test.ts tests/voice-bridge.test.ts tests/tts-bridge.test.ts
cd cowork && npx playwright test e2e/cowork-smoke.spec.ts --reporter=line --workers=1
cd cowork && npx playwright test e2e/companion-panel.spec.ts --reporter=line --workers=1
cd cowork && npx playwright test e2e/recent-features-smoke.spec.ts e2e/companion-panel.spec.ts --workers=1
cd cowork && npx playwright test e2e/codebuddy-settings.spec.ts e2e/recent-features-smoke.spec.ts e2e/companion-panel.spec.ts e2e/companion-live.spec.ts --reporter=line --workers=1
cd cowork && npx vitest runResult: root and Cowork typechecks passed, Vite E2E build passed with existing chunk/dynamic-import warnings, Kyutai/voice bridge unit tests passed (31 tests), focused companion panel Playwright passed (1 test), recent-features IPC smoke passed (9 tests), Cowork smoke passed (29 tests), focused companion/settings/recent-feature Playwright suite passed (11 passed, 1 live hardware test skipped unless COWORK_LIVE_COMPANION=1), and Cowork Vitest passed (205 files, 1318 tests).
git clone https://github.com/phuetz/code-buddy.git
cd code-buddy
npm install
npm run dev # Development mode
npm test # Run all tests
npm run validate # Lint + typecheck + test (run before committing)See Development for architecture details and coding conventions.
MIT License - see LICENSE for details.
Report Bug | Request Feature | Star on GitHub
Multi-AI: Grok | Claude | ChatGPT | Gemini | LM Studio | Ollama | AWS Bedrock | Azure | Groq | Together | Fireworks | OpenRouter | vLLM | Copilot | Mistral