diff --git a/MONTECARLO.md b/MONTECARLO.md deleted file mode 100644 index 84ec706..0000000 --- a/MONTECARLO.md +++ /dev/null @@ -1,612 +0,0 @@ -# Monte Carlo fanout — step-by-step orch guide - -A self-contained, fully-worked walkthrough of `llmux orch` using the -canonical fanout pattern: **one coordinator dispatches the same task -to N workers in parallel, waits for all replies, then synthesizes.** -The demo task is Monte Carlo π estimation (each worker throws darts, -the coordinator averages), but the orchestration shape — fan-out → wait -→ fan-in — generalizes to any embarrassingly-parallel multi-agent task -(survey N agents on the same question, run a benchmark across N models, -collect votes, gather alternative implementations of the same prompt). - -If you only want the quickstart, jump to [Run it](#run-it). The rest of -this document walks through every moving part in detail so you can -adapt the pattern to your own task without guessing. - ---- - -## What this demonstrates - -1. **The fanout primitive.** A single agent (`claude-coord`) sends the - same prompt to four other agent sessions, each running a different - CLI vendor (`claude`, `agy`, `opencode`, `codex`). The four workers - reply independently. The coordinator collects all four replies - before producing a final answer. -2. **Method-vs-recipe separation.** The coordinator owns the **method** - (how to aggregate). The workers are **generic** — they have no - Monte Carlo knowledge. The recipe to execute is inlined into each - task body so the workers just follow instructions. This means the - same worker fleet can run *any* fanout task; only the coordinator's - skill file changes per task type. -3. **At-least-once durable messaging.** Every message is a markdown - file in a git repo on disk. You can `git log` the transport at any - point during or after the run and replay exactly what happened. -4. **Reply threading.** Workers' replies carry `re: ` - in frontmatter, so the coordinator knows which dispatch each reply - answers — even though all four arrive interleaved. - ---- - -## Architecture at a glance - -``` - ┌─────────────────────────────┐ - │ llmuxd (HTTP + WS daemon) │ - │ localhost:3001 │ - └──────────────┬──────────────┘ - │ - ┌────────────────────┼────────────────────┐ - │ │ │ - ┌────────▼────────┐ ┌────────▼────────┐ ┌────────▼────────┐ - │ tmux: claude- │ │ tmux: agy │ │ tmux: opencode │ - │ coord │ │ │ │ │ - │ (claude CLI) │ │ (agy CLI) │ │ (opencode CLI) │ - │ alias= │ │ alias=agy │ │ alias=opencode │ - │ claude-coord │ │ │ │ │ - └─────────────────┘ └─────────────────┘ └─────────────────┘ - │ │ │ - │ (each agent's │ │ - │ bash tool runs │ │ - │ `llmux orch …`) │ │ - ▼ ▼ ▼ - ┌────────────────────────────────────────────────────────┐ - │ Orch transport (git repo) │ - │ $XDG_DATA_HOME/llmux/orchestration/ │ - │ data/actors/*.md ← personas + skills │ - │ data/channels/main/ ← messages (sharded by date │ - │ YYYY/MM/DD/*.md dispatches + replies, │ - │ threaded by re:) │ - │ .git/ ← full audit log │ - └────────────────────────────────────────────────────────┘ - - Per-message claim locks live OUTSIDE the transport, in the - machine-local state dir: - $XDG_STATE_HOME/llmux/orch/orchestration/claims/ - ▲ - │ (web UI mirrors transport) - ┌────────────────────┴────────────────────┐ - │ Web: http://:3001/orch │ - │ "Channels" page — threaded inbox, │ - │ alias chips, replied-set ack │ - └─────────────────────────────────────────┘ -``` - -Everything is local-host, local-disk. There's no network round-trip for -message delivery — the bus is a git repo on the same filesystem as the -daemon. Replication to a remote is optional and one-way (DR mirror). - ---- - -## Prerequisites - -### 1. Required CLIs - -```sh -for c in llmux tmux python3 claude agy opencode codex; do - command -v "$c" >/dev/null && echo "[ok] $c" || echo "[MISSING] $c" -done -``` - -- **`llmux`** — `v0.35.0` or newer (orch + fleet support). Install via - `npm i -g @cordfuse/llmux@latest`. Verify with `llmux --version`. -- **`tmux`** — the session backend. v3.2+ recommended (for `-e KEY=VAL` - env injection at session create). -- **`python3`** — used inside the worker recipe for the actual dart - throws (real RNG; LLM-generated randomness is unusable for Monte - Carlo, so the recipe shells out to Python's `random`). -- **Agent CLIs** — `claude`, `agy`, `opencode`, `codex`. Each must be - installed and authenticated independently. The fleet binds one - agent CLI per worker session, so all four need to be operational - before `fleet start` is run. - -If you don't have all four agent CLIs, you can edit `fleet.yaml` -(see [Adapting the fleet](#adapting-the-fleet)) and drop / swap -workers — the demo works with any N ≥ 1 worker. - -### 2. Optional: `gemini` as a fifth worker - -The shipped `fleet.yaml` does not include `gemini`. Gemini CLI's free -OAuth tier was sunset on 2026-06-18; if you have a paid API key the -CLI still works and you can add gemini back as a fifth worker — see -[Adapting the fleet](#adapting-the-fleet). - -### 3. llmuxd running - -```sh -# Check -curl -s http://localhost:3001/api/version -# {"version":"0.36.3"} -``` - -If not running: - -```sh -llmux server start 3001 & -``` - -The daemon hosts the web UI at `http://localhost:3001/` (web nav -label **Channels** at `/orch`) and is what `llmux orch fleet` talks to -when spawning sessions. - -### 4. Disk layout - -After init, two directories matter: - -| Path | What lives there | -|---|---| -| `~/.local/share/llmux/orchestration/` | The transport. **This is a git repo.** Actors, skills, channel messages. Authoritative for the bus. | -| `~/.local/state/llmux/orch/orchestration/` | Orch machine-local state — per-message **claim locks** + the local ack set. Not part of the git transport (a claim is a local truth, not a bus message). | -| `~/.local/state/llmuxd/` | Daemon machine-local state (sessions.json, auth tokens). Unrelated to orch. | - -If `XDG_DATA_HOME` / `XDG_STATE_HOME` are set, those override -`~/.local/share` / `~/.local/state` per XDG spec. - ---- - -## Run it - -This section is the happy path with zero detail. Skip to [Phase-by-phase -walkthrough](#phase-by-phase-walkthrough) if you want to see what each -command actually does. - -```sh -git clone https://github.com/cordfuse/llmux.git -cd llmux - -# 1. One-time: init the transport (creates the git repo on disk) -llmux orch init - -# 2. One-time: install the actors + skill into the transport -cp examples/monte-carlo/actors/*.md \ - ~/.local/share/llmux/orchestration/data/actors/ -mkdir -p ~/.local/share/llmux/orchestration/data/actors/skills -cp examples/monte-carlo/skills/*.md \ - ~/.local/share/llmux/orchestration/data/actors/skills/ -( cd ~/.local/share/llmux/orchestration \ - && git add -A \ - && git commit -m "actors: monte-carlo demo" ) - -# 3. Spawn the fleet (5 tmux sessions: 1 coordinator + 4 workers) and -# fire the coordinator's bootstrap, which kicks off the whole run. -llmux orch fleet start --file examples/monte-carlo/fleet.yaml - -# 4. Watch the coordinator pane print the final π estimate. -tmux attach -t claude-coord # Ctrl-b d to detach -``` - -A successful run prints something like: - -``` -| alias | hits | throws | -| ------------- | ----- | ------ | -| claude-worker | 7865 | 10000 | -| agy | 7853 | 10000 | -| opencode | 7848 | 10000 | -| codex | 7810 | 10000 | -| total | 31376 | 40000 | - -π ≈ 3.1376 (error vs math.pi: 0.004) -``` - -Numbers vary every run (real RNG). π should land within ±0.02 of -`math.pi` (3.14159…). End-to-end takes ~1–3 minutes depending on -which CLI is slowest to wake up. - ---- - -## Phase-by-phase walkthrough - -What each step actually does on disk and over the bus. - -### Phase 1 — `llmux orch init` - -Creates a fresh git repo at `~/.local/share/llmux/orchestration/` -with this layout: - -``` -~/.local/share/llmux/orchestration/ -├── .git/ ← full audit log -├── README.md ← transport-local readme -├── PROTOCOL.md ← message format reference -└── data/ - ├── actors/ ← actor (participant) definitions - │ └── operator.md ← default human operator persona - └── channels/ - └── main/ ← default channel (empty until first send) -``` - -If you run `--remote git@host:llmux-orch.git`, the daemon also -configures a one-way async backup push. The remote is restore-only — -there's no `git pull` semantics; the local transport is authoritative. - -### Phase 2 — Install actors + skill - -The fleet's six personas (one coordinator + four workers + an unused -`gemini.md` worker template) are copied into `data/actors/`. The -coordinator's skill file goes under `data/actors/skills/`. Then a -git commit records the addition. - -After this step: - -``` -data/actors/ -├── operator.md -├── claude-coord.md ← coordinator persona (includes the skill) -├── claude-worker.md ← worker persona -├── agy.md ← worker persona (Antigravity CLI) -├── opencode.md ← worker persona (OpenCode CLI) -├── codex.md ← worker persona (Codex CLI) -├── gemini.md ← worker persona (optional — see adapting) -└── skills/ - └── montecarlo-coordinate.md ← the dart recipe + aggregation formula -``` - -The git commit is what makes them appear in the web UI's alias picker -and in `llmux orch status --json` (which reads from the working tree -but also references commit identity). - -### Phase 3 — `llmux orch fleet start --file examples/monte-carlo/fleet.yaml` - -Reads `fleet.yaml`, then for each session entry: - -1. **Spawn** — if a tmux session with that `name` doesn't exist, - create it via `llmux session start --agent --orch-alias `. - The `--orch-alias` flag injects `$LLMUX_ORCH_ALIAS=` into - the agent's spawn env so the agent's own bash tool can call - `llmux orch send / inbox / reply / ack` without ever passing - `--alias` explicitly. -2. **Bootstrap** — if the session entry has a `bootstrap:` field, - send that text as a one-shot prompt to the agent (via - `llmux session send --body ""`). - -The coordinator's bootstrap is the trigger: - -> Start a Monte Carlo π estimation run NOW. You are claude-coord. -> Read your skill: `cat ~/.local/share/llmux/orchestration/data/actors/skills/montecarlo-coordinate.md`. -> Follow it to completion. Use N=10000 darts per worker. The 4 workers -> are: claude-worker, agy, opencode, codex. Begin. - -The workers' bootstraps are all variants of: - -> You are an llmux orch worker. Poll your inbox via `llmux orch next --alias --json`. -> The task body contains a recipe — follow it exactly. Reply via -> `llmux orch reply --alias `. Then stop. - -So after `fleet start` returns, five tmux sessions exist, all five -agents are alive, the coordinator has read its skill, and the workers -are in polling loops. - -### Phase 4 — Coordinator dispatches - -The coordinator's first action (per the skill) is to send four task -messages, one per worker: - -```sh -llmux orch send --alias claude-coord --to claude-worker --body "throw 10000 darts. ..." -llmux orch send --alias claude-coord --to agy --body "throw 10000 darts. ..." -llmux orch send --alias claude-coord --to opencode --body "throw 10000 darts. ..." -llmux orch send --alias claude-coord --to codex --body "throw 10000 darts. ..." -``` - -Each `orch send` writes a markdown file under -`data/channels/main/`. The filename is sortable (timestamp-derived) so -the inbox sees messages in send order. - -A message on disk looks like this. The **id is the relPath itself** -(date-sharded), not a frontmatter field: - -``` -data/channels/main/2026/06/25/191208123Z-a1b2c3d4.md -``` - -``` ---- -from: claude-coord -to: claude-worker -timestamp: 2026-06-25T19:12:08.123Z ---- - -throw 10000 darts. Use this exact Python script (real RNG — LLM-generated -randomness is unusable for Monte Carlo): - - python3 -c "import random; n=10000; h=sum(1 for _ in range(n) if random.random()**2+random.random()**2<=1); print(h)" - -Reply with strict JSON on one line: - - {"alias":"claude-worker","hits":H,"throws":10000} -``` - -Frontmatter is minimal by design: `from`, `to` (string or list), an -ISO `timestamp`, and an optional `re:` for replies. The channel is -implicit in the directory path (`data/channels//…`). Each -`orch send` is also a git commit, so you can `git log -p` the -transport later and replay the exact prompts the workers received. - -### Phase 5 — Workers claim, execute, reply - -Each worker's bootstrap put it in a loop on `llmux orch next --alias - --json`. `next` is the **claim** verb — it atomically picks the -oldest message addressed to the caller that no one has claimed yet, -records a claim lock in `data/claims.json`, and prints the message. -Two consumers can't double-process the same message because of the -lock. - -The worker: - -1. Reads the task body from `next`'s output. -2. Executes the recipe — in this demo, runs the inline `python3 -c "..."` - to get a hit count. -3. Calls `llmux orch reply --alias ''` - to post a reply. The reply file's frontmatter carries - `re: ` so it's threaded under the original. -4. Per the bootstrap, stops. - -A reply on disk (note `re:` carries the parent's relPath, and `to:` -is set to the parent's `from`): - -``` -data/channels/main/2026/06/25/191246789Z-7f3e1c20.md -``` - -``` ---- -from: claude-worker -to: claude-coord -re: 2026/06/25/191208123Z-a1b2c3d4.md -timestamp: 2026-06-25T19:12:46.789Z ---- - -{"alias":"claude-worker","hits":7865,"throws":10000} -``` - -### Phase 6 — Coordinator polls + aggregates - -While the workers work, the coordinator polls: - -```sh -llmux orch inbox --alias claude-coord --json -``` - -The `--json` form returns `{ messages: [...], nextCursor: "" }`. -The coordinator's skill says to re-poll every ~10s, passing back the -`nextCursor` as `--since` to only get *new* messages, until it has -exactly four replies (one `re:` matching each dispatch it sent). - -Once all four are in, the coordinator runs the aggregation formula -from the skill: - -``` -pi_estimate = 4 * sum(hits) / sum(throws) -``` - -…and prints the markdown table + final π line into its own tmux pane. -That output is what you see when you `tmux attach -t claude-coord`. - -### Phase 7 (optional) — Ack the replies - -The skill suggests: - -```sh -llmux orch ack --alias claude-coord -``` - -…for each reply. `ack` marks a message as processed in the local -ack set so it stops showing up in subsequent `inbox` calls. It does -not delete the message from disk — `git log` still has it. Ack is -just inbox hygiene. - ---- - -## Watching it run - -Multiple useful views, pick any: - -```sh -# Coordinator pane — where the final π lands -tmux attach -t claude-coord # Ctrl-b d to detach - -# Any worker — see it claim, run python, reply -tmux attach -t agy # or claude-worker / opencode / codex - -# Web UI — threaded inbox view -xdg-open http://localhost:3001/orch # the "Channels" page - -# Message count grows in real time (expect 8 = 4 dispatches + 4 replies) -watch -n 2 'find ~/.local/share/llmux/orchestration/data/channels -name "*.md" | wc -l' - -# Git audit log — every send/reply is a commit -git -C ~/.local/share/llmux/orchestration log --oneline - -# Inspect a specific message -ls ~/.local/share/llmux/orchestration/data/channels/main/ | tail -cat ~/.local/share/llmux/orchestration/data/channels/main/.md -``` - ---- - -## Stop, reset, clean up - -```sh -# Kill the 5 tmux sessions (transport preserved — git history is your log) -llmux orch fleet stop --file examples/monte-carlo/fleet.yaml - -# Fully clean slate (wipe transport + claim/ack state). Irreversible. -rm -rf ~/.local/share/llmux/orchestration ~/.local/state/llmux/orch - -# Stop the daemon -llmux server stop -``` - -If you only want to re-run *without* nuking the transport, just -`fleet stop` then `fleet start` again — the previous run's messages -stay in the channel as history, and the coordinator's polling logic -uses `--since` so it won't re-aggregate old replies. - ---- - -## The pieces dissected - -### `fleet.yaml` (annotated) - -```yaml -sessions: - - name: claude-coord # tmux session name (visible in tmux ls) - agent: claude # which agent CLI to spawn — must be in - # llmux's agent registry (claude / agy / - # opencode / codex / gemini / qwen) - orch_alias: claude-coord # bus identity for this session. Sets - # $LLMUX_ORCH_ALIAS in the agent's env - # so `llmux orch …` calls default to it. - bootstrap: | # optional one-shot prompt fired after spawn - Start a Monte Carlo π estimation run NOW. ... - - - name: claude-worker - agent: claude - orch_alias: claude-worker - bootstrap: | - You are an llmux orch worker. Poll your inbox via - `llmux orch next --alias claude-worker --json`. ... - - # ... three more worker entries with the same shape, different agent CLIs -``` - -Optional fields: - -- `cwd:` — working directory for the agent session. Defaults to `$HOME`. - -That's the full schema today (`name`, `agent`, `orch_alias`, `cwd`, -`bootstrap`). If you need extra CLI flags or env vars for an agent, -configure them at the agent-registry level (see `llmux agent list`) -rather than per-fleet-entry. - -**Re-running `fleet start`** skips the spawn for sessions that already -exist with the same name, but it **does re-fire each session's -bootstrap prompt every time**. So if you only want to spawn missing -sessions without re-triggering the coordinator, kill the coordinator's -bootstrap before re-running, or comment out its `bootstrap:` block -temporarily. - -### The actor files - -Actor files are markdown with YAML frontmatter, stored at -`data/actors/.md`. Required frontmatter: - -```yaml ---- -alias: claude-coord # unique on this bus -name: Claude (Coordinator) # human-readable -description: Monte Carlo π estimation orchestrator -species: machine # machine | human -includes: # optional — pull in skills - - ./skills/montecarlo-coordinate.md ---- -``` - -Body is the persona prompt the agent reads when bootstrapping. The -`includes:` field lets a persona reference one or more skill files -that get rendered inline when the agent reads its actor file. That's -how the coordinator gets the Monte Carlo recipe without it being -hardcoded into the agent CLI itself. - -The workers (`claude-worker.md`, `agy.md`, `opencode.md`, `codex.md`) -have no `includes:` — they're deliberately generic. They know how to -poll/claim/reply on the bus, and they execute whatever recipe arrives -in a task body. They have zero Monte Carlo knowledge. - -### The skill — `skills/montecarlo-coordinate.md` - -This is the only Monte Carlo specific file in the whole setup. It -documents: - -- **Dispatch** — what to put in each worker's task body (the inline - python recipe + the JSON reply shape). -- **Wait** — how to poll for replies (`inbox --json` every ~10s, - threading by `re:`, ~3 min hard timeout per worker). -- **Aggregate** — the `pi = 4 * hits / throws` formula and the output - table shape. -- **Cleanup** — optional `ack` per reply. - -To run a *different* fanout task, you write a new skill file and a -new coordinator actor that `includes:` it. The worker fleet is -unchanged. - ---- - -## Adapting the fleet - -### Add gemini as a fifth worker (paid API key required) - -`examples/monte-carlo/actors/gemini.md` is already shipped (in case you -want it). Just add the session to `fleet.yaml`: - -```yaml - - name: gemini - agent: gemini - orch_alias: gemini - bootstrap: | - You are an llmux orch worker. Poll your inbox via `llmux orch next --alias gemini --json`. The task body contains a recipe — follow it exactly. Reply via `llmux orch reply --alias gemini `. Then stop. -``` - -And update the coordinator's bootstrap to dispatch to 5 workers: -change `The 4 workers are: claude-worker, agy, opencode, codex.` to -`The 5 workers are: claude-worker, agy, opencode, codex, gemini.` - -### Drop a worker you don't have installed - -Delete the session entry from `fleet.yaml` and remove that alias from -the coordinator's bootstrap line. The aggregation formula has no -fixed worker count — it sums whatever replies come back. - -### Use the pattern for a non-π task - -The whole point of the method-vs-recipe split is that this is easy: - -1. Write a new skill `data/actors/skills/-coordinate.md` - with three sections: **Dispatch** (what the worker recipe is), - **Wait** (when to stop polling), **Aggregate** (how to combine - replies). -2. Write a new coordinator actor `data/actors/.md` that - `includes:` the new skill. -3. Copy `examples/monte-carlo/fleet.yaml` to a new file, point the - coordinator session at your new actor + change the bootstrap to - trigger the new task. -4. `llmux orch fleet start --file .yaml`. - -The workers don't need to change. They're a reusable fleet. - ---- - -## Troubleshooting - -| Symptom | Cause + fix | -|---|---| -| `MISSING: llmux orch fleet` from prereqs script | Old `llmux` — `npm i -g @cordfuse/llmux@latest`. Requires v0.35.0+. | -| `fleet start` fails with "agent not found" | The agent CLI for that worker isn't installed or isn't on PATH. Either install it, or remove that worker from `fleet.yaml`. | -| Worker session is stuck on a permission prompt (opencode does this for `/tmp`) | `tmux attach -t opencode`, hit Enter to approve "Allow once". Future spawns: set the CLI's "always allow" for the path. | -| Coordinator polls forever, never finishes | One or more workers never replied. `tmux attach -t ` for each worker; investigate the pane. Common: worker hit a rate limit, or the agent CLI failed to invoke `python3`. | -| Coordinator's bootstrap never fires | `llmuxd` wasn't running when `fleet start` ran, or the tmux session already existed before `fleet start` and the bootstrap quietly failed. `llmux server start 3001`, then re-run `fleet start` — bootstraps re-fire for every existing session. | -| Worker claims a message but never replies | Look in `~/.local/state/llmux/orch/orchestration/claims/` for a file named after the message (with `/` → `__`). It's a JSON record `{alias, claimedAt, heartbeatAt}`. Either wait for the claim TTL (5 min) to expire, or `llmux orch release --alias ` to drop the claim manually. | -| Aggregation runs but π is wildly off (>0.1 error) | Check whether all workers actually used real RNG. Some agents may "helpfully" estimate `hits` instead of running the python. The recipe explicitly forbids this — re-read the worker pane and adjust. | -| Web UI at `/orch` shows nothing | Check `llmux orch status` — if transport path is empty, you forgot Phase 2 (no actors installed). | -| Replies arrive but coordinator sees them as un-threaded | The worker called `orch send` instead of `orch reply `. `reply` is what populates the `re:` field. The skill is explicit; if a worker is going off-script, tighten the bootstrap wording. | - ---- - -## Reference - -- **Orch design** — [ORCHESTRATION-DESIGN.md](ORCHESTRATION-DESIGN.md) — full design behind the transport, claim model, and at-least-once semantics. -- **Orch CLI surface** — `llmux orch --help` and `llmux orch --help`. -- **Fleet YAML schema** — top-of-file comments at `examples/monte-carlo/fleet.yaml` and `packages/llmux/src/orch/fleet.ts`. -- **Transport on disk** — `~/.local/share/llmux/orchestration/` is a normal git repo. `git log`, `git show `, `git diff` it freely. Restore from clone with `git clone ~/.local/share/llmux/orchestration`. -- **Web UI** — `http://:3001/orch` (nav label **Channels**). REST endpoints under `/api/orch/*`. -- **Crosstalk sibling** — same on-disk message format, cross-machine system-level multi-user variant: `@cordfuse/crosstalk`. diff --git a/ORCHESTRATION-DESIGN.md b/ORCHESTRATION-DESIGN.md deleted file mode 100644 index 71aa048..0000000 --- a/ORCHESTRATION-DESIGN.md +++ /dev/null @@ -1,235 +0,0 @@ -# llmux Orchestration — Design - -> Status: **draft**, branch `feat/orchestration`. Target release: **v1.0.0** (the "feature-complete enough to drop the leading 0" moment). Cherry-picked from `cordfuse/crosstalk` v7's engine; crosstalk continues independently as the cross-machine + headless-invocation variant. - -## Problem - -llmux today runs **one agent per tmux session**. Sessions are isolated — there's no first-class way for one session's agent to send a message to another and get a reply. Operators wire it up by hand (paste, send-keys, manual nudging). - -We want a **single-host orchestration layer baked into llmux** so sessions can exchange messages durably, with at-least-once delivery and dedup, **without** needing to spin up a separate transport service (crosstalk) and without crossing the user/system privilege boundary. - -Steve's framing — *"an instance markdown says 'run a subagent that watches for git-transport messages and acts on the ones broadcast or targeted for this llmux session.'"* The watcher = the agent's own read-act-write rhythm; the dispatcher = a small in-process loop inside llmuxd. - -## How we got here - -Spitballed across two design rounds (mac+Steve, then me+Steve, 2026-06-20): - -1. **First round** considered SQLite-in-dotfolder. Rejected after deeper review: SQLite is mutable by default, so recovering git's "cumulative-for-free" property required adding events tables, claim TTLs with heartbeat extension, migrations, bespoke debug tooling. At human poll rates, SQLite's perf win isn't material; the schema overhead exceeded the benefit. -2. **Second round** considered bridging crosstalk (system-level) and llmux (user-level) via an adapter. Rejected: crosstalk's design is system-level (`/var/lib/crosstalk-*`, multi-user-eventually), llmux is per-user. Bridging means cross-privilege config references and a `runtime: llmux` flag in crosstalk's models.yaml that points at user-side instance config. Structural impedance. -3. **Landed:** lift the bits from crosstalk that work, drop them into llmux as a user-mode, single-host, single-dispatcher orchestration engine. Git transport in llmux's dotfolder, with an **optional remote for disaster recovery only** (one-way mirror, no pull-rebase semantics needed). - -## Architecture - -``` -┌───────────────────────────────────────────────────────────────┐ -│ llmuxd (user-mode, single per-operator) │ -│ │ -│ ┌────────────────────────────────────────────────────────┐ │ -│ │ tmux sessions (interactive agents) │ │ -│ │ ┌─────────┐ ┌─────────┐ ┌─────────┐ │ │ -│ │ │ session │ │ session │ │ session │ ← agents call │ │ -│ │ │ (alias │ │ (alias │ │ (caller)│ `llmux orch ...`│ │ -│ │ │ bot-a) │ │ bot-b) │ │ │ from bash tool │ │ -│ │ └────┬────┘ └────┬────┘ └────┬────┘ │ │ -│ └──────│───────────│───────────│────────────────────────┘ │ -│ ▼ ▼ ▼ │ -│ ┌──────────────────────────────────────────────────────┐ │ -│ │ Orch CLI: `llmux orch ` │ │ -│ │ inbox · send · reply · release · status · init │ │ -│ └────────────────────┬─────────────────────────────────┘ │ -│ ▼ │ -│ ┌──────────────────────────────────────────────────────┐ │ -│ │ packages/llmux/src/orch/ (cherry-picked + refactored│ │ -│ │ from crosstalk/engine/src/) │ │ -│ │ dispatch.ts · transport.ts (local git only) · │ │ -│ │ resolve.ts · frontmatter.ts · filenames.ts · │ │ -│ │ replies.ts · activation.ts · state.ts │ │ -│ │ + optional async backup-push │ │ -│ └────────────────────┬─────────────────────────────────┘ │ -│ ▼ │ -│ ┌──────────────────────────────────────────────────────┐ │ -│ │ Git repo: $XDG_CONFIG_HOME/llmux/orchestration/ │ │ -│ │ data/channels//YYYY/MM/DD/HHMMSSmmmZ-..md │ │ -│ │ data/cursors/ │ │ -│ │ (optional remote: `git remote add origin ` — │ │ -│ │ push-only, async, for DR only — NOT for sync) │ │ -│ └──────────────────────────────────────────────────────┘ │ -└───────────────────────────────────────────────────────────────┘ -``` - -## What gets cherry-picked from `cordfuse/crosstalk` - -Source path: `cordfuse/crosstalk/engine/src/` → `cordfuse/llmux/packages/llmux/src/orch/`. **One-time fork**, not a sync. llmux owns its copy after the cherry-pick and can diverge freely. - -| Source module | Take? | Refactor on landing | -|---|---|---| -| `dispatch.ts` (461 lines) | yes | drop multi-dispatcher coordination (`dispatchers.ts` collaborator); drop heartbeat-to-`/var/lib/crosstalk-state`; route logs through llmuxd's log buffer | -| `transport.ts` (236 lines) | yes — **heavy trim** | drop `gitPull`/`recoverInterruptedGit`/push-conflict-retry (no remote sync); keep `cursorBaseline` and `newFilesSince`; add `backupPushAsync` (~30 lines) | -| `resolve.ts` (100 lines) | yes | alias resolution as-is; "model" → "alias" rename for llmux semantics | -| `frontmatter.ts`, `filenames.ts`, `replies.ts`, `activation.ts`, `state.ts` | yes | rewire state paths from `/var/lib/crosstalk-state` → `$XDG_STATE_HOME/llmux/orch/` | -| `channel.ts`, `run.ts`, `stop.ts` | maybe | TBD per first-pass scope | - -## What does NOT come from crosstalk - -| Source module | Why not | -|---|---| -| `invoke.ts` (headless `claude --print` spawning) | llmux already runs agents in interactive sessions; agents act via their own runtime, not via dispatcher-spawned subprocesses | -| `models.ts` (yaml registry) | llmux's instance config IS the registry — each instance has an `orch_alias` field; no separate yaml | -| `workflow.ts` (state machines) | out of scope for v1.0; prompt-level concern | -| `dispatchers.ts` (multi-dispatcher coordination) | single-host = single dispatcher; no coordination needed | -| `api.ts` (HTTP server) | llmux already exposes HTTP; orch is a sub-noun of the existing API | -| `init.ts`, `status.ts`, `up.ts`, `down.ts` (operator CLI) | replaced by `llmux orch ` | - -## Net size estimate - -Cherry-pick ~800 lines from crosstalk's ~3000-line engine, simplify to ~500-600 lines as it lands in llmux. Plus ~150 lines of new code for the optional async backup-push and the `llmux orch` CLI surface. - -## Local-git semantics - -The dispatcher operates on a normal git repo at `$XDG_CONFIG_HOME/llmux/orchestration/`. Single writer (llmuxd is the only process making commits); commits are atomic via git's index lock. No remote required. - -### Optional remote (DR only) - -``` -llmux orch init [--remote git@github.com:you/llmux-orch-backup.git] -``` - -If a remote is configured: -- After each commit (or debounced every N seconds, configurable), llmuxd kicks an **async background `git push`**. -- Push failures are logged but never block the dispatcher. The local repo is the source of truth; the remote is a snapshot. -- **No `git pull`**, ever. The remote is write-only from llmux's perspective. -- Restore: operator manually `git clone $XDG_CONFIG_HOME/llmux/orchestration` after disaster. - -This means the remote uses standard SSH/HTTPS git auth (whatever the operator's git already uses) — no new auth path to invent. - -## Message addressing (mirrors crosstalk's PROTOCOL) - -- `to: ` — addressed to a specific participant -- `to: all` — broadcast (all alive watchers see it; claim race resolves) -- `re: ` — reply to a previous message (links into a thread) -- Filenames: `data/channels//YYYY/MM/DD/HHMMSSmmmZ-{8hex}.md` (crosstalk's filename scheme verbatim — chronological, collision-free, sortable) -- Frontmatter: `from`, `to`, `re`, `at`, optional kind/payload fields - -## Actors live in the transport - -The transport repo carries **actor definitions** as well as messages. Same design ethos as crosstalk's `data/models.yaml` (v7) / `local/actors/*.md` (v6): the transport is the source of truth for who's playing on the bus, with all the cumulative/audit/DR-restore benefits that flow from being in git. - -Steve's framing: **actor = skills + personality**. The personality lives in the actor markdown body; skills are referenced via an `includes:` list (resolved against local files in v1.0, against [skills.sh](https://skills.sh) in v1.1+). - -### Storage - -``` -$XDG_CONFIG_HOME/llmux/orchestration/ - data/ - actors/ - bot-a.md - bot-b.md - reviewer.md - channels/ - main/YYYY/MM/DD/HHMMSSmmmZ-{8hex}.md - cursors/ - bot-a - bot-b -``` - -One markdown file per actor in `data/actors/.md`. Filename = alias. - -### Actor file shape - -```markdown ---- -alias: bot-a -name: Bot A -description: Code reviewer specialised in TypeScript and bun -includes: - - ./skills/local/typescript-review.md # local file in transport - # - skills.sh/typescript-review # v1.1+ — resolved against skills.sh ---- - -# Persona - -You are a thoughtful, terse code reviewer. You focus on clarity, -correctness, and minimal abstraction. You favour KISS over clever. -You respond with structured feedback — strengths, concerns, suggestions — -and stop when there's nothing further to add. - -# Skills (auto-included from the includes: list above) -``` - -Frontmatter fields: -- `alias` — must match the filename stem (sanity check on load) -- `name`, `description` — skills.sh-compatible frontmatter (matches the format imprint + imprint-chefremy + toneai-nux-imprint shipped on 2026-06-20) -- `includes` — list of skill refs. v1.0 supports `./relative/path.md` (resolved inside the transport). v1.1+ adds `skills.sh/` (resolved against the canonical registry). - -### How instance config references actors - -The llmux instance config gets a single new field — a pointer, not a definition: - -```ts -interface InstanceConfig { - // ...existing fields (agent, cwd, model, etc.)... - orch_alias?: string; // points at data/actors/.md in the transport -} -``` - -### How llmuxd composes the effective system prompt - -At session start, if `orch_alias` is set: - -1. Read `data/actors/.md` from the transport -2. Resolve the `includes:` list (concat local files; v1.1+ also resolves skills.sh refs) -3. Compose the effective system prompt = - ``` - {actor body} - {included skills, concatenated} - {orch participation stanza} ← see "Participation" below - ``` -4. Hand to the tmux session - -### Edge cases - -- **`orch_alias` set but no actor file** → log a warn, fall back to the instance's inline `system_prompt` (or none if not set) -- **Actor file edited mid-session** → not re-read; only at session start (KISS; matches crosstalk's behaviour) -- **Two instances with the same `orch_alias`** → intentional, this is the concurrent-watcher pattern (multiple workers exercising the claim race) -- **`includes:` ref unresolvable** → log a warn per ref, skip it, continue with whatever resolved - -## Participation: how an llmux instance becomes orchestratable - -1. **Instance config gets `orch_alias?: string`**. If set, the instance is addressable on the bus and llmuxd reads the matching actor file from the transport. -2. **System-prompt stanza** auto-appended (after the actor body + skills) for instances with an alias: - ``` - You are participant `` in the llmux orchestration bus. - Inbox poll: llmux orch inbox --alias - Reply: llmux orch reply "" - Send: llmux orch send --to "" - Poll periodically; act on messages addressed to you or broadcast. - ``` -3. **CLI invocation** via the agent's bash tool. No new IPC, no new SDK — the agent shells out to the existing `llmux` binary. - -## #1845 carry-over (ephemeral-alias semantics) - -First-boot cursor seeds to HEAD. Pre-boot messages to a never-booted alias are not delivered. Same rule we just locked in for crosstalk yesterday; same comment in dispatch.ts. - -## Build plan - -| Phase | Commit | Deliverable | -|---|---|---| -| 1 | `f...` (this commit) | this design doc + branch rename | -| 2 | next | cherry-pick + simplify the 8-9 modules into `packages/llmux/src/orch/`; local-git transport only (no remote yet); typecheck passes | -| 3 | next | actor loader + `includes:` resolver (local files only); `llmux orch init` lays down `data/actors/` skeleton | -| 4 | next | optional async backup-push + `llmux orch init --remote` flag | -| 5 | next | `llmux orch ` CLI surface + instance-prompt stanza wiring + `orch_alias` instance-config field | -| 6 | next | smoke test (two simulated sessions + broadcast claim-race + reply round-trip + actor-loaded prompt assembly) | -| 7 | release | bump to v1.0.0, PR, merge, tag, publish | -| v1.1 | follow-up | `skills.sh/` resolver inside the `includes:` list | - -## Crosstalk stays put - -`cordfuse/crosstalk` continues as the **cross-machine + headless-invocation** variant. It is NOT deprecated. The two products diverge cleanly: - -- **crosstalk** — system-level, multi-host, dispatches headless agents (`claude --print`), git transport with remote sync -- **llmux orch** — user-level, single-host, agents act inside their own interactive sessions, git transport with optional DR-only remote - -Cherry-pick is a **fork**, not a sync. After phase 2, llmux's copy of the dispatcher diverges as needed; crosstalk's continues to evolve for its use case. No coupling. - -— cachy, 2026-06-20 diff --git a/README.md b/README.md index a9fc06d..4cdcb08 100644 --- a/README.md +++ b/README.md @@ -260,7 +260,7 @@ session/agent verb and it routes over HTTP instead of operating locally: ```bash export LLMUX_SERVER=http://192.0.2.10:3001 # or https://.tailnet.ts.net -export LLMUX_TOKEN=sas_… # mint with `llmux token create` +export LLMUX_TOKEN=sas_… # mint with `llmux token create --username ` llmux session list llmux session prompt main "tomorrow's plan?" diff --git a/V2-SYSTEM-AUTH-DESIGN.md b/V2-SYSTEM-AUTH-DESIGN.md index 28be39b..a05361d 100644 --- a/V2-SYSTEM-AUTH-DESIGN.md +++ b/V2-SYSTEM-AUTH-DESIGN.md @@ -1,5 +1,14 @@ # llmux v2 — System-Mode Daemon + Application-Layer Multi-User Auth +> **Note (v0.37.0, 2026-06-26):** orch references in this doc describe +> the design as originally conceived. The orch bus + `/api/orch/*` +> surface + `LLMUX_ORCH_ALIAS` env + the `/var/lib/llmux/orchestration/` +> transport directory were yanked from llmux in v0.37.0; v2 user-account +> auth (the substantive content of this doc) survives without them. Read +> the orch passages as historical context for *why* v2 auth was designed +> with identity binding — the binding still earns its keep against the +> v2 token store, just no longer for orch message authorship. + > Status: **design draft**, branch `feat/v2`. Target release: **v2** (when the multi-tenant trigger conditions surface — see V1.x vs V2 below). > > This document describes the v2 architecture: a single-service-user daemon owning all its own state, with multi-user authentication implemented at the application layer. **The daemon never touches any `/home/*` path.**