diff --git a/.env.example b/.env.example index 1dc480c..462a595 100644 --- a/.env.example +++ b/.env.example @@ -3,6 +3,9 @@ # OPENCLAW_BASE_URL=http://127.0.0.1:18789 # OPENCLAW_API_KEY= +# OpenRouter (voice messages, Ask mode, smart titles) is configured in the app +# under Settings → "Voice & Ask", not here — the key is stored in the local DB. + PORT=3000 # Default (when unset): ~/.iclaw/data/iclaw.db # DB_PATH=./data/iclaw.db diff --git a/AGENTS.md b/AGENTS.md index 8e1128d..283e6b1 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -19,7 +19,7 @@ These are intentional non-goals. Don't add them. If a request lands here that ne - **Not a generic OpenAI-compat client.** We migrated off `/v1/chat/completions` in commit `a92c10e`. Do not re-introduce an HTTP chat completion path or SSE-based streaming — everything chat-related is native WS. - **Not a multi-provider chat tool.** No fallback to Ollama, vLLM, Claude API, OpenAI direct, etc. OpenClaw is the only backend; switch model providers in `openclaw.json`. - **Not a remote/hosted product.** Loopback-only. No multi-user, no auth layer, no team features. Threat model is "this user on this machine". -- **Not a re-implementation of OpenClaw's agent runtime.** We don't compact transcripts, run tools, manage `MEMORY.md`, or wrap the LLM. We display what the gateway produces and (separately) maintain a project-scoped fact layer that's *additional* to OpenClaw memory, not a replacement. +- **For the default Full Power chat, not a re-implementation of OpenClaw's agent runtime.** On that path we don't compact transcripts, run tools, or wrap the LLM — we display what the gateway produces and (separately) maintain a project-scoped fact layer *additional* to OpenClaw memory. (The Work / Safe work / Incognito modes are the deliberate exception — a small, self-contained agent loop in `packages/iclaw-runtime`; see below. Don't grow it into a general OpenClaw replacement.) - **Not heavyweight on the frontend.** No build step beyond `tsc`. Plain CSS, vanilla JS on the client, EJS for views. If you want to add React/Vue/Svelte, open an issue first — the answer is probably no for the current scope. ## Architecture you need to know @@ -62,6 +62,26 @@ These are intentional non-goals. Don't add them. If a request lands here that ne Browser ↔ server is **one persistent WebSocket** at `/ws`. Chat traffic (send, abort, exec approval, streaming turn events, cross-tab sync) flows through it. HTTP routes exist for page rendering (EJS), form actions, the `/media/*` proxy, and JSON endpoints under `/api/gateway/*`. Every HTTP mutation that touches a chat also emits the matching `chat-updated` / `chat-deleted` over WS so other tabs catch up instantly. +### Runtime modes (Work / Safe work / Incognito) + +The default **Full Power** mode goes to the gateway (above). The other three modes +are served by the bundled **iClaw runtime** (`packages/iclaw-runtime`), a small +agent runtime kept deliberately minimal: + +- The host calls it over HTTP on `127.0.0.1:7430` (`src/services/workRuntime.ts` → + runtime `index.ts`). The model loop runs **on the host** via OpenRouter. +- Tool/shell execution is isolated in a **Docker sandbox**, one model per file: + `secure-runner.ts` (Safe work / Incognito — everything runs in a per-turn + container) and `work-container.ts` (Work — agent loop + file tools on the host, + only `run_command` containerized with per-folder `:ro`/`:rw` mounts). Both share + ONE image, `container/secure-sandbox.Dockerfile` (tag `iclaw-secure:latest`, + built via `npm run build:secure-image`); `node:22` is only an emergency fallback. +- The live runtime is just 9 files (`index.ts`, `sessions.ts`, `secure-runner.ts`, + `work-container.ts`, `agent/{loop,tools,security,prompt-dump}.ts`, `log.ts`). + Path/secret enforcement lives in `agent/security.ts`. Needs Docker + an + OpenRouter key; without either, these modes are unavailable and fall back to + Full Power. This is NOT the OpenClaw path — keep the two cleanly separate. + ### The canonical client paths | What you want | Use | diff --git a/CHANGELOG.md b/CHANGELOG.md index f15a769..ae718c3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,15 @@ All notable changes to iClaw are documented here. The format follows [Keep a Cha - **Hover-intent auto-close (3.5 s)** for both menus. Mouse leaves the menu → 3.5 s timer; returns → timer resets. Replaces the schedule menu's old 10 s blanket timeout. - **Dynamic favicon.** The browser tab icon is now canvas-rendered with Apple-style rounded corners, and carries up to two status dots aggregated across all chats/tasks: 🟠 needs-human, 🔵 chat ready / task-review, 🟢 working (priority orange → blue → green). It's a derived view of the sidebar status dots — repainted only when the verdict actually changes (debounced), never animated, so steady-state cost is zero. +#### Remote Access (E2E alpha) +- Open the local iClaw UI from another device through an **iclaw-relay** tunnel — no inbound port, no SSH forwarding (**Settings → Remote Access**). Two setups: *local mode* (iClaw + OpenClaw on one laptop) and *host mode* (always-on machine, browse from phone/laptop). Enabled by `ICLAW_RELAY_URL` + `OPAQUE_SERVER_SETUP` on the iClaw host. +- **OPAQUE login** (`@serenity-kit/opaque`): the tunnel passphrase is never sent in plaintext over the wire; a tunneled `POST /__ra/login` is rejected (426) in favour of the OPAQUE `start`/`finish` handshake. +- **End-to-end encrypted transport**: HTTP and WebSocket payloads are encrypted between the browser and the *local* iClaw (`/__ra/e2e/http`, `/__ra/e2e/ws`); the relay forwards opaque envelopes (`ct`, `sid`) and sees only metadata (subdomain, timing, sizes). +- **Trusted device sessions**: Ed25519 challenge-response lets a browser reconnect without retyping the passphrase; devices are listed and individually revocable in Settings. +- **Relay access-token gate** (`?access=` → HttpOnly cookie) blocks visitors who only guess the subdomain. **New access link** rotates the token and forbids old links; **Disable** revokes the tunnel and its relay registration. +- Adversarial smoke bundle (`npm run test:ra-smoke`) plus a relay frame-capture scanner (`npm run scan:relay-capture`) assert no plaintext passphrase, cookie, HTML, or chat JSON ever crosses the relay. +- **Alpha** — not externally audited; the relay still sees routing metadata. Security model and setup in [docs/REMOTE_ACCESS.md](../docs/REMOTE_ACCESS.md). + ### Removed - Native browser tooltip on sidebar chat items (`title=""`). The full title is already visible inline; the tooltip just got in the way of the new hover-hold gesture. diff --git a/README.md b/README.md index 5a87464..07906c4 100644 --- a/README.md +++ b/README.md @@ -5,13 +5,12 @@

iClaw

- Chat UI for OpenClaw Gateway — runs on your machine, stores history locally. + Minimalist agent UI - give it a project, it works only within it. Isolated memory, isolated folders

- CI - npm npm downloads + Reddit MIT

@@ -19,41 +18,33 @@ ## Install -Requires [Node.js 20+](https://nodejs.org) and a running [OpenClaw Gateway](https://docs.openclaw.ai). +Requires [Node.js 20+](https://nodejs.org) ```bash npx @iclawapp/iclaw ``` -The terminal shows a short status line and **press `g`** to open the UI in your browser. It picks the next free port if `3000` is busy, and exits without starting a second copy if iClaw is already running. +Press **`g`** in the terminal to open the UI in your browser -Your chat history is saved to `~/.iclaw/data/iclaw.db`. - -Optional env vars: `PORT` (preferred port, default `3000`), `ICLAW_OPEN_BROWSER=1` (also open the browser tab on start). - -## Encrypted chat sharing (optional) - -Hit **Share** in any chat to get an encrypted link. The chat is encrypted in your browser (AES-256-GCM); the server stores ciphertext only. Supports password protection, burn-after-read, and TTL. - -Powered by [iClaw-cloud](https://github.com/iClawApp/iClaw-cloud) — defaults to `https://app.iclaw.digital`. - -## Remote Access (alpha) +## Star history -Open the iClaw UI from another device through an **iclaw-relay** tunnel (Settings → Remote Access). +[![Star History Chart](https://api.star-history.com/svg?repos=iClawApp/iClaw&type=Date)](https://www.star-history.com/#iClawApp/iClaw&Date) -**Security model (summary):** +## Chat modes -- **Relay access token** — blocks visitors who only guess the subdomain. -- **OPAQUE login** — passphrase is not sent in plaintext over the tunnel. -- **Encrypted HTTP/WebSocket** (E2E alpha) — payloads are encrypted between the browser and **local** iClaw; the relay forwards encrypted frames only. -- **Device sessions** — trusted browsers can reconnect without retyping the passphrase. -- **Alpha** — not externally audited. The relay still sees metadata (subdomain, timing, sizes, E2E endpoint paths). +| Mode | What it does | Backend | +| --- | --- | --- | +| **Work** | AI edits files in folders you pick; you approve every change. | iClaw runtime | +| **Safe work & Internet research** | Locked Docker sandbox — run untrusted code and research the web. | iClaw runtime | +| **Full Power** (default) | The full OpenClaw agent — files, tools, shell, browser. | OpenClaw Gateway | +| **Incognito** | Private, read-only research — reads files & the web, never writes, nothing saved. | iClaw runtime | -Two setups: **local mode** (iClaw + OpenClaw on the same laptop) and **host mode** (always-on machine, browse from phone/laptop). See [docs/REMOTE_ACCESS.md](../docs/REMOTE_ACCESS.md). +The three runtime modes need **Docker** running and an **OpenRouter key** (Settings); without either the composer falls back to Full Power. Architecture: [AGENTS.md](AGENTS.md). -Env on the iClaw host: `ICLAW_RELAY_URL` and `OPAQUE_SERVER_SETUP` (required when Remote Access is enabled). +## More ---- +- **Encrypted chat sharing** — hit **Share** in any chat for an end-to-end encrypted link (AES-256-GCM; password, burn-after-read, TTL). Powered by [iClaw-cloud](https://github.com/iClawApp/iClaw-cloud). +- **Remote Access** (alpha) — reach the UI from another device over an iclaw-relay tunnel (Settings → Remote Access). Not externally audited yet — see [docs/REMOTE_ACCESS.md](docs/REMOTE_ACCESS.md). ## For developers @@ -62,28 +53,9 @@ git clone https://github.com/iClawApp/iClaw.git cd iClaw && npm install && npm run dev ``` -| | Default | -| --- | --- | -| Web UI | http://localhost:3000 | -| Gateway | http://127.0.0.1:18789 (`OPENCLAW_BASE_URL`) | - -Optional env vars: [.env.example](.env.example). -Architecture + coding rules: [AGENTS.md](AGENTS.md). -Remote Access alpha: [docs/REMOTE_ACCESS.md](../docs/REMOTE_ACCESS.md), smoke [docs/REMOTE_ACCESS_SMOKE.md](../docs/REMOTE_ACCESS_SMOKE.md). - -```bash -npm run test:ra-smoke # E2E adversarial smoke (vitest) -npm run scan:relay-capture -- frames.ndjson # scan relay frame capture -``` - -## Star history - -[![Star History Chart](https://api.star-history.com/svg?repos=iClawApp/iClaw&type=Date)](https://www.star-history.com/#iClawApp/iClaw&Date) - -## Contributing - -See [CONTRIBUTING.md](CONTRIBUTING.md), [ROADMAP.md](ROADMAP.md), [CHANGELOG.md](CHANGELOG.md). Bug reports and small PRs welcome — for bigger changes open an issue first. +Web UI on http://localhost:3000, Gateway on http://127.0.0.1:18789 (`OPENCLAW_BASE_URL`). Env vars: [.env.example](.env.example) · Architecture & coding rules: [AGENTS.md](AGENTS.md). -## License +## License and contributing +[CONTRIBUTING.md](CONTRIBUTING.md) · [ROADMAP.md](ROADMAP.md) · [CHANGELOG.md](CHANGELOG.md) MIT — see [LICENSE](LICENSE). diff --git a/docs/project-skills-spec.md b/docs/project-skills-spec.md new file mode 100644 index 0000000..2559025 --- /dev/null +++ b/docs/project-skills-spec.md @@ -0,0 +1,563 @@ +# Per-Project Skills (Inbox-Gated Self-Learning) — Implementation Spec + +> Audience: an engineer/agent implementing this feature in the iClaw monorepo. +> Read this top-to-bottom before writing code. It tells you exactly which +> existing code to mirror, what to build, and in what order. + +## 0. One-paragraph summary + +Give iClaw a **closed learning loop scoped to projects**: after a chat turn (or +session), a **background reviewer** distills reusable *procedural skills* from +what just happened. Distilled skills do **not** activate automatically — they +land in a **per-project inbox** as *suggestions*. The user accepts/edits/rejects +them (exactly like the existing **project fact suggestions**). Accepted skills +become **active** project skills, stored as **`SKILL.md` markdown** (agentskills.io +format), and their one-line `description` is injected as a **summary index** into +the Work/Secure/Execute system prompt; the full body is loaded on demand. The +user sees a **Skills panel per project** showing each skill's name + description +(the "summary per skill" the product owner asked for). This mirrors Hermes's +self-improvement loop but adds a human-acceptance gate, which neutralizes the two +big risks (skill sprawl and prompt-injection-planted skills). + +This is the **procedural** half of memory. iClaw already has the **declarative** +half (project *facts*). We are duplicating the facts pattern for skills, then +layering Hermes's distillation + the SKILL.md format on top. + +--- + +## 1. Existing patterns to MIRROR (read these first) + +The whole feature is "facts, but richer." Study these before designing: + +| Concern | Facts implementation (mirror it) | +|---|---| +| DB tables | `src/db/database.ts` → `project_facts`, `project_fact_suggestions` (lines ~52–77) | +| Data layer | `src/services/store.ts` → `projectFacts` (~368) and `projectFactSuggestions` (~452) objects | +| Types | `src/types/index.ts` → `ProjectFact`, `ProjectFactSuggestion` | +| Distillation pipeline | `src/services/projectMemory.ts` → `scheduleProjectFactExtraction`, `runProjectFactExtraction`, `runThrowawayTurn`, `isDuplicateFact`, `parseExtractedFactLines` | +| Pipeline trigger | `src/services/chatRunner.ts:676` calls `scheduleProjectFactExtraction({...})` after a successful assistant reply | +| Prompt injection (Work/Secure) | `src/services/chatRunner.ts` `buildWorkSystemPrompt` (~849) prepends facts; the runtime composes the final system prompt in `packages/iclaw-runtime/src/agent/loop.ts` `buildSystemPrompt` | +| Prompt injection (Execute/gateway) | `src/services/projectMemory.ts` `buildGatewayUserMessage` prepends facts to the gateway turn | +| REST: inbox accept/reject | `src/routes/chats.ts` → `GET /:id/fact-suggestions`, `POST /:id/fact-suggestions/:sid/accept`, `.../reject` (~98–158) | +| REST: active-item CRUD | `src/routes/projects.ts` → `PATCH /:id/facts/:factId`, `POST /:id/facts/:factId/delete` (~210–237) | +| WS events | `src/types/protocol.ts` → `project-fact-suggestions`, `project-fact-suggestion-removed`, `project-fact-added/updated/deleted`, `project-facts-synced` (~159–179) | +| UI: inbox cards | `public/js/iclaw.js` → `fact-suggestions-card`, `fact-suggestion-row[data-suggestion-id]`, `removeFactSuggestionRow`, `existingFactSuggestionIds`, `cancelFactSuggestionRowExpiry` | +| UI: active-item panel | the project facts list (search `project-fact` in `public/js/iclaw.js` and the projects/settings view) | + +**Rule of thumb:** for every `fact` symbol there should be a parallel `skill` +symbol. Keep names consistent (`projectSkills`, `projectSkillSuggestions`, +`scheduleProjectSkillReview`, `project-skill-suggestions`, etc.). + +--- + +## 2. What to take from Hermes (the "best of") + +Local clone: `~/.hermes/hermes-agent`. Key references: + +1. **Background-review fork with a restricted toolset.** + `agent/background_review.py` → `spawn_background_review` forks a daemon that + replays the conversation snapshot in a copy of the agent whose tools are + **whitelisted to memory + skill writes only** (everything else denied at + runtime). Main conversation + prompt cache are untouched. + - **Take:** the *pattern* of a separate, least-privilege reviewer that only + emits memory/skill writes. In iClaw this maps to `runThrowawayTurn` + (`projectMemory.ts`) — a throwaway gateway session — except the reviewer + output is parsed into **inbox suggestions**, never written live. + +2. **Be ACTIVE but CLASS-LEVEL (anti-sprawl).** + `_SKILL_REVIEW_PROMPT` in `background_review.py`: "most sessions produce at + least one skill update… Target shape: CLASS-LEVEL skills, each with a rich + `SKILL.md` and a `references/` directory… Not a long flat list of narrow + one-session-one-skill entries. Patch the existing skill if it covers the + learning." + - **Take:** the reviewer must prefer **patching an existing skill** over + creating a new one, and skills should be broad/reusable, not one-offs. + This is what keeps the library from rotting. + +3. **`SKILL.md` format (agentskills.io standard).** + Example: `~/.hermes/hermes-agent/skills/yuanbao/SKILL.md`. Frontmatter: + ```yaml + --- + name: + description: "" + version: 1.0.0 + metadata: + iclaw: + tags: [..] + related_skills: [..] + --- + # + ## <procedure sections...> + ``` + - **Take:** store skills in exactly this format. `description` is the summary + shown in the UI and injected into the prompt index. Body is the procedure. + +4. **Nudge intervals (optional, phase 2).** + `agent/agent_init.py` → `_skill_nudge_interval = 10`. A lighter in-band + reminder every N turns, complementary to background review. + - **Take later:** start with background review only; add nudges if needed. + +5. **Skill manager tool.** `tools/skill_manager_tool.py` → `_create_skill(name, + content, category)`. We don't need the agent to write skills live (the + reviewer does), but we DO need a **read** tool (`get_skill`) so the acting + agent can load a full skill body on demand. See §7. + +What we deliberately **diverge** from Hermes on: +- Hermes skills are **global + category-foldered**; ours are **per-project** + (keyed by `project_id`), matching iClaw facts. (Add an optional `global` scope + at acceptance time — see §5.) +- Hermes writes skills **live**; we route through an **inbox** (human gate). + +--- + +## 3. Core design decisions (locked) + +1. **Inbox-gated.** Distilled skills are `project_skill_suggestions` rows, never + active until accepted. This is the safety spine — do not skip it. +2. **Per-project scope**, with an optional `global` scope chosen at acceptance. +3. **`SKILL.md` markdown** is the storage format for both active skills and + suggestions (the suggestion already holds the full proposed body). +4. **Two suggestion kinds:** `new` (a new skill) and `patch` (an update to an + existing skill `target_skill_id`, carrying the full proposed new body; show a + diff in the UI). +5. **Provenance + trust.** Every suggestion records `source_chat_id` and an + `untrusted` flag (true when the source turn ingested untrusted content — for + now set heuristically: secure-mode turns, or turns that fetched web/email/ + Telegram content). Untrusted suggestions are **always** inbox-gated and + visually flagged. (Auto-accept of trusted suggestions is a future toggle — + see §14; default = everything is gated.) +6. **Summaries-only in the prompt.** Inject `name: description` lines for active + skills (token-bounded, like facts). Full body via `get_skill` tool on demand. +7. **Cost-aware trigger.** Do NOT run the reviewer every turn. Trigger on: + session end / chat idle, OR every N (config, default 8) *substantive* turns + (turns that used tools), whichever comes first. (Facts extraction runs per + turn because it's cheap and append-only; skill review is heavier, so throttle.) + +--- + +## 4. Data model + +Add to `src/db/database.ts` (mirror the facts tables; the schema is applied with +`CREATE TABLE IF NOT EXISTS`, so just append — no migration framework needed, +same as facts): + +```sql +-- Active, accepted project skills (procedural memory). Stored as SKILL.md. +CREATE TABLE IF NOT EXISTS project_skills ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + project_id INTEGER REFERENCES projects(id) ON DELETE CASCADE, -- NULL = global skill + name TEXT NOT NULL, -- kebab-case, unique within scope + description TEXT NOT NULL, -- one-line summary (shown to user + prompt index) + body TEXT NOT NULL, -- full SKILL.md (frontmatter + procedure) + tags TEXT, -- JSON array of strings (optional) + source_chat_id INTEGER REFERENCES chats(id) ON DELETE SET NULL, + usage_count INTEGER NOT NULL DEFAULT 0, + version INTEGER NOT NULL DEFAULT 1, + created_at TEXT NOT NULL DEFAULT (datetime('now')), + updated_at TEXT NOT NULL DEFAULT (datetime('now')) +); +CREATE INDEX IF NOT EXISTS idx_skills_project ON project_skills(project_id, id); +-- Uniqueness within a scope (project_id may be NULL for global). SQLite treats +-- NULLs as distinct, which is fine — enforce global uniqueness in the store layer. +CREATE UNIQUE INDEX IF NOT EXISTS idx_skills_scope_name + ON project_skills(project_id, name); + +-- Inbox: proposed skills awaiting user acceptance. +CREATE TABLE IF NOT EXISTS project_skill_suggestions ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + project_id INTEGER NOT NULL REFERENCES projects(id) ON DELETE CASCADE, + chat_id INTEGER NOT NULL REFERENCES chats(id) ON DELETE CASCADE, + kind TEXT NOT NULL DEFAULT 'new', -- 'new' | 'patch' + target_skill_id INTEGER REFERENCES project_skills(id) ON DELETE CASCADE, -- for 'patch' + name TEXT NOT NULL, + description TEXT NOT NULL, + body TEXT NOT NULL, -- full proposed SKILL.md + tags TEXT, -- JSON array (optional) + untrusted INTEGER NOT NULL DEFAULT 0, -- 1 if source turn ingested untrusted content + assistant_message_id INTEGER REFERENCES messages(id) ON DELETE SET NULL, + created_at TEXT NOT NULL DEFAULT (datetime('now')) +); +CREATE INDEX IF NOT EXISTS idx_skill_suggestions_chat ON project_skill_suggestions(chat_id, id); +CREATE INDEX IF NOT EXISTS idx_skill_suggestions_project ON project_skill_suggestions(project_id, id); +``` + +--- + +## 5. Store layer (`src/services/store.ts`) + +Add two objects next to `projectFacts` / `projectFactSuggestions`. Mirror their +method shapes exactly. + +```ts +export const projectSkills = { + listByProject(projectId: number): ProjectSkill[] // active skills for a project (+ global merged in by caller if desired) + listGlobal(): ProjectSkill[] + get(id: number): ProjectSkill | undefined + getByName(projectId: number | null, name: string): ProjectSkill | undefined + // index = name + description only (for prompt injection / panel list) + listIndex(projectId: number): { id: number; name: string; description: string }[] + create(opts: { projectId: number | null; name; description; body; tags?; sourceChatId?: number|null }): ProjectSkill + update(id: number, patch: { description?; body?; tags?; name? }): void // bumps version + updated_at + incrementUsage(id: number): void + remove(id: number): void +}; + +export const projectSkillSuggestions = { + listByChat(chatId: number): ProjectSkillSuggestion[] + listByProject(projectId: number): ProjectSkillSuggestion[] + get(id: number): ProjectSkillSuggestion | undefined + insert(opts: { projectId; chatId; kind:'new'|'patch'; targetSkillId?: number|null; name; description; body; tags?; untrusted?: boolean; assistantMessageId: number|null }): ProjectSkillSuggestion + remove(id: number): void +}; +``` + +Notes: +- `update` increments `version` and sets `updated_at`; on accepting a `patch`, + call `projectSkills.update(target, {...})`. +- Enforce scope+name uniqueness in `create` (look up `getByName` first; if it + exists, treat as an update rather than insert — the route layer decides). +- `tags` round-trips as `JSON.stringify` / `JSON.parse`. + +--- + +## 6. Types (`src/types/index.ts`) + +```ts +export interface ProjectSkill { + id: number; + project_id: number | null; // null = global + name: string; + description: string; + body: string; + tags: string | null; // JSON + source_chat_id: number | null; + usage_count: number; + version: number; + created_at: string; + updated_at: string; +} + +export interface ProjectSkillSuggestion { + id: number; + project_id: number; + chat_id: number; + kind: 'new' | 'patch'; + target_skill_id: number | null; + name: string; + description: string; + body: string; + tags: string | null; + untrusted: number; // 0 | 1 + assistant_message_id: number | null; + created_at: string; +} +``` + +--- + +## 7. Skill exposure to the agent (read path) + +Active skills must reach the agent in two layers, matching how facts do it: + +### 7a. Summary index in the system prompt +- **Work/Secure (iclaw-runtime):** the host builds the project block in + `chatRunner.ts buildWorkSystemPrompt`. Add a **"Skills available"** section: + one `- <name>: <description>` line per active skill (project + global), + token-bounded the same way facts are (see `buildGatewayUserMessage`'s budget + logic). Tell the model: *"To use a skill, call `get_skill` with its name to + load the full procedure."* + - The runtime's `packages/iclaw-runtime/src/agent/loop.ts buildSystemPrompt` + already appends the host-supplied `systemPrompt`. No runtime change needed + for the index itself — it rides in via `buildWorkSystemPrompt`. +- **Execute (gateway):** add the same index block in `buildGatewayUserMessage` + (`projectMemory.ts`) alongside the facts block. + +### 7b. `get_skill` tool (load full body on demand) +- **Work/Secure runtime:** add a `get_skill` tool to + `packages/iclaw-runtime/src/agent/tools.ts` (`TOOL_DEFINITIONS` + a handler in + `executeTool`). Problem: the runtime is a separate process and does not have + the iClaw DB. Two options — pick (A) for the MVP: + - **(A) Inject bodies at session create.** When the host creates the work + session (`createWorkSession` in `workRuntime.ts` / `runWorkModeTurn` in + `chatRunner.ts`), pass the active skills (name + body) for the chat's + project as a new `skills` field; the runtime stores them on the session and + `get_skill(name)` reads from that in-memory map. Re-inject when skills + change (the session is already recreated on folder/mode changes — see + `workSessions` signature logic; add skills to that signature so accepting a + skill recreates the session with the new set). Simple, no cross-process DB. + - (B) Add an HTTP endpoint on the host that the runtime calls to fetch a skill + body by `(projectId, name)`. More moving parts; defer. +- **Execute (gateway):** the gateway agent already has tools; expose `get_skill` + via the same mechanism facts use, or simply inline the full bodies of the + top-N most relevant skills if token budget allows. MVP: index-only + instruct + the agent that bodies can be requested; wire the actual fetch in phase 2 if + the gateway can't call back. (Coordinate with how Execute mode tools are + registered — see `openclawWs`.) + +> MVP shortcut acceptable: if `get_skill` plumbing is heavy, start by injecting +> the **full bodies** of up to ~3 small active skills (most-recently-used) and +> just the index for the rest. Make the body-vs-index cutoff a constant. + +On a skill being used (agent calls `get_skill`), bump `usage_count` (host side). + +--- + +## 8. The distillation pipeline (`src/services/projectSkills.ts` — NEW) + +Model it on `projectMemory.ts`. New module, parallel functions. + +### 8a. Trigger +- Add `scheduleProjectSkillReview(opts)` and call it from the **same place** + facts are scheduled (`chatRunner.ts:676` area) **but throttled**: + - Maintain a per-chat counter of substantive (tool-using) turns. + - Fire the reviewer when counter ≥ `SKILL_REVIEW_TURN_INTERVAL` (default 8) OR + on session end (hook where a Work/Secure/Execute session is torn down). + - Also expose a manual "review now" affordance later (phase 2). +- Guard like facts: only when `chat.shares_to_project`, project exists, etc. + +### 8b. Reviewer call (least-privilege, throwaway) +- Reuse `runThrowawayTurn` (gateway 'main' agent) OR a direct OpenRouter call + (see `loadOpenRouterConfig`) — pick whichever the surrounding mode already + uses; gateway throwaway is the established pattern. +- Feed it: the recent turn(s) transcript (user + assistant + which tools ran), + PLUS the **existing skill index** (names + descriptions) for the project so it + can decide patch-vs-new and avoid duplicates. +- The reviewer must output **structured** results so we can build suggestions. + Use a strict, parseable format (JSON preferred; fall back to a line protocol + if the model is unreliable). Target schema: + ```json + { + "skills": [ + { + "action": "new" | "patch", + "target": "<existing-skill-name, only for patch>", + "name": "<kebab-case>", + "description": "<one line>", + "tags": ["..."], + "body": "<full SKILL.md markdown>" + } + ] + } + ``` + If nothing qualifies, the model returns `{"skills": []}` (or literal `NONE`). + +### 8c. Reviewer prompt (adapt Hermes `_SKILL_REVIEW_PROMPT`) +Encode these rules (paraphrase from `~/.hermes/hermes-agent/agent/background_review.py`): +- Prefer **patching** an existing skill over creating a near-duplicate. +- Skills should be **class-level / reusable**, not one-shot task logs. +- A skill captures a *procedure/convention/workflow* discovered this session + that will help next time (commands that worked, gotchas, project conventions, + tool quirks) — NOT transient task state, NOT secrets/paths/credentials (same + exclusions as the facts prompt; reuse that exclusion list). +- Output strictly the JSON above. Same language as the technical content. + +### 8d. Build suggestions +- Parse the JSON. For each entry: + - Dedup against existing **active skills** and **pending suggestions** for the + project (reuse `isDuplicateFact`-style normalization on `name`+`description`). + - For `patch`: resolve `target` name → `target_skill_id`; if not found, + downgrade to `new`. + - Determine `untrusted` (see §10). + - Insert into `project_skill_suggestions`. +- Broadcast a WS event (`project-skill-suggestions`) like + `project-fact-suggestions`, so the chat UI shows inbox cards immediately. + +### 8e. Compaction (phase 2) +Facts compact when the table grows (`compactProjectFacts`). Skills mostly +self-limit via class-level discipline + user pruning, so defer compaction. + +--- + +## 9. REST routes + +### Inbox (per chat) — mirror `src/routes/chats.ts` fact-suggestion routes +- `GET /chats/:id/skill-suggestions` → list for chat. +- `POST /chats/:id/skill-suggestions/:sid/accept` + - body: `{ scope?: 'project' | 'global', body?, description?, name?, tags? }` + (the optional fields let the UI submit user **edits** before accepting). + - if `kind === 'patch'`: `projectSkills.update(target_skill_id, {...})`. + - if `kind === 'new'`: `projectSkills.create({...scope...})`. + - remove the suggestion; broadcast `project-skill-suggestion-removed` + + `project-skill-added`/`project-skill-updated`. + - If accepting changes the active skill set for any open Work/Secure chat in + that project, invalidate those sessions so the new skill is injected (tie + into the `workSessions` signature — see §7b option A). +- `POST /chats/:id/skill-suggestions/:sid/reject` → remove + broadcast removed. + +### Active skills (per project) — mirror `src/routes/projects.ts` facts CRUD +- `GET /projects/:id/skills` → list active (project + global) with index info. +- `GET /projects/:id/skills/:skillId` → full body (for the view/edit modal). +- `PATCH /projects/:id/skills/:skillId` → edit description/body/tags/name. +- `POST /projects/:id/skills/:skillId/delete`. + +--- + +## 10. Security (do not cut corners here) + +This feature is a **prompt-injection amplifier** if done naively: untrusted +content (a web page, an email, a Telegram message) could contain text that the +reviewer distills into a *standing instruction*. The inbox is the primary +defense; reinforce it: + +1. **Everything is inbox-gated by default.** No code path writes an active skill + without an explicit `accept` call from the user. +2. **`untrusted` provenance flag.** Set `untrusted = 1` when the source turn + ingested external content. MVP heuristic: `true` if the turn ran in **Secure + mode** with network enabled, OR used a fetch/`run_command curl`/email/Telegram + tool. Surface this on the inbox card ("learned from untrusted content — + review carefully"). +3. **The reviewer is least-privilege by construction.** It only emits suggestion + text; it has no tools and cannot act. (This is the iClaw analogue of Hermes's + whitelisted reviewer.) +4. **Never distill secrets.** Reuse the facts exclusion list (no tokens, keys, + `.env`, paths, hostnames). Additionally, strip/refuse any skill body that + contains secret-looking patterns (reuse `findBlockedPattern` from + `packages/iclaw-runtime/src/agent/security.ts` conceptually, or a host-side + equivalent). +5. **No auto-accept in MVP.** The trust-level auto-accept (phase 2, §14) must + only ever apply to **trusted** (non-ingested) sessions, never `untrusted=1`. + +Cross-reference: this connects to the broader threat-model discussion (lethal +trifecta, host-mediated creds, egress allowlist). A planted skill that says +"always send X to Y" is only dangerous if the acting agent also has a send +capability — keep capability scoping per task regardless of this feature. + +--- + +## 11. WS protocol (`src/types/protocol.ts`) + +Add events mirroring the fact events: +```ts +| { type: 'project-skill-suggestions'; chatId: number; projectId: number; + projectName: string; + suggestions: { id: number; kind: 'new'|'patch'; name: string; + description: string; untrusted: boolean; targetSkillId: number|null }[] } +| { type: 'project-skill-suggestion-removed'; chatId: number; suggestionId: number } +| { type: 'project-skill-added'; projectId: number; skill: ProjectSkill } +| { type: 'project-skill-updated'; projectId: number; skill: ProjectSkill } +| { type: 'project-skill-deleted'; projectId: number; skillId: number } +``` +(Card list carries only summary fields; the full body is fetched via REST when +the user expands/edits — keeps WS frames small.) + +--- + +## 12. UI (`public/js/iclaw.js` + views) + +### 12a. Inbox cards (in the chat) — extend the fact-suggestion card +Mirror `fact-suggestions-card` / `fact-suggestion-row`. Differences vs facts: +- Each row shows **name + description** (the summary) and a **kind badge** + (`New skill` / `Updates "<target>"`). +- An **untrusted** badge when `untrusted` (amber, "from untrusted content"). +- A **Preview/Edit** affordance: clicking fetches the full body + (`GET /chats/:id/skill-suggestions` returns bodies, or fetch per-suggestion) + and shows it in an expandable area / modal with an editable textarea. +- Accept button → `POST .../accept` with any edits + a scope toggle + (Project / Global). Reject button → `.../reject`. +- Reuse `existingSuggestionIds` / expiry helpers (`cancelFactSuggestionRowExpiry` + analogue) so the inbox self-cleans. + +### 12b. Project Skills panel — mirror the project facts list +- A "Skills" section on the project view (next to Facts): list each active skill + as **name — description**, with view (open body modal), edit, delete. +- This list IS the "short summary per skill" deliverable. +- Live-update via the new `project-skill-*` WS events. + +### 12c. Composer indicator (optional, phase 2) +A small "N skills" chip in the composer for Work/Execute, like the work-folders +count, so the user knows skills are active for this project. + +--- + +## 13. Wiring checklist (file-by-file) + +1. `src/db/database.ts` — add the two tables + indexes (§4). +2. `src/types/index.ts` — add `ProjectSkill`, `ProjectSkillSuggestion` (§6). +3. `src/services/store.ts` — add `projectSkills`, `projectSkillSuggestions` (§5). +4. `src/services/projectSkills.ts` (NEW) — reviewer pipeline (§8). +5. `src/services/chatRunner.ts` — call `scheduleProjectSkillReview` (throttled), + add the skills index to `buildWorkSystemPrompt`, and (option A §7b) pass + active skills into `createWorkSession` + include them in the `workSessions` + recreation signature. +6. `src/services/workRuntime.ts` — add `skills` to `CreateSessionOptions` and + send it. +7. `packages/iclaw-runtime/src/index.ts` + `sessions.ts` — accept `skills` on + session create, store on the session. +8. `packages/iclaw-runtime/src/agent/tools.ts` + `loop.ts` — add `get_skill` + tool reading the session's skill map; bump usage via an event/log the host + can observe (or skip usage tracking in MVP). +9. `src/services/projectMemory.ts` — add skills index to `buildGatewayUserMessage` + (Execute path). +10. `src/routes/chats.ts` — skill-suggestion inbox routes (§9). +11. `src/routes/projects.ts` — active-skill CRUD routes (§9). +12. `src/types/protocol.ts` — WS events (§11). +13. `public/js/iclaw.js` (+ relevant `views/*.ejs`, `public/css/style.css`) — + inbox cards + project Skills panel (§12). + +--- + +## 14. Phasing + +**MVP (ship first):** +- Tables, store, types. +- Reviewer pipeline (throttled, JSON output, dedup, patch-vs-new). +- Inbox suggestions + WS + chat cards (accept/edit/reject, scope, untrusted badge). +- Project Skills panel (list/view/edit/delete) — the summary view. +- Prompt **index** injection (Work/Secure via `buildWorkSystemPrompt`, Execute + via `buildGatewayUserMessage`). For bodies, use the "inline top-N small skills" + shortcut (§7b) — defer real `get_skill`. + +**Phase 2:** +- `get_skill` tool (runtime in-memory map, option A) + usage tracking. +- Trust-level auto-accept of **trusted** suggestions (config per project; + untrusted always gated). +- Nudge intervals (Hermes-style in-band reminder). +- Skill compaction / merge when a project accumulates many. +- Composer "N skills" chip. + +--- + +## 15. Acceptance criteria (definition of done for MVP) + +- [ ] After ~8 tool-using turns in a project chat (or on session end), a skill + suggestion can appear in the chat as an inbox card with name + description. +- [ ] No skill ever becomes active without the user clicking Accept. +- [ ] Accepting a `new` suggestion creates a `project_skills` row (SKILL.md + body); accepting a `patch` updates the target and bumps `version`. +- [ ] The user can edit the body before accepting. +- [ ] Suggestions from untrusted sessions show the untrusted badge. +- [ ] The project Skills panel lists each active skill as name — description, + with view/edit/delete, live-updating over WS. +- [ ] Active skills' `name: description` index appears in the Work/Secure and + Execute system prompts, token-bounded. +- [ ] Rejecting / deleting works and broadcasts the removal. +- [ ] Type-checks clean: `npm run typecheck` (host) and runtime `tsc --noEmit` + (ignore the pre-existing `agent-to-agent` / `db-v2.test` errors). + +--- + +## 16. Open decisions (resolve with product owner if blocked) + +1. **Reviewer model:** gateway 'main' agent (consistent with facts) vs a cheap + OpenRouter model (`SUMMARY_MODEL` style). Default: reuse the facts pattern + (gateway throwaway) for MVP. +2. **Trigger cadence:** every-N-turns vs session-end vs both. Default: both, + N=8. +3. **Global scope at acceptance:** include the Project/Global toggle in MVP, or + project-only first? Default: include the toggle (cheap, and useful). +4. **`get_skill` vs inline bodies:** MVP uses inline-top-N; confirm that's + acceptable or prioritize the tool. + +--- + +## 17. Don't break + +- Facts must keep working untouched — skills are additive and parallel. +- The Work/Secure session recreation logic (`workSessions` signature in + `chatRunner.ts`) already recreates on folder/mode change; if you add skills to + the injected session state, **add skills to that signature** so accepting a + skill takes effect without a manual restart (this is the same class of bug we + already fixed for folder-access and mode changes — see git history + `fix(work): recreate session on Work<->Secure mode switch`). +- Keep the reviewer **off the hot path** (fire-and-forget, like + `scheduleProjectFactExtraction`); never block the user's turn on it. diff --git a/package-lock.json b/package-lock.json index 6a72132..9b17974 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,13 +1,16 @@ { "name": "@iclawapp/iclaw", - "version": "0.1.5", + "version": "0.3.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "@iclawapp/iclaw", - "version": "0.1.5", + "version": "0.3.0", "license": "MIT", + "workspaces": [ + "packages/*" + ], "dependencies": { "@noble/hashes": "^2.2.0", "@serenity-kit/opaque": "^1.1.0", @@ -573,6 +576,10 @@ "node": ">=18" } }, + "node_modules/@iclaw/runtime": { + "resolved": "packages/iclaw-runtime", + "link": true + }, "node_modules/@jridgewell/resolve-uri": { "version": "3.1.2", "resolved": "https://registry.npmjs.org/@jridgewell/resolve-uri/-/resolve-uri-3.1.2.tgz", @@ -1299,6 +1306,182 @@ "url": "https://opencollective.com/vitest" } }, + "node_modules/@vscode/ripgrep": { + "version": "1.18.0", + "resolved": "https://registry.npmjs.org/@vscode/ripgrep/-/ripgrep-1.18.0.tgz", + "integrity": "sha512-ns5lWe44tSfbTMbVUsyB+I1819PVSw4AdpgK0RNkzfWfwy6+3IUNSxwSrfTno1/oWaS/hERNz+XLWVyga2aJBQ==", + "license": "MIT", + "optionalDependencies": { + "@vscode/ripgrep-darwin-arm64": "1.18.0", + "@vscode/ripgrep-darwin-x64": "1.18.0", + "@vscode/ripgrep-linux-arm": "1.18.0", + "@vscode/ripgrep-linux-arm64": "1.18.0", + "@vscode/ripgrep-linux-ia32": "1.18.0", + "@vscode/ripgrep-linux-ppc64": "1.18.0", + "@vscode/ripgrep-linux-riscv64": "1.18.0", + "@vscode/ripgrep-linux-s390x": "1.18.0", + "@vscode/ripgrep-linux-x64": "1.18.0", + "@vscode/ripgrep-win32-arm64": "1.18.0", + "@vscode/ripgrep-win32-ia32": "1.18.0", + "@vscode/ripgrep-win32-x64": "1.18.0" + } + }, + "node_modules/@vscode/ripgrep-darwin-arm64": { + "version": "1.18.0", + "resolved": "https://registry.npmjs.org/@vscode/ripgrep-darwin-arm64/-/ripgrep-darwin-arm64-1.18.0.tgz", + "integrity": "sha512-r3ktHSvbFycQNF6sl7sNDPocpsI7J+mEzh1IaZFkY0spm3k2Z9t8hPAeOK7+p0l6p6/swkQC14XWX01low+94Q==", + "cpu": [ + "arm64" + ], + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ] + }, + "node_modules/@vscode/ripgrep-darwin-x64": { + "version": "1.18.0", + "resolved": "https://registry.npmjs.org/@vscode/ripgrep-darwin-x64/-/ripgrep-darwin-x64-1.18.0.tgz", + "integrity": "sha512-25b4gWbL138dGuQU244ebCKKc0q05ULBMoFSz9oAEUHNeqK/lOJViDS7DRvbDazzAzSEdan391Znks/R5mkaTQ==", + "cpu": [ + "x64" + ], + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ] + }, + "node_modules/@vscode/ripgrep-linux-arm": { + "version": "1.18.0", + "resolved": "https://registry.npmjs.org/@vscode/ripgrep-linux-arm/-/ripgrep-linux-arm-1.18.0.tgz", + "integrity": "sha512-GDAvufNDHu8zqLEmXstalQF0Wh6wQvdsBi/Vg3Yi3CK4a8XoFXqqXVEHEZ9xQz3t0NfoSEc9JbvK9DDS6FxyxQ==", + "cpu": [ + "arm" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@vscode/ripgrep-linux-arm64": { + "version": "1.18.0", + "resolved": "https://registry.npmjs.org/@vscode/ripgrep-linux-arm64/-/ripgrep-linux-arm64-1.18.0.tgz", + "integrity": "sha512-lQ/5zTG++U0E3IhVgS4EPTTn/U4okncaRMM5GOFfOYZywS4nuD31GhkHbNYlDk5CuDC68+hYJ0/eQeyCKJDA+g==", + "cpu": [ + "arm64" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@vscode/ripgrep-linux-ia32": { + "version": "1.18.0", + "resolved": "https://registry.npmjs.org/@vscode/ripgrep-linux-ia32/-/ripgrep-linux-ia32-1.18.0.tgz", + "integrity": "sha512-YWLkSUtFd4Jh5EepIhA9RJSfv3uMAVMo+2rBIGHPBnvgLrZciIs2cDKei1/p6Wc/aCzUoHyMAg2R6tw4ZCBKGg==", + "cpu": [ + "ia32" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@vscode/ripgrep-linux-ppc64": { + "version": "1.18.0", + "resolved": "https://registry.npmjs.org/@vscode/ripgrep-linux-ppc64/-/ripgrep-linux-ppc64-1.18.0.tgz", + "integrity": "sha512-quXVY8fwQ8O/lvU1yrSqSl3jlUzysRSb+AfUfCL/tRtphxsKlFvPAejryZ6vg4Bgvn8XL74xb4qMCDmWgYrT5w==", + "cpu": [ + "ppc64" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@vscode/ripgrep-linux-riscv64": { + "version": "1.18.0", + "resolved": "https://registry.npmjs.org/@vscode/ripgrep-linux-riscv64/-/ripgrep-linux-riscv64-1.18.0.tgz", + "integrity": "sha512-f5kBQBrWfQt8Q7OhSORuNDei5dkYagBj3y4jImSUXGMy8B/Ke7SltSRcUtjPv166FAFfHCAmWuZp3+cWnX2/Vw==", + "cpu": [ + "riscv64" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@vscode/ripgrep-linux-s390x": { + "version": "1.18.0", + "resolved": "https://registry.npmjs.org/@vscode/ripgrep-linux-s390x/-/ripgrep-linux-s390x-1.18.0.tgz", + "integrity": "sha512-rTOcJFGGcl2c07RUOWUo4U1ndnemKhY6A9hnMB18uk7jSgJc0d/QLBGWMWpumdtoJtpizn/wIv5mXIisJukusQ==", + "cpu": [ + "s390x" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@vscode/ripgrep-linux-x64": { + "version": "1.18.0", + "resolved": "https://registry.npmjs.org/@vscode/ripgrep-linux-x64/-/ripgrep-linux-x64-1.18.0.tgz", + "integrity": "sha512-mQ3bVrUpnD2vs7QT0vX90Lt0cnUq467uFtEktIdsJJmW296RoSULRGqWgzG1AKxyBpNDD6l4ZO4qKf6SgyC23Q==", + "cpu": [ + "x64" + ], + "license": "MIT", + "optional": true, + "os": [ + "linux" + ] + }, + "node_modules/@vscode/ripgrep-win32-arm64": { + "version": "1.18.0", + "resolved": "https://registry.npmjs.org/@vscode/ripgrep-win32-arm64/-/ripgrep-win32-arm64-1.18.0.tgz", + "integrity": "sha512-vfTIjq1OHnzUjxZcHVQAMbnggp8dpGf+0QKFOZHwWPqFwXxQC8eCWM+5NUdoJ6yrElCeMzoUTXoK/LdZaniB+Q==", + "cpu": [ + "arm64" + ], + "license": "MIT", + "optional": true, + "os": [ + "win32" + ] + }, + "node_modules/@vscode/ripgrep-win32-ia32": { + "version": "1.18.0", + "resolved": "https://registry.npmjs.org/@vscode/ripgrep-win32-ia32/-/ripgrep-win32-ia32-1.18.0.tgz", + "integrity": "sha512-//rfAE+BOw5AC2EMmepmiE36jUuevtQYNQqqlw1s3m9FlRxjxEut97RkRPHAu9BG4mSojatZx+kXZXNdyI9caQ==", + "cpu": [ + "ia32" + ], + "license": "MIT", + "optional": true, + "os": [ + "win32" + ] + }, + "node_modules/@vscode/ripgrep-win32-x64": { + "version": "1.18.0", + "resolved": "https://registry.npmjs.org/@vscode/ripgrep-win32-x64/-/ripgrep-win32-x64-1.18.0.tgz", + "integrity": "sha512-KNPvtElldqILHdnAetujPaowkNbpqJy3ssIGGN6F6Kve9Qi+nNLI2DN01O83JjCEVQbCzl8Ov3QZ9Eov3BR8Dg==", + "cpu": [ + "x64" + ], + "license": "MIT", + "optional": true, + "os": [ + "win32" + ] + }, "node_modules/accepts": { "version": "1.3.8", "resolved": "https://registry.npmjs.org/accepts/-/accepts-1.3.8.tgz", @@ -2847,6 +3030,24 @@ "wrappy": "1" } }, + "node_modules/openai": { + "version": "6.41.0", + "resolved": "https://registry.npmjs.org/openai/-/openai-6.41.0.tgz", + "integrity": "sha512-IGWPopZq6Rjoynjfb3NSLf/z2MTw7UiOsm9TAjPGAjUESH7Uq41Trg4QWehBEn58p74i+m7uoRPV2vXcpPXhyA==", + "license": "Apache-2.0", + "peerDependencies": { + "ws": "^8.18.0", + "zod": "^3.25 || ^4.0" + }, + "peerDependenciesMeta": { + "ws": { + "optional": true + }, + "zod": { + "optional": true + } + } + }, "node_modules/parseurl": { "version": "1.3.3", "resolved": "https://registry.npmjs.org/parseurl/-/parseurl-1.3.3.tgz", @@ -3828,6 +4029,23 @@ "optional": true } } + }, + "packages/iclaw-runtime": { + "name": "@iclaw/runtime", + "version": "0.1.0", + "license": "MIT", + "dependencies": { + "@vscode/ripgrep": "^1.18.0", + "openai": "^6.41.0" + }, + "devDependencies": { + "@types/node": "^22.10.0", + "tsx": "^4.19.0", + "typescript": "^5.7.0" + }, + "engines": { + "node": ">=20" + } } } } diff --git a/package.json b/package.json index 6fed699..e26bfa6 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@iclawapp/iclaw", - "version": "0.2.1", + "version": "0.3.0", "description": "Local web UI for OpenClaw Gateway", "license": "MIT", "homepage": "https://github.com/iClawApp/iClaw", @@ -14,6 +14,9 @@ "engines": { "node": ">=20" }, + "workspaces": [ + "packages/*" + ], "bin": { "iclaw": "bin/iclaw.js" }, @@ -29,6 +32,7 @@ "prebuild": "node scripts/copy-opaque-vendor.mjs", "build": "tsc", "start": "node dist/index.js", + "build:secure-image": "bash packages/iclaw-runtime/container/build-secure.sh", "typecheck": "tsc --noEmit", "pretest": "node scripts/copy-opaque-vendor.mjs", "test": "vitest run", diff --git a/packages/iclaw-runtime/container/.dockerignore b/packages/iclaw-runtime/container/.dockerignore new file mode 100644 index 0000000..598ce19 --- /dev/null +++ b/packages/iclaw-runtime/container/.dockerignore @@ -0,0 +1,2 @@ +agent-runner/node_modules +agent-runner/dist diff --git a/packages/iclaw-runtime/container/build-secure.sh b/packages/iclaw-runtime/container/build-secure.sh new file mode 100755 index 0000000..2d4b049 --- /dev/null +++ b/packages/iclaw-runtime/container/build-secure.sh @@ -0,0 +1,34 @@ +#!/usr/bin/env bash +# Build the iClaw Secure-Mode sandbox image (Chromium + agent-browser). +# Usage: ./build-secure.sh [--cjk] +set -euo pipefail + +cd "$(dirname "$0")" + +# macOS: iClaw's engine is Colima, and the runtime looks for this image inside +# its own Colima VM. Build into that same VM (start it if needed, route the build +# to its context) so Secure Mode actually finds the image — not Docker Desktop. +if [[ "$(uname -s)" == "Darwin" ]]; then + # Find colima/docker whether installed via Homebrew or downloaded by iClaw into + # ~/.iclaw/engine (no-brew installs), plus Homebrew's bin dirs. + export PATH="${ICLAW_ENGINE_DIR:-$HOME/.iclaw/engine}/bin:/opt/homebrew/bin:/usr/local/bin:$PATH" + if ! command -v colima >/dev/null 2>&1; then + echo "ERROR: colima not found. Install it first (iClaw installs it automatically; or: brew install colima docker)." >&2 + exit 1 + fi + colima start iclaw + export DOCKER_CONTEXT="colima-iclaw" +fi + +IMAGE_TAG="${ICLAW_SECURE_IMAGE:-iclaw-secure:latest}" +CJK="false" +[[ "${1:-}" == "--cjk" ]] && CJK="true" + +echo "Building ${IMAGE_TAG} (CJK fonts: ${CJK})..." +DOCKER_BUILDKIT=1 docker build \ + -f secure-sandbox.Dockerfile \ + --build-arg INSTALL_CJK_FONTS="${CJK}" \ + -t "${IMAGE_TAG}" \ + . + +echo "Done. Secure Mode will use ${IMAGE_TAG}." diff --git a/packages/iclaw-runtime/container/secure-sandbox.Dockerfile b/packages/iclaw-runtime/container/secure-sandbox.Dockerfile new file mode 100644 index 0000000..6337c57 --- /dev/null +++ b/packages/iclaw-runtime/container/secure-sandbox.Dockerfile @@ -0,0 +1,63 @@ +# syntax=docker/dockerfile:1.7 +# iClaw sandbox image — the SINGLE image shared by Safe work and Work mode. +# +# A long-lived `sleep` container the host docker-execs into, one turn at a time +# (Safe work: src/secure-runner.ts) or bind-mounts the user's folders into for +# `run_command` (Work: src/work-container.ts → resolveWorkImage prefers this tag). +# +# Toolset is curated 80/20: the ~20% of CLIs that ~80% of agent turns reach for, +# and nothing heavy. Deliberately EXCLUDED: a browser (Chromium + agent-browser +# ~870MB — web access is via curl/wget, and our search runs a different path), +# language toolchains/compilers, and other large runtimes. The agent self-installs +# the long tail at runtime WITHOUT root into /workspace/.tools (on PATH), which +# lives in the bind-mounted, TTL-reaped workspace — installs persist across +# container restarts within a session and auto-delete when the workspace expires. +# For a heavier base (e.g. a Go/Rust toolchain), point ICLAW_WORK_IMAGE at a +# custom image instead of bloating this shared one. +# +# Built via container/build-secure.sh → tag `iclaw-secure:latest`. +# Runs as the non-root `node` user. + +FROM node:22-slim + +# Curated 80/20 CLIs. curl/wget + ca-certificates power web access and non-root +# self-installs (downloading static binaries). git, python3 and an editor cover +# the bulk of real agent work. All baked in (not apt-installed at runtime) +# because the container runs as non-root `node` and is recreated after idle reap +# — a runtime `apt install` can't work and wouldn't persist anyway. +RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \ + --mount=type=cache,target=/var/lib/apt,sharing=locked \ + apt-get update && apt-get install -y --no-install-recommends \ + ca-certificates \ + curl \ + wget \ + git \ + ripgrep \ + python3 \ + nano \ + unzip \ + zip \ + xz-utils \ + bzip2 \ + jq \ + less \ + tree \ + && rm -rf /var/lib/apt/lists/* + +# ---- Self-installed tools (no root) ---------------------------------------- +# Drop static binaries into /workspace/.tools/bin, or `npm i -g <pkg>` (prefix +# below). Inside the bind-mounted, TTL-reaped workspace, so they persist across +# container restarts within a session and are auto-deleted when the workspace +# expires (~7 days) — see createSecureWorkspace / destroySecureWorkspace. +ENV NPM_CONFIG_PREFIX=/workspace/.tools/npm +ENV PATH="/workspace/.tools/bin:/workspace/.tools/npm/bin:$PATH" + +# ---- Workspace + non-root user --------------------------------------------- +# /workspace is the host bind-mount target. Home must be writable for tool caches. +RUN mkdir -p /workspace && chown -R node:node /workspace && chmod 777 /home/node + +USER node +WORKDIR /workspace + +# Default command; the host overrides argv with `sleep 3600` at run time. +CMD ["sleep", "3600"] diff --git a/packages/iclaw-runtime/package.json b/packages/iclaw-runtime/package.json new file mode 100644 index 0000000..0d7f790 --- /dev/null +++ b/packages/iclaw-runtime/package.json @@ -0,0 +1,26 @@ +{ + "name": "@iclaw/runtime", + "version": "0.1.0", + "description": "iClaw agent runtime — agent loop on the host, Docker sandbox per turn (Work / Safe work / Incognito), OpenRouter-only.", + "license": "MIT", + "type": "module", + "main": "dist/index.js", + "scripts": { + "build": "tsc && node -e \"const fs=require('fs');fs.mkdirSync('dist/agent',{recursive:true});fs.copyFileSync('src/agent/social-fetch.mjs','dist/agent/social-fetch.mjs')\"", + "dev": "tsx src/index.ts", + "start": "node dist/index.js", + "typecheck": "tsc --noEmit" + }, + "dependencies": { + "@vscode/ripgrep": "^1.18.0", + "openai": "^6.41.0" + }, + "devDependencies": { + "@types/node": "^22.10.0", + "tsx": "^4.19.0", + "typescript": "^5.7.0" + }, + "engines": { + "node": ">=20" + } +} diff --git a/packages/iclaw-runtime/src/agent/loop.ts b/packages/iclaw-runtime/src/agent/loop.ts new file mode 100644 index 0000000..47288f2 --- /dev/null +++ b/packages/iclaw-runtime/src/agent/loop.ts @@ -0,0 +1,553 @@ +/** + * Model-agnostic agent loop. + * + * Calls any OpenRouter model that supports tool calling. + * Streams text deltas and tool events back to the caller via async generator. + */ +import OpenAI from 'openai'; + +import { TOOL_DEFINITIONS, WEB_FETCH_TOOL, WEB_SEARCH_TOOL, READ_SUMMARY_TOOL, ANALYZE_LINK_TOOL, SHOW_IMAGE_TOOL, executeTool, normalizeFetchUrl, normalizeSearchQuery, type ToolContext, type ToolName, type SavingsNote, type ImageRef } from './tools.js'; +import { SOCIAL_SEARCH_TOOL } from './social.js'; +import { dumpPrompt, newTurnId } from './prompt-dump.js'; + +export interface AgentOptions { + apiKey: string; + model: string; + allowedFolders: string[]; + /** Per-folder access levels. When omitted, all allowed folders are writable. */ + folderAccess?: { path: string; readonly: boolean }[]; + /** Shell backend for run_command (Docker sandbox). Omit to disable commands. */ + runShell?: (command: string, cwd: string) => Promise<string>; + /** + * Sandbox backend for analyze_link (yt-dlp). Runs in the session's container + * so yt-dlp never parses untrusted data on the host. Omit to drop the tool + * (no Docker → analyze_link not offered, falls back to web_fetch/web_search). + */ + linkSandbox?: (command: string) => Promise<string>; + /** + * Incognito (read-only, ephemeral): file reads are unrestricted (read + * anywhere; secrets still refused), write_file is disabled, run_command is + * sandboxed read-only, and the `web_fetch` research tool is exposed. + */ + incognito?: boolean; + systemPrompt?: string; + /** + * Image data URLs (`data:<mime>;base64,…`) for THIS turn's user message — + * files the user dropped into the chat. Sent once as vision blocks so the + * model literally sees them; NOT stored in history (one-shot, expensive). + */ + images?: string[]; + onWriteApproval?: (filePath: string, content: string) => Promise<boolean>; + /** Abort the in-flight turn (user pressed Stop). Stops the model stream and + * ends the loop cleanly between rounds. */ + signal?: AbortSignal; +} + +export type AgentEvent = + | { type: 'text'; content: string } + | { type: 'tool_start'; name: string; input: unknown } + | { type: 'tool_result'; name: string; result: string } + | { type: 'note'; note: SavingsNote } + | { type: 'image'; path: string; mime: string; fileName: string; bytes: number } + | { type: 'approval_request'; changeId: string; path: string; content: string } + | { type: 'done'; tokens?: number; cached?: number } + | { type: 'error'; message: string }; + +export type Message = OpenAI.Chat.ChatCompletionMessageParam; + +/** Max tool-call rounds per turn before we stop (env-tunable for long tasks). */ +const MAX_ROUNDS = Math.max(1, Number(process.env.ICLAW_MAX_ROUNDS) || 40); + +/** Turn a provider/SDK error into a concise, user-facing message. */ +function describeApiError(err: unknown): string { + const e = err as { + status?: number; + code?: number | string; + error?: { code?: number; metadata?: { error_type?: string } }; + message?: string; + }; + const code = e?.status ?? e?.code ?? e?.error?.code; + const isRateLimit = code === 429 || e?.error?.metadata?.error_type === 'rate_limit_exceeded'; + if (isRateLimit) { + return 'Rate limit reached on the model provider (429). Wait a few seconds and try again, ' + + 'or switch to a model with higher limits (ICLAW_MODEL).'; + } + return e?.message || String(err); +} + +// Mid-turn compaction budget: once the in-flight message array passes this many +// chars, stub out all but a few tool outputs (they're already acted on; the model +// can re-run a tool if it truly needs the data again). Lowered to 16k — recent +// work turns plateau around 15–20k, so a 32k gate almost never fired and the +// whole history was resent every round (O(n²) tokens). Like Hermes' compressor we +// keep the FIRST few tool outputs (early task context) as well as the last few. +const INTURN_COMPACT_CHARS = Number(process.env.ICLAW_INTURN_COMPACT_CHARS) || 16_000; +const INTURN_KEEP_TOOL_MSGS = Number(process.env.ICLAW_INTURN_KEEP_TOOL_MSGS) || 6; +const INTURN_KEEP_FIRST_TOOL_MSGS = Number(process.env.ICLAW_INTURN_KEEP_FIRST_TOOL_MSGS) || 2; + +/** + * Shrink old tool-result messages in-place when the turn's context grows too + * large. Keeps message structure (assistant↔tool pairing + tool_call_ids) intact + * for API validity — only replaces stale tool *content* with a short stub. No + * extra model call. Shared by the Work/Incognito loop and the Sandbox loop. + * + * Protects the first `INTURN_KEEP_FIRST_TOOL_MSGS` and last `INTURN_KEEP_TOOL_MSGS` + * tool outputs; only the middle gets stubbed (mirrors Hermes' protect_first_n / + * protect_last_n). + */ +export function shrinkOldToolOutputs(messages: Message[]): void { + let total = 0; + for (const m of messages) total += typeof m.content === 'string' ? m.content.length : 0; + if (total <= INTURN_COMPACT_CHARS) return; + + const toolIdx: number[] = []; + for (let i = 0; i < messages.length; i++) if (messages[i].role === 'tool') toolIdx.push(i); + const keepFirst = INTURN_KEEP_FIRST_TOOL_MSGS; + const keepLast = INTURN_KEEP_TOOL_MSGS; + for (let k = keepFirst; k < toolIdx.length - keepLast; k++) { + const i = toolIdx[k]; + const content = (messages[i] as { content?: unknown }).content; + if (typeof content === 'string' && content.length > 160) { + messages[i] = { + ...messages[i], + content: `[earlier tool output omitted to save context — was ${content.length.toLocaleString()} chars; re-run the tool if you need it again]`, + } as Message; + } + } +} + +/** + * Prompt caching (#2): mark the static prefix (system prompt + tool schemas) with + * an OpenRouter/Anthropic `cache_control: ephemeral` breakpoint so it isn't re- + * billed at full price on every round. The system block is the one guaranteed- + * stable prefix every round (tools ride with it), so one breakpoint there caches + * ~1.2k tokens of overhead per round. OpenRouter strips the hint for providers + * that don't support it (e.g. DeepSeek already caches automatically), so this is + * safe across models. Off only when ICLAW_PROMPT_CACHE=off. + * + * Returns a NEW array (doesn't mutate `messages`) with the system content + * converted to the parts form Anthropic needs for a cache breakpoint. + */ +export function withPromptCaching(messages: Message[]): Message[] { + if (process.env.ICLAW_PROMPT_CACHE === 'off') return messages; + const i = messages.findIndex((m) => m.role === 'system'); + if (i === -1) return messages; + const sys = messages[i]; + if (typeof sys.content !== 'string' || !sys.content) return messages; + const out = messages.slice(); + out[i] = { + role: 'system', + content: [ + { type: 'text', text: sys.content, cache_control: { type: 'ephemeral' } }, + ], + } as unknown as Message; + return out; +} + +/** + * Tool-loop guardrail (#5): a turn can waste rounds (and tokens) re-issuing the + * SAME tool call with identical args — a stuck model retrying a failed read, or + * looping. Track call signatures; once one repeats past the limit, short-circuit + * with a nudge instead of executing it again (mirrors Hermes' tool_loop_guardrails). + */ +const TOOL_REPEAT_LIMIT = Math.max(1, Number(process.env.ICLAW_TOOL_REPEAT_LIMIT) || 3); + +/** + * Collapse a tool call into a repeat-signature. Keyed on EXACT args by default, + * but web tools get a semantic key so the model can't dodge the guard by + * rewording: `web_fetch` keys on the normalized URL alone (a tweaked `focus` or + * `#anchor` is the SAME page), `web_search`/`social_search` on the normalized + * query (word order / quoting / operators don't change the results). Malformed + * args fall back to the exact-args key. + */ +function toolRepeatSignature(name: string, rawArgs: string): string { + try { + if (name === 'web_fetch') { + const a = JSON.parse(rawArgs) as { url?: unknown }; + if (a.url) return `web_fetch:${normalizeFetchUrl(String(a.url))}`; + } else if (name === 'web_search' || name === 'social_search') { + const a = JSON.parse(rawArgs) as { query?: unknown }; + if (a.query) return `${name}:${normalizeSearchQuery(String(a.query))}`; + } + } catch { + // fall through to exact-args signature + } + return `${name}:${rawArgs}`; +} + +export function makeToolGuard(): { check(name: string, rawArgs: string): string | null } { + const counts = new Map<string, number>(); + return { + check(name: string, rawArgs: string): string | null { + const sig = toolRepeatSignature(name, rawArgs); + const n = (counts.get(sig) ?? 0) + 1; + counts.set(sig, n); + if (n > TOOL_REPEAT_LIMIT) { + if (name === 'web_fetch') { + return `Guardrail: you've already fetched this URL ${n - 1} times this turn — the page is identical ` + + `no matter what 'focus' or '#anchor' you pass. Not fetching it again. Use what you already have, ` + + `fetch a DIFFERENT url, or tell the user the data isn't available there.`; + } + if (name === 'web_search' || name === 'social_search') { + return `Guardrail: you've already run this search ${n - 1} times this turn — rewording it returns the ` + + `same results. Not running it again. Try a genuinely different source or tool, or tell the user what ` + + `you couldn't find.`; + } + return `Guardrail: you've already called ${name} with these exact arguments ${n - 1} times this turn ` + + `and got the same result. Not running it again. Change the arguments, try a different tool, ` + + `or tell the user what's blocking you.`; + } + return null; + }, + }; +} + +/** Dead-round circuit-breaker (#3): this many CONSECUTIVE rounds whose tool calls + * ALL came back empty / failed / timed-out / guard-blocked means the agent is + * spinning on data it can't get (e.g. login-gated stats). We then force it to + * conclude from what it has instead of grinding to MAX_ROUNDS. Env-tunable. */ +const DEAD_ROUND_LIMIT = Math.max(1, Number(process.env.ICLAW_DEAD_ROUND_LIMIT) || 5); + +/** True for a tool result that carried no new signal — fuels DEAD_ROUND_LIMIT. + * Matches the leading text of every "nothing useful" path in tools.ts + * (fetch/search failures, timeouts, empty results, the guard nudge). */ +export function isLowValueResult(s: string): boolean { + const t = s.trimStart(); + if (t.length === 0) return true; + return ( + t.startsWith('Fetch failed') || + t.startsWith('Search failed') || + t.startsWith('Guardrail:') || + t.startsWith('No results for') || + t.startsWith('No files found') || + t.startsWith('(empty') || + t.startsWith('Only absolute http') || + t.startsWith('Security error:') || + /\btimed out after\b/.test(t) + ); +} + +const DEFAULT_SYSTEM = `Work Mode: read/search/edit/create files in the selected folders, run shell commands, and research the web (web_search then web_fetch). +Prefer edit_file over rewriting whole files. For a large file you only need the gist of, use read_summary (cheap) instead of read_file. The user approves every write in the UI — don't paste file contents or ask "is this correct?"; just act, then briefly say what you did. +Be efficient: chain shell steps with && in one call, don't repeat commands. Never go outside the allowed folders. +Keep replies short — don't echo back long file listings or file contents; summarize in a line or two.`; + +/** + * System-install policy (Section 3): the agent must not change the user's + * computer. Shared by Work and Safe prompts. run_command runs in an isolated + * sandbox, so installs there are harmless AND don't reach the host — which is + * exactly why we say it in the prompt rather than guarding run_command (a guard + * would wrongly block legitimate in-sandbox installs). + */ +export const HOST_INSTALL_POLICY = `System changes (important): your run_command runs in an isolated sandbox, so installing tools there is fine for the task but does NOT install anything on the user's actual computer. Never attempt or claim to make host system changes — installing Python, Node, Git, Docker, Homebrew or system packages on their machine, or editing their PATH. If a task genuinely needs something installed on their computer, tell them it changes their system, give the official install command or link, and ask them to run it themselves.`; + +const INCOGNITO_SYSTEM = `Incognito: private, READ-ONLY research. You can read files anywhere, search, run a read-only shell in the selected folders, and use web_search/web_fetch. +You CANNOT write — never claim you saved or changed anything. This chat is ephemeral: nothing is stored. Be concise; put findings in your reply.`; + +/** + * Build the per-turn system message. The base rules and the folder-access + * summary are ALWAYS included so the model knows which folders are read-only + * (and won't waste calls writing to them); any host-supplied prompt (project + * context) is appended rather than replacing the base. + */ +/** One-line identity so the model doesn't hallucinate being Claude/GPT. */ +function identityLine(model: string): string { + return `You are iClaw, a private AI assistant${model ? `, powered by ${model}` : ''}. If asked what you are, say exactly that — never claim to be ChatGPT, Claude, Gemini or any other product.`; +} + +function buildSystemPrompt(opts: AgentOptions): string { + if (opts.incognito) { + const parts = [identityLine(opts.model), INCOGNITO_SYSTEM]; + if (opts.allowedFolders.length) { + parts.push( + `\nShell folders (read-only): ${opts.allowedFolders.join(', ')}. File reads aren't limited to these; secrets are always refused.`, + ); + } else { + parts.push('\nNo shell folders selected — run_command is off; use read_file / search_files / web_fetch.'); + } + if (opts.systemPrompt?.trim()) parts.push(`\n${opts.systemPrompt.trim()}`); + return parts.join('\n'); + } + + const parts = [identityLine(opts.model), DEFAULT_SYSTEM, HOST_INSTALL_POLICY]; + + const folders = opts.folderAccess?.length + ? opts.folderAccess + : opts.allowedFolders.map((path) => ({ path, readonly: false })); + if (folders.length) { + const lines = folders.map((f) => `- ${f.path} (${f.readonly ? 'read-only' : 'rw'})`); + parts.push( + `\nFolders (use these exact paths):\n${lines.join('\n')}\n` + + `Never write or run commands in a read-only folder — say it's read-only and offer a rw folder instead.`, + ); + } + + if (opts.systemPrompt?.trim()) parts.push(`\n${opts.systemPrompt.trim()}`); + return parts.join('\n'); +} + +/** + * Run one user turn. Yields events as the agent works. + * Handles multi-step tool loops automatically. + */ +export async function* runAgentTurn( + history: Message[], + userMessage: string, + opts: AgentOptions, +): AsyncGenerator<AgentEvent> { + const client = new OpenAI({ + baseURL: 'https://openrouter.ai/api/v1', + apiKey: opts.apiKey, + }); + + // A tool may emit a savings note mid-call (analyze_link). Collect it here and + // flush as a `note` event right after the tool_result it belongs to. + const pendingNotes: SavingsNote[] = []; + const pendingImages: ImageRef[] = []; + const toolCtx: ToolContext = { + allowedFolders: opts.allowedFolders, + folderAccess: opts.folderAccess, + runShell: opts.runShell, + linkSandbox: opts.linkSandbox, + readOnly: opts.incognito, + readAnywhere: opts.incognito, + requestWriteApproval: opts.onWriteApproval ?? (async () => true), + onNote: (note) => pendingNotes.push(note), + onImage: (image) => pendingImages.push(image), + // Fresh per turn → web_fetch dedups repeat pulls of the same URL within this + // turn (and never leaks fetched bodies across turns). + fetchCache: new Map<string, string>(), + }; + + // Per-mode tool set. Incognito is read-only, so don't ship write_file/edit_file + // schemas it can't use (saves prompt tokens). Web research tools run host-side + // and are never exposed to Secure Mode (it has its own loop + container network + // gate that a host-side fetch would bypass). + const fileTools = opts.incognito + ? TOOL_DEFINITIONS.filter((t) => t.function.name !== 'write_file' && t.function.name !== 'edit_file') + : TOOL_DEFINITIONS; + // analyze_link (yt-dlp in the session container) is offered only when a + // sandbox backend is wired (Docker up). Without it, the model uses web_fetch. + const tools = [ + ...fileTools, READ_SUMMARY_TOOL, WEB_FETCH_TOOL, WEB_SEARCH_TOOL, + // show_image lets the agent surface a real image file inline. Not in + // Incognito — that turn is ephemeral, so there's no message to attach to. + ...(opts.incognito ? [] : [SHOW_IMAGE_TOOL]), + // analyze_link + social_search both run in the session container, so both + // are offered only when a sandbox backend is wired. + ...(opts.linkSandbox ? [ANALYZE_LINK_TOOL, SOCIAL_SEARCH_TOOL] : []), + ]; + + // When the user dropped image(s), send the turn's user message as a + // multimodal content array (text + image blocks) so a vision model sees them. + // History keeps only the text form (the caller stores `userMessage`), so the + // images aren't re-sent every round. + const userMsg: OpenAI.Chat.ChatCompletionUserMessageParam = opts.images?.length + ? { + role: 'user', + content: [ + { type: 'text', text: userMessage }, + ...opts.images.map((url) => ({ type: 'image_url' as const, image_url: { url } })), + ], + } + : { role: 'user', content: userMessage }; + + const messages: Message[] = [ + { role: 'system', content: buildSystemPrompt(opts) }, + ...history, + userMsg, + ]; + + // Token usage across all rounds (dev-mode display). `turnCached` = how many + // prompt tokens were served from the provider's prefix cache. + let turnTokens = 0; + let turnCached = 0; + const dumpTurnId = newTurnId(); + const dumpMode = opts.incognito ? 'incognito' : 'work'; + const guard = makeToolGuard(); + + // Paragraph break to insert before the FIRST text of a round that follows a + // tool call. The model streams a fresh sentence each round ("…the repo" → + // run_command → "Oh, the repo is there!"); consumers concatenate text deltas + // raw, so without this the two segments glue into "…the repoOh, the repo…". + let pendingSeparator = ''; + + // Dead-round circuit-breaker state: count consecutive all-low-value rounds; + // once we trip, force the next round to answer with tools disabled. + let deadStreak = 0; + let forceConclude = false; + + // Max tool-call rounds to prevent infinite loops (env-tunable: ICLAW_MAX_ROUNDS). + for (let round = 0; round < MAX_ROUNDS; round++) { + // User pressed Stop between rounds → end cleanly (partial text already sent). + if (opts.signal?.aborted) { + yield { type: 'done', tokens: turnTokens || undefined, cached: turnCached || undefined }; + return; + } + let textBuffer = ''; + const toolCallBuffers: Record<string, { name: string; arguments: string }> = {}; + + // Dev mode: persist exactly what we're about to send (incl. tool schemas). + dumpPrompt({ turnId: dumpTurnId, mode: dumpMode, model: opts.model, round, messages, tools }); + + let stream: AsyncIterable<OpenAI.Chat.ChatCompletionChunk>; + try { + stream = await client.chat.completions.create( + { + model: opts.model, + messages: withPromptCaching(messages), + tools: tools as unknown as OpenAI.Chat.ChatCompletionTool[], + // Normally 'auto'; after the dead-round breaker trips we send 'none' so + // the model MUST produce a final answer instead of calling more tools. + tool_choice: forceConclude ? 'none' : 'auto', + stream: true, + stream_options: { include_usage: true }, // final chunk carries token usage + }, + { signal: opts.signal }, + ); + } catch (err) { + // Aborted by the user → not an error; end the turn cleanly. + if (opts.signal?.aborted) { + yield { type: 'done', tokens: turnTokens || undefined, cached: turnCached || undefined }; + return; + } + yield { type: 'error', message: describeApiError(err) }; + return; + } + + let finishReason: string | null = null; + + // The provider can inject an error MID-STREAM (e.g. a 429 surfaced as a JSON + // error in the SSE body). Catch it here so the turn ends with a clean error + // event instead of throwing out of the generator and leaving a truncated reply. + try { + for await (const chunk of stream) { + // Usage rides in a final chunk that has no choices — capture it first. + if (chunk.usage?.total_tokens) turnTokens += chunk.usage.total_tokens; + const cached = (chunk.usage as { prompt_tokens_details?: { cached_tokens?: number } } | undefined) + ?.prompt_tokens_details?.cached_tokens; + if (cached) turnCached += cached; + const choice = chunk.choices[0]; + if (!choice) continue; + + finishReason = choice.finish_reason ?? finishReason; + const delta = choice.delta; + + // Text content + if (delta.content) { + // First text after a tool round → emit the paragraph break so this + // new segment doesn't fuse onto the previous round's last word. + if (pendingSeparator) { + yield { type: 'text', content: pendingSeparator }; + pendingSeparator = ''; + } + textBuffer += delta.content; + yield { type: 'text', content: delta.content }; + } + + // Tool call accumulation + if (delta.tool_calls) { + for (const tc of delta.tool_calls) { + const idx = String(tc.index ?? 0); + if (!toolCallBuffers[idx]) { + toolCallBuffers[idx] = { name: '', arguments: '' }; + } + if (tc.function?.name) toolCallBuffers[idx].name += tc.function.name; + if (tc.function?.arguments) toolCallBuffers[idx].arguments += tc.function.arguments; + } + } + } + } catch (err) { + // Aborted by the user → not an error; end the turn cleanly. + if (opts.signal?.aborted) { + yield { type: 'done', tokens: turnTokens || undefined, cached: turnCached || undefined }; + return; + } + yield { type: 'error', message: describeApiError(err) }; + return; + } + + const toolCalls = Object.values(toolCallBuffers); + + // No tool calls → model is done + if (toolCalls.length === 0 || finishReason === 'stop') { + if (textBuffer) { + messages.push({ role: 'assistant', content: textBuffer }); + } + yield { type: 'done', tokens: turnTokens || undefined, cached: turnCached || undefined }; + return; + } + + // Build assistant message with tool calls + const assistantMsg: OpenAI.Chat.ChatCompletionAssistantMessageParam = { + role: 'assistant', + content: textBuffer || null, + tool_calls: toolCalls.map((tc, i) => ({ + id: `call_${i}`, + type: 'function' as const, + function: { name: tc.name, arguments: tc.arguments }, + })), + }; + messages.push(assistantMsg); + + // Execute each tool call + const roundResults: string[] = []; + for (let i = 0; i < toolCalls.length; i++) { + const tc = toolCalls[i]; + let parsedArgs: Record<string, unknown> = {}; + try { + parsedArgs = JSON.parse(tc.arguments); + } catch { + parsedArgs = {}; + } + + yield { type: 'tool_start', name: tc.name, input: parsedArgs }; + + // Guardrail: refuse to re-run an identical call that's already looping. + const blocked = guard.check(tc.name, tc.arguments); + const result = blocked ?? (await executeTool(tc.name as ToolName, parsedArgs, toolCtx)); + roundResults.push(result); + + yield { type: 'tool_result', name: tc.name, result }; + while (pendingNotes.length) yield { type: 'note', note: pendingNotes.shift()! }; + while (pendingImages.length) { + const im = pendingImages.shift()!; + yield { type: 'image', path: im.path, mime: im.mime, fileName: im.fileName, bytes: im.bytes }; + } + + messages.push({ + role: 'tool', + tool_call_id: `call_${i}`, + content: result, + }); + } + // This round spoke before calling tools, and didn't end on a newline — the + // next round's text is a separate thought, so arm a paragraph break for it. + // (Leave a previously-armed break intact for tool-only rounds with no text.) + if (textBuffer && !/\n\s*$/.test(textBuffer)) pendingSeparator = '\n\n'; + + // Mid-turn compaction: on a long multi-round task the accumulated tool + // outputs would be resent every round (O(n²) tokens). Stub out old ones. + shrinkOldToolOutputs(messages); + + // Dead-round circuit-breaker: if EVERY tool result this round was empty/ + // failed/timed-out/guard-blocked, the agent made no progress. After + // DEAD_ROUND_LIMIT such rounds in a row, force a conclusion (tools off next + // round) so we don't grind to MAX_ROUNDS re-trying data we can't reach. + const allDead = roundResults.length > 0 && roundResults.every(isLowValueResult); + deadStreak = allDead ? deadStreak + 1 : 0; + if (deadStreak >= DEAD_ROUND_LIMIT && !forceConclude) { + forceConclude = true; + messages.push({ + role: 'user', + content: + `[system] The last ${deadStreak} tool rounds all came back empty, failed, timed out, or ` + + `duplicate — you're not getting closer. Stop calling tools and answer now with what you've ` + + `gathered, stating plainly which parts you could NOT find or verify. Do not invent numbers.`, + }); + } + // Continue loop — model will see tool results and respond + } + + yield { type: 'error', message: `Reached the step limit (${MAX_ROUNDS} tool rounds). Send "continue" to keep going, or raise ICLAW_MAX_ROUNDS.` }; +} diff --git a/packages/iclaw-runtime/src/agent/prompt-dump.ts b/packages/iclaw-runtime/src/agent/prompt-dump.ts new file mode 100644 index 0000000..f4d5c21 --- /dev/null +++ b/packages/iclaw-runtime/src/agent/prompt-dump.ts @@ -0,0 +1,121 @@ +/** + * Dev-mode prompt dumper. + * + * When ICLAW_DEV_MODE=true, every request we send to the model (the full + * messages array + tool schemas, i.e. exactly what burns prompt tokens) is + * appended to a per-turn JSONL file, with a per-section size breakdown so you + * can see WHAT is heavy (system prompt vs tool schemas vs history). + * + * Self-limiting: files older than 24h are deleted, and the directory is trimmed + * (oldest-first) to stay under 50 MB. Best-effort and fully wrapped in try/catch + * — dumping must never break or slow down a real turn. + * + * Location: $ICLAW_DEV_PROMPT_DIR or ~/.iclaw/dev-prompts/<turnId>.jsonl + */ +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; + +const TTL_MS = 24 * 60 * 60 * 1000; +const MAX_BYTES = 50 * 1024 * 1024; +const SWEEP_INTERVAL_MS = 30_000; + +let lastSweep = 0; + +function enabled(): boolean { + return process.env.ICLAW_DEV_MODE === 'true'; +} + +function dir(): string { + return process.env.ICLAW_DEV_PROMPT_DIR || path.join(os.homedir(), '.iclaw', 'dev-prompts'); +} + +export function newTurnId(): string { + return `${Date.now().toString(36)}-${Math.random().toString(36).slice(2, 8)}`; +} + +interface DumpRecord { + turnId: string; + mode: string; + model: string; + round: number; + messages: readonly unknown[]; + tools?: readonly unknown[]; +} + +function summarize(messages: readonly unknown[], tools?: readonly unknown[]) { + const charsByRole: Record<string, number> = {}; + let messageChars = 0; + for (const raw of messages) { + const m = raw as { role?: string; content?: unknown }; + const role = m.role || 'unknown'; + const c = typeof m.content === 'string' + ? m.content.length + : JSON.stringify(m.content ?? '').length; + charsByRole[role] = (charsByRole[role] || 0) + c; + messageChars += c; + } + const toolChars = tools ? JSON.stringify(tools).length : 0; + return { + chars_by_role: charsByRole, + chars_messages: messageChars, + chars_tools: toolChars, + // ~4 chars/token — a rough guide, not the billed count. + approx_tokens: Math.ceil((messageChars + toolChars) / 4), + }; +} + +export function dumpPrompt(rec: DumpRecord): void { + if (!enabled()) return; + try { + const d = dir(); + fs.mkdirSync(d, { recursive: true }); + const line = JSON.stringify({ + ts: new Date().toISOString(), + mode: rec.mode, + model: rec.model, + round: rec.round, + sizes: summarize(rec.messages, rec.tools), + messages: rec.messages, + tools: rec.tools, + }) + '\n'; + fs.appendFileSync(path.join(d, `${rec.turnId}.jsonl`), line); + sweep(d); + } catch { + // dev-only; never surface + } +} + +function sweep(d: string): void { + const now = Date.now(); + if (now - lastSweep < SWEEP_INTERVAL_MS) return; + lastSweep = now; + try { + const files = fs.readdirSync(d) + .filter((f) => f.endsWith('.jsonl')) + .map((f) => { + const p = path.join(d, f); + try { const st = fs.statSync(p); return { p, mtime: st.mtimeMs, size: st.size }; } + catch { return null; } + }) + .filter((x): x is { p: string; mtime: number; size: number } => x != null); + + // 1) TTL: drop anything older than 24h. + const live: { p: string; mtime: number; size: number }[] = []; + for (const f of files) { + if (now - f.mtime > TTL_MS) { try { fs.rmSync(f.p, { force: true }); } catch { /* */ } } + else live.push(f); + } + // 2) Size cap: trim oldest until under 50 MB. + let total = live.reduce((a, f) => a + f.size, 0); + if (total > MAX_BYTES) { + live.sort((a, b) => a.mtime - b.mtime); + for (const f of live) { + if (total <= MAX_BYTES) break; + try { fs.rmSync(f.p, { force: true }); total -= f.size; } catch { /* */ } + } + } + } catch { + // ignore + } +} diff --git a/packages/iclaw-runtime/src/agent/security.ts b/packages/iclaw-runtime/src/agent/security.ts new file mode 100644 index 0000000..a0c7c54 --- /dev/null +++ b/packages/iclaw-runtime/src/agent/security.ts @@ -0,0 +1,137 @@ +/** + * Path security for Work Mode. + * All file operations must pass validatePath() before execution. + */ +import fs from 'node:fs'; +import path from 'node:path'; + +const BLOCKED_PATTERNS = [ + '.env', '.ssh', '.gnupg', '.gpg', '.aws', '.azure', '.gcloud', '.kube', '.docker', + 'id_rsa', 'id_ed25519', 'id_dsa', 'private_key', + '.netrc', '.npmrc', '.pypirc', '.secret', +]; + +const BLOCKED_EXTENSIONS = ['.pem', '.key', '.p12', '.pfx', '.cert', '.crt']; + +const BLOCKED_NAMES = [ + 'credentials', 'credentials.json', 'credentials.yaml', 'credentials.yml', + 'secrets', 'secrets.json', 'secrets.yaml', 'secrets.yml', + 'token', 'tokens', +]; + +export class SecurityError extends Error { + constructor(message: string) { + super(message); + this.name = 'SecurityError'; + } +} + +/** Expand a leading ~ and resolve to an absolute path (no symlink resolution). */ +function expandPath(inputPath: string): string { + const expanded = inputPath.startsWith('~/') || inputPath === '~' + ? path.join(process.env.HOME ?? '', inputPath.slice(1)) + : inputPath; + return path.resolve(expanded); +} + +/** + * Return the secret-bearing pattern a real path matches (in any component, by + * extension, or by basename), or null if clean. Shared by validatePath (host + * file tools) and validateMountRoot (container mounts) so both apply the SAME + * deny-list — a path the file tools refuse to touch is also never mounted. + */ +export function findBlockedPattern(realPath: string): string | null { + const parts = realPath.split(path.sep); + for (const part of parts) { + for (const blocked of BLOCKED_PATTERNS) { + if (part === blocked || part.startsWith(blocked + '.')) return blocked; + } + } + const ext = path.extname(realPath).toLowerCase(); + if (BLOCKED_EXTENSIONS.includes(ext)) return ext; + const basename = path.basename(realPath).toLowerCase(); + for (const blocked of BLOCKED_NAMES) { + if (basename === blocked || basename.startsWith(blocked + '.')) return blocked; + } + return null; +} + +/** Resolve and validate a path against allowed folders. Throws SecurityError if blocked. */ +export function validatePath(inputPath: string, allowedFolders: string[]): string { + if (!inputPath) throw new SecurityError('Empty path'); + + const resolved = expandPath(inputPath); + + // Symlink resolution — prevent escape via symlinks + let realPath: string; + try { + realPath = fs.realpathSync(resolved); + } catch { + // File doesn't exist yet (write case) — use resolved path + realPath = resolved; + } + + const blocked = findBlockedPattern(realPath); + if (blocked) { + throw new SecurityError(`Path matches blocked pattern "${blocked}": ${realPath}`); + } + + // Must be under at least one allowed folder + if (allowedFolders.length > 0) { + const allowed = allowedFolders.some((folder) => { + try { + const realFolder = fs.realpathSync(path.resolve(folder)); + return realPath === realFolder || realPath.startsWith(realFolder + path.sep); + } catch { + return false; + } + }); + + if (!allowed) { + throw new SecurityError( + `Path "${realPath}" is outside allowed folders: ${allowedFolders.join(', ')}`, + ); + } + } + + return realPath; +} + +/** + * Validate a folder that's about to be bind-mounted into the command sandbox. + * Resolves ~ and symlinks, then rejects any secret-bearing root (e.g. ~/.ssh, + * a folder named `credentials`) via the SAME deny-list the host file tools use. + * Returns the real (symlink-resolved) path to mount. Throws SecurityError. + * + * Note: this guards the mount ROOT. It deliberately does not try to hide secret + * files nested inside an otherwise-legitimate folder — granting a shell into a + * folder inherently exposes its contents. Callers therefore only mount folders + * the user explicitly selected (never a broad fallback like all of $HOME). + */ +export function validateMountRoot(folderPath: string): string { + if (!folderPath) throw new SecurityError('Empty mount path'); + const realPath = fs.realpathSync(expandPath(folderPath)); + const blocked = findBlockedPattern(realPath); + if (blocked) { + throw new SecurityError(`Folder matches blocked pattern "${blocked}": ${realPath}`); + } + return realPath; +} + +/** Check if a path is read-write allowed (vs read-only). */ +export function isWriteAllowed( + realPath: string, + allowedFolders: { path: string; readonly?: boolean }[], +): boolean { + for (const folder of allowedFolders) { + try { + const realFolder = fs.realpathSync(path.resolve(folder.path)); + if (realPath === realFolder || realPath.startsWith(realFolder + path.sep)) { + return !folder.readonly; + } + } catch { + continue; + } + } + return false; +} diff --git a/packages/iclaw-runtime/src/agent/social-fetch.mjs b/packages/iclaw-runtime/src/agent/social-fetch.mjs new file mode 100644 index 0000000..17d3b43 --- /dev/null +++ b/packages/iclaw-runtime/src/agent/social-fetch.mjs @@ -0,0 +1,161 @@ +#!/usr/bin/env node +// Keyless social-source fetcher. Self-contained: Node built-ins only (global +// fetch, Node 18+), NO imports — so it ships as one asset and runs as-is inside +// the sandbox container via `node` (honouring the network gate, exactly like +// analyze_link runs yt-dlp in-sandbox). Reads params from env, prints a +// normalized JSON payload between markers. Never throws: each source is isolated, +// a failure contributes [] + an error note so the others still return. +// +// Sources (all keyless / free — no API keys): +// reddit -> /search.rss?q= (+ optional shreddit /svc comment enrichment) +// hackernews -> hn.algolia.com/api/v1/search +// +// Reddit technique (RSS discovery + shreddit comment scrape) is adapted from +// mvanhorn/last30days-skill (MIT) — the .json API is dead (403), these endpoints +// are the keyless state of the art. Post upvote score is NOT recoverable keyless. +// +// Env in: SOCIAL_QUERY, SOCIAL_SOURCES(csv), SOCIAL_LIMIT, SOCIAL_TIME, SOCIAL_WITH_COMMENTS +// Std out: "__SOCIAL_JSON__\n" + JSON {query, counts, errors, results[]} + +const UA = + 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 ' + + '(KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36'; + +async function get(url, { accept = '*/*', timeoutMs = 15000 } = {}) { + const ctrl = new AbortController(); + const t = setTimeout(() => ctrl.abort(), timeoutMs); + try { + const r = await fetch(url, { headers: { 'User-Agent': UA, Accept: accept }, signal: ctrl.signal }); + if (!r.ok) throw new Error(`HTTP ${r.status}`); + return await r.text(); + } finally { + clearTimeout(t); + } +} + +function decodeEntities(s) { + return (s || '') + .replace(/&/g, '&').replace(/</g, '<').replace(/>/g, '>') + .replace(/"/g, '"').replace(/�?39;/g, "'").replace(/ /g, ' ') + .replace(/ /g, ' '); +} +const stripTags = (s) => decodeEntities((s || '').replace(/<[^>]+>/g, '')).replace(/\s+/g, ' ').trim(); +const tokens = (s) => new Set((s || '').toLowerCase().match(/[a-z0-9]+/g) || []); +function relevance(query, title) { + const q = tokens(query); + if (q.size === 0) return 0; + const t = tokens(title); + let n = 0; + for (const w of q) if (t.has(w)) n++; + return Math.round((n / q.size) * 1000) / 1000; +} + +// ── Reddit ──────────────────────────────────────────────────────────────────── +async function shredditComments(sub, postId, max = 3) { + const url = `https://www.reddit.com/svc/shreddit/comments/r/${sub}/t3_${postId}`; + let html; + try { html = await get(url, { timeoutMs: 12000 }); } catch { return [null, []]; } + const totalM = html.match(/total-comments="(\d+)"/); + const total = totalM ? Number(totalM[1]) : null; + const comments = []; + const re = /<shreddit-comment(?=[\s>])([^>]*)>/g; + let m; + while ((m = re.exec(html))) { + const attrs = m[1]; + const attr = (k) => (attrs.match(new RegExp(`${k}="([^"]*)"`, 'i')) || [])[1]; + if (Number(attr('depth') || '0') !== 0) continue; // top-level only + const thingId = attr('thingId'); + let body = ''; + if (thingId) { + const bIdx = html.indexOf(`id="${thingId}-post-rtjson-content"`); + if (bIdx !== -1) { + const seg = html.slice(bIdx, bIdx + 2500); + const ps = [...seg.matchAll(/<p[^>]*>([\s\S]*?)<\/p>/g)].map((x) => x[1]); + body = stripTags(ps.join(' ')).slice(0, 280); + } + } + const score = attr('score'); + comments.push({ author: attr('author') || null, score: score != null ? Number(score) : null, body }); + } + comments.sort((x, y) => (y.score ?? -1) - (x.score ?? -1)); + return [total, comments.slice(0, max)]; +} + +async function fetchReddit(query, limit, time, withComments) { + const url = `https://www.reddit.com/search.rss?q=${encodeURIComponent(query)}&sort=relevance&t=${time}`; + const xml = await get(url, { accept: 'application/atom+xml' }); + const items = []; + for (const block of xml.split('<entry>').slice(1)) { + const entry = block.split('</entry>')[0]; + const title = decodeEntities((entry.match(/<title>([\s\S]*?)<\/title>/) || [])[1] || '').trim(); + const href = (entry.match(/<link[^>]*href="([^"]+)"/) || [])[1] || ''; + const author = decodeEntities((entry.match(/<author>[\s\S]*?<name>([\s\S]*?)<\/name>/) || [])[1] || '').trim(); + const created = (entry.match(/<(?:published|updated)>([\s\S]*?)<\/(?:published|updated)>/) || [])[1] || ''; + const subM = href.match(/\/r\/([^/]+)\//); + const isPost = href.includes('/comments/'); + items.push({ + platform: 'reddit', type: isPost ? 'post' : 'subreddit', title, url: href, + author: author || null, subreddit: subM ? subM[1] : null, created: created || null, + score: null, snippet: '', relevance: relevance(query, title), + }); + } + const posts = items.filter((i) => i.type === 'post').sort((a, b) => b.relevance - a.relevance); + const ranked = posts.concat(items.filter((i) => i.type !== 'post')).slice(0, limit); + + if (withComments) { + const targets = ranked.filter((i) => i.type === 'post' && i.subreddit).slice(0, 3); + await Promise.all(targets.map(async (it) => { + const pid = (it.url.match(/\/comments\/([A-Za-z0-9]+)/) || [])[1]; + if (!pid) return; + const [total, comments] = await shredditComments(it.subreddit, pid); + it.num_comments = total; + if (comments.length) it.comments = comments; + })); + } + return ranked; +} + +// ── Hacker News (Algolia) ─────────────────────────────────────────────────────── +const HN_WINDOW = { day: 86400, week: 604800, month: 2592000, year: 31536000 }; +async function fetchHackerNews(query, limit, time) { + let url = `https://hn.algolia.com/api/v1/search?query=${encodeURIComponent(query)}&tags=story&hitsPerPage=${limit}`; + const secs = HN_WINDOW[time]; + if (secs) url += `&numericFilters=created_at_i>${Math.floor(Date.now() / 1000) - secs}`; + const data = JSON.parse(await get(url, { accept: 'application/json' })); + return (data.hits || []).slice(0, limit).map((h) => ({ + platform: 'hackernews', type: 'story', title: h.title || h.story_title || '', + url: h.url || `https://news.ycombinator.com/item?id=${h.objectID}`, + hn_url: `https://news.ycombinator.com/item?id=${h.objectID}`, + author: h.author ?? null, score: h.points ?? null, num_comments: h.num_comments ?? null, + created: h.created_at ?? null, snippet: (h.story_text || '').slice(0, 240), + relevance: relevance(query, h.title || ''), + })); +} + +const SOURCES = { reddit: fetchReddit, hackernews: fetchHackerNews }; + +async function main() { + const query = (process.env.SOCIAL_QUERY || '').trim(); + const sources = (process.env.SOCIAL_SOURCES || 'reddit,hackernews').split(',').map((s) => s.trim()).filter(Boolean); + const limit = Math.max(1, Math.min(50, Number(process.env.SOCIAL_LIMIT || '25'))); + const time = process.env.SOCIAL_TIME || 'month'; + const withComments = ['1', 'true', 'yes'].includes((process.env.SOCIAL_WITH_COMMENTS || '').toLowerCase()); + + const out = { query, counts: {}, errors: {}, results: [] }; + if (!query) { console.log('__SOCIAL_JSON__'); console.log(JSON.stringify({ ...out, error: 'empty query' })); return; } + + await Promise.all(sources.map(async (src) => { + const fn = SOURCES[src]; + if (!fn) { out.errors[src] = 'unknown source'; out.counts[src] = 0; return; } + try { + const res = await fn(query, limit, time, withComments); + out.counts[src] = res.length; out.results.push(...res); + } catch (e) { + out.errors[src] = `${e.name}: ${e.message}`; out.counts[src] = 0; + } + })); + + console.log('__SOCIAL_JSON__'); + console.log(JSON.stringify(out)); +} +main(); diff --git a/packages/iclaw-runtime/src/agent/social.ts b/packages/iclaw-runtime/src/agent/social.ts new file mode 100644 index 0000000..e750ef0 --- /dev/null +++ b/packages/iclaw-runtime/src/agent/social.ts @@ -0,0 +1,184 @@ +// ── social_search (sandboxed keyless social-media search) ───────────────────── +// +// A unified "give keywords → get posts" tool over FREE keyless sources (no API +// keys): Reddit (RSS discovery + shreddit comment scrape) and HackerNews +// (Algolia). Designed to grow to more sources (Lemmy, GitHub, Polymarket…). +// +// Like analyze_link, the actual fetch runs INSIDE the sandbox container via the +// injected `runInSandbox` callback, so all egress honours the same container +// network gate — a host-side fetch would bypass Secure Mode's boundary (the same +// reason web_search/web_fetch are excluded there). The engine is a self-contained +// Node script (social-fetch.mjs) shipped as an asset; the container already has +// `node`, so nothing self-installs. +// +// Kept OUT of TOOL_DEFINITIONS and appended by the runner (like analyze_link). + +import { readFileSync } from 'node:fs'; +import { fileURLToPath } from 'node:url'; +import type { SavingsNote } from './tools.js'; + +export const SOCIAL_SOURCES = ['reddit', 'hackernews'] as const; +const SOCIAL_RESULT_MAX_CHARS = Number(process.env.ICLAW_SOCIAL_MAX) || 14_000; + +export const SOCIAL_SEARCH_TOOL = { + type: 'function' as const, + function: { + name: 'social_search', + description: + 'Search social platforms by KEYWORDS and get real posts — free, no API keys. ' + + 'Sources: reddit (posts + optional top comments), hackernews (stories with points/comments). ' + + 'Runs in the sandbox. Use with_comments:true to also pull top comments for the top Reddit posts. ' + + 'Note: Reddit post upvote counts are not available without a paid key (comment counts/scores are).', + parameters: { + type: 'object', + properties: { + query: { type: 'string', description: 'Keywords / search terms' }, + sources: { + type: 'array', + items: { type: 'string', enum: SOCIAL_SOURCES as unknown as string[] }, + description: 'Which platforms to search (default: all).', + }, + time: { + type: 'string', + enum: ['day', 'week', 'month', 'year', 'all'], + description: 'Recency window (default month).', + }, + limit: { type: 'number', description: 'Max results per source (default 25, max 50).' }, + with_comments: { type: 'boolean', description: 'Also fetch top comments for the top Reddit posts.' }, + }, + required: ['query'], + }, + }, +} as const; + +interface SocialItem { + platform: string; + type: string; + title: string; + url: string; + hn_url?: string; + author: string | null; + subreddit?: string | null; + score?: number | null; + num_comments?: number | null; + created: string | null; + snippet?: string; + relevance: number; + comments?: { author: string | null; score: number | null; body: string }[]; +} +interface SocialPayload { + query: string; + counts: Record<string, number>; + errors: Record<string, string>; + results: SocialItem[]; + error?: string; +} + +/** Single-quote a string for safe embedding in a bash command. */ +function shQuote(s: string): string { + return `'${s.replace(/'/g, `'\\''`)}'`; +} + +// The fetcher is plain JS (no imports) shipped next to this module; base64 it once +// and drop it into the container's workspace cache, then run with `node`. +const FETCHER_URL = new URL('./social-fetch.mjs', import.meta.url); +let fetcherB64: string | null = null; +function getFetcherB64(): string { + if (fetcherB64 === null) fetcherB64 = readFileSync(fileURLToPath(FETCHER_URL)).toString('base64'); + return fetcherB64; +} + +function buildSocialCommand(p: { + query: string; sources: string; limit: number; time: string; withComments: boolean; +}): string { + // Write the fetcher to $HOME — writable in BOTH sandboxes regardless of the + // container uid. Secure runs as `node` (owns /workspace), but Work runs as the + // HOST uid with folders at /work/<n> and NO writable /workspace, so a + // /workspace path fails with EACCES there. $HOME is set in both (Work: + // HOME=/tmp, Secure: /home/node), so it's the one path safe everywhere. + const script = '"$HOME/.iclaw-social-fetch.mjs"'; + return [ + 'set -e', + `echo ${shQuote(getFetcherB64())} | base64 -d > ${script}`, + `SOCIAL_QUERY=${shQuote(p.query)} SOCIAL_SOURCES=${shQuote(p.sources)} ` + + `SOCIAL_LIMIT=${shQuote(String(p.limit))} SOCIAL_TIME=${shQuote(p.time)} ` + + `SOCIAL_WITH_COMMENTS=${shQuote(p.withComments ? '1' : '0')} ` + + `node ${script}`, + ].join('\n'); +} + +/** Render the normalized payload into compact text for the model. */ +function formatPayload(d: SocialPayload): string { + const counts = Object.entries(d.counts).map(([k, v]) => `${k}: ${v}`).join(', ') || 'none'; + const errs = Object.entries(d.errors || {}); + const lines: string[] = [`Social search "${d.query}" — ${counts}` + (errs.length ? ` (errors: ${errs.map(([k, v]) => `${k}=${v}`).join('; ')})` : '')]; + + for (const platform of Object.keys(d.counts)) { + const rows = d.results.filter((r) => r.platform === platform); + if (!rows.length) continue; + lines.push(`\n### ${platform}`); + rows.forEach((r, i) => { + const tag = r.subreddit ? `[r/${r.subreddit}] ` : ''; + lines.push(`${i + 1}. ${tag}${r.title || '(untitled)'}\n ${r.url}`); + const meta: string[] = []; + if (r.author) meta.push(`by ${r.author}`); + if (r.score != null) meta.push(`${r.score} points`); + if (r.num_comments != null) meta.push(`${r.num_comments} comments`); + if (meta.length) lines.push(` ${meta.join(' · ')}`); + for (const c of r.comments || []) { + lines.push(` • [${c.score ?? '?'}] ${c.author ?? 'anon'}: ${c.body || '(no text)'}`); + } + }); + } + return lines.join('\n'); +} + +/** + * social_search implementation. `runInSandbox` runs a bash command inside the + * session's container; `networkEnabled` mirrors the chat's network gate. + */ +export async function socialSearch( + args: Record<string, unknown>, + deps: { runInSandbox: (command: string) => Promise<string>; networkEnabled: boolean; onNote?: (note: SavingsNote) => void }, +): Promise<string> { + const query = String(args.query ?? '').trim(); + if (!query) return 'social_search needs a non-empty query (keywords/terms).'; + if (!deps.networkEnabled) { + return 'social_search needs network, which is currently OFF for this chat. Ask the user to enable network, then retry.'; + } + + const requested = Array.isArray(args.sources) ? args.sources.map(String) : [...SOCIAL_SOURCES]; + const sources = requested.filter((s) => (SOCIAL_SOURCES as readonly string[]).includes(s)); + if (!sources.length) return `social_search: unknown source(s). Supported: ${SOCIAL_SOURCES.join(', ')}.`; + const limit = Math.max(1, Math.min(50, Number(args.limit) || 25)); + const time = ['day', 'week', 'month', 'year', 'all'].includes(String(args.time)) ? String(args.time) : 'month'; + const withComments = args.with_comments === true; + + let raw: string; + try { + raw = await deps.runInSandbox(buildSocialCommand({ query, sources: sources.join(','), limit, time, withComments })); + } catch (err) { + return `social_search failed to run in the sandbox: ${err instanceof Error ? err.message : String(err)}`; + } + + const marker = raw.lastIndexOf('__SOCIAL_JSON__'); + if (marker === -1) return `social_search: no result from the sandbox. Output: ${raw.slice(0, 400)}`; + let payload: SocialPayload; + try { + payload = JSON.parse(raw.slice(marker + '__SOCIAL_JSON__'.length).trim()); + } catch { + return `social_search: could not parse result. Output: ${raw.slice(marker, marker + 400)}`; + } + + const total = Object.values(payload.counts || {}).reduce((a, b) => a + b, 0); + if (!total) { + const errs = Object.entries(payload.errors || {}).map(([k, v]) => `${k}: ${v}`).join('; '); + return `social_search found nothing for "${query}".${errs ? ` Source errors — ${errs}.` : ''} Try different keywords or sources.`; + } + + const text = formatPayload(payload); + if (text.length > SOCIAL_RESULT_MAX_CHARS) { + return text.slice(0, SOCIAL_RESULT_MAX_CHARS) + `\n\n…[truncated — narrow the query or lower limit]`; + } + return text; +} diff --git a/packages/iclaw-runtime/src/agent/tools.ts b/packages/iclaw-runtime/src/agent/tools.ts new file mode 100644 index 0000000..a96c85a --- /dev/null +++ b/packages/iclaw-runtime/src/agent/tools.ts @@ -0,0 +1,1310 @@ +/** + * Tool definitions (JSON schema for the model) + implementations. + * All file operations are validated against allowedFolders. + */ +import fs from 'node:fs'; +import path from 'node:path'; +import { execFile } from 'node:child_process'; +import { promisify } from 'node:util'; +import { createRequire } from 'node:module'; + +import { validatePath, isWriteAllowed, SecurityError } from './security.js'; + +// Bundled ripgrep binary — shipped as a dependency so it's ALWAYS present (no +// per-host "is rg installed?" branching, no grep fallback). We drive it with +// our own ignore list (below), so it works the same with or without git. +// @vscode/ripgrep is CommonJS with no type declarations, so pull rgPath through +// createRequire rather than an ESM import (avoids the export-shape mismatch). +const { rgPath } = createRequire(import.meta.url)('@vscode/ripgrep') as { rgPath: string }; + +const execFileAsync = promisify(execFile); + +// Refuse to slurp a huge file into memory at all (use search_files instead). +const MAX_FILE_BYTES = Number(process.env.ICLAW_MAX_FILE_BYTES) || 5_000_000; +const COMMAND_TIMEOUT = 30_000; +const WEB_FETCH_TIMEOUT = 20_000; +const WEB_FETCH_MAX_CHARS = 20_000; + +// ── Token-saving output caps ────────────────────────────────────────────────── +// Tool outputs are the biggest token sink in multi-turn chats: they land in the +// history and get resent every round. Cap what we hand back to the model. +const MAX_FILE_READ_CHARS = Number(process.env.ICLAW_MAX_FILE_READ) || 16_000; +const MAX_CMD_OUTPUT_CHARS = Number(process.env.ICLAW_MAX_CMD_OUTPUT) || 8_000; +const MAX_LIST_ENTRIES = Number(process.env.ICLAW_MAX_LIST_ENTRIES) || 200; +// search_files: cap matching lines returned per file so a high-frequency term +// can't dump a whole file into history. +const MAX_MATCH_LINES_PER_FILE = Number(process.env.ICLAW_MAX_MATCH_LINES) || 30; + +// Directories to never descend into when searching. Different stacks bury huge +// generated trees in different places (JS node_modules, Rust target, Python +// .venv/__pycache__, Java .gradle, …); scanning them is what made unscoped +// searches time out. This is OUR list, fed to ripgrep as ignore globs, so it +// applies whether or not the user has git. ripgrep additionally skips binary +// files (videos/photos/archives/office docs) and follows no symlinks — exactly +// the behaviour a non-technical user's media-heavy folders need. +// Override the whole set with ICLAW_SEARCH_EXCLUDE_DIRS (comma-separated). +const SEARCH_EXCLUDE_DIRS = (process.env.ICLAW_SEARCH_EXCLUDE_DIRS + ? process.env.ICLAW_SEARCH_EXCLUDE_DIRS.split(',').map((s) => s.trim()).filter(Boolean) + : ['node_modules', '.git', '.svn', '.hg', '.cache', '.next', '.nuxt', 'dist', 'build', + 'out', 'target', 'vendor', '.venv', 'venv', '.tox', '__pycache__', '.mypy_cache', + '.pytest_cache', '.gradle', '.idea', 'Pods', '.terraform', '.tools']); + +// Cheap-model summarizer (read_summary, web_fetch summarize). Moves the cost of +// reading big content onto a cheap model; the expensive model + history only +// carry the short summary. +const SUMMARY_MODEL = process.env.ICLAW_SUMMARY_MODEL || 'google/gemini-2.5-flash-lite'; +const SUMMARY_MAX_INPUT_CHARS = Number(process.env.ICLAW_SUMMARY_MAX_INPUT) || 60_000; + +export const TOOL_OUTPUT_MAX_CHARS = MAX_CMD_OUTPUT_CHARS; + +/** + * Compress raw command output before it lands in history (rtk-inspired, but + * generic + lossless-for-signal — no per-command filters that could hide a + * failing test or an error the agent needs). Runs BEFORE clampMiddle so the cap + * keeps real content instead of progress-bar/ANSI noise. Four safe passes: + * 1. strip ANSI escape codes (colours, cursor moves) — pure noise to a model, + * and confirmed present in real run_command output (e.g. "\x1b[95m…\x1b[0m"); + * 2. collapse carriage-return progress lines to their final state (npm/pip/ + * docker/git "Downloading 12%…100%" → just the last frame); + * 3. dedupe runs of ≥3 identical consecutive lines into one + "(×N)"; + * 4. squeeze 3+ blank lines down to one. + */ +// eslint-disable-next-line no-control-regex +const ANSI_RE = /\x1b\[[0-9;?]*[ -/]*[@-~]|\x1b\][^\x07\x1b]*(?:\x07|\x1b\\)|\x1b[@-Z\\-_]/g; + +export function compressCommandOutput(s: string): string { + if (!s) return s; + let text = s.replace(ANSI_RE, ''); + // Carriage-return progress: keep only what survives the last \r on each line. + text = text + .split('\n') + .map((line) => (line.includes('\r') ? line.slice(line.lastIndexOf('\r') + 1) : line)) + .join('\n'); + // Dedupe runs of ≥3 identical, non-blank consecutive lines into one + "(×N)". + const lines = text.split('\n'); + const out: string[] = []; + for (let i = 0; i < lines.length; ) { + let j = i + 1; + while (j < lines.length && lines[j] === lines[i]) j++; + const run = j - i; + if (run >= 3 && lines[i].trim()) out.push(`${lines[i]} …(×${run})`); + else for (let k = i; k < j; k++) out.push(lines[k]); + i = j; + } + return out.join('\n').replace(/\n{3,}/g, '\n\n'); +} + +/** Keep the head and tail of long output (errors are usually at the end). */ +export function clampMiddle(s: string, max: number): string { + if (s.length <= max) return s; + const head = Math.ceil(max * 0.6); + const tail = max - head; + return s.slice(0, head) + + `\n\n…[truncated ${(s.length - max).toLocaleString()} of ${s.length.toLocaleString()} chars — refine the command or use search_files/read_file]…\n\n` + + s.slice(s.length - tail); +} + +// ── Tool JSON schemas (sent to the model) ──────────────────────────────────── + +export const TOOL_DEFINITIONS = [ + { + type: 'function' as const, + function: { + name: 'list_files', + description: 'List a directory. Dirs show as "[dir] name/", files as "[file] name".', + parameters: { + type: 'object', + properties: { path: { type: 'string', description: 'Directory path' } }, + required: ['path'], + }, + }, + }, + { + type: 'function' as const, + function: { + name: 'read_file', + description: 'Read a file.', + parameters: { + type: 'object', + properties: { path: { type: 'string', description: 'File path' } }, + required: ['path'], + }, + }, + }, + { + type: 'function' as const, + function: { + name: 'search_files', + description: 'Find files. To LOCATE a file by its name (e.g. "where is config.json?"), pass "name" — it returns just the matching paths, cheaply. To search file CONTENTS, pass "query". Omit "path" to cover ALL folders you have access to in one call — do that instead of searching them one by one.', + parameters: { + type: 'object', + properties: { + path: { type: 'string', description: 'Directory to search. Omit to search every accessible folder in one call.' }, + name: { type: 'string', description: 'Locate files by NAME or fragment (case-insensitive), returning paths only. Use this to find where a file lives — not "query".' }, + query: { type: 'string', description: 'Search file CONTENTS for this string. Use "name" instead when you just want to locate a file.' }, + filePattern: { type: 'string', description: 'Optional glob to limit which files "query" searches, e.g. "*.ts"' }, + }, + required: [], + }, + }, + }, + { + type: 'function' as const, + function: { + name: 'write_file', + description: 'Create or overwrite a whole file (needs approval). Use edit_file to change existing files.', + parameters: { + type: 'object', + properties: { + path: { type: 'string', description: 'File path' }, + content: { type: 'string', description: 'Full file content' }, + }, + required: ['path', 'content'], + }, + }, + }, + { + type: 'function' as const, + function: { + name: 'edit_file', + description: 'Replace an exact, UNIQUE snippet in a file (old_string→new_string). Preferred for edits; needs approval.', + parameters: { + type: 'object', + properties: { + path: { type: 'string', description: 'File path' }, + old_string: { type: 'string', description: 'Exact unique text to replace' }, + new_string: { type: 'string', description: 'Replacement' }, + }, + required: ['path', 'old_string', 'new_string'], + }, + }, + }, + { + type: 'function' as const, + function: { + name: 'run_command', + description: 'Run a shell command in an allowed folder. Chain steps with && to save calls.', + parameters: { + type: 'object', + properties: { + command: { type: 'string', description: 'Command' }, + cwd: { type: 'string', description: 'Working dir (allowed folder)' }, + }, + required: ['command', 'cwd'], + }, + }, + }, +] as const; + +/** + * Web research tool. Kept OUT of TOOL_DEFINITIONS and appended by the agent loop + * only when enabled (Incognito), so it never reaches Secure Mode — there, all + * network must go through the container's `--network` flag, and a host-side + * fetch would bypass that boundary. + */ +export const WEB_FETCH_TOOL = { + type: 'function' as const, + function: { + name: 'web_fetch', + description: 'Fetch an http(s) URL and return its text (HTML stripped). Read-only. By default returns a concise summary (cheaper). GitHub repo/file URLs are auto-redirected to raw markdown/source, and Reddit links to old.reddit.com (server-rendered, fetchable). Set summarize:false when you need the EXACT text.', + parameters: { + type: 'object', + properties: { + url: { type: 'string', description: 'Absolute http(s) URL. A github.com repo or /blob/ file link is fetched as raw README/source; a reddit.com link via old.reddit.com.' }, + summarize: { type: 'boolean', description: 'Defaults to true (concise gist). Set false to get the exact page text — do this for full lists, precise numbers/dates, source code, or anything you will quote verbatim.' }, + focus: { type: 'string', description: 'What the summary should focus on (only used when summarizing).' }, + }, + required: ['url'], + }, + }, +} as const; + +/** + * read_summary — read a file and return a SHORT summary via a cheap model, + * instead of dumping the whole file into the expensive model's context (and + * history). Host-loop only (Work / Incognito); kept out of Secure, since the + * summary call is a host-side network request that would bypass the sandbox's + * container network gate. + */ +export const READ_SUMMARY_TOOL = { + type: 'function' as const, + function: { + name: 'read_summary', + description: 'Read a file and return a SHORT summary (cheap model) — for when you just need the gist of a large file. Use read_file when you need exact content to edit.', + parameters: { + type: 'object', + properties: { + path: { type: 'string', description: 'File path' }, + focus: { type: 'string', description: 'What to focus the summary on (optional)' }, + }, + required: ['path'], + }, + }, +} as const; + +/** + * Web search — kept OUT of TOOL_DEFINITIONS (like web_fetch) and appended by the + * agent loop only for the host-loop modes (Work / Incognito), never Secure + * (host-side network would bypass the sandbox's container network gate). + */ +export const WEB_SEARCH_TOOL = { + type: 'function' as const, + function: { + name: 'web_search', + description: 'Search the web; returns top results (title, url, snippet). Then web_fetch a URL for details.', + parameters: { + type: 'object', + properties: { + query: { type: 'string', description: 'Search query' }, + count: { type: 'number', description: 'Max results (default 6)' }, + }, + required: ['query'], + }, + }, +} as const; + +/** + * show_image — display an image file to the user INLINE in the chat. + * + * Kept OUT of TOOL_DEFINITIONS and appended by the loop (like the web tools). + * Use after creating or locating an image the user should actually SEE — a + * chart, diagram, QR code, screenshot, rendered figure. The path must be a real + * image file inside one of the allowed folders. + */ +export const SHOW_IMAGE_TOOL = { + type: 'function' as const, + function: { + name: 'show_image', + description: + 'Display an image FILE to the user inline in the chat. Use right after you create or find an image ' + + 'the user should see (a chart/plot, diagram, QR code, screenshot, rendered figure). Pass the path to a ' + + 'PNG/JPG/GIF/WebP/SVG inside an allowed folder. Do NOT use for text output — only real image files.', + parameters: { + type: 'object', + properties: { + path: { type: 'string', description: 'Path to the image file (PNG/JPG/GIF/WebP/SVG) inside an allowed folder.' }, + caption: { type: 'string', description: 'Optional short caption shown with the image.' }, + }, + required: ['path'], + }, + }, +} as const; + +/** + * analyze_link — extract the *content* behind a URL (not the raw HTML). + * + * Kept OUT of TOOL_DEFINITIONS and appended by the loop, like the web tools. + * Unlike web_fetch/web_search (host-side), this runs ENTIRELY inside the + * sandbox: the loop injects a `runInSandbox` callback wired to the container, + * so the network egress respects the same container network gate. First handler + * is YouTube (subtitles via yt-dlp); the registry is built to grow to more + * sites without touching the schema. + * + * `mode` is the token lever: "summary" (default) hands the extracted text to a + * cheap model focused on `purpose` and returns only the gist; "full" returns the + * cleaned text (clamped). Always prefer "summary" unless you need exact wording. + */ +export const ANALYZE_LINK_TOOL = { + type: 'function' as const, + function: { + name: 'analyze_link', + description: + 'Extract and read the content behind a URL (currently: YouTube video subtitles/transcript). ' + + 'Runs in the sandbox. Use mode:"summary" (default, cheap) for the gist; mode:"full" only when ' + + 'you need exact wording. Always pass a short "purpose" so the summary keeps what you actually need. ' + + 'For plain web articles use web_fetch instead.', + parameters: { + type: 'object', + properties: { + url: { type: 'string', description: 'Absolute http(s) URL (e.g. a YouTube video link)' }, + mode: { + type: 'string', + enum: ['summary', 'full'], + description: 'summary = cheap, purpose-focused gist (default); full = cleaned transcript (clamped)', + }, + purpose: { + type: 'string', + description: 'One short sentence on what you need from the link — drives the summary extraction', + }, + }, + required: ['url'], + }, + }, +} as const; + +export type ToolName = + | 'list_files' | 'read_file' | 'read_summary' | 'search_files' | 'write_file' | 'edit_file' + | 'run_command' | 'web_fetch' | 'web_search' | 'analyze_link' | 'social_search' | 'show_image'; + +/** A host image the agent asked to display inline (via show_image). */ +export interface ImageRef { + /** Absolute HOST path of the image (already inside an allowed folder). */ + path: string; + mime: string; + fileName: string; + bytes: number; +} + +/** + * A token/cost saving worth surfacing in the chat. Emitted when a tool fed the + * main model a cheap SUMMARY instead of the full extracted content (e.g. + * analyze_link summary mode reads a long video transcript but hands the model + * only the gist). The loop forwards it as a `note` event → a chat system note. + */ +export interface SavingsNote { + /** Which tool produced the saving — drives the chat-note wording. */ + kind: 'analyze_link' | 'read_file' | 'read_summary' | 'run_command' | 'search'; + /** Short human label for the source, e.g. "YouTube transcript". */ + source: string; + /** + * Whole-percent of content NOT sent to the main model (1 - delivered/full). + * Present only when we actually hold BOTH numbers as a byproduct of normal + * work (read_file/read_summary/run_command/analyze_link). Omitted for `search`, + * where ripgrep trims long lines but never reports how many bytes it dropped — + * so we surface a quantity-free "trimmed oversized output" note instead of a + * fabricated percentage. + */ + savedPct?: number; + /** Chars the model would have ingested otherwise (full). */ + fullChars?: number; + /** Chars actually delivered to the model. */ + deliveredChars?: number; +} + +// Only post a "saved you X%" note when the win is real and worth mentioning — +// a tiny truncation isn't worth a chat row. Both gates must pass. +const SAVINGS_MIN_FULL_CHARS = 4_000; +const SAVINGS_MIN_PCT = 30; + +/** + * Emit a "saved you X%" note IFF we already hold both the full and delivered + * sizes (zero extra work — just a subtraction) and the saving clears the gates. + * Used by read_file / read_summary / run_command truncation. Never fabricates a + * baseline; if the numbers aren't both in hand, the caller simply doesn't call. + */ +function emitTruncationSaving( + ctx: ToolContext, + kind: SavingsNote['kind'], + source: string, + fullChars: number, + deliveredChars: number, +): void { + if (!ctx.onNote || fullChars < SAVINGS_MIN_FULL_CHARS || deliveredChars >= fullChars) return; + const savedPct = Math.round((1 - deliveredChars / fullChars) * 100); + if (savedPct < SAVINGS_MIN_PCT) return; + ctx.onNote({ kind, source, savedPct, fullChars, deliveredChars }); +} + +// ── Tool context (injected per-session) ────────────────────────────────────── + +export interface ToolContext { + allowedFolders: string[]; + /** + * Per-folder access levels (path + readonly flag). When provided, write_file + * is denied for paths under a read-only folder. When omitted (e.g. restored + * sessions) all allowed folders are treated as writable. + */ + folderAccess?: { path: string; readonly: boolean }[]; + /** + * Runs a shell command for run_command. Injected so the host never executes + * bash directly: Work Mode wires this to a Docker container with per-folder + * :ro/:rw mounts (the kernel enforces read-only). When omitted (no Docker), + * run_command is disabled and returns a guidance message — the strict + * fallback that keeps read-only an honest guarantee. + */ + runShell?: (command: string, cwd: string) => Promise<string>; + /** + * Runs an analyze_link helper command inside the session's sandbox container + * (warm-reused; yt-dlp self-installs once). Injected so yt-dlp — which parses + * untrusted YouTube data — never runs on the host. Omitted when no Docker → + * analyze_link returns a guidance message. + */ + linkSandbox?: (command: string) => Promise<string>; + /** + * Incognito (read-only): write_file is denied outright (nothing ever hits + * disk), and run_command is only reachable via a read-only sandbox. + */ + readOnly?: boolean; + /** + * Incognito: file reads (read/list/search) are NOT restricted to + * allowedFolders — the agent may read anywhere on the host. The secret + * deny-list (BLOCKED_PATTERNS in security.ts) still applies, so .ssh/.env/ + * credentials etc. are refused regardless. + */ + readAnywhere?: boolean; + /** Called when agent wants to write — returns true if approved, false if rejected. */ + requestWriteApproval: (filePath: string, content: string) => Promise<boolean>; + /** + * Optional sink for a user-visible "saved N% cost" note (analyze_link summary + * mode). The agent loop wires this to forward a `note` event to the chat. + */ + onNote?: (note: SavingsNote) => void; + /** + * Optional sink for show_image: an image file (already validated to live + * inside an allowed folder) the agent wants displayed inline in the chat. The + * loop wires this to emit an `image` event the host turns into an attachment. + */ + onImage?: (image: ImageRef) => void; + /** + * Within-turn cache of fetched page bodies, keyed by normalized URL + * (`normalizeFetchUrl`). A research turn often re-fetches the same page with a + * tweaked `focus`; with this set, web_fetch serves the body from memory and + * re-summarizes for the new focus instead of re-hitting the network. Created + * fresh per turn by the loop; absent → caching off (each fetch goes to network). + */ + fetchCache?: Map<string, string>; +} + +/** Folders to validate reads against — empty (anywhere) for Incognito. */ +function readFolders(ctx: ToolContext): string[] { + return ctx.readAnywhere ? [] : ctx.allowedFolders; +} + +// ── Tool implementations ────────────────────────────────────────────────────── + +export async function executeTool( + name: ToolName, + args: Record<string, unknown>, + ctx: ToolContext, +): Promise<string> { + try { + switch (name) { + case 'list_files': return await listFiles(args, ctx); + case 'read_file': return await readFile(args, ctx); + case 'read_summary': return await readSummary(args, ctx); + case 'search_files': return await searchFiles(args, ctx); + case 'write_file': return await writeFile(args, ctx); + case 'edit_file': return await editFile(args, ctx); + case 'run_command': return await runCommand(args, ctx); + case 'web_fetch': return await webFetch(args, ctx); + case 'web_search': return await webSearch(args); + case 'show_image': return showImage(args, ctx); + case 'analyze_link': + if (!ctx.linkSandbox) { + return 'analyze_link needs a sandbox container (Docker), which is unavailable here. Use web_fetch/web_search instead.'; + } + return await analyzeLink(args, { runInSandbox: ctx.linkSandbox, networkEnabled: true, onNote: ctx.onNote }); + case 'social_search': { + if (!ctx.linkSandbox) { + return 'social_search needs a sandbox container (Docker), which is unavailable here. Use web_search instead.'; + } + const { socialSearch } = await import('./social.js'); + return await socialSearch(args, { runInSandbox: ctx.linkSandbox, networkEnabled: true, onNote: ctx.onNote }); + } + default: return `Unknown tool: ${name}`; + } + } catch (err) { + if (err instanceof SecurityError) return `Security error: ${err.message}`; + return `Error: ${err instanceof Error ? err.message : String(err)}`; + } +} + +async function listFiles(args: Record<string, unknown>, ctx: ToolContext): Promise<string> { + const dir = validatePath(args.path as string, readFolders(ctx)); + const entries = fs.readdirSync(dir, { withFileTypes: true }); + // Explicit [dir]/[file] labels (+ trailing slash on dirs) so the model can + // reliably tell directories from files and recurse without guessing. + const lines = entries + .sort((a, b) => Number(b.isDirectory()) - Number(a.isDirectory()) || a.name.localeCompare(b.name)) + .map((e) => (e.isDirectory() ? `[dir] ${e.name}/` : `[file] ${e.name}`)); + if (lines.length === 0) return '(empty directory)'; + // Cap big listings — the full list otherwise lands in history and is resent + // every round. + if (lines.length > MAX_LIST_ENTRIES) { + const shown = lines.slice(0, MAX_LIST_ENTRIES); + return `${shown.join('\n')}\n…[+${lines.length - MAX_LIST_ENTRIES} more entries — narrow with search_files]`; + } + return lines.join('\n'); +} + +async function readFile(args: Record<string, unknown>, ctx: ToolContext): Promise<string> { + const filePath = validatePath(args.path as string, readFolders(ctx)); + const stat = fs.statSync(filePath); + if (stat.size > MAX_FILE_BYTES) { + return `File too large to read whole (${stat.size.toLocaleString()} bytes). Use search_files to find the part you need.`; + } + const content = fs.readFileSync(filePath, 'utf-8'); + if (content.length > MAX_FILE_READ_CHARS) { + // We already read the whole file; handing the model only the head is a real, + // free-to-measure saving (full = content.length, delivered = the cap). + emitTruncationSaving(ctx, 'read_file', path.basename(filePath), content.length, MAX_FILE_READ_CHARS); + return content.slice(0, MAX_FILE_READ_CHARS) + + `\n\n…[truncated: showing first ${MAX_FILE_READ_CHARS.toLocaleString()} of ${content.length.toLocaleString()} chars. Use search_files for specific content, or read_summary for the gist.]`; + } + return content; +} + +async function readSummary(args: Record<string, unknown>, ctx: ToolContext): Promise<string> { + const filePath = validatePath(args.path as string, readFolders(ctx)); + const stat = fs.statSync(filePath); + if (stat.size > MAX_FILE_BYTES) { + return `File too large (${stat.size.toLocaleString()} bytes). Use search_files for specific content.`; + } + const content = fs.readFileSync(filePath, 'utf-8'); + if (!content.trim()) return '(empty file)'; + const summary = await summarizeText(content, args.focus ? String(args.focus) : undefined); + // Model ingests the short summary instead of the full file — both sizes in hand. + emitTruncationSaving(ctx, 'read_summary', path.basename(filePath), content.length, summary.length); + return `Summary of ${path.basename(filePath)} (${content.length.toLocaleString()} chars). ` + + `Call read_file for the exact content if you need to edit it.\n\n${summary}`; +} + +async function searchFiles(args: Record<string, unknown>, ctx: ToolContext): Promise<string> { + const query = (args.query as string | undefined) ?? ''; + const name = (args.name as string | undefined)?.trim(); + const pattern = (args.filePattern as string | undefined) ?? ''; + const rawPath = (args.path as string | undefined)?.trim(); + + if (!name && !query) { + return 'Provide "name" to locate a file by its name, or "query" to search file contents.'; + } + + // Resolve the search roots. With an explicit path, validate it lies inside the + // allowed folders (unchanged). WITHOUT one, sweep EVERY allowed folder in a + // single ripgrep call — a Work chat can grant many folders, and making the + // model search them one-by-one burns rounds and bloats context. ripgrep takes + // multiple path args, so all folders are covered in one process. + // (Incognito reads anywhere and has no fixed root set, so it still needs an + // explicit path — there's no sane "search the whole disk" default.) + let roots: string[]; + if (rawPath) { + try { + roots = [validatePath(rawPath, readFolders(ctx))]; + } catch (e) { + // Don't just bounce the model with a bare error — point it at the cheap + // fix (omit path → all allowed folders) so it doesn't waste a round + // guessing another path. + if (e instanceof SecurityError) { + const allowed = readFolders(ctx); + return `Security error: ${e.message}. Tip: omit "path" to search every allowed folder at once` + + `${allowed.length ? ` (${allowed.join(', ')})` : ''}.`; + } + throw e; + } + } else { + const allowed = readFolders(ctx); + if (allowed.length === 0) { + return 'Provide a "path" to search — no folder is granted to search across.'; + } + roots = allowed; + } + + const excludeGlobs = SEARCH_EXCLUDE_DIRS.flatMap((d) => ['-g', `!**/${d}/**`]); + + // NAME mode: locate files by filename across all roots and return paths only — + // no content, no per-line context. This is the right primitive for "where is + // file X" (a content grep for a filename matches every log/trace that mentions + // it, dumping huge irrelevant lines). `rg --files` lists files honouring our + // excludes; --iglob filters by name (case-insensitive, friendlier for users). + if (name) { + const glob = /[*?]/.test(name) ? name : `*${name}*`; + const fileArgs = ['--files', '--hidden', '--no-messages', ...excludeGlobs, '--iglob', glob, '--', ...roots]; + let out = ''; + try { + ({ stdout: out } = await execFileAsync(rgPath, fileArgs, { timeout: COMMAND_TIMEOUT, maxBuffer: 8 * 1024 * 1024 })); + } catch (err) { + const e = err as { killed?: boolean; signal?: string; stdout?: string }; + if (e.killed || e.signal === 'SIGTERM' || e.signal === 'SIGKILL') { + return 'Search did not finish in time (the tree is large). This is NOT a confirmation that the file is missing — search a more specific subfolder.'; + } + out = e.stdout ?? ''; + } + const found = out.trim().split('\n').filter(Boolean).slice(0, 40); + if (found.length === 0) return 'No files found with that name.'; + const more = out.trim().split('\n').filter(Boolean).length > 40 ? '\n…[more — narrow the name]' : ''; + return found.join('\n') + more; + } + + // CONTENT mode: find the files that contain `query`. ripgrep skips binary + // files (videos / photos / archives / office docs) and never follows symlinks + // for free; we pass our own exclude list as ignore globs so generated/ + // dependency trees are pruned with or without git. This keeps the search fast + // and — paired with the honest timeout below — truthful (an unscoped grep over + // a 40GB+ tree used to time out, misreported as "No matches found", i.e. a + // false "file doesn't exist"). + // --hidden searches dotfiles too (rg still always skips .git); -F = literal + // (plain substring); --max-filesize guards against slurping a giant file. + const rgArgs = ['--hidden', '--no-messages', '-l', '-F', '-m', '1', + '--max-filesize', `${MAX_FILE_BYTES}`, + ...(pattern ? ['-g', pattern] : []), + ...excludeGlobs, + '--', query, ...roots]; + + let stdout = ''; + try { + ({ stdout } = await execFileAsync(rgPath, rgArgs, { timeout: COMMAND_TIMEOUT, maxBuffer: 8 * 1024 * 1024 })); + } catch (err) { + const e = err as { killed?: boolean; signal?: string; code?: number; stdout?: string }; + // CRITICAL: a timeout/kill is NOT "no matches". Say so explicitly so the + // model narrows the search instead of telling the user the file is absent. + if (e.killed || e.signal === 'SIGTERM' || e.signal === 'SIGKILL') { + return ( + `Search did not finish in time (the tree is large` + + `${pattern ? '' : ' and no filePattern was given, so every file was scanned'}). ` + + `This is NOT a confirmation that the file is missing. Narrow it: search a more ` + + `specific subfolder, or pass filePattern (e.g. "*.sh").` + ); + } + // ripgrep exits 1 on no matches, 2 on partial read errors (matches on + // readable files may still be in stdout). Keep whatever it printed. + stdout = e.stdout ?? ''; + } + + const files = stdout.trim().split('\n').filter(Boolean).slice(0, 20); + if (files.length === 0) return 'No matches found.'; + + // Show matching lines for the first 5 files. Cap matches PER FILE (-m) so a + // file with thousands of hits can't dump itself into history (resent every + // round), and clamp the combined output as a backstop. + const results: string[] = []; + let trimmedLines = 0; // long match lines ripgrep clamped (markers it printed) + for (const file of files.slice(0, 5)) { + // --max-columns caps each match LINE's length: minified JS / JSON / log + // (.jsonl trace) files can have single lines tens of thousands of chars + // long, and dumping even a few of them blew the token budget. -preview keeps + // a short head of the line instead of omitting it outright. + const { stdout: ctx2 } = await execFileAsync( + rgPath, ['-n', '--no-messages', '-F', '-m', String(MAX_MATCH_LINES_PER_FILE), + '--max-columns', '200', '--max-columns-preview', '--', query, file], { timeout: 5000 }, + ).catch(() => ({ stdout: '' })); + trimmedLines += (ctx2.match(/\[\.\.\. omitted end of long line\]/g) || []).length; + const trimmed = ctx2.trim(); + const lineCount = trimmed ? trimmed.split('\n').length : 0; + const more = lineCount >= MAX_MATCH_LINES_PER_FILE ? `\n…[more matches — refine the query]` : ''; + results.push(`${file}:\n${trimmed}${more}`); + } + if (files.length > 5) results.push(`...and ${files.length - 5} more files`); + // Qualitative saving: ripgrep never tells us HOW MANY bytes it dropped from a + // long line, only that it did. So when several oversized lines were trimmed we + // post a quantity-free note (no fabricated %), gated so it's not noise. + if (ctx.onNote && trimmedLines >= 2) { + ctx.onNote({ kind: 'search', source: 'your files' }); + } + return clampMiddle(results.join('\n\n'), MAX_CMD_OUTPUT_CHARS); +} + +async function writeFile(args: Record<string, unknown>, ctx: ToolContext): Promise<string> { + if (ctx.readOnly) { + return 'write_file is disabled in Incognito mode. Incognito is read-only and never writes to disk — ' + + 'summarize or return the content in your reply instead.'; + } + const filePath = validatePath(args.path as string, ctx.allowedFolders); + const content = args.content as string; + + if (ctx.folderAccess && !isWriteAllowed(filePath, ctx.folderAccess)) { + return `Write denied: "${filePath}" is in a read-only folder. Ask the user to grant read & write access to this folder.`; + } + + const approved = await ctx.requestWriteApproval(filePath, content); + if (!approved) return `Write rejected by user: ${filePath}`; + + fs.mkdirSync(path.dirname(filePath), { recursive: true }); + fs.writeFileSync(filePath, content, 'utf-8'); + return `Written: ${filePath}`; +} + +const SHOW_IMAGE_MIME: Record<string, string> = { + '.png': 'image/png', '.jpg': 'image/jpeg', '.jpeg': 'image/jpeg', + '.gif': 'image/gif', '.webp': 'image/webp', '.svg': 'image/svg+xml', +}; +const SHOW_IMAGE_MAX_BYTES = 20 * 1024 * 1024; + +/** + * show_image: hand the host an image file (inside an allowed folder) to render + * inline in the chat. Path is validated against allowedFolders exactly like + * write_file — so the model must pass the HOST path it was given for the folder, + * not a container `/work/N` path. We only signal (via ctx.onImage); the host + * copies it into the chat's uploads and attaches it to the reply. + */ +function showImage(args: Record<string, unknown>, ctx: ToolContext): string { + const raw = args.path; + if (typeof raw !== 'string' || !raw.trim()) return 'show_image needs a "path" to an image file.'; + let filePath: string; + try { + filePath = validatePath(raw, ctx.allowedFolders); + } catch (e) { + if (e instanceof SecurityError) { + return `Security error: ${e.message}. Pass the image's path inside an allowed folder` + + `${ctx.allowedFolders.length ? ` (${ctx.allowedFolders.join(', ')})` : ''} — not a /work/N container path.`; + } + throw e; + } + const ext = path.extname(filePath).toLowerCase(); + const mime = SHOW_IMAGE_MIME[ext]; + if (!mime) return `show_image supports image files only (png/jpg/gif/webp/svg); got "${ext || 'no extension'}".`; + let st: fs.Stats; + try { st = fs.statSync(filePath); } catch { return `No such file: ${filePath}`; } + if (!st.isFile()) return `Not a file: ${filePath}`; + if (st.size === 0) return `That image file is empty: ${filePath}`; + if (st.size > SHOW_IMAGE_MAX_BYTES) { + return `Image too large to display (${st.size.toLocaleString()} bytes; max ${SHOW_IMAGE_MAX_BYTES.toLocaleString()}).`; + } + if (!ctx.onImage) return 'Showing images inline is not available in this mode.'; + ctx.onImage({ path: filePath, mime, fileName: path.basename(filePath), bytes: st.size }); + return `Displayed ${path.basename(filePath)} to the user in the chat.`; +} + +async function editFile(args: Record<string, unknown>, ctx: ToolContext): Promise<string> { + if (ctx.readOnly) { + return 'edit_file is disabled in Incognito mode (read-only). Return the change in your reply instead.'; + } + const filePath = validatePath(args.path as string, ctx.allowedFolders); + const oldStr = String(args.old_string ?? ''); + const newStr = String(args.new_string ?? ''); + if (!oldStr) return 'edit_file requires old_string — the exact text to replace.'; + + if (ctx.folderAccess && !isWriteAllowed(filePath, ctx.folderAccess)) { + return `Edit denied: "${filePath}" is in a read-only folder. Ask the user to grant read & write access.`; + } + + let content: string; + try { + content = fs.readFileSync(filePath, 'utf-8'); + } catch { + return `File not found (read it / create with write_file first): ${filePath}`; + } + + const first = content.indexOf(oldStr); + if (first === -1) { + return 'old_string not found. Read the file and copy the exact text to replace (including whitespace/indentation).'; + } + if (content.indexOf(oldStr, first + oldStr.length) !== -1) { + return 'old_string is not unique — it appears more than once. Include more surrounding context so it matches exactly one place.'; + } + + const next = content.slice(0, first) + newStr + content.slice(first + oldStr.length); + + // Reuse the write-approval flow; show the resulting full content so the UI + // diff/preview reflects what will land on disk. + const approved = await ctx.requestWriteApproval(filePath, next); + if (!approved) return `Edit rejected by user: ${filePath}`; + + fs.writeFileSync(filePath, next, 'utf-8'); + return `Edited: ${filePath}`; +} + +async function runCommand(args: Record<string, unknown>, ctx: ToolContext): Promise<string> { + // Validate cwd is inside an allowed folder up front (clear error before we + // hand off to the sandbox, and the container only mounts allowed folders). + const cwd = validatePath(args.cwd as string, ctx.allowedFolders); + const command = args.command as string; + + // No sandbox available → run_command is disabled. We never fall back to host + // bash, because that can't enforce per-folder read-only. File tools still + // work (write_file is path-checked on the host). + if (!ctx.runShell) { + return 'run_command is unavailable. Shell commands run in a Docker sandbox, which needs both ' + + '(1) Docker installed and running, and (2) at least one folder explicitly selected for this chat. ' + + 'Ask the user to start Docker and/or add a folder. Meanwhile read_file / search_files / write_file still work.'; + } + + // The sandbox mounts read-only folders as :ro, so the kernel — not us — + // rejects any write outside the read & write folders. Commands may freely + // read from read-only folders. + const out = await ctx.runShell(command, cwd); + // Compress noise (ANSI, progress bars, repeated lines) THEN cap verbose output + // (test runs, build logs) so the cap keeps real signal, not flooding history. + const delivered = clampMiddle(compressCommandOutput(out), MAX_CMD_OUTPUT_CHARS); + // The command's full output is already in hand; if we capped it, that's a real + // saving (the model would otherwise ingest all of it, every round). + emitTruncationSaving(ctx, 'run_command', 'command output', out.length, delivered.length); + return delivered; +} + +// ── web_fetch (read-only research) ──────────────────────────────────────────── + +/** Crude HTML → readable text. Good enough for research summaries, not parsing. */ +function htmlToText(html: string): string { + return html + .replace(/<script[\s\S]*?<\/script>/gi, ' ') + .replace(/<style[\s\S]*?<\/style>/gi, ' ') + .replace(/<!--[\s\S]*?-->/g, ' ') + .replace(/<\/(p|div|li|h[1-6]|tr|section|article|header|footer)>/gi, '\n') + .replace(/<br\s*\/?>/gi, '\n') + .replace(/<[^>]+>/g, ' ') + .replace(/ /g, ' ') + .replace(/&/g, '&') + .replace(/</g, '<') + .replace(/>/g, '>') + .replace(/"/g, '"') + .replace(/'/g, "'") + .replace(/[ \t]+/g, ' ') + .replace(/\n{3,}/g, '\n\n') + .trim(); +} + +// Search-query terms dropped before normalizing (boolean/scope operators + a few +// fillers) so two searches that differ only by operator/word-order collapse to +// the same repeat-signature. Kept tiny on purpose — over-collapsing distinct +// queries would wrongly trip the loop guard. +const SEARCH_STOPWORDS = new Set([ + 'or', 'and', 'the', 'a', 'an', 'of', 'to', 'for', 'in', 'on', 'vs', 'site', 'intitle', 'inurl', +]); + +/** + * Normalize a URL for within-turn fetch dedup AND the loop guard: lowercase the + * host, drop the `#fragment` (it never changes what a fetch returns), and strip + * a bare trailing slash. So `…/page`, `…/page/`, and `…/page#overview` collapse + * to one key — the model can't dodge the repeat-guard by tweaking the anchor. + */ +export function normalizeFetchUrl(raw: string): string { + try { + const u = new URL(String(raw).trim()); + u.hash = ''; + u.hostname = u.hostname.toLowerCase(); + let s = u.toString(); + if (s.endsWith('/') && u.pathname === '/' && !u.search) s = s.slice(0, -1); + return s; + } catch { + return String(raw).trim(); + } +} + +// GitHub repo/file *pages* are mostly UI chrome (the awesome-ai-agents repo page +// is ~1.4 MB of nav/file-tree/JS vs a ~220 KB raw README), and htmlToText keeps +// that noise then truncates at 20k — so the actual content gets buried/cut. +// Redirect a repo URL to its raw README and a /blob/ file URL to the raw file, so +// web_fetch pulls clean markdown/source from the first byte. Non-content GitHub +// routes (issues, pull, tree, releases, wiki, user profiles, …) pass through. +// Reddit links get routed to old.reddit.com (server-rendered); other untouched. +const GITHUB_NON_REPO = new Set([ + 'features', 'about', 'pricing', 'marketplace', 'sponsors', 'settings', 'notifications', + 'explore', 'topics', 'trending', 'collections', 'events', 'enterprise', 'team', 'login', + 'join', 'search', 'orgs', 'apps', 'security', 'readme', 'site', 'contact', 'customer-stories', + 'solutions', 'resources', 'git-guides', 'mobile', +]); + +export function canonicalizeFetchUrl(raw: string): string { + let u: URL; + try { u = new URL(String(raw).trim()); } catch { return String(raw).trim(); } + const host = u.hostname.toLowerCase(); + + // GitHub repo/file pages → raw markdown/source. + if (host === 'github.com' || host === 'www.github.com') { + const seg = u.pathname.split('/').filter(Boolean); + // /owner/repo/blob/<ref>/<path...> → raw file (any "view file" link) + if (seg.length >= 5 && seg[2] === 'blob') { + const [owner, repo, , ref, ...rest] = seg; + return `https://raw.githubusercontent.com/${owner}/${repo}/${ref}/${rest.join('/')}`; + } + // /owner/repo (exactly) → raw README on the default branch (HEAD resolves it) + if (seg.length === 2 && !GITHUB_NON_REPO.has(seg[0].toLowerCase())) { + const [owner, repo] = seg; + return `https://raw.githubusercontent.com/${owner}/${repo.replace(/\.git$/, '')}/HEAD/README.md`; + } + return raw; + } + + // Reddit: the JSON API and new-reddit are blocked / JS-only from many IPs + // (the .json endpoint 403s even with a browser UA; www.reddit returns an empty + // JS shell). old.reddit.com is server-rendered — the real post AND comments are + // in the HTML. Route every reddit host there (idempotent), dropping a trailing + // `.json` (it 403s) and the hash so the variants collapse to one fetch + cache. + if (host === 'reddit.com' || host.endsWith('.reddit.com')) { + u.hostname = 'old.reddit.com'; + // drop a trailing `.json` (403s) and a trailing slash so the www / old / + // .json / slash variants all collapse to one fetch + cache entry. + u.pathname = u.pathname.replace(/\.json$/, '').replace(/(.)\/$/, '$1'); + u.hash = ''; + return u.toString(); + } + + return raw; +} + +/** + * Normalize a search query into a repeat-signature for the loop guard: lowercase, + * strip punctuation/quotes/operators, drop stopwords, then sort the UNIQUE terms. + * Two searches that differ only by word order, quoting, or a repeated word map to + * the same key; genuinely different term sets stay distinct (so a real new search + * isn't blocked). + */ +export function normalizeSearchQuery(raw: string): string { + const terms = String(raw) + .toLowerCase() + .replace(/[^\p{L}\p{N}\s]/gu, ' ') + .split(/\s+/) + .filter((t) => t && !SEARCH_STOPWORDS.has(t)); + return Array.from(new Set(terms)).sort().join(' '); +} + +async function webFetch(args: Record<string, unknown>, ctx?: ToolContext): Promise<string> { + const inputUrl = String(args.url ?? '').trim(); + if (!/^https?:\/\/\S+$/i.test(inputUrl)) { + return 'Only absolute http(s) URLs are allowed.'; + } + // Redirect GitHub repo/file pages to their raw markdown/source before anything + // else, so the cache + fetch both work on the clean URL. + const url = canonicalizeFetchUrl(inputUrl); + const key = normalizeFetchUrl(url); + const cache = ctx?.fetchCache; + + // Within-turn cache: if we already pulled this page this turn, re-use the body + // (the model loves to re-fetch the same URL with a tweaked `focus`). We cache + // the extracted body, not the summary, so a new `focus` re-summarizes from + // memory with NO network round-trip. Misses go to the network as before; + // failures are NOT cached, so a transient error can still be retried. + let text = cache?.get(key); + const fromCache = text !== undefined; + let status = 200; + if (text === undefined) { + const ctrl = new AbortController(); + const timer = setTimeout(() => ctrl.abort(), WEB_FETCH_TIMEOUT); + try { + const res = await fetch(url, { + signal: ctrl.signal, + redirect: 'follow', + headers: { 'User-Agent': 'iClaw-Incognito/1.0', Accept: 'text/html,application/json;q=0.9,*/*;q=0.8' }, + }); + status = res.status; + const ct = res.headers.get('content-type') || ''; + const raw = await res.text(); + text = /html/i.test(ct) ? htmlToText(raw) : raw.trim(); + cache?.set(key, text); + } catch (err) { + const msg = err instanceof Error && err.name === 'AbortError' + ? `timed out after ${WEB_FETCH_TIMEOUT / 1000}s` + : err instanceof Error ? err.message : String(err); + return `Fetch failed (${url}): ${msg}`; + } finally { + clearTimeout(timer); + } + } + + const note = fromCache ? '(already fetched this turn — served from cache; no new request)\n\n' : ''; + + // Summarize by default (cheaper, smaller history); the model opts out with + // summarize:false when it needs the exact text (lists, numbers, code, quotes). + if (args.summarize !== false && text) { + const summary = await summarizeText(text, args.focus ? String(args.focus) : undefined); + return `${note}Summary of ${url}:\n\n${summary}`; + } + if (text.length > WEB_FETCH_MAX_CHARS) { + text = text.slice(0, WEB_FETCH_MAX_CHARS) + `\n\n[truncated at ${WEB_FETCH_MAX_CHARS} chars]`; + } + return `${note}HTTP ${status} — ${url}\n\n${text || '(empty response body)'}`; +} + +// ── cheap-model summarizer (read_summary, web_fetch summarize) ──────────────── + +/** + * Summarize text with a cheap model via OpenRouter. Faithful + dense; preserves + * exact names/numbers. Degrades gracefully to a truncation if there's no key or + * the call fails, so callers always get usable output. + */ +async function summarizeText(text: string, focus?: string): Promise<string> { + const key = process.env.ICLAW_OPENROUTER_API_KEY || ''; + if (!key || !text.trim()) return clampMiddle(text, MAX_CMD_OUTPUT_CHARS); + const base = (process.env.OPENROUTER_BASE_URL || 'https://openrouter.ai/api/v1').replace(/\/+$/, ''); + const input = text.length > SUMMARY_MAX_INPUT_CHARS ? text.slice(0, SUMMARY_MAX_INPUT_CHARS) : text; + const ctrl = new AbortController(); + const timer = setTimeout(() => ctrl.abort(), WEB_FETCH_TIMEOUT); + try { + const res = await fetch(`${base}/chat/completions`, { + method: 'POST', + signal: ctrl.signal, + headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${key}` }, + body: JSON.stringify({ + model: SUMMARY_MODEL, + temperature: 0.2, + messages: [ + { + role: 'system', + content: + 'You compress a document for another AI agent. Produce a dense, faithful summary: ' + + 'what it is, its structure, and the key facts. Preserve exact names, numbers, paths and ' + + 'identifiers. No preamble, no fluff. If the input was truncated, say so at the end.', + }, + { role: 'user', content: (focus ? `Focus on: ${focus}\n\n---\n` : '') + input }, + ], + }), + }); + if (!res.ok) throw new Error(`summary HTTP ${res.status}`); + const data = await res.json() as { choices?: { message?: { content?: string } }[] }; + const out = data.choices?.[0]?.message?.content; + return (typeof out === 'string' && out.trim()) + ? out.trim() + (text.length > SUMMARY_MAX_INPUT_CHARS ? '\n\n[note: input was truncated before summarizing]' : '') + : clampMiddle(text, MAX_CMD_OUTPUT_CHARS); + } catch { + return clampMiddle(text, MAX_CMD_OUTPUT_CHARS); + } finally { + clearTimeout(timer); + } +} + +// ── web_search (OpenRouter by default, DuckDuckGo as keyless fallback) ──────── + +interface SearchHit { title: string; url: string; snippet: string } + +function formatHits(query: string, hits: SearchHit[], provider: string): string { + if (hits.length === 0) return `No results for "${query}".`; + const lines = hits.map((h, i) => `${i + 1}. ${h.title}\n ${h.url}${h.snippet ? `\n ${h.snippet}` : ''}`); + return `Web search (${provider}) — "${query}":\n\n${lines.join('\n\n')}`; +} + +/** + * Zero-config default: use OpenRouter's built-in web search via the SAME key the + * runtime already uses for chat — no separate search account/key to set up. The + * `web` plugin runs a search and the response carries `url_citation` annotations + * (title/url/snippet). Costs a small per-result fee on the user's existing + * OpenRouter credits. + */ +async function openRouterSearch(query: string, count: number, signal: AbortSignal): Promise<SearchHit[]> { + const key = process.env.ICLAW_OPENROUTER_API_KEY || ''; + if (!key) throw new Error('no OpenRouter key'); + const base = (process.env.OPENROUTER_BASE_URL || 'https://openrouter.ai/api/v1').replace(/\/+$/, ''); + const model = process.env.ICLAW_SEARCH_MODEL || process.env.ICLAW_MODEL || 'google/gemini-2.5-flash'; + const res = await fetch(`${base}/chat/completions`, { + method: 'POST', + signal, + headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${key}` }, + body: JSON.stringify({ + model, + plugins: [{ id: 'web', max_results: count }], + messages: [{ role: 'user', content: `Find the most relevant, recent web results for: ${query}` }], + }), + }); + if (!res.ok) throw new Error(`OpenRouter HTTP ${res.status}`); + const data = await res.json() as { + choices?: { message?: { annotations?: { type?: string; url_citation?: { url?: string; title?: string; content?: string } }[] } }[]; + }; + const anns = data.choices?.[0]?.message?.annotations ?? []; + return anns + .filter((a) => a.type === 'url_citation' && a.url_citation?.url) + .map((a) => ({ + title: a.url_citation!.title || a.url_citation!.url!, + url: a.url_citation!.url!, + snippet: (a.url_citation!.content ?? '').replace(/\s+/g, ' ').trim().slice(0, 300), + })) + .slice(0, count); +} + +async function duckDuckGoSearch(query: string, count: number, signal: AbortSignal): Promise<SearchHit[]> { + const u = `https://html.duckduckgo.com/html/?q=${encodeURIComponent(query)}`; + const res = await fetch(u, { signal, headers: { 'User-Agent': 'Mozilla/5.0 iClaw-Incognito/1.0' } }); + const html = await res.text(); + const hits: SearchHit[] = []; + const re = /<a[^>]*class="result__a"[^>]*href="([^"]+)"[^>]*>([\s\S]*?)<\/a>/g; + let m: RegExpExecArray | null; + while ((m = re.exec(html)) && hits.length < count) { + let url = m[1]; + const uddg = /[?&]uddg=([^&]+)/.exec(url); // DDG wraps links in a redirect + if (uddg) url = decodeURIComponent(uddg[1]); + const title = m[2].replace(/<[^>]+>/g, '').replace(/&/g, '&').trim(); + if (url.startsWith('http')) hits.push({ title, url, snippet: '' }); + } + return hits; +} + +async function webSearch(args: Record<string, unknown>): Promise<string> { + const query = String(args.query ?? '').trim(); + if (!query) return 'web_search requires a query.'; + const count = Math.min(10, Math.max(1, Number(args.count) || 6)); + const ctrl = new AbortController(); + const timer = setTimeout(() => ctrl.abort(), WEB_FETCH_TIMEOUT); + try { + // 1) OpenRouter web search — the zero-config default (reuses the chat key). + if (process.env.ICLAW_OPENROUTER_API_KEY) { + try { + const hits = await openRouterSearch(query, count, ctrl.signal); + if (hits.length) return formatHits(query, hits, 'OpenRouter'); + } catch { /* fall through */ } + } + // 2) Keyless last resort. + return formatHits(query, await duckDuckGoSearch(query, count, ctrl.signal), 'DuckDuckGo'); + } catch (err) { + const msg = err instanceof Error && err.name === 'AbortError' + ? `timed out after ${WEB_FETCH_TIMEOUT / 1000}s` + : err instanceof Error ? err.message : String(err); + return `Search failed for "${query}": ${msg}.`; + } finally { + clearTimeout(timer); + } +} + +// ── analyze_link (sandboxed link → content extraction) ─────────────────────── +// +// Engine: yt-dlp (1000+ site extractors → easy to grow past YouTube). Shipped as +// a self-contained binary self-installed into /workspace/.tools/bin on first use +// — deliberately NOT baked into the image (keeps it lean; the binary persists in +// the workspace across container restarts, like uv). All network egress happens +// inside the sandbox via the injected runInSandbox callback, so it honours the +// same container network gate as run_command. + +const ANALYZE_LINK_FULL_MAX_CHARS = Number(process.env.ICLAW_ANALYZE_LINK_MAX) || 16_000; +// Subtitle language priority (yt-dlp glob syntax). Tunable; first available wins. +const ANALYZE_LINK_SUB_LANGS = process.env.ICLAW_ANALYZE_LINK_LANGS || 'en.*,uk.*'; +// Only surface a "saved N%" note when the win is real: the transcript is at +// least this long AND we trimmed at least this fraction. Avoids noise on clips. +const ANALYZE_LINK_SAVINGS_MIN_CHARS = 2_000; +const ANALYZE_LINK_SAVINGS_MIN_PCT = 25; + +/** Single-quote a string for safe embedding in a bash command. */ +function shQuote(s: string): string { + return `'${s.replace(/'/g, `'\\''`)}'`; +} + +/** Pull the 11-ish-char video id out of any YouTube URL shape, else null. */ +function extractYouTubeId(url: string): string | null { + try { + const u = new URL(url); + const host = u.hostname.replace(/^www\./, ''); + if (host === 'youtu.be') return u.pathname.slice(1).split('/')[0] || null; + if (host.endsWith('youtube.com') || host.endsWith('youtube-nocookie.com')) { + if (u.pathname === '/watch') return u.searchParams.get('v'); + const m = u.pathname.match(/^\/(?:shorts|embed|live|v)\/([^/?#]+)/); + if (m) return m[1]; + } + return null; + } catch { + return null; + } +} + +/** + * WebVTT → dense plain text. Drops the header, cue timestamps/ids and inline + * styling tags, and de-duplicates consecutive identical lines (YouTube + * auto-captions emit each segment twice — once styled, once plain). + */ +function cleanVtt(vtt: string): string { + const out: string[] = []; + let last = ''; + for (const raw of vtt.split(/\r?\n/)) { + if (!raw.trim()) continue; + if (/^WEBVTT/.test(raw)) continue; + if (/^(Kind|Language):/i.test(raw)) continue; + if (raw.includes('-->')) continue; // timestamp cue line + if (/^\d+$/.test(raw.trim())) continue; // numeric cue id + const line = raw + .replace(/<[^>]+>/g, '') // inline <00:00.000> / <c> tags + .replace(/ /g, ' ') + .replace(/&/g, '&') + .replace(/</g, '<') + .replace(/>/g, '>') + .replace(/'/g, "'") + .trim(); + if (!line || line === last) continue; + out.push(line); + last = line; + } + return out.join(' ').replace(/\s{2,}/g, ' ').trim(); +} + +/** + * Bash run inside the sandbox: ensure yt-dlp, fetch subtitles (+ info json) for + * the video without downloading it, then print a marker-delimited payload the + * host parses. `set -e`-safe: every fallible step is guarded so we always exit 0 + * with a parseable result (success, no-subs, or install error). + */ +function buildYouTubeSubsCommand(videoId: string, url: string): string { + const out = `/workspace/.cache/links/${videoId}`; // videoId is [\w-]-validated + return [ + 'set -e', + 'BIN=/workspace/.tools/bin', + 'mkdir -p "$BIN" /workspace/.cache/links', + 'export PATH="$BIN:$PATH"', + // Install yt-dlp if missing OR if a cached binary can\'t actually run (e.g. a + // wrong-arch download from a previous version) — self-heals the stale binary. + 'if ! (command -v yt-dlp >/dev/null 2>&1 && yt-dlp --version >/dev/null 2>&1); then', + // Pick the binary matching the container arch. yt-dlp_linux is x86_64-only; + // arm64 containers (e.g. Docker on Apple Silicon) need the aarch64 build. + ' case "$(uname -m)" in aarch64|arm64) YDL=yt-dlp_linux_aarch64 ;; armv7l|armhf) YDL=yt-dlp_linux_armv7l ;; *) YDL=yt-dlp_linux ;; esac', + ' curl -fsSL "https://github.com/yt-dlp/yt-dlp/releases/latest/download/$YDL" -o "$BIN/yt-dlp" 2>/dev/null && chmod +x "$BIN/yt-dlp" || { echo "__ERR__ could not download yt-dlp (network blocked?)"; exit 0; }', + ' hash -r 2>/dev/null || true', + ' yt-dlp --version >/dev/null 2>&1 || { echo "__ERR__ yt-dlp not runnable after install (arch=$(uname -m))"; exit 0; }', + 'fi', + `OUT=${shQuote(out)}`, + 'rm -f "$OUT"*.vtt "$OUT".info.json 2>/dev/null || true', + `ERR=$(yt-dlp --quiet --no-warnings --skip-download --write-subs --write-auto-subs --write-info-json --sub-langs ${shQuote(ANALYZE_LINK_SUB_LANGS)} --sub-format vtt -o "$OUT.%(ext)s" ${shQuote(url)} 2>&1) || true`, + 'f=$(ls "$OUT"*.vtt 2>/dev/null | head -1)', + 'if [ -n "$f" ]; then', + ' echo "__META__"', + ' jq -r \'[.title, .uploader, .duration_string] | map(select(. != null)) | join(" | ")\' "$OUT.info.json" 2>/dev/null || true', + ' echo "__VTT__"', + ' cat "$f"', + 'else', + ' echo "__NOSUBS__"', + ' echo "$ERR" | tail -n 5', + 'fi', + ].join('\n'); +} + +/** + * analyze_link implementation. `runInSandbox` runs a bash command inside the + * session's container; `networkEnabled` mirrors the chat's network gate. + */ +export async function analyzeLink( + args: Record<string, unknown>, + deps: { + runInSandbox: (command: string) => Promise<string>; + networkEnabled: boolean; + onNote?: (note: SavingsNote) => void; + }, +): Promise<string> { + const url = String(args.url ?? '').trim(); + if (!/^https?:\/\/\S+$/i.test(url)) return 'analyze_link needs an absolute http(s) URL.'; + const mode = args.mode === 'full' ? 'full' : 'summary'; + const purpose = args.purpose ? String(args.purpose) : undefined; + + if (!deps.networkEnabled) { + return 'analyze_link needs network, which is currently OFF for this chat. Ask the user to enable network, then retry.'; + } + + const videoId = extractYouTubeId(url); + if (!videoId || !/^[\w-]{6,}$/.test(videoId)) { + return 'analyze_link currently supports only YouTube video links. For web pages or articles, use web_fetch instead.'; + } + + let raw: string; + try { + raw = await deps.runInSandbox(buildYouTubeSubsCommand(videoId, url)); + } catch (err) { + return `analyze_link failed to run in the sandbox: ${err instanceof Error ? err.message : String(err)}`; + } + + if (raw.includes('__ERR__')) { + return `analyze_link: ${raw.split('__ERR__')[1].trim().slice(0, 300) || 'sandbox error'}`; + } + if (raw.includes('__NOSUBS__') || (!raw.includes('__VTT__'))) { + const detail = (raw.split('__NOSUBS__')[1] ?? raw).trim().slice(0, 250); + const blocked = /sign in|confirm.*bot|429|HTTP Error 403|blocked|not a bot/i.test(detail); + return blocked + ? `analyze_link: YouTube blocked the request from the sandbox IP, or the video is restricted. ${detail}` + : `analyze_link: no subtitles available in ${ANALYZE_LINK_SUB_LANGS} (video may have none, or only other languages). ${detail}`; + } + + const meta = raw.includes('__META__') ? raw.split('__META__')[1].split('__VTT__')[0].trim() : ''; + const transcript = cleanVtt(raw.split('__VTT__')[1] ?? ''); + if (!transcript) return 'analyze_link: subtitles were fetched but empty after cleanup.'; + const header = meta ? `Video: ${meta}\n\n` : ''; + + if (mode === 'full') { + const body = + transcript.length > ANALYZE_LINK_FULL_MAX_CHARS + ? transcript.slice(0, ANALYZE_LINK_FULL_MAX_CHARS) + + `\n\n…[truncated ${(transcript.length - ANALYZE_LINK_FULL_MAX_CHARS).toLocaleString()} chars — re-call with mode:"summary" and a purpose]` + : transcript; + return `${header}Transcript (${transcript.length.toLocaleString()} chars):\n\n${body}`; + } + + const summary = await summarizeText(transcript, purpose); + + // Cost win: the main model ingests this short summary instead of the whole + // transcript (which it would also re-read every subsequent round). Surface the + // saving as a chat note. Char-count is a transparent token proxy; gate on a + // real win so we don't post noise for tiny clips. + if (deps.onNote && transcript.length >= ANALYZE_LINK_SAVINGS_MIN_CHARS && summary.length < transcript.length) { + const savedPct = Math.round((1 - summary.length / transcript.length) * 100); + if (savedPct >= ANALYZE_LINK_SAVINGS_MIN_PCT) { + deps.onNote({ + kind: 'analyze_link', + source: 'video transcript', + savedPct, + fullChars: transcript.length, + deliveredChars: summary.length, + }); + } + } + + return `${header}Transcript summary${purpose ? ` (focus: ${purpose})` : ''}:\n\n${summary}`; +} diff --git a/packages/iclaw-runtime/src/colima.ts b/packages/iclaw-runtime/src/colima.ts new file mode 100644 index 0000000..c09f591 --- /dev/null +++ b/packages/iclaw-runtime/src/colima.ts @@ -0,0 +1,103 @@ +/** + * Colima container engine (macOS) — runtime copy. + * + * On macOS, iClaw runs its Docker workloads on Colima — a free, lightweight, + * CLI-driven Docker engine in a small Linux VM — instead of Docker Desktop. + * A non-technical user never has to think about it: iClaw installs Colima, + * starts it on demand the first moment a task needs a container, and stops it + * again when idle. + * + * Colima can arrive two ways (see scripts/install-docker.sh): via Homebrew when + * it's present, or — on a clean Mac with no Homebrew — by downloading pinned + * colima/lima/docker binaries into ~/.iclaw/engine. Either way iClaw owns a + * DEDICATED Colima profile ("iclaw") — its own VM — so it never disturbs the + * user's other containers/profiles, and it pins its own `docker` CLI calls to + * that profile's context via DOCKER_CONTEXT, leaving the GLOBAL context alone. + * + * Off macOS this module is dormant: callers gate every use on `isMac` (Linux: + * user-managed dockerd; Windows: Docker Desktop). + */ +import { execFile } from 'node:child_process'; +import { existsSync } from 'node:fs'; +import { homedir, platform as osPlatform } from 'node:os'; +import { delimiter, join } from 'node:path'; +import { promisify } from 'node:util'; + +const execFileAsync = promisify(execFile); + +/** True on macOS, where iClaw uses Colima as its container engine. */ +export const isMac = osPlatform() === 'darwin'; + +/** Dedicated Colima profile iClaw owns (its own VM — never the user's other work). */ +export const COLIMA_PROFILE = 'iclaw'; +/** Docker context Colima creates for that profile (`colima start iclaw`). */ +export const COLIMA_CONTEXT = `colima-${COLIMA_PROFILE}`; + +/** Where the no-brew installer drops the downloaded engine binaries (colima, limactl, docker). */ +export const ENGINE_BIN = join( + process.env.ICLAW_ENGINE_DIR || join(homedir(), '.iclaw', 'engine'), + 'bin', +); + +/** Generous ceiling for a first `colima start` (downloads the guest image + boots). */ +const START_TIMEOUT = 300_000; +const PROBE_TIMEOUT = 8_000; +const STOP_TIMEOUT = 60_000; + +/** Add a dir to PATH (front = higher priority) if it exists and isn't already there. */ +function addToPath(dir: string, front: boolean): void { + if (!existsSync(dir)) return; + const parts = (process.env.PATH ?? '').split(delimiter).filter(Boolean); + if (parts.includes(dir)) return; + process.env.PATH = front + ? `${dir}${delimiter}${parts.join(delimiter)}` + : `${parts.join(delimiter)}${delimiter}${dir}`; +} + +/** + * Set up this process's (and its children's) environment for iClaw's Colima + * engine on macOS: + * - ensure Homebrew's bin dirs are on PATH (a GUI-launched .app starts with a + * minimal PATH that omits them, so a brew-installed colima/docker wouldn't be + * found otherwise), and + * - put our own downloaded engine bin (~/.iclaw/engine/bin) FIRST, so a no-brew + * install wins, and + * - pin every `docker` call to iClaw's Colima context, leaving the user's GLOBAL + * docker context untouched. + * Idempotent; honours an explicit DOCKER_CONTEXT already in the environment. + * No-op off macOS. Safe to call from every process entry point — and again right + * after an install, to pick up freshly downloaded binaries. + */ +export function ensureColimaEnv(): void { + if (!isMac) return; + addToPath('/opt/homebrew/bin', false); + addToPath('/usr/local/bin', false); + addToPath(ENGINE_BIN, true); // our pinned binaries take priority + if (!process.env.DOCKER_CONTEXT) { + process.env.DOCKER_CONTEXT = COLIMA_CONTEXT; + } +} + +/** Colima CLI present on PATH (the engine is installed). */ +export async function colimaInstalled(): Promise<boolean> { + try { + await execFileAsync('colima', ['version'], { timeout: PROBE_TIMEOUT }); + return true; + } catch { + return false; + } +} + +/** + * Start iClaw's Colima VM (creating it on first run). Blocks until the daemon is + * ready; idempotent — "already running" is a success. First run can take ~1–2 min + * (guest image download); subsequent starts ~20s. + */ +export async function colimaStart(): Promise<void> { + await execFileAsync('colima', ['start', COLIMA_PROFILE], { timeout: START_TIMEOUT }); +} + +/** Stop iClaw's Colima VM (only ours — never the user's other profiles). */ +export async function colimaStop(): Promise<void> { + await execFileAsync('colima', ['stop', COLIMA_PROFILE], { timeout: STOP_TIMEOUT }); +} diff --git a/packages/iclaw-runtime/src/docker-lifecycle.ts b/packages/iclaw-runtime/src/docker-lifecycle.ts new file mode 100644 index 0000000..7c04102 --- /dev/null +++ b/packages/iclaw-runtime/src/docker-lifecycle.ts @@ -0,0 +1,242 @@ +/** + * On-demand Docker lifecycle. + * + * Docker is only needed when a task actually runs a sandboxed command (Work + * run_command / analyze_link, or any Safe-Mode turn). So instead of nagging the + * user up front, we start Docker ourselves the first moment a task needs it, and + * — only if WE started it — stop it again once it's been idle and there are no + * containers left running (so we never kill the user's own Docker work). + * + * ensureDockerForTask() → true if Docker is usable now (started it if needed) + * startDockerIdleReaper() → background loop that stops a self-started, idle, + * container-free daemon after ICLAW_DOCKER_IDLE_STOP_MS + * + * Linux is start-only-by-the-user (the daemon needs sudo), so there we never + * auto-start or auto-stop. + */ +import { execFile } from 'node:child_process'; +import { existsSync, mkdirSync, rmSync, statSync, utimesSync, writeFileSync } from 'node:fs'; +import { homedir, platform as osPlatform } from 'node:os'; +import { join } from 'node:path'; +import { promisify } from 'node:util'; + +import { colimaInstalled, colimaStart, colimaStop, ensureColimaEnv, isMac } from './colima.js'; +import { log } from './log.js'; + +const execFileAsync = promisify(execFile); + +// macOS: put colima/docker on PATH and pin every `docker` call in this process to +// iClaw's Colima VM (not the user's global context / Docker Desktop). No-op off macOS. +ensureColimaEnv(); + +const PROBE_TIMEOUT = 8_000; +const START_POLL_ATTEMPTS = 30; +const START_POLL_INTERVAL = 2_000; +/** Idle window before stopping a daemon we started (default 15 min). */ +const IDLE_STOP_MS = Number(process.env.ICLAW_DOCKER_IDLE_STOP_MS) || 15 * 60_000; +const REAPER_INTERVAL = 60_000; + +/** + * Durable "iClaw started Docker" marker. Its EXISTENCE means some iClaw instance + * started the daemon (so we may stop it); its MTIME is the last time a task used + * Docker (the idle-stop countdown). On disk — NOT in memory — so the ownership + * survives a runtime restart/crash: a fresh process inherits it and its reaper + * still stops the daemon a previous process started. Cleared when we stop Docker + * (or could be removed by the user). Single file, shared across installs; the + * "no running containers" guard is what actually keeps it safe. + */ +const OWN_MARKER = process.env.ICLAW_DOCKER_OWN_MARKER || join(homedir(), '.iclaw', '.docker-owned'); + +let ensureInFlight: Promise<boolean> | null = null; + +function sleep(ms: number): Promise<void> { + return new Promise((r) => setTimeout(r, ms)); +} + +/** Claim Docker ownership durably (we started it). */ +function markOwned(): void { + try { + mkdirSync(join(OWN_MARKER, '..'), { recursive: true }); + writeFileSync(OWN_MARKER, ''); + } catch { + /* best-effort — without the marker we just won't auto-stop */ + } +} + +/** Refresh last-use (marker mtime) — only if we own Docker. */ +function touchOwned(): void { + if (!existsSync(OWN_MARKER)) return; + try { + const now = new Date(); + utimesSync(OWN_MARKER, now, now); + } catch { + /* ignore */ + } +} + +/** Last-use time (ms) if iClaw owns the daemon, else null. */ +function ownedSince(): number | null { + try { + return statSync(OWN_MARKER).mtimeMs; + } catch { + return null; + } +} + +/** Release ownership (after we stop Docker). */ +function clearOwned(): void { + try { + rmSync(OWN_MARKER, { force: true }); + } catch { + /* ignore */ + } +} + +async function reachable(): Promise<boolean> { + try { + await execFileAsync('docker', ['info'], { timeout: PROBE_TIMEOUT }); + return true; + } catch { + return false; + } +} + +async function installed(): Promise<boolean> { + if (isMac) return colimaInstalled(); + try { + await execFileAsync('docker', ['--version'], { timeout: PROBE_TIMEOUT }); + return true; + } catch { + return false; + } +} + +async function launchDaemon(): Promise<void> { + const plat = osPlatform(); + if (plat === 'darwin') { + // macOS engine is Colima, not Docker Desktop: start iClaw's own VM. Blocks + // until the daemon is ready (idempotent if already running). + await colimaStart(); + } else if (plat === 'win32') { + await execFileAsync('cmd', ['/c', 'start', '', 'Docker Desktop'], { timeout: PROBE_TIMEOUT }); + } + // Linux: starting dockerd needs sudo — leave it to the user. +} + +/** + * Ensure Docker is usable for a task RIGHT NOW. If it's down but installed, start + * it ourselves and wait (up to ~60s). Returns false when Docker is missing or + * couldn't be started (caller then disables run_command / fails the Safe turn). + * Single-flight so concurrent tools don't each try to launch the daemon. + */ +export function ensureDockerForTask(): Promise<boolean> { + if (!ensureInFlight) { + ensureInFlight = (async () => { + if (await reachable()) { + // Already up. Refresh the idle countdown only if WE own it (a prior + // iClaw run started it) — never claim a daemon the user started. + touchOwned(); + return true; + } + if (!(await installed()) || osPlatform() === 'linux') return false; + log.info('Docker is down but a task needs it — starting it'); + try { + await launchDaemon(); + } catch { + return false; // Docker.app not found / launch failed + } + for (let i = 0; i < START_POLL_ATTEMPTS; i++) { + await sleep(START_POLL_INTERVAL); + if (await reachable()) { + markOwned(); // durable: survives a runtime restart + log.info('Docker started by iClaw (will auto-stop when idle + empty)'); + return true; + } + } + return false; + })().finally(() => { + ensureInFlight = null; + }); + } + return ensureInFlight; +} + +/** Reset the idle-stop countdown — call whenever a container is used. */ +export function markDockerUse(): void { + touchOwned(); +} + +/** Number of running containers, or -1 if the probe failed. */ +async function runningContainerCount(): Promise<number> { + try { + const { stdout } = await execFileAsync('docker', ['ps', '-q'], { timeout: PROBE_TIMEOUT }); + return stdout.split('\n').filter((s) => s.trim()).length; + } catch { + return -1; + } +} + +async function quitDocker(): Promise<void> { + const plat = osPlatform(); + try { + if (plat === 'darwin') { + // Stop only iClaw's own Colima VM — never the user's other profiles/engines. + await colimaStop(); + } else if (plat === 'win32') { + await execFileAsync('taskkill', ['/IM', 'Docker Desktop.exe', '/F'], { timeout: 15_000 }); + } + } catch (err) { + log.warn('Stopping Docker failed', { error: err instanceof Error ? err.message : String(err) }); + } +} + +/** + * Background loop: stop Docker once it's been idle past IDLE_STOP_MS AND has no + * running containers AND iClaw owns it (the durable marker — set when WE started + * it, surviving restarts). The container check is the key safety net: if the + * user (or one of our warm sandboxes) still has a container up, we leave the + * daemon alone. + */ +export function startDockerIdleReaper(): void { + setInterval(async () => { + const since = ownedSince(); + if (since == null) return; // not started by iClaw → never touch + if (Date.now() - since < IDLE_STOP_MS) return; + const n = await runningContainerCount(); + if (n !== 0) return; // containers still running (or probe failed) → don't touch + log.info('Docker idle + no containers — stopping the daemon iClaw started'); + await quitDocker(); + clearOwned(); + }, REAPER_INTERVAL).unref(); +} + +let shutdownStopDone = false; + +/** + * On graceful shutdown, stop a Docker daemon WE started if it's container-free — + * so quitting iClaw doesn't leave our daemon running. Skipped (fast exit, no + * churn) when we don't own it, or when any container is up (a warm sandbox from + * a task that just ran, or the user's own work): in that case the durable marker + * is left behind so the next runtime can stop it later. Hard-capped so a hung + * Docker can't block the exit. + */ +export async function stopDockerOnShutdown(maxMs = 8_000): Promise<void> { + if (shutdownStopDone) return; + shutdownStopDone = true; + if (ownedSince() == null) return; // not ours → leave it + await Promise.race([ + (async () => { + const n = await runningContainerCount(); + if (n !== 0) return; // containers up → don't kill work; marker persists + log.info('Shutting down — stopping the idle Docker iClaw started'); + await quitDocker(); + clearOwned(); + })(), + sleep(maxMs), + ]); +} + +/** Test/inspection helper. */ +export function _state(): { owned: boolean; ownedSince: number | null } { + return { owned: ownedSince() != null, ownedSince: ownedSince() }; +} diff --git a/packages/iclaw-runtime/src/index.ts b/packages/iclaw-runtime/src/index.ts new file mode 100644 index 0000000..43ab917 --- /dev/null +++ b/packages/iclaw-runtime/src/index.ts @@ -0,0 +1,236 @@ +/** + * iClaw Runtime — HTTP server for the Work / Safe work / Incognito modes. + * + * Model-agnostic agent loop runs HERE on the host (via OpenRouter); tool/shell + * execution is isolated in a per-turn Docker sandbox (secure-runner.ts / + * work-container.ts). No SQLite — sessions live in memory + the secure-mode + * workspace dir under ~/.iclaw/secure. + * + * API: + * POST /sessions → { sessionId } + * POST /sessions/:id/messages → 202 + * POST /sessions/:id/abort → 200 (stop in-flight turn, keep session) + * GET /sessions/:id/events → SSE stream + * DELETE /sessions/:id → 200 + * GET /health → 200 + */ +import http from 'node:http'; + +import { ensureColimaEnv } from './colima.js'; +import { createSession, getSession, deleteSession, abortSession, attachSseClient, detachSseClient, sendMessage, getSessionInfo, exportSessionWorkspace, applySessionChanges, sweepExpiredSessions, startContainerReaper, loadPersistedSessions, type RuntimeAttachment } from './sessions.js'; +import { killOrphanContainers } from './secure-runner.js'; +import { startDockerIdleReaper, stopDockerOnShutdown } from './docker-lifecycle.js'; +import { killOrphanWorkContainers } from './work-container.js'; + +// macOS: put colima/docker on PATH and route every `docker` call in this process +// (the container sandboxes in secure-runner.ts / work-container.ts included) to +// iClaw's Colima VM. +ensureColimaEnv(); + +const PORT = parseInt(process.env.ICLAW_RUNTIME_PORT || '7430', 10); +const SECRET = process.env.ICLAW_RUNTIME_SECRET || ''; +const API_KEY = process.env.ICLAW_OPENROUTER_API_KEY || ''; +// Default agent model. Override per-install via ICLAW_MODEL in .env. +const DEFAULT_MODEL = process.env.ICLAW_MODEL || 'deepseek/deepseek-v4-flash'; + +function authOk(req: http.IncomingMessage): boolean { + if (!SECRET) { + const addr = req.socket.remoteAddress ?? ''; + return addr === '127.0.0.1' || addr === '::1' || addr === '::ffff:127.0.0.1'; + } + return req.headers['x-iclaw-token'] === SECRET; +} + +function json(res: http.ServerResponse, status: number, body: unknown): void { + const payload = JSON.stringify(body); + res.writeHead(status, { 'Content-Type': 'application/json' }); + res.end(payload); +} + +function readBody(req: http.IncomingMessage): Promise<unknown> { + return new Promise((resolve, reject) => { + let raw = ''; + req.on('data', (c) => { raw += c; }); + req.on('end', () => { + try { resolve(JSON.parse(raw || '{}')); } + catch { resolve({}); } + }); + req.on('error', reject); + }); +} + +const server = http.createServer(async (req, res) => { + if (!authOk(req)) return json(res, 401, { error: 'unauthorized' }); + + const url = new URL(req.url ?? '/', `http://localhost`); + const parts = url.pathname.split('/').filter(Boolean); + + // GET /health + if (req.method === 'GET' && parts[0] === 'health') { + return json(res, 200, { ok: true }); + } + + // GET /sessions/:id/info + if (req.method === 'GET' && parts[0] === 'sessions' && parts[2] === 'info') { + const info = getSessionInfo(parts[1]); + if (!info) return json(res, 404, { error: 'session not found' }); + return json(res, 200, info); + } + + // POST /sessions + if (req.method === 'POST' && parts[0] === 'sessions' && parts.length === 1) { + const body = await readBody(req) as { allowedFolders?: string[]; folderAccess?: { path: string; readonly: boolean }[]; copyFolders?: string[]; model?: string; secure?: boolean; incognito?: boolean; systemPrompt?: string; key?: string; history?: { role: string; content: string }[] }; + // folderAccess (when present) is the source of truth for per-folder read/ + // write; derive allowedFolders paths from it so the two never drift. + const folderAccess = Array.isArray(body.folderAccess) + ? body.folderAccess + .filter((f) => f && typeof f.path === 'string' && f.path) + .map((f) => ({ path: f.path, readonly: f.readonly !== false })) + : undefined; + const allowedFolders = folderAccess + ? folderAccess.map((f) => f.path) + : (body.allowedFolders ?? []); + const copyFolders = Array.isArray(body.copyFolders) + ? body.copyFolders.filter((p) => typeof p === 'string' && p) + : undefined; + const sessionId = createSession({ + allowedFolders, + folderAccess, + copyFolders, + model: body.model ?? DEFAULT_MODEL, + apiKey: API_KEY, + secure: body.secure ?? false, + incognito: body.incognito ?? false, + systemPrompt: body.systemPrompt, + key: body.key, + history: body.history, + }); + return json(res, 201, { sessionId }); + } + + // POST /sessions/:id/messages + if (req.method === 'POST' && parts[0] === 'sessions' && parts[2] === 'messages') { + const sessionId = parts[1]; + if (!getSession(sessionId)) return json(res, 404, { error: 'session not found' }); + const body = await readBody(req) as { content?: string; networkEnabled?: boolean; ttlDays?: number; attachments?: RuntimeAttachment[]; copyFolders?: string[] }; + if (!body.content?.trim()) return json(res, 400, { error: 'content required' }); + const attachments = Array.isArray(body.attachments) + ? body.attachments.filter( + (a): a is RuntimeAttachment => + !!a && typeof a.path === 'string' && !!a.path && + typeof a.mimeType === 'string' && typeof a.fileName === 'string', + ) + : undefined; + const copyFolders = Array.isArray(body.copyFolders) + ? body.copyFolders.filter((p) => typeof p === 'string' && p) + : undefined; + sendMessage(sessionId, body.content, body.networkEnabled, body.ttlDays, attachments, copyFolders).catch(console.error); + return json(res, 202, { queued: true }); + } + + // POST /sessions/:id/abort — stop the in-flight turn (keeps the session). + if (req.method === 'POST' && parts[0] === 'sessions' && parts[2] === 'abort') { + const sessionId = parts[1]; + if (!getSession(sessionId)) return json(res, 404, { error: 'session not found' }); + return json(res, 200, { aborted: abortSession(sessionId) }); + } + + // GET /sessions/:id/events (SSE) + if (req.method === 'GET' && parts[0] === 'sessions' && parts[2] === 'events') { + const sessionId = parts[1]; + if (!getSession(sessionId)) return json(res, 404, { error: 'session not found' }); + res.writeHead(200, { + 'Content-Type': 'text/event-stream', + 'Cache-Control': 'no-cache', + 'Connection': 'keep-alive', + 'X-Accel-Buffering': 'no', + }); + res.write(': connected\n\n'); + attachSseClient(sessionId, res); + req.on('close', () => detachSseClient(sessionId)); + return; + } + + // POST /sessions/:id/export — copy the Safe sandbox to a host folder. + if (req.method === 'POST' && parts[0] === 'sessions' && parts[2] === 'export') { + if (!getSession(parts[1])) return json(res, 404, { error: 'session not found' }); + const body = await readBody(req) as { destDir?: string }; + const result = exportSessionWorkspace(parts[1], typeof body.destDir === 'string' ? body.destDir : undefined); + if (!result) return json(res, 400, { error: 'not a Safe session' }); + return json(res, 200, result); + } + + // POST /sessions/:id/apply — copy the sandbox's changes back to the originals. + if (req.method === 'POST' && parts[0] === 'sessions' && parts[2] === 'apply') { + if (!getSession(parts[1])) return json(res, 404, { error: 'session not found' }); + const results = applySessionChanges(parts[1]); + if (!results) return json(res, 400, { error: 'not a Safe session' }); + return json(res, 200, { results }); + } + + // DELETE /sessions/:id + if (req.method === 'DELETE' && parts[0] === 'sessions' && parts.length === 2) { + deleteSession(parts[1]); + return json(res, 200, { stopped: true }); + } + + json(res, 404, { error: 'not found' }); +}); + +server.on('error', (err: NodeJS.ErrnoException) => { + if (err.code === 'EADDRINUSE') { + console.error(`[iclaw-runtime] port ${PORT} already in use — waiting 3s and retrying`); + setTimeout(() => server.listen(PORT, '127.0.0.1'), 3000); + } else { + console.error('[iclaw-runtime] server error', err); + process.exit(1); + } +}); + +server.listen(PORT, '127.0.0.1', () => { + console.error(`[iclaw-runtime] listening on port ${PORT}, model=${DEFAULT_MODEL}`); +}); + +// Periodic cleanup of expired sessions (every hour) +setInterval(() => { + const removed = sweepExpiredSessions(); + if (removed > 0) console.error(`[iclaw-runtime] swept ${removed} expired session(s)`); +}, 3600_000).unref(); + +// Reap idle Secure-Mode sandbox containers (keeps RAM in check across chats). +startContainerReaper(30_000); + +// Stop a Docker daemon WE started once it's idle and container-free (never +// touches a daemon the user started, nor one with running containers). +startDockerIdleReaper(); + +// Restore persisted Secure sessions so workspaces + TTL survive restarts +// (expired ones are deleted), then kill stray containers from the old process. +const { restored, expired } = loadPersistedSessions(); +if (restored > 0 || expired > 0) { + console.error(`[iclaw-runtime] restored ${restored} secure session(s), deleted ${expired} expired`); +} +killOrphanContainers() + .then((n) => { if (n > 0) console.error(`[iclaw-runtime] killed ${n} orphan container(s)`); }) + .catch((err) => console.error('[iclaw-runtime] container cleanup failed', err)); +killOrphanWorkContainers() + .then((n) => { if (n > 0) console.error(`[iclaw-runtime] killed ${n} orphan work container(s)`); }) + .catch((err) => console.error('[iclaw-runtime] work container cleanup failed', err)); + +let shuttingDown = false; +async function shutdown(): Promise<void> { + if (shuttingDown) return; + shuttingDown = true; + // Stop a Docker daemon WE started if it's idle/container-free (capped so a + // hung Docker can't block the exit). No-op when we don't own it or a + // container is still up — the durable marker is left for the next runtime. + try { + await stopDockerOnShutdown(); + } catch { + /* best-effort */ + } + server.close(); + process.exit(0); +} +process.on('SIGTERM', shutdown); +process.on('SIGINT', shutdown); diff --git a/packages/iclaw-runtime/src/install-id.ts b/packages/iclaw-runtime/src/install-id.ts new file mode 100644 index 0000000..adffc2a --- /dev/null +++ b/packages/iclaw-runtime/src/install-id.ts @@ -0,0 +1,26 @@ +/** + * Per-install identity for container labelling. + * + * Every sandbox container is stamped with `--label iclaw-install=<slug>` so the + * orphan reapers only ever touch THIS install's containers — two iClaw installs + * (different checkouts) on one machine can't kill each other's sandboxes. + * + * The slug is sha1 of this install's absolute path (like NanoClaw's + * install-slug): deterministic across restarts, unique across checkouts. + * Overridable via ICLAW_INSTALL_ID for unusual setups. + */ +import { createHash } from 'node:crypto'; +import { dirname, resolve } from 'node:path'; +import { fileURLToPath } from 'node:url'; + +function computeSlug(): string { + const override = process.env.ICLAW_INSTALL_ID?.trim(); + if (override) return override.replace(/[^A-Za-z0-9_.-]/g, '').slice(0, 32) || 'iclaw'; + // This file lives at <install>/packages/iclaw-runtime/(src|dist)/install-id.* + const here = dirname(fileURLToPath(import.meta.url)); + const installRoot = resolve(here, '..'); + return createHash('sha1').update(installRoot).digest('hex').slice(0, 8); +} + +/** `iclaw-install=<slug>` — the value passed to `docker run --label` / `ps --filter`. */ +export const INSTALL_LABEL = `iclaw-install=${computeSlug()}`; diff --git a/packages/iclaw-runtime/src/log.ts b/packages/iclaw-runtime/src/log.ts new file mode 100644 index 0000000..d1e820c --- /dev/null +++ b/packages/iclaw-runtime/src/log.ts @@ -0,0 +1,64 @@ +const LEVELS = { debug: 20, info: 30, warn: 40, error: 50, fatal: 60 } as const; +type Level = keyof typeof LEVELS; + +const COLORS: Record<Level, string> = { + debug: '\x1b[34m', + info: '\x1b[32m', + warn: '\x1b[33m', + error: '\x1b[31m', + fatal: '\x1b[41m\x1b[37m', +}; +const KEY_COLOR = '\x1b[35m'; +const MSG_COLOR = '\x1b[36m'; +const RESET = '\x1b[39m'; +const FULL_RESET = '\x1b[0m'; + +const threshold = LEVELS[(process.env.LOG_LEVEL as Level) || 'info'] ?? LEVELS.info; + +function formatErr(err: unknown): string { + if (err instanceof Error) { + return `{ type: "${err.constructor.name}", message: "${err.message}", stack: ${err.stack} }`; + } + return JSON.stringify(err); +} + +function formatData(data: Record<string, unknown>): string { + const parts: string[] = []; + for (const [k, v] of Object.entries(data)) { + parts.push(`${KEY_COLOR}${k}${RESET}=${k === 'err' ? formatErr(v) : JSON.stringify(v)}`); + } + return parts.length ? ' ' + parts.join(' ') : ''; +} + +function ts(): string { + const d = new Date(); + const hh = String(d.getHours()).padStart(2, '0'); + const mm = String(d.getMinutes()).padStart(2, '0'); + const ss = String(d.getSeconds()).padStart(2, '0'); + const ms = String(d.getMilliseconds()).padStart(3, '0'); + return `${hh}:${mm}:${ss}.${ms}`; +} + +function emit(level: Level, msg: string, data?: Record<string, unknown>): void { + if (LEVELS[level] < threshold) return; + const tag = `${COLORS[level]}${level.toUpperCase()}${level === 'fatal' ? FULL_RESET : RESET}`; + const stream = LEVELS[level] >= LEVELS.warn ? process.stderr : process.stdout; + stream.write(`[${ts()}] ${tag} ${MSG_COLOR}${msg}${RESET}${data ? formatData(data) : ''}\n`); +} + +export const log = { + debug: (msg: string, data?: Record<string, unknown>) => emit('debug', msg, data), + info: (msg: string, data?: Record<string, unknown>) => emit('info', msg, data), + warn: (msg: string, data?: Record<string, unknown>) => emit('warn', msg, data), + error: (msg: string, data?: Record<string, unknown>) => emit('error', msg, data), + fatal: (msg: string, data?: Record<string, unknown>) => emit('fatal', msg, data), +}; + +process.on('uncaughtException', (err) => { + log.fatal('Uncaught exception', { err }); + process.exit(1); +}); + +process.on('unhandledRejection', (reason) => { + log.error('Unhandled rejection', { err: reason }); +}); diff --git a/packages/iclaw-runtime/src/secure-export.ts b/packages/iclaw-runtime/src/secure-export.ts new file mode 100644 index 0000000..5bb9773 --- /dev/null +++ b/packages/iclaw-runtime/src/secure-export.ts @@ -0,0 +1,198 @@ +/** + * Safe Mode export / apply. + * + * After working in the sandbox the user may want the results OUT of it: + * export → copy the whole sandbox workspace to a host folder (a fresh place; + * nothing of theirs is overwritten). Good for "give me the output". + * apply → for each folder that was copied IN (see secure-ingest's + * .iclaw-ingest.json), copy the new/changed files back to the + * ORIGINAL folder. Additive + overwrite only — never deletes the + * user's files — and refuses secret roots. This is the explicit + * "apply the sandbox's changes to my real project" action. + * + * Both are user-initiated (button + confirm); apply is the only path that ever + * writes to the user's real files, and only to folders they themselves chose to + * ingest. + */ +import { + cpSync, mkdirSync, readFileSync, writeFileSync, existsSync, statSync, readdirSync, +} from 'node:fs'; +import { homedir } from 'node:os'; +import { join, relative, sep, dirname } from 'node:path'; + +import { validateMountRoot } from './agent/security.js'; +import { log } from './log.js'; + +/** Workspace internals that must never be exported/applied. */ +const INTERNAL = new Set(['.tools', '.iclaw-ingest.json']); +const SKIP_DIRS = new Set(['node_modules', '.git']); + +export interface ExportResult { + ok: boolean; + /** Host path the workspace was copied to (on success). */ + path?: string; + files?: number; + error?: string; +} + +export interface ApplyFileChange { + /** Original-folder-relative path that was written. */ + path: string; + kind: 'created' | 'modified'; +} + +export interface ApplyResult { + ok: boolean; + /** Original host folder the changes were applied to. */ + source?: string; + applied?: ApplyFileChange[]; + error?: string; +} + +interface IngestLogEntry { + kind: string; + source: string; + ok: boolean; + target?: string; +} + +/** Bounded recursive file count. */ +function countFiles(dir: string, cap = 50_000): number { + let total = 0; + const stack = [dir]; + while (stack.length && total < cap) { + const cur = stack.pop()!; + let entries; + try { entries = readdirSync(cur, { withFileTypes: true }); } + catch { continue; } + for (const e of entries) { + if (e.isDirectory()) stack.push(join(cur, e.name)); + else total++; + } + } + return total; +} + +/** Copy the whole sandbox (minus workspace internals) to a fresh host folder. */ +export function exportWorkspace(workspaceDir: string, destDir?: string): ExportResult { + try { + const base = destDir && destDir.trim() ? destDir : join(homedir(), 'Downloads'); + mkdirSync(base, { recursive: true }); + const target = join(base, `iclaw-sandbox-${Date.now()}`); + cpSync(workspaceDir, target, { + recursive: true, + filter: (src) => { + const rel = relative(workspaceDir, src); + if (!rel) return true; // the root itself + const top = rel.split(sep)[0]; + return !INTERNAL.has(top); + }, + }); + return { ok: true, path: target, files: countFiles(target) }; + } catch (err) { + return { ok: false, error: err instanceof Error ? err.message : String(err) }; + } +} + +function readIngestLog(workspaceDir: string): IngestLogEntry[] { + try { + const raw = JSON.parse(readFileSync(join(workspaceDir, '.iclaw-ingest.json'), 'utf-8')); + return Array.isArray(raw?.results) ? (raw.results as IngestLogEntry[]) : []; + } catch { + return []; + } +} + +/** True when two files differ by size or content (cheap size check first). */ +function differs(a: string, b: string): boolean { + try { + const sa = statSync(a); + const sb = statSync(b); + if (sa.size !== sb.size) return true; + return !readFileSync(a).equals(readFileSync(b)); + } catch { + return true; // missing target / unreadable → treat as changed + } +} + +/** + * Copy new/changed files from `sandboxRoot` back into `originalRoot`. Additive + + * overwrite only (never deletes); skips node_modules/.git. Returns what changed. + */ +function syncBack(sandboxRoot: string, originalRoot: string): ApplyFileChange[] { + const applied: ApplyFileChange[] = []; + const stack = ['']; + while (stack.length) { + const rel = stack.pop()!; + const cur = rel ? join(sandboxRoot, rel) : sandboxRoot; + let entries; + try { entries = readdirSync(cur, { withFileTypes: true }); } + catch { continue; } + for (const e of entries) { + const childRel = rel ? join(rel, e.name) : e.name; + if (e.isDirectory()) { + if (SKIP_DIRS.has(e.name)) continue; + stack.push(childRel); + continue; + } + const from = join(sandboxRoot, childRel); + const to = join(originalRoot, childRel); + const existed = existsSync(to); + if (existed && !differs(from, to)) continue; // unchanged + mkdirSync(dirname(to), { recursive: true }); + cpSync(from, to); + applied.push({ path: childRel, kind: existed ? 'modified' : 'created' }); + } + } + return applied; +} + +/** + * Apply the sandbox's changes back to the original folders that were ingested. + * Only `folder` sources have an original to write to (repo/zip/url don't). Each + * original is re-validated (refuses secret roots) before any write. + */ +export function applyChanges(workspaceDir: string): ApplyResult[] { + const entries = readIngestLog(workspaceDir).filter( + (e) => e.ok && e.kind === 'folder' && e.target, + ); + const out: ApplyResult[] = []; + for (const e of entries) { + try { + // Re-validate the destination root (refuses secret-bearing roots, resolves + // symlinks) — the same gate ingest used, now guarding the write-back. + const originalRoot = validateMountRoot(e.source); + const sandboxRoot = join(workspaceDir, e.target!); + if (!existsSync(sandboxRoot)) { + out.push({ ok: false, source: e.source, error: 'sandbox copy missing' }); + continue; + } + const applied = syncBack(sandboxRoot, originalRoot); + out.push({ ok: true, source: originalRoot, applied }); + } catch (err) { + const error = err instanceof Error ? err.message : String(err); + log.warn('Safe-mode apply failed', { source: e.source, error }); + out.push({ ok: false, source: e.source, error }); + } + } + return out; +} + +/** One-line summaries for the UI. */ +export function describeExport(r: ExportResult): string { + return r.ok ? `Exported ${r.files ?? '?'} files to ${r.path}` : `Export failed: ${r.error}`; +} + +export function describeApply(results: ApplyResult[]): string { + if (results.length === 0) return 'Nothing to apply — no folders were copied into this sandbox.'; + const lines: string[] = []; + for (const r of results) { + if (!r.ok) { + lines.push(`✗ ${r.source}: ${r.error}`); + continue; + } + const n = r.applied?.length ?? 0; + lines.push(n === 0 ? `• ${r.source}: no changes` : `• ${r.source}: applied ${n} file(s)`); + } + return lines.join('\n'); +} diff --git a/packages/iclaw-runtime/src/secure-ingest.ts b/packages/iclaw-runtime/src/secure-ingest.ts new file mode 100644 index 0000000..ec14ddf --- /dev/null +++ b/packages/iclaw-runtime/src/secure-ingest.ts @@ -0,0 +1,202 @@ +/** + * Safe Mode source ingest. + * + * Safe Mode = "I don't trust this thing. Work on a COPY in a sandbox." So before + * the agent touches anything, the untrusted source is brought INTO the isolated + * sandbox workspace (~/.iclaw/secure/<id>/), and the container only ever mounts + * that workspace. The user's original files/repos are never modified — we copy, + * clone, extract, or download into the sandbox; the originals stay put. + * + * Supported sources: + * folder → recursive copy of a local directory (secret roots refused) + * repo → shallow `git clone` of a remote http(s)/git URL + * zip → extract a local .zip into the workspace + * url → download a single artifact into the workspace + * + * Each ingest lands in its own subdirectory so multiple sources don't collide, + * and a `.iclaw-ingest.json` log is written for the report / audit trail. + */ +import { execFile } from 'node:child_process'; +import { + cpSync, mkdirSync, statSync, writeFileSync, readdirSync, existsSync, realpathSync, +} from 'node:fs'; +import { basename, join } from 'node:path'; +import { promisify } from 'node:util'; + +import { validateMountRoot } from './agent/security.js'; +import { log } from './log.js'; + +const execFileAsync = promisify(execFile); + +const CLONE_TIMEOUT = Number(process.env.ICLAW_INGEST_CLONE_TIMEOUT) || 180_000; +const DOWNLOAD_TIMEOUT = Number(process.env.ICLAW_INGEST_DOWNLOAD_TIMEOUT) || 120_000; +/** Cap the file count we report (a cheap, bounded walk — not a full inventory). */ +const COUNT_CAP = 50_000; + +export type IngestSource = + | { kind: 'folder'; path: string } + | { kind: 'repo'; url: string } + | { kind: 'zip'; path: string } + | { kind: 'url'; url: string }; + +export interface IngestResult { + kind: IngestSource['kind']; + /** Human description of the source (path or URL). */ + source: string; + ok: boolean; + /** Workspace-relative subdir the source landed in (on success). */ + target?: string; + /** Approximate file count copied/cloned/extracted. */ + files?: number; + /** Failure reason (on error). */ + error?: string; +} + +function sourceLabel(s: IngestSource): string { + return s.kind === 'repo' || s.kind === 'url' ? s.url : s.path; +} + +/** A non-colliding subdir name inside the workspace for `base`. */ +function uniqueTarget(workspaceDir: string, base: string): string { + const safe = base.replace(/[^A-Za-z0-9._-]/g, '_').replace(/^\.+/, '') || 'source'; + let name = safe; + let n = 1; + while (existsSync(join(workspaceDir, name))) name = `${safe}-${n++}`; + return name; +} + +/** Bounded recursive file count — never walks more than COUNT_CAP entries. */ +function countFiles(dir: string): number { + let total = 0; + const stack = [dir]; + while (stack.length && total < COUNT_CAP) { + const cur = stack.pop()!; + let entries; + try { entries = readdirSync(cur, { withFileTypes: true }); } + catch { continue; } + for (const e of entries) { + if (e.isDirectory()) stack.push(join(cur, e.name)); + else total++; + if (total >= COUNT_CAP) break; + } + } + return total; +} + +/** Derive a sane subdir name from a clone/download URL. */ +function nameFromUrl(url: string): string { + try { + const u = new URL(url); + const last = u.pathname.split('/').filter(Boolean).pop() || u.hostname; + return last.replace(/\.git$/i, ''); + } catch { + return 'source'; + } +} + +async function ingestFolder(workspaceDir: string, path: string): Promise<IngestResult> { + // validateMountRoot resolves symlinks AND refuses secret-bearing roots + // (~/.ssh, .env, credentials, …) — same gate Work Mode uses for mounts. + const real = validateMountRoot(path); + if (!statSync(real).isDirectory()) throw new Error('not a folder'); + const target = uniqueTarget(workspaceDir, basename(real)); + const dest = join(workspaceDir, target); + // Copy a COPY — the original is never touched. Skip the VCS/dep noise that + // bloats the sandbox without adding signal. + cpSync(real, dest, { + recursive: true, + filter: (src) => !/(^|[\\/])(node_modules|\.git)([\\/]|$)/.test(src.slice(real.length)), + }); + return { kind: 'folder', source: path, ok: true, target, files: countFiles(dest) }; +} + +async function ingestRepo(workspaceDir: string, url: string): Promise<IngestResult> { + if (!/^(https?|git):\/\//i.test(url)) { + throw new Error('only http(s):// or git:// repo URLs are allowed'); + } + const target = uniqueTarget(workspaceDir, nameFromUrl(url)); + const dest = join(workspaceDir, target); + // --depth 1, no checkout of submodules — we want a working copy, fast. git + // clone does not execute repo code, so cloning untrusted repos on the host is + // safe; the agent only ever RUNS the code later inside the sandbox. + await execFileAsync('git', ['clone', '--depth', '1', '--', url, dest], { timeout: CLONE_TIMEOUT }); + return { kind: 'repo', source: url, ok: true, target, files: countFiles(dest) }; +} + +async function ingestZip(workspaceDir: string, path: string): Promise<IngestResult> { + const real = realpathSync(path); + if (!statSync(real).isFile()) throw new Error('not a file'); + const target = uniqueTarget(workspaceDir, basename(real).replace(/\.zip$/i, '')); + const dest = join(workspaceDir, target); + mkdirSync(dest, { recursive: true }); + // `unzip` is present on macOS and most Linux; surfaces a clear error if not. + await execFileAsync('unzip', ['-q', '-o', real, '-d', dest], { timeout: DOWNLOAD_TIMEOUT }); + return { kind: 'zip', source: path, ok: true, target, files: countFiles(dest) }; +} + +async function ingestUrl(workspaceDir: string, url: string): Promise<IngestResult> { + if (!/^https?:\/\//i.test(url)) throw new Error('only http(s):// URLs are allowed'); + const dir = join(workspaceDir, 'downloads'); + mkdirSync(dir, { recursive: true }); + const file = uniqueTarget(dir, nameFromUrl(url) || 'download'); + const res = await fetch(url, { signal: AbortSignal.timeout(DOWNLOAD_TIMEOUT) }); + if (!res.ok) throw new Error(`download failed (HTTP ${res.status})`); + const buf = Buffer.from(await res.arrayBuffer()); + writeFileSync(join(dir, file), buf); + return { kind: 'url', source: url, ok: true, target: `downloads/${file}`, files: 1 }; +} + +async function ingestOne(workspaceDir: string, s: IngestSource): Promise<IngestResult> { + switch (s.kind) { + case 'folder': return ingestFolder(workspaceDir, s.path); + case 'repo': return ingestRepo(workspaceDir, s.url); + case 'zip': return ingestZip(workspaceDir, s.path); + case 'url': return ingestUrl(workspaceDir, s.url); + } +} + +/** + * Ingest every source into the sandbox workspace, never throwing — a failed + * source becomes a `{ ok: false, error }` result so the others still land. Writes + * a `.iclaw-ingest.json` log into the workspace for the report / audit trail. + */ +export async function ingestSources( + workspaceDir: string, + sources: IngestSource[], +): Promise<IngestResult[]> { + const results: IngestResult[] = []; + for (const s of sources) { + try { + results.push(await ingestOne(workspaceDir, s)); + } catch (err) { + const error = err instanceof Error ? err.message : String(err); + log.warn('Safe-mode ingest failed', { source: sourceLabel(s), error }); + results.push({ kind: s.kind, source: sourceLabel(s), ok: false, error }); + } + } + try { + writeFileSync( + join(workspaceDir, '.iclaw-ingest.json'), + JSON.stringify({ at: new Date().toISOString(), results }, null, 2), + 'utf-8', + ); + } catch { + /* best-effort log */ + } + return results; +} + +/** One-line user-facing summary of an ingest pass (prepended to the turn). */ +export function describeIngest(results: IngestResult[]): string { + if (results.length === 0) return ''; + const ok = results.filter((r) => r.ok); + const failed = results.filter((r) => !r.ok); + const lines = ['[Safe sandbox] Working on a COPY — your original files are unchanged.']; + for (const r of ok) { + lines.push(` • copied ${r.source} → /workspace/${r.target} (${r.files ?? '?'} files)`); + } + for (const r of failed) { + lines.push(` • could not ingest ${r.source}: ${r.error}`); + } + return lines.join('\n'); +} diff --git a/packages/iclaw-runtime/src/secure-runner.ts b/packages/iclaw-runtime/src/secure-runner.ts new file mode 100644 index 0000000..647cdab --- /dev/null +++ b/packages/iclaw-runtime/src/secure-runner.ts @@ -0,0 +1,610 @@ +/** + * Secure Mode runner. + * + * Agent loop runs on the HOST (can reach OpenRouter). + * Tool execution happens INSIDE a Docker container (isolated). + * + * Per-turn container lifecycle: + * start container → execute turn → stop container → (keep workspace dir) + * Next turn: start NEW container with SAME workspace volume. + * This allows changing network settings per message while preserving files. + */ +import { execFile } from 'node:child_process'; +import { mkdtempSync, mkdirSync, rmSync, writeFileSync, readFileSync, existsSync, chmodSync, readdirSync, statSync } from 'node:fs'; +import { homedir } from 'node:os'; +import { join, basename, resolve, extname, sep } from 'node:path'; +import { randomUUID } from 'node:crypto'; +import { promisify } from 'node:util'; + +import type OpenAI from 'openai'; +import type { AgentEvent, AgentOptions, Message } from './agent/loop.js'; +import type { SavingsNote } from './agent/tools.js'; +import { shrinkOldToolOutputs, withPromptCaching, makeToolGuard, HOST_INSTALL_POLICY } from './agent/loop.js'; +import { log } from './log.js'; +import { INSTALL_LABEL } from './install-id.js'; +import { dumpPrompt, newTurnId } from './agent/prompt-dump.js'; + +const execFileAsync = promisify(execFile); + +// Curated secure sandbox image: a small CLI toolset, no browser (build-secure.sh). +// The agent reaches the web via `curl` and can self-install more tools into +// /workspace/.tools (no root). Runs as the non-root `node` user — see +// secure-sandbox.Dockerfile. +const SECURE_IMAGE = process.env.ICLAW_SECURE_IMAGE || 'iclaw-secure:latest'; +// Emergency fallback when the curated image isn't built yet — e.g. a fresh +// `npx @iclawapp/iclaw` install, which doesn't ship the Dockerfile. A public +// base `docker run` can auto-pull, so Safe work degrades to a leaner toolset +// instead of hard-failing (matches Work mode in work-container.ts + the AGENTS.md +// contract that "node:22 is only an emergency fallback"). Full node:22 (not +// -slim) carries curl/wget/git so web research still works; the non-root user +// and the /workspace/.tools PATH the curated image bakes in are re-applied as run +// flags in startContainer (fallbackArgs) so the fallback keeps the SAME +// guarantees. Override with ICLAW_SECURE_FALLBACK_IMAGE. +const FALLBACK_IMAGE = process.env.ICLAW_SECURE_FALLBACK_IMAGE || 'node:22'; +let resolvedSecureImage: string | null = null; +const CONTAINER_TIMEOUT = 30_000; +/** Max tool-call rounds per turn (env-tunable for long multi-step tasks). */ +const MAX_ROUNDS = Math.max(1, Number(process.env.ICLAW_MAX_ROUNDS) || 40); + +export type SecureEvent = AgentEvent; + +const SECURE_PREFIX = 'iclaw-secure-'; + +// Persistent root for Secure workspaces. NOT tmpdir(): macOS prunes /var/folders +// after a few idle days, which is shorter than our 7-day TTL. A stable dir means +// workspaces (and their TTL) genuinely survive runtime restarts. +const SECURE_DATA_DIR = process.env.ICLAW_SECURE_DATA_DIR || join(homedir(), '.iclaw', 'secure'); + +/** Metadata persisted alongside a workspace so sessions survive restarts. */ +export interface SessionMeta { + key?: string; // stable identity (e.g. "chat:156") for reconnection + lastActivity: number; // ms; TTL counts from here + ttlMs: number; // 0 = never + secure: boolean; + model?: string; +} + +function metaPathFor(workspaceDir: string): string { + return `${workspaceDir}.meta.json`; +} + +/** + * Create a persistent workspace directory. Lives under SECURE_DATA_DIR so it + * survives restarts (reaped only by TTL). + */ +export function createSecureWorkspace(): string { + mkdirSync(SECURE_DATA_DIR, { recursive: true }); + const dir = mkdtempSync(join(SECURE_DATA_DIR, `${SECURE_PREFIX}`)); + // The sandbox runs as the non-root `node` user; make the bind-mounted dir + // writable to it (mkdtemp defaults to 0700 owned by the host user). + try { chmodSync(dir, 0o777); } catch {} + // Pre-create the runtime tool dirs the container puts on PATH (.tools/bin for + // downloaded static binaries, .tools/npm for `npm i -g`). They live inside the + // workspace, so self-installed tools persist across container restarts and are + // auto-deleted with the workspace when its TTL expires. World-writable so the + // non-root container user can install into them. + for (const sub of ['.tools', '.tools/bin', '.tools/npm']) { + const p = join(dir, sub); + try { mkdirSync(p, { recursive: true }); chmodSync(p, 0o777); } catch {} + } + return dir; +} + +/** Destroy a workspace directory and its metadata sidecar. */ +export function destroySecureWorkspace(workspaceDir: string): void { + try { rmSync(workspaceDir, { recursive: true, force: true }); } catch {} + try { rmSync(metaPathFor(workspaceDir), { force: true }); } catch {} +} + +/** Persist session metadata next to (not inside) the workspace. */ +export function writeSessionMeta(workspaceDir: string, meta: SessionMeta): void { + try { writeFileSync(metaPathFor(workspaceDir), JSON.stringify(meta), 'utf-8'); } catch {} +} + +/** List every persisted workspace dir with its metadata (for startup reload). */ +export function listPersistedWorkspaces(): { dir: string; meta: SessionMeta | null }[] { + const out: { dir: string; meta: SessionMeta | null }[] = []; + let entries; + try { entries = readdirSync(SECURE_DATA_DIR, { withFileTypes: true }); } + catch { return out; } + for (const entry of entries) { + if (!entry.isDirectory() || !entry.name.startsWith(SECURE_PREFIX)) continue; + const dir = join(SECURE_DATA_DIR, entry.name); + let meta: SessionMeta | null = null; + try { meta = JSON.parse(readFileSync(metaPathFor(dir), 'utf-8')) as SessionMeta; } catch {} + out.push({ dir, meta }); + } + return out; +} + +/** + * Kill containers left running by a previous runtime process. Containers are + * pure runtime (no data), so this is always safe — unlike workspace dirs, which + * carry the user's files and are reaped only by TTL (see sessions.ts reload). + */ +export async function killOrphanContainers(): Promise<number> { + try { + // Scope to THIS install's Secure containers (name AND install label) so a + // second iClaw install can't reap ours. + const { stdout } = await execFileAsync( + 'docker', ['ps', '-aq', '--filter', `name=${SECURE_PREFIX}`, '--filter', `label=${INSTALL_LABEL}`], + { timeout: 10_000 }, + ); + const ids = stdout.split('\n').map((s) => s.trim()).filter(Boolean); + if (ids.length === 0) return 0; + await execFileAsync('docker', ['rm', '-f', ...ids], { timeout: 30_000 }).catch(() => {}); + return ids.length; + } catch { + return 0; // Docker not installed/running — nothing to clean. + } +} + +async function imageExists(image: string): Promise<boolean> { + try { + await execFileAsync('docker', ['image', 'inspect', image], { timeout: 8_000 }); + return true; + } catch { + return false; + } +} + +/** + * Pick the Safe-work image: the curated sandbox if it's been built (or an + * explicit ICLAW_SECURE_IMAGE override, trusted as-is so a registry ref can + * auto-pull), otherwise the public fallback so a fresh install degrades + * gracefully instead of hard-failing. Cached after the first resolve — a build + * done mid-session is picked up on the next runtime restart, as in Work mode. + */ +async function resolveSecureImage(): Promise<string> { + if (resolvedSecureImage) return resolvedSecureImage; + if (process.env.ICLAW_SECURE_IMAGE || (await imageExists(SECURE_IMAGE))) { + resolvedSecureImage = SECURE_IMAGE; + } else { + log.warn( + 'Curated sandbox image not built — Safe work falling back to a leaner base. ' + + 'Build it with `npm run build:secure-image` for the full toolset.', + { fallback: FALLBACK_IMAGE, image: SECURE_IMAGE }, + ); + resolvedSecureImage = FALLBACK_IMAGE; + } + return resolvedSecureImage; +} + +/** + * Start a container with the given workspace. Returns containerName. + * Fail-closed: if Docker can't start the sandbox we throw, so the caller + * aborts the turn instead of silently running file ops outside isolation. + * + * The container is long-lived (`sleep`) and reused across turns by the session + * manager — see sessions.ts. It is recreated only when the network setting + * changes or after an idle period. + */ +export async function startContainer(workspaceDir: string, networkEnabled: boolean): Promise<string> { + const containerName = `iclaw-secure-${randomUUID().slice(0, 8)}`; + + const networkArgs = networkEnabled ? [] : ['--network', 'none']; + + const image = await resolveSecureImage(); + // The curated image bakes in `USER node` (non-root) and a /workspace/.tools + // PATH for rootless self-installs. A public fallback base defaults to ROOT and + // lacks those envs, so on the fallback we re-apply them as run flags to keep + // the SAME non-root + rootless-install guarantees. Empty for the curated image, + // so its run command stays byte-for-byte unchanged. + const fallbackArgs = image === FALLBACK_IMAGE + ? [ + '--user', '1000:1000', + '-e', 'HOME=/tmp', + '-e', 'NPM_CONFIG_PREFIX=/workspace/.tools/npm', + '-e', 'PATH=/workspace/.tools/bin:/workspace/.tools/npm/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', + ] + : []; + + try { + // `-d` returns once the container is up (and pulls the image on first run), + // so we await the real result instead of guessing with a fixed sleep. + await execFileAsync('docker', [ + 'run', '--rm', '-d', + '--name', containerName, + '--label', INSTALL_LABEL, + ...networkArgs, + ...fallbackArgs, + // No browser any more, so the default 64MB /dev/shm is fine and 512MB is + // plenty of headroom for shell/node tasks. Tunable via env. + '--memory', process.env.ICLAW_SECURE_MEMORY || '512m', + '--cpus', process.env.ICLAW_SECURE_CPUS || '1', + '-v', `${workspaceDir}:/workspace:rw`, + '--workdir', '/workspace', + image, + 'sleep', '86400', + ], { timeout: 60_000 }); + } catch (err) { + const e = err as { stderr?: string; message?: string }; + throw new Error( + `Secure sandbox failed to start — is Docker running? ${(e.stderr || e.message || '').slice(0, 200)}`, + ); + } + return containerName; +} + +/** Stop a container (workspace is preserved on host). */ +export function stopContainer(containerName: string): void { + try { execFile('docker', ['rm', '-f', containerName], () => {}); } catch {} +} + +/** True if a container with this name is up. Used for warm-reuse liveness. */ +export async function isContainerRunning(containerName: string): Promise<boolean> { + try { + const { stdout } = await execFileAsync( + 'docker', ['inspect', '-f', '{{.State.Running}}', containerName], + { timeout: 5_000 }, + ); + return stdout.trim() === 'true'; + } catch { + return false; + } +} + +/** Execute a command inside the sandbox container. */ +async function execInContainer(containerName: string, command: string): Promise<string> { + const dev = process.env.ICLAW_DEV_MODE === 'true'; + const startedAt = Date.now(); + // Don't dump giant commands (e.g. base64-injected tool scripts like + // social_search) into the log — keep the head + the useful env/arg tail. + const cmdLog = command.length > 400 + ? `${command.slice(0, 120)} …[+${command.length - 340} chars]… ${command.slice(-220)}` + : command; + try { + const { stdout, stderr } = await execFileAsync( + 'docker', ['exec', containerName, 'bash', '-c', command], + { timeout: CONTAINER_TIMEOUT }, + ); + const out = [stdout, stderr].filter(Boolean).join('\n').trim() || '(no output)'; + if (dev) log.info('secure run_command', { code: 0, ms: Date.now() - startedAt, bytes: out.length, cmd: cmdLog }); + return out; + } catch (err: unknown) { + const e = err as { stdout?: string; stderr?: string; message?: string; killed?: boolean; signal?: string; code?: number }; + const timedOut = Boolean(e.killed) || e.signal === 'SIGTERM' || e.signal === 'SIGKILL'; + const partial = [e.stdout, e.stderr].filter(Boolean).join('\n').trim(); + if (dev) { + log.warn('secure run_command failed', { + ms: Date.now() - startedAt, timedOut, + killed: Boolean(e.killed), signal: e.signal ?? null, code: e.code ?? null, + bytes: partial.length, cmd: cmdLog, + }); + } + // Not dev-gated: a timeout kill must reach the MODEL. In Secure Mode the + // sandbox is `--network none` unless the user enables it, so a `curl`/`git` + // hang here is the common case — say so instead of returning empty text the + // model misreads as a clean run. + if (timedOut) { + return (partial ? partial + '\n\n' : '') + + `[command killed after ${Math.round(CONTAINER_TIMEOUT / 1000)}s — it timed out and did NOT finish. ` + + `Likely a network hang (this sandbox has no internet unless the user turns the network toggle ON) ` + + `or an interactive prompt. Do not assume it completed.]`; + } + return partial || e.message || 'Error'; + } +} + +function readFromWorkspace(workspaceDir: string, filePath: string): string { + const safe = basename(filePath); + const full = join(workspaceDir, safe); + if (!existsSync(full)) return `File not found: ${safe}`; + return readFileSync(full, 'utf-8'); +} + +function writeToWorkspace(workspaceDir: string, filePath: string, content: string): string { + const safe = basename(filePath); + const full = join(workspaceDir, safe); + writeFileSync(full, content, 'utf-8'); + return `Written: /workspace/${safe}`; +} + +function editInWorkspace(workspaceDir: string, filePath: string, oldStr: string, newStr: string): string { + if (!oldStr) return 'edit_file requires old_string — the exact text to replace.'; + const safe = basename(filePath); + const full = join(workspaceDir, safe); + if (!existsSync(full)) return `File not found: ${safe}`; + const content = readFileSync(full, 'utf-8'); + const first = content.indexOf(oldStr); + if (first === -1) return 'old_string not found — copy the exact text (including whitespace) from the file.'; + if (content.indexOf(oldStr, first + oldStr.length) !== -1) { + return 'old_string is not unique — add more surrounding context so it matches one place.'; + } + writeFileSync(full, content.slice(0, first) + newStr + content.slice(first + oldStr.length), 'utf-8'); + return `Edited: /workspace/${safe}`; +} + +/** + * Run one turn in Secure Mode. + * + * The container is owned and reused by the session manager (warm reuse): it is + * passed in here, and NOT torn down at the end of the turn. Lifecycle (create + * on network change, reap on idle) lives in sessions.ts. + */ +export async function* runSecureTurn( + history: { role: string; content: string }[], + userMessage: string, + opts: { + apiKey: string; + model: string; + workspaceDir: string; + containerName: string; + networkEnabled?: boolean; + systemPrompt?: string; + signal?: AbortSignal; + /** Image data URLs for dropped files — shown to the model as vision blocks. */ + images?: string[]; + }, +): AsyncGenerator<SecureEvent> { + const networkEnabled = opts.networkEnabled ?? false; + + // The sandbox base prompt (identity + operating rules) is ALWAYS included; any + // caller-supplied prompt (e.g. project context) is APPENDED, not replaced — + // otherwise a project chat would lose the identity and the sandbox rules. + const secureBasePrompt = `You are iClaw, a private AI assistant${opts.model ? `, powered by ${opts.model}` : ''}. If asked what you are, say exactly that — never claim to be ChatGPT, Claude, Gemini or another product. +You are running in a secure isolated sandbox. This is the "work on a COPY" mode: the user does NOT trust the source, so everything happens inside /workspace and their real computer is never touched. +You can run commands and read/write files in /workspace only. +What's already here: any folders the user selected are COPIED into /workspace (originals untouched), and any files they dropped are saved here too. To examine an untrusted repo, archive, or URL, bring it in yourself — \`git clone <url>\`, \`unzip <file>\`, or \`curl -O <url>\` — into /workspace. Never expect or use host paths; only /workspace exists. +Work efficiently — each tool call is one step and steps are limited. Chain related +shell commands into a single run_command with && (e.g. \`git clone <url> repo && cd repo && pip install -r requirements.txt\`) instead of one command per step, and don't re-run exploratory commands you've already seen. +Preinstalled CLIs: git, rg (ripgrep), jq, curl, node, unzip/zip, less, tree. +You can install more tools yourself, no root needed${networkEnabled ? '' : ' (requires network, which is currently OFF — ask the user to enable it)'}: download a static binary into /workspace/.tools/bin (already on PATH) with curl, or run \`npm i -g <pkg>\`. Self-installed tools live in the workspace, so they persist across turns and are removed automatically when the workspace expires. +Network is ${networkEnabled ? 'enabled' : 'disabled'}.${ + networkEnabled + ? '\nTo fetch web pages or APIs, use run_command with `curl -s <url>` (there is no browser in this sandbox).' + + '\nFor a link\'s actual content, prefer the analyze_link tool (YouTube videos: subtitles/transcript). ' + + 'Use mode:"summary" with a short purpose by default to save tokens; mode:"full" only when you need exact wording.' + : '' + } +Be concise. +${HOST_INSTALL_POLICY}`; + const systemPrompt = opts.systemPrompt?.trim() + ? `${secureBasePrompt}\n\n${opts.systemPrompt.trim()}` + : secureBasePrompt; + + const gen = runSecureAgentLoop( + history.map((m) => ({ role: m.role as 'user' | 'assistant', content: m.content })), + userMessage, + { + apiKey: opts.apiKey, + model: opts.model, + allowedFolders: [opts.workspaceDir], + signal: opts.signal, + images: opts.images, + systemPrompt, + onWriteApproval: async () => true, + }, + opts.containerName, + opts.workspaceDir, + networkEnabled, + ); + + for await (const event of gen) { + yield event; + } +} + +async function* runSecureAgentLoop( + history: Message[], + userMessage: string, + opts: AgentOptions, + containerName: string, + workspaceDir: string, + networkEnabled: boolean, +): AsyncGenerator<SecureEvent> { + const OpenAI = (await import('openai')).default; + const { TOOL_DEFINITIONS, ANALYZE_LINK_TOOL, SHOW_IMAGE_TOOL, analyzeLink, clampMiddle, compressCommandOutput, TOOL_OUTPUT_MAX_CHARS } = + await import('./agent/tools.js'); + const { SOCIAL_SEARCH_TOOL, socialSearch } = await import('./agent/social.js'); + + // analyze_link and social_search are appended (not part of core TOOL_DEFINITIONS) + // and run their network work INSIDE this container, so they respect the same + // network gate. show_image lets the agent surface an image it produced in /workspace. + const tools = [...TOOL_DEFINITIONS, ANALYZE_LINK_TOOL, SOCIAL_SEARCH_TOOL, SHOW_IMAGE_TOOL]; + + // Buffer for analyze_link savings notes, flushed as `note` events per tool call. + const savingsNotes: SavingsNote[] = []; + // Buffer for show_image requests, flushed as `image` events per tool call. + const pendingImages: { path: string; mime: string; fileName: string; bytes: number }[] = []; + const SECURE_IMAGE_MIME: Record<string, string> = { + '.png': 'image/png', '.jpg': 'image/jpeg', '.jpeg': 'image/jpeg', + '.gif': 'image/gif', '.webp': 'image/webp', '.svg': 'image/svg+xml', + }; + + const client = new OpenAI({ + baseURL: 'https://openrouter.ai/api/v1', + apiKey: opts.apiKey, + }); + + // Dropped images → multimodal user message (text + vision blocks), seen once. + const userMsg: OpenAI.Chat.ChatCompletionUserMessageParam = opts.images?.length + ? { + role: 'user', + content: [ + { type: 'text', text: userMessage }, + ...opts.images.map((url) => ({ type: 'image_url' as const, image_url: { url } })), + ], + } + : { role: 'user', content: userMessage }; + + const messages: Message[] = [ + { role: 'system', content: opts.systemPrompt ?? 'You are a helpful assistant in a secure sandbox.' }, + ...history, + userMsg, + ]; + + let turnTokens = 0; + let turnCached = 0; + const dumpTurnId = newTurnId(); + const guard = makeToolGuard(); + // Paragraph break for the first text of a post-tool round, so streamed + // segments across a tool boundary don't glue (see loop.ts for the full story). + let pendingSeparator = ''; + for (let round = 0; round < MAX_ROUNDS; round++) { + // User pressed Stop between rounds → end cleanly (partial text already sent). + if (opts.signal?.aborted) { + yield { type: 'done', tokens: turnTokens || undefined, cached: turnCached || undefined }; + return; + } + let textBuffer = ''; + const toolCallBuffers: Record<string, { name: string; arguments: string }> = {}; + + dumpPrompt({ turnId: dumpTurnId, mode: 'secure', model: opts.model, round, messages, tools }); + + // eslint-disable-next-line @typescript-eslint/no-explicit-any + let stream: AsyncIterable<any>; + try { + stream = await client.chat.completions.create( + { + model: opts.model, + messages: withPromptCaching(messages), + // eslint-disable-next-line @typescript-eslint/no-explicit-any + tools: tools as any, + tool_choice: 'auto', + stream: true, + stream_options: { include_usage: true }, + }, + { signal: opts.signal }, + ); + } catch (err) { + if (opts.signal?.aborted) { + yield { type: 'done', tokens: turnTokens || undefined, cached: turnCached || undefined }; + return; + } + yield { type: 'error', message: err instanceof Error ? err.message : String(err) }; + return; + } + + let finishReason: string | null = null; + try { + for await (const chunk of stream) { + if (chunk.usage?.total_tokens) turnTokens += chunk.usage.total_tokens; + if (chunk.usage?.prompt_tokens_details?.cached_tokens) turnCached += chunk.usage.prompt_tokens_details.cached_tokens; + const choice = chunk.choices[0]; + if (!choice) continue; + finishReason = choice.finish_reason ?? finishReason; + const delta = choice.delta; + if (delta.content) { + if (pendingSeparator) { + yield { type: 'text', content: pendingSeparator }; + pendingSeparator = ''; + } + textBuffer += delta.content; + yield { type: 'text', content: delta.content }; + } + if (delta.tool_calls) { + for (const tc of delta.tool_calls) { + const idx = String(tc.index ?? 0); + if (!toolCallBuffers[idx]) toolCallBuffers[idx] = { name: '', arguments: '' }; + if (tc.function?.name) toolCallBuffers[idx].name += tc.function.name; + if (tc.function?.arguments) toolCallBuffers[idx].arguments += tc.function.arguments; + } + } + } + } catch (err) { + // Stop pressed mid-stream → clean end; otherwise surface the error. + if (opts.signal?.aborted) { + yield { type: 'done', tokens: turnTokens || undefined, cached: turnCached || undefined }; + return; + } + yield { type: 'error', message: err instanceof Error ? err.message : String(err) }; + return; + } + + const toolCalls = Object.values(toolCallBuffers); + if (toolCalls.length === 0 || finishReason === 'stop') { + if (textBuffer) messages.push({ role: 'assistant', content: textBuffer }); + yield { type: 'done', tokens: turnTokens || undefined, cached: turnCached || undefined }; + return; + } + + messages.push({ + role: 'assistant', + content: textBuffer || null, + tool_calls: toolCalls.map((tc, i) => ({ + id: `call_${i}`, + type: 'function' as const, + function: { name: tc.name, arguments: tc.arguments }, + })), + } as Message); + + for (let i = 0; i < toolCalls.length; i++) { + const tc = toolCalls[i]; + let args: Record<string, unknown> = {}; + try { args = JSON.parse(tc.arguments); } catch {} + + yield { type: 'tool_start', name: tc.name, input: args }; + + let result: string; + const blocked = guard.check(tc.name, tc.arguments); + if (blocked) { + result = blocked; + } else if (tc.name === 'run_command') { + result = clampMiddle(compressCommandOutput(await execInContainer(containerName, String(args.command ?? ''))), TOOL_OUTPUT_MAX_CHARS); + } else if (tc.name === 'write_file') { + result = writeToWorkspace(workspaceDir, String(args.path ?? 'file.txt'), String(args.content ?? '')); + } else if (tc.name === 'edit_file') { + result = editInWorkspace(workspaceDir, String(args.path ?? ''), String(args.old_string ?? ''), String(args.new_string ?? '')); + } else if (tc.name === 'read_file') { + result = readFromWorkspace(workspaceDir, String(args.path ?? '')); + } else if (tc.name === 'list_files') { + result = await execInContainer(containerName, 'ls -la /workspace'); + } else if (tc.name === 'search_files') { + result = await execInContainer(containerName, `grep -r ${JSON.stringify(String(args.query ?? ''))} /workspace 2>/dev/null | head -20`); + } else if (tc.name === 'analyze_link') { + // Runs its fetch inside this same container (network honours the gate). + result = await analyzeLink(args, { + runInSandbox: (command) => execInContainer(containerName, command), + networkEnabled, + onNote: (note) => savingsNotes.push(note), + }); + } else if (tc.name === 'social_search') { + // Keyless social fetch — runs in this container, honours the network gate. + result = await socialSearch(args, { + runInSandbox: (command) => execInContainer(containerName, command), + networkEnabled, + onNote: (note) => savingsNotes.push(note), + }); + } else if (tc.name === 'show_image') { + // The workspace dir IS the host side of the container's /workspace mount, + // so an image written there (by run_command or write_file) is already on + // disk here. Resolve within the workspace (no ../ escape), then queue it. + const rel = String(args.path ?? '').replace(/^\/workspace\/?/, ''); + const full = resolve(workspaceDir, rel); + if (full !== workspaceDir && !full.startsWith(workspaceDir + sep)) { + result = 'show_image: the path must stay inside /workspace.'; + } else { + const ext = extname(full).toLowerCase(); + const mime = SECURE_IMAGE_MIME[ext]; + let st: ReturnType<typeof statSync> | null = null; + try { st = statSync(full); } catch { st = null; } + if (!mime) { + result = `show_image supports image files only (png/jpg/gif/webp/svg); got "${ext || 'no extension'}".`; + } else if (!st || !st.isFile() || st.size === 0) { + result = `No such image file: ${rel || String(args.path ?? '')}`; + } else if (st.size > 20 * 1024 * 1024) { + result = `Image too large to display (${st.size.toLocaleString()} bytes; max ${(20 * 1024 * 1024).toLocaleString()}).`; + } else { + pendingImages.push({ path: full, mime, fileName: basename(full), bytes: st.size }); + result = `Displayed ${basename(full)} to the user in the chat.`; + } + } + } else { + result = `Tool not available in secure mode: ${tc.name}`; + } + + yield { type: 'tool_result', name: tc.name, result }; + while (savingsNotes.length) yield { type: 'note', note: savingsNotes.shift()! }; + while (pendingImages.length) { + const im = pendingImages.shift()!; + yield { type: 'image', path: im.path, mime: im.mime, fileName: im.fileName, bytes: im.bytes }; + } + messages.push({ role: 'tool', tool_call_id: `call_${i}`, content: result }); + } + if (textBuffer && !/\n\s*$/.test(textBuffer)) pendingSeparator = '\n\n'; + shrinkOldToolOutputs(messages); // mid-turn compaction (see loop.ts) + } + + yield { type: 'error', message: `Reached the step limit (${MAX_ROUNDS} tool rounds). Send "continue" to keep going, or raise ICLAW_MAX_ROUNDS.` }; +} diff --git a/packages/iclaw-runtime/src/sessions.ts b/packages/iclaw-runtime/src/sessions.ts new file mode 100644 index 0000000..4617a0b --- /dev/null +++ b/packages/iclaw-runtime/src/sessions.ts @@ -0,0 +1,799 @@ +/** + * In-memory session manager for Work and Secure modes. + */ +import http from 'node:http'; +import { randomUUID } from 'node:crypto'; +import { readdirSync, statSync, readFileSync, copyFileSync } from 'node:fs'; +import { basename, dirname, join } from 'node:path'; + +import { runAgentTurn, type Message } from './agent/loop.js'; +import type { AgentEvent } from './agent/loop.js'; +import { validateMountRoot } from './agent/security.js'; +import { + runSecureTurn, createSecureWorkspace, destroySecureWorkspace, + startContainer, stopContainer, isContainerRunning, + writeSessionMeta, listPersistedWorkspaces, +} from './secure-runner.js'; +import { + resolveWorkImage, startWorkContainer, execInWorkContainer, + toWorkMounts, type WorkMount, +} from './work-container.js'; +import { ingestSources, describeIngest, type IngestSource } from './secure-ingest.js'; +import { + exportWorkspace, applyChanges, type ExportResult, type ApplyResult, +} from './secure-export.js'; +import { ensureDockerForTask, markDockerUse } from './docker-lifecycle.js'; + +export interface SessionOptions { + allowedFolders: string[]; + /** + * Per-folder access levels (path + readonly flag), parallel to allowedFolders. + * When omitted, every allowed folder is treated as writable. Used by Work Mode + * to enforce read-only folders; Secure Mode leaves it unset (workspace is RW). + */ + folderAccess?: { path: string; readonly: boolean }[]; + /** + * Safe Mode only: host folders to COPY into the sandbox workspace on the first + * turn (originals never touched). Realizes "I added a folder and it got copied + * into the sandbox" — distinct from Work Mode's live bind mounts. + */ + copyFolders?: string[]; + model: string; + apiKey: string; + secure?: boolean; + /** + * Incognito (read-only, ephemeral): runs the host loop like Work, but writes + * are denied, reads are unrestricted, the shell sandbox forces every folder + * to :ro, and web_fetch is available. Mutually exclusive with `secure`. + */ + incognito?: boolean; + networkEnabled?: boolean; + systemPrompt?: string; + /** Stable identity (e.g. "chat:156") for reconnecting to a workspace. */ + key?: string; + /** + * Optional compacted prior history (from the host's DB) used to seed context + * after a restart. Only applied when the session has no history yet — never + * clobbers a live session's accumulated context. + */ + history?: { role: string; content: string }[]; +} + +/** + * A file the user dropped into the chat, forwarded from the host. `path` is the + * absolute on-disk location of the persisted upload (the runtime shares the + * host filesystem), so the runtime can read it directly — to stage it into a + * Secure workspace, grant Work read access, or base64 an image for vision. + */ +export interface RuntimeAttachment { + path: string; + mimeType: string; + fileName: string; +} + +interface Session { + id: string; + opts: SessionOptions; + history: Message[]; + sseClient: http.ServerResponse | null; + pending: AgentEvent[]; + /** Persistent workspace dir for Secure Mode (survives container restarts). */ + secureWorkspaceDir?: string; + /** Safe Mode: folder paths already copied into the workspace (ingest once each). */ + ingestedFolders?: Set<string>; + /** + * Warm Secure-Mode sandbox container, reused across turns. Recreated only + * when the network setting changes or after the container is reaped for idle; + * the workspace dir outlives it either way. + */ + secureContainer?: { name: string; networkEnabled: boolean; lastUsed: number; inUse: boolean }; + /** + * Warm Work-Mode command sandbox, reused across turns (parallel to + * secureContainer — a session is exactly one mode). Holds the user's chosen + * folders bind-mounted :ro/:rw; only run_command runs inside it. + */ + workContainer?: { name: string; lastUsed: number; inUse: boolean }; + /** Stable identity for reconnection across restarts. */ + key?: string; + /** Last activity timestamp (ms). Updated on each message. */ + lastActivity: number; + /** TTL in ms after last activity before cleanup. 0 = never. Default 7 days. */ + ttlMs: number; + /** Controller for the in-flight turn — abort() stops the model stream and + * ends the agent loop (user pressed Stop). Set per turn, cleared after. */ + abort?: AbortController; +} + +const DEFAULT_TTL_MS = 7 * 86400_000; +// Warm container idle reap: many chats → keep this short so idle sandboxes +// don't pile up RAM. (The workspace dir persists; only the container is freed.) +const CONTAINER_IDLE_MS = 2 * 60_000; +// Cap concurrent warm sandbox containers; LRU-evict the rest. +const MAX_WARM_CONTAINERS = 4; + +// Live-session re-compaction: once in-memory history exceeds the trigger, fold +// the older turns into a summary so a long-running session (no restart) keeps a +// bounded context instead of growing unbounded. Tunable via env. +const HISTORY_COMPACT_TRIGGER = Number(process.env.ICLAW_HISTORY_COMPACT_TRIGGER) || 40; +const HISTORY_KEEP_RECENT = Number(process.env.ICLAW_HISTORY_KEEP_RECENT) || 16; +// Also compact by SIZE, not just message count: a few big messages bloat the +// resent context as much as many small ones. ~24k chars ≈ ~6k tokens. +const HISTORY_COMPACT_CHARS = Number(process.env.ICLAW_HISTORY_COMPACT_CHARS) || 24_000; +const SUMMARY_MODEL = process.env.ICLAW_SUMMARY_MODEL || 'google/gemini-2.5-flash-lite'; +const OPENROUTER_BASE = process.env.OPENROUTER_BASE_URL?.replace(/\/+$/, '') || 'https://openrouter.ai/api/v1'; +const SUMMARY_SYSTEM = + 'You compress conversation history into a concise, information-dense summary. ' + + 'Preserve facts, decisions, user preferences, names, numbers, file/work state, and open threads. ' + + 'Merge any existing summary with the new messages. Output only the summary.'; +const sessions = new Map<string, Session>(); +// Maps a stable key (e.g. "chat:156") to the current in-memory session id, so a +// chat reconnects to the same persisted workspace after a restart. +const keyToId = new Map<string, string>(); + +/** Persist TTL/activity metadata so the session survives a runtime restart. */ +function persistMeta(session: Session): void { + if (!session.opts.secure || !session.secureWorkspaceDir) return; + writeSessionMeta(session.secureWorkspaceDir, { + key: session.key, + lastActivity: session.lastActivity, + ttlMs: session.ttlMs, + secure: true, + model: session.opts.model, + }); +} + +/** + * Create a session, or — if a stable `key` is given and a live session already + * exists for it — reuse it (refreshing runtime params like model/systemPrompt/ + * apiKey while preserving the workspace, history, TTL and lastActivity). + */ +export function createSession(opts: SessionOptions): string { + if (opts.key) { + const existingId = keyToId.get(opts.key); + const existing = existingId ? sessions.get(existingId) : undefined; + if (existing) { + // Reconnect: keep state, refresh the volatile runtime params. + existing.opts = { ...existing.opts, ...opts }; + // Seed context only if empty (e.g. restored-from-disk after a restart) — + // never clobber a live session's accumulated history. + if (existing.history.length === 0) seedHistory(existing, opts.history); + return existing.id; + } + } + + const id = randomUUID(); + const session: Session = { + id, opts, history: [], sseClient: null, pending: [], + key: opts.key, lastActivity: Date.now(), ttlMs: DEFAULT_TTL_MS, + }; + seedHistory(session, opts.history); + if (opts.secure) { + session.secureWorkspaceDir = createSecureWorkspace(); + } + sessions.set(id, session); + if (opts.key) keyToId.set(opts.key, id); + persistMeta(session); + return id; +} + +/** Seed a session's history from compacted prior turns (host-supplied). */ +function seedHistory(session: Session, history?: { role: string; content: string }[]): void { + if (!history?.length) return; + session.history = history.map((m) => ({ role: m.role, content: m.content }) as Message); +} + +/** Set TTL (days) for a session. 0 = never expire. */ +export function setSessionTtl(id: string, ttlDays: number): void { + const session = sessions.get(id); + if (session) { + session.ttlMs = ttlDays <= 0 ? 0 : ttlDays * 86400_000; + persistMeta(session); + } +} + +/** + * Reload persisted Secure sessions at startup so workspaces (and their TTL) + * survive a runtime restart. Expired ones are deleted; the rest become live + * shell sessions that a chat reconnects to via its key. apiKey comes from env; + * model/systemPrompt are refreshed when the host next calls createSession. + */ +export function loadPersistedSessions(): { restored: number; expired: number } { + const now = Date.now(); + let restored = 0; + let expired = 0; + const apiKey = process.env.ICLAW_OPENROUTER_API_KEY || ''; + + for (const { dir, meta } of listPersistedWorkspaces()) { + if (!meta || (meta.ttlMs > 0 && now - meta.lastActivity > meta.ttlMs)) { + destroySecureWorkspace(dir); + expired++; + continue; + } + const id = randomUUID(); + const session: Session = { + id, + opts: { + allowedFolders: [dir], + model: meta.model || (process.env.ICLAW_MODEL || ''), + apiKey, + secure: true, + }, + history: [], + sseClient: null, + pending: [], + secureWorkspaceDir: dir, + key: meta.key, + lastActivity: meta.lastActivity, + ttlMs: meta.ttlMs, + }; + sessions.set(id, session); + if (meta.key) keyToId.set(meta.key, id); + restored++; + } + return { restored, expired }; +} + +/** Sweep expired sessions — destroys workspace + frees memory. Runs periodically. */ +export function sweepExpiredSessions(): number { + const now = Date.now(); + let removed = 0; + for (const [id, session] of sessions) { + if (session.ttlMs > 0 && now - session.lastActivity > session.ttlMs) { + if (session.secureContainer) stopContainer(session.secureContainer.name); + if (session.workContainer) stopContainer(session.workContainer.name); + if (session.secureWorkspaceDir) destroySecureWorkspace(session.secureWorkspaceDir); + if (session.key) keyToId.delete(session.key); + session.sseClient?.end(); + sessions.delete(id); + removed++; + } + } + return removed; +} + +/** + * Stop warm Secure containers that have been idle longer than CONTAINER_IDLE_MS. + * The session and its workspace dir survive — the next message re-creates the + * container. Driven by a short interval (startContainerReaper). + */ +export function reapIdleContainers(): number { + const now = Date.now(); + let reaped = 0; + for (const session of sessions.values()) { + const c = session.secureContainer; + if (c && !c.inUse && now - c.lastUsed > CONTAINER_IDLE_MS) { + stopContainer(c.name); + session.secureContainer = undefined; + reaped++; + } + const w = session.workContainer; + if (w && !w.inUse && now - w.lastUsed > CONTAINER_IDLE_MS) { + stopContainer(w.name); + session.workContainer = undefined; + reaped++; + } + } + return reaped; +} + +/** Start the idle-container reaper. Returns a stop handle. */ +export function startContainerReaper(intervalMs = 30_000): () => void { + const t = setInterval(reapIdleContainers, intervalMs); + t.unref?.(); + return () => clearInterval(t); +} + +/** + * Ensure the session has a running sandbox container matching `networkEnabled`, + * reusing the existing one when possible (warm reuse). Recreates it when the + * network setting changed or the old container died. Enforces a global cap on + * concurrent warm containers by LRU-evicting the least recently used. + * + * Throws (fail-closed) if Docker can't start the sandbox. + */ +async function ensureSecureContainer(session: Session, networkEnabled: boolean): Promise<string> { + const existing = session.secureContainer; + if (existing && existing.networkEnabled === networkEnabled && await isContainerRunning(existing.name)) { + existing.lastUsed = Date.now(); + return existing.name; + } + // Stale, dead, or network setting changed → drop the old one. + if (existing) { + stopContainer(existing.name); + session.secureContainer = undefined; + } + + evictWarmContainersIfNeeded(session); + + const name = await startContainer(session.secureWorkspaceDir!, networkEnabled); + session.secureContainer = { name, networkEnabled, lastUsed: Date.now(), inUse: false }; + return name; +} + +/** + * LRU-evict warm containers (excluding `keep`) until under the cap. Counts both + * Secure and Work sandboxes — a session has at most one, and both consume RAM. + */ +function evictWarmContainersIfNeeded(keep: Session): void { + type Warm = { session: Session; kind: 'secure' | 'work'; name: string; lastUsed: number }; + const warm: Warm[] = []; + for (const s of sessions.values()) { + if (s === keep) continue; + if (s.secureContainer && !s.secureContainer.inUse) { + warm.push({ session: s, kind: 'secure', name: s.secureContainer.name, lastUsed: s.secureContainer.lastUsed }); + } + if (s.workContainer && !s.workContainer.inUse) { + warm.push({ session: s, kind: 'work', name: s.workContainer.name, lastUsed: s.workContainer.lastUsed }); + } + } + while (warm.length >= MAX_WARM_CONTAINERS) { + warm.sort((a, b) => a.lastUsed - b.lastUsed); + const victim = warm.shift(); + if (!victim) break; + stopContainer(victim.name); + if (victim.kind === 'secure') victim.session.secureContainer = undefined; + else victim.session.workContainer = undefined; + } +} + +/** + * Ensure the Work session has a running command sandbox with `mounts`. Reuses + * the warm container when alive; recreates it if it died. Throws (fail-closed) + * if Docker can't start it, so run_command surfaces the error instead of + * leaking to the host. + */ +async function ensureWorkContainer(session: Session, mounts: WorkMount[], image: string): Promise<string> { + const existing = session.workContainer; + if (existing && await isContainerRunning(existing.name)) { + existing.lastUsed = Date.now(); + existing.inUse = true; + return existing.name; + } + if (existing) { + stopContainer(existing.name); + session.workContainer = undefined; + } + evictWarmContainersIfNeeded(session); + const name = await startWorkContainer(mounts, image); + session.workContainer = { name, lastUsed: Date.now(), inUse: true }; + return name; +} + +export function getSession(id: string): Session | undefined { + return sessions.get(id); +} + +/** + * Abort the in-flight turn for a session WITHOUT tearing the session down + * (unlike deleteSession, the workspace/container survive). The agent loop stops + * its model stream and ends cleanly. No-op if nothing is running. + */ +export function abortSession(id: string): boolean { + const session = sessions.get(id); + if (!session?.abort) return false; + session.abort.abort(); + return true; +} + +export function deleteSession(id: string): void { + const session = sessions.get(id); + if (session?.secureContainer) stopContainer(session.secureContainer.name); + if (session?.workContainer) stopContainer(session.workContainer.name); + if (session?.secureWorkspaceDir) { + destroySecureWorkspace(session.secureWorkspaceDir); + } + if (session?.key) keyToId.delete(session.key); + sessions.delete(id); +} + +/** Get workspace info for a session. */ +export function getSessionInfo(id: string): { workspaceSize: number; secure: boolean } | null { + const session = sessions.get(id); + if (!session) return null; + let workspaceSize = 0; + if (session.secureWorkspaceDir) { + workspaceSize = getDirSize(session.secureWorkspaceDir); + } + return { workspaceSize, secure: session.opts.secure ?? false }; +} + +/** Export a Safe session's sandbox to a host folder. Null if not a Safe session. */ +export function exportSessionWorkspace(id: string, destDir?: string): ExportResult | null { + const session = sessions.get(id); + if (!session?.secureWorkspaceDir) return null; + return exportWorkspace(session.secureWorkspaceDir, destDir); +} + +/** Apply a Safe session's changes back to the original ingested folders. */ +export function applySessionChanges(id: string): ApplyResult[] | null { + const session = sessions.get(id); + if (!session?.secureWorkspaceDir) return null; + return applyChanges(session.secureWorkspaceDir); +} + +function getDirSize(dir: string): number { + try { + let total = 0; + const walk = (d: string) => { + for (const entry of readdirSync(d, { withFileTypes: true })) { + const full = `${d}/${entry.name}`; + if (entry.isDirectory()) walk(full); + else total += statSync(full).size; + } + }; + walk(dir); + return total; + } catch { return 0; } +} + +/** Update network setting for a secure session (takes effect on next turn). */ +export function setNetworkEnabled(id: string, enabled: boolean): void { + const session = sessions.get(id); + if (session) session.opts.networkEnabled = enabled; +} + +export function attachSseClient(id: string, res: http.ServerResponse): void { + const session = sessions.get(id); + if (!session) return; + session.sseClient = res; + for (const event of session.pending) writeSse(res, event); + session.pending = []; +} + +export function detachSseClient(id: string): void { + const session = sessions.get(id); + if (session) session.sseClient = null; +} + +/** Summarize messages with the cheap model. Best-effort — null on any failure. */ +async function summarizeMessages(apiKey: string, msgs: Message[]): Promise<string | null> { + if (!apiKey || msgs.length === 0) return null; + const text = msgs + .map((m) => `${m.role}: ${String(m.content).slice(0, 2000)}`) + .join('\n\n'); + try { + const res = await fetch(`${OPENROUTER_BASE}/chat/completions`, { + method: 'POST', + headers: { Authorization: `Bearer ${apiKey}`, 'Content-Type': 'application/json' }, + body: JSON.stringify({ + model: SUMMARY_MODEL, + stream: false, + temperature: 0, + max_tokens: 900, + messages: [ + { role: 'system', content: SUMMARY_SYSTEM }, + { role: 'user', content: `Messages to summarize:\n${text}\n\nReturn the updated summary.` }, + ], + }), + }); + if (!res.ok) return null; + const j = await res.json() as { choices?: { message?: { content?: string } }[] }; + return j?.choices?.[0]?.message?.content?.trim() || null; + } catch { + return null; + } +} + +/** + * Re-compact a long-running session's in-memory history: fold older turns into + * one summary message, keep the most recent verbatim. Bounds context for + * sessions that live for hours without a restart. Best-effort — leaves history + * untouched if summarization fails. + */ +function historyChars(h: Message[]): number { + let n = 0; + for (const m of h) { + n += typeof m.content === 'string' ? m.content.length : JSON.stringify(m.content ?? '').length; + } + return n; +} + +async function compactHistoryIfNeeded(session: Session): Promise<void> { + const h = session.history; + // Trigger on message count OR total size — whichever hits first. + const over = h.length > HISTORY_COMPACT_TRIGGER || historyChars(h) > HISTORY_COMPACT_CHARS; + if (!over) return; + // Need enough older messages to make folding worthwhile. + if (h.length <= HISTORY_KEEP_RECENT + 1) return; + const recent = h.slice(-HISTORY_KEEP_RECENT); + const older = h.slice(0, -HISTORY_KEEP_RECENT); + const summary = await summarizeMessages(session.opts.apiKey, older); + if (!summary) return; + session.history = [ + { role: 'system', content: `Summary of earlier conversation (compacted):\n${summary}` } as Message, + ...recent, + ]; +} + +/** + * Make dropped files usable by this turn's agent, and describe them to the model. + * + * - Images → base64 data URLs returned in `images`, shown as vision blocks so + * the model literally sees them (e.g. a screenshot). + * - Other files → made readable, then pointed at by path in `noticeText`: + * · Secure: copied into the workspace dir (appears at /workspace/<name>). + * · Work/Incognito: the file already lives on the host; we grant the + * session read access to its folder and give the model the absolute path. + * + * `noticeText` is appended to the user's message so the model KNOWS a file was + * attached (the bug being fixed: previously the agent had no idea) and how to + * reach it. Best-effort per file — a failure to stage one is reported inline, + * never throws. + */ +function stageAttachments( + session: Session, + attachments: RuntimeAttachment[] | undefined, +): { noticeText: string; images: string[] } { + if (!attachments?.length) return { noticeText: '', images: [] }; + const images: string[] = []; + const lines: string[] = []; + + for (const att of attachments) { + if (att.mimeType.startsWith('image/')) { + try { + const b64 = readFileSync(att.path).toString('base64'); + images.push(`data:${att.mimeType};base64,${b64}`); + lines.push(`- "${att.fileName}" — image, shown to you directly in this message.`); + continue; + } catch { + // Fall through and treat it as a regular file (path notice only). + } + } + + if (session.opts.secure && session.secureWorkspaceDir) { + const safe = basename(att.fileName) || basename(att.path); + try { + copyFileSync(att.path, join(session.secureWorkspaceDir, safe)); + lines.push(`- "${att.fileName}" (${att.mimeType}) — in /workspace; read_file (path "${safe}") only if you need its contents.`); + } catch (err) { + lines.push(`- "${att.fileName}" — could not be staged (${err instanceof Error ? err.message : String(err)}).`); + } + } else { + // Work / Incognito read the host filesystem directly. Grant this session + // read access to the upload's folder so validatePath lets read_file in. + const dir = dirname(att.path); + if (!session.opts.allowedFolders.includes(dir)) { + session.opts.allowedFolders = [...session.opts.allowedFolders, dir]; + } + lines.push(`- "${att.fileName}" (${att.mimeType}) — available at "${att.path}"; read_file it only if you need its contents.`); + } + } + + const noticeText = lines.length + ? `\n\n[The user attached ${attachments.length} file(s) to THIS message:\n${lines.join('\n')}\nUse them if relevant to the request.]` + : ''; + return { noticeText, images }; +} + +export async function sendMessage(sessionId: string, content: string, networkEnabled?: boolean, ttlDays?: number, attachments?: RuntimeAttachment[], copyFolders?: string[]): Promise<void> { + const session = sessions.get(sessionId); + if (!session) throw new Error(`Session not found: ${sessionId}`); + + // Reset TTL countdown on activity + session.lastActivity = Date.now(); + if (ttlDays !== undefined) session.ttlMs = ttlDays <= 0 ? 0 : ttlDays * 86400_000; + + // Safe Mode: merge in any folders the user added since the session started, so + // a mid-chat addition gets copied into the sandbox on this turn (not silently + // dropped). The ingest step below copies only the not-yet-ingested ones. + if (copyFolders?.length && session.opts.secure) { + const merged = new Set([...(session.opts.copyFolders ?? []), ...copyFolders]); + session.opts.copyFolders = [...merged]; + } + + // Per-message network override + if (networkEnabled !== undefined && session.opts.secure) { + session.opts.networkEnabled = networkEnabled; + } + + // Persist the refreshed activity/TTL so it survives a runtime restart. + persistMeta(session); + + // Keep a long-running session's context bounded (summarize old turns). + await compactHistoryIfNeeded(session); + + // Fresh abort controller for this turn (user Stop → abortSession()). + const abort = new AbortController(); + session.abort = abort; + + // Make any dropped files reachable + tell the model about them. `messageText` + // (content + notice) is what goes to the model AND into history, so follow-up + // turns remember a file was shared; `images` ride as vision blocks for one turn. + const { noticeText, images } = stageAttachments(session, attachments); + const messageText = content + noticeText; + + if (session.opts.secure) { + const workspaceDir = session.secureWorkspaceDir!; + const netEnabled = session.opts.networkEnabled ?? false; + + // COPY the user's chosen folders into the isolated workspace — Safe Mode + // works on a copy, so the originals are never touched. Each folder is copied + // once (tracked in ingestedFolders), so folders added mid-chat are picked up + // on the next turn. The summary is prepended so the model knows what's in + // /workspace and tells the user their files are unchanged. + let turnText = messageText; + const ingested = (session.ingestedFolders ??= new Set<string>()); + const pending = (session.opts.copyFolders ?? []).filter((p) => !ingested.has(p)); + if (pending.length) { + // Mark up front so a failed copy isn't retried every turn. + for (const p of pending) ingested.add(p); + const sources: IngestSource[] = pending.map((p) => ({ kind: 'folder', path: p })); + const results = await ingestSources(workspaceDir, sources); + for (const r of results) { + if (!r.ok) emit(session, { type: 'error', message: `Sandbox ingest: ${r.source} — ${r.error}` }); + } + const summary = describeIngest(results); + if (summary) turnText = `${summary}\n\n${messageText}`; + } + + // A Safe turn IS the sandbox, so it needs Docker now — start it ourselves if + // it's down (it'll auto-stop when idle). Only fail if it's missing/unstartable. + if (!(await ensureDockerForTask())) { + emit(session, { + type: 'error', + message: 'Safe work needs Docker, which isn’t running and couldn’t be started automatically. Install or start Docker, then try again.', + }); + return; + } + markDockerUse(); + + // Warm reuse: get (or lazily start) the session's sandbox container. + // Fail-closed — if Docker can't start it, surface an error, don't run. + let containerName: string; + try { + containerName = await ensureSecureContainer(session, netEnabled); + } catch (err) { + emit(session, { type: 'error', message: err instanceof Error ? err.message : String(err) }); + return; + } + + const secureGen = runSecureTurn( + session.history.map((m) => ({ role: m.role as string, content: String(m.content) })), + turnText, + { + apiKey: session.opts.apiKey, + model: session.opts.model, + workspaceDir, + containerName, + networkEnabled: netEnabled, + systemPrompt: session.opts.systemPrompt, + signal: abort.signal, + images, + }, + ); + let assistantText = ''; + // Pin the container so the idle reaper / LRU never frees it mid-turn. + if (session.secureContainer) session.secureContainer.inUse = true; + try { + for await (const event of secureGen) { + emit(session, event as AgentEvent); + if (event.type === 'text') assistantText += event.content; + } + } finally { + if (session.secureContainer) { + session.secureContainer.inUse = false; + session.secureContainer.lastUsed = Date.now(); + } + if (session.abort === abort) session.abort = undefined; + } + if (assistantText) { + session.history.push({ role: 'user', content: turnText }); + session.history.push({ role: 'assistant', content: assistantText }); + } + return; + } + + // run_command runs in a Docker sandbox with per-folder :ro/:rw mounts so the + // kernel enforces read-only. Without Docker we leave runShell undefined → + // run_command is disabled (file tools still work). The container is created + // lazily on first command, so chat/read-only turns never spin one up. + // + // Only EXPLICITLY chosen folders are mounted — never the broad $HOME fallback + // (mounting all of home would expose ~/.ssh etc. to the shell). Each folder is + // validated against the secret deny-list and dropped (with a notice) if it + // names a sensitive root. + // Incognito forces every mounted folder to :ro — the shell may read but the + // kernel rejects all writes, matching the read-only contract. + const incognito = !!session.opts.incognito; + let runShell: ((command: string, cwd: string) => Promise<string>) | undefined; + let linkSandbox: ((command: string) => Promise<string>) | undefined; + + // Validate the explicitly-chosen folders into bind mounts (empty when none + // selected). This is host-only (no Docker needed), so it always runs. + const validated: { path: string; readonly: boolean }[] = []; + for (const f of session.opts.folderAccess ?? []) { + try { + validated.push({ path: validateMountRoot(f.path), readonly: incognito ? true : f.readonly }); + } catch (err) { + emit(session, { + type: 'error', + message: `Folder excluded from commands: ${err instanceof Error ? err.message : String(err)}`, + }); + } + } + const mounts: WorkMount[] = toWorkMounts(validated); + + // We DON'T gate on Docker being up. Docker is started lazily inside the tool, + // the first moment a task actually needs it (ensureDockerForTask) — so pure + // file-edit / read-only turns never touch Docker, and the user is never nagged + // to start it up front. If Docker is missing or can't be started, the tool + // returns a guidance message and file tools still work. + const ensureSandbox = async (): Promise<boolean> => { + if (!(await ensureDockerForTask())) return false; + markDockerUse(); + return true; + }; + // analyze_link runs yt-dlp inside the session's container (warm-reused, so + // yt-dlp self-installs once) — never on the host. Offered even with no folders + // (network-only container). Runs at "/" (it writes to its own scratch paths). + linkSandbox = async (command) => { + if (!(await ensureSandbox())) { + return 'analyze_link needs a Docker sandbox, which isn’t running and couldn’t be started. Use web_fetch/web_search instead.'; + } + const image = await resolveWorkImage(); + const name = await ensureWorkContainer(session, mounts, image); + return execInWorkContainer(name, command, '/', { mounts }); + }; + // run_command needs a real folder to operate in, so it stays gated on mounts. + if (mounts.length) { + runShell = async (command, cwd) => { + if (!(await ensureSandbox())) { + return 'run_command needs a Docker sandbox. Docker isn’t running and couldn’t be started automatically — start (or install) Docker and try again. File read/write tools still work without it.'; + } + const image = await resolveWorkImage(); + const name = await ensureWorkContainer(session, mounts, image); + // scan: report created/modified/deleted files after the command so the + // user sees exactly what changed inside their allowed folders. + return execInWorkContainer(name, command, cwd, { mounts, scan: true }); + }; + } + + const gen = runAgentTurn(session.history, messageText, { + apiKey: session.opts.apiKey, + model: session.opts.model, + allowedFolders: session.opts.allowedFolders, + folderAccess: session.opts.folderAccess, + runShell, + linkSandbox, + incognito, + systemPrompt: session.opts.systemPrompt, + signal: abort.signal, + images, + onWriteApproval: async (filePath, fileContent) => { + emit(session, { type: 'approval_request', changeId: randomUUID(), path: filePath, content: fileContent }); + return true; + }, + }); + + let assistantText = ''; + try { + for await (const event of gen) { + emit(session, event); + if (event.type === 'text') assistantText += event.content; + } + } finally { + // Release the warm container so the idle reaper / LRU can free it. + if (session.workContainer) { + session.workContainer.inUse = false; + session.workContainer.lastUsed = Date.now(); + } + if (session.abort === abort) session.abort = undefined; + } + + if (assistantText) { + session.history.push({ role: 'user', content: messageText }); + session.history.push({ role: 'assistant', content: assistantText }); + } +} + +function emit(session: Session, event: AgentEvent): void { + if (session.sseClient && !session.sseClient.writableEnded) { + writeSse(session.sseClient, event); + } else { + session.pending.push(event); + } +} + +function writeSse(res: http.ServerResponse, event: AgentEvent): void { + try { res.write(`data: ${JSON.stringify(event)}\n\n`); } catch {} +} diff --git a/packages/iclaw-runtime/src/work-container.ts b/packages/iclaw-runtime/src/work-container.ts new file mode 100644 index 0000000..7bddd26 --- /dev/null +++ b/packages/iclaw-runtime/src/work-container.ts @@ -0,0 +1,464 @@ +/** + * Work Mode command sandbox. + * + * Work Mode = "I trust this folder. Work on the REAL files, but only inside it." + * The agent loop and the file tools (read/write/list/search) run on the host — + * they're path-checked (validatePath) and gated by per-write approval. The one + * tool that can't be safely contained on the host is `run_command`: arbitrary + * `bash -c` can write/delete anywhere, ignoring our per-folder flags. + * + * So `run_command` is routed into a Docker container. Each allowed folder is + * bind-mounted to a NORMALIZED container path — `/work/0`, `/work/1`, … — with + * `:ro` or `:rw` per its access level. This scheme is identical on macOS, Linux + * and Windows (no same-path mounts, which can't work on Windows where the host + * path is `C:\…`). The kernel enforces the flags: a write into a `:ro` mount + * fails with "Read-only file system", and anything outside the mounts is simply + * invisible — the command literally cannot see the rest of the computer. + * + * The model speaks in HOST paths (the file tools and the folder list it sees use + * them), so run_command's host `cwd` and any host folder roots in the command + * are translated to their `/work/<n>` equivalents before exec — see + * hostToContainer / translateCommandPaths. + * + * After each command we scan the writable mounts and report which files were + * created / modified / deleted, reassuring the user that all changes stayed + * inside the allowed folders. + * + * When Docker is unavailable, `run_command` is disabled (strict fallback) — the + * host file tools still work, so read-only stays an honest guarantee. + */ +import { execFile } from 'node:child_process'; +import { randomUUID } from 'node:crypto'; +import { promisify } from 'node:util'; + +import { log } from './log.js'; +import { INSTALL_LABEL } from './install-id.js'; + +const execFileAsync = promisify(execFile); + +const WORK_PREFIX = 'iclaw-work-'; +const COMMAND_TIMEOUT = Number(process.env.ICLAW_WORK_COMMAND_TIMEOUT) || 60_000; +// `docker run` auto-pulls the image on first use; allow time for that. +const START_TIMEOUT = Number(process.env.ICLAW_WORK_START_TIMEOUT) || 300_000; +/** Cap the per-command file scan so a huge tree can't stall the turn. */ +const SCAN_MAX_FILES = Number(process.env.ICLAW_WORK_SCAN_MAX_FILES) || 20_000; +/** Cap how many changed paths we list back (the rest collapse to "+N more"). */ +const SCAN_MAX_REPORT = 50; + +// The shared iClaw sandbox image — ONE image for both Safe work and Work mode +// (container/secure-sandbox.Dockerfile, 80/20 toolset). Resolved lazily. +const SANDBOX_IMAGE = process.env.ICLAW_SECURE_IMAGE || 'iclaw-secure:latest'; +// Last-resort fallback when the shared image hasn't been built yet: a public +// base that already ships bash, git and node, so Work still runs (with a leaner +// toolset) on a fresh checkout. Auto-pulled by `docker run`. +const FALLBACK_IMAGE = process.env.ICLAW_WORK_SLIM_IMAGE || 'node:22'; + +/** + * One allowed folder, mapped from its host path to a normalized container path. + * `label` (the folder's basename) is only used to make reports readable. + */ +export interface WorkMount { + /** Validated absolute host path (the source of the bind mount). */ + path: string; + /** Normalized in-container mount point: `/work/<n>`. */ + containerPath: string; + /** Read-only (`:ro`) when true, read & write (`:rw`) otherwise. */ + readonly: boolean; + /** Folder basename, for human-readable change reports. */ + label: string; +} + +/** Folder basename for a host path (handles both separators). */ +function basenameOf(hostPath: string): string { + const parts = hostPath.replace(/[\\/]+$/, '').split(/[\\/]/); + return parts[parts.length - 1] || hostPath; +} + +/** + * Assign normalized `/work/<n>` container paths to validated allowed folders. + * Order is stable (index = position), so the same folder set always maps the + * same way within a session. + */ +export function toWorkMounts(folders: { path: string; readonly: boolean }[]): WorkMount[] { + return folders.map((f, i) => ({ + path: f.path, + containerPath: `/work/${i}`, + readonly: f.readonly, + label: basenameOf(f.path), + })); +} + +/** Path key for prefix comparison: forward slashes; lowercased on Windows. */ +function normKey(p: string, plat: NodeJS.Platform): string { + const slashed = p.replace(/\\/g, '/').replace(/\/+$/, ''); + return plat === 'win32' ? slashed.toLowerCase() : slashed; +} + +/** + * Map a host path (cwd or any path under an allowed folder) to its container + * path. Returns null when the path isn't inside any mount — the caller then + * refuses, so a command can never `cd` outside the allowed folders. + */ +export function hostToContainer( + hostPath: string, + mounts: WorkMount[], + plat: NodeJS.Platform = process.platform, +): string | null { + const target = normKey(hostPath, plat); + // Longest root first so a nested mount wins over its parent. + const sorted = [...mounts].sort((a, b) => b.path.length - a.path.length); + for (const m of sorted) { + const root = normKey(m.path, plat); + if (target === root) return m.containerPath; + if (target.startsWith(root + '/')) { + return m.containerPath + target.slice(root.length); + } + } + return null; +} + +/** Map a container path (`/work/<n>/…`) back to its host path, for reports. */ +export function containerToHost(containerPath: string, mounts: WorkMount[]): string { + for (const m of mounts) { + if (containerPath === m.containerPath) return m.path; + if (containerPath.startsWith(m.containerPath + '/')) { + const rel = containerPath.slice(m.containerPath.length + 1); + const sep = m.path.includes('\\') ? '\\' : '/'; + return m.path.replace(/[\\/]+$/, '') + sep + rel.replace(/\//g, sep); + } + } + return containerPath; +} + +function escapeRegex(s: string): string { + return s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); +} + +/** + * Rewrite host folder roots inside a shell command to their container paths. + * + * The model emits host paths (that's all it sees), so a command may reference + * `~/Projects/app/...` or `C:\Users\me\app\...`. We replace each allowed-folder + * root (in both native and forward-slash form) with its `/work/<n>` mount and + * normalize the tail to forward slashes, stopping at whitespace, quotes, and + * shell metacharacters so a path can't swallow the rest of the command. The + * structured `cwd` is mapped separately (hostToContainer); this catches inline + * absolute paths in the command body. + */ +export function translateCommandPaths( + command: string, + mounts: WorkMount[], + plat: NodeJS.Platform = process.platform, +): string { + let out = command; + const flags = plat === 'win32' ? 'gi' : 'g'; + // Tail must start with a separator (or be empty) so the root only matches at a + // path boundary — `/a` must not match inside `/abc`. The lookahead then + // asserts the token really ends (whitespace, quote, shell metachar, or EOL). + const TAIL = "((?:[/\\\\][^\\s\"'`&|;<>()]*)?)"; + const BOUND = "(?=[\\s\"'`&|;<>()]|$)"; + // Longest root first so a nested mount wins over its parent. + const sorted = [...mounts].sort((a, b) => b.path.length - a.path.length); + for (const m of sorted) { + for (const variant of new Set([m.path, m.path.replace(/\\/g, '/')])) { + const esc = escapeRegex(variant); + out = out.replace( + new RegExp(esc + TAIL + BOUND, flags), + (_full, tail: string) => m.containerPath + (tail || '').replace(/\\/g, '/'), + ); + } + } + return out; +} + +let dockerOk: boolean | null = null; +let resolvedImage: string | null = null; + +/** True if a usable Docker daemon is reachable. Cached after the first probe. */ +export async function dockerAvailable(): Promise<boolean> { + if (dockerOk !== null) return dockerOk; + try { + await execFileAsync('docker', ['info'], { timeout: 8_000 }); + dockerOk = true; + } catch { + dockerOk = false; + } + return dockerOk; +} + +async function imageExists(image: string): Promise<boolean> { + try { + await execFileAsync('docker', ['image', 'inspect', image], { timeout: 8_000 }); + return true; + } catch { + return false; + } +} + +/** + * Pick the image for Work Mode commands. Order: + * 1. ICLAW_WORK_IMAGE — explicit override (e.g. a heavier python/go toolchain). + * 2. The shared sandbox image (iclaw-secure:latest) — the canonical choice, + * identical to what Safe work uses, so one build serves both modes. + * 3. Fallback public base (node:22) only when the shared image isn't built — + * logged as a warning so the leaner toolset isn't a silent surprise. + * Cached after the first resolve. + */ +export async function resolveWorkImage(): Promise<string> { + if (resolvedImage) return resolvedImage; + if (process.env.ICLAW_WORK_IMAGE) { + resolvedImage = process.env.ICLAW_WORK_IMAGE; + } else if (await imageExists(SANDBOX_IMAGE)) { + resolvedImage = SANDBOX_IMAGE; + } else { + log.warn( + 'Shared sandbox image not built — Work Mode falling back to a leaner base. ' + + 'Build it with `npm run build:secure-image` for the full toolset.', + { fallback: FALLBACK_IMAGE, sandbox: SANDBOX_IMAGE }, + ); + resolvedImage = FALLBACK_IMAGE; + } + return resolvedImage; +} + +/** + * Start a long-lived Work container with the given folder mounts. Returns the + * container name. Fail-closed: throws if Docker can't start it, so the caller + * keeps run_command disabled instead of silently running unsandboxed. + */ +export async function startWorkContainer(mounts: WorkMount[], image: string): Promise<string> { + const name = `${WORK_PREFIX}${randomUUID().slice(0, 8)}`; + const mountArgs: string[] = []; + for (const m of mounts) { + // Source = native host path (the Docker CLI handles the `X:\` drive prefix + // on Windows); target = normalized `/work/<n>` mount point. + mountArgs.push('-v', `${m.path}:${m.containerPath}:${m.readonly ? 'ro' : 'rw'}`); + } + // Default working dir: first writable folder, else the first mount. + const home = mounts.find((m) => !m.readonly) ?? mounts[0]; + + // Run the command process as the HOST user, so files it creates in the user's + // real (bind-mounted) folders are owned by the user — not root or the image's + // uid 1000. Without this, run_command output in a :rw folder lands root-owned + // and the user can't edit/delete it (visible on Linux; on macOS the engine's + // VM file-sharing remaps ownership so it's hidden there). Skip when we'd add no value or can't: + // running as root already (uid 0), already matching the image user (1000), or + // Windows (getuid undefined). HOME=/tmp gives the now-nameless uid a writable + // home for tool caches (npm, git) since /home/node isn't writable by it. + const uid = process.getuid?.(); + const gid = process.getgid?.(); + const userArgs = + uid != null && uid !== 0 && uid !== 1000 + ? ['--user', `${uid}:${gid ?? uid}`, '-e', 'HOME=/tmp'] + : []; + + try { + await execFileAsync('docker', [ + 'run', '--rm', '-d', + '--name', name, + '--label', INSTALL_LABEL, + '--memory', process.env.ICLAW_WORK_MEMORY || '768m', + '--cpus', process.env.ICLAW_WORK_CPUS || '1', + ...userArgs, + ...mountArgs, + ...(home ? ['--workdir', home.containerPath] : []), + image, + 'sleep', '86400', + ], { timeout: START_TIMEOUT }); + } catch (err) { + const e = err as { stderr?: string; message?: string }; + throw new Error( + `Work sandbox failed to start — is Docker running? ${(e.stderr || e.message || '').slice(0, 200)}`, + ); + } + return name; +} + +export interface ExecOptions { + /** Folder mappings — used to translate the host cwd + command paths. */ + mounts?: WorkMount[]; + /** Run a post-command change scan and append a report (Work run_command). */ + scan?: boolean; +} + +/** + * Execute a command inside a Work container at the given HOST cwd. The cwd and + * any host folder roots in the command are translated to their `/work/<n>` + * container paths first; a cwd outside every mount is refused (the command can + * never operate outside the allowed folders). With `scan`, a before/after diff + * of the writable mounts is appended so the user sees exactly what changed. + */ +export async function execInWorkContainer( + name: string, + command: string, + hostCwd: string, + opts: ExecOptions = {}, +): Promise<string> { + const mounts = opts.mounts ?? []; + // Translate the host cwd → container path. '/' (linkSandbox) and already- + // container paths fall through unchanged. + let containerCwd = hostCwd; + if (mounts.length) { + const mapped = hostToContainer(hostCwd, mounts); + if (mapped) { + containerCwd = mapped; + } else if (!hostCwd.startsWith('/work/') && hostCwd !== '/') { + return `Refused: working directory "${hostCwd}" is outside the folders you allowed for this chat.`; + } + } + const containerCmd = translateCommandPaths(command, mounts); + + const before = opts.scan ? await snapshotWritable(name, mounts) : null; + const output = await rawExec(name, containerCmd, containerCwd); + if (!before) return output; + + const after = await snapshotWritable(name, mounts); + const report = renderChangeReport(before, after, mounts); + return report ? `${output}\n\n${report}` : output; +} + +/** Bare `docker exec` — returns combined stdout/stderr (or the error text). */ +async function rawExec(name: string, command: string, cwd: string): Promise<string> { + const dev = process.env.ICLAW_DEV_MODE === 'true'; + const startedAt = Date.now(); + // Don't dump giant commands (e.g. base64-injected tool scripts like + // social_search) into the log — keep the head + the useful env/arg tail. + const cmdLog = command.length > 400 + ? `${command.slice(0, 120)} …[+${command.length - 340} chars]… ${command.slice(-220)}` + : command; + try { + const { stdout, stderr } = await execFileAsync( + 'docker', ['exec', '--workdir', cwd, name, 'bash', '-lc', command], + { timeout: COMMAND_TIMEOUT }, + ); + const out = [stdout, stderr].filter(Boolean).join('\n').trim() || '(no output)'; + if (dev) log.info('run_command', { cwd, code: 0, ms: Date.now() - startedAt, bytes: out.length, cmd: cmdLog }); + return out; + } catch (err: unknown) { + const e = err as { stdout?: string; stderr?: string; message?: string; killed?: boolean; signal?: string; code?: number }; + // execFile rejects on a non-zero exit AND on the timeout kill — tell them + // apart: a SIGTERM/SIGKILL with `killed` is OUR timeout firing, not the + // command exiting on its own. + const timedOut = Boolean(e.killed) || e.signal === 'SIGTERM' || e.signal === 'SIGKILL'; + const partial = [e.stdout, e.stderr].filter(Boolean).join('\n').trim(); + if (dev) { + log.warn('run_command failed', { + cwd, ms: Date.now() - startedAt, timedOut, + killed: Boolean(e.killed), signal: e.signal ?? null, code: e.code ?? null, + bytes: partial.length, cmd: cmdLog, + }); + } + // CRITICAL (not dev-gated): a timeout kill must be visible to the MODEL. + // Otherwise it reads the near-empty output of a hung command (e.g. an + // `npm install` with no container network) as "ran fine, did nothing" and + // misdiagnoses it as a permissions/sandbox bug — exactly what happened in + // practice. State plainly that the command was killed and did NOT finish. + if (timedOut) { + return (partial ? partial + '\n\n' : '') + + `[command killed after ${Math.round(COMMAND_TIMEOUT / 1000)}s — it timed out and did NOT finish. ` + + `Most likely it hung on the network (npm/pip/git/curl with no connectivity in this sandbox) or on an ` + + `interactive prompt. Do not assume anything it was downloading or building completed.]`; + } + return partial || e.message || 'Error'; + } +} + +type FileSnapshot = Map<string, string>; // container path → mtime (epoch float) + +/** + * Snapshot files in the WRITABLE mounts (read-only mounts can't change, so we + * skip them). Excludes .git / node_modules churn so the report stays meaningful. + * Best-effort: returns an empty snapshot if `find` fails or the tree is huge. + */ +async function snapshotWritable(name: string, mounts: WorkMount[]): Promise<FileSnapshot> { + const snap: FileSnapshot = new Map(); + const writable = mounts.filter((m) => !m.readonly); + if (writable.length === 0) return snap; + const roots = writable.map((m) => m.containerPath); + // GNU find (the container is always Linux): print "<mtime>\t<path>" per file. + const findCmd = + `find ${roots.map((r) => `'${r}'`).join(' ')} -type f ` + + `-not -path '*/.git/*' -not -path '*/node_modules/*' ` + + `-printf '%T@\\t%p\\n' 2>/dev/null | head -n ${SCAN_MAX_FILES}`; + try { + const { stdout } = await execFileAsync( + 'docker', ['exec', name, 'bash', '-lc', findCmd], + { timeout: COMMAND_TIMEOUT }, + ); + for (const line of stdout.split('\n')) { + const tab = line.indexOf('\t'); + if (tab === -1) continue; + snap.set(line.slice(tab + 1), line.slice(0, tab)); + } + } catch { + /* best-effort — an empty/partial snapshot just yields a sparser report. */ + } + return snap; +} + +interface ChangeSet { + created: string[]; + modified: string[]; + deleted: string[]; +} + +function diffSnapshots(before: FileSnapshot, after: FileSnapshot): ChangeSet { + const created: string[] = []; + const modified: string[] = []; + const deleted: string[] = []; + for (const [p, mtime] of after) { + const prev = before.get(p); + if (prev === undefined) created.push(p); + else if (prev !== mtime) modified.push(p); + } + for (const p of before.keys()) if (!after.has(p)) deleted.push(p); + return { created, modified, deleted }; +} + +/** Human-readable change block appended to run_command output (host paths). */ +function renderChangeReport( + before: FileSnapshot, + after: FileSnapshot, + mounts: WorkMount[], +): string { + const { created, modified, deleted } = diffSnapshots(before, after); + const total = created.length + modified.length + deleted.length; + const lines: string[] = ['── File changes (Work sandbox) ──']; + if (total === 0) { + lines.push('No files were created, modified, or deleted.'); + return lines.join('\n'); + } + const section = (mark: string, label: string, paths: string[]): void => { + if (paths.length === 0) return; + const shown = paths.slice(0, SCAN_MAX_REPORT); + for (const p of shown) lines.push(`${mark} ${label}: ${containerToHost(p, mounts)}`); + if (paths.length > shown.length) { + lines.push(` …and ${paths.length - shown.length} more ${label} file(s)`); + } + }; + section('+', 'created', created); + section('~', 'modified', modified); + section('-', 'deleted', deleted); + // By construction the shell only sees mounted folders, so every change above + // is inside an allowed folder — state it plainly for reassurance. + lines.push('All changes were inside the folders you allowed.'); + return lines.join('\n'); +} + +/** Kill Work containers orphaned by a previous runtime process. */ +export async function killOrphanWorkContainers(): Promise<number> { + try { + // Scope to THIS install's Work containers: name prefix AND install label are + // ANDed, so a second iClaw install never reaps ours (nor we theirs). + const { stdout } = await execFileAsync( + 'docker', ['ps', '-aq', '--filter', `name=${WORK_PREFIX}`, '--filter', `label=${INSTALL_LABEL}`], + { timeout: 10_000 }, + ); + const ids = stdout.split('\n').map((s) => s.trim()).filter(Boolean); + if (ids.length === 0) return 0; + await execFileAsync('docker', ['rm', '-f', ...ids], { timeout: 30_000 }).catch(() => {}); + return ids.length; + } catch { + return 0; + } +} diff --git a/packages/iclaw-runtime/tsconfig.json b/packages/iclaw-runtime/tsconfig.json new file mode 100644 index 0000000..c83c533 --- /dev/null +++ b/packages/iclaw-runtime/tsconfig.json @@ -0,0 +1,20 @@ +{ + "compilerOptions": { + "target": "ES2022", + "module": "NodeNext", + "moduleResolution": "NodeNext", + "lib": ["ES2022"], + "outDir": "./dist", + "rootDir": "./src", + "strict": true, + "esModuleInterop": true, + "skipLibCheck": true, + "forceConsistentCasingInFileNames": true, + "resolveJsonModule": true, + "declaration": true, + "declarationMap": true, + "sourceMap": true + }, + "include": ["src/**/*"], + "exclude": ["node_modules", "dist"] +} diff --git a/public/css/style.css b/public/css/style.css index 538de1d..152355a 100644 --- a/public/css/style.css +++ b/public/css/style.css @@ -874,6 +874,16 @@ code { display: none !important; } +/* Header tools toggle between the draft (new-chat) and live-chat states via the + `hidden` attribute — these carry explicit `display`, so the UA [hidden] rule + alone wouldn't win. Agent also hides in non-Full-Power modes (see iclaw.js). */ +.agent-form[hidden], +.share-form[hidden], +.delete-form[hidden], +.share-btn[hidden] { + display: none !important; +} + /* All header controls (selects, toggles, buttons) standardise on 32px height + 10px radius so the row reads as one consistent strip of pills. */ .chat-header .btn--sm { @@ -882,34 +892,6 @@ code { border-radius: 10px; } -.reasoning-toggle { - display: inline-flex; - align-items: center; - gap: 8px; - height: 32px; - padding: 0 10px; - font-size: 0.78rem; - color: var(--muted); - border-radius: 10px; - border: 1px solid color-mix(in srgb, var(--border) 60%, var(--header-surface)); - background: transparent; - cursor: pointer; - user-select: none; - transition: background 0.15s ease, border-color 0.15s ease, color 0.12s ease; -} -.reasoning-toggle:hover { - color: var(--text); - background: color-mix(in srgb, var(--hover) 50%, transparent); - border-color: color-mix(in srgb, var(--border) 95%, var(--header-surface)); -} -.reasoning-toggle input[type="checkbox"] { - margin: 0; - width: 14px; - height: 14px; - accent-color: var(--md-link); - cursor: pointer; -} - /* ---------- Exec approval card ---------- */ .exec-approval-card { @@ -981,33 +963,6 @@ code { opacity: 0.7; } -/* ---------- Reasoning block ---------- */ - -.reasoning-block { - border-left: 3px solid color-mix(in srgb, var(--text) 25%, transparent); - background: color-mix(in srgb, var(--text) 3%, var(--surface)); - padding: 8px 12px; - font-size: 0.88em; - color: var(--muted); - font-style: italic; -} -.reasoning-block .role { - font-size: 0.75em; - text-transform: lowercase; - opacity: 0.7; - margin-bottom: 4px; -} -.reasoning-block .reasoning-body { - white-space: pre-wrap; - word-break: break-word; - font-family: inherit; -} -.reasoning-block.active::after { - content: "…"; - margin-left: 4px; - opacity: 0.5; -} - /* ---------- Slash autocomplete menu ---------- */ /* Could be replaced wholesale by `.menu` / `.menu-item` primitives once we refactor iclaw.js to emit those class names. The position/anchor + the @@ -1067,28 +1022,6 @@ code { --_btn-bg-hover: color-mix(in srgb, var(--danger) 8%, transparent); } -.gateway { - font-size: 0.72rem; - padding: 4px 9px; - border-radius: 6px; - background: var(--msg-user-fill); - border: 1px solid var(--msg-user-stroke); - font-weight: 500; - letter-spacing: -0.01em; -} -.gateway.ok { color: var(--ok); border-color: var(--ok-border); } -.gateway.down { color: var(--down); border-color: var(--down-soft-border); } -.gateway.degraded { - color: var(--warn); - border-color: color-mix(in srgb, var(--warn) 40%, var(--border)); - background: color-mix(in srgb, var(--warn) 8%, var(--surface)); -} -.gateway.shutdown { - color: var(--down); - border-color: color-mix(in srgb, var(--down) 55%, var(--border)); - background: color-mix(in srgb, var(--down) 8%, var(--surface)); - font-weight: 600; -} /* ---------- Welcome / empty ---------- */ @@ -1101,12 +1034,13 @@ code { .empty-state { text-align: center; - margin: 48px auto 32px; - max-width: 22rem; + margin: 80px auto 32px; + max-width: 32rem; padding: 0 20px; font-size: 0.95rem; line-height: 1.5; letter-spacing: -0.01em; + white-space: nowrap; } /* ---------- Messages (ChatGPT-like thread) ---------- */ @@ -1223,6 +1157,31 @@ code { } /* ChatGPT hides role chrome; keep labels for screen readers only */ +/* Dev-mode token-usage badge (a subtle trailing line under a message). */ +.msg-tokens { + display: block; + margin-top: 4px; + font-size: 11px; + font-variant-numeric: tabular-nums; + letter-spacing: 0.01em; + color: color-mix(in srgb, var(--text) 42%, transparent); + user-select: none; +} +.msg.user .msg-tokens { text-align: right; } + +/* Dev-mode chat-wide token total (chat header). */ +.chat-token-total { + align-self: center; + flex: none; + padding: 2px 8px; + border-radius: 999px; + font-size: 11px; + font-variant-numeric: tabular-nums; + white-space: nowrap; + color: color-mix(in srgb, var(--text) 55%, transparent); + background: color-mix(in srgb, var(--text) 8%, transparent); +} + .msg .role { position: absolute; width: 1px; @@ -1245,7 +1204,8 @@ code { font-size: 0.8125rem; color: var(--muted); font-style: normal; - margin-bottom: 4px; + /* Status now sits BELOW the streamed text → put the gap above it. */ + margin-top: 4px; font-weight: 500; } .msg.assistant.streaming .stream-status::before { @@ -1699,21 +1659,53 @@ code { cursor: pointer; } -/* tables (GFM) */ -.msg-body table { - border-collapse: collapse; +/* tables (GFM) — wrapped in `.md-table-wrap` (see enhanceTables) so wide + tables scroll horizontally instead of breaking the thread layout, and the + rounded frame + zebra rows read cleanly. */ +.md-table-wrap { margin: 0.5em 0 1em; + max-width: 100%; + overflow-x: auto; + border: 1px solid var(--border); + border-radius: 10px; + -webkit-overflow-scrolling: touch; +} +.msg-body table { + /* `separate` (not `collapse`) so the wrapper's border-radius clips cleanly */ + border-collapse: separate; + border-spacing: 0; + margin: 0; + width: 100%; + min-width: max-content; font-size: 0.9em; + background: var(--surface); } .msg-body th, .msg-body td { - border: 1px solid var(--border); - padding: 6px 10px; + border-bottom: 1px solid var(--border); + border-right: 1px solid var(--border); + padding: 7px 12px; text-align: left; + vertical-align: top; } +/* edges: no double border against the rounded frame */ +.msg-body th:last-child, +.msg-body td:last-child { border-right: none; } +.msg-body tbody tr:last-child td { border-bottom: none; } .msg-body th { background: var(--th-bg); font-weight: 600; + white-space: nowrap; +} +.msg-body tbody tr:nth-child(even) { + background: color-mix(in srgb, var(--th-bg) 45%, transparent); +} +.msg-body tbody tr:hover { + background: color-mix(in srgb, var(--md-link) 8%, transparent); +} +/* compact, readable link text inside cells (full URL kept on the href + title) */ +.msg-body td a { + word-break: break-all; } /* When the assistant body is the streaming target, keep our markdown rules */ @@ -1731,56 +1723,102 @@ code { --composer-secret-strip-gap: var(--space-1); } +/* ── Composer field — new layout: textarea top, toolbar bottom ── */ .composer-field { - margin-top: -8px; margin-bottom: var(--space-4); display: flex; - align-items: flex-end; - gap: 6px; - padding: 6px 6px 6px 16px; + flex-direction: column; + padding: 12px 12px 8px 16px; background: var(--surface); border: 1px solid var(--border); - border-radius: 22px; - box-shadow: 0 1px 2px rgba(0, 0, 0, 0.04); - transition: border-color 0.15s ease, box-shadow 0.15s ease; + border-radius: 18px; + box-shadow: 0 1px 2px rgba(0,0,0,0.04); + transition: border-color 0.15s, box-shadow 0.15s; } .composer-field:focus-within { border-color: color-mix(in srgb, var(--text) 22%, var(--border)); - box-shadow: - 0 1px 2px rgba(0, 0, 0, 0.04), - 0 0 0 3px color-mix(in srgb, var(--text) 8%, transparent); + box-shadow: 0 1px 2px rgba(0,0,0,0.04), 0 0 0 3px color-mix(in srgb, var(--text) 8%, transparent); } +/* Textarea */ .composer-field .composer-input-stack { - flex: 1; - min-width: 0; - display: flex; - flex-direction: column; - align-items: stretch; + width: 100%; } - .composer-field .composer-input-stack textarea { - flex: 1; - min-width: 0; + width: 100%; margin: 0; - padding: 8px 0; + padding: 0 0 10px; border: none; background: transparent; - border-radius: 0; font-family: inherit; font-size: 0.95em; - line-height: 1.45; + line-height: 1.5; resize: none; min-height: 24px; max-height: 200px; field-sizing: content; } -.composer-field .composer-input-stack textarea:focus { - outline: none; +.composer-field .composer-input-stack textarea:focus { outline: none; } +.composer-field .composer-input-stack textarea::placeholder { color: var(--muted); opacity: 0.55; } + +/* Toolbar row */ +.composer-toolbar { + display: flex; + align-items: center; + justify-content: space-between; + gap: 4px; +} +.composer-toolbar__left { + display: flex; + align-items: center; + gap: 2px; +} +.composer-toolbar__right { + display: flex; + align-items: center; + gap: 4px; } -.composer-field .composer-input-stack textarea::placeholder { + +/* Generic toolbar button */ +.composer-toolbar-btn { + display: inline-flex; + align-items: center; + gap: 5px; + padding: 5px 8px; + border: none; + border-radius: 8px; + background: transparent; color: var(--muted); - opacity: 0.5; + font: inherit; + font-size: var(--text-sm); + cursor: pointer; + transition: background 0.12s, color 0.12s; + white-space: nowrap; +} +.composer-toolbar-btn:hover, +.composer-toolbar-btn[aria-expanded='true'] { + background: color-mix(in srgb, var(--text) 8%, transparent); + color: var(--text); +} + +/* Mode selector in toolbar */ +.composer-toolbar-modes { position: relative; } +.composer-mode-btn { + font-weight: 500; +} +.composer-mode-btn svg { opacity: 0.6; } +.composer-mode-menu { + bottom: calc(100% + 6px); + left: 0; + min-width: 264px; +} +.composer-mode-menu-item[aria-checked='true'] { + background: color-mix(in srgb, var(--accent) 10%, transparent); +} +.composer-mode-menu-item__desc { + font-size: var(--text-xs); + color: var(--muted); + line-height: 1.35; } .composer-send { @@ -3491,7 +3529,6 @@ code { flex-shrink: 0; } -.chat-header .reasoning-toggle .header-tool-icon, .chat-header .share-toggle .header-tool-icon { display: none; } @@ -3596,11 +3633,26 @@ code { background: transparent; } .project-pick-title { - margin: 0 0 22px; + margin: 0 0 6px; font-size: 1.05rem; font-weight: 650; letter-spacing: -0.02em; } +.project-pick-hint { + margin: 0 0 22px; + font-size: var(--text-sm); + color: var(--muted); +} +.project-pick-hint kbd { + font: inherit; + font-size: 0.8em; + padding: 1px 6px; + border: 1px solid var(--border); + border-bottom-width: 2px; + border-radius: 6px; + background: color-mix(in srgb, var(--text) 5%, var(--surface)); + color: var(--text); +} .project-pick-grid { display: flex; flex-wrap: wrap; @@ -4692,31 +4744,331 @@ code { /* ---------- Composer attachments ---------- */ -.composer-attach { +/* attach button inherits .composer-toolbar-btn */ +.composer-attach { padding: 5px 7px; border-radius: 8px; } +.composer-attach:active { transform: scale(0.94); } + +/* Work Mode folders button */ +.composer-network-toggle[hidden] { display: none; } +.composer-network-toggle[data-network='on'] { color: var(--accent); } +.composer-work-folders[hidden] { display: none; } +.composer-work-folders__count { + font-size: 11px; + font-weight: 600; + min-width: 14px; + text-align: center; +} + +/* Work folders modal */ +.work-folders-modal { + position: fixed; + inset: 0; + z-index: 200; + display: flex; + align-items: flex-end; + justify-content: center; + padding-bottom: 80px; +} +.work-folders-modal[hidden] { display: none; } +.work-folders-modal__backdrop { + position: absolute; + inset: 0; + background: rgba(0,0,0,0.4); +} +.work-folders-modal__panel { + position: relative; + background: var(--surface); + border: 1px solid var(--border); + border-radius: 12px; + padding: 20px; + width: min(480px, 94vw); + display: flex; + flex-direction: column; + gap: 12px; + box-shadow: 0 8px 32px rgba(0,0,0,0.2); +} +.work-folders-modal__head { } +.work-folders-modal__head-row { + display: flex; + align-items: flex-start; + justify-content: space-between; + gap: 12px; +} +.work-folders-modal__title { margin: 0; font-size: var(--text-base); font-weight: 600; } +.work-folders-modal__hint { margin: 4px 0 0; font-size: var(--text-sm); } +.work-folders-list { + list-style: none; + margin: 0; + padding: 0; + display: flex; + flex-direction: column; + gap: 6px; + max-height: 200px; + overflow-y: auto; +} +.work-folders-list:empty::before { + content: 'No folders added yet'; + color: var(--muted); + font-size: var(--text-sm); + display: block; + padding: 4px 0; +} +.work-folders-list__item { + display: flex; + align-items: center; + gap: 8px; + background: color-mix(in srgb, var(--text) 5%, transparent); + border-radius: 6px; + padding: 6px 10px; + font-size: var(--text-sm); + font-family: monospace; +} +.work-folders-list__path { flex: 1; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; } +.work-folders-list__access { + flex-shrink: 0; + border: 1px solid var(--border, color-mix(in srgb, var(--text) 18%, transparent)); + background: transparent; + color: var(--muted); + cursor: pointer; + border-radius: 999px; + padding: 2px 10px; + font-family: inherit; + font-size: var(--text-sm); + white-space: nowrap; +} +.work-folders-list__access[data-write="1"] { + color: var(--accent); + border-color: color-mix(in srgb, var(--accent) 45%, transparent); + background: color-mix(in srgb, var(--accent) 10%, transparent); +} +.work-folders-list__access:hover { color: var(--text); } +.work-folders-list__remove { + flex-shrink: 0; + border: none; + background: transparent; + color: var(--muted); + cursor: pointer; + padding: 0 2px; + font-size: 16px; + line-height: 1; +} +.work-folders-list__remove:hover { color: var(--text); } +.work-folders-add { + display: flex; + align-items: center; + gap: 8px; + flex-wrap: wrap; +} +.work-folders-add__or { + font-size: var(--text-sm); + flex-shrink: 0; +} +.work-folders-input { + flex: 1; + padding: 6px 10px; + border: 1px solid var(--border); + border-radius: 6px; + background: var(--bg); + color: var(--text); + font-size: var(--text-sm); + font-family: monospace; +} +.work-folders-input:focus { outline: none; border-color: var(--accent); } +.work-folders-modal__actions { display: flex; justify-content: flex-end; } + +/* Speech-to-text mic button — mirrors .composer-attach. Recording = red pulse; + busy (transcribing) swaps the mic for a spinner. */ +.composer-mic { flex: 0 0 auto; align-self: center; + position: relative; display: inline-flex; align-items: center; justify-content: center; width: 32px; height: 32px; margin: 0; - margin-left: -8px; padding: 0; border: none; border-radius: 50%; background: transparent; color: var(--muted); cursor: pointer; + touch-action: none; transition: background 0.15s ease, color 0.15s ease, transform 0.1s ease; } -.composer-attach:hover { +.composer-mic:hover { background: color-mix(in srgb, var(--text) 8%, transparent); color: var(--text); } -.composer-attach:active { transform: scale(0.94); } - -.composer-attach-menu, +.composer-mic:active { transform: scale(0.94); } +.composer-mic__busy { display: none; } +.composer-mic[data-state='recording'] { + color: var(--danger); + background: color-mix(in srgb, var(--danger) 12%, transparent); + animation: composer-mic-pulse 1.2s ease-in-out infinite; +} +.composer-mic[data-state='recording']:hover { + color: var(--danger); + background: color-mix(in srgb, var(--danger) 18%, transparent); +} +.composer-mic[data-state='busy'] { cursor: progress; } +.composer-mic[data-state='busy'] .composer-mic__idle { display: none; } +.composer-mic[data-state='busy'] .composer-mic__busy { + display: inline-flex; + animation: composer-mic-spin 0.8s linear infinite; +} +@keyframes composer-mic-pulse { + 0%, 100% { opacity: 1; } + 50% { opacity: 0.45; } +} +@keyframes composer-mic-spin { + to { transform: rotate(360deg); } +} + +/* --- Voice recording (hold-to-record) — UX ported from the Flutter app --- */ +/* Active/locked mic: filled accent with an amplitude halo driven by --mic-amp. */ +.composer-mic[data-state='recording'], +.composer-mic[data-state='locked'] { + color: #fff; + background: var(--accent); + animation: none; + box-shadow: 0 0 0 calc(var(--mic-amp, 0) * 10px) + color-mix(in srgb, var(--accent) 30%, transparent); +} +.composer-mic[data-state='recording']:hover, +.composer-mic[data-state='locked']:hover { + color: #fff; + background: var(--accent); +} +.composer-mic.is-cancel[data-state='recording'], +.composer-mic.is-cancel[data-state='recording']:hover { + background: var(--danger); + box-shadow: 0 0 0 calc(var(--mic-amp, 0) * 10px) + color-mix(in srgb, var(--danger) 30%, transparent); +} +.composer-mic__send { display: none; } +.composer-mic[data-state='locked'] .composer-mic__idle { display: none; } +.composer-mic[data-state='locked'] .composer-mic__send { display: inline-flex; } + +/* The recording bar takes the textarea's place while the mic is held. */ +.composer-field.is-recording .composer-input-stack textarea { display: none; } +.composer-recording { + display: none; + align-items: center; + gap: 10px; + min-height: 24px; + padding: 2px 0 10px; +} +.composer-recording:not([hidden]) { display: flex; } +.composer-recording__dot { + flex: 0 0 auto; + width: 9px; + height: 9px; + border-radius: 50%; + background: var(--danger); + animation: composer-rec-dot 1.1s ease-in-out infinite; +} +@keyframes composer-rec-dot { + 0%, 100% { opacity: 1; transform: scale(1); } + 50% { opacity: 0.35; transform: scale(0.78); } +} +.composer-recording__time { + flex: 0 0 auto; + min-width: 62px; + color: var(--text); + font-weight: 600; + font-size: 0.9em; + font-variant-numeric: tabular-nums; + font-feature-settings: 'tnum'; +} +.composer-recording__wave { + flex: 1 1 auto; + min-width: 0; + height: 26px; + display: block; +} +.composer-recording__hint { + flex: 0 0 auto; + color: var(--muted); + font-size: 0.8em; + font-weight: 500; + white-space: nowrap; +} +.composer-recording__cancel { + display: none; + flex: 0 0 auto; + border: none; + background: transparent; + color: var(--danger); + font: inherit; + font-size: 0.85em; + font-weight: 600; + cursor: pointer; + padding: 4px 10px; + border-radius: 8px; +} +.composer-recording__cancel:hover { + background: color-mix(in srgb, var(--danger) 12%, transparent); +} +.composer-recording.is-locked .composer-recording__cancel { display: inline-flex; } +.composer-recording.is-cancel .composer-recording__dot { background: var(--danger); } +.composer-recording.is-cancel .composer-recording__time, +.composer-recording.is-cancel .composer-recording__hint { color: var(--danger); } +/* Quick-tap coaching hint */ +.composer-recording.is-hint .composer-recording__dot, +.composer-recording.is-hint .composer-recording__time, +.composer-recording.is-hint .composer-recording__wave, +.composer-recording.is-hint .composer-recording__cancel { display: none; } +.composer-recording.is-hint .composer-recording__hint { color: var(--muted); } + +/* Floating lock indicator above the mic. */ +.composer-lock-hint { + position: absolute; + display: none; + flex-direction: column; + align-items: center; + gap: 3px; + transform: translateX(-50%); + padding: 9px 7px; + border-radius: 999px; + background: var(--surface); + border: 1px solid var(--border); + box-shadow: 0 4px 16px rgba(0, 0, 0, 0.16); + color: var(--muted); + z-index: var(--z-popover); + pointer-events: none; + transition: background 0.12s ease, color 0.12s ease, border-color 0.12s ease; +} +.composer-lock-hint:not([hidden]) { display: flex; } +.composer-lock-hint__chev { + transform: translateY(calc((1 - var(--lock-progress, 0)) * 5px)); + opacity: calc(0.35 + var(--lock-progress, 0) * 0.65); + transition: transform 0.08s linear, opacity 0.08s linear; +} +.composer-lock-hint.is-locked { + background: var(--accent); + color: #fff; + border-color: transparent; +} +.composer-lock-hint.is-locked .composer-lock-hint__chev { display: none; } + +@media (max-width: 430px) { + .composer-recording__hint { display: none; } + .composer-recording.is-hint .composer-recording__hint { display: inline; } +} +@media (prefers-reduced-motion: reduce) { + .composer-recording__dot { animation: none; } +} + +/* While recording, hide everything but the waveform + active mic. Use + visibility (not display:none) so the toolbar keeps its layout — otherwise the + lone right group collapses to the left under `justify-content: space-between` + and the mic appears to "teleport" left. */ +.composer-field.is-recording .composer-toolbar__left, +.composer-field.is-recording .composer-send { visibility: hidden; } + +.composer-attach-menu, .composer-secret-pick-menu { position: absolute; left: var(--space-3); @@ -4862,6 +5214,82 @@ code { transition: opacity 0.12s ease; } +/* Full Power selected while OpenClaw is off: swap the textarea for a message, + * but keep the toolbar (and its mode switcher) live so the user can switch. */ +.composer-exec-msg { + display: none; + align-items: center; + justify-content: center; + text-align: center; + min-height: 24px; + padding: 0 0 10px; + color: var(--muted); + font-size: 0.95em; + font-weight: 500; +} +.composer.is-exec-disabled .composer-input-stack textarea { display: none; } +.composer.is-exec-disabled .composer-exec-msg { display: flex; } + +/* Docker-required mode (Safe work) selected while Docker is off: same overlay + * treatment as the Full Power / OpenClaw case. */ +.composer-docker-msg { + display: none; + align-items: center; + justify-content: center; + flex-wrap: wrap; + gap: 6px; + text-align: center; + min-height: 24px; + padding: 0 0 10px; + color: var(--muted); + font-size: 0.95em; + font-weight: 500; +} +.composer.is-docker-disabled .composer-input-stack textarea { display: none; } +.composer.is-docker-disabled .composer-docker-msg { display: flex; } +.composer-docker-size { color: var(--muted); font-weight: 400; opacity: 0.8; } +.composer-docker-or { color: var(--muted); font-weight: 400; } + +/* Inline call-to-action button for installing Docker (Safe overlay). */ +.composer-docker-btn { + appearance: none; + border: 1px solid var(--border); + background: var(--msg-user-fill); + color: var(--text); + font: inherit; + font-weight: 600; + font-size: 0.92em; + padding: 3px 10px; + border-radius: 6px; + cursor: pointer; +} +.composer-docker-btn:hover { border-color: var(--ok-border); } +.composer-docker-btn:disabled { opacity: 0.6; cursor: default; } + +/* Mode menu item muted while its backend is unreachable: still clickable + * (selecting it surfaces the input overlay that explains why). The suffix is + * backend-specific so Execute reads "needs OpenClaw" and Safe "needs Docker". */ +.composer-mode-menu-item.is-unavailable { + opacity: 0.5; +} +.composer-mode-menu-item.is-unavailable[data-mode="execute"] .menu-item__title::after { + content: ' · needs OpenClaw'; + font-weight: 400; + color: var(--muted); +} +.composer-mode-menu-item.is-unavailable[data-requires-docker="1"] .menu-item__title::after { + content: ' · needs Docker'; + font-weight: 400; + color: var(--muted); +} +/* Last so it wins for Safe work (needs both Docker and a key) — the key is the + * blocker to surface first. */ +.composer-mode-menu-item.is-unavailable[data-requires-key="1"] .menu-item__title::after { + content: ' · needs OpenRouter'; + font-weight: 400; + color: var(--muted); +} + /* ---------- Message attachments (persisted user uploads) ---------- */ .msg-attachments { @@ -6882,12 +7310,10 @@ a.project-tab.project-tab--tasks:hover { display: none; } - .chat-header .reasoning-toggle .header-tool-icon, .chat-header .share-toggle .header-tool-icon { display: block; } - .chat-header .reasoning-toggle, .chat-header .share-toggle { position: relative; justify-content: center; @@ -6899,7 +7325,6 @@ a.project-tab.project-tab--tasks:hover { gap: 0; } - .chat-header .reasoning-toggle input[type="checkbox"], .chat-header .share-toggle input[type="checkbox"] { position: absolute; inset: 0; @@ -6911,7 +7336,6 @@ a.project-tab.project-tab--tasks:hover { accent-color: transparent; } - .chat-header .reasoning-toggle:has(input:checked), .chat-header .share-toggle:has(input:checked) { color: var(--md-link); background: color-mix(in srgb, var(--md-link) 12%, transparent); @@ -6944,13 +7368,6 @@ a.project-tab.project-tab--tasks:hover { border-radius: 12px; } - .draft-header .gateway { - flex-shrink: 0; - max-width: none; - white-space: nowrap; - font-size: 0.72rem; - } - .project-header { display: grid; grid-template-columns: 44px minmax(0, 1fr); @@ -7222,3 +7639,846 @@ a.project-tab.project-tab--tasks:hover { touch-action: auto; } } + +/* ── Secure workspace bar ── */ +.secure-workspace-bar { + /* Mirror .composer-secret-ui exactly so the two share identical insets — they + never show at once (refreshSecureBar hides the bar while the secret UI is + up). Absolute (not in-flow) so the bottom gap matches the secret strip's + `bottom: var(--space-4)` instead of sitting above the composer's padding. */ + position: absolute; + left: calc(20px + var(--space-2)); + right: calc(20px + var(--space-2)); + bottom: var(--space-4); + margin: 0; + padding: 0 var(--space-2); + display: flex; + align-items: center; + gap: 6px; + min-height: var(--composer-secret-row-h, 22px); + font-size: var(--text-xs); + color: var(--muted); +} +.secure-workspace-bar[hidden] { display: none; } +.secure-workspace-bar__size[hidden] { display: none; } +.secure-workspace-bar__sep { opacity: 0.4; } +.secure-workspace-bar__change { + border: none; + background: transparent; + color: var(--muted); + font: inherit; + font-size: var(--text-xs); + cursor: pointer; + text-decoration: underline; + padding: 0; +} +.secure-workspace-bar__change:hover { color: var(--text); } +.secure-workspace-bar__action { + border: none; + background: transparent; + color: var(--muted); + font: inherit; + font-size: var(--text-xs); + cursor: pointer; + text-decoration: underline; + padding: 0; + margin-left: 8px; +} +.secure-workspace-bar__action:hover { color: var(--text); } +.secure-workspace-bar__action:disabled { opacity: 0.6; cursor: default; text-decoration: none; } +.secure-workspace-bar__destroy { + border: none; + background: transparent; + color: var(--muted); + font: inherit; + font-size: var(--text-xs); + cursor: pointer; + text-decoration: underline; + padding: 0; + margin-left: 8px; +} +.secure-workspace-bar__destroy:hover { color: var(--down, #d9534f); } +.secure-workspace-bar__destroy:disabled { opacity: 0.6; cursor: default; text-decoration: none; } + +/* ── Incognito (ephemeral, read-only) ─────────────────────────────────────── */ +/* "Blacked out" identity — neutral/dark, no colour. */ + +/* Divider that sets Incognito apart at the bottom of the mode menu. */ +.composer-mode-menu-sep { + height: 1px; + margin: 6px 10px; + background: color-mix(in srgb, var(--text) 14%, transparent); +} + +/* Immersive surface: hide the chat list entirely — exit Incognito (the × at + top-left, or pick another mode) to see your chats again. */ +body.incognito-mode .app { grid-template-columns: 1fr; } +body.incognito-mode #app-sidebar { display: none; } + +/* No delete and no cloud-share while incognito. */ +body.incognito-mode #share-btn, +body.incognito-mode .delete-form { display: none !important; } + +/* Darken the conversation column (overlay only — keeps the base bg). */ +body.incognito-mode .col-main { + background-image: + radial-gradient(130% 60% at 50% 0%, rgba(0, 0, 0, 0.30), transparent 70%); +} + +/* Exit button — top-left, only while incognito. */ +.incognito-exit { + position: fixed; + top: 14px; + left: 14px; + z-index: 60; + display: none; + align-items: center; + justify-content: center; + width: 34px; + height: 34px; + padding: 0; + border-radius: 50%; + border: 1px solid color-mix(in srgb, var(--text) 18%, transparent); + background: var(--sidebar-bg, color-mix(in srgb, var(--text) 10%, transparent)); + color: var(--text); + cursor: pointer; + line-height: 0; +} +.incognito-exit:hover { background: color-mix(in srgb, var(--text) 20%, var(--sidebar-bg, transparent)); } +body.incognito-mode .incognito-exit { display: inline-flex; } +/* Make room for the × so it doesn't sit on top of the chat title. */ +body.incognito-mode .chat-header { padding-left: 58px; } + +.incognito-banner { + display: flex; + align-items: center; + gap: 10px; + width: min(var(--thread-max-width), 100%); + margin: 14px auto 6px; + padding: 10px 16px; + border-radius: 12px; + font-size: var(--text-sm); + line-height: 1.4; + color: var(--muted); + background: color-mix(in srgb, var(--text) 7%, transparent); + border: 1px solid color-mix(in srgb, var(--text) 16%, transparent); +} +.incognito-banner__icon { + flex: none; + width: 18px; + height: 18px; + color: var(--text); + opacity: 0.75; +} +.incognito-banner__dot { display: none; } /* superseded by the icon */ + +/* Composer: neutral emphasis, no colour. */ +body.incognito-mode .composer-field { + border-color: color-mix(in srgb, var(--text) 26%, var(--border, transparent)); + background: rgba(0, 0, 0, 0.10); +} +body.incognito-mode .composer-field:focus-within { + border-color: color-mix(in srgb, var(--text) 42%, transparent); +} +body.incognito-mode .composer-mode-btn, +body.incognito-mode .composer-mode-btn__label { + color: var(--text); + font-weight: 600; +} +/* Pop up above the bar, anchored to it — never reflow the page. */ +.secure-ttl-menu { + top: auto; + bottom: calc(100% + 4px); + right: calc(var(--space-2) * 2); + left: auto; + min-width: 120px; +} + +/* ---------------- project skills (procedural memory) ---------------- */ + +/* Chat inbox card — mirrors .fact-suggestions-card. */ +.msg.system.skill-suggestions-card { + align-self: stretch; + max-width: min(var(--thread-max-width), 100%); + width: 100%; + margin: 0 auto; + padding: 0; + text-align: left; + background: transparent; + border: none; + border-radius: 0; +} + +.skill-suggestions-shell { + border-radius: 14px; + border: 1px solid color-mix(in srgb, var(--border) 88%, transparent); + background: color-mix(in srgb, var(--surface) 55%, var(--chat-pane-bg)); + box-shadow: 0 1px 3px rgba(0, 0, 0, 0.05); + overflow: hidden; +} + +@media (prefers-color-scheme: dark) { + .skill-suggestions-shell { + box-shadow: 0 1px 4px rgba(0, 0, 0, 0.35); + } +} + +.skill-suggestions-lead { + margin: 0; + padding: 10px 14px 8px; + font-size: 0.8125rem; + font-weight: 500; + letter-spacing: -0.01em; + color: var(--muted); + line-height: 1.4; +} + +.skill-suggestions-list { + list-style: none; + margin: 0; + padding: 0 8px 8px; + display: flex; + flex-direction: column; + gap: 8px; +} + +.skill-suggestion-row { + display: flex; + flex-direction: column; + gap: 8px; + padding: 12px; + border-radius: 10px; + background: color-mix(in srgb, var(--surface) 40%, transparent); +} + +.skill-suggestion-head { + display: flex; + flex-wrap: wrap; + gap: 6px; + align-items: center; +} + +.skill-suggestion-badge, +.skill-scope-badge { + display: inline-flex; + align-items: center; + font-size: 0.6875rem; + font-weight: 600; + letter-spacing: 0.01em; + padding: 2px 8px; + border-radius: 999px; + line-height: 1.5; +} + +.skill-suggestion-badge--new, +.skill-scope-badge { + background: color-mix(in srgb, var(--accent, #4a8) 18%, transparent); + color: color-mix(in srgb, var(--accent, #4a8) 88%, var(--text)); +} + +.skill-suggestion-badge--patch { + background: color-mix(in srgb, #c9a227 22%, transparent); + color: color-mix(in srgb, #9a7b10 90%, var(--text)); +} + +.skill-suggestion-badge--untrusted { + background: color-mix(in srgb, var(--danger, #d35) 16%, transparent); + color: color-mix(in srgb, var(--danger, #d35) 90%, var(--text)); +} + +.skill-suggestion-name { + width: 100%; + box-sizing: border-box; + font-size: 0.9375rem; + font-weight: 600; + font-family: var(--mono-font, ui-monospace, monospace); + padding: 6px 8px; + border: 1px solid color-mix(in srgb, var(--border) 80%, transparent); + border-radius: 8px; + background: var(--chat-pane-bg); + color: var(--text); +} + +.skill-suggestion-desc, +.skill-suggestion-body { + width: 100%; + box-sizing: border-box; + font-size: 0.875rem; + line-height: 1.5; + padding: 6px 8px; + border: 1px solid color-mix(in srgb, var(--border) 80%, transparent); + border-radius: 8px; + background: var(--chat-pane-bg); + color: var(--text); + resize: vertical; +} + +.skill-suggestion-body { + font-family: var(--mono-font, ui-monospace, monospace); + font-size: 0.8125rem; +} + +.skill-suggestion-bodywrap > summary { + cursor: pointer; + font-size: 0.8125rem; + color: var(--muted); + padding: 2px 0; + user-select: none; +} + +.skill-suggestion-controls { + display: flex; + flex-wrap: wrap; + align-items: center; + justify-content: space-between; + gap: 10px; +} + +.skill-suggestion-scope { + display: inline-flex; + align-items: center; + gap: 6px; + font-size: 0.8125rem; + color: var(--muted); + cursor: pointer; +} + +.skill-suggestion-actions { + display: flex; + gap: 8px; + margin-left: auto; +} + +.skill-suggestion-btn { + font-size: 0.8125rem; + font-weight: 600; + padding: 6px 14px; + border-radius: 8px; + border: 1px solid transparent; + cursor: pointer; +} + +.skill-suggestion-accept { + background: var(--accent, #4a8); + color: #fff; +} + +.skill-suggestion-accept:disabled { + opacity: 0.6; + cursor: default; +} + +.skill-suggestion-reject { + background: transparent; + border-color: color-mix(in srgb, var(--border) 80%, transparent); + color: var(--muted); +} + +.skill-suggestion-reject:hover { + color: var(--text); +} + +/* Simplified card: one plain title + summary; everything technical is folded + into the «Деталі» <details> below. */ +.skill-suggestion-simple { + display: flex; + flex-direction: column; + gap: 3px; +} + +.skill-suggestion-title { + display: flex; + align-items: center; + gap: 6px; + font-size: 0.9375rem; + font-weight: 600; + letter-spacing: -0.01em; + color: var(--text); + line-height: 1.35; +} + +.skill-suggestion-summary { + font-size: 0.875rem; + line-height: 1.45; + color: var(--muted); +} + +.skill-suggestion-updates { + align-self: flex-start; + margin-top: 2px; + font-size: 0.6875rem; + font-weight: 600; + color: color-mix(in srgb, #c9a227 90%, var(--text)); +} + +.skill-suggestion-info { + font-size: 0.8125rem; + color: var(--muted); + cursor: help; + opacity: 0.7; +} +.skill-suggestion-info:hover, +.skill-suggestion-info:focus { opacity: 1; } + +/* The fold. Closed by default — non-technical users never need to open it. */ +.skill-suggestion-advanced > summary { + cursor: pointer; + font-size: 0.75rem; + color: var(--muted); + padding: 2px 0; + user-select: none; + list-style: none; +} +.skill-suggestion-advanced > summary::-webkit-details-marker { display: none; } +.skill-suggestion-advanced > summary::before { content: '⋯ '; } +.skill-suggestion-advanced[open] > summary::before { content: '▾ '; } + +.skill-suggestion-advanced { + display: flex; + flex-direction: column; + gap: 8px; +} +.skill-suggestion-advanced[open] { gap: 8px; } + +.skill-suggestion-field { + display: flex; + flex-direction: column; + gap: 3px; +} + +.skill-suggestion-field-label { + font-size: 0.6875rem; + font-weight: 600; + letter-spacing: 0.01em; + color: var(--muted); +} + +/* Actions are a direct row child now; right-align them. */ +.skill-suggestion-row > .skill-suggestion-actions { + justify-content: flex-end; +} + +/* Project page — skills list (mirrors .fact). */ +.project-chats > li.skill { + margin: 0; + padding: 12px 16px 14px; + background: transparent; +} + +.project-chats > li.skill .project-row-head { + margin-bottom: 6px; + display: flex; + align-items: center; + gap: 8px; +} + +.skill-name { + display: block; + width: 100%; + box-sizing: border-box; + margin: 0 0 6px; + border: none; + background: transparent; + padding: 2px 0; + font-size: 0.9375rem; + font-weight: 600; + font-family: var(--mono-font, ui-monospace, monospace); + color: var(--text); +} + +.skill-description { + display: block; + width: 100%; + box-sizing: border-box; + margin: 0 0 6px; + border: none; + background: transparent; + padding: 2px 0; + font-size: 0.9375rem; + line-height: 1.5; + resize: vertical; + min-height: 2.5rem; + color: var(--text); +} + +.skill-name:focus, +.skill-description:focus, +.skill-body:focus { + outline: none; +} + +.skill-body-details > summary { + cursor: pointer; + font-size: 0.8125rem; + color: var(--muted); + padding: 4px 0; + user-select: none; +} + +.skill-body { + display: block; + width: 100%; + box-sizing: border-box; + margin: 6px 0 0; + padding: 8px; + border: 1px solid color-mix(in srgb, var(--border) 80%, transparent); + border-radius: 8px; + background: var(--chat-pane-bg); + color: var(--text); + font-family: var(--mono-font, ui-monospace, monospace); + font-size: 0.8125rem; + line-height: 1.5; + resize: vertical; +} + +/* -------------------------------------------------------------------------- */ +/* First-run welcome / onboarding (views/welcome.ejs) */ +/* Apple "setup sheet" feel: one centered card, large title, calm monochrome. */ +/* -------------------------------------------------------------------------- */ + +.onb { + min-height: 100dvh; + display: flex; + align-items: center; + justify-content: center; + padding: var(--space-9); + background: var(--bg); +} + +.onb-card-shell { + position: relative; + width: 100%; + max-width: 30rem; + background: var(--surface); + border: 1px solid var(--border); + border-radius: var(--radius-xl); + box-shadow: var(--shadow-lg); + padding: var(--space-10); + text-align: center; +} + +.onb-step { animation: onb-fade 0.18s ease; } + +@keyframes onb-fade { + from { opacity: 0; transform: translateY(4px); } + to { opacity: 1; transform: none; } +} + +.onb-logo { + font-weight: 700; + letter-spacing: -0.02em; + font-size: var(--text-lg); + color: var(--muted); + margin-bottom: var(--space-7); +} + +.onb-title { + font-size: 1.6rem; + font-weight: 650; + letter-spacing: -0.02em; + margin: 0 0 var(--space-4); + color: var(--text); +} + +.onb-sub { + font-size: var(--text-base); + line-height: 1.5; + color: var(--muted); + margin: 0 auto var(--space-9); + max-width: 26rem; +} + +/* Big tappable option rows (Apple list-row style) */ +.onb-option { + display: flex; + align-items: center; + gap: var(--space-6); + width: 100%; + text-align: left; + background: var(--surface); + border: 1px solid var(--border); + border-radius: var(--radius-lg); + padding: var(--space-7); + margin-bottom: var(--space-5); + cursor: pointer; + transition: border-color 0.12s ease, background 0.12s ease, box-shadow 0.12s ease; +} +.onb-option:hover:not(:disabled) { + border-color: color-mix(in srgb, var(--text) 28%, var(--border)); + box-shadow: var(--shadow-md); +} +.onb-option:focus-visible { outline: none; box-shadow: var(--focus-ring); } + +.onb-option--disabled, +.onb-option:disabled { + cursor: default; + opacity: 0.65; +} + +.onb-option-main { display: flex; flex-direction: column; gap: var(--space-2); flex: 1; } +.onb-option-name { + font-size: var(--text-md); + font-weight: 600; + color: var(--text); + display: flex; + align-items: center; + gap: var(--space-4); +} +.onb-option-desc { font-size: var(--text-sm); line-height: 1.45; color: var(--muted); } + +.onb-option-chevron { + font-size: 1.5rem; + color: var(--muted); + line-height: 1; +} + +.onb-badge { + font-size: var(--text-xs); + font-weight: 600; + color: var(--muted); + background: color-mix(in srgb, var(--text) 7%, var(--surface)); + border: 1px solid var(--border); + border-radius: var(--radius-pill); + padding: 1px var(--space-4); +} + +.onb-skip { + display: inline-block; + margin-top: var(--space-5); + background: none; + border: none; + color: var(--muted); + font-size: var(--text-sm); + cursor: pointer; + padding: var(--space-4); +} +.onb-skip:hover { color: var(--text); text-decoration: underline; } + +/* Connect-a-model chooser modal — shown when a user who skipped onboarding + (no OpenRouter key, no reachable OpenClaw) tries to send. Reuses .onb-* rows. */ +.connect-modal { + position: fixed; + inset: 0; + z-index: 1000; + display: flex; + align-items: center; + justify-content: center; + padding: var(--space-6); +} +.connect-modal[hidden] { display: none; } +.connect-modal__backdrop { + position: absolute; + inset: 0; + background: rgba(0, 0, 0, 0.55); +} +.connect-modal__card { + position: relative; + z-index: 1; + margin: 0; + text-align: center; + animation: onb-fade 0.18s ease; +} +.connect-modal__close { + position: absolute; + top: var(--space-5); + right: var(--space-5); + width: 1.75rem; + height: 1.75rem; + display: grid; + place-items: center; + background: none; + border: none; + border-radius: var(--radius-lg); + color: var(--muted); + font-size: 1.4rem; + line-height: 1; + cursor: pointer; +} +.connect-modal__close:hover { + color: var(--text); + background: color-mix(in srgb, var(--text) 8%, transparent); +} +.connect-modal__note { + margin: var(--space-6) 0 0; + font-size: var(--text-sm); + color: var(--muted); + line-height: 1.45; +} + +.onb-back { + position: absolute; + top: var(--space-7); + left: var(--space-7); + background: none; + border: none; + color: var(--muted); + font-size: var(--text-sm); + cursor: pointer; +} +.onb-back:hover { color: var(--text); } + +/* Key step */ +.onb-steps { + text-align: left; + margin: var(--space-8) auto var(--space-9); + padding-left: 1.25rem; + max-width: 26rem; + color: var(--text); + font-size: var(--text-sm); + line-height: 1.6; +} +.onb-steps li { margin-bottom: var(--space-4); } +.onb-steps a { color: var(--info); } + +.onb-key-form { + display: flex; + gap: var(--space-4); + margin-bottom: var(--space-4); +} +.onb-key-input { + flex: 1; + font-family: var(--mono-font, ui-monospace, monospace); + font-size: var(--text-sm); + padding: var(--space-6); + border: 1px solid var(--border); + border-radius: var(--radius-md); + background: var(--bg); + color: var(--text); +} +.onb-key-input:focus-visible { outline: none; box-shadow: var(--focus-ring); border-color: color-mix(in srgb, var(--text) 28%, var(--border)); } +.onb-key-submit { white-space: nowrap; } + +.onb-key-error { color: var(--danger); font-size: var(--text-sm); margin: 0 0 var(--space-4); line-height: 1.45; } +.onb-key-error.is-warn { color: var(--warn); } +.onb-key-note { color: var(--muted); font-size: var(--text-xs); margin: 0; } + +/* Honest background-prep line */ +.onb-prep[hidden] { display: none; } +.onb-prep { + display: flex; + align-items: center; + justify-content: center; + gap: var(--space-4); + margin-top: var(--space-9); + padding-top: var(--space-7); + border-top: 1px solid var(--border); + font-size: var(--text-xs); + color: var(--muted); +} +.onb-prep-dot { + width: 7px; + height: 7px; + border-radius: var(--radius-circle); + background: var(--warn); + animation: onb-pulse 1.2s ease-in-out infinite; +} +.onb-prep-dot.is-done { + background: var(--ok); + animation: none; +} +@keyframes onb-pulse { + 0%, 100% { opacity: 0.35; } + 50% { opacity: 1; } +} + +@media (max-width: 32rem) { + .onb { padding: 0; align-items: stretch; } + .onb-card-shell { + border: none; + border-radius: 0; + box-shadow: none; + max-width: none; + padding: var(--space-10) var(--space-8); + display: flex; + flex-direction: column; + justify-content: center; + min-height: 100dvh; + } +} + +/* -------------------------------------------------------------------------- */ +/* First-chat conversational welcome (views/index.ejs draft thread) */ +/* Reads like the assistant's own first message: avatar + bubble + chips. */ +/* -------------------------------------------------------------------------- */ + +.welcome-card { + /* Overrides .empty-state (center + nowrap + narrow) — this is a left-aligned, + wrapping message bubble, not a one-line hint. */ + display: flex; + gap: var(--space-6); + align-items: flex-start; + width: 100%; + max-width: var(--thread-max-width); + box-sizing: border-box; + margin: var(--space-10) auto var(--space-9); + padding: 0 var(--space-7); + text-align: left; + white-space: normal; + animation: onb-fade 0.2s ease; +} + +.welcome-avatar { + flex: 0 0 auto; + width: 34px; + height: 34px; + border-radius: var(--radius-circle); + display: flex; + align-items: center; + justify-content: center; + font-size: var(--text-sm); + font-weight: 700; + letter-spacing: -0.02em; + color: var(--surface); + background: var(--accent); +} + +.welcome-bubble { + flex: 1; + min-width: 0; + color: var(--msg-prose, var(--text)); + font-size: var(--text-base); + line-height: 1.55; +} +.welcome-bubble p { margin: 0 0 var(--space-5); } +.welcome-greeting { font-weight: 650; font-size: var(--text-lg); color: var(--text); } +/* Extra breathing room below the lead — reads like a blank line before the list. */ +.welcome-lead { color: var(--text); margin-bottom: var(--space-9); } +.welcome-reassure { color: var(--muted); font-size: var(--text-sm); } + +.welcome-list { + list-style: disc; + margin: 0 0 var(--space-6); + padding-left: 1.25em; + display: flex; + flex-direction: column; + gap: var(--space-5); +} +.welcome-list li { + /* Same size as the lead line above it. */ + line-height: 1.5; + color: var(--text); + padding-left: 0.25em; +} + +.welcome-chips { + display: flex; + flex-wrap: wrap; + gap: var(--space-4); + margin-top: var(--space-6); +} +.welcome-chip { + border: 1px solid var(--border); + border-radius: var(--radius-pill); + background: var(--surface); + color: var(--text); + font-size: var(--text-sm); + padding: var(--space-4) var(--space-6); + cursor: pointer; + transition: border-color 0.12s ease, background 0.12s ease, box-shadow 0.12s ease; +} +.welcome-chip:hover { + border-color: color-mix(in srgb, var(--text) 28%, var(--border)); + background: var(--hover); + box-shadow: var(--shadow-sm); +} +.welcome-chip:focus-visible { outline: none; box-shadow: var(--focus-ring); } diff --git a/public/js/iclaw.js b/public/js/iclaw.js index 2bbeea9..66dfefe 100644 --- a/public/js/iclaw.js +++ b/public/js/iclaw.js @@ -12,6 +12,20 @@ // ------------------------------------------------------------------------- // shared DOM handles + state // ------------------------------------------------------------------------- + // In-app navigation. Through a tunnel a full browser navigation bounces via + // the passphrase gate ("Checking this device…") because it can't be E2E + // wrapped. When the encrypted transport is installed it exposes + // window.iclawE2eNavigate, which pulls the next page over the existing channel + // and swaps the document in place — no gate round trip. Outside the tunnel + // (helper absent) this is a normal navigation. + function goTo(url) { + if (typeof window.iclawE2eNavigate === 'function') { + window.iclawE2eNavigate(url); + return; + } + window.location.assign(url); + } + const messagesEl = document.getElementById('messages'); function getMessagesThreadEl() { return messagesEl?.querySelector(':scope > .messages-thread') ?? null; @@ -42,6 +56,1349 @@ const rawChatId = messagesEl?.dataset.chatId; const startedOnDraft = messagesEl?.dataset.draft === '1' || !rawChatId; let activeChatId = startedOnDraft ? null : Number(rawChatId); + + // ------------------------------------------------------------------------- + // composer mode (Ask / Execute). Mode rides along with each sent message. + // The set of selectable modes is rendered server-side from the config in + // services/chatModes.ts, so adding a mode there surfaces it here with no + // client change. Default + back-compat fallback is 'execute'. + // ------------------------------------------------------------------------- + const MODE_STORAGE_KEY = rawChatId ? `iclaw:composer-mode:${rawChatId}` : 'iclaw:composer-mode'; + const composerModesEl = document.getElementById('composer-modes'); + const composerModeBtn = document.getElementById('composer-mode-btn'); + const composerModeMenu = document.getElementById('composer-mode-menu'); + const composerModeLabel = document.getElementById('composer-mode-label'); + const composerModeDefault = + composerModesEl?.dataset.defaultMode || 'execute'; + const composerModeIds = composerModeMenu + ? Array.from(composerModeMenu.querySelectorAll('.composer-mode-menu-item')).map( + (el) => el.dataset.mode, + ) + : [composerModeDefault]; + let selectedComposerMode = composerModeDefault; + + // True when an OpenRouter key is configured. Derived from the rendered menu: + // locked (needs-key) items are only emitted when there's no key. Drives the + // connect chooser + the Full Power "switch mode" overlay (the runtime modes + // are the only fallback for a dead gateway, and they need the key). + const openRouterReady = + !!composerModeMenu && + !composerModeMenu.querySelector('.composer-mode-menu-item[data-requires-key="1"]'); + + /** True when a mode is shown but locked behind a missing OpenRouter key. */ + function isModeLocked(id) { + if (!composerModeMenu) return false; + const el = composerModeMenu.querySelector( + '.composer-mode-menu-item[data-mode="' + id + '"]', + ); + return !!el && el.dataset.requiresKey === '1'; + } + + /** Currently selected send mode (always one of the rendered, enabled ids). */ + function getComposerMode() { + return composerModeIds.includes(selectedComposerMode) + ? selectedComposerMode + : composerModeDefault; + } + + function setComposerMode(mode, opts) { + let next = composerModeIds.includes(mode) ? mode : composerModeDefault; + // Don't land on a locked mode that isn't the default (e.g. a chat last used + // in a runtime mode after the key was removed) — fall back to the default. + // The default itself MAY be a locked Work (no OpenClaw + no key); that's + // intended, and the connect chooser fires on first send. + if (isModeLocked(next)) next = composerModeDefault; + // Remember the mode we leave when entering Incognito, so the × can restore it. + if (next === 'incognito' && selectedComposerMode !== 'incognito') { + incognitoReturnMode = selectedComposerMode; + } + selectedComposerMode = next; + if (composerModeBtn) composerModeBtn.dataset.mode = next; + if (composerModeMenu) { + composerModeMenu.querySelectorAll('.composer-mode-menu-item').forEach((el) => { + const on = el.dataset.mode === next; + el.setAttribute('aria-checked', on ? 'true' : 'false'); + if (composerModeLabel && on) { + const t = el.querySelector('.menu-item__title'); + composerModeLabel.textContent = t ? t.textContent : next; + } + if (on) { + const desc = el.querySelector('.composer-mode-menu-item__desc'); + if (composerModeBtn && desc) composerModeBtn.title = desc.textContent || ''; + } + }); + } + // Never persist incognito as a chat's default mode — it's a transient, + // explicitly-entered surface, not a sticky preference. + if ((!opts || opts.persist !== false) && next !== 'incognito') { + try { localStorage.setItem(MODE_STORAGE_KEY, next); } catch (_) {} + // Persist server-side too, so the mode sticks across page navigation and + // syncs across devices — not just this browser's localStorage. Drafts have + // no chat id yet (mode rides along with the first message); once the chat + // exists the server row (chats.mode) is the source of truth on reload. + if (activeChatId != null) { + fetch('/chats/' + encodeURIComponent(activeChatId) + '/mode', { + method: 'POST', + headers: { 'content-type': 'application/json' }, + body: JSON.stringify({ mode: next }), + }).catch(function () {}); + } + } + // Incognito: tint the surface + show the "nothing saved" banner, and start a + // fresh ephemeral session each time the mode is (re)entered. + if (typeof syncIncognitoSurface === 'function') syncIncognitoSurface(next); + syncExecuteAvailability(); + if (typeof syncDockerAvailability === 'function') syncDockerAvailability(); + syncAgentVisibility(next); + } + + /** + * The Agent picker selects an OpenClaw agent session, which only affects Full + * Power (Execute). Work / Safe work / Incognito route to iclaw-runtime and + * ignore it — so hide it there instead of implying a choice that does nothing. + * Mode is per-message, so this re-runs on every mode change. (When only Execute + * is selectable the mode menu isn't rendered and this never hides it.) + */ + function syncAgentVisibility(mode) { + const m = mode || getComposerMode(); + document.querySelectorAll('.agent-form').forEach((el) => { + el.hidden = m !== 'execute'; + }); + } + + // ── Full Power (Execute) gating ─────────────────────────────────────────── + // Execute routes to the OpenClaw gateway; the runtime modes don't. When the + // gateway is off we mute the Full Power option and, if it's the selected mode, + // cover the input with an explanation (same treatment as the drag-drop overlay) + // so the user switches mode instead of typing into a dead end. + const composerExecMsg = document.getElementById('composer-exec-msg'); + + // ── Docker gating (Safe-work sandbox) ───────────────────────────────────── + // We only block Safe work when Docker is NOT INSTALLED ('missing'). When it's + // merely stopped, the runtime starts it on demand the moment a task needs it + // (and auto-stops it when idle), so we don't block or nag. Work/Incognito are + // never blocked — file tools run on the host, run_command starts Docker lazily. + const composerDockerMsg = document.getElementById('composer-docker-msg'); + let dockerState = 'unknown'; + let dockerSizeHint = ''; + let dockerPollTimer = null; + + // Live "is the gateway usable for Full Power" flag. Seeded from the server via + // the composer form's data-gateway-ok, then kept current by applyGatewayStatus + // on every status change. + let gatewayOk = (function seedGatewayOk() { + if (form && form.dataset.gatewayOk != null) return form.dataset.gatewayOk !== '0'; + return true; // no signal → don't block + })(); + + // The sidebar "Start OpenClaw" banner only makes sense when the user actually + // wants Full Power — the runtime modes (Work / Safe work / Incognito) never + // touch the gateway. Tracked here so BOTH a gateway-status change and a mode + // change re-evaluate the banner (see refreshGatewayOfflineBanner). + let gatewayOffline = false; + + /** True only when OpenClaw is reachable for Full Power (Execute). */ + function isExecuteAvailable() { + return gatewayOk; + } + + /** Reflect gateway availability on the Full Power option + the input overlay. */ + function syncExecuteAvailability() { + const avail = isExecuteAvailable(); + if (composerModeMenu) { + const execItem = composerModeMenu.querySelector( + '.composer-mode-menu-item[data-mode="execute"]', + ); + if (execItem) execItem.classList.toggle('is-unavailable', !avail); + } + // Full Power with the gateway down is only a "switch mode below" situation + // when the runtime modes are actually usable — i.e. an OpenRouter key is + // set. Without a key those modes are locked, so don't disable the input + // here; the submit handler surfaces the connect chooser instead. + const blocked = + !avail && getComposerMode() === 'execute' && openRouterReady; + if (form) form.classList.toggle('is-exec-disabled', blocked); + if (composerExecMsg) { + composerExecMsg.setAttribute('aria-hidden', blocked ? 'false' : 'true'); + } + refreshComposerInputDisabled(); + // The mode may have just changed — the sidebar "Start OpenClaw" banner is + // gated on Full Power too, so keep it in sync from the same funnel. + refreshGatewayOfflineBanner(); + } + + /** + * The composer input is disabled while ANY blocking overlay is up — Full + * Power with OpenClaw off, or a Docker-required mode with Docker off. Both + * overlays cover the textarea, so the send target would be hidden anyway; + * disabling input + send keeps keyboard submit from firing into nothing. + */ + function refreshComposerInputDisabled() { + const blocked = !!( + form && + (form.classList.contains('is-exec-disabled') || + form.classList.contains('is-docker-disabled')) + ); + if (input) input.disabled = blocked; + const sb = document.getElementById('composer-send-btn'); + if (sb) sb.disabled = blocked; + } + + /** True when the given mode can't run without a Docker daemon (Safe work). */ + function modeRequiresDocker(mode) { + if (!composerModeMenu) return false; + const item = composerModeMenu.querySelector( + '.composer-mode-menu-item[data-mode="' + mode + '"]', + ); + return !!item && item.dataset.requiresDocker === '1'; + } + + /** + * Reflect Docker state. We only BLOCK when Docker is genuinely unusable — + * i.e. not installed ('missing') — and only for a mode that can't run without + * it (Safe work). When Docker is merely stopped we don't block or nag: the + * runtime starts it on demand the moment a task needs it (and auto-stops it + * when idle). Work/Incognito are never blocked — their file tools run on the + * host and run_command starts Docker lazily. No-op until the first poll. + */ + function syncDockerAvailability() { + const missing = dockerState === 'missing'; + if (composerModeMenu) { + composerModeMenu.querySelectorAll('.composer-mode-menu-item').forEach((el) => { + if (el.dataset.requiresDocker === '1') { + // A key-locked mode (no OpenRouter) stays locked regardless of Docker. + el.classList.toggle('is-unavailable', missing || el.dataset.requiresKey === '1'); + } + }); + } + const mode = getComposerMode(); + const blocked = missing && modeRequiresDocker(mode); + if (form) form.classList.toggle('is-docker-disabled', blocked); + if (composerDockerMsg) { + composerDockerMsg.setAttribute('aria-hidden', blocked ? 'false' : 'true'); + } + applyDockerActionLabels(); + refreshComposerInputDisabled(); + } + + /** Button copy reflects whether Docker is missing (Install) or just idle (Start). */ + function applyDockerActionLabels() { + const installing = dockerState === 'installing'; + const starting = dockerState === 'starting'; + const needsInstall = dockerState === 'missing'; + let label = needsInstall ? 'Install Docker' : 'Start Docker'; + if (installing) label = 'Installing Docker…'; + else if (starting) label = 'Starting Docker…'; + const busy = installing || starting; + + const mainBtn = document.getElementById('composer-docker-action'); + if (mainBtn) { + mainBtn.textContent = label; + mainBtn.disabled = busy; + } + const sizeEl = document.getElementById('composer-docker-size'); + if (sizeEl) sizeEl.textContent = needsInstall && !busy && dockerSizeHint ? dockerSizeHint : ''; + } + + function applyDockerState(next, sizeHint) { + dockerState = next || 'unknown'; + if (typeof sizeHint === 'string' && sizeHint) dockerSizeHint = sizeHint; + syncDockerAvailability(); + } + + function pollDockerStatus(opts) { + fetch('/api/docker/status', { headers: { Accept: 'application/json' } }) + .then((r) => (r.ok ? r.json() : null)) + .then((data) => { + if (!data) return; + applyDockerState(data.state, data.sizeHint); + // Keep polling only while a start/install is in flight; otherwise the + // state is settled and the next change is user-driven (button click). + if (dockerState === 'installing' || dockerState === 'starting') { + dockerPollTimer = setTimeout(() => pollDockerStatus(opts), 2500); + } + }) + .catch(() => { + if (opts && opts.keepAlive) dockerPollTimer = setTimeout(() => pollDockerStatus(opts), 4000); + }); + } + + function triggerDockerAction() { + // Missing → install (which also starts); installed-but-idle → just start. + const endpoint = dockerState === 'missing' ? '/api/docker/install' : '/api/docker/start'; + // Optimistic transient state so the button shows progress immediately. + applyDockerState(dockerState === 'missing' ? 'installing' : 'starting', dockerSizeHint); + fetch(endpoint, { method: 'POST', headers: { Accept: 'application/json' } }) + .then((r) => (r.ok ? r.json() : null)) + .then((data) => { + if (data && data.state) applyDockerState(data.state, dockerSizeHint); + if (dockerPollTimer) clearTimeout(dockerPollTimer); + pollDockerStatus({ keepAlive: true }); + }) + .catch(() => { + // Endpoint failed (e.g. non-localhost) — re-probe so the UI is honest. + if (dockerPollTimer) clearTimeout(dockerPollTimer); + pollDockerStatus({}); + }); + } + + (function initDockerGate() { + const mainBtn = document.getElementById('composer-docker-action'); + if (mainBtn) mainBtn.addEventListener('click', triggerDockerAction); + syncDockerAvailability(); + // First real status read; thereafter polling only runs during an action. + pollDockerStatus({}); + })(); + + // ── Incognito (ephemeral, never persisted) ──────────────────────────────── + // The conversation lives only in this tab: messages render locally, the turn + // streams over `incognito-*` WS events keyed by `activeIncognitoKey`, and + // nothing is written to the DB. Reloading the page clears it. + let activeIncognitoKey = null; + let incognitoReturnMode = composerModeDefault; + + function newIncognitoKey() { + return 'inc-' + Date.now().toString(36) + '-' + Math.random().toString(36).slice(2, 8); + } + + /** + * Leave Incognito → return to wherever we came from (that chat + the sidebar + * come back). Incognito ran on a fresh ephemeral surface, so we just navigate + * back; the ephemeral conversation is discarded. + */ + function exitIncognito() { + let origin = '/'; + try { + origin = sessionStorage.getItem('iclaw:incognito-origin') || '/'; + sessionStorage.removeItem('iclaw:incognito-origin'); + } catch (_) {} + window.location.assign(origin); + } + + /** The fixed top-left × shown only in incognito. Created once, CSS toggles it. */ + function ensureIncognitoExitButton() { + if (document.getElementById('incognito-exit')) return; + const btn = document.createElement('button'); + btn.id = 'incognito-exit'; + btn.type = 'button'; + btn.className = 'incognito-exit'; + btn.setAttribute('aria-label', 'Exit Incognito'); + btn.title = 'Exit Incognito'; + btn.innerHTML = + '<svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" ' + + 'stroke-width="2.2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true">' + + '<path d="M18 6 6 18"/><path d="m6 6 12 12"/></svg>'; + btn.addEventListener('click', exitIncognito); + document.body.appendChild(btn); + } + + function syncIncognitoSurface(mode) { + const on = mode === 'incognito'; + document.body.classList.toggle('incognito-mode', on); + if (on) ensureIncognitoExitButton(); + let banner = document.getElementById('incognito-banner'); + if (on) { + if (!banner) { + banner = document.createElement('div'); + banner.id = 'incognito-banner'; + banner.className = 'incognito-banner'; + banner.innerHTML = + '<svg class="incognito-banner__icon" viewBox="0 0 24 24" fill="none" ' + + 'stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true">' + + '<path d="M2 12s3.5-7 10-7 10 7 10 7"/><path d="m4 4 16 16"/>' + + '<path d="M9.9 9.9a3 3 0 0 0 4.2 4.2"/></svg>' + + '<span>Incognito — read-only research. Your chat list is hidden, and this ' + + 'conversation isn’t saved or added to project memory.</span>'; + const root = typeof messagesAppendRoot === 'function' ? messagesAppendRoot() : messagesEl; + (root || messagesEl)?.prepend(banner); + } + // Fresh session whenever incognito is (re)selected. + activeIncognitoKey = null; + } else if (banner) { + banner.remove(); + } + } + + /** + * Finalize the streamed incognito reply (no message-appended for ephemeral + * turns): render the full markdown, strip the streaming chrome, and release + * the composer. Shared by the `incognito-turn-ended` event and the stop button. + */ + function finalizeIncognitoStream() { + setStopVisible(false); + cancelStreamRender(); + if (currentStreamEl) { + const body = currentStreamEl.querySelector('.stream-body, .msg-body'); + if (body && currentStreamFullText.trim()) { + body.classList.remove('stream-body'); + body.innerHTML = renderMarkdown(currentStreamFullText); + decorateMessageBody(body); + currentStreamEl.classList.remove('streaming', 'stream-waiting', 'stream-tool', 'stream-generating'); + const st = currentStreamEl.querySelector('.stream-status'); + if (st) { stopStreamStatusDotAnim(st); st.remove(); } + } else { + currentStreamEl.remove(); + } + currentStreamEl = null; + } + currentStreamFullText = ''; + streamShownLen = 0; + if (inFlight) { inFlight = false; if (waitingItems[0]) flushNextQueued(); } + } + + function closeComposerModeMenu() { + if (!composerModeMenu) return; + composerModeMenu.hidden = true; + if (composerModeBtn) composerModeBtn.setAttribute('aria-expanded', 'false'); + } + + if (composerModeBtn && composerModeMenu) { + // Initial mode precedence: + // 1. server-persisted chat mode (data-chat-mode = chats.mode) — authoritative, + // survives navigation and syncs across devices. + // 2. per-chat localStorage (legacy / offline fallback, existing chats only). + // 3. the UI default (Work) — for new chats we ignore any stale GLOBAL + // localStorage value so a fresh chat always starts on the default. + let stored = null; + if (rawChatId) { + try { stored = localStorage.getItem(MODE_STORAGE_KEY); } catch (_) {} + } + const chatMode = composerModesEl ? composerModesEl.dataset.chatMode : ''; + setComposerMode(chatMode || stored || composerModeDefault, { persist: false }); + // Entered via "Incognito" elsewhere → ?mode=incognito on a fresh surface. + try { + const _qp = new URLSearchParams(window.location.search); + if (_qp.get('mode') === 'incognito' && composerModeIds.includes('incognito')) { + setComposerMode('incognito', { persist: false }); + window.history.replaceState({}, '', window.location.pathname); // tidy the URL + } + } catch (_) {} + + composerModeBtn.addEventListener('click', (e) => { + e.stopPropagation(); + const open = composerModeMenu.hidden; + composerModeMenu.hidden = !open; + composerModeBtn.setAttribute('aria-expanded', open ? 'true' : 'false'); + }); + const MODE_PLACEHOLDERS = { + work: 'Work inside selected folders', + secure: 'Run risky tasks in isolation', + incognito: 'Private read-only research — nothing saved', + execute: 'Use full iClaw power', + }; + + function updateComposerPlaceholder(mode) { + if (input) input.placeholder = MODE_PLACEHOLDERS[mode] || 'Ask anything'; + } + + composerModeMenu.addEventListener('click', (e) => { + const item = e.target.closest('.composer-mode-menu-item'); + if (!item) return; + // Locked behind a missing OpenRouter key — re-offer connecting instead of + // switching into a mode that can't run. + if (item.dataset.requiresKey === '1') { + closeComposerModeMenu(); + openConnectChooser(); + return; + } + // Incognito is a separate ephemeral surface, not a flag on the current + // chat: open a fresh blank chat instead of converting this one. Remember + // where we came from so the × can bring us back. + if (item.dataset.mode === 'incognito' && !document.body.classList.contains('incognito-mode')) { + try { sessionStorage.setItem('iclaw:incognito-origin', window.location.pathname + window.location.search); } catch (_) {} + window.location.assign('/?mode=incognito'); + return; + } + setComposerMode(item.dataset.mode); + closeComposerModeMenu(); + updateComposerPlaceholder(item.dataset.mode); + if (typeof updateWorkFoldersButton === 'function') updateWorkFoldersButton(); + input?.focus(); + }); + + // Set placeholder on initial load + updateComposerPlaceholder(getComposerMode()); + document.addEventListener('click', (e) => { + if (composerModeMenu.hidden) return; + if (composerModesEl && !composerModesEl.contains(e.target)) closeComposerModeMenu(); + }); + document.addEventListener('keydown', (e) => { + if (e.key === 'Escape' && !composerModeMenu.hidden) closeComposerModeMenu(); + }); + } + + // ------------------------------------------------------------------------- + // Work Mode — folders picker + // ------------------------------------------------------------------------- + const workFoldersBtn = document.getElementById('composer-work-folders-btn'); + const workFoldersModal = document.getElementById('work-folders-modal'); + const workFoldersList = document.getElementById('work-folders-list'); + const workFoldersInput = document.getElementById('work-folders-input'); + const workFoldersAddBtn = document.getElementById('work-folders-add-btn'); + const workFoldersBrowseBtn = document.getElementById('work-folders-browse-btn'); + const workFoldersClose = document.getElementById('work-folders-close'); + const workFoldersBackdrop = document.getElementById('work-folders-backdrop'); + const workFoldersCount = document.getElementById('composer-work-folders-count'); + + function workFoldersKey() { + const pid = messagesEl?.dataset.projectId; + if (pid) return `iclaw:work-folders:project:${pid}`; + return 'iclaw:work-folders:no-project'; + } + + // Folders are stored as { path, write }. Older clients stored bare path + // strings — migrate those to writable (their effective behavior at the time) + // so upgrading doesn't silently revoke access on existing folders. Newly + // added folders default to read-only (see addFolder / browse below). + function getWorkFolders() { + let raw; + try { raw = JSON.parse(localStorage.getItem(workFoldersKey()) || '[]'); } + catch { return []; } + if (!Array.isArray(raw)) return []; + return raw + .map((f) => (typeof f === 'string' + ? { path: f, write: true } + : (f && typeof f === 'object' && typeof f.path === 'string' + ? { path: f.path, write: f.write === true } + : null))) + .filter(Boolean); + } + + function saveWorkFolders(folders) { + try { localStorage.setItem(workFoldersKey(), JSON.stringify(folders)); } catch {} + } + + function escAttr(s) { + return String(s).replace(/&/g, '&').replace(/"/g, '"').replace(/</g, '<').replace(/>/g, '>'); + } + + function renderWorkFoldersList() { + if (!workFoldersList) return; + const folders = getWorkFolders(); + workFoldersList.innerHTML = ''; + for (const f of folders) { + const p = escAttr(f.path); + const li = document.createElement('li'); + li.className = 'work-folders-list__item'; + const label = f.write ? 'Read & write' : 'Read-only'; + li.innerHTML = + `<span class="work-folders-list__path" title="${p}">${p}</span>` + + `<button type="button" class="work-folders-list__access" data-path="${p}" data-write="${f.write ? '1' : '0'}" ` + + `title="Click to toggle access">${label}</button>` + + `<button type="button" class="work-folders-list__remove" data-path="${p}" aria-label="Remove">×</button>`; + workFoldersList.appendChild(li); + } + if (workFoldersCount) { + workFoldersCount.textContent = folders.length > 0 ? String(folders.length) : ''; + } + } + + // ── Network toggle (Secure Mode) ────────────────────────────────────────── + const networkToggleBtn = document.getElementById('composer-network-toggle-btn'); + + function networkKey() { + const pid = messagesEl?.dataset.projectId; + return pid ? `iclaw:secure-network:project:${pid}` : 'iclaw:secure-network:no-project'; + } + + function getNetworkEnabled() { + try { return localStorage.getItem(networkKey()) === 'on'; } catch { return false; } + } + + function setNetworkEnabledStorage(on) { + try { localStorage.setItem(networkKey(), on ? 'on' : 'off'); } catch {} + } + + function updateNetworkToggle() { + if (!networkToggleBtn) return; + const mode = getComposerMode(); + networkToggleBtn.hidden = (mode !== 'secure'); + const on = getNetworkEnabled(); + networkToggleBtn.dataset.network = on ? 'on' : 'off'; + networkToggleBtn.title = on ? 'Network is ON - click to disable' : 'Network is OFF - click to enable'; + networkToggleBtn.style.color = on ? 'var(--accent)' : ''; + } + + networkToggleBtn?.addEventListener('click', () => { + const on = !getNetworkEnabled(); + setNetworkEnabledStorage(on); + updateNetworkToggle(); + }); + + function updateWorkFoldersButton() { + if (!workFoldersBtn) return; + const mode = getComposerMode(); + // Incognito also uses folders — as read-only roots for its sandboxed shell. + workFoldersBtn.hidden = (mode !== 'work' && mode !== 'incognito'); + renderWorkFoldersList(); + updateNetworkToggle(); + } + + if (workFoldersBtn && workFoldersModal) { + workFoldersBtn.addEventListener('click', () => { + renderWorkFoldersList(); + workFoldersModal.hidden = false; + workFoldersInput?.focus(); + }); + + workFoldersList?.addEventListener('click', (e) => { + const removeBtn = e.target.closest('.work-folders-list__remove'); + if (removeBtn) { + const path = removeBtn.dataset.path; + saveWorkFolders(getWorkFolders().filter((f) => f.path !== path)); + renderWorkFoldersList(); + return; + } + const accessBtn = e.target.closest('.work-folders-list__access'); + if (accessBtn) { + const path = accessBtn.dataset.path; + const folders = getWorkFolders().map((f) => + f.path === path ? { ...f, write: !f.write } : f); + saveWorkFolders(folders); + renderWorkFoldersList(); + } + }); + + function addFolderPath(val) { + if (!val) return; + const folders = getWorkFolders(); + if (!folders.some((f) => f.path === val)) { + // New folders default to read-only; user opts into write explicitly. + folders.push({ path: val, write: false }); + saveWorkFolders(folders); + renderWorkFoldersList(); + } + } + + function addFolder() { + addFolderPath(workFoldersInput?.value.trim()); + if (workFoldersInput) workFoldersInput.value = ''; + } + + workFoldersAddBtn?.addEventListener('click', addFolder); + workFoldersInput?.addEventListener('keydown', (e) => { + if (e.key === 'Enter') { e.preventDefault(); addFolder(); } + }); + + workFoldersBrowseBtn?.addEventListener('click', async () => { + workFoldersBrowseBtn.disabled = true; + try { + const res = await fetch('/api/pick-folder', { method: 'POST' }); + if (res.status === 204) return; // user cancelled + const data = await res.json(); + if (data.path) addFolderPath(data.path); + } catch (e) { + console.error('pick-folder failed', e); + } finally { + workFoldersBrowseBtn.disabled = false; + } + }); + + const closeModal = () => { workFoldersModal.hidden = true; }; + workFoldersClose?.addEventListener('click', closeModal); + workFoldersBackdrop?.addEventListener('click', closeModal); + document.addEventListener('keydown', (e) => { + if (e.key === 'Escape' && !workFoldersModal.hidden) closeModal(); + }); + } + + updateWorkFoldersButton(); + + // ── Secure workspace bar ───────────────────────────────────────────────── + const secureBar = document.getElementById('secure-workspace-bar'); + const secureSizeEl = document.getElementById('secure-workspace-size'); + const secureTtlEl = document.getElementById('secure-workspace-ttl'); + const secureChangeBtn = document.getElementById('secure-workspace-change'); + const secureTtlMenu = document.getElementById('secure-ttl-menu'); + + function secureTtlKey() { + const pid = messagesEl?.dataset.projectId; + return `iclaw:secure-ttl:${pid ? 'project:' + pid : 'chat:' + rawChatId}`; + } + + function getSecureTtl() { + try { + const raw = localStorage.getItem(secureTtlKey()); + return raw ? JSON.parse(raw) : { ttlDays: 7, lastActivity: Date.now() }; + } catch { return { ttlDays: 7, lastActivity: Date.now() }; } + } + + function saveSecureTtl(ttlDays) { + try { + localStorage.setItem(secureTtlKey(), JSON.stringify({ ttlDays, lastActivity: Date.now() })); + } catch {} + } + + function formatBytes(bytes) { + if (bytes < 1024) return bytes + ' B'; + if (bytes < 1024 * 1024) return (bytes / 1024).toFixed(1) + ' KB'; + return (bytes / (1024 * 1024)).toFixed(1) + ' MB'; + } + + function formatTtlRemaining(ttlDays, lastActivity) { + if (ttlDays === 0) return 'never expires'; + // +15s demo buffer: holds the full day count for ~15s after each reset so + // users can see the countdown tick down (proof the TTL resets on activity). + const expiresAt = lastActivity + ttlDays * 86400_000 + 15_000; + const remaining = expiresAt - Date.now(); + if (remaining <= 0) return 'expired'; + const days = Math.floor(remaining / 86400_000); + const hours = Math.floor((remaining % 86400_000) / 3600_000); + const mins = Math.floor((remaining % 3600_000) / 60_000); + const secs = Math.floor((remaining % 60_000) / 1000); + // More than 2 days: show only days (no hours) + if (days > 2) return `expires in ${days}d`; + if (days > 0) return `expires in ${days}d ${hours}h`; + if (hours > 0) return `expires in ${hours}h ${mins}m`; + if (mins > 0) return `expires in ${mins}m ${secs}s`; + return `expires in ${secs}s`; + } + + // True when the secret hint UI is visible — it takes priority over the secure bar. + function secretUiVisible() { + const el = document.getElementById('composer-secret-ui'); + return el && !el.hidden; + } + + // Update only the TTL text (cheap, local — ticks every second). + function tickSecureTtl() { + if (!secureBar || secureBar.hidden) return; + const ttl = getSecureTtl(); + if (secureTtlEl) secureTtlEl.textContent = formatTtlRemaining(ttl.ttlDays, ttl.lastActivity); + } + + async function refreshSecureBar() { + if (!secureBar) return; + const mode = getComposerMode(); + // Hide if not in Secure Mode, or if the secret UI is currently showing. + if (mode !== 'secure' || secretUiVisible() || !rawChatId) { secureBar.hidden = true; return; } + + try { + const res = await fetch(`/chats/${rawChatId}/workspace-info`); + const data = await res.json(); + // Only surface the bar once a secure session actually exists — i.e. after + // the first message. Nothing is created on the host until then. + if (!data.active) { secureBar.hidden = true; return; } + secureBar.hidden = false; + tickSecureTtl(); + if (secureSizeEl) { + // Only show a size once there's actually something in the workspace — + // "0 B" is noise. + const size = data.workspaceSize != null ? data.workspaceSize : 0; + if (size > 0) { + secureSizeEl.textContent = formatBytes(size); + secureSizeEl.hidden = false; + } else { + secureSizeEl.textContent = ''; + secureSizeEl.hidden = true; + } + } + } catch { secureBar.hidden = true; } + } + + // Live countdown — updates the TTL text every second. + setInterval(tickSecureTtl, 1000); + + secureChangeBtn?.addEventListener('click', (e) => { + e.stopPropagation(); + if (secureTtlMenu) { + secureTtlMenu.hidden = !secureTtlMenu.hidden; + } + }); + + secureTtlMenu?.addEventListener('click', (e) => { + const item = e.target.closest('[data-ttl]'); + if (!item) return; + saveSecureTtl(Number(item.dataset.ttl)); + secureTtlMenu.hidden = true; + refreshSecureBar(); + }); + + // Export the sandbox out to a host folder (default ~/Downloads). Read-only — + // copies the sandbox contents to a fresh place, touches nothing of the user's. + const secureExportBtn = document.getElementById('secure-workspace-export'); + secureExportBtn?.addEventListener('click', async (e) => { + e.stopPropagation(); + if (!rawChatId) return; + secureExportBtn.disabled = true; + const prev = secureExportBtn.textContent; + secureExportBtn.textContent = 'Exporting…'; + try { + const res = await fetch(`/chats/${rawChatId}/export-sandbox`, { + method: 'POST', + headers: { 'Content-Type': 'application/json', Accept: 'application/json' }, + body: '{}', + }); + const data = await res.json(); + if (data && data.ok) alert(`Exported ${data.files != null ? data.files + ' files' : 'sandbox'} to:\n${data.path}`); + else alert(`Export failed: ${(data && data.error) || 'unknown error'}`); + } catch { alert('Export failed.'); } + secureExportBtn.disabled = false; + secureExportBtn.textContent = prev || 'Export'; + }); + + // Apply: copy the sandbox's new/changed files back to the ORIGINAL folders. + // This writes to the user's real files, so it's confirmed and additive only. + const secureApplyBtn = document.getElementById('secure-workspace-apply'); + secureApplyBtn?.addEventListener('click', async (e) => { + e.stopPropagation(); + if (!rawChatId) return; + if (!confirm('Apply the sandbox\'s new and changed files back to your original folders? Existing files may be overwritten; nothing is deleted.')) return; + secureApplyBtn.disabled = true; + const prev = secureApplyBtn.textContent; + secureApplyBtn.textContent = 'Applying…'; + try { + const res = await fetch(`/chats/${rawChatId}/apply-sandbox`, { + method: 'POST', + headers: { Accept: 'application/json' }, + }); + const data = await res.json(); + const results = (data && data.results) || []; + if (results.length === 0) { + alert('Nothing to apply — no folders were copied into this sandbox.'); + } else { + const lines = results.map((r) => r.ok + ? `• ${r.source}: ${(r.applied && r.applied.length) || 0} file(s)` + : `✗ ${r.source}: ${r.error}`); + alert('Applied changes:\n' + lines.join('\n')); + } + } catch { alert('Apply failed.'); } + secureApplyBtn.disabled = false; + secureApplyBtn.textContent = prev || 'Apply changes'; + }); + + // Destroy the sandbox: deletes the copied workspace + container. The next + // message starts a fresh sandbox (re-copying any selected folders). + const secureDestroyBtn = document.getElementById('secure-workspace-destroy'); + secureDestroyBtn?.addEventListener('click', async (e) => { + e.stopPropagation(); + if (!rawChatId) return; + if (!confirm('Destroy this sandbox? The copied files and anything created in it are deleted. Your original files are untouched.')) return; + secureDestroyBtn.disabled = true; + const prev = secureDestroyBtn.textContent; + secureDestroyBtn.textContent = 'Destroying…'; + try { + await fetch(`/chats/${rawChatId}/destroy-workspace`, { + method: 'POST', + headers: { Accept: 'application/json' }, + }); + } catch { /* best-effort */ } + secureDestroyBtn.disabled = false; + secureDestroyBtn.textContent = prev || 'Destroy'; + refreshSecureBar(); + }); + + document.addEventListener('click', (e) => { + if (secureTtlMenu && !secureTtlMenu.hidden && !secureChangeBtn?.contains(e.target) && !secureTtlMenu.contains(e.target)) { + secureTtlMenu.hidden = true; + } + }); + + // Refresh bar when mode changes + composerModeMenu?.addEventListener('click', () => setTimeout(refreshSecureBar, 50)); + refreshSecureBar(); + + // ------------------------------------------------------------------------- + // Speech-to-text (mic). Records via MediaRecorder, POSTs the clip to + // /api/stt, and inserts the returned transcript into the composer textarea. + // Only wired when the server rendered the button (OPENROUTER_API_KEY set). + // Hold to record; slide left to cancel; slide up to lock hands-free. + // ------------------------------------------------------------------------- + const micBtn = document.getElementById('composer-mic-btn'); + if (micBtn && navigator.mediaDevices && window.MediaRecorder) { + // Telegram/WhatsApp-style hold-to-record gesture ported from the Flutter + // app: hold the mic to record, slide left to cancel, slide up to lock + // hands-free. A Web Audio meter drives the live waveform + amplitude halo. + // The post-release pipeline (audio → /api/stt → transcript) is unchanged. + const recEl = document.getElementById('composer-recording'); + const recTimeEl = document.getElementById('composer-recording-time'); + const recHintEl = document.getElementById('composer-recording-hint'); + const recCancelBtn = document.getElementById('composer-recording-cancel'); + const recWave = document.getElementById('composer-recording-wave'); + const lockHintEl = document.getElementById('composer-lock-hint'); + const composerFieldEl = micBtn.closest('.composer-field'); + + const CANCEL_DX = 72; // px dragged left to arm cancel + const LOCK_DY = 96; // px dragged up to lock hands-free + const MIN_MS = 800; // discard clips shorter than this + const REST_HINT = '‹ slide to cancel · slide up to lock'; + + let mediaRecorder = null; + let mediaStream = null; + let micChunks = []; + // phase: 'idle' | 'starting' | 'recording' | 'locked' + let phase = 'idle'; + let pendingSend = false; + let willCancel = false; + let ignoreNextUp = false; + let startedAt = 0; + let activePointerId = null; + let startX = 0; + let startY = 0; + let holdHintTimer = 0; + // Caret captured when recording starts, so the transcript is inserted at + // the cursor (dictate-into-text) instead of appended at the end. + let savedSelStart = null; + let savedSelEnd = null; + + // Web Audio meter (live waveform + amplitude halo). + let audioCtx = null; + let analyser = null; + let sourceNode = null; + let meterRaf = 0; + let timeData = null; + let waveSamples = []; + let accentColor = '#4f8cff'; + let dangerColor = '#e5484d'; + + function setMicState(state) { + micBtn.dataset.state = state; + micBtn.setAttribute( + 'aria-pressed', + state === 'recording' || state === 'locked' ? 'true' : 'false', + ); + micBtn.title = + state === 'recording' + ? 'Release to send · slide to cancel' + : state === 'locked' + ? 'Tap to send' + : state === 'busy' + ? 'Transcribing…' + : 'Hold to record'; + } + + function pickMicMime() { + if (!window.MediaRecorder || !MediaRecorder.isTypeSupported) return ''; + const cands = [ + 'audio/webm;codecs=opus', + 'audio/webm', + 'audio/ogg;codecs=opus', + 'audio/mp4', + 'audio/mpeg', + ]; + return cands.find((t) => MediaRecorder.isTypeSupported(t)) || ''; + } + + function stopMicStream() { + if (mediaStream) { + mediaStream.getTracks().forEach((t) => t.stop()); + mediaStream = null; + } + } + + function insertTranscript(text) { + if (!input || !text) return; + const val = input.value || ''; + // Insert at the caret captured when recording began; fall back to the end + // when the field wasn't focused (savedSel* are null). + let start = typeof savedSelStart === 'number' ? savedSelStart : val.length; + let end = typeof savedSelEnd === 'number' ? savedSelEnd : val.length; + start = Math.max(0, Math.min(start, val.length)); + end = Math.max(start, Math.min(end, val.length)); + const before = val.slice(0, start); + const after = val.slice(end); + // Pad so dictated words don't collide with the surrounding text. + const lead = before && !/\s$/.test(before) ? ' ' : ''; + const trail = after && !/^\s/.test(after) ? ' ' : ''; + const piece = lead + text + trail; + input.value = before + piece + after; + const caret = before.length + (lead + text).length; + input.dispatchEvent(new Event('input', { bubbles: true })); + input.focus(); + try { + input.setSelectionRange(caret, caret); + } catch (_) {} + // Keep the caret in sync for a possible next dictation. + savedSelStart = caret; + savedSelEnd = caret; + } + + async function transcribeBlob(blob) { + const res = await fetch('/api/stt', { + method: 'POST', + headers: { 'Content-Type': blob.type || 'audio/webm' }, + body: blob, + }); + if (!res.ok) { + let msg = 'HTTP ' + res.status; + try { + const j = await res.json(); + if (j && j.error) msg = j.error; + } catch (_) {} + throw new Error(msg); + } + const j = await res.json(); + return j && typeof j.text === 'string' ? j.text.trim() : ''; + } + + async function onMicStop() { + stopMeter(); + stopMicStream(); + const send = pendingSend; + const type = (mediaRecorder && mediaRecorder.mimeType) || 'audio/webm'; + const blob = new Blob(micChunks, { type }); + micChunks = []; + if (!send || !blob.size) { + setMicState('idle'); + return; + } + setMicState('busy'); + try { + const text = await transcribeBlob(blob); + if (text) insertTranscript(text); + } catch (err) { + window.alert( + 'Transcription failed: ' + (err && err.message ? err.message : 'unknown error'), + ); + } finally { + setMicState('idle'); + } + } + + // ----- recording UI ----- + function fmtTime(ms) { + const cs = Math.floor(ms / 10) % 100; + const totalSec = Math.floor(ms / 1000); + const p2 = (n) => String(n).padStart(2, '0'); + return p2(Math.floor(totalSec / 60)) + ':' + p2(totalSec % 60) + '.' + p2(cs); + } + + function readThemeColors() { + try { + const cs = getComputedStyle(document.documentElement); + const a = cs.getPropertyValue('--accent').trim(); + const d = cs.getPropertyValue('--danger').trim(); + if (a) accentColor = a; + if (d) dangerColor = d; + } catch (_) {} + } + + function sizeWaveCanvas() { + if (!recWave) return; + const dpr = window.devicePixelRatio || 1; + const w = recWave.clientWidth || 160; + recWave.width = Math.max(1, Math.round(w * dpr)); + recWave.height = Math.round(26 * dpr); + } + + function positionLockHint() { + if (!lockHintEl || !composerFieldEl) return; + const r = micBtn.getBoundingClientRect(); + const fr = composerFieldEl.getBoundingClientRect(); + lockHintEl.style.left = r.left - fr.left + r.width / 2 + 'px'; + lockHintEl.style.bottom = fr.bottom - r.top + 10 + 'px'; + } + + function showRecordingUI() { + readThemeColors(); + window.clearTimeout(holdHintTimer); + if (composerFieldEl) composerFieldEl.classList.add('is-recording'); + if (recEl) { + recEl.classList.remove('is-cancel', 'is-locked', 'is-hint'); + recEl.hidden = false; + } + micBtn.classList.remove('is-cancel'); + if (recHintEl) recHintEl.textContent = REST_HINT; + if (recTimeEl) recTimeEl.textContent = '00:00.00'; + sizeWaveCanvas(); + if (lockHintEl) { + lockHintEl.classList.remove('is-locked'); + lockHintEl.style.setProperty('--lock-progress', '0'); + lockHintEl.hidden = false; + positionLockHint(); + } + } + + function hideRecordingUI() { + if (composerFieldEl) composerFieldEl.classList.remove('is-recording'); + if (recEl) { + recEl.hidden = true; + recEl.classList.remove('is-cancel', 'is-locked', 'is-hint'); + } + if (lockHintEl) lockHintEl.hidden = true; + micBtn.classList.remove('is-cancel'); + micBtn.style.setProperty('--mic-amp', '0'); + } + + function setCancelArmed(on) { + if (willCancel === on) return; + willCancel = on; + if (recEl) recEl.classList.toggle('is-cancel', on); + micBtn.classList.toggle('is-cancel', on); + if (recHintEl && phase === 'recording') { + recHintEl.textContent = on ? 'release to cancel' : REST_HINT; + } + } + + function setLockProgress(p) { + if (lockHintEl) { + lockHintEl.style.setProperty('--lock-progress', String(Math.max(0, Math.min(1, p)))); + } + } + + function flashHoldHint() { + // Quick tap (no real recording) → briefly coach the user to hold. + if (!recEl) return; + if (composerFieldEl) composerFieldEl.classList.add('is-recording'); + recEl.classList.add('is-hint'); + recEl.hidden = false; + if (recHintEl) recHintEl.textContent = 'hold the mic to record'; + window.clearTimeout(holdHintTimer); + holdHintTimer = window.setTimeout(() => { + recEl.classList.remove('is-hint'); + if (phase === 'idle') { + recEl.hidden = true; + if (composerFieldEl) composerFieldEl.classList.remove('is-recording'); + } + }, 1100); + } + + // ----- Web Audio meter ----- + function startMeter(stream) { + try { + const Ctx = window.AudioContext || window.webkitAudioContext; + audioCtx = new Ctx(); + if (audioCtx.state === 'suspended') audioCtx.resume().catch(() => {}); + sourceNode = audioCtx.createMediaStreamSource(stream); + analyser = audioCtx.createAnalyser(); + analyser.fftSize = 512; + analyser.smoothingTimeConstant = 0.65; + sourceNode.connect(analyser); + timeData = new Uint8Array(analyser.fftSize); + waveSamples = []; + } catch (_) { + // Meter is best-effort; recording still works without it. + } + meterRaf = requestAnimationFrame(meterFrame); + } + + function stopMeter() { + if (meterRaf) cancelAnimationFrame(meterRaf); + meterRaf = 0; + try { if (sourceNode) sourceNode.disconnect(); } catch (_) {} + try { if (audioCtx) audioCtx.close(); } catch (_) {} + sourceNode = null; + analyser = null; + audioCtx = null; + timeData = null; + } + + function meterFrame() { + meterRaf = requestAnimationFrame(meterFrame); + if (phase === 'recording' || phase === 'locked') { + if (recTimeEl) recTimeEl.textContent = fmtTime(Date.now() - startedAt); + } + let level = 0; + if (analyser && timeData) { + analyser.getByteTimeDomainData(timeData); + let sum = 0; + for (let i = 0; i < timeData.length; i++) { + const v = (timeData[i] - 128) / 128; + sum += v * v; + } + level = Math.min(1, Math.sqrt(sum / timeData.length) * 2.4); + } + micBtn.style.setProperty('--mic-amp', level.toFixed(3)); + waveSamples.push(level); + drawWave(); + } + + function drawWave() { + if (!recWave) return; + const ctx = recWave.getContext('2d'); + if (!ctx) return; + const dpr = window.devicePixelRatio || 1; + const w = recWave.width; + const h = recWave.height; + ctx.clearRect(0, 0, w, h); + const barW = 2 * dpr; + const step = barW + 2 * dpr; + const n = Math.max(1, Math.floor(w / step)); + if (waveSamples.length > n) waveSamples = waveSamples.slice(-n); + const data = waveSamples; + const mid = h / 2; + ctx.fillStyle = willCancel ? dangerColor : accentColor; + for (let i = 0; i < data.length; i++) { + const bh = Math.max(2 * dpr, data[i] * h * 0.92); + const x = w - (data.length - i) * step; + const r = barW / 2; + ctx.beginPath(); + if (ctx.roundRect) ctx.roundRect(x, mid - bh / 2, barW, bh, r); + else ctx.rect(x, mid - bh / 2, barW, bh); + ctx.fill(); + } + } + + // ----- lifecycle ----- + async function beginRecording() { + try { + mediaStream = await navigator.mediaDevices.getUserMedia({ audio: true }); + } catch (_) { + phase = 'idle'; + activePointerId = null; + hideRecordingUI(); + window.alert('Microphone access was blocked. Allow it in your browser to use voice input.'); + return; + } + // The user may have released or cancelled while the permission prompt was + // open. If we're no longer arming, drop the freshly acquired stream. + if (phase !== 'starting') { + stopMicStream(); + return; + } + micChunks = []; + const mimeType = pickMicMime(); + try { + mediaRecorder = mimeType + ? new MediaRecorder(mediaStream, { mimeType }) + : new MediaRecorder(mediaStream); + } catch (_) { + mediaRecorder = new MediaRecorder(mediaStream); + } + mediaRecorder.addEventListener('dataavailable', (e) => { + if (e.data && e.data.size) micChunks.push(e.data); + }); + mediaRecorder.addEventListener('stop', onMicStop); + mediaRecorder.start(); + phase = 'recording'; + startedAt = Date.now(); + pendingSend = false; + willCancel = false; + setMicState('recording'); + showRecordingUI(); + startMeter(mediaStream); + try { if (input) input.blur(); } catch (_) {} + } + + function finishRecording(send) { + if (phase !== 'recording' && phase !== 'locked') return; + const longEnough = Date.now() - startedAt >= MIN_MS; + pendingSend = !!send && longEnough && !willCancel; + if (send && !longEnough && navigator.vibrate) { + try { navigator.vibrate([15, 40, 15]); } catch (_) {} + } + phase = 'idle'; + willCancel = false; + ignoreNextUp = false; + activePointerId = null; + hideRecordingUI(); + if (send && !longEnough) flashHoldHint(); + if (mediaRecorder && mediaRecorder.state !== 'inactive') { + try { mediaRecorder.stop(); } catch (_) { onMicStop(); } + } else { + onMicStop(); + } + } + + function cancelRecording() { + finishRecording(false); + } + + function lockRecording() { + if (phase !== 'recording') return; + phase = 'locked'; + ignoreNextUp = true; + setCancelArmed(false); + setMicState('locked'); + if (recEl) recEl.classList.add('is-locked'); + if (recHintEl) recHintEl.textContent = 'tap mic to send'; + if (lockHintEl) { + lockHintEl.classList.add('is-locked'); + setLockProgress(1); + } + try { micBtn.releasePointerCapture(activePointerId); } catch (_) {} + sizeWaveCanvas(); + } + + // ----- pointer gesture ----- + micBtn.addEventListener('pointerdown', (e) => { + e.preventDefault(); + if (micBtn.dataset.state === 'busy') return; + // In locked mode a fresh tap on the mic sends. + if (phase === 'locked') { + finishRecording(true); + return; + } + if (phase !== 'idle') return; + // Capture the caret so the transcript lands where the user put it. Only + // when the textarea is focused; otherwise append at the end (null). + if (input && document.activeElement === input) { + savedSelStart = input.selectionStart; + savedSelEnd = input.selectionEnd; + } else { + savedSelStart = null; + savedSelEnd = null; + } + phase = 'starting'; + willCancel = false; + activePointerId = e.pointerId; + startX = e.clientX; + startY = e.clientY; + try { micBtn.setPointerCapture(e.pointerId); } catch (_) {} + beginRecording(); + }); + + micBtn.addEventListener('pointermove', (e) => { + if (phase !== 'recording') return; + if (activePointerId !== null && e.pointerId !== activePointerId) return; + const dx = startX - e.clientX; // leftward positive + const dy = startY - e.clientY; // upward positive + setCancelArmed(dx >= CANCEL_DX); + if (willCancel) { + setLockProgress(0); + return; + } + setLockProgress(dy / LOCK_DY); + if (dy >= LOCK_DY) lockRecording(); + }); + + function onPointerEnd(e) { + if (activePointerId !== null && e.pointerId !== activePointerId) return; + if (phase === 'starting') { + // Released before recording actually began → treat as a quick tap. + phase = 'idle'; + activePointerId = null; + stopMicStream(); + flashHoldHint(); + return; + } + if (phase === 'recording') { + finishRecording(!willCancel); + return; + } + if (phase === 'locked' && ignoreNextUp) { + // The release that triggered the lock — keep recording hands-free. + ignoreNextUp = false; + } + } + + micBtn.addEventListener('pointerup', onPointerEnd); + micBtn.addEventListener('pointercancel', (e) => { + if (activePointerId !== null && e.pointerId !== activePointerId) return; + if (phase === 'starting') { + phase = 'idle'; + activePointerId = null; + stopMicStream(); + return; + } + if (phase === 'recording') cancelRecording(); + }); + + if (recCancelBtn) { + recCancelBtn.addEventListener('click', (e) => { + e.preventDefault(); + cancelRecording(); + }); + } + + window.addEventListener('resize', () => { + if (phase === 'recording' || phase === 'locked') { + sizeWaveCanvas(); + positionLockHint(); + } + }); + } + /** Clears timed reply-quote highlights in the transcript. */ let replyJumpHighlightTimer = null; let replyJumpHighlightFadeTimer = null; @@ -129,8 +1486,15 @@ } else { messagesEl.dataset.projectId = ''; } + if (typeof updateWorkFoldersButton === 'function') updateWorkFoldersButton(); } history.replaceState(null, '', '/chats/' + id); + promoteDraftHeaderTools(id, payload.projectId); + // Migrate mode from the draft fallback key to the per-chat key + try { + const draftMode = localStorage.getItem('iclaw:composer-mode'); + if (draftMode) localStorage.setItem(`iclaw:composer-mode:${id}`, draftMode); + } catch (_) {} applyTitleForActive(payload.title || 'New chat'); if (ws && ws.readyState === WebSocket.OPEN) { wsSend({ type: 'subscribe', chatId: id }); @@ -149,6 +1513,35 @@ syncComposerSecretUi(); } + /** + * Promote the dormant draft header tools into a live chat's tools once the row + * exists. The draft renders them hidden + action-less (see + * partials/header-chat-tools.ejs); here we fill in the /chats/:id actions and + * reveal them — so Share, Delete (and Suggest-facts, when the draft + * chose a project) show up immediately instead of only after a reload. Queries + * are scoped to the header so they don't hit the share modal's own .share-form. + */ + function promoteDraftHeaderTools(id, projectId) { + if (!startedOnDraft) return; + const header = document.querySelector('.chat-header-tools'); + if (!header) return; + const shareBtn = document.getElementById('share-btn'); + if (shareBtn) shareBtn.hidden = false; + const delForm = header.querySelector('.delete-form'); + if (delForm) { + delForm.setAttribute('action', '/chats/' + id + '/delete'); + delForm.hidden = false; + } + const hasProject = + (projectId != null && Number.isFinite(Number(projectId))) || + (draftChosenProjectId != null && Number.isFinite(draftChosenProjectId)); + const sharesForm = header.querySelector('.share-form'); + if (sharesForm && hasProject) { + sharesForm.setAttribute('action', '/chats/' + id + '/shares'); + sharesForm.hidden = false; + } + } + async function ensureDraftChatRow() { if (activeChatId != null) return; const body = { agent: draftAgentSelect?.value || 'openclaw/default' }; @@ -326,6 +1719,21 @@ void commitDraftFromCard(card); }); + // Space = "No project" — a quick skip past project selection. Guarded to the + // active picking stage so it never fires afterwards, and ignored while a text + // field is focused. commitDraftFromCard's re-entry guard makes this safe even + // if a card button also has focus. + document.addEventListener('keydown', (e) => { + if (e.key !== ' ' && e.code !== 'Space') return; + if (draftProjectLocked || !draftBody.classList.contains('is-picking')) return; + const t = e.target; + if (t && (t.tagName === 'INPUT' || t.tagName === 'TEXTAREA' || t.isContentEditable)) return; + const none = projectPickEl.querySelector('.project-pick-card--none'); + if (!none) return; + e.preventDefault(); + void commitDraftFromCard(none); + }); + const initSel = (projectPickEl.dataset.initialProjectId || '').trim(); if (initSel !== '') { const esc = @@ -334,9 +1742,35 @@ : initSel.replace(/\\/g, '\\\\').replace(/"/g, '\\"'); const card = projectPickEl.querySelector('.project-pick-card[data-project-id="' + esc + '"]'); if (card) queueMicrotask(() => void commitDraftFromCard(card)); + } else if (draftBody?.dataset.autoNone === '1') { + // First-ever chat: skip the "Choose a project" step so a new user lands + // straight on the welcome greeting + composer (No project). + const card = projectPickEl.querySelector('.project-pick-card--none'); + if (card) queueMicrotask(() => void commitDraftFromCard(card)); } } + // Welcome-card suggestion chips: drop the text into the composer and send it, + // so a non-technical user gets going with one click. + function initWelcomeChips() { + const card = document.getElementById('welcome-card'); + if (!card) return; + card.addEventListener('click', (e) => { + const chip = e.target.closest('.welcome-chip'); + if (!chip) return; + const prompt = chip.getAttribute('data-prompt') || chip.textContent || ''; + const ta = document.getElementById('composer-input'); + const form = document.getElementById('send-form'); + if (!ta || !form) return; + ta.value = prompt; + ta.dispatchEvent(new Event('input', { bubbles: true })); + if (typeof form.requestSubmit === 'function') form.requestSubmit(); + else form.dispatchEvent(new Event('submit', { cancelable: true, bubbles: true })); + }); + } + + initWelcomeChips(); + initDraftProjectPick(); // serializes turns per chat too, so this is just for the visible label const waitingItems = []; @@ -379,6 +1813,10 @@ /** the assistant DOM node we're streaming into right now */ let currentStreamEl = null; let currentStreamFullText = ''; + /** Pending requestAnimationFrame id for the streaming typewriter (0 = none). */ + let streamRenderRaf = 0; + /** How many chars of currentStreamFullText the typewriter has revealed. */ + let streamShownLen = 0; /** Debounce hljs while `turn-delta` re-renders markdown (innerHTML each chunk). */ let streamSyntaxHlTimer = null; @@ -629,6 +2067,63 @@ refreshProjectTabLabels(active || 'chats'); } + function syncProjectSkillsTabCountFromDom() { + const root = document.querySelector('main.project-page[data-project-id]'); + const btn = root?.querySelector('[data-project-tab="skills"]'); + const ul = document.getElementById('skills-list'); + if (!btn || !ul) return; + btn.setAttribute('data-tab-count', String(ul.querySelectorAll('li.skill').length)); + const active = root.querySelector('.project-tab.is-active')?.getAttribute('data-project-tab'); + refreshProjectTabLabels(active || 'chats'); + } + + /** Build a skill row matching `views/project.ejs` (WS-driven updates on project page). */ + function buildSkillLi(s) { + const li = document.createElement('li'); + li.className = 'skill'; + li.dataset.skillId = String(s.id); + li.dataset.skillVersion = String(s.version != null ? s.version : 1); + const isGlobal = s.project_id == null; + const titleRaw = + s.source_chat_title != null && String(s.source_chat_title).trim() !== '' + ? String(s.source_chat_title).trim() + : 'Chat'; + const head = + '<div class="project-row-head muted">' + + (isGlobal + ? '<span class="skill-scope-badge" title="Available to every project">Global</span>' + : '') + + (s.source_chat_id != null + ? '<a href="/chats/' + + s.source_chat_id + + '" class="project-chat-source">' + + escapeHtml(titleRaw) + + '</a>' + : '<span>—</span>') + + '</div>'; + li.innerHTML = + head + + '<input class="skill-name" aria-label="Skill name" spellcheck="false" value="' + + escapeHtml(s.name || '') + + '" />' + + '<textarea class="skill-description" aria-label="Skill summary" rows="2">' + + escapeHtml(s.description || '') + + '</textarea>' + + '<details class="skill-body-details"><summary>View / edit procedure</summary>' + + '<textarea class="skill-body" aria-label="Skill procedure (SKILL.md)" rows="10">' + + escapeHtml(s.body || '') + + '</textarea></details>' + + '<div class="fact-meta">' + + '<button type="button" class="fact-delete skill-delete" aria-label="Remove skill">Remove</button></div>'; + const nameEl = li.querySelector('.skill-name'); + if (nameEl) nameEl.dataset.saved = String(s.name || '').trim(); + const descEl = li.querySelector('.skill-description'); + if (descEl) descEl.dataset.saved = String(s.description || '').trim(); + const bodyEl = li.querySelector('.skill-body'); + if (bodyEl) bodyEl.dataset.saved = String(s.body || '').trim(); + return li; + } + /** Build a secrets row matching `views/project.ejs` (WS-driven updates on project page). */ function buildProjectSecretRowLi(secret) { const label = String(secret?.label ?? ''); @@ -689,10 +2184,30 @@ const CODE_COPIED_ICON_SVG = '<svg class="code-copy-icon code-copy-icon--ok" viewBox="0 0 24 24" width="18" height="18" aria-hidden="true" focusable="false"><path fill="currentColor" d="M9 16.17L4.83 12l-1.42 1.41L9 19L21 7l-1.41-1.41L9 16.17z"/></svg>'; + /** Strip protocol/`www.`/trailing slash; middle-truncate pathological URLs. */ + function prettifyUrlText(href) { + let s = String(href || '') + .replace(/^https?:\/\//i, '') + .replace(/^www\./i, '') + .replace(/\/+$/, ''); + if (s.length > 60) s = s.slice(0, 42) + '…' + s.slice(-15); + return s; + } function decorateLinks(root) { root.querySelectorAll('a[href]').forEach((a) => { a.target = '_blank'; a.rel = 'noopener noreferrer'; + // Inside tables, collapse long bare-URL link text to a compact, readable + // form (full URL stays on the href + title). Leaves `[label](url)` links + // and prose URLs untouched. Idempotent: after rewrite text !== href. + if (a.closest('td, th')) { + const href = a.getAttribute('href') || ''; + const txt = (a.textContent || '').trim(); + if (txt && txt === href && /^https?:\/\//i.test(href)) { + if (!a.title) a.title = href; + a.textContent = prettifyUrlText(href); + } + } }); } @@ -700,7 +2215,7 @@ function enhanceCodeBlocks(root) { if (!root || root.nodeType !== 1) return; const pres = root.querySelectorAll( - '.msg-body pre, .stream-body pre, .reasoning-body pre, .task-log-entry-body pre', + '.msg-body pre, .stream-body pre, .task-log-entry-body pre', ); pres.forEach((pre) => { if (pre.parentElement?.classList.contains('code-block-wrap')) return; @@ -720,6 +2235,23 @@ }); } + /** Wrap GFM tables in a horizontal-scroll frame (after markdown → DOM). */ + function enhanceTables(root) { + if (!root || root.nodeType !== 1) return; + const tables = root.querySelectorAll( + '.msg-body table, .stream-body table, .task-log-entry-body table', + ); + tables.forEach((table) => { + if (table.parentElement?.classList.contains('md-table-wrap')) return; + const parent = table.parentElement; + if (!parent) return; + const wrap = document.createElement('div'); + wrap.className = 'md-table-wrap'; + parent.insertBefore(wrap, table); + wrap.appendChild(table); + }); + } + function clearStreamSyntaxHighlightSchedule() { if (streamSyntaxHlTimer != null) { clearTimeout(streamSyntaxHlTimer); @@ -740,16 +2272,32 @@ const hl = window.hljs; if (!root || root.nodeType !== 1 || !hl || typeof hl.highlightElement !== 'function') return; root.querySelectorAll( - '.msg-body pre code, .stream-body pre code, .reasoning-body pre code, .task-log-entry-body pre code', + '.msg-body pre code, .stream-body pre code, .task-log-entry-body pre code', ).forEach((code) => { const pre = code.parentElement; if (!pre || pre.tagName !== 'PRE') return; if (pre.closest('.exec-approval-card')) return; if (code.classList.contains('hljs')) return; - try { - hl.highlightElement(code); - } catch (_) { - /* unknown language / empty */ + // Only highlight when the fence declares a language hljs actually knows. + // On a bare/unknown fence, highlightElement() falls back to auto-detection, + // which mis-guesses prose as Ruby/Perl (an apostrophe opens a "string" and + // the github-dark theme dims whole paragraphs). Leave those as plain text. + const langClass = [...code.classList].find((c) => c.startsWith('language-')); + const lang = langClass && langClass.slice(9); + const known = + lang && + lang !== 'text' && + lang !== 'plaintext' && + typeof hl.getLanguage === 'function' && + hl.getLanguage(lang); + if (known) { + try { + hl.highlightElement(code); + } catch (_) { + code.classList.add('hljs'); /* keep block styling even if hljs throws */ + } + } else { + code.classList.add('hljs'); /* plaintext — styled, no auto-detect */ } }); } @@ -761,6 +2309,7 @@ function decorateMessageBody(root, opts) { decorateLinks(root); enhanceCodeBlocks(root); + enhanceTables(root); if (opts && opts.deferSyntaxHighlight) { scheduleStreamSyntaxHighlight(root); return; @@ -923,6 +2472,63 @@ function scrollToBottom() { if (messagesEl) messagesEl.scrollTop = messagesEl.scrollHeight; } + + /** True when the messages list is at/near the bottom (within `threshold` px). */ + function isNearBottom(threshold = 120) { + if (!messagesEl) return true; + return messagesEl.scrollHeight - messagesEl.scrollTop - messagesEl.clientHeight <= threshold; + } + + /** + * Smooth "typewriter" streaming. Backend deltas arrive in bursts and can stall + * for ~a second; we decouple what's shown from what's arrived. `currentStream + * FullText` is the target (grows as deltas arrive); `streamShownLen` is how much + * we've revealed. Each animation frame we reveal a few more characters, so the + * user sees a steady flow no matter how lumpy the source is. + * + * The reveal is adaptive — a larger backlog drains faster (so we never fall + * seconds behind on a big burst) while a small backlog types out gently. Re- + * rendering happens at most once per frame (re-parsing the whole markdown + + * replacing innerHTML per token would be O(n²) and thrash layout). Auto-scroll + * fires only when the user is already at the bottom, so scrolling up mid-stream + * no longer yanks them back down. + */ + function renderStreamFrame() { + streamRenderRaf = 0; + const el = currentStreamEl; + if (!el || !messagesEl?.contains(el)) return; + const body = el.querySelector('.stream-body, .msg-body'); + if (!body) return; + + const target = currentStreamFullText; + if (streamShownLen < target.length) { + const remaining = target.length - streamShownLen; + // ~120 chars/s floor (2 per frame), faster as the backlog grows. + const step = Math.max(2, Math.ceil(remaining / 8)); + streamShownLen = Math.min(target.length, streamShownLen + step); + const stick = isNearBottom(); + body.innerHTML = renderMarkdown(target.slice(0, streamShownLen)); + decorateMessageBody(body, { deferSyntaxHighlight: true }); + if (stick) scrollToBottom(); + } + + // Keep going while there's still backlog; otherwise idle until the next + // delta re-arms the loop via ensureTyping(). + if (streamShownLen < currentStreamFullText.length) { + streamRenderRaf = requestAnimationFrame(renderStreamFrame); + } + } + + /** Start the typewriter loop if it isn't already running. */ + function ensureTyping() { + if (!streamRenderRaf) streamRenderRaf = requestAnimationFrame(renderStreamFrame); + } + + /** Stop the typewriter and reset the revealed position (call when finalizing). */ + function cancelStreamRender() { + if (streamRenderRaf) { cancelAnimationFrame(streamRenderRaf); streamRenderRaf = 0; } + streamShownLen = 0; + } /** Build the HTML block for persisted attachments (image inline / file as link). */ function attachmentsHtml(attachments) { if (!Array.isArray(attachments) || attachments.length === 0) return ''; @@ -962,6 +2568,30 @@ return '<div class="msg-attachments">' + items + '</div>'; } + /** Dev mode: add a live message's tokens to the chat-wide running total. */ + function bumpChatTokenTotal(tokens) { + if (!window.__ICLAW_DEV__ || !tokens) return; + const el = document.getElementById('chat-token-total'); + if (!el) return; + const cur = (Number(el.dataset.total) || 0) + Number(tokens); + el.dataset.total = String(cur); + el.textContent = cur.toLocaleString() + ' tok total'; + el.hidden = false; + } + + /** Dev mode: show token usage (+cache hits) on a message bubble + chat total. */ + function applyTokenBadge(el, tokens, cached) { + if (!window.__ICLAW_DEV__ || !tokens || !el) return; + if (el.querySelector(':scope > .msg-tokens')) return; + const c = Number(cached) || 0; + const span = document.createElement('span'); + span.className = 'msg-tokens'; + span.title = 'Tokens spent on this reply' + (c ? ' (' + c.toLocaleString() + ' served from cache)' : ''); + span.textContent = Number(tokens).toLocaleString() + ' tok' + (c ? ' · ' + c.toLocaleString() + ' cached' : ''); + el.appendChild(span); + bumpChatTokenTotal(tokens); + } + function appendMessage(msg, opts) { if (!messagesEl) return null; clearEmptyState(); @@ -999,6 +2629,7 @@ '<div class="msg-body">' + renderMessageHtml(msg.content || '') + '</div>' + attachmentsHtml(msg.attachments); decorateMessageBody(div); + applyTokenBadge(div, msg.tokens, msg.cached_tokens); messagesAppendRoot().appendChild(div); scrollToBottom(); return div; @@ -1006,15 +2637,19 @@ function appendStreamingAssistant() { if (!messagesEl) return null; clearEmptyState(); + // First turn of a chat can be a cold start (model/Docker warming up, 20–60s). + // Say "Warming up…" instead of "Thinking…" so the wait reads as honest setup, + // not a hang. Detected by the absence of any prior assistant message. + const isFirstTurn = !messagesAppendRoot()?.querySelector('.msg.assistant'); const div = document.createElement('div'); div.className = 'msg assistant streaming stream-waiting'; div.innerHTML = '<div class="role">assistant</div>' + - '<div class="stream-status"></div>' + - '<div class="msg-body stream-body"></div>'; + '<div class="msg-body stream-body"></div>' + + '<div class="stream-status"></div>'; messagesAppendRoot().appendChild(div); const st = div.querySelector('.stream-status'); - if (st) setStreamStatusLabel(st, 'Thinking…'); + if (st) setStreamStatusLabel(st, isFirstTurn ? 'Warming up…' : 'Thinking…'); scrollToBottom(); return div; } @@ -1251,88 +2886,263 @@ }); } - async function processFactSuggestionExpiryQueue() { - if (processingFactSuggestionExpiry) return; - processingFactSuggestionExpiry = true; - try { - while (factSuggestionExpiryQueue.length > 0) { - const row = factSuggestionExpiryQueue[0]; - if (!row || !document.contains(row)) { - factSuggestionExpiryQueue.shift(); - continue; - } - const btn = row.querySelector('.fact-suggestion-reject'); - if (!btn) { - factSuggestionExpiryQueue.shift(); - continue; - } - await runSingleFactRejectExpiry(row, btn); - factSuggestionExpiryQueue.shift(); - } - } finally { - processingFactSuggestionExpiry = false; - if (factSuggestionExpiryQueue.length > 0) void processFactSuggestionExpiryQueue(); - } + async function processFactSuggestionExpiryQueue() { + if (processingFactSuggestionExpiry) return; + processingFactSuggestionExpiry = true; + try { + while (factSuggestionExpiryQueue.length > 0) { + const row = factSuggestionExpiryQueue[0]; + if (!row || !document.contains(row)) { + factSuggestionExpiryQueue.shift(); + continue; + } + const btn = row.querySelector('.fact-suggestion-reject'); + if (!btn) { + factSuggestionExpiryQueue.shift(); + continue; + } + await runSingleFactRejectExpiry(row, btn); + factSuggestionExpiryQueue.shift(); + } + } finally { + processingFactSuggestionExpiry = false; + if (factSuggestionExpiryQueue.length > 0) void processFactSuggestionExpiryQueue(); + } + } + + function appendFactSuggestionsCard(opts) { + if (!messagesEl) return; + const { projectId, chatId, suggestions, projectName } = opts; + if (!suggestions || suggestions.length === 0) return; + clearEmptyState(); + const safeName = escapeHtml((projectName || '').trim() || 'project'); + const rowsHtml = suggestions.map(buildFactSuggestionRowHtml).filter(Boolean).join(''); + if (!rowsHtml) return; + + const pidEsc = String(projectId); + const cidEsc = String(chatId); + const existing = messagesEl.querySelector( + '.fact-suggestions-card[data-project-id="' + pidEsc + '"][data-chat-id="' + cidEsc + '"]', + ); + if (existing) { + const ul = existing.querySelector('.fact-suggestions-list'); + if (!ul) return; + const tpl = document.createElement('template'); + tpl.innerHTML = rowsHtml.trim(); + const added = []; + tpl.content.childNodes.forEach((n) => { + if (n.nodeType !== 1) return; + const el = /** @type {HTMLElement} */ (n); + const sid = el.dataset.suggestionId; + if (!sid || ul.querySelector('.fact-suggestion-row[data-suggestion-id="' + sid + '"]')) return; + ul.appendChild(el); + added.push(el); + }); + scrollToBottom(); + enqueueFactSuggestionExpiryRows(added); + return; + } + + const card = document.createElement('div'); + card.className = 'msg system fact-suggestions-card'; + card.dataset.projectId = pidEsc; + card.dataset.chatId = cidEsc; + card.innerHTML = + '<div class="fact-suggestions-shell">' + + '<p class="fact-suggestions-lead">Save to project memory «' + + safeName + + '»?</p>' + + '<ul class="fact-suggestions-list" role="list">' + + rowsHtml + + '</ul>' + + '</div>'; + messagesAppendRoot().appendChild(card); + scrollToBottom(); + const addedRows = Array.from(card.querySelectorAll('.fact-suggestion-row')); + enqueueFactSuggestionExpiryRows(addedRows); + } + + async function loadPendingFactSuggestions() { + if (activeChatId == null || !messagesEl) return; + try { + const res = await fetch('/chats/' + encodeURIComponent(activeChatId) + '/fact-suggestions', { + headers: { Accept: 'application/json' }, + }); + if (!res.ok) return; + const data = await res.json(); + const list = Array.isArray(data.suggestions) ? data.suggestions : []; + if (list.length === 0) return; + const first = list[0]; + const pid = first ? Number(first.project_id) : NaN; + if (!Number.isFinite(pid)) return; + const have = existingFactSuggestionIds(); + const fresh = list + .filter((s) => s && Number.isFinite(Number(s.id)) && !have.has(Number(s.id))) + .map((s) => ({ id: Number(s.id), content: String(s.content ?? '') })); + const pname = + typeof data.projectName === 'string' && data.projectName.trim() + ? data.projectName.trim() + : null; + const projectLabel = pname || 'project'; + appendFactSuggestionsCard({ + projectId: pid, + chatId: activeChatId, + projectName: projectLabel, + suggestions: fresh, + }); + } catch (_) { + /* ignore */ + } + } + + // ------------------------------------------------------------------------- + // skill suggestions (inbox-gated procedural memory) — chat cards + // Mirrors fact suggestions, but with no auto-reject: a skill is a standing + // instruction, so acceptance is always a deliberate user action. + // ------------------------------------------------------------------------- + + function existingSkillSuggestionIds() { + const ids = new Set(); + if (!messagesEl) return ids; + messagesEl.querySelectorAll('.skill-suggestion-row[data-suggestion-id]').forEach((el) => { + const n = Number(el.dataset.suggestionId); + if (Number.isFinite(n)) ids.add(n); + }); + return ids; + } + + function removeSkillSuggestionRow(chatId, sid) { + if (!messagesEl || chatId !== activeChatId) return; + const row = messagesEl.querySelector( + '.skill-suggestion-row[data-suggestion-id="' + sid + '"]', + ); + if (!row) return; + const card = row.closest('.skill-suggestions-card'); + row.remove(); + if (card && !card.querySelector('.skill-suggestion-row')) card.remove(); } - function appendFactSuggestionsCard(opts) { + // "handle-sandbox-network" → "Handle sandbox network": a readable title from + // the kebab slug, used until/unless the server sends a friendlier s.title. + function humanizeSkillName(name) { + const s = String(name || '').replace(/[-_]+/g, ' ').trim(); + return s ? s.charAt(0).toUpperCase() + s.slice(1) : ''; + } + + // A non-technical person sees ONE plain sentence + two buttons. Everything + // technical (kebab name, full procedure, "all projects", provenance) is folded + // into a collapsed «Деталі» block — still in the DOM, so the accept handler + // reads the same fields whether or not it's ever opened. + function buildSkillSuggestionRowHtml(s) { + const id = Number(s.id); + if (!Number.isFinite(id)) return ''; + const kind = s.kind === 'patch' ? 'patch' : 'new'; + const humanTitle = + (s.title && String(s.title).trim()) || humanizeSkillName(s.name) || 'це'; + const updatesLine = + kind === 'patch' + ? '<div class="skill-suggestion-updates">Доповнює те, що вже вмію</div>' + : ''; + // Provenance kept — but as a quiet ⓘ, not a red "untrusted" badge. + const infoIcon = s.untrusted + ? '<span class="skill-suggestion-info" tabindex="0" role="img" aria-label="Підказано з матеріалу, що міг містити зовнішній вміст — перевір у «Деталях»" title="Підказано з матеріалу, що міг містити зовнішній (недовірений) вміст. Переглянь «Деталі», перш ніж зберігати.">ⓘ</span>' + : ''; + return ( + '<li class="skill-suggestion-row" data-suggestion-id="' + + id + + '" data-kind="' + + kind + + '" role="listitem">' + + '<div class="skill-suggestion-simple">' + + '<div class="skill-suggestion-title">' + + escapeHtml(humanTitle) + + infoIcon + + '</div>' + + (s.description + ? '<div class="skill-suggestion-summary">' + escapeHtml(s.description) + '</div>' + : '') + + updatesLine + + '</div>' + + '<details class="skill-suggestion-advanced"><summary>Деталі</summary>' + + '<label class="skill-suggestion-field"><span class="skill-suggestion-field-label">Назва</span>' + + '<input class="skill-suggestion-name" aria-label="Skill name" spellcheck="false" value="' + + escapeHtml(s.name || '') + + '" /></label>' + + '<label class="skill-suggestion-field"><span class="skill-suggestion-field-label">Опис</span>' + + '<textarea class="skill-suggestion-desc" aria-label="Skill summary" rows="2">' + + escapeHtml(s.description || '') + + '</textarea></label>' + + '<label class="skill-suggestion-field"><span class="skill-suggestion-field-label">Що саме робити</span>' + + '<textarea class="skill-suggestion-body" aria-label="Skill procedure (SKILL.md)" rows="10">' + + escapeHtml(s.body || '') + + '</textarea></label>' + + '<label class="skill-suggestion-scope"><input type="checkbox" class="skill-suggestion-global" /> Використовувати в усіх проєктах</label>' + + '</details>' + + '<div class="skill-suggestion-actions">' + + '<button type="button" class="skill-suggestion-btn skill-suggestion-reject" data-suggestion-id="' + + id + + '">Не треба</button>' + + '<button type="button" class="skill-suggestion-btn skill-suggestion-accept" data-suggestion-id="' + + id + + '">Запам\'ятати</button>' + + '</div></li>' + ); + } + + function appendSkillSuggestionsCard(opts) { if (!messagesEl) return; const { projectId, chatId, suggestions, projectName } = opts; if (!suggestions || suggestions.length === 0) return; clearEmptyState(); const safeName = escapeHtml((projectName || '').trim() || 'project'); - const rowsHtml = suggestions.map(buildFactSuggestionRowHtml).filter(Boolean).join(''); + const rowsHtml = suggestions.map(buildSkillSuggestionRowHtml).filter(Boolean).join(''); if (!rowsHtml) return; const pidEsc = String(projectId); const cidEsc = String(chatId); const existing = messagesEl.querySelector( - '.fact-suggestions-card[data-project-id="' + pidEsc + '"][data-chat-id="' + cidEsc + '"]', + '.skill-suggestions-card[data-project-id="' + pidEsc + '"][data-chat-id="' + cidEsc + '"]', ); if (existing) { - const ul = existing.querySelector('.fact-suggestions-list'); + const ul = existing.querySelector('.skill-suggestions-list'); if (!ul) return; const tpl = document.createElement('template'); tpl.innerHTML = rowsHtml.trim(); - const added = []; tpl.content.childNodes.forEach((n) => { if (n.nodeType !== 1) return; - const el = /** @type {HTMLElement} */ (n); + const el = n; const sid = el.dataset.suggestionId; - if (!sid || ul.querySelector('.fact-suggestion-row[data-suggestion-id="' + sid + '"]')) return; + if (!sid || ul.querySelector('.skill-suggestion-row[data-suggestion-id="' + sid + '"]')) + return; ul.appendChild(el); - added.push(el); }); scrollToBottom(); - enqueueFactSuggestionExpiryRows(added); return; } const card = document.createElement('div'); - card.className = 'msg system fact-suggestions-card'; + card.className = 'msg system skill-suggestions-card'; card.dataset.projectId = pidEsc; card.dataset.chatId = cidEsc; card.innerHTML = - '<div class="fact-suggestions-shell">' + - '<p class="fact-suggestions-lead">Save to project memory «' + + '<div class="skill-suggestions-shell">' + + '<p class="skill-suggestions-lead">💡 Запам\'ятати це для «' + safeName + - '»?</p>' + - '<ul class="fact-suggestions-list" role="list">' + + '», щоб наступного разу зробити швидше?</p>' + + '<ul class="skill-suggestions-list" role="list">' + rowsHtml + - '</ul>' + - '</div>'; + '</ul></div>'; messagesAppendRoot().appendChild(card); scrollToBottom(); - const addedRows = Array.from(card.querySelectorAll('.fact-suggestion-row')); - enqueueFactSuggestionExpiryRows(addedRows); } - async function loadPendingFactSuggestions() { + async function loadPendingSkillSuggestions() { if (activeChatId == null || !messagesEl) return; try { - const res = await fetch('/chats/' + encodeURIComponent(activeChatId) + '/fact-suggestions', { - headers: { Accept: 'application/json' }, - }); + const res = await fetch( + '/chats/' + encodeURIComponent(activeChatId) + '/skill-suggestions', + { headers: { Accept: 'application/json' } }, + ); if (!res.ok) return; const data = await res.json(); const list = Array.isArray(data.suggestions) ? data.suggestions : []; @@ -1340,19 +3150,25 @@ const first = list[0]; const pid = first ? Number(first.project_id) : NaN; if (!Number.isFinite(pid)) return; - const have = existingFactSuggestionIds(); + const have = existingSkillSuggestionIds(); const fresh = list .filter((s) => s && Number.isFinite(Number(s.id)) && !have.has(Number(s.id))) - .map((s) => ({ id: Number(s.id), content: String(s.content ?? '') })); + .map((s) => ({ + id: Number(s.id), + kind: s.kind === 'patch' ? 'patch' : 'new', + name: String(s.name ?? ''), + description: String(s.description ?? ''), + body: String(s.body ?? ''), + untrusted: !!s.untrusted, + })); const pname = typeof data.projectName === 'string' && data.projectName.trim() ? data.projectName.trim() - : null; - const projectLabel = pname || 'project'; - appendFactSuggestionsCard({ + : 'project'; + appendSkillSuggestionsCard({ projectId: pid, chatId: activeChatId, - projectName: projectLabel, + projectName: pname, suggestions: fresh, }); } catch (_) { @@ -1369,6 +3185,7 @@ serverId: row.id, id: String(row.id), content: row.content, + mode: row.mode || undefined, }; if (row.reply_to_message_id != null && row.reply_quote) { item.replyTo = { @@ -1398,6 +3215,7 @@ async function enqueueQueueOnServer(chatId, draft) { const body = { content: draft.content }; + if (draft.mode) body.mode = draft.mode; if (draft.replyTo) body.replyTo = draft.replyTo; if (draft.inlineSecrets && draft.inlineSecrets.length > 0) { body.inlineSecrets = draft.inlineSecrets; @@ -1592,6 +3410,59 @@ .catch(() => {}); return; } + const sAcc = e.target.closest('.skill-suggestion-accept'); + const sRej = e.target.closest('.skill-suggestion-reject'); + if (sAcc || sRej) { + const row = (sAcc || sRej).closest('.skill-suggestion-row'); + const sid = Number((sAcc || sRej).dataset.suggestionId); + if (!Number.isFinite(sid) || activeChatId == null) return; + e.preventDefault(); + if (sRej) { + fetch( + '/chats/' + + encodeURIComponent(activeChatId) + + '/skill-suggestions/' + + encodeURIComponent(sid) + + '/reject', + { method: 'POST', headers: { Accept: 'application/json' } }, + ) + .then((res) => { + if (res.ok) removeSkillSuggestionRow(activeChatId, sid); + }) + .catch(() => {}); + return; + } + // Accept — submit any inline edits + the scope choice. + const name = row?.querySelector('.skill-suggestion-name')?.value?.trim() || ''; + const description = row?.querySelector('.skill-suggestion-desc')?.value?.trim() || ''; + const body = row?.querySelector('.skill-suggestion-body')?.value?.trim() || ''; + const global = !!row?.querySelector('.skill-suggestion-global')?.checked; + const payload = { scope: global ? 'global' : 'project' }; + if (name) payload.name = name; + if (description) payload.description = description; + if (body) payload.body = body; + if (sAcc) sAcc.disabled = true; + fetch( + '/chats/' + + encodeURIComponent(activeChatId) + + '/skill-suggestions/' + + encodeURIComponent(sid) + + '/accept', + { + method: 'POST', + headers: { 'content-type': 'application/json', Accept: 'application/json' }, + body: JSON.stringify(payload), + }, + ) + .then((res) => { + if (res.ok) removeSkillSuggestionRow(activeChatId, sid); + else if (sAcc) sAcc.disabled = false; + }) + .catch(() => { + if (sAcc) sAcc.disabled = false; + }); + return; + } const status = e.target.closest('.stream-status.has-detail'); if (!status) return; const expanded = status.classList.toggle('detail-expanded'); @@ -2364,7 +4235,7 @@ headers: { Accept: 'application/json' }, }); if (!res.ok) throw new Error(String(res.status)); - window.location.assign('/'); + goTo('/'); } catch (err) { console.error('[iclaw] mark unread failed', err); } @@ -2376,7 +4247,12 @@ f.method = 'POST'; f.action = '/chats/' + encodeURIComponent(cid) + '/delete'; document.body.appendChild(f); - f.submit(); + // requestSubmit() (not submit()) fires a real submit event, so the E2E + // SPA layer can take it over the encrypted channel instead of a full + // navigation that bounces off the gate. Falls back to a normal submit + // when that layer isn't present (local, non-tunnel). + if (typeof f.requestSubmit === 'function') f.requestSubmit(); + else f.submit(); } }); @@ -2544,7 +4420,7 @@ if ( el.classList.contains('streaming') || el.classList.contains('fact-suggestions-card') || - el.classList.contains('reasoning-block') + el.classList.contains('skill-suggestions-card') ) { return null; } @@ -2753,6 +4629,7 @@ if (activeChatId != null) { wsSend({ type: 'subscribe', chatId: activeChatId }); loadPendingFactSuggestions(); + loadPendingSkillSuggestions(); } /* If the socket dropped while we had pending task-create records (or * was never up when the server emitted 'ready'), reconcile against the @@ -2820,9 +4697,10 @@ }); } if (msg.chatId === activeChatId && msg.title != null) applyTitleForActive(msg.title); - // Sync header controls when another tab/CLI flipped these. - if (msg.chatId === activeChatId && msg.reasoningMode !== undefined && reasoningToggle) { - reasoningToggle.checked = msg.reasoningMode !== 'off'; + // Composer mode changed elsewhere (another tab/device) → mirror it here + // without re-persisting (avoids a broadcast loop). + if (msg.chatId === activeChatId && msg.mode !== undefined) { + setComposerMode(msg.mode, { persist: false }); } if (msg.chatId === activeChatId && msg.projectId !== undefined && messagesEl) { messagesEl.dataset.projectId = @@ -2837,7 +4715,7 @@ case 'chat-deleted': sidebarRemoveChat(msg.chatId); - if (msg.chatId === activeChatId) window.location.assign('/'); + if (msg.chatId === activeChatId) goTo('/'); return; case 'chat-unread': @@ -2885,6 +4763,7 @@ // rather than left orphaned above the final message. const target = ensureStreamEl(); if (target) { + cancelStreamRender(); // final render below is authoritative target.classList.remove( 'streaming', 'stream-waiting', 'stream-tool', 'stream-generating', ); @@ -2900,6 +4779,18 @@ body.innerHTML = renderMarkdown(msg.message.content || ''); decorateMessageBody(body); } + // Inline images the agent surfaced via show_image. The streaming + // bubble only built a text body, so append the attachments block here + // (the non-streaming appendMessage path already includes it). Reload + // renders them the same way, from msg.attachments. + if (Array.isArray(msg.message.attachments) && msg.message.attachments.length) { + target.querySelector('.msg-attachments-finalized')?.remove(); + const wrap = document.createElement('div'); + wrap.className = 'msg-attachments-finalized'; + wrap.innerHTML = attachmentsHtml(msg.message.attachments); + target.appendChild(wrap); + } + applyTokenBadge(target, msg.message.tokens, msg.message.cached_tokens); currentStreamEl = null; currentStreamFullText = ''; } else { @@ -2916,6 +4807,7 @@ setWorkingDot(msg.chatId, true); if (msg.chatId !== activeChatId) return; setStopVisible(true); + streamShownLen = 0; // fresh typewriter for the new turn ensureStreamEl(); { const status = currentStreamEl?.querySelector('.stream-status'); @@ -2945,12 +4837,9 @@ status.hidden = true; } } - const body = el.querySelector('.stream-body, .msg-body'); - if (body) { - body.innerHTML = renderMarkdown(currentStreamFullText); - decorateMessageBody(body, { deferSyntaxHighlight: true }); - } - scrollToBottom(); + // Feed the typewriter — it reveals the accumulated text smoothly, + // a few chars per frame, instead of dumping each lumpy delta at once. + ensureTyping(); return; } @@ -3008,13 +4897,13 @@ setWorkingDot(msg.chatId, false); if (msg.chatId !== activeChatId) return; setStopVisible(false); - finalizeReasoningBlock(); // Tear down a streaming element that nobody finalized — e.g. an // abort with no streamed text (skipPersist on the server → no // `message-appended` to clean it up), or any other edge where the // turn ends without an assistant message. Without this, the // "Finishing…" / "Thinking…" status would sit on the page until // the user reloaded. + cancelStreamRender(); if (currentStreamEl && currentStreamEl.classList.contains('streaming')) { const body = currentStreamEl.querySelector('.stream-body, .msg-body'); const hasContent = !!(body && body.textContent && body.textContent.trim()); @@ -3052,7 +4941,6 @@ return; } setStopVisible(false); - finalizeReasoningBlock(); if (currentStreamEl) { const st = currentStreamEl.querySelector('.stream-status'); stopStreamStatusDotAnim(st); @@ -3072,6 +4960,44 @@ return; } + /* ---- incognito (ephemeral; keyed, never persisted) ---- */ + case 'incognito-turn-delta': { + if (msg.key !== activeIncognitoKey) return; + const el = ensureStreamEl(); + currentStreamFullText += msg.text; + if (el.classList.contains('stream-waiting') || el.classList.contains('stream-tool')) { + el.classList.remove('stream-waiting', 'stream-tool'); + el.classList.add('stream-generating'); + const status = el.querySelector('.stream-status'); + if (status) { stopStreamStatusDotAnim(status); status.hidden = true; } + } + ensureTyping(); + return; + } + + case 'incognito-turn-tool': + // Tool activity indicator could go here; ignored for now. + return; + + case 'incognito-error': { + if (msg.key !== activeIncognitoKey) return; + const div = document.createElement('div'); + div.className = 'msg system error'; + div.innerHTML = + '<div class="role">error</div>' + + '<div class="msg-body">' + escapeHtml('Error: ' + msg.message) + '</div>'; + messagesAppendRoot()?.appendChild(div); + return; + } + + case 'incognito-turn-ended': { + if (msg.key !== activeIncognitoKey) return; + const finishedEl = currentStreamEl; + finalizeIncognitoStream(); + applyTokenBadge(finishedEl, msg.tokens, msg.cached); + return; + } + case 'project-fact-suggestions': { if (msg.chatId !== activeChatId) return; const have = existingFactSuggestionIds(); @@ -3090,6 +5016,68 @@ removeFactSuggestionRow(msg.chatId, msg.suggestionId); return; + case 'project-skill-suggestions': { + if (msg.chatId !== activeChatId) return; + const have = existingSkillSuggestionIds(); + const fresh = (msg.suggestions || []).filter((s) => s && !have.has(Number(s.id))); + if (fresh.length === 0) return; + appendSkillSuggestionsCard({ + projectId: msg.projectId, + chatId: msg.chatId, + projectName: typeof msg.projectName === 'string' ? msg.projectName : 'project', + suggestions: fresh, + }); + return; + } + + case 'project-skill-suggestion-removed': + removeSkillSuggestionRow(msg.chatId, msg.suggestionId); + return; + + case 'project-skill-added': { + if (currentProjectPageId() !== msg.projectId) return; + const ul = document.getElementById('skills-list'); + if (!ul) return; + ul.querySelector('li.project-chats-empty')?.remove(); + // Replace if a row with this id already exists (e.g. accept-as-update). + ul.querySelector('li.skill[data-skill-id="' + msg.skill.id + '"]')?.remove(); + ul.insertBefore(buildSkillLi(msg.skill), ul.firstChild); + syncProjectSkillsTabCountFromDom(); + return; + } + + case 'project-skill-updated': { + if (currentProjectPageId() !== msg.projectId) return; + const ul = document.getElementById('skills-list'); + if (!ul) return; + const existing = ul.querySelector('li.skill[data-skill-id="' + msg.skill.id + '"]'); + const fresh = buildSkillLi(msg.skill); + if (existing) existing.replaceWith(fresh); + else { + ul.querySelector('li.project-chats-empty')?.remove(); + ul.insertBefore(fresh, ul.firstChild); + } + syncProjectSkillsTabCountFromDom(); + return; + } + + case 'project-skill-deleted': { + if (currentProjectPageId() !== msg.projectId) return; + document + .querySelector('#skills-list li.skill[data-skill-id="' + msg.skillId + '"]') + ?.remove(); + const ul = document.getElementById('skills-list'); + if (ul && !ul.querySelector('li.skill')) { + const empty = document.createElement('li'); + empty.className = 'project-chats-empty muted'; + empty.textContent = + 'No skills yet. Accept a skill suggestion in a chat for this project.'; + ul.appendChild(empty); + } + syncProjectSkillsTabCountFromDom(); + return; + } + case 'project-created': { const arr = window.__ICLAW_PROJECTS__; if (Array.isArray(arr)) { @@ -3153,7 +5141,7 @@ }); window.__ICLAW_PROJECTS__ = (window.__ICLAW_PROJECTS__ || []).filter((p) => p && p.id !== pid); removeProjectsHubRow(pid); - if (currentProjectPageId() === pid) window.location.assign('/projects'); + if (currentProjectPageId() === pid) goTo('/projects'); return; } @@ -3288,12 +5276,6 @@ return; } - case 'turn-reasoning': { - if (msg.chatId !== activeChatId) return; - appendReasoningChunk(msg.text); - return; - } - case 'gateway-session-changed': // Informational — for now we don't auto-refetch the sidebar. Logging // this lets future iterations decide what to do without changing the @@ -3376,55 +5358,73 @@ const gatewayBannerStart = document.getElementById('sidebar-gateway-start'); let gatewayStatusPollTimer = null; - function setGatewayOfflineBannerVisible(visible) { - if (gatewayBanner) gatewayBanner.hidden = !visible; + function setGatewayOffline(offline) { + gatewayOffline = offline; + refreshGatewayOfflineBanner(); + } + + /** + * Show the "Start OpenClaw" banner only when the gateway is offline AND the + * user actually wants Full Power. Work / Safe work / Incognito run on the + * iclaw-runtime and never touch the gateway, so nudging them to start it is + * just noise — same `execute` gate the composer overlay uses. Queries the DOM + * directly (not the closure const) so it's safe to call from the early + * mode-change funnel before the gateway-banner const is initialised. + */ + function refreshGatewayOfflineBanner() { + const banner = document.getElementById('sidebar-gateway-banner'); + if (!banner) return; + banner.hidden = !(gatewayOffline && getComposerMode() === 'execute'); } function applyGatewayStatus(status, detail) { - const badge = document.getElementById('gateway-badge'); // "degraded" (gateway answered /health but the WS RPC can't get through) // gets its own in-page banner on the home/projects pages, so the sidebar // "Start OpenClaw" banner is reserved for a genuinely-offline gateway — // there's nothing to "start" when it's already running. const offline = status === 'down' || status === 'shutdown'; - setGatewayOfflineBannerVisible(offline); - - if (!badge) return; - badge.classList.remove('ok', 'down', 'degraded', 'shutdown'); - if (status === 'ok') { - badge.classList.add('ok'); - badge.textContent = 'OpenClaw: connected'; - } else if (status === 'degraded') { - badge.classList.add('degraded'); - badge.textContent = 'OpenClaw: unreachable'; - } else if (status === 'shutdown') { - badge.classList.add('shutdown'); - badge.textContent = 'OpenClaw: shutting down'; - } else { - badge.classList.add('down'); - badge.textContent = 'OpenClaw: off'; - } - const baseUrl = badge.dataset.baseUrl || ''; - badge.title = baseUrl; + setGatewayOffline(offline); + + // Keep Full Power gating current on every page. + gatewayOk = status === 'ok'; + if (typeof syncExecuteAvailability === 'function') syncExecuteAvailability(); } (function initGatewayOfflineBanner() { - const badge = document.getElementById('gateway-badge'); // Only a genuinely-offline gateway shows the sidebar "Start OpenClaw" banner. // "degraded" is handled by the in-page banner instead. - if (badge && badge.classList.contains('down')) { - setGatewayOfflineBannerVisible(true); - return; - } if (!gatewayBanner) return; void fetch('/api/gateway/status', { headers: { Accept: 'application/json' } }) .then((res) => (res.ok ? res.json() : null)) .then((data) => { - if (data && data.up !== true) setGatewayOfflineBannerVisible(true); + if (data && data.up !== true) setGatewayOffline(true); }) .catch(() => {}); })(); + // ── Connect-a-model chooser ─────────────────────────────────────────────── + // Shown when someone who skipped onboarding (no OpenRouter key, no reachable + // OpenClaw) actually tries to send — re-offers the onboarding choice instead + // of a dead end. The submit handler calls openConnectChooser(); this is a + // function declaration so it's callable regardless of definition order. + function openConnectChooser() { + const modal = document.getElementById('connect-modal'); + if (modal) modal.hidden = false; + } + (function initConnectChooser() { + const modal = document.getElementById('connect-modal'); + if (!modal) return; + const close = () => { modal.hidden = true; }; + modal.querySelectorAll('[data-connect-close]').forEach((el) => { + el.addEventListener('click', close); + }); + const pick = document.getElementById('connect-pick-openrouter'); + if (pick) pick.addEventListener('click', () => { window.location.href = '/welcome'; }); + document.addEventListener('keydown', (e) => { + if (e.key === 'Escape' && !modal.hidden) close(); + }); + })(); + function stopGatewayStatusPoll() { if (gatewayStatusPollTimer != null) { clearInterval(gatewayStatusPollTimer); @@ -3517,10 +5517,38 @@ if (item.serverId != null) ownQueueIds.delete(item.serverId); renderQueue(); inFlight = true; + + // Incognito: ephemeral turn — render locally, stream over `incognito-*` + // events, never persist. No pending-id (no message-appended adopts it) and + // no server chat is created. + if (item.mode === 'incognito') { + if (!activeIncognitoKey) activeIncognitoKey = newIncognitoKey(); + appendMessage({ role: 'user', content: item.content, mode: 'incognito' }); + currentStreamFullText = ''; + streamShownLen = 0; + currentStreamEl = ensureStreamEl(); + setStopVisible(true); + const wf = (typeof getWorkFolders === 'function' ? getWorkFolders() : []) + .map((f) => ({ path: f.path, readonly: true })); + const ok = wsSend({ + type: 'incognito-send', + key: activeIncognitoKey, + requestId: 'r-' + Date.now() + '-' + Math.random().toString(36).slice(2, 6), + content: item.content, + workFolders: wf.length ? wf : undefined, + }); + if (!ok) { + inFlight = false; + addWaitingItem(item, { at: 'front' }); + renderQueue(); + } + return; + } + // Optimistically append user msg. Mark it as pending-id so the // upcoming `message-appended` for the same user msg adopts this node // instead of duplicating. - const optimistic = { role: 'user', content: item.content }; + const optimistic = { role: 'user', content: item.content, mode: item.mode }; if (item.replyTo) { optimistic.reply_to_message_id = item.replyTo.messageId; optimistic.reply_quote = item.replyTo.quote; @@ -3539,12 +5567,32 @@ } appendMessage(optimistic, { pendingId: true }); currentStreamFullText = ''; + streamShownLen = 0; currentStreamEl = ensureStreamEl(); const payload = { type: 'send', requestId: 'r-' + Date.now() + '-' + Math.random().toString(36).slice(2, 6), content: item.content, }; + if (item.mode) payload.mode = item.mode; + if (item.mode === 'work') { + const wf = getWorkFolders ? getWorkFolders() : []; + if (wf.length > 0) { + payload.workFolders = wf.map((f) => ({ path: f.path, readonly: !f.write })); + } + } + if (item.mode === 'secure') { + payload.networkEnabled = typeof getNetworkEnabled === 'function' ? getNetworkEnabled() : false; + // Reset TTL countdown on each message + send TTL to runtime + if (typeof saveSecureTtl === 'function') { + const cur = getSecureTtl(); + payload.ttlDays = cur.ttlDays; + saveSecureTtl(cur.ttlDays); + // Session is created host-side during this turn; re-check a few times + // so the bar appears as soon as it exists (not before the first message). + [250, 800, 1600, 3000].forEach((d) => setTimeout(refreshSecureBar, d)); + } + } if (item.replyTo) { payload.replyTo = { messageId: item.replyTo.messageId, @@ -4140,6 +6188,14 @@ * selection-only changes are correct without waiting for debounce. */ function applyComposerSecretStripLayout() { + applyComposerSecretStripLayoutInner(); + // Secret strip and the secure-workspace bar share the bottom row; whenever + // the strip's visibility changes, re-evaluate the bar so it yields to the + // strip ("Selection in message") and reappears once the strip is gone. + if (typeof refreshSecureBar === 'function') refreshSecureBar(); + } + + function applyComposerSecretStripLayoutInner() { if (!composerSecretUi) return; if (composerSecretModal && !composerSecretModal.hidden) { cancelComposerSelectionHintReveal(); @@ -4617,6 +6673,19 @@ form.addEventListener('submit', async (e) => { e.preventDefault(); if (startedOnDraft && !draftProjectLocked) return; + // Don't send into a dead end. Full Power needs a reachable OpenClaw; the + // runtime modes (incl. a Work default with no OpenClaw on the device) need + // an OpenRouter key. When neither is usable, re-offer connecting a model. + const _sendMode = getComposerMode(); + const _modeRunnable = + _sendMode === 'execute' ? isExecuteAvailable() : openRouterReady; + if (!_modeRunnable) { + // Full Power with a key has usable runtime fallbacks → nudge to switch + // mode; otherwise there's no working backend → re-offer connecting. + if (_sendMode === 'execute' && openRouterReady) syncExecuteAvailability(); + else openConnectChooser(); + return; + } // If the schedule menu was just opened by a long-press, the bubbling // click on the send button would otherwise submit a regular message. if (scheduleMenuJustOpened || isScheduleMenuOpen()) { @@ -4651,6 +6720,7 @@ replyTo: replySnap || undefined, attachments: attachmentsSnap.length > 0 ? attachmentsSnap : undefined, inlineSecrets, + mode: getComposerMode(), }; let queued; const persistOnServer = @@ -5330,7 +7400,7 @@ openBtn.dataset.bound = '1'; openBtn.addEventListener('click', () => { removeTaskCreateBanner(pendingId); - window.location.href = '/tasks/' + encodeURIComponent(taskId); + goTo('/tasks/' + encodeURIComponent(taskId)); }); } } @@ -6128,6 +8198,14 @@ } if (stopBtn) { stopBtn.addEventListener('click', () => { + // Incognito has no DB chat — abort by ephemeral key and finalize locally. + if (getComposerMode() === 'incognito' && activeIncognitoKey) { + wsSend({ type: 'incognito-abort', key: activeIncognitoKey }); + finalizeIncognitoStream(); + stopBtn.disabled = true; + setTimeout(() => { stopBtn.disabled = false; }, 3000); + return; + } if (activeChatId == null) return; wsSend({ type: 'abort', chatId: activeChatId }); // Optimistically disable until server confirms via turn-error/ended, @@ -6137,37 +8215,6 @@ }); } - // ------------------------------------------------------------------------- - // Chat header extras: Reasoning toggle - // (Interrupt moved to per-queue-item buttons; Compact removed — OpenClaw - // auto-compacts at context limit, and users who want it can still type - // /compact in the composer.) - // ------------------------------------------------------------------------- - const reasoningToggle = document.getElementById('chat-reasoning-toggle'); - - if (reasoningToggle && activeChatId != null) { - reasoningToggle.addEventListener('change', async () => { - const mode = reasoningToggle.checked ? 'on' : 'off'; - try { - // Server route does two things atomically: persists the iClaw mirror - // and calls `sessions.patch({ reasoningLevel })` on the OpenClaw - // gateway. No more /reasoning slash kludge. - const res = await fetch( - '/chats/' + encodeURIComponent(activeChatId) + '/reasoning', - { - method: 'POST', - headers: { 'content-type': 'application/json' }, - body: JSON.stringify({ mode }), - }, - ); - if (!res.ok) throw new Error('HTTP ' + res.status); - } catch { - // revert if the server didn't accept - reasoningToggle.checked = !reasoningToggle.checked; - } - }); - } - // ------------------------------------------------------------------------- // Daily-reset policy banner — surfaces when OpenClaw's default "reset every // morning at 04:00" policy is active for `direct` (dashboard) sessions. @@ -6592,38 +8639,6 @@ }); } - // ------------------------------------------------------------------------- - // Reasoning text rendering - // ------------------------------------------------------------------------- - function appendReasoningChunk(text) { - if (!messagesEl || !text) return; - let block = messagesEl.querySelector('.reasoning-block.active'); - if (!block) { - block = document.createElement('div'); - block.className = 'msg assistant reasoning-block active'; - block.innerHTML = - '<div class="role">reasoning</div>' + - '<div class="msg-body reasoning-body"></div>'; - // Insert above any currently-streaming assistant element so the user - // sees thinking → answer, not answer → thinking. - const appendRoot = messagesAppendRoot(); - if (currentStreamEl && appendRoot && currentStreamEl.parentElement === appendRoot) { - appendRoot.insertBefore(block, currentStreamEl); - } else if (appendRoot) { - appendRoot.appendChild(block); - } - } - const body = block.querySelector('.reasoning-body'); - if (body) body.textContent += text; - scrollToBottom(); - } - function finalizeReasoningBlock() { - if (!messagesEl) return; - messagesEl.querySelectorAll('.reasoning-block.active').forEach((b) => { - b.classList.remove('active'); - }); - } - // ------------------------------------------------------------------------- // Slash autocomplete (`/` at composer start → commands.list) // ------------------------------------------------------------------------- @@ -6707,7 +8722,7 @@ const taskAskModal = document.getElementById('task-ask-modal'); if (taskAskModal && !taskAskModal.hidden) return; if (location.pathname === '/' || location.pathname === '') return; - window.location.assign('/'); + goTo('/'); }); function renderSlashMenu() { @@ -6845,6 +8860,70 @@ ); } + // ------------------------------------------------------------------------- + // project page — skills list (fetch + WS sync from other tabs) + // ------------------------------------------------------------------------- + const skillsListEl = document.getElementById('skills-list'); + if (skillsListEl && projectPageId != null) { + skillsListEl + .querySelectorAll('.skill-name, .skill-description, .skill-body') + .forEach((el) => { + el.dataset.saved = el.value.trim(); + }); + skillsListEl.addEventListener('click', (e) => { + const btn = e.target.closest('.skill-delete'); + if (!btn) return; + const li = btn.closest('li.skill'); + const skillId = li?.dataset.skillId; + if (!skillId) return; + e.preventDefault(); + fetch( + '/projects/' + + encodeURIComponent(projectPageId) + + '/skills/' + + encodeURIComponent(skillId) + + '/delete', + { method: 'POST', headers: { Accept: 'application/json' } }, + ).catch(() => {}); + }); + skillsListEl.addEventListener( + 'blur', + (e) => { + const el = e.target.closest('.skill-name, .skill-description, .skill-body'); + if (!el || !skillsListEl.contains(el)) return; + const li = el.closest('li.skill'); + const skillId = li?.dataset.skillId; + if (!skillId) return; + const next = el.value.trim(); + if (!next) return; + if (next === (el.dataset.saved || '').trim()) return; + const field = el.classList.contains('skill-name') + ? 'name' + : el.classList.contains('skill-description') + ? 'description' + : 'body'; + const payload = {}; + payload[field] = next; + fetch( + '/projects/' + + encodeURIComponent(projectPageId) + + '/skills/' + + encodeURIComponent(skillId), + { + method: 'PATCH', + headers: { 'content-type': 'application/json', Accept: 'application/json' }, + body: JSON.stringify(payload), + }, + ) + .then((res) => { + if (res.ok) el.dataset.saved = next; + }) + .catch(() => {}); + }, + true, + ); + } + function collapseProjectSecretRow(li) { li.classList.remove('project-secret-row--revealed'); const preview = li.querySelector('.project-secret-reveal'); @@ -6984,6 +9063,7 @@ const panels = { chats: document.getElementById('project-panel-chats'), memory: document.getElementById('project-panel-memory'), + skills: document.getElementById('project-panel-skills'), links: document.getElementById('project-panel-links'), files: document.getElementById('project-panel-files'), secrets: document.getElementById('project-panel-secrets'), @@ -6992,6 +9072,7 @@ !tabs.length || !panels.chats || !panels.memory || + !panels.skills || !panels.links || !panels.files || !panels.secrets @@ -7035,7 +9116,7 @@ const sel = document.getElementById('task-project-filter'); if (!sel) return; sel.addEventListener('change', () => { - window.location.href = '/tasks' + tasksBoardQueryFromFilterValue(sel.value); + goTo('/tasks' + tasksBoardQueryFromFilterValue(sel.value)); }); } @@ -7165,7 +9246,7 @@ at: Date.now(), }), ); - window.location.href = '/tasks'; + goTo('/tasks'); } function redirectToTasksAfterRetry(taskId, title) { @@ -7177,7 +9258,7 @@ at: Date.now(), }), ); - window.location.href = '/tasks'; + goTo('/tasks'); } function redirectToTasksAfterResumeSubmit(taskId, title, humanInput) { @@ -7190,7 +9271,7 @@ at: Date.now(), }), ); - window.location.href = '/tasks'; + goTo('/tasks'); } async function hydrateTaskApproveRunFlash() { @@ -7764,8 +9845,8 @@ div.className = 'msg assistant streaming stream-waiting'; div.innerHTML = '<div class="role">assistant</div>' + - '<div class="stream-status"></div>' + - '<div class="msg-body stream-body"></div>'; + '<div class="msg-body stream-body"></div>' + + '<div class="stream-status"></div>'; taskAskLive.thread.appendChild(div); const st = div.querySelector('.stream-status'); if (st) setStreamStatusLabel(st, 'Thinking…'); @@ -8312,7 +10393,7 @@ }); const data = await res.json().catch(() => ({})); if (!res.ok) throw new Error(data.error || res.statusText); - window.location.href = '/tasks'; + goTo('/tasks'); } catch (err) { alert(err instanceof Error ? err.message : String(err)); deleteBtn.disabled = false; @@ -8338,7 +10419,7 @@ if (failBtn) failBtn.disabled = true; try { await postAction('/complete', { status: 'done' }); - window.location.href = '/tasks'; + goTo('/tasks'); } catch (err) { alert(err instanceof Error ? err.message : String(err)); doneBtn.disabled = false; @@ -8352,7 +10433,7 @@ if (doneBtn) doneBtn.disabled = true; try { await postAction('/complete', { status: 'failed' }); - window.location.href = '/tasks'; + goTo('/tasks'); } catch (err) { alert(err instanceof Error ? err.message : String(err)); failBtn.disabled = false; diff --git a/public/js/ra-e2e-transport.mjs b/public/js/ra-e2e-transport.mjs index 4df76da..796ffd2 100644 --- a/public/js/ra-e2e-transport.mjs +++ b/public/js/ra-e2e-transport.mjs @@ -383,25 +383,53 @@ let installed = false; /** Load workspace HTML via encrypted /__ra/e2e/http (after OPAQUE login). */ export async function navigateViaE2eDocument(nextUrl) { - const installed = await installRaE2eTransport(); - if (!installed) { + const ready = await installRaE2eTransport(); + if (!ready) { throw new Error('Encrypted session not ready. Sign in with your passphrase again.'); } - const path = - typeof nextUrl === 'string' && nextUrl.startsWith('/') - ? nextUrl - : typeof nextUrl === 'string' && nextUrl.startsWith('http') - ? new URL(nextUrl).pathname + new URL(nextUrl).search - : '/'; - const res = await window.fetch(path, { - method: 'GET', - credentials: 'same-origin', - headers: { Accept: 'text/html,application/xhtml+xml' }, - }); - if (!res.ok) { - throw new Error('Could not load workspace (HTTP ' + res.status + ')'); + const toPath = function (u) { + try { + const url = new URL(u, location.origin); + if (url.origin !== location.origin) return null; + return url.pathname + url.search; + } catch { + return null; + } + }; + let path = toPath(nextUrl) || '/'; + let html = null; + // Follow same-origin redirects over the encrypted channel. The relay/server + // returns 3xx as-is (it doesn't follow them), so without this a redirecting + // route — e.g. the 302 after a POST, or a GET that redirects — would fall + // through to a full navigation and bounce off the gate again. + for (let hop = 0; hop < 5; hop++) { + const res = await window.fetch(path, { + method: 'GET', + credentials: 'same-origin', + headers: { Accept: 'text/html,application/xhtml+xml' }, + redirect: 'manual', + }); + if (res.status >= 300 && res.status < 400) { + const next = toPath(res.headers.get('location') || ''); + if (!next) throw new Error('Could not follow redirect (HTTP ' + res.status + ')'); + path = next; + continue; + } + if (!res.ok) { + throw new Error('Could not load workspace (HTTP ' + res.status + ')'); + } + html = await res.text(); + break; + } + if (html == null) { + throw new Error('Too many redirects loading workspace'); + } + // Reflect the final URL (after any redirects) in the address bar. + try { + history.replaceState({ iclawE2eSpa: true }, '', path); + } catch { + // ignore } - const html = await res.text(); document.open(); document.write(html); document.close(); @@ -457,3 +485,287 @@ export async function installRaE2eTransport() { installed = true; return true; } + +/* ---------------------------------------------------------- SPA routing -- */ +// Through a tunnel, every *full-page* navigation is answered with the +// passphrase gate ("Checking this device…") — the browser's own navigation +// request can't be E2E-wrapped, only fetch()/WebSocket can. So clicking from +// page to page flashes the gate each time while it re-establishes the channel +// and pulls the page over /__ra/e2e/http. +// +// These helpers keep navigation inside the already-installed encrypted +// transport: intercept in-app link clicks (and expose e2eNavigate() for the +// few programmatic navigations in iclaw.js), fetch the next page over the +// existing channel, and replace the document in place. iclaw.js still boots +// exactly once per page (document.write re-runs it), so there's no +// double-binding of its document/WebSocket listeners — same single-load model, +// just without the gate round trip. + +let spaNavInFlight = false; +let spaNavTimer = null; + +function isE2eEnabled() { + const meta = document.querySelector('meta[name="iclaw-ra-e2e"]'); + return !!(meta && meta.getAttribute('content') === 'true'); +} + +function beginSpaNav() { + spaNavInFlight = true; + // Self-healing latch: if the destination never re-initialises (e.g. its boot + // import fails after document.write), don't block navigation forever. Timers + // survive document.open(), so this clears even on the failure path. + try { + if (spaNavTimer) clearTimeout(spaNavTimer); + } catch { + // ignore + } + spaNavTimer = setTimeout(function () { + spaNavInFlight = false; + }, 20000); +} + +/** + * Navigate to a same-origin in-app URL over the encrypted transport, without + * bouncing through the passphrase gate. Falls back to a normal navigation when + * E2E isn't active or the encrypted load fails. Also exposed on + * window.iclawE2eNavigate so iclaw.js can route its programmatic navigations. + */ +export async function e2eNavigate(nextUrl, opts) { + if (!isE2eEnabled()) { + window.location.assign(nextUrl); + return; + } + let path; + try { + const u = new URL(nextUrl, location.origin); + if (u.origin !== location.origin) { + window.location.assign(nextUrl); + return; + } + path = u.pathname + u.search + u.hash; + } catch { + window.location.assign(nextUrl); + return; + } + if (spaNavInFlight) return; + beginSpaNav(); + try { + if (opts && opts.replace) history.replaceState({ iclawE2eSpa: true }, '', path); + else history.pushState({ iclawE2eSpa: true }, '', path); + } catch { + // ignore — address bar stays put, the content still swaps below + } + // Retry once before surrendering to a full navigation: a transient transport + // hiccup (a dropped relay frame, a decrypt miss) shouldn't bounce the user + // through the gate when a second encrypted attempt would just work. + for (let attempt = 0; attempt < 2; attempt++) { + try { + await navigateViaE2eDocument(path); + return; // success — the document is being replaced + } catch { + if (attempt === 0) { + await new Promise(function (r) { + setTimeout(r, 150); + }); + continue; + } + spaNavInFlight = false; + // Encrypted load failed twice — fall back to a real navigation (gate resume). + window.location.assign(path); + return; + } + } +} + +function spaShouldIntercept(a) { + if (!a) return false; + const target = a.getAttribute('target'); + if (target && target !== '_self') return false; // _blank etc. → let it open + if (a.hasAttribute('download')) return false; + if (a.dataset && typeof a.dataset.noSpa !== 'undefined') return false; // opt-out + const href = a.getAttribute('href'); + if (!href || href.charAt(0) === '#') return false; + let u; + try { + u = new URL(href, location.href); + } catch { + return false; + } + if (u.origin !== location.origin) return false; // external link + const p = u.pathname; + if ( + p.startsWith('/__ra/') || + p.startsWith('/js/') || + p.startsWith('/css/') || + p.startsWith('/uploads/') || + p.startsWith('/favicon') + ) { + return false; // non-page / asset / gate endpoints + } + // Same path, only a fragment differs → let the browser scroll natively. + if (u.pathname === location.pathname && u.search === location.search && u.hash) { + return false; + } + return true; +} + +function onSpaClick(e) { + if (e.defaultPrevented) return; // iclaw.js already handled this click + if (e.button !== 0 || e.metaKey || e.ctrlKey || e.shiftKey || e.altKey) return; + const start = e.target; + const a = start && start.closest ? start.closest('a[href]') : null; + if (!a || !spaShouldIntercept(a)) return; + const u = new URL(a.getAttribute('href'), location.href); + e.preventDefault(); + e2eNavigate(u.pathname + u.search + u.hash); +} + +function onSpaPopState() { + if (!isE2eEnabled()) return; + if (spaNavInFlight) return; + beginSpaNav(); + // The browser already moved the address bar; just render the current URL. + navigateViaE2eDocument(location.pathname + location.search).catch(function () { + spaNavInFlight = false; + window.location.reload(); + }); +} + +function spaToPath(u) { + try { + const url = new URL(u, location.href); + if (url.origin !== location.origin) return null; + return url.pathname + url.search; + } catch { + return null; + } +} + +async function submitFormOverE2e(form, method, actionPath) { + if (spaNavInFlight) return; + beginSpaNav(); + try { + const body = new URLSearchParams(new FormData(form)).toString(); + let res; + if (method === 'get') { + const base = actionPath.split('#')[0].split('?')[0]; + res = await window.fetch(base + (body ? '?' + body : ''), { + method: 'GET', + credentials: 'same-origin', + headers: { Accept: 'text/html,application/xhtml+xml' }, + redirect: 'manual', + }); + } else { + res = await window.fetch(actionPath, { + method: method.toUpperCase(), + credentials: 'same-origin', + headers: { + 'Content-Type': 'application/x-www-form-urlencoded;charset=UTF-8', + Accept: 'text/html,application/xhtml+xml', + }, + body, + redirect: 'manual', + }); + } + if (res.status >= 300 && res.status < 400) { + // Post/Redirect/Get: load the redirect target over the encrypted channel. + const target = spaToPath(res.headers.get('location') || '/'); + spaNavInFlight = false; + if (target == null) { + window.location.assign(res.headers.get('location') || '/'); + return; + } + return e2eNavigate(target); + } + if (res.ok) { + const html = await res.text(); + try { + history.pushState({ iclawE2eSpa: true }, '', actionPath); + } catch { + // ignore + } + document.open(); + document.write(html); + document.close(); + return; + } + // Non-OK with a body (e.g. a re-rendered form with validation errors) — + // show it; otherwise fall back to a real submit. + const errHtml = await res.text().catch(() => ''); + if (errHtml) { + document.open(); + document.write(errHtml); + document.close(); + return; + } + spaNavInFlight = false; + form.submit(); + } catch (err) { + spaNavInFlight = false; + try { + form.submit(); + } catch { + // ignore + } + } +} + +function onSpaSubmit(e) { + if (e.defaultPrevented) return; // JS-handled form (onsubmit="return false", etc.) + const form = e.target; + if (!form || form.tagName !== 'FORM') return; + if (form.dataset && typeof form.dataset.noSpa !== 'undefined') return; // opt-out + // Only forms that point at a concrete endpoint. Forms without an explicit + // action (the composer, search, inline-edit forms) are app-JS-driven — never + // ours to take over. + const actionAttr = form.getAttribute('action'); + if (!actionAttr) return; + const method = (form.getAttribute('method') || 'get').toLowerCase(); + if (method !== 'get' && method !== 'post') return; // skip method="dialog", etc. + if ((form.getAttribute('enctype') || '').toLowerCase().indexOf('multipart') !== -1) { + return; // file uploads — let the browser handle them + } + const actionPath = spaToPath(actionAttr); + if (actionPath == null) return; // external / unparseable action + const p = actionPath.split('?')[0]; + if ( + p.startsWith('/__ra/') || + p.startsWith('/js/') || + p.startsWith('/css/') || + p.startsWith('/uploads/') + ) { + return; + } + e.preventDefault(); + submitFormOverE2e(form, method, actionPath); +} + +/** + * Install in-app navigation interception. Called from every workspace page's + * boot (head.ejs), which re-runs after each document.write swap. + * + * Listeners are bound on window and rebound *idempotently* (remove-then-add) on + * every page. In theory window listeners survive document.open() (same realm), + * so binding once would do — but if a swap ever drops them, an unbound page + * would silently let chat-link clicks full-navigate and flash the gate. Removing + * first guarantees exactly one handler whether or not they persisted. + */ +export function setupE2eSpaNavigation() { + if (!isE2eEnabled()) return; + // Fresh page after a swap — release the latch and its safety timer. + spaNavInFlight = false; + try { + if (spaNavTimer) clearTimeout(spaNavTimer); + } catch { + // ignore + } + spaNavTimer = null; + // Bubble phase: anything iclaw.js handled (and stopped/prevented) never + // reaches window, so we only take over genuinely unclaimed link/form events. + window.removeEventListener('click', onSpaClick); + window.removeEventListener('submit', onSpaSubmit); + window.removeEventListener('popstate', onSpaPopState); + window.addEventListener('click', onSpaClick); + window.addEventListener('submit', onSpaSubmit); + window.addEventListener('popstate', onSpaPopState); +} diff --git a/scripts/install-docker.sh b/scripts/install-docker.sh new file mode 100755 index 0000000..edab297 --- /dev/null +++ b/scripts/install-docker.sh @@ -0,0 +1,130 @@ +#!/usr/bin/env bash +# install-docker.sh — best-effort container-engine install for iClaw Work/Safe modes. +# +# Triggered by the composer's "Install" button via POST /api/docker/install +# (localhost only). Idempotent: a no-op when the engine is already present. The +# caller starts the engine and polls readiness afterwards, so this script only +# has to land the binaries. +# +# macOS — the engine is Colima (a lightweight Linux VM, NOT Docker Desktop, so no +# GUI, no licence, nothing for a non-technical user to configure). Two ways to +# land it, in order of preference: +# 1. Homebrew present → `brew install colima docker` (brew keeps it updated). +# 2. No Homebrew → download PINNED colima + lima + docker binaries straight +# from the projects' official releases into ~/.iclaw/engine, +# verifying SHA-256. No brew, no Xcode CLT, no sudo — so it +# works on a clean Mac with nothing installed. +# Linux — official get.docker.com convenience script + docker group. +set -euo pipefail + +# ---- pinned engine versions + SHA-256 (verified against the official releases) ---- +# colima/lima: GitHub release asset digests · docker: download.docker.com static +# (no published checksum — hashes computed at pin time). To bump: change the +# version and the matching per-arch SHA-256 below. +COLIMA_VER="0.10.3" +LIMA_VER="2.1.2" +DOCKER_VER="29.5.3" +# colima-Darwin-<arch> +COLIMA_SHA_arm64="980ad8bf61a4ca370243f4cb41401a61276dcd2c2502bee7b9b86f9250169f34" +COLIMA_SHA_x86_64="3082737fe8a98afda11cba7d9a20b6e56fe80c6153464beda04bec630758770b" +# lima-<ver>-Darwin-<arch>.tar.gz (bundles limactl + share/lima incl. the native guest agent) +LIMA_SHA_arm64="7081d03d01511f20c4a3b38d8120428ef1c66e4b21ec9b54017bc65da60b031f" +LIMA_SHA_x86_64="3dc5218c7b0cc14126fb6e3ae6f174f026660e4e2cdffcb34b16e5a2f415eb45" +# docker-<ver>.tgz (download.docker.com/mac/static/stable/<arch>) +DOCKER_SHA_aarch64="a579c5fb15bebb35dc443cdf6f17b076b6c90afa6cd0e51463b1608e5b235536" +DOCKER_SHA_x86_64="db73fa6cdeb6a5a3b646fe18dec4cdb48ade3b5cea6bc069afcc389b5d1cb819" + +ENGINE_DIR="${ICLAW_ENGINE_DIR:-$HOME/.iclaw/engine}" + +# fetch_verify URL DEST EXPECTED_SHA256 — download over HTTPS, fail hard on mismatch. +fetch_verify() { + local url="$1" dest="$2" sha="$3" + echo " ↓ $(basename "$dest")" + curl -fSL --retry 3 "$url" -o "$dest" + if ! echo "${sha} ${dest}" | shasum -a 256 -c - >/dev/null 2>&1; then + echo "ERROR: SHA-256 mismatch for $url — refusing to install." >&2 + rm -f "$dest" + exit 1 + fi +} + +# Download + verify pinned colima/lima/docker into ENGINE_DIR. No Homebrew needed. +install_engine_no_brew() { + local m carch larch darch csha lsha dsha + m="$(uname -m)" + case "$m" in + arm64) carch=arm64; larch=arm64; darch=aarch64 + csha=$COLIMA_SHA_arm64; lsha=$LIMA_SHA_arm64; dsha=$DOCKER_SHA_aarch64 ;; + x86_64) carch=x86_64; larch=x86_64; darch=x86_64 + csha=$COLIMA_SHA_x86_64; lsha=$LIMA_SHA_x86_64; dsha=$DOCKER_SHA_x86_64 ;; + *) echo "ERROR: unsupported macOS architecture '$m'." >&2; exit 1 ;; + esac + + local bin tmp + bin="$ENGINE_DIR/bin" + mkdir -p "$bin" + tmp="$(mktemp -d)" + trap 'rm -rf "$tmp"' RETURN + + echo "Homebrew not found — installing the Colima engine directly into $ENGINE_DIR" + echo "(downloading pinned binaries; no brew, no Xcode tools, no password)…" + + # colima — single self-contained binary + fetch_verify "https://github.com/abiosoft/colima/releases/download/v${COLIMA_VER}/colima-Darwin-${carch}" \ + "$bin/colima" "$csha" + chmod +x "$bin/colima" + + # lima — tarball lays down bin/limactl + share/lima (limactl finds its share via ../share) + fetch_verify "https://github.com/lima-vm/lima/releases/download/v${LIMA_VER}/lima-${LIMA_VER}-Darwin-${larch}.tar.gz" \ + "$tmp/lima.tgz" "$lsha" + tar -xzf "$tmp/lima.tgz" -C "$ENGINE_DIR" + + # docker — static CLI client (tgz contains docker/docker) + fetch_verify "https://download.docker.com/mac/static/stable/${darch}/docker-${DOCKER_VER}.tgz" \ + "$tmp/docker.tgz" "$dsha" + tar -xzf "$tmp/docker.tgz" -C "$tmp" + install -m 0755 "$tmp/docker/docker" "$bin/docker" + + if "$bin/colima" version >/dev/null 2>&1; then + echo "Colima engine ready: $("$bin/colima" version 2>/dev/null | head -1)" + else + echo "ERROR: colima failed to run after install." >&2 + exit 1 + fi +} + +case "$(uname -s)" in + Darwin) + # On macOS the engine is Colima. A bare `docker` CLI may already exist (e.g. + # left over from Docker Desktop), so key the "already installed" check on + # colima specifically — on PATH or in our own engine dir. + if command -v colima >/dev/null 2>&1 || [ -x "$ENGINE_DIR/bin/colima" ]; then + echo "colima already installed." + exit 0 + fi + if command -v brew >/dev/null 2>&1; then + echo "Installing Colima + Docker CLI via Homebrew (no Docker Desktop)…" + brew install colima docker + else + install_engine_no_brew + fi + ;; + Linux) + if command -v docker >/dev/null 2>&1; then + echo "docker already installed: $(docker --version 2>/dev/null || echo unknown)" + exit 0 + fi + echo "Installing Docker Engine via get.docker.com…" + curl -fsSL https://get.docker.com | sh + if command -v sudo >/dev/null 2>&1; then + sudo usermod -aG docker "$USER" || true + echo "NOTE: log out and back in for docker group membership to take effect." + fi + ;; + *) + echo "ERROR: unsupported platform $(uname -s). Install Docker manually from https://docker.com" >&2 + exit 1 + ;; +esac + +echo "container engine install step done." diff --git a/src/app.ts b/src/app.ts index 493f8aa..29b82f6 100644 --- a/src/app.ts +++ b/src/app.ts @@ -8,6 +8,7 @@ import { projectsRouter } from './routes/projects'; import { agentsRouter } from './routes/sessions'; import { mediaRouter } from './routes/media'; import { gatewayRouter } from './routes/gateway'; +import { dockerRouter } from './routes/docker'; import { updateRouter } from './routes/update'; import { tasksRouter } from './routes/tasks'; import { projects, scheduledMessages } from './services/store'; @@ -71,6 +72,7 @@ export function createApp(): express.Express { })); res.locals.scheduledChatIds = scheduledMessages.chatIdsWithPending(); res.locals.scheduledChatCounts = scheduledMessages.pendingCountByChatId(); + res.locals.devMode = process.env.ICLAW_DEV_MODE === 'true'; next(); }); @@ -125,6 +127,7 @@ export function createApp(): express.Express { app.use('/', settingsRouter); app.use('/api/agents', agentsRouter); app.use('/api/gateway', gatewayRouter); + app.use('/api/docker', dockerRouter); app.use('/api/update', updateRouter); app.use('/api/remote-access', remoteAccessApiRouter); app.use('/media', mediaRouter); diff --git a/src/db/database.ts b/src/db/database.ts index 6b0401d..edf58d8 100644 --- a/src/db/database.ts +++ b/src/db/database.ts @@ -23,8 +23,6 @@ CREATE TABLE IF NOT EXISTS chats ( shares_to_project INTEGER NOT NULL DEFAULT 1, -- Optional per-session model override applied via sessions.patch. model_override TEXT, - -- Reasoning visibility mirror; actual state lives on the gateway. - reasoning_mode TEXT NOT NULL DEFAULT 'off', created_at TEXT NOT NULL DEFAULT (datetime('now')), updated_at TEXT NOT NULL DEFAULT (datetime('now')), title_manual INTEGER NOT NULL DEFAULT 0, @@ -44,6 +42,8 @@ CREATE TABLE IF NOT EXISTS messages ( reply_to_role TEXT, /** JSON array of {url, mimeType, fileName, sizeBytes} for user-attached files. NULL when no attachments. */ attachments TEXT, + /** Send mode: 'ask' | 'execute' (see services/chatModes.ts). Legacy rows default to 'execute'. */ + mode TEXT NOT NULL DEFAULT 'execute', created_at TEXT NOT NULL DEFAULT (datetime('now')) ); @@ -74,6 +74,45 @@ CREATE TABLE IF NOT EXISTS project_fact_suggestions ( CREATE INDEX IF NOT EXISTS idx_fact_suggestions_chat ON project_fact_suggestions(chat_id, id); +-- Active, accepted project skills (procedural memory). Stored as SKILL.md. +-- The declarative half is project_facts; this is the procedural half. +CREATE TABLE IF NOT EXISTS project_skills ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + project_id INTEGER REFERENCES projects(id) ON DELETE CASCADE, -- NULL = global skill + name TEXT NOT NULL, -- kebab-case, unique within scope + description TEXT NOT NULL, -- one-line summary (shown to user + prompt index) + body TEXT NOT NULL, -- full SKILL.md (frontmatter + procedure) + tags TEXT, -- JSON array of strings (optional) + source_chat_id INTEGER REFERENCES chats(id) ON DELETE SET NULL, + usage_count INTEGER NOT NULL DEFAULT 0, + version INTEGER NOT NULL DEFAULT 1, + created_at TEXT NOT NULL DEFAULT (datetime('now')), + updated_at TEXT NOT NULL DEFAULT (datetime('now')) +); +CREATE INDEX IF NOT EXISTS idx_skills_project ON project_skills(project_id, id); +-- Uniqueness within a scope (project_id may be NULL for global). SQLite treats +-- NULLs as distinct, so global uniqueness is also enforced in the store layer. +CREATE UNIQUE INDEX IF NOT EXISTS idx_skills_scope_name + ON project_skills(project_id, name); + +-- Inbox: proposed skills awaiting user acceptance. Never active until accepted. +CREATE TABLE IF NOT EXISTS project_skill_suggestions ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + project_id INTEGER NOT NULL REFERENCES projects(id) ON DELETE CASCADE, + chat_id INTEGER NOT NULL REFERENCES chats(id) ON DELETE CASCADE, + kind TEXT NOT NULL DEFAULT 'new', -- 'new' | 'patch' + target_skill_id INTEGER REFERENCES project_skills(id) ON DELETE CASCADE, -- for 'patch' + name TEXT NOT NULL, + description TEXT NOT NULL, + body TEXT NOT NULL, -- full proposed SKILL.md + tags TEXT, -- JSON array (optional) + untrusted INTEGER NOT NULL DEFAULT 0, -- 1 if source turn ingested untrusted content + assistant_message_id INTEGER REFERENCES messages(id) ON DELETE SET NULL, + created_at TEXT NOT NULL DEFAULT (datetime('now')) +); +CREATE INDEX IF NOT EXISTS idx_skill_suggestions_chat ON project_skill_suggestions(chat_id, id); +CREATE INDEX IF NOT EXISTS idx_skill_suggestions_project ON project_skill_suggestions(project_id, id); + CREATE TABLE IF NOT EXISTS project_secrets ( id INTEGER PRIMARY KEY AUTOINCREMENT, project_id INTEGER REFERENCES projects(id) ON DELETE CASCADE, @@ -111,6 +150,8 @@ CREATE TABLE IF NOT EXISTS queued_messages ( attachments TEXT, /** JSON array of {slot, label, plain} for [[iclaw:sN]] markers; resolved on flush. */ inline_secrets TEXT, + /** Send mode chosen at enqueue time; preserved so flush sends with it. */ + mode TEXT NOT NULL DEFAULT 'execute', /** Lower sorts first; promote-to-front uses values below the current min. */ position INTEGER NOT NULL DEFAULT 0, created_at TEXT NOT NULL DEFAULT (datetime('now')) @@ -232,6 +273,17 @@ CREATE TABLE IF NOT EXISTS remote_access_state ( updated_at INTEGER NOT NULL DEFAULT (CAST(strftime('%s','now') AS INTEGER) * 1000) ); +-- Context compaction cache. One row per chat: a rolling summary of all turns +-- up to (and including) up_to_message_id. Lets us feed the model +-- [summary] + [recent verbatim] so a chat's context is never cleared, only +-- compressed. Rebuilt incrementally as the chat grows. +CREATE TABLE IF NOT EXISTS chat_summaries ( + chat_id INTEGER PRIMARY KEY REFERENCES chats(id) ON DELETE CASCADE, + up_to_message_id INTEGER NOT NULL, + summary TEXT NOT NULL, + updated_at TEXT NOT NULL DEFAULT (datetime('now')) +); + -- Robustness: any new message bumps the parent chat's updated_at so sidebar -- sorting is always correct even if a caller forgets the manual chats.touch(). CREATE TRIGGER IF NOT EXISTS trg_chats_touch_on_message @@ -257,7 +309,15 @@ function ensureColumn(table: string, column: string, ddl: string): void { } } ensureColumn('messages', 'attachments', 'TEXT'); +ensureColumn('messages', 'mode', "TEXT NOT NULL DEFAULT 'execute'"); +ensureColumn('messages', 'tokens', 'INTEGER'); // dev-mode token usage (runtime modes) +ensureColumn('messages', 'cached_tokens', 'INTEGER'); // dev-mode: prompt tokens served from cache +ensureColumn('queued_messages', 'mode', "TEXT NOT NULL DEFAULT 'execute'"); ensureColumn('chats', 'chat_kind', "TEXT NOT NULL DEFAULT 'normal'"); +// Sticky composer send-mode per chat (null = use UI default). Persisted on change +// so it survives page navigation and syncs across devices — previously only the +// last sent message's mode + per-browser localStorage tracked this. +ensureColumn('chats', 'mode', 'TEXT'); ensureColumn('remote_access_tunnels', 'access_token', 'TEXT'); ensureColumn('remote_access_tunnels', 'opaque_registration_record', 'TEXT'); // Tunnel ownership secret. Proves to the relay that a re-registering client is diff --git a/src/db/kv.ts b/src/db/kv.ts new file mode 100644 index 0000000..7f061eb --- /dev/null +++ b/src/db/kv.ts @@ -0,0 +1,31 @@ +/** + * Tiny typed accessor over the `iclaw_kv` table — a plain string key/value + * store for app-level settings that don't deserve their own table (e.g. the + * OpenRouter API key entered in Settings). + * + * Deliberately low-level: imports ONLY `db`, so modules like `config.ts` can + * read settings without pulling in the higher-level `store.ts` (which would + * create an import cycle: config → store → chatModes → openRouter → config). + */ + +import { db } from './database'; + +const getStmt = db.prepare('SELECT value FROM iclaw_kv WHERE key = ?'); +const setStmt = db.prepare( + `INSERT INTO iclaw_kv (key, value) VALUES (?, ?) + ON CONFLICT(key) DO UPDATE SET value = excluded.value`, +); +const delStmt = db.prepare('DELETE FROM iclaw_kv WHERE key = ?'); + +export function kvGet(key: string): string | null { + const row = getStmt.get(key) as { value: string } | undefined; + return row?.value ?? null; +} + +export function kvSet(key: string, value: string): void { + setStmt.run(key, value); +} + +export function kvDelete(key: string): void { + delStmt.run(key); +} diff --git a/src/index.ts b/src/index.ts index 2482111..0697140 100644 --- a/src/index.ts +++ b/src/index.ts @@ -6,6 +6,8 @@ import { openclaw } from './services/openclaw'; import { scheduler } from './services/scheduler'; import { gatewayEvents } from './services/gatewayEvents'; import { remoteAccess, setRemoteAccessQuiet } from './services/remoteAccess'; +import { runtimeProcess } from './services/runtimeProcess'; +import { probeOpenClawInstalled } from './services/openclawInstall'; import { setBoundLocalAddress } from './services/localAddress'; import { findAvailablePort, @@ -26,6 +28,12 @@ import { writeLockFile, } from './startup'; import { offerRemoteAccessOnboarding } from './remoteAccessOnboarding'; +import { ensureColimaEnv } from './services/colima'; + +// macOS: set up the Colima engine env (PATH to colima/docker + docker context) +// before we spawn the runtime sidecar, so the child — and its container +// commands — inherit it. +ensureColimaEnv(); const preferredPort = Number(process.env.PORT ?? 3000); const host = '127.0.0.1'; @@ -44,6 +52,7 @@ function gracefulShutdown( } removeLockFileIfOwned(); scheduler.stop(); + runtimeProcess.stop(); remoteAccess.shutdown(); const exitCode = signal === 'SIGINT' ? 130 : 0; @@ -108,7 +117,12 @@ async function main(): Promise<void> { server = createServer(app); attachWsServer(server); scheduler.start(); + runtimeProcess.start(); gatewayEvents.start(); + // Prime the OpenClaw install check before serving — the composer's default + // mode (Full Power vs Work) keys off it. Fast: present → quick; absent → + // ENOENT immediately. Never fatal. + await probeOpenClawInstalled().catch(() => {}); const stop = () => gracefulShutdown(server, 'SIGINT'); process.on('SIGINT', () => stop()); diff --git a/src/routes/chats.ts b/src/routes/chats.ts index 07c7c39..e2d1460 100644 --- a/src/routes/chats.ts +++ b/src/routes/chats.ts @@ -6,12 +6,15 @@ import { projects, projectFactSuggestions, projectFacts, + projectSkills, + projectSkillSuggestions, projectSecrets, secretUsableInChat, scheduledMessages, queuedMessages, tasks, enrichFactWithSourceChatTitle, + enrichSkillWithSourceChatTitle, } from '../services/store'; import { persistIncomingAttachments, type IncomingAttachment } from '../services/uploads'; import { @@ -24,8 +27,18 @@ import { openclawWs } from '../services/openclawWs'; import { openclaw, cloudShareBaseUrl } from '../services/openclaw'; import { chatStatus } from '../services/chatStatus'; import { wsHub } from '../services/wsHub'; -import { sendMessage } from '../services/chatRunner'; +import { sendMessage, getWorkSessionId, destroyWorkSession, exportChatSandbox, applyChatSandboxChanges } from '../services/chatRunner'; +import { getWorkspaceInfo } from '../services/workRuntime'; +import { + defaultComposerMode, + isSelectableMode, + isEphemeralMode, + listComposerModes, + normalizeChatMode, +} from '../services/chatModes'; +import type { ChatMode } from '../types'; import { shouldShowSendHint } from '../services/sendHint'; +import { openRouterEnabled } from '../services/openRouter'; export const chatsRouter: Router = Router(); @@ -152,23 +165,169 @@ chatsRouter.post('/:id/fact-suggestions/:suggestionId/reject', (req, res) => { res.type('application/json').json({ ok: true }); }); +/* ---------------- project skill suggestions (inbox-gated) ---------------- */ + +function parseTags(raw: unknown): string[] | null { + if (!Array.isArray(raw)) return null; + const tags = raw.filter((t): t is string => typeof t === 'string' && t.trim().length > 0); + return tags.length > 0 ? tags : null; +} + +/** Pending project-skill suggestions for this chat (JSON, includes full bodies). */ +chatsRouter.get('/:id/skill-suggestions', (req, res) => { + const id = Number(req.params.id); + if (!Number.isFinite(id) || !chats.get(id)) { + res.status(404).json({ error: 'chat not found' }); + return; + } + const suggestions = projectSkillSuggestions.listByChat(id); + const first = suggestions[0]; + const projectName = + first != null ? (projects.get(first.project_id)?.name?.trim() ?? 'project') : null; + res.type('application/json').json({ suggestions, projectName }); +}); + +chatsRouter.post('/:id/skill-suggestions/:suggestionId/accept', (req, res) => { + const chatId = Number(req.params.id); + const sid = Number(req.params.suggestionId); + if (!Number.isFinite(chatId) || !Number.isFinite(sid)) { + res.status(400).json({ error: 'invalid id' }); + return; + } + const chat = chats.get(chatId); + const sug = projectSkillSuggestions.get(sid); + if (!chat || !sug || sug.chat_id !== chatId) { + res.status(404).json({ error: 'not found' }); + return; + } + if (!chat.project_id || chat.project_id !== sug.project_id) { + res.status(400).json({ error: 'project mismatch' }); + return; + } + + // Optional user edits submitted alongside the accept. + const name = typeof req.body?.name === 'string' && req.body.name.trim() ? req.body.name : sug.name; + const description = + typeof req.body?.description === 'string' && req.body.description.trim() + ? req.body.description + : sug.description; + const body = typeof req.body?.body === 'string' && req.body.body.trim() ? req.body.body : sug.body; + const tags = parseTags(req.body?.tags) ?? (sug.tags ? (JSON.parse(sug.tags) as string[]) : null); + const scope = req.body?.scope === 'global' ? 'global' : 'project'; + const projectId = scope === 'global' ? null : sug.project_id; + + try { + if (sug.kind === 'patch' && sug.target_skill_id != null && projectSkills.get(sug.target_skill_id)) { + projectSkills.update(sug.target_skill_id, { name, description, body, tags }); + const updated = projectSkills.get(sug.target_skill_id)!; + projectSkillSuggestions.remove(sid); + wsHub.broadcastAll({ + type: 'project-skill-updated', + projectId: updated.project_id ?? sug.project_id, + skill: enrichSkillWithSourceChatTitle(updated), + }); + wsHub.broadcastAll({ type: 'project-skill-suggestion-removed', chatId, suggestionId: sid }); + res.type('application/json').json({ skill: updated }); + return; + } + + // 'new' (or a patch whose target vanished). Same-scope name collision → + // treat as an update of the existing skill rather than a duplicate insert. + const existing = projectSkills.getByName(projectId, name); + if (existing) { + projectSkills.update(existing.id, { name, description, body, tags }); + const updated = projectSkills.get(existing.id)!; + projectSkillSuggestions.remove(sid); + wsHub.broadcastAll({ + type: 'project-skill-updated', + projectId: updated.project_id ?? sug.project_id, + skill: enrichSkillWithSourceChatTitle(updated), + }); + wsHub.broadcastAll({ type: 'project-skill-suggestion-removed', chatId, suggestionId: sid }); + res.type('application/json').json({ skill: updated }); + return; + } + + const skill = projectSkills.create({ + projectId, + name, + description, + body, + tags, + sourceChatId: chatId, + }); + projectSkillSuggestions.remove(sid); + wsHub.broadcastAll({ + type: 'project-skill-added', + projectId: skill.project_id ?? sug.project_id, + skill: enrichSkillWithSourceChatTitle(skill), + }); + wsHub.broadcastAll({ type: 'project-skill-suggestion-removed', chatId, suggestionId: sid }); + res.type('application/json').json({ skill }); + } catch (err) { + res.status(400).json({ error: err instanceof Error ? err.message : 'accept failed' }); + } +}); + +chatsRouter.post('/:id/skill-suggestions/:suggestionId/reject', (req, res) => { + const chatId = Number(req.params.id); + const sid = Number(req.params.suggestionId); + if (!Number.isFinite(chatId) || !Number.isFinite(sid)) { + res.status(400).json({ error: 'invalid id' }); + return; + } + const sug = projectSkillSuggestions.get(sid); + if (!sug || sug.chat_id !== chatId) { + res.status(404).json({ error: 'not found' }); + return; + } + projectSkillSuggestions.remove(sid); + wsHub.broadcastAll({ type: 'project-skill-suggestion-removed', chatId, suggestionId: sid }); + res.type('application/json').json({ ok: true }); +}); + chatsRouter.get('/:id', async (req, res, next) => { try { const id = Number(req.params.id); const chat = chats.get(id); if (!chat) { - res.status(404).send('chat not found'); + // Stale link / deleted chat → send the user home instead of a dead-end + // 404 page. Keeps navigation friendly for non-technical users. + res.redirect('/'); return; } if (chats.markRead(id)) wsHub.broadcastAll({ type: 'chat-read', chatId: id }); const { agents, error: agentsError } = await getAgentsSafe(); + const chatMessages = messages.listByChat(id); + // The chat's "current" composer mode. Prefer the explicitly persisted + // chats.mode (set the moment the user picks a mode — survives navigation and + // syncs across devices). Fall back to the most recent message's mode for + // legacy chats that predate the column, then to the UI default (empty string). + let chatCurrentMode = ''; + if (chat.mode && isSelectableMode(chat.mode) && !isEphemeralMode(chat.mode as ChatMode)) { + chatCurrentMode = chat.mode; + } else { + // Only USER rows carry a user-chosen mode. Assistant rows mirror it, but + // synthetic system rows (e.g. the "saved X% tokens" savings badge) default + // to 'execute' — and being the newest rows they'd otherwise hijack this + // fallback, reopening a Work chat as Full Power. Walk back to the last + // user message instead. + for (let i = chatMessages.length - 1; i >= 0; i--) { + if (chatMessages[i].role !== 'user') continue; + const m = chatMessages[i].mode; + if (m && isSelectableMode(m) && !isEphemeralMode(m as ChatMode)) { + chatCurrentMode = m; + break; + } + } + } res.render('chat', { chats: chats.list(), allProjects: projects.list(), hasAnyTasks: tasks.hasAny(), taskStatusSignals: tasks.statusSignals(), activeChat: chat, - chatMessages: messages.listByChat(id), + chatMessages, agents, agentsError, defaultAgent: DEFAULT_AGENT, @@ -180,6 +339,16 @@ chatsRouter.get('/:id', async (req, res, next) => { scheduledList: scheduledMessages.listByChat(id), queueList: queuedMessages.listByChat(id), sendHintShow: shouldShowSendHint(), + chatModes: listComposerModes(), + defaultChatMode: defaultComposerMode(), + chatCurrentMode, + sttEnabled: openRouterEnabled(), + // Lets the composer lock the runtime modes (and the connect chooser fire) + // when no key is configured. + openRouterReady: openRouterEnabled(), + // Full Power (Execute) needs the gateway; agents.list succeeding implies it's + // reachable. Seeds the composer's Full Power gating (no badge on this page). + gatewayOk: !agentsError, }); } catch (err) { next(err); @@ -251,42 +420,34 @@ chatsRouter.post('/:id/shares', (req, res) => { res.redirect(`/chats/${id}`); }); -/** - * Toggle reasoning visibility on the active session by sending the slash - * command through the normal chat flow. The mode is mirrored locally so the - * UI toggle stays in sync across reloads. - */ -chatsRouter.post('/:id/reasoning', async (req, res) => { +// Persist the chat's sticky composer send-mode the moment the user picks it, so it +// survives navigation and syncs across devices instead of living only in this +// browser's localStorage. Pure iClaw UI state — the mode rides along with each +// sent message, so there's no gateway patch to make here. +chatsRouter.post('/:id/mode', (req, res) => { const id = Number(req.params.id); - const chat = chats.get(id); - if (!chat) { + if (!chats.get(id)) { res.status(404).json({ error: 'chat not found' }); return; } const raw = String(req.body?.mode ?? '').trim().toLowerCase(); - const mode: 'off' | 'on' | 'stream' = - raw === 'on' ? 'on' : raw === 'stream' ? 'stream' : 'off'; - // Mirror locally first — UI source of truth even if the gateway hiccups. - chats.setReasoningMode(id, mode); + // Only persist real, selectable, non-ephemeral modes — incognito is transient. + if (!isSelectableMode(raw) || isEphemeralMode(raw as ChatMode)) { + res.status(400).json({ error: 'invalid mode' }); + return; + } + chats.setChatMode(id, raw); + // A draft is hidden from the sidebar until its first user message. The client + // upserts a sidebar row on any `updatedAt`, so omit it for drafts — otherwise + // switching mode would leak the empty draft into the list. `mode` still + // broadcasts for cross-tab sync. wsHub.broadcastAll({ type: 'chat-updated', chatId: id, - reasoningMode: mode, - updatedAt: chats.get(id)!.updated_at, + mode: raw, + ...(chats.isDraft(id) ? {} : { updatedAt: chats.get(id)!.updated_at }), }); - // Push the real flip to OpenClaw. `sessions.patch` is the proper channel — - // it's what the dashboard uses. Failure here doesn't roll back the mirror; - // we surface the error to the caller so the UI can warn. - let gatewayWarning: string | null = null; - try { - await openclawWs.patchSession({ - sessionKey: chat.openclaw_session_id, - reasoningLevel: mode === 'off' ? null : mode, - }); - } catch (err) { - gatewayWarning = err instanceof Error ? err.message : String(err); - } - res.json({ id, mode, ...(gatewayWarning ? { gatewayWarning } : {}) }); + res.json({ id, mode: raw }); }); chatsRouter.post('/:id/unread', (req, res) => { @@ -619,6 +780,7 @@ chatsRouter.post('/:id/queue', (req, res) => { replyTo: replyTo ?? null, attachments: persistedAttachments, inlineSecrets: inlineSecrets ?? null, + mode: normalizeChatMode(req.body?.mode), }); wsHub.broadcastAll({ type: 'queue-added', chatId: id, item: row }); res.json({ item: row }); @@ -696,6 +858,7 @@ chatsRouter.post('/:id/queue/:queueId/flush', async (req, res) => { prePersistedAttachments: row.attachments && row.attachments.length > 0 ? row.attachments : undefined, requestId: String(req.body?.requestId ?? '').trim() || undefined, + mode: row.mode, }); res.json({ ok: true }); } catch (err) { @@ -934,3 +1097,49 @@ chatsRouter.patch('/:id/scheduled/:scheduledId', (req, res) => { .json({ error: err instanceof Error ? err.message : 'failed to update' }); } }); + +/** GET /chats/:id/workspace-info — workspace size for Work/Secure Mode. */ +chatsRouter.get('/:id/workspace-info', async (req, res) => { + const chatId = Number(req.params.id); + const sessionId = getWorkSessionId(chatId); + if (!sessionId) return res.json({ active: false }); + const info = await getWorkspaceInfo(sessionId); + res.json({ active: true, sessionId, ...info }); +}); + +/** + * POST /chats/:id/destroy-workspace — tear down the chat's Safe/Work sandbox: + * stops the container and deletes the workspace (the Safe-mode copy and all of + * its contents). The next message starts a fresh sandbox. + */ +chatsRouter.post('/:id/destroy-workspace', async (req, res) => { + const chatId = Number(req.params.id); + if (!Number.isFinite(chatId)) return res.status(400).json({ error: 'bad chat id' }); + const destroyed = await destroyWorkSession(chatId); + res.json({ destroyed }); +}); + +/** POST /chats/:id/export-sandbox — copy the Safe sandbox out to a host folder. */ +chatsRouter.post('/:id/export-sandbox', async (req, res) => { + const chatId = Number(req.params.id); + if (!Number.isFinite(chatId)) return res.status(400).json({ error: 'bad chat id' }); + const destDir = typeof req.body?.destDir === 'string' ? req.body.destDir : undefined; + try { + const result = await exportChatSandbox(chatId, destDir); + res.json(result); + } catch (err) { + res.status(502).json({ ok: false, error: err instanceof Error ? err.message : String(err) }); + } +}); + +/** POST /chats/:id/apply-sandbox — apply the sandbox's changes to the originals. */ +chatsRouter.post('/:id/apply-sandbox', async (req, res) => { + const chatId = Number(req.params.id); + if (!Number.isFinite(chatId)) return res.status(400).json({ error: 'bad chat id' }); + try { + const results = await applyChatSandboxChanges(chatId); + res.json({ results }); + } catch (err) { + res.status(502).json({ error: err instanceof Error ? err.message : String(err) }); + } +}); diff --git a/src/routes/docker.ts b/src/routes/docker.ts new file mode 100644 index 0000000..c766997 --- /dev/null +++ b/src/routes/docker.ts @@ -0,0 +1,41 @@ +/** + * Docker status + lifecycle for the chat composer's Docker gate. + * + * GET /api/docker/status — current daemon readiness (cached probe) + * POST /api/docker/start — launch an installed-but-idle daemon (localhost) + * POST /api/docker/install — install the engine, then start it (localhost) + * + * The actions run host commands, so they're localhost-only. They return at once + * with a `starting`/`installing` state; the composer polls /status until ready. + */ + +import { Router } from 'express'; +import { + DOCKER_SIZE_HINT, + getDockerState, + installDocker, + startDocker, +} from '../services/docker'; +import { isLocalhostRequest } from '../services/gatewayStart'; + +export const dockerRouter: Router = Router(); + +dockerRouter.get('/status', async (_req, res) => { + res.json({ state: await getDockerState(), sizeHint: DOCKER_SIZE_HINT }); +}); + +dockerRouter.post('/start', (req, res) => { + if (!isLocalhostRequest(req)) { + res.status(403).json({ error: 'forbidden' }); + return; + } + res.json({ state: startDocker() }); +}); + +dockerRouter.post('/install', (req, res) => { + if (!isLocalhostRequest(req)) { + res.status(403).json({ error: 'forbidden' }); + return; + } + res.json({ state: installDocker() }); +}); diff --git a/src/routes/index.ts b/src/routes/index.ts index bb1fbc2..6f5005d 100644 --- a/src/routes/index.ts +++ b/src/routes/index.ts @@ -1,18 +1,142 @@ -import { Router } from 'express'; +import express, { Router } from 'express'; +import { execFile } from 'node:child_process'; +import { promisify } from 'node:util'; import { chats, projectSecrets, projects, tasks } from '../services/store'; -import { openclaw } from '../services/openclaw'; +import { openclaw, cloudShareBaseUrl } from '../services/openclaw'; import { probeGateway } from '../services/gatewayProbe'; import { chatStatus } from '../services/chatStatus'; import { shouldShowSendHint } from '../services/sendHint'; +import { defaultComposerMode, listComposerModes } from '../services/chatModes'; +import { openRouterEnabled, transcribeAudio, isOpenRouterFailure } from '../services/openRouter'; +import { isOnboardingDone, setOnboardingDone } from '../services/config'; +import { startOnboardingPrep, getOnboardingEnv } from '../services/onboardingEnv'; + +const execFileAsync = promisify(execFile); export const indexRouter: Router = Router(); +/** + * First-run welcome screen. Shown until the user picks a power source (or skips) + * — gated solely on the `onboarding.done` flag so it never reappears after. + * Starts the background environment prep (Docker probe + image pre-pull) so the + * slow download happens while the user reads the copy / pastes their key. + */ +indexRouter.get('/welcome', (_req, res) => { + startOnboardingPrep(); + res.render('welcome', { + title: 'Welcome to iClaw', + hasKey: openRouterEnabled(), + }); +}); + +/** Honest background-prep status for the welcome progress line. */ +indexRouter.get('/api/onboarding/status', (_req, res) => { + res.json(getOnboardingEnv()); +}); + +/** Finish (or skip) onboarding — flips the flag so /welcome never shows again. */ +indexRouter.post('/api/onboarding/complete', (_req, res) => { + setOnboardingDone(); + res.json({ ok: true }); +}); + +/** Native OS folder picker — opens system dialog, returns selected path. */ +indexRouter.post('/api/pick-folder', async (_req, res) => { + try { + let folderPath: string; + if (process.platform === 'darwin') { + const { stdout } = await execFileAsync('osascript', [ + '-e', 'POSIX path of (choose folder with prompt "Select a folder for Work Mode")', + ]); + folderPath = stdout.trim().replace(/\/$/, ''); + } else if (process.platform === 'linux') { + const { stdout } = await execFileAsync('zenity', ['--file-selection', '--directory', '--title=Select folder for Work Mode']).catch(() => + execFileAsync('kdialog', ['--getexistingdirectory', process.env.HOME ?? '/']), + ); + folderPath = stdout.trim(); + } else { + return res.status(400).json({ error: 'Folder picker not supported on this platform' }); + } + if (!folderPath) return res.status(400).json({ error: 'No folder selected' }); + res.json({ path: folderPath }); + } catch (err: unknown) { + // User cancelled dialog — not an error + const msg = err instanceof Error ? err.message : String(err); + if (msg.includes('User canceled') || msg.includes('cancelled') || msg.includes('-128')) { + return res.status(204).end(); + } + res.status(500).json({ error: msg }); + } +}); + /** Draft composer — secret name check before the chat row exists. */ indexRouter.get('/api/secrets/check-label', (req, res) => { res.json({ available: projectSecrets.isLabelAvailable(String(req.query.label ?? '')) }); }); +/** Map an upload mime to the container hint OpenRouter expects for input_audio. */ +function audioFormatFromMime(mime: string): string { + const m = mime.toLowerCase(); + if (m.includes('webm')) return 'webm'; + if (m.includes('ogg')) return 'ogg'; + if (m.includes('wav')) return 'wav'; + if (m.includes('mpeg') || m.includes('mp3')) return 'mp3'; + if (m.includes('mp4') || m.includes('m4a') || m.includes('aac')) return 'm4a'; + if (m.includes('flac')) return 'flac'; + return 'webm'; +} + +/** + * Speech-to-text. The composer mic POSTs the recorded clip as a raw audio body + * (Content-Type carries the container). We transcribe it via OpenRouter and + * return `{ text }`. Gated on OPENROUTER_API_KEY — the mic button is only + * rendered when that's set; this 503 covers a stale client. + */ +indexRouter.post( + '/api/stt', + express.raw({ type: () => true, limit: '25mb' }), + async (req, res) => { + if (!openRouterEnabled()) { + res.status(503).json({ error: 'Speech-to-text is unavailable: set OPENROUTER_API_KEY.' }); + return; + } + const buf = req.body; + if (!Buffer.isBuffer(buf) || buf.length === 0) { + res.status(400).json({ error: 'No audio received.' }); + return; + } + const mime = String(req.headers['content-type'] || 'audio/webm'); + try { + const text = await transcribeAudio({ + audioBase64: buf.toString('base64'), + format: audioFormatFromMime(mime), + }); + res.json({ text }); + } catch (err) { + const msg = err instanceof Error ? err.message : String(err); + console.warn('[stt] transcription failed:', msg); + res + .status(502) + .json({ error: isOpenRouterFailure(err) ? 'Transcription failed (OpenRouter).' : msg }); + } + }, +); + indexRouter.get('/', async (req, res) => { + // First run: render the welcome flow in place of the empty chat. Rendered + // (not redirected) so `/` stays 200 — the CLI/instance readiness probe in + // startup.ts checks for a 200 that contains "iClaw". + if (!isOnboardingDone()) { + // Power users who already configured a key (env/prior run) shouldn't see + // the welcome at all — mark it done and fall through to the app. + if (openRouterEnabled()) { + setOnboardingDone(); + } else { + startOnboardingPrep(); + return res.render('welcome', { title: 'Welcome to iClaw', hasKey: openRouterEnabled() }); + } + } + const list = chats.list(); const allProjects = projects.list(); @@ -27,6 +151,9 @@ indexRouter.get('/', async (req, res) => { res.render('index', { chats: list, + // First-ever empty state → show the conversational welcome (and skip the + // project picker) instead of dropping a non-technical user into a blank chat. + isFirstChat: list.length === 0, allProjects, hasAnyTasks: tasks.hasAny(), taskStatusSignals: tasks.statusSignals(), @@ -39,7 +166,16 @@ indexRouter.get('/', async (req, res) => { agentsError, defaultAgent: 'openclaw/default', openclawBaseUrl: openclaw.baseUrl, + cloudShareBaseUrl, workingIds: chatStatus.workingIds(), sendHintShow: shouldShowSendHint(), + chatModes: listComposerModes(), + defaultChatMode: defaultComposerMode(), + sttEnabled: openRouterEnabled(), + // With an OpenRouter key the runtime modes (Work / Safe work / Incognito) + // work without OpenClaw — so a missing gateway must NOT block starting a chat. + openRouterReady: openRouterEnabled(), + // Full Power (Execute) needs a reachable gateway; seeds the composer gating. + gatewayOk: gatewayStatus === 'ok', }); }); diff --git a/src/routes/projects.ts b/src/routes/projects.ts index 8e7998a..68e2470 100644 --- a/src/routes/projects.ts +++ b/src/routes/projects.ts @@ -10,7 +10,7 @@ import { Router } from 'express'; import { listProjectLinkGroups } from '../services/projectLinks'; -import { chats, projects, projectFacts, projectSecrets, tasks, enrichFactsWithSourceChatTitles, enrichFactWithSourceChatTitle } from '../services/store'; +import { chats, projects, projectFacts, projectSkills, projectSecrets, tasks, enrichFactsWithSourceChatTitles, enrichFactWithSourceChatTitle, enrichSkillsWithSourceChatTitles, enrichSkillWithSourceChatTitle } from '../services/store'; import { chatStatus } from '../services/chatStatus'; import { wsHub } from '../services/wsHub'; import { openclaw } from '../services/openclaw'; @@ -77,7 +77,8 @@ projectsRouter.get('/:id', (req, res) => { const id = Number(req.params.id); const project = projects.get(id); if (!project) { - res.status(404).send('project not found'); + // Stale link / deleted project → home, not a dead-end 404 page. + res.redirect('/'); return; } const linkGroups = listProjectLinkGroups(id); @@ -89,6 +90,7 @@ projectsRouter.get('/:id', (req, res) => { project, projectChats: chats.listByProject(id), facts: enrichFactsWithSourceChatTitles(projectFacts.listByProject(id)), + skills: enrichSkillsWithSourceChatTitles(projectSkills.listForProject(id)), projectSecrets: projectSecrets.listMetaByProject(id), projectWebLinks: linkGroups.web, projectFileLinks: linkGroups.files, @@ -186,7 +188,8 @@ projectsRouter.get('/:id/secrets/:secretId/value', (req, res) => { projectsRouter.post('/:id/delete', (req, res) => { const id = Number(req.params.id); if (!projects.get(id)) { - res.status(404).send('project not found'); + // Already gone → mirror the success path back to the projects list. + res.redirect('/projects'); return; } const detachedChatIds = chats.listByProject(id).map((c) => c.id); @@ -247,3 +250,80 @@ projectsRouter.post('/:id/facts/:factId/delete', (req, res) => { if (wantsJson(req)) res.json({ id: factId, deleted: true }); else res.redirect(`/projects/${projectId}`); }); + +/* ---------------- skills (procedural memory; edit/delete here) ---------------- */ + +/** Skill belongs to the project when project-scoped, or is global (project_id null). */ +function skillVisibleToProject( + skill: { project_id: number | null } | undefined, + projectId: number, +): boolean { + return !!skill && (skill.project_id === projectId || skill.project_id === null); +} + +/** Active skills (project + global), index info only. */ +projectsRouter.get('/:id/skills', (req, res) => { + const id = Number(req.params.id); + if (!projects.get(id)) { + res.status(404).json({ error: 'project not found' }); + return; + } + res.json({ skills: enrichSkillsWithSourceChatTitles(projectSkills.listForProject(id)) }); +}); + +/** Full skill body (for the view/edit modal). */ +projectsRouter.get('/:id/skills/:skillId', (req, res) => { + const projectId = Number(req.params.id); + const skillId = Number(req.params.skillId); + const skill = projectSkills.get(skillId); + if (!projects.get(projectId) || !skillVisibleToProject(skill, projectId)) { + res.status(404).json({ error: 'not found' }); + return; + } + res.json({ skill: enrichSkillWithSourceChatTitle(skill!) }); +}); + +projectsRouter.patch('/:id/skills/:skillId', (req, res) => { + const projectId = Number(req.params.id); + const skillId = Number(req.params.skillId); + const skill = projectSkills.get(skillId); + if (!projects.get(projectId) || !skillVisibleToProject(skill, projectId)) { + res.status(404).json({ error: 'not found' }); + return; + } + const patch: { name?: string; description?: string; body?: string; tags?: string[] | null } = {}; + if (typeof req.body?.name === 'string' && req.body.name.trim()) patch.name = req.body.name; + if (typeof req.body?.description === 'string' && req.body.description.trim()) + patch.description = req.body.description; + if (typeof req.body?.body === 'string' && req.body.body.trim()) patch.body = req.body.body; + if (Array.isArray(req.body?.tags)) { + const tags = req.body.tags.filter((t: unknown): t is string => typeof t === 'string'); + patch.tags = tags.length > 0 ? tags : null; + } + try { + projectSkills.update(skillId, patch); + const updated = projectSkills.get(skillId)!; + wsHub.broadcastAll({ + type: 'project-skill-updated', + projectId, + skill: enrichSkillWithSourceChatTitle(updated), + }); + res.json(updated); + } catch (err) { + res.status(400).json({ error: err instanceof Error ? err.message : 'update failed' }); + } +}); + +projectsRouter.post('/:id/skills/:skillId/delete', (req, res) => { + const projectId = Number(req.params.id); + const skillId = Number(req.params.skillId); + const skill = projectSkills.get(skillId); + if (!skillVisibleToProject(skill, projectId)) { + res.status(404).json({ error: 'not found' }); + return; + } + projectSkills.remove(skillId); + wsHub.broadcastAll({ type: 'project-skill-deleted', projectId, skillId }); + if (wantsJson(req)) res.json({ id: skillId, deleted: true }); + else res.redirect(`/projects/${projectId}`); +}); diff --git a/src/routes/settings.ts b/src/routes/settings.ts index 480b82c..624ca25 100644 --- a/src/routes/settings.ts +++ b/src/routes/settings.ts @@ -1,9 +1,9 @@ /** * GET /settings — Settings page. * - * For now the only section is Remote Access, but the page is built as a - * sectioned scaffold so future settings can land alongside without a - * separate URL. + * Sectioned scaffold: Remote Access + OpenRouter (the key that unlocks voice + * messages, Ask mode, and smart titles). The key is stored in the local DB via + * Settings — not an env var — and takes effect on the next page load. */ import { Router } from 'express'; @@ -12,12 +12,20 @@ import { chats, projects, tasks } from '../services/store'; import { chatStatus } from '../services/chatStatus'; import { openclaw } from '../services/openclaw'; import { remoteAccess, ALLOWED_DURATIONS_MS } from '../services/remoteAccess'; +import { + maskOpenRouterApiKey, + setOpenRouterApiKey, + clearOpenRouterApiKey, +} from '../services/config'; +import { openRouterEnabled, fetchUsage, isOpenRouterFailure, validateKey } from '../services/openRouter'; +import { runtimeProcess } from '../services/runtimeProcess'; + export const settingsRouter = Router(); -settingsRouter.get('/settings', (_req, res) => { - res.render('settings', { +/** Locals every settings sub-page needs for the shared sidebar + shell. */ +function sidebarLocals() { + return { title: 'Settings — iClaw', - // Sidebar locals. chats: chats.list(), workingIds: chatStatus.workingIds(), allProjects: projects.list(), @@ -29,8 +37,95 @@ settingsRouter.get('/settings', (_req, res) => { activeTasksList: false, activeSettings: true, openclawBaseUrl: openclaw.baseUrl, - // Page-specific. + }; +} + +/** /settings → first sub-page. Each section is now its own page. */ +settingsRouter.get('/settings', (_req, res) => { + res.redirect('/settings/voice-ask'); +}); + +settingsRouter.get('/settings/voice-ask', (_req, res) => { + res.render('settings', { + ...sidebarLocals(), + settingsTab: 'voice-ask', + openRouter: { + hasKey: openRouterEnabled(), + maskedKey: maskOpenRouterApiKey(), + }, + }); +}); + +settingsRouter.get('/settings/remote-access', (_req, res) => { + res.render('settings', { + ...sidebarLocals(), + settingsTab: 'remote-access', tunnels: remoteAccess.list(), allowedDurationsMs: ALLOWED_DURATIONS_MS, }); }); + +/** + * Validate a key WITHOUT saving it — onboarding (and Settings) calls this before + * committing, so a dead/zero-balance key is caught up front instead of in chat. + */ +settingsRouter.post('/api/openrouter/validate', async (req, res) => { + const key = typeof req.body?.key === 'string' ? req.body.key.trim() : ''; + if (!key) { + res.status(400).json({ valid: false, reason: 'empty' }); + return; + } + try { + const result = await validateKey(key); + res.json(result); + } catch (err) { + const msg = err instanceof Error ? err.message : String(err); + res.status(502).json({ valid: false, reason: 'network', error: msg }); + } +}); + +/** Save / update the OpenRouter API key. Takes effect on the next page load. */ +settingsRouter.post('/api/openrouter/key', (req, res) => { + const key = typeof req.body?.key === 'string' ? req.body.key.trim() : ''; + if (!key) { + res.status(400).json({ error: 'Paste your OpenRouter API key first.' }); + return; + } + // OpenRouter keys look like `sk-or-...`. Be lenient (warn-not-block) so a + // future key format still works, but catch obvious paste mistakes. + if (!/^sk-or-/.test(key)) { + res.status(400).json({ error: 'That doesn’t look like an OpenRouter key (expected to start with “sk-or-”).' }); + return; + } + setOpenRouterApiKey(key); + // The runtime reads the key from the DB only at spawn — restart it so Work / + // Safe / Incognito modes work immediately after onboarding, without an app + // restart. + runtimeProcess.restart(); + res.json({ ok: true, maskedKey: maskOpenRouterApiKey() }); +}); + +/** Disconnect — remove the stored key. */ +settingsRouter.delete('/api/openrouter/key', (_req, res) => { + clearOpenRouterApiKey(); + // Drop the key from the running runtime too. + runtimeProcess.restart(); + res.json({ ok: true }); +}); + +/** Spend / credits readout for the connected key. */ +settingsRouter.get('/api/openrouter/usage', async (_req, res) => { + if (!openRouterEnabled()) { + res.status(404).json({ error: 'No OpenRouter key connected.' }); + return; + } + try { + const usage = await fetchUsage(); + res.json({ ok: true, usage }); + } catch (err) { + const msg = err instanceof Error ? err.message : String(err); + res + .status(502) + .json({ error: isOpenRouterFailure(err) ? 'Could not load usage from OpenRouter.' : msg }); + } +}); diff --git a/src/routes/ws.ts b/src/routes/ws.ts index 68fad60..20e7fab 100644 --- a/src/routes/ws.ts +++ b/src/routes/ws.ts @@ -8,13 +8,36 @@ import type { Server as HttpServer, IncomingMessage } from 'node:http'; import { WebSocketServer, type WebSocket } from 'ws'; import { chats } from '../services/store'; import { wsHub } from '../services/wsHub'; -import { sendMessage, abortChatRun } from '../services/chatRunner'; +import { sendMessage, abortChatRun, runIncognitoTurn, abortIncognito, type WorkFolder } from '../services/chatRunner'; import { openclawWs } from '../services/openclawWs'; import type { ClientMsg, ServerMsg } from '../types/protocol'; import type { InlineSecretWire } from '../services/inlineSecrets'; +import { normalizeChatMode } from '../services/chatModes'; const PATH = '/ws'; +/** + * Coerce the untrusted `workFolders` wire field into typed WorkFolder[]. Accepts + * either objects ({ path, readonly }) from the current client or bare path + * strings from older clients. Missing/unknown readonly defaults to true + * (read-only) — the safe default. Returns undefined when nothing valid is sent. + */ +function parseWorkFolders(raw: unknown): WorkFolder[] | undefined { + if (!Array.isArray(raw)) return undefined; + const out: WorkFolder[] = []; + for (const entry of raw) { + if (typeof entry === 'string') { + if (entry) out.push({ path: entry, readonly: true }); + } else if (entry && typeof entry === 'object') { + const o = entry as Record<string, unknown>; + if (typeof o.path === 'string' && o.path) { + out.push({ path: o.path, readonly: o.readonly !== false }); + } + } + } + return out.length > 0 ? out : undefined; +} + function send(socket: WebSocket, msg: ServerMsg): void { wsHub.send(socket, msg); } @@ -90,6 +113,12 @@ async function handleClientMsg(socket: WebSocket, msg: ClientMsg): Promise<void> replyTo: msg.replyTo, incomingAttachments: msg.attachments, inlineSecrets, + mode: normalizeChatMode((msg as { mode?: unknown }).mode), + networkEnabled: (msg as Record<string, unknown>).networkEnabled === true, + ttlDays: typeof (msg as Record<string, unknown>).ttlDays === 'number' + ? ((msg as Record<string, unknown>).ttlDays as number) + : undefined, + workFolders: parseWorkFolders((msg as Record<string, unknown>).workFolders), }); } catch (err) { // Errors are already broadcast via chatRunner; nothing more to do. @@ -106,6 +135,39 @@ async function handleClientMsg(socket: WebSocket, msg: ClientMsg): Promise<void> } return; + case 'incognito-send': { + const key = String((msg as { key?: unknown }).key ?? '').trim(); + const content = String(msg.content ?? '').trim(); + if (!key || !content) return; + let endedTokens: number | undefined; + let endedCached: number | undefined; + try { + const res = await runIncognitoTurn({ + key, + content, + workFolders: parseWorkFolders((msg as Record<string, unknown>).workFolders), + onEvent: (e) => { + if (e.type === 'text-delta') send(socket, { type: 'incognito-turn-delta', key, text: e.text }); + else if (e.type === 'tool') send(socket, { type: 'incognito-turn-tool', key, name: e.name }); + else if (e.type === 'error') send(socket, { type: 'incognito-error', key, message: e.message }); + }, + }); + endedTokens = res.tokens; + endedCached = res.cached; + } catch (err) { + send(socket, { type: 'incognito-error', key, message: err instanceof Error ? err.message : String(err) }); + } + // Always close the turn so the client can re-enable the composer. + send(socket, { type: 'incognito-turn-ended', key, tokens: endedTokens, cached: endedCached }); + return; + } + + case 'incognito-abort': { + const key = String((msg as { key?: unknown }).key ?? '').trim(); + if (key) await abortIncognito(key).catch((err) => console.error('[ws] incognito-abort failed', err)); + return; + } + case 'exec-approval': { const approvalId = String(msg.approvalId ?? '').trim(); const decision = msg.decision === 'denied' ? 'denied' : 'approved'; diff --git a/src/services/chatModes.ts b/src/services/chatModes.ts new file mode 100644 index 0000000..cc70b40 --- /dev/null +++ b/src/services/chatModes.ts @@ -0,0 +1,204 @@ +/** + * Single source of truth for chat "modes" — the distinctions the composer + * offers, plus a place to grow future modes without hardcoding them everywhere. + * + * Design notes + * ------------ + * - The wire/DB value is a plain string. The TS union (`ChatMode` in ../types) + * names the live modes; the catalog below can list `enabled: false` + * placeholders for modes we haven't built yet. Adding a live mode later = + * flip `enabled` and (if it needs a new union member) widen `ChatMode`. No DB + * migration is needed because the column is TEXT. + * - `DEFAULT_MODE` is 'execute' so anything that doesn't specify a mode — + * legacy rows, older clients, scheduled messages, task runs — behaves exactly + * as before. `normalizeChatMode()` enforces this fallback. + * + * Backends + * -------- + * - execute → the chat's main OpenClaw agent session (full tools). + * - work / secure / incognito → iclaw-runtime (our runtime), which requires + * OPENROUTER_API_KEY to be configured. + * + * Incognito (read-only, ephemeral) + * -------------------------------- + * A privacy mode: read files in ANY folder, read-only shell in a sandbox over + * folders the user selects, and web research — but NO writes and NO other + * actions. The conversation is ephemeral (kept only in the browser tab, never + * persisted) and contributes nothing to project memory. Enforcement lives in + * the runtime; the host treats incognito turns as non-persistent. + */ + +import type { ChatMode } from '../types'; +import { openRouterEnabled } from './openRouter'; +import { isOpenClawInstalled } from './openclawInstall'; + +export interface ChatModeDef { + /** Wire/DB value. */ + id: string; + /** Short label for the selector chip. */ + label: string; + /** One-liner shown in the selector / tooltip. */ + description: string; + /** Whether the mode is selectable today. Placeholders are `false`. */ + enabled: boolean; + /** Runs on iclaw-runtime (requires OPENROUTER_API_KEY), not the OpenClaw agent. */ + runtimeBacked: boolean; + /** Ephemeral, never-persisted conversation (incognito). */ + ephemeral: boolean; + /** + * Mode is unusable without a running Docker daemon. Safe work IS the Docker + * sandbox, so it fails closed without it. Work/Incognito only LOSE shell + * commands (run_command) — their file tools run on the host — so they stay + * usable and are NOT marked here; the UI shows a softer "shell needs Docker" + * hint for them instead of disabling the mode. + */ + requiresDocker: boolean; +} + +/** + * The full catalog. Order here is the order shown in the UI. Disabled entries + * are intentionally kept so the selector and any future router can enumerate + * the roadmap without scattering string literals across the codebase. + */ +export const CHAT_MODES: readonly ChatModeDef[] = [ + { + id: 'work', + label: 'Work', + description: 'AI edits files in folders you choose - you approve every change', + enabled: true, + runtimeBacked: true, + ephemeral: false, + requiresDocker: false, + }, + { + id: 'secure', + label: 'Safe work & Internet research', + description: 'Locked sandbox - safely run untrusted code and research the web, isolated from your system', + enabled: true, + runtimeBacked: true, + ephemeral: false, + requiresDocker: true, + }, + { + id: 'execute', + label: 'Full Power', + description: 'Full access via OpenClaw - for complex tasks that need more power', + enabled: true, + runtimeBacked: false, + ephemeral: false, + requiresDocker: false, + }, + // Set apart at the bottom (the composer renders a divider before any + // `ephemeral` mode) — it's a distinct, off-the-record surface. + { + id: 'incognito', + label: 'Incognito', + description: 'Private, read-only research - reads files & the web, never writes, nothing saved', + enabled: true, + runtimeBacked: true, + ephemeral: true, + requiresDocker: false, + }, + // --- Planned modes (not selectable yet) ------------------------------- + { + id: 'image', + label: 'Image', + description: 'Generate or edit images. (Coming soon.)', + enabled: false, + runtimeBacked: false, + ephemeral: false, + requiresDocker: false, + }, +] as const; + +/** Mode used whenever none is supplied or the supplied one is unknown/disabled. */ +export const DEFAULT_MODE: ChatMode = 'execute'; + +/** + * UI default for the composer's mode selector on a NEW/empty chat. Deliberately + * separate from DEFAULT_MODE: the latter is the backend normalization fallback + * (mode-less posted/API messages keep routing to the OpenClaw agent), while this + * only seeds what the picker shows when there's nothing else to go on. + */ +export const DEFAULT_COMPOSER_MODE: ChatMode = 'work'; + +/** Modes a client is allowed to select right now. */ +export const ENABLED_MODE_IDS: readonly string[] = CHAT_MODES.filter( + (m) => m.enabled, +).map((m) => m.id); + +/** + * A mode is available when it's enabled AND its backend is reachable. The + * runtime-backed modes (Work / Secure / Incognito) run on iclaw-runtime, which + * requires OpenRouter — so they're hidden (and posted values coerced to + * Execute) when no key is configured. + */ +function modeAvailable(def: ChatModeDef): boolean { + if (!def.enabled) return false; + if (def.runtimeBacked && !openRouterEnabled()) return false; + return true; +} + +/** Available modes only — backend source of truth for what may actually run. */ +export function listSelectableModes(): ChatModeDef[] { + return CHAT_MODES.filter(modeAvailable); +} + +/** + * Every ENABLED mode, each tagged with whether its backend is reachable right + * now. The composer always shows the full selector (so users discover the + * modes) and locks the ones that aren't usable yet — e.g. the runtime modes + * before an OpenRouter key is added. + */ +export function listComposerModes(): Array<ChatModeDef & { available: boolean }> { + return CHAT_MODES.filter((m) => m.enabled).map((m) => ({ + ...m, + available: modeAvailable(m), + })); +} + +function findMode(id: string): ChatModeDef | undefined { + return CHAT_MODES.find((m) => m.id === id); +} + +/** True only for an available (enabled + backend-reachable) known mode id. */ +export function isSelectableMode(id: string): boolean { + const def = findMode(id); + return Boolean(def && modeAvailable(def)); +} + +/** + * UI default for the composer's mode selector on a new chat: + * - an OpenRouter key is set → Work (the runtime path; the user's everyday mode). + * - no key, OpenClaw installed → Full Power (startable on demand). + * - no key, no OpenClaw → Work (locked until a key is added; the composer + * surfaces the connect chooser on first send). + * + * Deliberately does NOT prefer Full Power just because the gateway is up — a key + * user starts in Work and switches to Full Power explicitly when they want it. + */ +export function defaultComposerMode(): ChatMode { + if (openRouterEnabled()) return DEFAULT_COMPOSER_MODE; + return isOpenClawInstalled() ? 'execute' : DEFAULT_COMPOSER_MODE; +} + +/** + * Coerce any untrusted input (query body, WS frame, DB row) into a valid, + * currently-selectable `ChatMode`. Unknown, disabled, or missing → DEFAULT_MODE. + */ +export function normalizeChatMode(raw: unknown): ChatMode { + if (typeof raw !== 'string') return DEFAULT_MODE; + const id = raw.trim().toLowerCase(); + if (isSelectableMode(id)) return id as ChatMode; + return DEFAULT_MODE; +} + +/** Definition for a (normalized) mode — always defined for selectable modes. */ +export function getModeDef(mode: ChatMode): ChatModeDef { + return findMode(mode) ?? findMode(DEFAULT_MODE)!; +} + +/** True for the ephemeral, never-persisted incognito mode. */ +export function isEphemeralMode(mode: ChatMode): boolean { + return getModeDef(mode).ephemeral; +} diff --git a/src/services/chatRunner.ts b/src/services/chatRunner.ts index b233895..9629c9a 100644 --- a/src/services/chatRunner.ts +++ b/src/services/chatRunner.ts @@ -7,20 +7,29 @@ */ import { randomUUID } from 'node:crypto'; -import { chats, messages, projects, projectSecrets } from './store'; +import { chats, messages, projects, projectSecrets, projectFacts } from './store'; import { buildGatewayUserMessage, scheduleProjectFactExtraction } from './projectMemory'; +import { buildSkillsPromptBlock, scheduleProjectSkillReview } from './projectSkills'; +import { projectSkills } from './store'; import { chatStatus } from './chatStatus'; import { openclawWs, type TurnEvent } from './openclawWs'; import { deriveTitle, suggestChatTitleWithTimeout } from './chatTitle'; -import { toolActivityLabel } from './toolLabels'; +import { toolActivityLabel, toolActivityDetail } from './toolLabels'; import { wsHub } from './wsHub'; import { gatewayAttachmentsFromPersisted, persistIncomingAttachments, + persistAgentImage, + runtimeAttachmentsFromPersisted, type IncomingAttachment, type ProcessedAttachment, } from './uploads'; -import type { Message, MessageAttachment } from '../types'; +import { resolve as resolvePath, sep as pathSep, join as joinPath } from 'node:path'; +import { homedir } from 'node:os'; +import type { ChatMode, Message, MessageAttachment } from '../types'; +import { DEFAULT_MODE } from './chatModes'; +import { buildCompactedHistory } from './contextCompaction'; +import { createWorkSession, sendWorkMessage, subscribeWorkEvents, stopWorkSession, abortWorkSession, exportSandbox, applySandboxChanges, type ExportResult, type ApplyResult } from './workRuntime'; import { expandStoredSecretPlaceholdersForGateway, resolveInlineSecretMarkersInContent, @@ -30,6 +39,22 @@ import { const DEFAULT_AGENT = 'openclaw/default'; +/** + * A folder granted to a Work Mode chat, with its access level. `readonly: true` + * means the agent may read/list/search but not write_file or run_command under + * it; `readonly: false` grants read & write. New folders default to read-only. + */ +export interface WorkFolder { + path: string; + readonly: boolean; +} + +/** + * OpenClaw session key currently executing an Execute turn for a chat. Lets + * `abortChatRun` stop the right gateway run. Cleared in the runTurn `finally`. + */ +const activeRunSessionKeys = new Map<number, string>(); + /** * Map an iClaw agent label ("openclaw/default", "openclaw/code", ...) to the * raw OpenClaw agentId ("main", "code", ...). @@ -211,6 +236,11 @@ async function runTurnLocked(opts: { /** Attachments already saved under data/uploads (queued-message flush). */ prePersistedAttachments?: MessageAttachment[]; inlineSecrets?: InlineSecretWire[]; + /** 'execute' | 'work' | 'secure' | 'incognito'. Defaults to 'execute'. */ + mode?: ChatMode; + workFolders?: WorkFolder[]; + networkEnabled?: boolean; + ttlDays?: number; }): Promise<void> { const { chatId, @@ -221,8 +251,8 @@ async function runTurnLocked(opts: { prePersistedAttachments, inlineSecrets, } = opts; + const mode: ChatMode = opts.mode ?? DEFAULT_MODE; const chat = chats.get(chatId)!; - const sessionKey = await ensureSession(chatId); const projectId = chat.project_id ?? null; let storedUserContent = content; @@ -268,10 +298,12 @@ async function runTurnLocked(opts: { gatewayBody = formatReplyGatewayBlock(refExpanded, reply.quote) + gatewayBody; } - const gatewayMessage = + const gatewayMessageBase = chat.project_id != null && projects.get(chat.project_id) ? buildGatewayUserMessage(gatewayBody, chat.project_id) : gatewayBody; + // The turn is dispatched AFTER the user row is persisted (Ask needs + // userMsg.id to seed prior-thread context). See the Ask/Execute branch below. // Persist user message + broadcast (stored text keeps placeholders only). const replyToRole = @@ -289,6 +321,7 @@ async function runTurnLocked(opts: { } : null, persistedAttachments.length > 0 ? persistedAttachments : null, + mode, ); for (const sid of newSecretIds) { projectSecrets.setSourceMessage(sid, userMsg.id); @@ -350,6 +383,9 @@ async function runTurnLocked(opts: { let switchedToGenerating = false; let assistantText = ''; + // Whether this turn invoked any tool — used to throttle skill review to + // "substantive" turns (procedural learning comes from tool use, not chit-chat). + let usedTools = false; const onEvent = (ev: TurnEvent): void => { if (ev.type === 'text-delta') { @@ -360,6 +396,7 @@ async function runTurnLocked(opts: { assistantText += ev.text; wsHub.broadcastToChat(chatId, { type: 'turn-delta', chatId, text: ev.text }); } else if (ev.type === 'tool-start') { + usedTools = true; chatStatus.setActivity(chatId, { kind: 'tool', name: ev.name, @@ -394,13 +431,6 @@ async function runTurnLocked(opts: { phase: ev.phase, label: ev.label, }); - } else if (ev.type === 'reasoning') { - // Surface model reasoning only when the user opted in for this chat. - // We re-read chat row every event because the toggle can flip mid-turn. - const cur = chats.get(chatId); - if (cur && cur.reasoning_mode && cur.reasoning_mode !== 'off') { - wsHub.broadcastToChat(chatId, { type: 'turn-reasoning', chatId, text: ev.text }); - } } else if (ev.type === 'attachment') { const proxied = rewriteMediaUrl(ev.url); // Inline into the running text so the stream-renderer picks it up, AND @@ -421,16 +451,42 @@ async function runTurnLocked(opts: { } }; - const { - text: gatewayAccumulated, - aborted, - authoritativeText, - } = await openclawWs.runTurn({ - sessionKey, - message: gatewayMessage, - onEvent, - attachments: gatewayAttachments.length > 0 ? gatewayAttachments : undefined, - }); + // Work / Secure Mode — routes to iclaw-runtime, returns early. Forward dropped + // files so the runtime agent can read them (Work) / stage them (Secure) and + // see images — otherwise the model has no idea a file was attached. + if (mode === 'work' || mode === 'secure') { + const runtimeAttachments = runtimeAttachmentsFromPersisted(chatId, persistedAttachments); + await runWorkModeTurn({ chatId, content: gatewayMessageBase, onEvent, workFolders: opts.workFolders, secure: mode === 'secure', networkEnabled: opts.networkEnabled, ttlDays: opts.ttlDays, beforeMsgId: userMsg.id, reviewUserMessage: storedUserContent, attachments: runtimeAttachments }); + wsHub.broadcastAll({ type: 'turn-ended', chatId, title: chats.get(chatId)?.title ?? '', aborted: false }); + return; + } + + // Execute: the chat's own main-agent OpenClaw session, full tools. + // Provision the gateway session lazily, HERE — so Work / Safe work / Incognito + // chats (which use iclaw-runtime, not the gateway) can start and run even when + // OpenClaw is unreachable, as long as an OpenRouter key is configured. + const sessionKey = await ensureSession(chatId); + let gatewayAccumulated = ''; + let aborted = false; + let authoritativeText: string | null = null; + let turnUsage: { tokens: number | null; cached: number | null } = { tokens: null, cached: null }; + + activeRunSessionKeys.set(chatId, sessionKey); + try { + const turn = await openclawWs.runTurn({ + sessionKey, + message: gatewayMessageBase, + onEvent, + attachments: gatewayAttachments.length > 0 ? gatewayAttachments : undefined, + }); + gatewayAccumulated = turn.text; + aborted = turn.aborted; + authoritativeText = turn.authoritativeText; + // `usage` is absent from older stubs/mocks — keep the null default then. + if (turn.usage) turnUsage = turn.usage; + } finally { + activeRunSessionKeys.delete(chatId); + } // Picking the assistant text in priority order: // @@ -469,7 +525,12 @@ async function runTurnLocked(opts: { const skipAssistant = aborted && finalText.trim().length === 0; const assistantMsg = skipAssistant ? null - : messages.append(chatId, 'assistant', finalText, aborted ? 'aborted' : null); + : messages.append( + chatId, 'assistant', finalText, aborted ? 'aborted' : null, null, null, + // Stamp the execute (OpenClaw) mode + dev-mode token usage resolved from + // the gateway history slice, mirroring the Work/Secure path. + 'execute', turnUsage.tokens, turnUsage.cached, + ); if (assistantMsg) { wsHub.broadcastToChat(chatId, { type: 'message-appended', @@ -479,6 +540,7 @@ async function runTurnLocked(opts: { syncSidebarUnread(chatId); } + // Persistent "Stopped by user" marker. Lives in `messages` so it // survives page reload + lands in the iClaw-cloud share payload. // Rendered exactly like the existing "Task done: …" / "Task created: @@ -511,6 +573,22 @@ async function runTurnLocked(opts: { assistantText: finalText, assistantMessageId: assistantMsg.id, }); + + // Procedural memory: throttled skill review (heavier than fact extraction, + // so only every Nth substantive turn). MVP untrusted heuristic: a tool-using + // turn with network reach may have ingested external content (web/email/etc.), + // so its distilled skills are flagged for extra scrutiny in the inbox. + // (This site handles Ask/Execute; Work/Secure returns earlier — see above.) + scheduleProjectSkillReview({ + chatId, + projectId: chatAfter.project_id, + sharesToProject: Boolean(chatAfter.shares_to_project), + substantive: usedTools, + userMessage: storedUserContent, + assistantText: finalText, + assistantMessageId: assistantMsg.id, + untrusted: usedTools && Boolean(opts.networkEnabled), + }); } // Wait for the title task (if any) so the final `turn-ended` reflects it. @@ -584,6 +662,14 @@ export async function sendMessage(opts: { incomingAttachments?: IncomingAttachment[]; /** Files already on disk (queued-message flush). */ prePersistedAttachments?: MessageAttachment[]; + /** 'ask' | 'execute'. Defaults to 'execute' when omitted. */ + mode?: ChatMode; + /** Allowed folders for Work Mode, each with a read-only / read&write flag. */ + workFolders?: WorkFolder[]; + /** Network toggle for Secure Mode. */ + networkEnabled?: boolean; + /** TTL in days for Secure Mode workspace. */ + ttlDays?: number; }): Promise<{ chatId: number }> { let chatId = opts.chatId; let isFirstTurn = false; @@ -629,6 +715,10 @@ export async function sendMessage(opts: { incomingAttachments: opts.incomingAttachments, prePersistedAttachments: opts.prePersistedAttachments, inlineSecrets: opts.inlineSecrets, + mode: opts.mode, + workFolders: opts.workFolders, + networkEnabled: opts.networkEnabled, + ttlDays: opts.ttlDays, }), ); } catch (err) { @@ -663,7 +753,463 @@ export async function sendMessage(opts: { return { chatId }; } +/** Build system prompt for Work/Secure Mode including project context. */ +function buildWorkSystemPrompt(chatId: number): string { + const chat = chats.get(chatId); + const lines: string[] = []; + + if (chat?.project_id) { + const project = projects.get(chat.project_id); + if (project?.name) lines.push(`Project: ${project.name}`); + if (project?.description) lines.push(`Description: ${project.description}`); + + const facts = projectFacts.listByProject(chat.project_id, 20); + if (facts.length > 0) { + lines.push('\nProject context:'); + facts.forEach((f) => lines.push(`- ${f.content}`)); + } + + const skillsBlock = buildSkillsPromptBlock(chat.project_id); + if (skillsBlock) { + lines.push('\n' + skillsBlock); + } + } + + return lines.length > 0 ? lines.join('\n') : ''; +} + +/** + * Persistent Work Mode sessions — one per chat, reused across turns. We store + * the folder signature alongside the sessionId so a mid-chat change to the + * folder set or a folder's read-only/read&write flag forces the session to be + * recreated with the new access (folderAccess is fixed at session creation). + */ +const workSessions = new Map< + number, + { sessionId: string; foldersKey: string; secure: boolean; skillsKey: string } +>(); + +/** + * Stable signature of the folders granted to a chat. Order-independent so the + * key only changes when a path or its readonly flag actually changes. + */ +function foldersSignature(folders?: WorkFolder[]): string { + if (!folders?.length) return ''; + return JSON.stringify( + folders + .map((f) => ({ p: f.path, r: f.readonly })) + .sort((a, b) => a.p.localeCompare(b.p)), + ); +} + +/** + * Stable signature of the active skills injected into a chat's session. Changes + * when a skill is accepted/edited/deleted (id+version), so the work session is + * recreated with the new skill set without a manual restart — same class of fix + * as folder-access / Work<->Secure mode changes. + */ +function skillsSignature(chatId: number): string { + const chat = chats.get(chatId); + if (!chat?.project_id) return ''; + const idx = projectSkills.listForProject(chat.project_id); + if (idx.length === 0) return ''; + return idx + .map((s) => `${s.id}:${s.version}`) + .sort() + .join(','); +} + +/** + * Skill-review cadence for Work/Secure. Smaller than the Ask/Execute interval: + * these turns are agentic and tool-heavy, so each one is more likely to contain + * a reusable procedure worth distilling. + */ +const WORK_SKILL_REVIEW_INTERVAL = 4; + +/** Get the runtime sessionId for a chat (if active). */ +export function getWorkSessionId(chatId: number): string | undefined { + return workSessions.get(chatId)?.sessionId; +} + +/** + * Destroy a chat's Work/Safe runtime session — stops the container AND deletes + * the workspace (the Safe sandbox copy and anything in it). Backs the + * "Destroy sandbox" button. The map entry is cleared so the next turn starts a + * fresh session (re-ingesting any selected folders for Safe Mode). + */ +export async function destroyWorkSession(chatId: number): Promise<boolean> { + const sessionId = workSessions.get(chatId)?.sessionId; + workSessions.delete(chatId); + if (!sessionId) return false; + try { + await stopWorkSession(sessionId); + } catch { + /* best-effort — the map is already cleared, so a stale session just TTLs out. */ + } + return true; +} + +/** Export a chat's Safe sandbox to a host folder. */ +export async function exportChatSandbox(chatId: number, destDir?: string): Promise<ExportResult> { + const sessionId = workSessions.get(chatId)?.sessionId; + if (!sessionId) return { ok: false, error: 'no active sandbox for this chat' }; + return exportSandbox(sessionId, destDir); +} + +/** Apply a chat's Safe sandbox changes back to the original folders. */ +export async function applyChatSandboxChanges(chatId: number): Promise<ApplyResult[]> { + const sessionId = workSessions.get(chatId)?.sessionId; + if (!sessionId) return []; + return applySandboxChanges(sessionId); +} + +/** Secure workspaces live here (mirrors the runtime's SECURE_DATA_DIR default). */ +const SECURE_WORKSPACE_ROOT = process.env.ICLAW_SECURE_DATA_DIR || joinPath(homedir(), '.iclaw', 'secure'); + +/** + * Defense-in-depth for show_image: the runtime already validates the image is + * inside an allowed folder before emitting the event, but the host independently + * re-checks the absolute path lies under a root this chat legitimately controls + * — its granted Work folders, or (Secure) the runtime's workspace root — before + * copying anything into the chat's uploads. + */ +function imagePathAllowed(hostPath: string, opts: { workFolders?: WorkFolder[]; secure?: boolean }): boolean { + const abs = resolvePath(hostPath); + const roots: string[] = []; + if (opts.secure) roots.push(resolvePath(SECURE_WORKSPACE_ROOT)); + for (const f of opts.workFolders ?? []) roots.push(resolvePath(f.path)); + return roots.some((root) => abs === root || abs.startsWith(root + pathSep)); +} + +/** + * Route a Work Mode turn to iclaw-runtime. + * Reuses the session across turns to preserve conversation history. + */ +async function runWorkModeTurn(opts: { + chatId: number; + content: string; + onEvent: (event: TurnEvent) => void; + workFolders?: WorkFolder[]; + secure?: boolean; + networkEnabled?: boolean; + ttlDays?: number; + /** Current user message id — history before it seeds the session context. */ + beforeMsgId?: number; + /** Original (stored) user text for skill review — without injected project prefix. */ + reviewUserMessage?: string; + /** Dropped files (absolute host paths) to forward to the runtime agent. */ + attachments?: { path: string; mimeType: string; fileName: string }[]; +}): Promise<void> { + const { chatId, content, onEvent } = opts; + + // Mode and folder access are baked in at session creation. Tear the session + // down (so it's recreated below) when either changed since it was created: + // - the mode switched Work↔Secure, or + // - (Work only) the folder set or a read-only/read&write flag changed. + // Without this, switching mode or toggling a folder in the UI is silently + // ignored — the chat keeps running on the stale session. + const wantSecure = !!opts.secure; + const foldersKey = wantSecure ? '' : foldersSignature(opts.workFolders); + const skillsKey = skillsSignature(chatId); + const existing = workSessions.get(chatId); + if ( + existing && + (existing.secure !== wantSecure || + (!wantSecure && existing.foldersKey !== foldersKey) || + existing.skillsKey !== skillsKey) + ) { + await stopWorkSession(existing.sessionId).catch(() => {}); + workSessions.delete(chatId); + const note = + existing.secure !== wantSecure + ? `Mode changed — restarted the session in ${wantSecure ? 'Secure' : 'Work'} mode.` + : existing.skillsKey !== skillsKey && + existing.foldersKey === foldersKey + ? 'Project skills changed — restarted the work session with the updated skills.' + : 'Folder access changed — restarted the work session with the new permissions.'; + const sys = messages.append(chatId, 'system', note, null); + wsHub.broadcastToChat(chatId, { type: 'message-appended', chatId, message: sys }); + } + + // Reuse existing session for this chat, or create a new one + let sessionId = workSessions.get(chatId)?.sessionId; + if (!sessionId) { + try { + // Per-folder access drives both the allowed-path list and read-only + // enforcement. Secure Mode ignores it (the sandbox workspace is the only + // mount); when no folders are chosen we fall back to HOME with write + // access, preserving prior behavior. + const hasFolders = !opts.secure && !!opts.workFolders?.length; + const folderAccess = hasFolders ? opts.workFolders : undefined; + const allowedFolders = hasFolders + ? opts.workFolders!.map((f) => f.path) + : [process.env.HOME ?? ''].filter(Boolean); + // Safe Mode: the folders the user picked are COPIED into the isolated + // sandbox (originals untouched), not bind-mounted live like Work Mode. + const copyFolders = + opts.secure && opts.workFolders?.length + ? opts.workFolders.map((f) => f.path) + : undefined; + // Seed the (possibly restored) session with compacted prior history from + // our DB, so context survives runtime restarts (older turns summarized). + const history = opts.beforeMsgId + ? await buildCompactedHistory(chatId, opts.beforeMsgId) + : undefined; + sessionId = await createWorkSession({ + allowedFolders, + folderAccess, + copyFolders, + secure: opts.secure, + systemPrompt: buildWorkSystemPrompt(chatId), + // Stable key → the chat reconnects to its persisted Secure workspace + // (and its running TTL) after a runtime restart. + key: `chat:${chatId}`, + history, + }); + workSessions.set(chatId, { sessionId, foldersKey, secure: wantSecure, skillsKey }); + } catch (err) { + const note = `Work Mode runtime unavailable. (${err instanceof Error ? err.message : String(err)})`; + const sys = messages.append(chatId, 'system', note, 'work-unavailable'); + wsHub.broadcastToChat(chatId, { type: 'message-appended', chatId, message: sys }); + return; + } + } + + // Safe Mode: forward the current folder selection every turn so folders added + // mid-chat get copied into the live sandbox (the runtime ingests new ones only). + const turnCopyFolders = + opts.secure && opts.workFolders?.length + ? opts.workFolders.map((f) => f.path) + : undefined; + try { + await sendWorkMessage(sessionId, content, opts.networkEnabled, opts.ttlDays, opts.attachments, turnCopyFolders); + } catch (err) { + // Session may have expired — retry with a fresh one + workSessions.delete(chatId); + const note = `Work Mode session lost, please resend. (${err instanceof Error ? err.message : String(err)})`; + const sys = messages.append(chatId, 'system', note, null); + wsHub.broadcastToChat(chatId, { type: 'message-appended', chatId, message: sys }); + return; + } + + await new Promise<void>((resolve) => { + let accumulated = ''; + // Images the agent surfaced via show_image this turn — attached to the reply + // row so they render inline (on `done`, and again from the DB on reload). + const turnAttachments: MessageAttachment[] = []; + // Savings notes arrive mid-stream (when a tool truncates), but we want them + // rendered BELOW the assistant's reply, not above it. So buffer them and + // flush as system rows only after the assistant row is appended on `done`. + const savingsTexts: string[] = []; + const unsubscribe = subscribeWorkEvents( + sessionId!, + (event) => { + if (event.type === 'text') { + accumulated += event.content; + onEvent({ type: 'text-delta', text: event.content }); + } else if (event.type === 'tool_start') { + // Surface what the agent is doing right now (live status label, e.g. + // "Searching social media…"). Reuses the same turn-tool pipeline the + // gateway path uses; without this Work/Secure showed only "Thinking…". + onEvent({ + type: 'tool-start', + name: event.name, + label: toolActivityLabel(event.name), + detail: toolActivityDetail(event.name, event.input), + }); + } else if (event.type === 'tool_result') { + onEvent({ type: 'tool-end', name: event.name }); + } else if (event.type === 'note') { + // A tool gave the model less than the full content — surface the saving + // as a friendly, plain-language chat note (no jargon: no "tokens", + // "output", "command"). When we have a real measured percentage we show + // it; otherwise (search line-trimming, where ripgrep never reports how + // much it dropped) we stay quantity-free rather than invent a number. + // The "ніж звичайні помічники" framing is a category claim anchored to + // the naive approach (load the whole file) — never a per-competitor + // benchmark, which we don't measure. + const n = event.note; + const text = + typeof n.savedPct === 'number' + ? `iClaw обробив це на ${n.savedPct}% економніше, ніж звичайні помічники 💚` + : `iClaw обробив це економніше, ніж звичайні помічники 💚`; + if (!savingsTexts.includes(text)) savingsTexts.push(text); // dedupe within a turn + } else if (event.type === 'image') { + // show_image: copy the agent's image (already on the host — a Work + // bind-mount or a Secure workspace) into this chat's uploads and queue + // it as an attachment. Re-validate the path host-side before touching + // the file; silently skip anything that fails (never break the turn). + if (imagePathAllowed(event.path, opts)) { + const att = persistAgentImage(chatId, event.path); + if (att) turnAttachments.push(att); + } + } else if (event.type === 'done') { + if (accumulated.trim() || turnAttachments.length > 0) { + // Stamp the assistant row with the actual turn mode (secure/work), + // not the append() default of 'execute'. Keeps history + UI honest. + // `tokens` powers the dev-mode usage badge (null when not reported). + const assistantMsg = messages.append( + chatId, 'assistant', accumulated, null, null, + turnAttachments.length > 0 ? turnAttachments : null, + opts.secure ? 'secure' : 'work', event.tokens ?? null, event.cached ?? null, + ); + wsHub.broadcastToChat(chatId, { type: 'message-appended', chatId, message: assistantMsg }); + + // Procedural memory: review Work/Secure turns too. These are agentic + // and tool-capable by nature, so every completed turn counts as + // "substantive". Untrusted heuristic: a turn with network reach may + // have ingested external content (web/email/etc.). Throttled with a + // smaller interval than Ask/Execute. Fire-and-forget. + const chatNow = chats.get(chatId); + if (chatNow?.project_id != null && projects.get(chatNow.project_id)) { + scheduleProjectSkillReview({ + chatId, + projectId: chatNow.project_id, + sharesToProject: Boolean(chatNow.shares_to_project), + substantive: true, + userMessage: opts.reviewUserMessage ?? content, + assistantText: accumulated, + assistantMessageId: assistantMsg.id, + untrusted: Boolean(opts.networkEnabled), + interval: WORK_SKILL_REVIEW_INTERVAL, + }); + } + } + // Now that the reply row exists, drop the savings note(s) UNDER it. + for (const text of savingsTexts) { + const sys = messages.append(chatId, 'system', text, 'savings'); + wsHub.broadcastToChat(chatId, { type: 'message-appended', chatId, message: sys }); + } + unsubscribe(); + resolve(); + } else if (event.type === 'error') { + const sys = messages.append(chatId, 'system', `Work Mode error: ${event.message}`, null); + wsHub.broadcastToChat(chatId, { type: 'message-appended', chatId, message: sys }); + unsubscribe(); + resolve(); + } + }, + (err) => { + // Ignore "aborted" — it just means the SSE stream closed normally + if (err.message !== 'aborted') { + const sys = messages.append(chatId, 'system', `Work Mode connection error: ${err.message}`, null); + wsHub.broadcastToChat(chatId, { type: 'message-appended', chatId, message: sys }); + } + resolve(); + }, + ); + }); +} + +/** + * Incognito turns — fully ephemeral. Keyed by a client-generated string (the + * browser's in-RAM chat id), NOT a DB chat. Nothing is persisted: no message + * rows, no chat row, no facts/skills review. Output streams to the caller, who + * forwards it to the originating socket only. The runtime session enforces + * read-only + read-anywhere + web_fetch (incognito:true). + */ +const incognitoSessions = new Map<string, { sessionId: string; foldersKey: string }>(); + +export type IncognitoEvent = + | { type: 'text-delta'; text: string } + | { type: 'tool'; name: string } + | { type: 'error'; message: string }; + +export async function runIncognitoTurn(opts: { + key: string; + content: string; + workFolders?: WorkFolder[]; + onEvent: (event: IncognitoEvent) => void; +}): Promise<{ tokens?: number; cached?: number }> { + const { key, content, onEvent } = opts; + + // Recreate the runtime session if the selected folders changed (the shell's + // :ro mounts are baked in at creation), mirroring the Work path. + const foldersKey = foldersSignature(opts.workFolders); + const existing = incognitoSessions.get(key); + if (existing && existing.foldersKey !== foldersKey) { + await stopWorkSession(existing.sessionId).catch(() => {}); + incognitoSessions.delete(key); + } + + let sessionId = incognitoSessions.get(key)?.sessionId; + if (!sessionId) { + try { + const folderAccess = opts.workFolders?.length ? opts.workFolders : undefined; + // No HOME fallback: incognito reads anywhere via the runtime, and with no + // folders the read-only shell is simply unavailable (file/web tools work). + const allowedFolders = folderAccess ? folderAccess.map((f) => f.path) : []; + sessionId = await createWorkSession({ + allowedFolders, + folderAccess, + incognito: true, + key: `incognito:${key}`, + }); + incognitoSessions.set(key, { sessionId, foldersKey }); + } catch (err) { + onEvent({ type: 'error', message: `Incognito runtime unavailable. (${err instanceof Error ? err.message : String(err)})` }); + return {}; + } + } + + try { + await sendWorkMessage(sessionId, content); + } catch (err) { + incognitoSessions.delete(key); + onEvent({ type: 'error', message: `Incognito session lost, please resend. (${err instanceof Error ? err.message : String(err)})` }); + return {}; + } + + return await new Promise<{ tokens?: number; cached?: number }>((resolve) => { + const unsubscribe = subscribeWorkEvents( + sessionId!, + (event) => { + if (event.type === 'text') { + onEvent({ type: 'text-delta', text: event.content }); + } else if (event.type === 'tool_start') { + onEvent({ type: 'tool', name: event.name }); + } else if (event.type === 'done') { + unsubscribe(); + resolve({ tokens: event.tokens, cached: event.cached }); + } else if (event.type === 'error') { + onEvent({ type: 'error', message: event.message }); + unsubscribe(); + resolve({}); + } + }, + (err) => { + if (err.message !== 'aborted') onEvent({ type: 'error', message: err.message }); + resolve({}); + }, + ); + }); +} + +/** Stop and forget an incognito session (abort / tab close). */ +export async function abortIncognito(key: string): Promise<void> { + const e = incognitoSessions.get(key); + if (!e) return; + incognitoSessions.delete(key); + await stopWorkSession(e.sessionId).catch(() => {}); +} + export async function abortChatRun(chatId: number): Promise<void> { + // Execute: abort the OpenClaw run on the session actually executing this turn. + const active = activeRunSessionKeys.get(chatId); + if (active && active.startsWith('agent:')) { + await openclawWs.abortRun(active); + return; + } + // Work / Secure: a runtime turn. Abort it without destroying the session — + // the workspace and warm container survive (unlike stopWorkSession). This is + // what makes the header Stop button work for Work and Sandbox, not just + // OpenClaw/Execute. + const workSessionId = getWorkSessionId(chatId); + if (workSessionId) { + await abortWorkSession(workSessionId).catch(() => {}); + return; + } + // Fallback: abort the chat's main OpenClaw session if it's mid-run. const chat = chats.get(chatId); if (!chat) return; if (chat.openclaw_session_id?.startsWith('agent:')) { diff --git a/src/services/chatTitle.ts b/src/services/chatTitle.ts index aa86e34..d574cbc 100644 --- a/src/services/chatTitle.ts +++ b/src/services/chatTitle.ts @@ -1,4 +1,6 @@ import { openclawWs } from './openclawWs'; +import { complete, openRouterEnabled } from './openRouter'; +import { loadOpenRouterConfig } from './config'; export const TITLE_LIMIT = 60; /** @@ -87,7 +89,39 @@ function modelToAgentId(model: string): string { return model.startsWith('openclaw/') ? model.slice('openclaw/'.length) : model; } -export async function suggestChatTitle(opts: { +/** + * Cheap single-shot title via OpenRouter. Returns null on failure or a rejected + * suggestion so the caller can fall back to the OpenClaw path. + */ +async function suggestChatTitleViaOpenRouter(userMessage: string): Promise<string | null> { + try { + const cfg = loadOpenRouterConfig(); + const text = await complete({ + model: cfg.titleModel, + messages: [{ role: 'user', content: buildTitlePrompt(userMessage) }], + temperature: 0.3, + maxTokens: 32, + }); + const cleaned = normalizeSuggestedTitle(text); + if (!cleaned) { + console.warn('[chatTitle] openrouter rejected suggestion:', JSON.stringify(text.slice(0, 120))); + return null; + } + return cleaned; + } catch (err) { + console.warn( + '[chatTitle] openrouter failed, will fall back to OpenClaw:', + err instanceof Error ? err.message : err, + ); + return null; + } +} + +/** + * Title via a throw-away OpenClaw agent session — the original path, kept as + * the fallback when OpenRouter is unconfigured (or its call failed). + */ +async function suggestChatTitleViaOpenClaw(opts: { model: string; userMessage: string; }): Promise<string | null> { @@ -125,6 +159,23 @@ export async function suggestChatTitle(opts: { } } +/** + * Suggest a chat title. Prefers OpenRouter (cheap, no agent run) when a key is + * configured; falls back to a throw-away OpenClaw session when OpenRouter is + * unconfigured or its call fails/returns an unusable suggestion. + */ +export async function suggestChatTitle(opts: { + model: string; + userMessage: string; +}): Promise<string | null> { + if (openRouterEnabled()) { + const viaOpenRouter = await suggestChatTitleViaOpenRouter(opts.userMessage); + if (viaOpenRouter) return viaOpenRouter; + // null → OpenRouter unusable this time; fall through to OpenClaw. + } + return suggestChatTitleViaOpenClaw(opts); +} + /** Race title generation against a hard budget; null on timeout. */ export async function suggestChatTitleWithTimeout( opts: Parameters<typeof suggestChatTitle>[0], diff --git a/src/services/colima.ts b/src/services/colima.ts new file mode 100644 index 0000000..e44f124 --- /dev/null +++ b/src/services/colima.ts @@ -0,0 +1,103 @@ +/** + * Colima container engine (macOS). + * + * On macOS, iClaw runs its Docker workloads on Colima — a free, lightweight, + * CLI-driven Docker engine in a small Linux VM — instead of Docker Desktop. + * A non-technical user never has to think about it: iClaw installs Colima, + * starts it on demand the first moment a task needs a container, and stops it + * again when idle. + * + * Colima can arrive two ways (see scripts/install-docker.sh): via Homebrew when + * it's present, or — on a clean Mac with no Homebrew — by downloading pinned + * colima/lima/docker binaries into ~/.iclaw/engine. Either way iClaw owns a + * DEDICATED Colima profile ("iclaw") — its own VM — so it never disturbs the + * user's other containers/profiles, and it pins its own `docker` CLI calls to + * that profile's context via DOCKER_CONTEXT, leaving the GLOBAL context alone. + * + * Off macOS this module is dormant: callers gate every use on `isMac` (Linux: + * user-managed dockerd; Windows: Docker Desktop). + */ +import { execFile } from 'node:child_process'; +import { existsSync } from 'node:fs'; +import { homedir, platform as osPlatform } from 'node:os'; +import { delimiter, join } from 'node:path'; +import { promisify } from 'node:util'; + +const execFileAsync = promisify(execFile); + +/** True on macOS, where iClaw uses Colima as its container engine. */ +export const isMac = osPlatform() === 'darwin'; + +/** Dedicated Colima profile iClaw owns (its own VM — never the user's other work). */ +export const COLIMA_PROFILE = 'iclaw'; +/** Docker context Colima creates for that profile (`colima start iclaw`). */ +export const COLIMA_CONTEXT = `colima-${COLIMA_PROFILE}`; + +/** Where the no-brew installer drops the downloaded engine binaries (colima, limactl, docker). */ +export const ENGINE_BIN = join( + process.env.ICLAW_ENGINE_DIR || join(homedir(), '.iclaw', 'engine'), + 'bin', +); + +/** Generous ceiling for a first `colima start` (downloads the guest image + boots). */ +const START_TIMEOUT = 300_000; +const PROBE_TIMEOUT = 8_000; +const STOP_TIMEOUT = 60_000; + +/** Add a dir to PATH (front = higher priority) if it exists and isn't already there. */ +function addToPath(dir: string, front: boolean): void { + if (!existsSync(dir)) return; + const parts = (process.env.PATH ?? '').split(delimiter).filter(Boolean); + if (parts.includes(dir)) return; + process.env.PATH = front + ? `${dir}${delimiter}${parts.join(delimiter)}` + : `${parts.join(delimiter)}${delimiter}${dir}`; +} + +/** + * Set up this process's (and its children's) environment for iClaw's Colima + * engine on macOS: + * - ensure Homebrew's bin dirs are on PATH (a GUI-launched .app starts with a + * minimal PATH that omits them, so a brew-installed colima/docker wouldn't be + * found otherwise), and + * - put our own downloaded engine bin (~/.iclaw/engine/bin) FIRST, so a no-brew + * install wins, and + * - pin every `docker` call to iClaw's Colima context, leaving the user's GLOBAL + * docker context untouched. + * Idempotent; honours an explicit DOCKER_CONTEXT already in the environment. + * No-op off macOS. Safe to call from every process entry point — and again right + * after an install, to pick up freshly downloaded binaries. + */ +export function ensureColimaEnv(): void { + if (!isMac) return; + addToPath('/opt/homebrew/bin', false); + addToPath('/usr/local/bin', false); + addToPath(ENGINE_BIN, true); // our pinned binaries take priority + if (!process.env.DOCKER_CONTEXT) { + process.env.DOCKER_CONTEXT = COLIMA_CONTEXT; + } +} + +/** Colima CLI present on PATH (the engine is installed). */ +export async function colimaInstalled(): Promise<boolean> { + try { + await execFileAsync('colima', ['version'], { timeout: PROBE_TIMEOUT }); + return true; + } catch { + return false; + } +} + +/** + * Start iClaw's Colima VM (creating it on first run). Blocks until the daemon is + * ready; idempotent — "already running" is a success. First run can take ~1–2 min + * (guest image download); subsequent starts ~20s. + */ +export async function colimaStart(): Promise<void> { + await execFileAsync('colima', ['start', COLIMA_PROFILE], { timeout: START_TIMEOUT }); +} + +/** Stop iClaw's Colima VM (only ours — never the user's other profiles). */ +export async function colimaStop(): Promise<void> { + await execFileAsync('colima', ['stop', COLIMA_PROFILE], { timeout: STOP_TIMEOUT }); +} diff --git a/src/services/config.ts b/src/services/config.ts index b1556a9..226d207 100644 --- a/src/services/config.ts +++ b/src/services/config.ts @@ -1,6 +1,7 @@ import { readFileSync, existsSync } from 'node:fs'; import { homedir } from 'node:os'; import { join } from 'node:path'; +import { kvGet, kvSet, kvDelete } from '../db/kv'; export interface OpenClawConfig { baseUrl: string; @@ -44,6 +45,90 @@ export function loadCloudShareBaseUrl(): string { return base; } +export interface OpenRouterConfig { + /** Empty when unconfigured — gates Ask/STT and title routing. */ + apiKey: string; + baseUrl: string; + /** Model for Ask turns. */ + askModel: string; + /** Model for chat-title generation (cheapest sensible default). */ + titleModel: string; + /** Cheap model for context compaction (summarizing old turns). */ + summaryModel: string; + /** Multimodal model used for speech-to-text transcription. */ + sttModel: string; + /** OpenRouter app-attribution headers (optional, for rankings). */ + referer: string; + appTitle: string; +} + +/** KV key under which the user's OpenRouter API key is stored (set in Settings). */ +const OPENROUTER_API_KEY_KV = 'openrouter.api_key'; + +/** + * Direct OpenRouter access for the tool-less features (Ask, titles, STT). + * + * The API key is entered by the user in Settings and stored in the local DB + * (`iclaw_kv`) — NOT an env var. Models default to a cheap, fast, multimodal + * flash model; advanced users can still override per-feature via the optional + * `ICLAW_*_MODEL` env vars (undocumented). + */ +export function loadOpenRouterConfig(): OpenRouterConfig { + const apiKey = (kvGet(OPENROUTER_API_KEY_KV) ?? '').trim(); + const baseUrl = ( + process.env.OPENROUTER_BASE_URL?.trim() || 'https://openrouter.ai/api/v1' + ).replace(/\/+$/, ''); + const askModel = process.env.ICLAW_ASK_MODEL?.trim() || 'google/gemini-2.5-flash'; + const titleModel = process.env.ICLAW_TITLE_MODEL?.trim() || 'google/gemini-2.5-flash'; + // Cheap/fast model for compaction; overridable. Falls back to truncation if + // the call fails, so an invalid slug degrades gracefully. + const summaryModel = process.env.ICLAW_SUMMARY_MODEL?.trim() || 'google/gemini-2.5-flash-lite'; + const sttModel = process.env.ICLAW_STT_MODEL?.trim() || 'google/gemini-2.5-flash'; + const referer = process.env.OPENROUTER_REFERER?.trim() || 'https://iclaw.digital'; + const appTitle = process.env.OPENROUTER_APP_TITLE?.trim() || 'iClaw'; + return { apiKey, baseUrl, askModel, titleModel, summaryModel, sttModel, referer, appTitle }; +} + +/** Persist the user's OpenRouter API key (from Settings). Empty/blank clears it. */ +export function setOpenRouterApiKey(key: string): void { + const trimmed = key.trim(); + if (trimmed) kvSet(OPENROUTER_API_KEY_KV, trimmed); + else kvDelete(OPENROUTER_API_KEY_KV); +} + +/** Remove the stored OpenRouter API key (disconnect). */ +export function clearOpenRouterApiKey(): void { + kvDelete(OPENROUTER_API_KEY_KV); +} + +/** KV key recording that the user has been through (or skipped) the welcome flow. */ +const ONBOARDING_DONE_KV = 'onboarding.done'; + +/** + * Has the first-run welcome flow been completed (or explicitly skipped)? + * This is the sole gate for showing /welcome — independent of whether a key is + * set, so a user can dismiss the screen and still configure things later. + */ +export function isOnboardingDone(): boolean { + return kvGet(ONBOARDING_DONE_KV) === '1'; +} + +/** Mark the welcome flow complete so it never shows again (unless reset). */ +export function setOnboardingDone(): void { + kvSet(ONBOARDING_DONE_KV, '1'); +} + +/** + * A privacy-preserving display form of the stored key: keeps the `sk-or-` + * prefix and the last 4 chars, masks the middle. Empty string when unset. + */ +export function maskOpenRouterApiKey(): string { + const key = (kvGet(OPENROUTER_API_KEY_KV) ?? '').trim(); + if (!key) return ''; + if (key.length <= 12) return '••••' + key.slice(-2); + return key.slice(0, 7) + '••••••••' + key.slice(-4); +} + export function loadOpenClawConfig(): OpenClawConfig { const envToken = process.env.OPENCLAW_API_KEY?.trim(); const envUrl = process.env.OPENCLAW_BASE_URL?.trim(); diff --git a/src/services/contextCompaction.ts b/src/services/contextCompaction.ts new file mode 100644 index 0000000..2c36b89 --- /dev/null +++ b/src/services/contextCompaction.ts @@ -0,0 +1,140 @@ +/** + * Context compaction — "never clear the chat, just compress old turns". + * + * Each chat's full transcript lives permanently in the `messages` table. For + * the model's context window we build a bounded view: + * + * [rolling summary of older turns] + [last N turns verbatim] + [new message] + * + * The summary is produced by a cheap model (config.summaryModel) and cached in + * `chat_summaries`, rebuilt incrementally as the chat grows. Old context is + * therefore compressed, never dropped — full detail is always recoverable from + * the DB. Used by Ask directly, and to seed Work/Secure runtime sessions. + */ +import { db } from '../db/database'; +import { messages } from './store'; +import { loadOpenRouterConfig } from './config'; +import { complete, type OpenRouterMessage } from './openRouter'; + +/** Keep this many most-recent messages verbatim; older ones get summarized. */ +const RECENT_VERBATIM_COUNT = 16; +/** Don't compact until there are clearly "old" turns beyond the verbatim window. */ +const MIN_OLDER_TO_SUMMARIZE = 4; +/** Clip each old message to this many chars when feeding the summarizer. */ +const SUMMARIZE_PER_MSG_CHARS = 2000; +/** Cap the generated summary length. */ +const SUMMARY_MAX_TOKENS = 900; + +interface SummaryRow { + up_to_message_id: number; + summary: string; +} + +function getSummaryRow(chatId: number): SummaryRow | undefined { + return db + .prepare('SELECT up_to_message_id, summary FROM chat_summaries WHERE chat_id = ?') + .get(chatId) as SummaryRow | undefined; +} + +function upsertSummaryRow(chatId: number, upToId: number, summary: string): void { + db.prepare( + `INSERT INTO chat_summaries (chat_id, up_to_message_id, summary, updated_at) + VALUES (?, ?, ?, datetime('now')) + ON CONFLICT(chat_id) DO UPDATE SET + up_to_message_id = excluded.up_to_message_id, + summary = excluded.summary, + updated_at = excluded.updated_at`, + ).run(chatId, upToId, summary); +} + +interface TurnMsg { id: number; role: string; content: string } + +function toModelMsg(m: TurnMsg): OpenRouterMessage { + return { role: m.role === 'assistant' ? 'assistant' : 'user', content: m.content.replace(/\s+$/, '') }; +} + +function formatForSummary(msgs: TurnMsg[]): string { + return msgs + .map((m) => { + const who = m.role === 'assistant' ? 'Assistant' : 'User'; + let text = m.content.trim(); + if (text.length > SUMMARIZE_PER_MSG_CHARS) text = text.slice(0, SUMMARIZE_PER_MSG_CHARS) + '…'; + return `${who}: ${text}`; + }) + .join('\n\n'); +} + +const SUMMARY_SYSTEM_PROMPT = [ + 'You compress conversation history into a concise, information-dense summary.', + 'Preserve: facts, user preferences and decisions, established context, names,', + 'numbers, file/work state, and any open threads or unfinished tasks.', + 'Merge the existing summary with the new messages into one updated summary.', + 'Do not add commentary, greetings, or meta text — output only the summary.', +].join(' '); + +/** + * Produce (or extend) the summary covering `older`. Incremental: re-summarizes + * only the messages added since the cached summary. Best-effort — returns the + * previous (possibly stale) summary, or null, if the model call fails. + */ +async function getOrBuildSummary(chatId: number, older: TurnMsg[], boundaryId: number): Promise<string | null> { + const row = getSummaryRow(chatId); + if (row && row.up_to_message_id >= boundaryId) return row.summary; + + const prevSummary = row?.summary ?? ''; + const newOlder = row ? older.filter((m) => m.id > row.up_to_message_id) : older; + if (newOlder.length === 0) return prevSummary || null; + + const cfg = loadOpenRouterConfig(); + if (!cfg.apiKey) return prevSummary || null; + + try { + const summary = (await complete({ + model: cfg.summaryModel, + temperature: 0, + maxTokens: SUMMARY_MAX_TOKENS, + messages: [ + { role: 'system', content: SUMMARY_SYSTEM_PROMPT }, + { + role: 'user', + content: + `Existing summary:\n${prevSummary || '(none yet)'}\n\n` + + `New messages to fold in:\n${formatForSummary(newOlder)}\n\n` + + 'Return the updated summary.', + }, + ], + })).trim(); + if (!summary) return prevSummary || null; + upsertSummaryRow(chatId, boundaryId, summary); + return summary; + } catch { + return prevSummary || null; // degrade: keep stale summary rather than fail the turn + } +} + +/** + * Build a compacted history for a chat: messages with id < beforeMsgId, as + * [summary?] + [recent verbatim]. Small chats are returned verbatim (no call). + */ +export async function buildCompactedHistory(chatId: number, beforeMsgId: number): Promise<OpenRouterMessage[]> { + const all = messages + .listByChat(chatId) + .filter((m) => m.id < beforeMsgId && (m.role === 'user' || m.role === 'assistant') && m.content.trim()) + .map((m) => ({ id: m.id, role: m.role, content: m.content })); + + if (all.length <= RECENT_VERBATIM_COUNT + MIN_OLDER_TO_SUMMARIZE) { + return all.map(toModelMsg); + } + + const recent = all.slice(-RECENT_VERBATIM_COUNT); + const older = all.slice(0, -RECENT_VERBATIM_COUNT); + const boundaryId = older[older.length - 1].id; + + const summary = await getOrBuildSummary(chatId, older, boundaryId); + const out: OpenRouterMessage[] = []; + if (summary) { + out.push({ role: 'system', content: `Summary of earlier conversation (compacted):\n${summary}` }); + } + for (const m of recent) out.push(toModelMsg(m)); + return out; +} diff --git a/src/services/docker.ts b/src/services/docker.ts new file mode 100644 index 0000000..6c2f70f --- /dev/null +++ b/src/services/docker.ts @@ -0,0 +1,190 @@ +/** + * Docker readiness + lifecycle for the chat UI. + * + * Work mode's file tools run on the host (no Docker), but `run_command` and the + * whole Safe-work sandbox need a running daemon. This service backs the chat + * composer's Docker gate: a cached status probe plus best-effort "start" and + * "install" actions the user can trigger from the Install/Start button. + * + * It never blocks: the action endpoints kick the work off in the background and + * flip the cached state to `starting`/`installing`, which the composer polls + * via GET /api/docker/status until it settles on `ready` (or back to + * `stopped`/`missing` if it didn't take). + * + * Mirrors the onboarding probe in onboardingEnv.ts, but that one is a one-shot + * welcome-screen state machine; this is the live, re-pollable runtime service. + */ + +import { execFile } from 'node:child_process'; +import { platform as osPlatform } from 'node:os'; +import path from 'node:path'; +import { promisify } from 'node:util'; + +import { colimaInstalled, colimaStart, ensureColimaEnv, isMac } from './colima'; + +const execFileAsync = promisify(execFile); + +// macOS: put colima/docker on PATH and pin every `docker` call to iClaw's Colima +// VM (not the user's global context / Docker Desktop). No-op off macOS. +ensureColimaEnv(); + +export type DockerState = + | 'ready' // daemon reachable (`docker info` ok) + | 'stopped' // binary present, daemon down + | 'missing' // binary not installed + | 'starting' // we're launching the daemon + | 'installing'; // we're installing the engine + +/** + * Human-readable footprint shown on the Install button. macOS installs Colima + * (a lightweight VM engine, no Docker Desktop); other platforms install Docker. + */ +export const DOCKER_SIZE_HINT = isMac + ? '~600 MB download · ~2 GB on disk · no Docker Desktop' + : '~600 MB download · ~4 GB on disk'; + +const PROBE_TIMEOUT = 8_000; +const CACHE_MS = 4_000; +const POLL_ATTEMPTS = 30; +const POLL_INTERVAL = 2_000; +const INSTALL_TIMEOUT = 600_000; + +let cached: DockerState = 'missing'; +let cachedAt = 0; +let probeInFlight: Promise<DockerState> | null = null; +/** Single-flight for start/install — only one host action runs at a time. */ +let actionInFlight: Promise<DockerState> | null = null; + +function sleep(ms: number): Promise<void> { + return new Promise((r) => setTimeout(r, ms)); +} + +/** Daemon reachable (`docker info` exits 0). */ +async function reachable(): Promise<boolean> { + try { + await execFileAsync('docker', ['info'], { timeout: PROBE_TIMEOUT }); + return true; + } catch { + return false; + } +} + +/** Engine installed: Colima on macOS, the Docker CLI elsewhere (daemon may be down). */ +async function installed(): Promise<boolean> { + if (isMac) return colimaInstalled(); + try { + await execFileAsync('docker', ['--version'], { timeout: PROBE_TIMEOUT }); + return true; + } catch { + return false; + } +} + +async function probe(): Promise<DockerState> { + if (await reachable()) return 'ready'; + if (await installed()) return 'stopped'; + return 'missing'; +} + +/** Re-probe and cache the settled state (ready/stopped/missing). */ +async function settle(): Promise<DockerState> { + cached = await probe(); + cachedAt = Date.now(); + return cached; +} + +/** + * Cached status. Never overrides an in-flight action (`starting`/`installing`) + * and dedupes concurrent probes. Refreshes once the cache is stale. + */ +export async function getDockerState(): Promise<DockerState> { + if (cached === 'starting' || cached === 'installing') return cached; + if (Date.now() - cachedAt < CACHE_MS) return cached; + if (!probeInFlight) { + probeInFlight = settle().finally(() => { + probeInFlight = null; + }); + } + return probeInFlight; +} + +/** Best-effort daemon launch for the current platform. */ +async function launchDaemon(): Promise<void> { + const plat = osPlatform(); + try { + if (plat === 'darwin') { + // macOS engine is Colima, not Docker Desktop: start iClaw's own VM. This + // blocks until the daemon is ready (idempotent if already up), so the + // poll below is just a fast confirmation. + await colimaStart(); + } else if (plat === 'win32') { + await execFileAsync('cmd', ['/c', 'start', '', 'Docker Desktop'], { timeout: PROBE_TIMEOUT }); + } + // Linux: starting dockerd needs sudo, which can't prompt from this web + // context — leave it to the user. The poll below still picks it up if they + // start it manually. + } catch { + /* engine launch failed (not installed / boot error) — the poll reports the real state. */ + } +} + +/** Poll until the daemon accepts connections (a cold Colima/Docker start can take 20–40s). */ +async function pollReady(): Promise<boolean> { + for (let i = 0; i < POLL_ATTEMPTS; i++) { + await sleep(POLL_INTERVAL); + if (await reachable()) return true; + } + return false; +} + +/** + * Start an installed-but-idle Docker, in the background. Returns immediately + * with `starting`; the cached state settles to `ready`/`stopped` once the poll + * resolves. Single-flight: a concurrent call rides the same action. + */ +export function startDocker(): DockerState { + if (!actionInFlight) { + cached = 'starting'; + cachedAt = Date.now(); + actionInFlight = (async () => { + await launchDaemon(); + await pollReady(); + return settle(); + })().finally(() => { + actionInFlight = null; + }); + } + return cached; +} + +const INSTALL_SCRIPT = path.resolve(__dirname, '../../scripts/install-docker.sh'); + +/** + * Install the container engine — macOS: Colima (Homebrew when present, else + * downloaded binaries in ~/.iclaw/engine); Linux: get.docker.com — then start it, + * all in the background. Returns immediately with `installing`; the cached state + * settles once the install + start + poll resolve. + */ +export function installDocker(): DockerState { + if (!actionInFlight) { + cached = 'installing'; + cachedAt = Date.now(); + actionInFlight = (async () => { + try { + await execFileAsync('bash', [INSTALL_SCRIPT], { timeout: INSTALL_TIMEOUT }); + } catch { + /* Install failed (no brew + no network, offline, declined sudo) — settle() + reports whether anything landed; the UI keeps the Install button. */ + } + // A no-brew install drops colima/docker into ~/.iclaw/engine/bin — re-run so + // that dir is on PATH before we try to launch the daemon. + ensureColimaEnv(); + await launchDaemon(); + await pollReady(); + return settle(); + })().finally(() => { + actionInFlight = null; + }); + } + return cached; +} diff --git a/src/services/onboardingEnv.ts b/src/services/onboardingEnv.ts new file mode 100644 index 0000000..179dd38 --- /dev/null +++ b/src/services/onboardingEnv.ts @@ -0,0 +1,157 @@ +/** + * First-run environment detection for the welcome flow. + * + * The onboarding screen shows an HONEST background progress line while the user + * reads the welcome copy / pastes their OpenRouter key. We do NOT install + * anything here — the container engine (Colima on macOS) and the OpenClaw + * gateway are installed elsewhere (the composer's Install button / the user). + * What we CAN do without the user noticing: + * - probe whether Docker is reachable (`docker info`), + * - if it is and the Work/Safe sandbox base image is missing, pre-pull it in + * the background (the one genuinely slow step we can hide behind the key + * entry), and + * - probe whether the OpenClaw gateway is reachable (Full Power). + * + * Everything is best-effort and cached — the screen polls `getOnboardingEnv()` + * for a status line, never blocking the actual "Continue" action. + */ + +import { execFile } from 'node:child_process'; +import { promisify } from 'node:util'; + +import { colimaInstalled, ensureColimaEnv, isMac } from './colima'; +import { probeGateway } from './gatewayProbe'; + +const execFileAsync = promisify(execFile); + +// macOS: probe Colima (iClaw's engine) — put it on PATH and route docker calls to +// its context. +ensureColimaEnv(); + +// Mirror work-container.ts's image resolution so onboarding agrees with what +// Work/Safe actually use. The canonical image is the prebuilt sandbox; the +// public base is only a last-resort fallback we pre-pull when NOTHING usable is +// present yet. If the user already has either, there is nothing to download. +const SANDBOX_IMAGE = process.env.ICLAW_SECURE_IMAGE || 'iclaw-secure:latest'; +/** Base image Work/Safe modes fall back to — public, so `docker pull` works on a fresh box. */ +const FALLBACK_IMAGE = process.env.ICLAW_WORK_SLIM_IMAGE || 'node:22'; + +type DockerState = + | 'unknown' // not probed yet + | 'missing' // Docker binary not installed + | 'stopped' // installed but daemon not running (and we couldn't start it) + | 'starting' // installed, daemon down — we're trying to start it + | 'pulling' // daemon up, downloading the sandbox image + | 'ready'; // daemon up and an image is available + +export interface OnboardingEnv { + /** Docker engine + sandbox image readiness (gates Work / Safe work). */ + docker: DockerState; + /** OpenClaw gateway reachable (gates Full Power / Execute). */ + openclaw: 'unknown' | 'connected' | 'off'; +} + +let state: OnboardingEnv = { docker: 'unknown', openclaw: 'unknown' }; +let prepStarted = false; + +/** Daemon reachable (`docker info` exits 0). */ +async function dockerReachable(): Promise<boolean> { + try { + await execFileAsync('docker', ['info'], { timeout: 8_000 }); + return true; + } catch { + return false; + } +} + +/** Engine installed: Colima on macOS, the Docker CLI elsewhere (daemon may be down). */ +async function dockerInstalled(): Promise<boolean> { + if (isMac) return colimaInstalled(); + try { + await execFileAsync('docker', ['--version'], { timeout: 8_000 }); + return true; + } catch { + return false; + } +} + +async function imageExists(image: string): Promise<boolean> { + try { + await execFileAsync('docker', ['image', 'inspect', image], { timeout: 8_000 }); + return true; + } catch { + return false; + } +} + +/** Pull the fallback base image in the background, flipping docker → ready when done. */ +function pullImageInBackground(image: string): void { + state.docker = 'pulling'; + // 10 min ceiling — a slow connection on a multi-hundred-MB image. + execFileAsync('docker', ['pull', image], { timeout: 600_000 }) + .then(() => { + state.docker = 'ready'; + }) + .catch(() => { + // Pull failed (offline, disk, daemon hiccup). `docker run` still + // auto-pulls on first real use, so this is non-fatal — leave it as + // 'pulling' is misleading, so report 'ready' (Docker is installed; the + // image just arrives lazily later). + state.docker = 'ready'; + }); +} + +/** + * Daemon is up: mark ready if a usable image is already present, otherwise + * pre-pull the public fallback in the background. + */ +async function ensureImage(): Promise<void> { + if ((await imageExists(SANDBOX_IMAGE)) || (await imageExists(FALLBACK_IMAGE))) { + state.docker = 'ready'; + } else { + pullImageInBackground(FALLBACK_IMAGE); + } +} + +/** + * Kick off detection + the (slow) image pre-pull exactly once. Safe to call on + * every /welcome render — it no-ops after the first call. + */ +export function startOnboardingPrep(): void { + if (prepStarted) return; + prepStarted = true; + + void (async () => { + // 1. Daemon already up (the user started it)? Pre-pull the sandbox image so + // the first Work/Safe use is fast. + if (await dockerReachable()) { + await ensureImage(); + return; + } + // 2. Installed but idle → DON'T start it here. Onboarding is informational; + // the runtime starts Docker on demand the moment a task needs it + // (docker-lifecycle.ts) and auto-stops it when idle. Starting it here + // instead would be a transparent start the runtime can't track → it would + // never get auto-stopped. So we just report the state. + if (await dockerInstalled()) { + state.docker = 'stopped'; + return; + } + // 3. Not installed at all. + state.docker = 'missing'; + })(); + + void (async () => { + try { + const { gatewayStatus } = await probeGateway('onboarding'); + state.openclaw = gatewayStatus === 'ok' ? 'connected' : 'off'; + } catch { + state.openclaw = 'off'; + } + })(); +} + +/** Current best-effort environment snapshot for the welcome progress line. */ +export function getOnboardingEnv(): OnboardingEnv { + return { ...state }; +} diff --git a/src/services/openRouter.ts b/src/services/openRouter.ts new file mode 100644 index 0000000..b31a13f --- /dev/null +++ b/src/services/openRouter.ts @@ -0,0 +1,370 @@ +/** + * Direct OpenRouter client — the FIRST non-OpenClaw LLM path in iClaw. + * + * Used for the lightweight, tool-less features that should NOT spin up a full + * OpenClaw agent run: + * - Chat titles → a cheap single-shot completion. + * - Background sub-tasks (fact extraction, fact compaction, skill review) via + * services/subtaskLlm.ts (preferred over a throwaway OpenClaw turn). + * - Speech-to-text → audio transcription via a multimodal model. + * + * Talks the OpenAI-compatible `/chat/completions` endpoint OpenRouter exposes. + * Streaming responses are parsed here (server-sent events over `fetch`) and the + * caller re-emits deltas through `wsHub` — the same shape `openclawWs.runTurn` + * uses, so the browser stream renderer is identical for both backends. + * + * No SDK / extra deps: Node 18+ (we run 25) ships global `fetch` + streams. + * + * Availability is config-driven: when `OPENROUTER_API_KEY` is unset, the + * features that REQUIRE OpenRouter (Ask, STT) are hidden/refused, and title + * generation falls back to the OpenClaw path. See `loadOpenRouterConfig`. + */ + +import { loadOpenRouterConfig } from './config'; + +export interface OpenRouterMessage { + role: 'system' | 'user' | 'assistant'; + content: string; +} + +/** True when an API key is configured — gates Ask/STT and title routing. */ +export function openRouterEnabled(): boolean { + return Boolean(loadOpenRouterConfig().apiKey); +} + +/** + * Failures from the OpenRouter bridge are tagged with this prefix so callers + * (chatRunner) can distinguish them from OpenClaw gateway failures and surface + * a sensible user message instead of leaking raw text. + */ +function fail(message: string): never { + throw new Error(`openrouter: ${message}`); +} + +export function isOpenRouterFailure(err: unknown): boolean { + const t = (err instanceof Error ? err.message : String(err)).trim(); + return /^openrouter:/i.test(t); +} + +function buildHeaders(apiKey: string, referer: string, appTitle: string): Record<string, string> { + const h: Record<string, string> = { + Authorization: `Bearer ${apiKey}`, + 'Content-Type': 'application/json', + }; + // OpenRouter uses these for app attribution / rankings. Optional, harmless. + if (referer) h['HTTP-Referer'] = referer; + if (appTitle) h['X-Title'] = appTitle; + return h; +} + +interface ChatCompletionOpts { + messages: OpenRouterMessage[]; + /** Defaults to the configured Ask model. */ + model?: string; + temperature?: number; + maxTokens?: number; + /** Abort the in-flight request (Stop button). */ + signal?: AbortSignal; +} + +/** + * Non-streaming completion. Returns the full assistant text. Used for titles + * and any short single-shot call. + */ +export async function complete(opts: ChatCompletionOpts): Promise<string> { + const cfg = loadOpenRouterConfig(); + if (!cfg.apiKey) fail('no API key configured (set OPENROUTER_API_KEY)'); + const model = opts.model ?? cfg.askModel; + + let res: Response; + try { + res = await fetch(`${cfg.baseUrl}/chat/completions`, { + method: 'POST', + headers: buildHeaders(cfg.apiKey, cfg.referer, cfg.appTitle), + body: JSON.stringify({ + model, + messages: opts.messages, + stream: false, + thinking: { type: 'disabled' }, + ...(opts.temperature != null ? { temperature: opts.temperature } : {}), + ...(opts.maxTokens != null ? { max_tokens: opts.maxTokens } : {}), + }), + signal: opts.signal, + }); + } catch (err) { + fail(`request failed: ${err instanceof Error ? err.message : String(err)}`); + } + if (!res.ok) fail(`HTTP ${res.status}: ${(await res.text().catch(() => '')).slice(0, 300)}`); + + const json = (await res.json().catch(() => null)) as { + choices?: Array<{ message?: { content?: string } }>; + } | null; + return json?.choices?.[0]?.message?.content ?? ''; +} + +interface StreamCompletionOpts extends ChatCompletionOpts { + /** Fires for every text delta as it streams in. */ + onDelta: (text: string) => void; +} + +/** + * Streaming completion. Calls `onDelta` for each token chunk and resolves with + * the full accumulated text. Parses OpenRouter's SSE stream (`data: {json}` + * lines, terminated by `data: [DONE]`). + */ +export async function streamComplete(opts: StreamCompletionOpts): Promise<string> { + const cfg = loadOpenRouterConfig(); + if (!cfg.apiKey) fail('no API key configured (set OPENROUTER_API_KEY)'); + const model = opts.model ?? cfg.askModel; + + let res: Response; + try { + res = await fetch(`${cfg.baseUrl}/chat/completions`, { + method: 'POST', + headers: buildHeaders(cfg.apiKey, cfg.referer, cfg.appTitle), + body: JSON.stringify({ + model, + messages: opts.messages, + stream: true, + // Disable extended thinking — Ask mode is for quick answers, not deep reasoning + thinking: { type: 'disabled' }, + ...(opts.temperature != null ? { temperature: opts.temperature } : {}), + ...(opts.maxTokens != null ? { max_tokens: opts.maxTokens } : {}), + }), + signal: opts.signal, + }); + } catch (err) { + fail(`request failed: ${err instanceof Error ? err.message : String(err)}`); + } + if (!res.ok) fail(`HTTP ${res.status}: ${(await res.text().catch(() => '')).slice(0, 300)}`); + if (!res.body) fail('no response body to stream'); + + const reader = res.body.getReader(); + const decoder = new TextDecoder(); + let buffer = ''; + let full = ''; + + // SSE frames are separated by a blank line. We accumulate across chunks + // because a single network chunk can split a `data:` line mid-JSON. + const consumeLine = (line: string): boolean => { + const trimmed = line.trim(); + if (!trimmed.startsWith('data:')) return false; + const payload = trimmed.slice('data:'.length).trim(); + if (payload === '[DONE]') return true; + try { + const json = JSON.parse(payload) as { + choices?: Array<{ delta?: { content?: string } }>; + }; + const delta = json.choices?.[0]?.delta?.content; + if (delta) { + full += delta; + opts.onDelta(delta); + } + } catch { + // OpenRouter sends `: OPENROUTER PROCESSING` keep-alive comments and the + // occasional partial line — ignore anything that isn't valid JSON. + } + return false; + }; + + try { + for (;;) { + const { done, value } = await reader.read(); + if (done) break; + buffer += decoder.decode(value, { stream: true }); + let nl: number; + while ((nl = buffer.indexOf('\n')) !== -1) { + const line = buffer.slice(0, nl); + buffer = buffer.slice(nl + 1); + if (consumeLine(line)) { + await reader.cancel().catch(() => {}); + return full; + } + } + } + // Flush any trailing buffered line (stream ended without final newline). + if (buffer.trim()) consumeLine(buffer); + } finally { + reader.releaseLock?.(); + } + return full; +} + +/** + * Transcribe audio to text via a multimodal model. Sends the clip as an + * `input_audio` content part to /chat/completions and asks for a verbatim + * transcript. Returns the transcript (possibly empty). Throws `openrouter:` on + * failure. + * + * NOTE: accepted audio formats depend on the model. Gemini-class models take + * common containers, but the browser's MediaRecorder output varies (webm/opus + * on Chrome, mp4/aac on Safari). If a model rejects the upload, point + * `ICLAW_STT_MODEL` at one that accepts that container. + */ +export async function transcribeAudio(opts: { + audioBase64: string; + /** Container hint for the model: "webm" | "mp3" | "wav" | "m4a" | "ogg" | … */ + format: string; + model?: string; + signal?: AbortSignal; +}): Promise<string> { + const cfg = loadOpenRouterConfig(); + if (!cfg.apiKey) fail('no API key configured (set OPENROUTER_API_KEY)'); + const model = opts.model ?? cfg.sttModel; + + let res: Response; + try { + res = await fetch(`${cfg.baseUrl}/chat/completions`, { + method: 'POST', + headers: buildHeaders(cfg.apiKey, cfg.referer, cfg.appTitle), + body: JSON.stringify({ + model, + stream: false, + messages: [ + { + role: 'user', + content: [ + { + type: 'text', + text: + 'Transcribe this audio verbatim. Output ONLY the transcript text — ' + + 'no preamble, quotes, or commentary. If there is no speech, output nothing.', + }, + { type: 'input_audio', input_audio: { data: opts.audioBase64, format: opts.format } }, + ], + }, + ], + }), + signal: opts.signal, + }); + } catch (err) { + fail(`request failed: ${err instanceof Error ? err.message : String(err)}`); + } + if (!res.ok) fail(`HTTP ${res.status}: ${(await res.text().catch(() => '')).slice(0, 300)}`); + + const json = (await res.json().catch(() => null)) as { + choices?: Array<{ message?: { content?: string } }>; + } | null; + return (json?.choices?.[0]?.message?.content ?? '').trim(); +} + +export interface KeyValidation { + /** The key authenticates against OpenRouter. */ + valid: boolean; + /** USD remaining if known; null when unlimited/unknown. */ + remaining: number | null; + /** Optional key label from OpenRouter. */ + label?: string; + /** Machine-readable reason when invalid: 'empty' | 'unauthorized' | 'http' | 'network'. */ + reason?: string; +} + +/** + * Validate an OpenRouter key WITHOUT storing it — used by onboarding/Settings to + * catch a dead, mistyped, or zero-balance key before it silently breaks the + * first chat. Hits `/auth/key` (cheap, no completion spend) with the candidate + * key and reports validity + remaining credit. Network failures are reported as + * `valid: false, reason: 'network'` so the UI can say "couldn't verify" rather + * than wrongly rejecting a good key. + */ +export async function validateKey(key: string, signal?: AbortSignal): Promise<KeyValidation> { + const trimmed = key.trim(); + if (!trimmed) return { valid: false, remaining: null, reason: 'empty' }; + + const cfg = loadOpenRouterConfig(); + const headers = buildHeaders(trimmed, cfg.referer, cfg.appTitle); + + let res: Response; + try { + res = await fetch(`${cfg.baseUrl}/auth/key`, { headers, signal }); + } catch { + return { valid: false, remaining: null, reason: 'network' }; + } + if (res.status === 401 || res.status === 403) { + return { valid: false, remaining: null, reason: 'unauthorized' }; + } + if (!res.ok) return { valid: false, remaining: null, reason: 'http' }; + + const j = (await res.json().catch(() => null)) as { + data?: { label?: string; usage?: number; limit?: number | null; limit_remaining?: number | null }; + } | null; + const d = j?.data ?? {}; + const usage = Number(d.usage ?? 0); + const limit = d.limit == null ? null : Number(d.limit); + const remaining = + d.limit_remaining == null ? (limit != null ? Math.max(limit - usage, 0) : null) : Number(d.limit_remaining); + return { valid: true, remaining, label: d.label }; +} + +export interface OpenRouterUsage { + /** USD spent so far on this key/account. */ + usage: number; + /** USD credit limit, or null when there is none. */ + limit: number | null; + /** USD remaining, or null when unknown/unlimited. */ + remaining: number | null; + /** Optional key label from OpenRouter. */ + label?: string; + isFreeTier?: boolean; +} + +/** + * Fetch spend/credit info for the configured key, for the Settings usage + * readout. Tries `/credits` (purchased credits + total usage) first, then + * falls back to `/auth/key` (usage + limit + label). Throws `openrouter:` on + * failure so the caller can show a friendly "couldn't load usage" note. + */ +export async function fetchUsage(signal?: AbortSignal): Promise<OpenRouterUsage> { + const cfg = loadOpenRouterConfig(); + if (!cfg.apiKey) fail('no API key configured (add it in Settings)'); + const headers = buildHeaders(cfg.apiKey, cfg.referer, cfg.appTitle); + + // Preferred: /credits → { data: { total_credits, total_usage } }. + try { + const res = await fetch(`${cfg.baseUrl}/credits`, { headers, signal }); + if (res.ok) { + const j = (await res.json().catch(() => null)) as { + data?: { total_credits?: number; total_usage?: number }; + } | null; + const d = j?.data; + if (d && (typeof d.total_usage === 'number' || typeof d.total_credits === 'number')) { + const usage = Number(d.total_usage ?? 0); + const credits = Number(d.total_credits ?? 0); + const limit = credits > 0 ? credits : null; + const remaining = limit != null ? Math.max(limit - usage, 0) : null; + return { usage, limit, remaining }; + } + } + } catch (err) { + if (signal?.aborted) fail('aborted'); + // fall through to /auth/key + } + + // Fallback: /auth/key → { data: { label, usage, limit, limit_remaining, is_free_tier } }. + let res: Response; + try { + res = await fetch(`${cfg.baseUrl}/auth/key`, { headers, signal }); + } catch (err) { + fail(`request failed: ${err instanceof Error ? err.message : String(err)}`); + } + if (!res.ok) fail(`HTTP ${res.status}: ${(await res.text().catch(() => '')).slice(0, 200)}`); + const j = (await res.json().catch(() => null)) as { + data?: { + label?: string; + usage?: number; + limit?: number | null; + limit_remaining?: number | null; + is_free_tier?: boolean; + }; + } | null; + const d = j?.data ?? {}; + const usage = Number(d.usage ?? 0); + const limit = d.limit == null ? null : Number(d.limit); + const remaining = + d.limit_remaining == null + ? limit != null + ? Math.max(limit - usage, 0) + : null + : Number(d.limit_remaining); + return { usage, limit, remaining, label: d.label, isFreeTier: d.is_free_tier }; +} diff --git a/src/services/openclawInstall.ts b/src/services/openclawInstall.ts new file mode 100644 index 0000000..f27d3ef --- /dev/null +++ b/src/services/openclawInstall.ts @@ -0,0 +1,42 @@ +import { execFile } from 'node:child_process'; +import { promisify } from 'node:util'; + +const execFileAsync = promisify(execFile); + +/** + * Whether the `openclaw` CLI is installed on this device. + * + * Probed once at startup and cached — `openclaw --version` is a subprocess, far + * too heavy for the per-render callers (e.g. defaultComposerMode). Distinct from + * "gateway reachable": OpenClaw can be installed but stopped, and Full Power + * starts it on demand — so the default-mode choice keys off INSTALL, not health. + * + * Defaults to `false` until the probe resolves, so a device without OpenClaw + * never momentarily defaults into a dead Full Power. A machine that HAS it is + * primed at startup (and onboarding's human-time gap covers any race). + */ +let installed = false; +let probed = false; + +export async function probeOpenClawInstalled(): Promise<boolean> { + try { + // Not installed → spawn rejects immediately (ENOENT); a present binary + // returns fast. The timeout only guards a pathological hang. + await execFileAsync('openclaw', ['--version'], { timeout: 3_000 }); + installed = true; + } catch { + installed = false; + } + probed = true; + return installed; +} + +/** Cached result of the startup probe. `false` until {@link probeOpenClawInstalled} runs. */ +export function isOpenClawInstalled(): boolean { + return installed; +} + +/** True once the startup probe has completed at least once. */ +export function openClawInstallProbed(): boolean { + return probed; +} diff --git a/src/services/openclawWs.ts b/src/services/openclawWs.ts index a0c0d83..1e3ff96 100644 --- a/src/services/openclawWs.ts +++ b/src/services/openclawWs.ts @@ -14,10 +14,17 @@ import { gatewayWs, type RawGatewayFrame } from './gatewayWs'; import { toolActivityLabel, lifecycleActivityLabel } from './toolLabels'; import { extractAssistantText, + extractTurnUsage, resolveFromHistorySlice, sliceFromLastUser, } from './turnReply'; +/** Token usage for one turn, resolved from the gateway history slice. */ +export interface TurnUsage { + tokens: number | null; + cached: number | null; +} + // ---------- shapes --------------------------------------------------------- export interface OpenClawAgent { @@ -39,6 +46,8 @@ export interface HistoryMessage { timestamp?: number; toolName?: string; isError?: boolean; + /** Per-message token usage (assistant rows) — powers the dev-mode badge. */ + usage?: unknown; } export type TurnEvent = @@ -54,8 +63,6 @@ export type TurnEvent = | { type: 'tool-end'; name: string; itemId?: string } | { type: 'lifecycle'; phase: string; label: string } | { type: 'attachment'; url: string; mime: string; label?: string; itemId?: string } - /** Model reasoning / analysis text — only emitted, chatRunner decides whether to surface. */ - | { type: 'reasoning'; text: string } | { type: 'text-final'; text: string }; // ---------- helpers -------------------------------------------------------- @@ -121,7 +128,9 @@ const HISTORY_FETCH_LIMIT = 1000; const HISTORY_FETCH_RETRIES = 3; const HISTORY_FETCH_RETRY_DELAY_MS = 150; -async function fetchAuthoritativeFromHistory(sessionKey: string): Promise<string | null> { +async function fetchAuthoritativeFromHistory( + sessionKey: string, +): Promise<{ text: string | null; usage: TurnUsage }> { let lastErr: unknown; for (let attempt = 0; attempt < HISTORY_FETCH_RETRIES; attempt++) { try { @@ -129,8 +138,12 @@ async function fetchAuthoritativeFromHistory(sessionKey: string): Promise<string 'chat.history', { sessionKey, limit: HISTORY_FETCH_LIMIT }, ); - const resolved = resolveFromHistorySlice(sliceFromLastUser(res.messages ?? [])); - return resolved.trim().length > 0 ? resolved : null; + const slice = sliceFromLastUser(res.messages ?? []); + const resolved = resolveFromHistorySlice(slice); + return { + text: resolved.trim().length > 0 ? resolved : null, + usage: extractTurnUsage(slice), + }; } catch (err) { lastErr = err; if (attempt < HISTORY_FETCH_RETRIES - 1) { @@ -144,7 +157,7 @@ async function fetchAuthoritativeFromHistory(sessionKey: string): Promise<string 'attempts:', lastErr instanceof Error ? lastErr.message : String(lastErr), ); - return null; + return { text: null, usage: { tokens: null, cached: null } }; } function pickAuthoritativeReplyText(opts: { @@ -195,12 +208,38 @@ const pendingAbortIntents = new Set<string>(); // ---------- public client ------------------------------------------------- +/** Short cache for agent-existence lookups so Ask routing doesn't hit agents.list every turn. */ +const AGENT_EXISTS_TTL_MS = 30_000; +const agentExistsCache = new Map<string, { at: number; exists: boolean }>(); + export const openclawWs = { async listAgents(): Promise<OpenClawAgent[]> { const res = await gatewayWs.request<{ agents: OpenClawAgent[] }>('agents.list', {}); return res.agents ?? []; }, + /** + * Whether an agent with this id is configured on the gateway. Cached for a + * few seconds. Returns false on any lookup failure (caller falls back to a + * non-agent path rather than erroring). Empty id is always false. + */ + async agentExists(agentId: string): Promise<boolean> { + const id = agentId.trim(); + if (!id) return false; + const hit = agentExistsCache.get(id); + const now = Date.now(); + if (hit && now - hit.at < AGENT_EXISTS_TTL_MS) return hit.exists; + try { + const agents = await this.listAgents(); + const exists = agents.some((a) => a.id === id); + agentExistsCache.set(id, { at: now, exists }); + return exists; + } catch { + // Gateway unreachable / RPC failed — don't cache, treat as absent. + return false; + } + }, + /** Create a new "dashboard" session for an agent. Empty params → default agent. */ async createSession(opts: { agentId?: string } = {}): Promise<OpenClawSession> { const params: Record<string, unknown> = {}; @@ -216,6 +255,25 @@ export const openclawWs = { await gatewayWs.request('sessions.delete', { key }); }, + /** + * Append a message into a session's transcript WITHOUT running the model + * (`chat.inject`). The gateway records it as an assistant note (zero cost, + * `model: "gateway-injected"`) and broadcasts a final `chat` event. Used to + * make the main Execute session aware of an out-of-band Ask exchange. + */ + async injectMessage(opts: { + sessionKey: string; + message: string; + label?: string; + }): Promise<void> { + if (!opts.sessionKey.startsWith('agent:')) return; + await gatewayWs.request('chat.inject', { + sessionKey: opts.sessionKey, + message: opts.message, + ...(opts.label ? { label: opts.label } : {}), + }); + }, + /** Fetch the canonical, UI-normalized transcript. */ async getHistory(sessionKey: string, limit = 200): Promise<HistoryMessage[]> { const res = await gatewayWs.request<{ messages: HistoryMessage[] }>('chat.history', { @@ -248,27 +306,6 @@ export const openclawWs = { return gatewayWs.request('commands.list', opts as Record<string, unknown>); }, - /** - * Patch a session's per-turn defaults (reasoning, model, thinking, etc.). - * Only the fields you pass are touched; the gateway leaves the rest alone. - * No-op when the session key isn't a real `agent:...` one yet. - */ - async patchSession(opts: { - sessionKey: string; - reasoningLevel?: string | null; - model?: string | null; - thinkingLevel?: string | null; - fastMode?: boolean | null; - }): Promise<void> { - if (!opts.sessionKey.startsWith('agent:')) return; - const params: Record<string, unknown> = { key: opts.sessionKey }; - if (opts.reasoningLevel !== undefined) params.reasoningLevel = opts.reasoningLevel; - if (opts.model !== undefined) params.model = opts.model; - if (opts.thinkingLevel !== undefined) params.thinkingLevel = opts.thinkingLevel; - if (opts.fastMode !== undefined) params.fastMode = opts.fastMode; - await gatewayWs.request('sessions.patch', params); - }, - /** Read the gateway's current full config (incl. `hash` needed for config.patch). */ async getConfig(): Promise<{ hash: string; config: Record<string, unknown> }> { const res = await gatewayWs.request<{ @@ -395,6 +432,12 @@ export const openclawWs = { * `null` on abort or when nothing usable was found. */ authoritativeText: string | null; + /** + * Token usage for this turn, summed across the assistant rows of the + * `chat.history` slice. `{ null, null }` on abort or when the gateway + * reported no usage (powers the dev-mode token badge — see chatRunner). + */ + usage: TurnUsage; }> { let runId: string | null = null; let accumulatedText = ''; @@ -516,19 +559,8 @@ export const openclawWs = { if (stream === 'item') { const kind = safeString(data.kind); - if (kind === 'analysis') { - // Reasoning / chain-of-thought. We always emit; chatRunner gates - // delivery to subscribers based on per-chat reasoning_mode. - const phase = data.phase; - if (phase === 'start' || phase === 'end' || phase === 'completed') return; - const text = - safeString(data.text) ?? - safeString(data.deltaText) ?? - safeString(data.content) ?? - ''; - if (text) opts.onEvent({ type: 'reasoning', text }); - return; - } + // Reasoning / chain-of-thought items are dropped (Reasoning feature removed). + if (kind === 'analysis') return; const name = safeString(data.name) ?? kind ?? 'tool'; const phase = data.phase; const itemId = safeString(data.itemId); @@ -716,10 +748,12 @@ export const openclawWs = { // Canonical reply: transcript/history first, then last `chat:final`, then stream. let authoritativeText: string | null = null; + let usage: TurnUsage = { tokens: null, cached: null }; if (!wasAborted) { const fromHistory = await fetchAuthoritativeFromHistory(opts.sessionKey); + usage = fromHistory.usage; authoritativeText = pickAuthoritativeReplyText({ - fromHistory, + fromHistory: fromHistory.text, fromTranscript: transcriptAssistantText, fromLastFinal: lastFinalText || accumulatedText, }); @@ -730,6 +764,7 @@ export const openclawWs = { text: lastFinalText || accumulatedText, aborted: wasAborted, authoritativeText, + usage, }; }, }; diff --git a/src/services/projectMemory.ts b/src/services/projectMemory.ts index 8c1c487..b082494 100644 --- a/src/services/projectMemory.ts +++ b/src/services/projectMemory.ts @@ -3,8 +3,9 @@ * facts after each turn (when enabled), and compact when the fact table grows. */ -import { openclawWs } from './openclawWs'; import { projectFacts, projects, chats, projectFactSuggestions, enrichFactsWithSourceChatTitles } from './store'; +import { buildSkillsPromptBlock } from './projectSkills'; +import { runSubtaskTurn } from './subtaskLlm'; import { wsHub } from './wsHub'; /** Rough token budget for the prepended project block (80/20 vs full window). */ @@ -19,14 +20,6 @@ const MAX_FACT_LINE_CHARS = 240; const compactingProjects = new Set<number>(); -/** - * Sub-tasks (extract + compact) always run against the gateway's default - * agent. Using whatever agent the chat is on (`openclaw/code`, etc.) made the - * two pipelines inconsistent and risked specialised agents underperforming on - * a plain text-extraction prompt. One agent, one behaviour. - */ -const SUBTASK_AGENT_ID = 'main'; - function approxTokens(s: string): number { return Math.ceil(s.length / CHARS_PER_TOKEN_EST); } @@ -55,11 +48,20 @@ export function buildGatewayUserMessage( ): string { if (!projects.get(projectId)) return storedUserContent; + const skillsBlock = buildSkillsPromptBlock(projectId); const all = projectFacts.listByProject(projectId, 200); - if (all.length === 0) return storedUserContent; + // Nothing to inject — neither facts nor skills. + if (all.length === 0 && !skillsBlock) return storedUserContent; + + const skillsPrefix = skillsBlock ? skillsBlock + '\n\n' : ''; + if (all.length === 0) { + // Skills-only injection (no facts yet). + return `[Project context for this workspace. Treat as background only; respond in direct conversation to the user (their message is after the separator).]\n${skillsPrefix}---\n[User message]\n${storedUserContent}`; + } const prefixBase = - '[Project context — shared facts for this workspace. Treat as background only; respond in direct conversation to the user (their message is after the separator).]\n'; + '[Project context — shared facts for this workspace. Treat as background only; respond in direct conversation to the user (their message is after the separator).]\n' + + skillsPrefix; const suffix = '\n---\n[User message]\n' + storedUserContent; const budget = Math.max( 200, @@ -159,33 +161,13 @@ function buildExtractPrompt(opts: { ].join('\n'); } -async function runThrowawayTurn(message: string): Promise<string> { - let sessionKey: string | null = null; - try { - const session = await openclawWs.createSession({ agentId: SUBTASK_AGENT_ID }); - sessionKey = session.key; - let acc = ''; - await openclawWs.runTurn({ - sessionKey: session.key, - message, - onEvent: (ev) => { - if (ev.type === 'text-delta') acc += ev.text; - else if (ev.type === 'text-final') acc = ev.text || acc; - }, - }); - return acc; - } finally { - if (sessionKey) openclawWs.deleteSession(sessionKey).catch(() => {}); - } -} - export async function extractFactsFromTurn(opts: { userMessage: string; assistantText: string; existingFacts: string[]; }): Promise<string[]> { const raw = await Promise.race([ - runThrowawayTurn(buildExtractPrompt(opts)), + runSubtaskTurn(buildExtractPrompt(opts), { maxTokens: 512 }), sleep<string>(EXTRACT_BUDGET_MS, ''), ]); if (raw == null || raw === '') return []; @@ -214,7 +196,7 @@ export async function compactProjectFacts(projectId: number): Promise<void> { if (contents.length <= FACT_COMPACTION_THRESHOLD) return; const raw = await Promise.race([ - runThrowawayTurn(buildCompactionPrompt(contents)), + runSubtaskTurn(buildCompactionPrompt(contents), { maxTokens: 1024 }), sleep<string>(EXTRACT_BUDGET_MS, ''), ]); if (raw == null || raw === '') return; diff --git a/src/services/projectSkills.ts b/src/services/projectSkills.ts new file mode 100644 index 0000000..8ebfc4a --- /dev/null +++ b/src/services/projectSkills.ts @@ -0,0 +1,413 @@ +/** + * Per-project procedural memory (the "skills" half of memory). After a chat + * turn, a throttled background reviewer distills reusable *procedural skills* + * from what just happened. Distilled skills do NOT activate automatically — + * they land in a per-project inbox as suggestions (project_skill_suggestions). + * The user accepts/edits/rejects them, exactly like project fact suggestions. + * + * This mirrors src/services/projectMemory.ts (the declarative "facts" half) and + * layers Hermes's distillation discipline (class-level skills, patch-over-new) + * plus the SKILL.md format on top, with a human-acceptance gate. + * + * SECURITY: the reviewer only ever emits inbox suggestions — it never writes an + * active skill. Untrusted turns (web/email/Telegram ingestion, secure+network) + * are flagged so the UI can warn the user. See docs/project-skills-spec.md §10. + */ + +import { + projectSkills, + projectSkillSuggestions, + projects, + chats, +} from './store'; +import type { ProjectSkillIndexRow } from './store'; +import { runSubtaskTurn } from './subtaskLlm'; +import { wsHub } from './wsHub'; + +/** Local copy (mirrors projectMemory.normalizeForDedup) — avoids a circular import. */ +function normalizeForDedup(s: string): string { + return s.trim().toLowerCase().replace(/\s+/g, ' '); +} + +/** Rough token budget for the prepended skills index block. */ +const CHARS_PER_TOKEN_EST = 4; +const SKILLS_INDEX_TARGET_TOKENS = 600; +const MAX_INDEX_LINES = 30; +const MAX_DESC_CHARS = 200; +/** MVP shortcut (§7b): inline the full bodies of up to N small skills. */ +const INLINE_BODY_COUNT = 3; +const INLINE_BODY_MAX_CHARS = 1200; +/** + * Default review cadence (Ask/Execute). Work/Secure turns are agentic and + * information-rich, so they pass a smaller interval (see WORK_REVIEW_INTERVAL + * in chatRunner) and get reviewed more often. + */ +const SKILL_REVIEW_TURN_INTERVAL = 8; +const REVIEW_BUDGET_MS = 90_000; +/** Max suggestions accepted from a single review pass. */ +const MAX_SUGGESTIONS_PER_REVIEW = 3; + +/** Per-chat counter of substantive (tool-using) turns since the last review. */ +const turnCounters = new Map<number, number>(); +/** Chats with a review in flight — never run two concurrently for one chat. */ +const reviewingChats = new Set<number>(); + +function approxTokens(s: string): number { + return Math.ceil(s.length / CHARS_PER_TOKEN_EST); +} + +function sleep<T = null>(ms: number, value: T | null = null): Promise<T | null> { + return new Promise((resolve) => setTimeout(() => resolve(value), ms)); +} + +/** + * Build the "Skills available" block injected into the system prompt (Work / + * Secure) or the gateway user message (Execute). Returns '' when the project + * has no active skills. Index lines are token-bounded like facts; the bodies of + * up to INLINE_BODY_COUNT small, most-recently-updated skills are inlined as an + * MVP shortcut for `get_skill` (spec §7b). + */ +export function buildSkillsPromptBlock(projectId: number): string { + if (!projects.get(projectId)) return ''; + const skills = projectSkills.listForProject(projectId); + if (skills.length === 0) return ''; + + const header = 'Skills available (procedural memory for this project):'; + let used = approxTokens(header); + const lines: string[] = []; + for (const s of skills) { + if (lines.length >= MAX_INDEX_LINES) break; + const desc = + s.description.length > MAX_DESC_CHARS + ? s.description.slice(0, MAX_DESC_CHARS - 1) + '…' + : s.description; + const line = `- ${s.name}: ${desc}`; + const t = approxTokens(line + '\n'); + if (used + t > SKILLS_INDEX_TARGET_TOKENS && lines.length > 0) break; + used += t; + lines.push(line); + } + + // Inline the bodies of a few small skills so the agent can follow them now, + // without a round-trip. Larger skills stay index-only (request on demand). + const inlined: string[] = []; + let inlinedCount = 0; + for (const s of skills) { + if (inlinedCount >= INLINE_BODY_COUNT) break; + if (s.body.length > INLINE_BODY_MAX_CHARS) continue; + inlined.push(`\n--- skill: ${s.name} ---\n${s.body.trim()}`); + inlinedCount++; + } + + const parts = [header, ...lines]; + if (inlined.length > 0) { + parts.push( + '\nFull procedures for the most relevant skills follow. For any other skill above, ask the user or apply the one-line summary.', + ...inlined, + ); + } else if (lines.length > 0) { + parts.push('\nApply a skill when its summary matches the task at hand.'); + } + return parts.join('\n'); +} + +// --------------------------------------------------------------------------- +// Reviewer +// --------------------------------------------------------------------------- + +interface ReviewedSkill { + action: 'new' | 'patch'; + target?: string; + name: string; + description: string; + tags?: string[]; + body: string; +} + +/** Extract the first balanced JSON object from a possibly fenced model output. */ +export function extractJsonObject(raw: string): string | null { + const t = raw.trim(); + if (!t || /^none\.?$/i.test(t)) return null; + const start = t.indexOf('{'); + if (start < 0) return null; + let depth = 0; + let inStr = false; + let esc = false; + for (let i = start; i < t.length; i++) { + const ch = t[i]; + if (inStr) { + if (esc) esc = false; + else if (ch === '\\') esc = true; + else if (ch === '"') inStr = false; + continue; + } + if (ch === '"') inStr = true; + else if (ch === '{') depth++; + else if (ch === '}') { + depth--; + if (depth === 0) return t.slice(start, i + 1); + } + } + return null; +} + +const KEBAB_RE = /^[a-z0-9]+(?:-[a-z0-9]+)*$/; + +function toKebab(name: string): string { + return name + .trim() + .toLowerCase() + .replace(/[^a-z0-9]+/g, '-') + .replace(/^-+|-+$/g, '') + .slice(0, 64); +} + +/** Parse + sanitize the reviewer JSON into well-formed skill candidates. */ +export function parseReviewedSkills(raw: string): ReviewedSkill[] { + const json = extractJsonObject(raw); + if (!json) return []; + let parsed: unknown; + try { + parsed = JSON.parse(json); + } catch { + return []; + } + const arr = (parsed as { skills?: unknown })?.skills; + if (!Array.isArray(arr)) return []; + const out: ReviewedSkill[] = []; + for (const item of arr) { + if (!item || typeof item !== 'object') continue; + const o = item as Record<string, unknown>; + const name = typeof o.name === 'string' ? toKebab(o.name) : ''; + const description = typeof o.description === 'string' ? o.description.trim() : ''; + const body = typeof o.body === 'string' ? o.body.trim() : ''; + if (!name || !KEBAB_RE.test(name) || !description || !body) continue; + const action = o.action === 'patch' ? 'patch' : 'new'; + const target = typeof o.target === 'string' ? toKebab(o.target) : undefined; + const tags = Array.isArray(o.tags) + ? o.tags.filter((t): t is string => typeof t === 'string').slice(0, 8) + : undefined; + out.push({ action, target, name, description, tags, body }); + if (out.length >= MAX_SUGGESTIONS_PER_REVIEW) break; + } + return out; +} + +function buildReviewPrompt(opts: { + transcript: string; + existingIndex: ProjectSkillIndexRow[]; +}): string { + const existingBlock = + opts.existingIndex.length > 0 + ? opts.existingIndex.map((s) => `- ${s.name}: ${s.description}`).join('\n') + : '(none yet)'; + return [ + 'TASK: From the conversation below, distill 0–3 reusable PROCEDURAL skills', + 'that would help an agent do similar work next time in this project.', + '', + 'A skill captures a procedure / convention / workflow discovered this session:', + 'commands that worked, project conventions, tool quirks, gotchas, repeatable', + 'multi-step processes. It is CLASS-LEVEL and reusable — NOT a one-shot task log.', + '', + 'Strongly PREFER patching an existing skill (action "patch", set "target" to', + 'its name) over creating a near-duplicate. Only create a "new" skill for a', + 'genuinely distinct capability. Most sessions produce zero or one skill update.', + '', + 'Hard skip (never emit): transient task state ("user is debugging X"),', + 'secrets/tokens/credentials/.env values, SSH/IPs/hostnames, user-local', + 'filesystem paths, one-off facts (those belong to project facts, not skills).', + '', + 'Existing skills (patch these instead of duplicating):', + existingBlock, + '', + 'Conversation:', + opts.transcript.slice(0, 16000), + '', + 'The "description" is read by a NON-TECHNICAL person on a save prompt (and also', + 'helps the agent pick this skill later). Write it as ONE short, plain sentence', + 'in everyday words: name the real-world situation and what it achieves. Avoid', + 'jargon — no "sandbox", "runtime", "container", "API", "stdout", "kebab", CLI', + 'flags, or file paths. Concrete and friendly, not a technical summary.', + 'Good: "How to finish a task on your own computer when the safe workspace has', + 'no internet." Bad: "Provides code to run locally when sandbox network is', + 'restricted." Keep the procedure detail in "body", not the description.', + '', + 'OUTPUT: strict JSON, no prose, no code fences:', + '{"skills":[{"action":"new"|"patch","target":"<existing-name, patch only>",', + '"name":"<kebab-case>","description":"<one plain-language sentence, no jargon>","tags":["..."],', + '"body":"<full SKILL.md markdown: frontmatter + procedure sections>"}]}', + 'If nothing qualifies, output exactly: {"skills":[]}', + 'Use the same language as the technical content.', + ].join('\n'); +} + +/** Build a compact transcript of the recent turn for the reviewer. */ +function buildTranscript(userMessage: string, assistantText: string): string { + const user = userMessage.replace(/\s+/g, ' ').trim().slice(0, 6000); + const assistant = assistantText.trim().slice(0, 12000); + return `User:\n${user}\n\nAssistant:\n${assistant}`; +} + +async function runProjectSkillReview(opts: { + chatId: number; + projectId: number; + userMessage: string; + assistantText: string; + assistantMessageId: number; + untrusted: boolean; +}): Promise<void> { + const chat = chats.get(opts.chatId); + if (!chat || chat.project_id !== opts.projectId) return; + if (!chat.shares_to_project) return; + if (!projects.get(opts.projectId)) return; + if (reviewingChats.has(opts.chatId)) return; + + reviewingChats.add(opts.chatId); + try { + const existingIndex = projectSkills.listIndex(opts.projectId); + const raw = await Promise.race([ + runSubtaskTurn( + buildReviewPrompt({ + transcript: buildTranscript(opts.userMessage, opts.assistantText), + existingIndex, + }), + { maxTokens: 4096 }, + ), + sleep<string>(REVIEW_BUDGET_MS, ''), + ]); + if (raw == null || raw === '') return; + + const reviewed = parseReviewedSkills(raw); + if (reviewed.length === 0) return; + + // Dedup pool: active skill names/descriptions + pending suggestion names. + const activeNames = new Set(existingIndex.map((s) => normalizeForDedup(s.name))); + const activeDescs = existingIndex.map((s) => normalizeForDedup(s.description)); + const pending = projectSkillSuggestions.listByProject(opts.projectId); + const pendingNames = new Set(pending.map((s) => normalizeForDedup(s.name))); + + const inserted: { + id: number; + kind: 'new' | 'patch'; + name: string; + description: string; + untrusted: boolean; + targetSkillId: number | null; + }[] = []; + + for (const r of reviewed) { + const normName = normalizeForDedup(r.name); + const normDesc = normalizeForDedup(r.description); + + // Resolve patch target; downgrade to 'new' if the target is unknown. + let kind: 'new' | 'patch' = r.action; + let targetSkillId: number | null = null; + if (kind === 'patch') { + const targetName = r.target || r.name; + const target = + projectSkills.getByName(opts.projectId, targetName) ?? + projectSkills.getByName(null, targetName); + if (target) { + targetSkillId = target.id; + } else { + kind = 'new'; + } + } + + // For 'new', skip if a same-named active/pending skill already exists, or + // if the description duplicates an existing skill's summary. + if (kind === 'new') { + if (activeNames.has(normName) || pendingNames.has(normName)) continue; + if (activeDescs.some((d) => d && (d === normDesc))) continue; + } + + const row = projectSkillSuggestions.insert({ + projectId: opts.projectId, + chatId: opts.chatId, + kind, + targetSkillId, + name: r.name, + description: r.description, + body: r.body, + tags: r.tags ?? null, + untrusted: opts.untrusted, + assistantMessageId: opts.assistantMessageId, + }); + pendingNames.add(normName); + inserted.push({ + id: row.id, + kind: row.kind, + name: row.name, + description: row.description, + untrusted: Boolean(row.untrusted), + targetSkillId: row.target_skill_id, + }); + } + + if (inserted.length > 0) { + const proj = projects.get(opts.projectId); + wsHub.broadcastAll({ + type: 'project-skill-suggestions', + chatId: opts.chatId, + projectId: opts.projectId, + projectName: proj?.name?.trim() || 'project', + suggestions: inserted, + }); + } + } catch (err) { + console.error( + '[projectSkills] review pipeline failed', + err instanceof Error ? err.message : err, + ); + } finally { + reviewingChats.delete(opts.chatId); + } +} + +/** + * Fire-and-forget skill review after a successful assistant reply. Throttled: + * only runs once every SKILL_REVIEW_TURN_INTERVAL substantive turns per chat, + * because skill review is heavier than fact extraction. Never blocks the turn. + */ +export function scheduleProjectSkillReview(opts: { + chatId: number; + projectId: number; + sharesToProject: boolean; + substantive: boolean; + userMessage: string; + assistantText: string; + assistantMessageId: number; + untrusted: boolean; + /** Turns between reviews for this chat. Defaults to the Ask/Execute cadence. */ + interval?: number; +}): void { + if (!opts.sharesToProject) return; + if (!opts.substantive) return; + + const interval = opts.interval && opts.interval > 0 ? opts.interval : SKILL_REVIEW_TURN_INTERVAL; + const next = (turnCounters.get(opts.chatId) ?? 0) + 1; + if (next < interval) { + turnCounters.set(opts.chatId, next); + return; + } + turnCounters.set(opts.chatId, 0); + + void runProjectSkillReview({ + chatId: opts.chatId, + projectId: opts.projectId, + userMessage: opts.userMessage, + assistantText: opts.assistantText, + assistantMessageId: opts.assistantMessageId, + untrusted: opts.untrusted, + }).catch((err) => { + console.error( + '[projectSkills] review failed', + err instanceof Error ? err.message : err, + ); + }); +} + +/** Test/maintenance helper: reset the in-memory turn counter for a chat. */ +export function resetSkillReviewCounter(chatId: number): void { + turnCounters.delete(chatId); +} diff --git a/src/services/remoteAccessE2eTransport.ts b/src/services/remoteAccessE2eTransport.ts index fdd6d4a..b1eb1bd 100644 --- a/src/services/remoteAccessE2eTransport.ts +++ b/src/services/remoteAccessE2eTransport.ts @@ -168,16 +168,6 @@ export function isE2ePlaintextTunnelExempt(ctx: E2ePlaintextTunnelContext): bool return false; } -/** @deprecated Use isE2ePlaintextTunnelExempt */ -export function isE2ePlaintextExemptPath( - path: string, - method = 'GET', - tunnelId = '', - cookieHeader?: string, -): boolean { - return isE2ePlaintextTunnelExempt({ method, path, tunnelId, cookieHeader }); -} - function decryptInbound( session: E2eTransportSession, wireRaw: string, diff --git a/src/services/runtimeProcess.ts b/src/services/runtimeProcess.ts new file mode 100644 index 0000000..b9ebf22 --- /dev/null +++ b/src/services/runtimeProcess.ts @@ -0,0 +1,111 @@ +/** + * Manages the iclaw-runtime sidecar process lifecycle. + * + * Spawns packages/iclaw-runtime/src/index.ts via tsx when Work Mode is + * needed. Restarts automatically on crash (up to MAX_RESTARTS times). + * Shuts down cleanly with the main iClaw process. + */ +import { spawn, type ChildProcess } from 'node:child_process'; +import path from 'node:path'; +import fs from 'node:fs'; +import { kvGet } from '../db/kv'; + +const RUNTIME_DIR = path.resolve(__dirname, '../../packages/iclaw-runtime'); +const RUNTIME_ENTRY = path.join(RUNTIME_DIR, 'src/index.ts'); +const MAX_RESTARTS = 5; +const RESTART_WINDOW_MS = 60_000; + +let child: ChildProcess | null = null; +let restartCount = 0; +let windowStart = Date.now(); +let stopping = false; + +function runtimeInstalled(): boolean { + return fs.existsSync(RUNTIME_ENTRY); +} + +function spawnRuntime(): void { + if (stopping || !runtimeInstalled()) return; + + // Pass OpenRouter key from iClaw's DB into the runtime environment + const openRouterKey = (kvGet('openrouter.api_key') ?? '').trim(); + const runtimeEnv: NodeJS.ProcessEnv = { + ...process.env, + ...(openRouterKey ? { ICLAW_OPENROUTER_API_KEY: openRouterKey } : {}), + }; + + child = spawn('npx', ['tsx', RUNTIME_ENTRY], { + cwd: RUNTIME_DIR, + stdio: ['ignore', 'pipe', 'pipe'], + env: runtimeEnv, + }); + + child.stdout?.on('data', (chunk: Buffer) => { + process.stdout.write(`[iclaw-runtime] ${chunk}`); + }); + child.stderr?.on('data', (chunk: Buffer) => { + process.stderr.write(`[iclaw-runtime] ${chunk}`); + }); + + child.on('exit', (code, signal) => { + child = null; + if (stopping) return; + + const now = Date.now(); + if (now - windowStart > RESTART_WINDOW_MS) { + restartCount = 0; + windowStart = now; + } + restartCount++; + + if (restartCount > MAX_RESTARTS) { + console.error( + `[iclaw-runtime] crashed ${MAX_RESTARTS} times in ${RESTART_WINDOW_MS / 1000}s — giving up`, + ); + return; + } + + const delay = Math.min(1000 * restartCount, 10_000); + console.warn( + `[iclaw-runtime] exited (code=${code ?? signal}) — restarting in ${delay}ms (attempt ${restartCount}/${MAX_RESTARTS})`, + ); + setTimeout(spawnRuntime, delay).unref(); + }); +} + +export const runtimeProcess = { + start(): void { + if (!runtimeInstalled()) return; + stopping = false; + spawnRuntime(); + }, + + stop(): void { + stopping = true; + if (child) { + child.kill('SIGTERM'); + child = null; + } + }, + + /** + * Restart so the runtime picks up a changed OpenRouter key (the key is read + * from the DB at spawn and passed via env — a running runtime can't see a key + * added/changed later, e.g. during onboarding). Safe to call even if the + * runtime isn't running yet: it just starts it. + */ + restart(): void { + if (!runtimeInstalled()) return; + this.stop(); + // Brief gap so port 7430 frees before respawn (the runtime also self-retries + // on a busy port, so this is just to avoid the noisy "port in use" log). + setTimeout(() => { + stopping = false; + spawnRuntime(); + }, 500).unref(); + }, + + get running(): boolean { + return child !== null && !stopping; + }, +}; diff --git a/src/services/store.ts b/src/services/store.ts index dbcc5fa..445adc8 100644 --- a/src/services/store.ts +++ b/src/services/store.ts @@ -4,11 +4,14 @@ import { deriveTitle } from './chatTitle'; import type { Chat, ChatKind, + ChatMode, Message, MessageAttachment, Project, ProjectFact, ProjectFactSuggestion, + ProjectSkill, + ProjectSkillSuggestion, ProjectSecret, QueuedMessage, ScheduledMessage, @@ -24,6 +27,7 @@ import type { TaskWithSteps, } from '../types'; import type { InlineSecretWire } from './inlineSecrets'; +import { DEFAULT_MODE } from './chatModes'; import { clampLogoColor, clampLogoEmoji } from '../constants/projectLogos'; // ---------- chats ---------- @@ -129,9 +133,10 @@ export const chats = { "UPDATE chats SET shares_to_project = ?, updated_at = datetime('now') WHERE id = ?", ).run(shares ? 1 : 0, id); }, - setReasoningMode(id: number, mode: 'off' | 'on' | 'stream'): void { + /** Persist the chat's sticky composer send-mode (see chats.mode). */ + setChatMode(id: number, mode: string): void { db.prepare( - "UPDATE chats SET reasoning_mode = ?, updated_at = datetime('now') WHERE id = ?", + "UPDATE chats SET mode = ?, updated_at = datetime('now') WHERE id = ?", ).run(mode, id); }, touch(id: number): void { @@ -210,6 +215,12 @@ export const messages = { finishReason: string | null = null, reply?: { replyToMessageId: number; replyQuote: string; replyToRole: string } | null, attachments?: MessageAttachment[] | null, + /** Send mode for user rows. Defaults to 'execute' (back-compat). */ + mode: ChatMode = DEFAULT_MODE, + /** Total tokens spent producing this message (dev-mode; assistant rows). */ + tokens: number | null = null, + /** Of `tokens`, prompt tokens served from cache (dev-mode). */ + cachedTokens: number | null = null, ): Message { const rid = reply?.replyToMessageId ?? null; const rq = reply?.replyQuote ?? null; @@ -218,9 +229,9 @@ export const messages = { attachments && attachments.length > 0 ? JSON.stringify(attachments) : null; const info = db .prepare( - 'INSERT INTO messages (chat_id, role, content, finish_reason, reply_to_message_id, reply_quote, reply_to_role, attachments) VALUES (?, ?, ?, ?, ?, ?, ?, ?)', + 'INSERT INTO messages (chat_id, role, content, finish_reason, reply_to_message_id, reply_quote, reply_to_role, attachments, mode, tokens, cached_tokens) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)', ) - .run(chatId, role, content, finishReason, rid, rq, rrole, att); + .run(chatId, role, content, finishReason, rid, rq, rrole, att, mode, tokens, cachedTokens); // chats.updated_at is bumped by the trg_chats_touch_on_message SQLite // trigger; no manual touch() needed here. We keep chats.touch() public // for callers that mutate parents without writing a message (e.g. @@ -479,6 +490,197 @@ export const projectFactSuggestions = { }, }; +// ---------- project skills (procedural memory; SKILL.md) ---------- + +/** name + description only — for prompt injection / panel list. */ +export interface ProjectSkillIndexRow { + id: number; + name: string; + description: string; +} + +export const projectSkills = { + /** Active skills for a project (does NOT include global; merge in the caller). */ + listByProject(projectId: number): ProjectSkill[] { + return db + .prepare( + 'SELECT * FROM project_skills WHERE project_id = ? ORDER BY updated_at DESC, id DESC', + ) + .all(projectId) as ProjectSkill[]; + }, + listGlobal(): ProjectSkill[] { + return db + .prepare( + 'SELECT * FROM project_skills WHERE project_id IS NULL ORDER BY updated_at DESC, id DESC', + ) + .all() as ProjectSkill[]; + }, + /** Active skills visible to a project: its own skills plus all global skills. */ + listForProject(projectId: number): ProjectSkill[] { + return [...this.listByProject(projectId), ...this.listGlobal()]; + }, + get(id: number): ProjectSkill | undefined { + return db.prepare('SELECT * FROM project_skills WHERE id = ?').get(id) as + | ProjectSkill + | undefined; + }, + getByName(projectId: number | null, name: string): ProjectSkill | undefined { + const trimmed = name.trim(); + if (projectId == null) { + return db + .prepare('SELECT * FROM project_skills WHERE project_id IS NULL AND name = ?') + .get(trimmed) as ProjectSkill | undefined; + } + return db + .prepare('SELECT * FROM project_skills WHERE project_id = ? AND name = ?') + .get(projectId, trimmed) as ProjectSkill | undefined; + }, + /** Index (id/name/description) of all skills visible to a project. */ + listIndex(projectId: number): ProjectSkillIndexRow[] { + return this.listForProject(projectId).map((s) => ({ + id: s.id, + name: s.name, + description: s.description, + })); + }, + create(opts: { + projectId: number | null; + name: string; + description: string; + body: string; + tags?: string[] | null; + sourceChatId?: number | null; + }): ProjectSkill { + const name = opts.name.trim(); + const description = opts.description.trim(); + const body = opts.body.trim(); + if (!name) throw new Error('skill name required'); + if (!description) throw new Error('skill description required'); + if (!body) throw new Error('skill body required'); + const tags = opts.tags && opts.tags.length > 0 ? JSON.stringify(opts.tags) : null; + const info = db + .prepare( + `INSERT INTO project_skills (project_id, name, description, body, tags, source_chat_id) + VALUES (?, ?, ?, ?, ?, ?)`, + ) + .run(opts.projectId, name, description, body, tags, opts.sourceChatId ?? null); + if (opts.projectId != null) projects.touch(opts.projectId); + return this.get(Number(info.lastInsertRowid))!; + }, + /** Patch description/body/tags/name; bumps version + updated_at. */ + update( + id: number, + patch: { description?: string; body?: string; tags?: string[] | null; name?: string }, + ): void { + const existing = this.get(id); + if (!existing) return; + const name = patch.name != null ? patch.name.trim() : existing.name; + const description = + patch.description != null ? patch.description.trim() : existing.description; + const body = patch.body != null ? patch.body.trim() : existing.body; + const tags = + patch.tags !== undefined + ? patch.tags && patch.tags.length > 0 + ? JSON.stringify(patch.tags) + : null + : existing.tags; + db.prepare( + `UPDATE project_skills + SET name = ?, description = ?, body = ?, tags = ?, + version = version + 1, updated_at = datetime('now') + WHERE id = ?`, + ).run(name, description, body, tags, id); + if (existing.project_id != null) projects.touch(existing.project_id); + }, + incrementUsage(id: number): void { + db.prepare('UPDATE project_skills SET usage_count = usage_count + 1 WHERE id = ?').run(id); + }, + remove(id: number): void { + const row = this.get(id); + db.prepare('DELETE FROM project_skills WHERE id = ?').run(id); + if (row?.project_id != null) projects.touch(row.project_id); + }, +}; + +// ---------- project skill suggestions (user confirm in chat; inbox-gated) ---------- + +export const projectSkillSuggestions = { + listByChat(chatId: number): ProjectSkillSuggestion[] { + return db + .prepare( + 'SELECT * FROM project_skill_suggestions WHERE chat_id = ? ORDER BY id ASC', + ) + .all(chatId) as ProjectSkillSuggestion[]; + }, + listByProject(projectId: number): ProjectSkillSuggestion[] { + return db + .prepare( + 'SELECT * FROM project_skill_suggestions WHERE project_id = ? ORDER BY id ASC', + ) + .all(projectId) as ProjectSkillSuggestion[]; + }, + get(id: number): ProjectSkillSuggestion | undefined { + return db.prepare('SELECT * FROM project_skill_suggestions WHERE id = ?').get(id) as + | ProjectSkillSuggestion + | undefined; + }, + insert(opts: { + projectId: number; + chatId: number; + kind: 'new' | 'patch'; + targetSkillId?: number | null; + name: string; + description: string; + body: string; + tags?: string[] | null; + untrusted?: boolean; + assistantMessageId: number | null; + }): ProjectSkillSuggestion { + const name = opts.name.trim(); + const description = opts.description.trim(); + const body = opts.body.trim(); + if (!name) throw new Error('suggestion name required'); + if (!description) throw new Error('suggestion description required'); + if (!body) throw new Error('suggestion body required'); + const tags = opts.tags && opts.tags.length > 0 ? JSON.stringify(opts.tags) : null; + const info = db + .prepare( + `INSERT INTO project_skill_suggestions + (project_id, chat_id, kind, target_skill_id, name, description, body, tags, untrusted, assistant_message_id) + VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`, + ) + .run( + opts.projectId, + opts.chatId, + opts.kind, + opts.targetSkillId ?? null, + name, + description, + body, + tags, + opts.untrusted ? 1 : 0, + opts.assistantMessageId ?? null, + ); + return this.get(Number(info.lastInsertRowid))!; + }, + remove(id: number): void { + db.prepare('DELETE FROM project_skill_suggestions WHERE id = ?').run(id); + }, +}; + +/** UI: attach source chat title from `chats` (not persisted on `project_skills`). */ +export function enrichSkillWithSourceChatTitle(skill: ProjectSkill): ProjectSkill { + const sid = skill.source_chat_id; + if (sid == null) return { ...skill }; + const c = chats.get(sid); + const source_chat_title = (c?.title ?? '').trim() || 'Chat'; + return { ...skill, source_chat_title }; +} + +export function enrichSkillsWithSourceChatTitles(skills: ProjectSkill[]): ProjectSkill[] { + return skills.map(enrichSkillWithSourceChatTitle); +} + // ---------- project secrets (tokens / API keys; placeholders in messages) ---------- export type SecretPickerRow = Omit<ProjectSecret, 'value'> & { @@ -767,7 +969,7 @@ export const queuedMessages = { listByChat(chatId: number): QueuedMessage[] { const rows = db .prepare( - 'SELECT id, chat_id, content, reply_to_message_id, reply_quote, reply_to_role, attachments, created_at FROM queued_messages WHERE chat_id = ? ORDER BY position ASC, id ASC', + 'SELECT id, chat_id, content, reply_to_message_id, reply_quote, reply_to_role, attachments, mode, created_at FROM queued_messages WHERE chat_id = ? ORDER BY position ASC, id ASC', ) .all(chatId) as QueuedRow[]; return rows.map(parseQueuedRow); @@ -775,7 +977,7 @@ export const queuedMessages = { get(id: number): QueuedMessage | undefined { const row = db .prepare( - 'SELECT id, chat_id, content, reply_to_message_id, reply_quote, reply_to_role, attachments, created_at FROM queued_messages WHERE id = ?', + 'SELECT id, chat_id, content, reply_to_message_id, reply_quote, reply_to_role, attachments, mode, created_at FROM queued_messages WHERE id = ?', ) .get(id) as QueuedRow | undefined; return row ? parseQueuedRow(row) : undefined; @@ -786,7 +988,7 @@ export const queuedMessages = { | undefined { const row = db .prepare( - 'SELECT id, chat_id, content, reply_to_message_id, reply_quote, reply_to_role, attachments, inline_secrets, created_at FROM queued_messages WHERE id = ?', + 'SELECT id, chat_id, content, reply_to_message_id, reply_quote, reply_to_role, attachments, mode, inline_secrets, created_at FROM queued_messages WHERE id = ?', ) .get(id) as (QueuedRow & { inline_secrets: string | null }) | undefined; if (!row) return undefined; @@ -808,6 +1010,7 @@ export const queuedMessages = { replyTo?: { messageId: number; quote: string; role?: string } | null; attachments?: MessageAttachment[] | null; inlineSecrets?: InlineSecretWire[] | null; + mode?: ChatMode; }): QueuedMessage { const trimmed = opts.content.trim(); const hasAttachments = opts.attachments && opts.attachments.length > 0; @@ -826,8 +1029,8 @@ export const queuedMessages = { .prepare( `INSERT INTO queued_messages ( chat_id, content, reply_to_message_id, reply_quote, reply_to_role, - attachments, inline_secrets, position - ) VALUES (?, ?, ?, ?, ?, ?, ?, ?)`, + attachments, inline_secrets, mode, position + ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)`, ) .run( opts.chatId, @@ -837,6 +1040,7 @@ export const queuedMessages = { opts.replyTo?.role ?? null, attachmentsJson, inlineJson, + opts.mode ?? DEFAULT_MODE, position, ); return this.get(Number(info.lastInsertRowid))!; diff --git a/src/services/subtaskLlm.ts b/src/services/subtaskLlm.ts new file mode 100644 index 0000000..6762154 --- /dev/null +++ b/src/services/subtaskLlm.ts @@ -0,0 +1,88 @@ +/** + * Shared runner for the background "throwaway" LLM sub-tasks — fact extraction, + * fact compaction, and skill review. These are cheap, tool-less, single-shot + * prompts that should NOT spin up a full OpenClaw agent run when a direct, + * cheaper path is available. + * + * Routing: + * 1. If OpenRouter is configured, run on the cheap `summaryModel` via a direct + * /chat/completions call (much cheaper than an OpenClaw agent turn). + * 2. Otherwise — or if the OpenRouter call fails — fall back to a throwaway + * OpenClaw session on the default `main` agent (the original behaviour). + * + * Only the background sub-tasks use this. The user-facing turns (Ask, Execute, + * Work, Secure) keep their own backends. + */ + +import { openclawWs } from './openclawWs'; +import { complete, openRouterEnabled, isOpenRouterFailure } from './openRouter'; +import { loadOpenRouterConfig } from './config'; + +/** + * Sub-tasks always run against the gateway's default agent when falling back to + * OpenClaw. Using whatever agent the chat is on risked specialised agents + * underperforming on a plain text-extraction prompt. One agent, one behaviour. + */ +const SUBTASK_AGENT_ID = 'main'; + +export interface SubtaskTurnOptions { + /** Optional system prompt (OpenRouter path only; ignored on OpenClaw fallback). */ + system?: string; + temperature?: number; + maxTokens?: number; +} + +/** Throwaway OpenClaw session — original behaviour, used as the fallback path. */ +async function runViaOpenClaw(message: string): Promise<string> { + let sessionKey: string | null = null; + try { + const session = await openclawWs.createSession({ agentId: SUBTASK_AGENT_ID }); + sessionKey = session.key; + let acc = ''; + await openclawWs.runTurn({ + sessionKey: session.key, + message, + onEvent: (ev) => { + if (ev.type === 'text-delta') acc += ev.text; + else if (ev.type === 'text-final') acc = ev.text || acc; + }, + }); + return acc; + } finally { + if (sessionKey) openclawWs.deleteSession(sessionKey).catch(() => {}); + } +} + +/** + * Run a single-shot background sub-task prompt. Prefers OpenRouter (cheap model) + * and falls back to OpenClaw when OpenRouter is unconfigured or the call throws. + * Returns the assistant text (possibly empty — callers parse/validate it). + */ +export async function runSubtaskTurn( + message: string, + opts: SubtaskTurnOptions = {}, +): Promise<string> { + if (openRouterEnabled()) { + try { + const cfg = loadOpenRouterConfig(); + const messages = []; + if (opts.system) messages.push({ role: 'system' as const, content: opts.system }); + messages.push({ role: 'user' as const, content: message }); + // Cheapest configured model — these tasks don't need a frontier model. + return await complete({ + messages, + model: cfg.summaryModel, + temperature: opts.temperature, + maxTokens: opts.maxTokens, + }); + } catch (err) { + // OpenRouter unavailable / errored → fall back to OpenClaw rather than + // dropping the sub-task. Log once; the fallback path is best-effort too. + console.error( + '[subtaskLlm] OpenRouter failed, falling back to OpenClaw', + isOpenRouterFailure(err) ? (err as Error).message : err, + ); + } + } + return runViaOpenClaw(message); +} diff --git a/src/services/toolLabels.ts b/src/services/toolLabels.ts index 3553cb9..ce5c153 100644 --- a/src/services/toolLabels.ts +++ b/src/services/toolLabels.ts @@ -1,7 +1,24 @@ -/** Human-readable status for an OpenClaw / OpenAI tool name. */ +/** Human-readable status for an iClaw-runtime / OpenClaw / OpenAI tool name. */ export function toolActivityLabel(name: string): string { const n = name.toLowerCase(); + // Exact matches for iClaw's own runtime tools — checked first so a specific + // tool (e.g. social_search) doesn't fall into the generic "search" bucket. + switch (n) { + case 'social_search': return 'Searching social media…'; + case 'web_search': return 'Searching the web…'; + case 'web_fetch': return 'Reading the web…'; + case 'read_summary': return 'Skimming a file…'; + case 'analyze_link': return 'Analyzing the link…'; + case 'show_image': return 'Sharing an image…'; + case 'search_files': return 'Searching files…'; + case 'list_files': return 'Listing files…'; + case 'read_file': return 'Reading a file…'; + case 'write_file': return 'Writing a file…'; + case 'edit_file': return 'Editing a file…'; + case 'run_command': return 'Running a command…'; + } + if (/web.?search|internet.?search|search|grep|find|lookup|browse/.test(n)) { return 'Searching…'; } @@ -28,6 +45,43 @@ export function toolActivityLabel(name: string): string { return (pretty ? pretty.charAt(0).toUpperCase() + pretty.slice(1) : name) + '…'; } +/** + * Short, safe "detail" for the expandable activity status — the actual query / + * URL / command / path the tool is working on (e.g. "Searching social media…" + * → "r/LocalLLaMA"). Pulled only from well-known arg fields and capped, so we + * never dump a large or unexpected input into the UI. Returns undefined when + * there's nothing useful to show. + */ +export function toolActivityDetail(name: string, input: unknown): string | undefined { + if (!input || typeof input !== 'object') return undefined; + const a = input as Record<string, unknown>; + const str = (v: unknown): string | undefined => + typeof v === 'string' && v.trim() ? v.trim() : undefined; + + let raw: string | undefined; + if (str(a.query)) { + raw = str(a.query); + } else if (str(a.command)) { + raw = str(a.command); + } else if (str(a.url)) { + const u = str(a.url)!; + try { + const parsed = new URL(u); + raw = parsed.host + parsed.pathname.replace(/\/$/, ''); + } catch { + raw = u; + } + } else if (str(a.path)) { + raw = str(a.path); + } else if (str(a.name)) { + raw = str(a.name); + } + if (!raw) return undefined; + + raw = raw.replace(/\s+/g, ' '); + return raw.length > 70 ? raw.slice(0, 69) + '…' : raw; +} + /** Human-readable label for gateway lifecycle phases. */ export function lifecycleActivityLabel(phase: string): string { switch (phase) { diff --git a/src/services/turnReply.ts b/src/services/turnReply.ts index de5e7c2..421c8bc 100644 --- a/src/services/turnReply.ts +++ b/src/services/turnReply.ts @@ -28,6 +28,71 @@ export interface HistoryMessageLike { toolName?: string; timestamp?: number; isError?: boolean; + /** + * Per-message token usage, present on assistant rows in OpenClaw's + * UI-normalized `chat.history`. The gateway sanitizes it to a flat object + * of numeric fields (field names vary by provider — see `pickUsageTotal`). + */ + usage?: unknown; +} + +/** Pull a finite number from a usage object under the first matching key. */ +function firstNumber(u: Record<string, unknown>, keys: string[]): number | null { + for (const k of keys) { + const v = u[k]; + if (typeof v === 'number' && Number.isFinite(v)) return v; + } + return null; +} + +/** + * Total tokens for one assistant row's `usage`. Prefers an explicit total; + * otherwise sums input + output. Field names cover OpenClaw's sanitized set + * (camelCase, snake_case, and Anthropic-style). Returns `null` when the row + * carries no usable usage numbers. + */ +function pickUsageTotal(usage: unknown): number | null { + if (!usage || typeof usage !== 'object') return null; + const u = usage as Record<string, unknown>; + const total = firstNumber(u, ['total_tokens', 'totalTokens', 'total']); + if (total != null) return total; + const input = firstNumber(u, ['input_tokens', 'inputTokens', 'input', 'prompt_tokens', 'promptTokens']); + const output = firstNumber(u, ['output_tokens', 'outputTokens', 'output', 'completion_tokens', 'completionTokens']); + if (input == null && output == null) return null; + return (input ?? 0) + (output ?? 0); +} + +/** Prompt tokens served from cache for one assistant row's `usage`, or null. */ +function pickUsageCached(usage: unknown): number | null { + if (!usage || typeof usage !== 'object') return null; + return firstNumber(usage as Record<string, unknown>, [ + 'cacheRead', + 'cache_read_input_tokens', + 'cachedTokens', + 'cached_tokens', + ]); +} + +/** + * Sum token usage across every assistant row in a turn-scoped slice. A native + * tool loop commits multiple assistant segments per turn (preamble + post-tool + * reply), each with its own `usage`, so the turn's true cost is their sum. + * Returns `null` for a field when no assistant row reported it. + */ +export function extractTurnUsage(slice: HistoryMessageLike[]): { + tokens: number | null; + cached: number | null; +} { + let tokens: number | null = null; + let cached: number | null = null; + for (const row of slice) { + if (row.role !== 'assistant') continue; + const t = pickUsageTotal(row.usage); + if (t != null) tokens = (tokens ?? 0) + t; + const c = pickUsageCached(row.usage); + if (c != null) cached = (cached ?? 0) + c; + } + return { tokens, cached }; } /** diff --git a/src/services/uploads.ts b/src/services/uploads.ts index bba9c58..0c8ea7a 100644 --- a/src/services/uploads.ts +++ b/src/services/uploads.ts @@ -15,7 +15,7 @@ */ import { randomUUID } from 'node:crypto'; -import { mkdirSync, readFileSync, writeFileSync } from 'node:fs'; +import { mkdirSync, readFileSync, writeFileSync, statSync, copyFileSync } from 'node:fs'; import { basename, join } from 'node:path'; import { resolveUploadsRoot as uploadsRootFromPaths } from '../paths'; import type { MessageAttachment } from '../types'; @@ -197,6 +197,32 @@ export function persistIncomingAttachments( return out; } +/** + * Resolve persisted attachments to absolute on-disk paths for the iclaw-runtime + * (Work/Secure/Incognito). The runtime shares the host filesystem, so it reads + * the file directly — no base64 roundtrip. Rows with an unexpected URL are + * skipped (rather than thrown) so one bad attachment can't sink the whole turn. + */ +export function runtimeAttachmentsFromPersisted( + chatId: number, + persisted: MessageAttachment[], +): { path: string; mimeType: string; fileName: string }[] { + if (!persisted.length) return []; + const chatDir = join(resolveUploadsRoot(), String(chatId)); + const prefix = `/uploads/${chatId}/`; + const out: { path: string; mimeType: string; fileName: string }[] = []; + for (const att of persisted) { + const url = att.url || ''; + if (!url.startsWith(prefix)) continue; + out.push({ + path: join(chatDir, basename(url)), + mimeType: att.mimeType || 'application/octet-stream', + fileName: att.fileName || 'attachment', + }); + } + return out; +} + /** * Rebuild OpenClaw gateway attachment payloads from rows already on disk * (queued messages persist files at enqueue time). @@ -238,3 +264,48 @@ export function gatewayAttachmentsFromPersisted( } return out; } + +/** Map a lowercase image extension to its MIME (the set show_image supports). */ +const AGENT_IMAGE_MIME: Record<string, string> = { + '.png': 'image/png', + '.jpg': 'image/jpeg', + '.jpeg': 'image/jpeg', + '.gif': 'image/gif', + '.webp': 'image/webp', + '.svg': 'image/svg+xml', +}; + +/** + * Persist an image the AGENT produced — already a real file on the host (a Work + * folder is bind-mounted live; a Secure workspace dir is the host side of the + * container mount) — into the chat's uploads dir, returning a MessageAttachment + * for the assistant row. Returns null (caller skips it) if the path isn't a + * readable, image-typed, size-capped regular file. + * + * Path TRUST is the caller's job: only pass a host path already confirmed to lie + * inside an allowed root. This copies (never moves) so the original is untouched. + */ +export function persistAgentImage(chatId: number, hostPath: string): MessageAttachment | null { + let st; + try { st = statSync(hostPath); } catch { return null; } + if (!st.isFile() || st.size === 0 || st.size > MAX_ATTACHMENT_BYTES) return null; + + const ext = (basename(hostPath).match(/\.[a-zA-Z0-9]+$/)?.[0] ?? '').toLowerCase(); + const mimeType = AGENT_IMAGE_MIME[ext]; + if (!mimeType) return null; // not a supported image type + + const chatDir = join(resolveUploadsRoot(), String(chatId)); + mkdirSync(chatDir, { recursive: true }); + const onDiskName = `${randomUUID()}${ext}`; + try { + copyFileSync(hostPath, join(chatDir, onDiskName)); + } catch { + return null; + } + return { + url: `/uploads/${chatId}/${onDiskName}`, + mimeType, + fileName: sanitizeFileName(basename(hostPath)), + sizeBytes: st.size, + }; +} diff --git a/src/services/workRuntime.ts b/src/services/workRuntime.ts new file mode 100644 index 0000000..e8870cf --- /dev/null +++ b/src/services/workRuntime.ts @@ -0,0 +1,235 @@ +/** + * iClaw Work Mode runtime client. + * + * Communicates with the `@iclaw/runtime` service (packages/iclaw-runtime) + * over HTTP. The runtime runs as a sidecar process on the same host. + * + * The runtime exposes: + * POST /sessions → { sessionId } + * POST /sessions/:id/messages → 202 + * GET /sessions/:id/events → SSE stream + * DELETE /sessions/:id → stop + */ +import http from 'http'; + +const RUNTIME_PORT = parseInt(process.env.ICLAW_RUNTIME_PORT || '7430', 10); +const RUNTIME_SECRET = process.env.ICLAW_RUNTIME_SECRET || ''; + +function runtimeHeaders(): Record<string, string> { + const h: Record<string, string> = { 'Content-Type': 'application/json' }; + if (RUNTIME_SECRET) h['x-iclaw-token'] = RUNTIME_SECRET; + return h; +} + +function request( + method: string, + path: string, + body?: unknown, +): Promise<{ status: number; data: unknown }> { + return new Promise((resolve, reject) => { + const payload = body != null ? JSON.stringify(body) : undefined; + const headers = runtimeHeaders(); + if (payload) headers['Content-Length'] = Buffer.byteLength(payload).toString(); + + const req = http.request( + { hostname: '127.0.0.1', port: RUNTIME_PORT, path, method, headers }, + (res) => { + let raw = ''; + res.on('data', (chunk) => { raw += chunk; }); + res.on('end', () => { + try { + resolve({ status: res.statusCode ?? 0, data: JSON.parse(raw) }); + } catch { + resolve({ status: res.statusCode ?? 0, data: raw }); + } + }); + }, + ); + req.on('error', reject); + if (payload) req.write(payload); + req.end(); + }); +} + +export interface CreateSessionOptions { + allowedFolders?: string[]; + /** + * Per-folder access levels. When provided, the runtime enforces read-only + * folders (denies write_file / run_command under them) and derives the + * allowed-path list from it. + */ + folderAccess?: { path: string; readonly: boolean }[]; + /** + * Safe Mode only: host folders to COPY into the sandbox workspace (originals + * never touched). Ignored in Work Mode, which bind-mounts folders live. + */ + copyFolders?: string[]; + model?: string; + secure?: boolean; + /** Incognito: read-only, read-anywhere, web_fetch enabled. Mutually exclusive with secure. */ + incognito?: boolean; + systemPrompt?: string; + /** Stable identity (e.g. "chat:156") so a chat reconnects to its workspace. */ + key?: string; + /** Compacted prior history to seed context (used after a restart). */ + history?: { role: string; content: string }[]; +} + +/** Create a new Work Mode session. Returns sessionId. */ +export async function createWorkSession(opts: CreateSessionOptions = {}): Promise<string> { + const body: Record<string, unknown> = { allowedFolders: opts.allowedFolders, secure: opts.secure }; + if (opts.incognito) body.incognito = true; + if (opts.folderAccess?.length) body.folderAccess = opts.folderAccess; + if (opts.copyFolders?.length) body.copyFolders = opts.copyFolders; + if (opts.model) body.model = opts.model; + if (opts.systemPrompt) body.systemPrompt = opts.systemPrompt; + if (opts.key) body.key = opts.key; + if (opts.history?.length) body.history = opts.history; + const res = await request('POST', '/sessions', body); + if (res.status !== 201) { + throw new Error(`Failed to create work session: ${JSON.stringify(res.data)}`); + } + return (res.data as { sessionId: string }).sessionId; +} + +/** A dropped file forwarded to the runtime: absolute host path + metadata. */ +export interface RuntimeAttachmentInput { + path: string; + mimeType: string; + fileName: string; +} + +/** Send a user message to a work session. */ +export async function sendWorkMessage(sessionId: string, content: string, networkEnabled?: boolean, ttlDays?: number, attachments?: RuntimeAttachmentInput[], copyFolders?: string[]): Promise<void> { + const body: Record<string, unknown> = { content }; + if (networkEnabled !== undefined) body.networkEnabled = networkEnabled; + if (ttlDays !== undefined) body.ttlDays = ttlDays; + if (attachments?.length) body.attachments = attachments; + // Safe Mode: lets the runtime copy folders the user added mid-chat into the + // sandbox on this turn (ignored in Work Mode / by non-secure sessions). + if (copyFolders?.length) body.copyFolders = copyFolders; + const res = await request('POST', `/sessions/${sessionId}/messages`, body); + if (res.status !== 202) { + throw new Error(`Failed to send work message: ${JSON.stringify(res.data)}`); + } +} + +/** Stop a work session. */ +export async function stopWorkSession(sessionId: string): Promise<void> { + await request('DELETE', `/sessions/${sessionId}`); +} + +/** + * Abort the in-flight turn for a session WITHOUT destroying it (workspace + + * container survive — unlike stopWorkSession). Backs the Stop button for + * Work / Secure / Incognito turns. + */ +export async function abortWorkSession(sessionId: string): Promise<void> { + await request('POST', `/sessions/${sessionId}/abort`); +} + +/** A "saved N% cost" note from the runtime (e.g. analyze_link summary mode). */ +export interface RuntimeSavingsNote { + kind: string; + /** Short human label for the source, e.g. "video transcript". */ + source: string; + /** Whole-percent saved. Absent for quantity-free notes (search line-trimming). */ + savedPct?: number; + fullChars?: number; + deliveredChars?: number; +} + +export type WorkEvent = + | { type: 'text'; content: string } + | { type: 'tool_start'; name: string; input?: unknown } + | { type: 'tool_result'; name: string; result?: string } + | { type: 'note'; note: RuntimeSavingsNote } + | { type: 'image'; path: string; mime: string; fileName: string; bytes: number } + | { type: 'done'; tokens?: number; cached?: number } + | { type: 'error'; message: string }; + +/** + * Subscribe to SSE events from a work session. + * Calls `onEvent` for each event, resolves when stream closes. + */ +export function subscribeWorkEvents( + sessionId: string, + onEvent: (event: WorkEvent) => void, + onError?: (err: Error) => void, +): () => void { + const headers = runtimeHeaders(); + const req = http.request( + { + hostname: '127.0.0.1', + port: RUNTIME_PORT, + path: `/sessions/${sessionId}/events`, + method: 'GET', + headers, + }, + (res) => { + let buf = ''; + res.on('data', (chunk: string) => { + buf += chunk; + const lines = buf.split('\n'); + buf = lines.pop() ?? ''; + for (const line of lines) { + if (line.startsWith('data: ')) { + try { + const event = JSON.parse(line.slice(6)) as WorkEvent; + onEvent(event); + } catch { + // ignore malformed SSE lines + } + } + } + }); + res.on('end', () => onEvent({ type: 'done' })); + res.on('error', (err) => onError?.(err)); + }, + ); + req.on('error', (err) => onError?.(err)); + req.end(); + + return () => req.destroy(); +} + +/** Get workspace info for a session. */ +export async function getWorkspaceInfo(sessionId: string): Promise<{ workspaceSize: number; secure: boolean } | null> { + try { + const res = await request('GET', `/sessions/${sessionId}/info`); + if (res.status !== 200) return null; + return res.data as { workspaceSize: number; secure: boolean }; + } catch { return null; } +} + +export interface ExportResult { ok: boolean; path?: string; files?: number; error?: string } +export interface ApplyResult { + ok: boolean; + source?: string; + applied?: { path: string; kind: 'created' | 'modified' }[]; + error?: string; +} + +/** Export a Safe sandbox to a host folder (default ~/Downloads). */ +export async function exportSandbox(sessionId: string, destDir?: string): Promise<ExportResult> { + const res = await request('POST', `/sessions/${sessionId}/export`, destDir ? { destDir } : {}); + if (res.status !== 200) throw new Error(`Export failed: ${JSON.stringify(res.data)}`); + return res.data as ExportResult; +} + +/** Apply a Safe sandbox's changes back to the originally-ingested folders. */ +export async function applySandboxChanges(sessionId: string): Promise<ApplyResult[]> { + const res = await request('POST', `/sessions/${sessionId}/apply`, {}); + if (res.status !== 200) throw new Error(`Apply failed: ${JSON.stringify(res.data)}`); + return (res.data as { results: ApplyResult[] }).results; +} + +/** True if the runtime service appears to be running. */ +export async function runtimeAvailable(): Promise<boolean> { + try { + const res = await request('GET', '/health'); + return res.status === 200; + } catch { + return false; + } +} diff --git a/src/types/index.ts b/src/types/index.ts index 7c74b97..34dc232 100644 --- a/src/types/index.ts +++ b/src/types/index.ts @@ -41,6 +41,8 @@ export interface QueuedMessage { reply_quote: string | null; reply_to_role: string | null; attachments: MessageAttachment[] | null; + /** Selected mode at enqueue time; preserved so the flush sends with it. */ + mode: ChatMode; created_at: string; } @@ -54,6 +56,42 @@ export interface ProjectFactSuggestion { created_at: string; } +/** + * Procedural memory: an accepted, active project skill stored as SKILL.md. + * `project_id === null` means a global skill (available to every project). + */ +export interface ProjectSkill { + id: number; + project_id: number | null; // null = global + name: string; + description: string; + body: string; + tags: string | null; // JSON array + source_chat_id: number | null; + usage_count: number; + version: number; + created_at: string; + updated_at: string; + /** Filled by server for UI (not a DB column). */ + source_chat_title?: string; +} + +/** LLM-proposed skill awaiting user accept/reject in the chat UI (inbox-gated). */ +export interface ProjectSkillSuggestion { + id: number; + project_id: number; + chat_id: number; + kind: 'new' | 'patch'; + target_skill_id: number | null; + name: string; + description: string; + body: string; + tags: string | null; + untrusted: number; // 0 | 1 + assistant_message_id: number | null; + created_at: string; +} + /** API key / token; message text uses `[[iclaw:secret:id|…]]` placeholders. */ export interface ProjectSecret { id: number; @@ -68,6 +106,24 @@ export interface ProjectSecret { export type ChatKind = 'normal' | 'draft' | 'task_execution'; +/** + * How a user message should be handled. + * + * - 'execute' — default: OpenClaw may use tools, files, shell, browser, etc. + * The back-compat fallback for any message whose `mode` is missing or + * unrecognized. + * - 'work' / 'secure' — run on iclaw-runtime (our runtime). + * - 'incognito' — read-only, ephemeral research on iclaw-runtime; never + * persisted (see services/chatModes.ts). + * + * Kept as a string union for the live modes, but the full catalog + * (incl. planned modes like image) lives in + * `services/chatModes.ts` so new modes can be added without touching this + * type everywhere. Storage columns are plain TEXT, so adding a mode later + * needs no DB migration. + */ +export type ChatMode = 'execute' | 'work' | 'secure' | 'incognito'; + export type TaskStatus = | 'planning' | 'ready' @@ -180,8 +236,9 @@ export interface Chat { shares_to_project: number; /** Optional per-chat model override applied via `sessions.patch` on OpenClaw. */ model_override: string | null; - /** Reasoning visibility mirror — 'off' | 'on' | 'stream'. */ - reasoning_mode: string; + /** Sticky composer send-mode for this chat (e.g. 'work' | 'secure' | 'execute'). + * null = never set → client uses the UI default. Survives navigation/devices. */ + mode: string | null; title_manual: number; unread: number; created_at: string; @@ -214,5 +271,15 @@ export interface Message { reply_to_role?: string | null; /** Persisted user-uploaded files (image / doc). `null` row column is parsed to undefined here. */ attachments?: MessageAttachment[] | null; + /** + * How this message was sent. Only meaningful on `user` rows; assistant / + * system rows default to 'execute'. Missing/legacy rows read back as + * 'execute' (DB column default), so old chats stay fully compatible. + */ + mode: ChatMode; + /** Total tokens spent producing this (assistant) message. Dev-mode only; null otherwise. */ + tokens?: number | null; + /** Of `tokens`, how many prompt tokens were served from the provider cache. Dev-mode. */ + cached_tokens?: number | null; created_at: string; } diff --git a/src/types/protocol.ts b/src/types/protocol.ts index 1ba4c9a..d7fdabd 100644 --- a/src/types/protocol.ts +++ b/src/types/protocol.ts @@ -10,9 +10,11 @@ */ import type { + ChatMode, Message, Project, ProjectFact, + ProjectSkill, QueuedMessage, ScheduledMessage, TaskWithSteps, @@ -39,6 +41,12 @@ export type ClientMsg = content: string; agent?: string; projectId?: number | null; + /** + * How to handle this message — 'execute' (full agent, default), + * 'work' / 'secure' / 'incognito' (iclaw-runtime). Omitted/unknown → + * server treats it as 'execute'. See services/chatModes.ts. + */ + mode?: ChatMode; /** Reply to an existing user/assistant row in this chat (quote ≤240 chars). */ replyTo?: { messageId: number; quote: string; role?: string }; /** @@ -62,6 +70,21 @@ export type ClientMsg = } /** Abort a running turn for this chat. */ | { type: 'abort'; chatId: number } + /** + * Incognito turn — ephemeral, never persisted. `key` is the browser's in-RAM + * chat id (not a DB chatId). The server streams `incognito-*` events back to + * THIS socket only. `workFolders` mirrors the `send` field (folders are forced + * read-only in incognito). + */ + | { + type: 'incognito-send'; + key: string; + content: string; + requestId?: string; + workFolders?: Array<{ path: string; readonly?: boolean } | string>; + } + /** Abort / forget an incognito session. */ + | { type: 'incognito-abort'; key: string } /** Resolve a pending exec approval — `decision` is 'approved' | 'denied'. */ | { type: 'exec-approval'; @@ -85,6 +108,12 @@ export type ServerMsg = | { type: 'hello'; serverStarted: number } | { type: 'pong' } + /* ---- incognito (ephemeral; keyed by the browser's in-RAM chat id) ---- */ + | { type: 'incognito-turn-delta'; key: string; text: string } + | { type: 'incognito-turn-tool'; key: string; name: string } + | { type: 'incognito-turn-ended'; key: string; tokens?: number; cached?: number } + | { type: 'incognito-error'; key: string; message: string } + /* ---- chat lifecycle ---- */ | { type: 'chat-created'; @@ -107,8 +136,8 @@ export type ServerMsg = projectName?: string | null; /** Toggle on whether the chat writes facts back to the project. */ sharesToProject?: boolean; - /** Reasoning visibility mode mirror — 'off' | 'on' | 'stream'. */ - reasoningMode?: 'off' | 'on' | 'stream'; + /** Sticky composer send-mode mirror (e.g. 'work' | 'secure' | 'execute'). */ + mode?: string; /** Present after mutations that bump `chats.updated_at` — flat sidebar order. */ updatedAt?: string; } @@ -171,6 +200,31 @@ export type ServerMsg = } | { type: 'project-fact-suggestion-removed'; chatId: number; suggestionId: number } + /* ---- project skills (procedural memory) ---- */ + | { type: 'project-skill-added'; projectId: number; skill: ProjectSkill } + | { type: 'project-skill-updated'; projectId: number; skill: ProjectSkill } + | { type: 'project-skill-deleted'; projectId: number; skillId: number } + /** + * After a turn, proposed skills the user can accept into project memory + * (confirm in chat). Card list carries summary fields only; the full body is + * fetched via REST when the user expands/edits — keeps WS frames small. + */ + | { + type: 'project-skill-suggestions'; + chatId: number; + projectId: number; + projectName: string; + suggestions: { + id: number; + kind: 'new' | 'patch'; + name: string; + description: string; + untrusted: boolean; + targetSkillId: number | null; + }[]; + } + | { type: 'project-skill-suggestion-removed'; chatId: number; suggestionId: number } + /* ---- scheduled messages (Telegram-style send-later) ---- */ | { type: 'scheduled-added'; chatId: number; scheduled: ScheduledMessage } | { type: 'scheduled-updated'; chatId: number; scheduled: ScheduledMessage } @@ -233,8 +287,6 @@ export type ServerMsg = approvalId: string; decision: string; } - /** A turn lost a reasoning/analysis chunk — only emitted when reasoning is on. */ - | { type: 'turn-reasoning'; chatId: number; text: string } /** Live mirror of the OpenClaw gateway health — drives the header badge. */ | { diff --git a/test/helpers/db.ts b/test/helpers/db.ts index bfb5ddc..a01daff 100644 --- a/test/helpers/db.ts +++ b/test/helpers/db.ts @@ -32,6 +32,8 @@ export function resetTestDb(): void { DELETE FROM task_context_snapshots; DELETE FROM project_fact_suggestions; DELETE FROM project_facts; + DELETE FROM project_skill_suggestions; + DELETE FROM project_skills; DELETE FROM project_secrets; DELETE FROM scheduled_messages; DELETE FROM queued_messages; diff --git a/test/integration/chatRunner.test.ts b/test/integration/chatRunner.test.ts index d2c792b..4fab513 100644 --- a/test/integration/chatRunner.test.ts +++ b/test/integration/chatRunner.test.ts @@ -304,34 +304,6 @@ describe('sendMessage — reply-to (quoted message)', () => { }); }); -describe('sendMessage — reasoning gate', () => { - it('does not broadcast turn-reasoning when chat.reasoning_mode = "off"', async () => { - openclawWsMock.runTurn.mockImplementationOnce(async (params) => { - params.onEvent({ type: 'reasoning', text: 'inner monologue' }); - params.onEvent({ type: 'text-final', text: 'final answer' }); - return { runId: 'r', text: 'final answer' }; - }); - await sendMessage({ content: 'plain' }); - expect(findChatBroadcast('turn-reasoning')).toBeUndefined(); - }); - - it('broadcasts turn-reasoning when chat.reasoning_mode != "off"', async () => { - // Create chat first, flip mode, then call sendMessage with that chatId. - openclawWsMock.runTurn.mockResolvedValueOnce({ runId: 'r0', text: 'ok' }); - const { chatId } = await sendMessage({ content: 'init' }); - chats.setReasoningMode(chatId, 'on'); - - broadcasts.toChat = []; - openclawWsMock.runTurn.mockImplementationOnce(async (params) => { - params.onEvent({ type: 'reasoning', text: 'analyzing…' }); - params.onEvent({ type: 'text-final', text: 'done' }); - return { runId: 'r1', text: 'done' }; - }); - await sendMessage({ chatId, content: 'second' }); - expect(findChatBroadcast('turn-reasoning')).toBeDefined(); - }); -}); - describe('sendMessage — attachment handling', () => { it('rewrites /api/chat/media URLs through /media proxy and inlines markdown', async () => { openclawWsMock.runTurn.mockImplementationOnce(async (params) => { diff --git a/test/integration/routes.chats.test.ts b/test/integration/routes.chats.test.ts index e3c62dd..2b71c0f 100644 --- a/test/integration/routes.chats.test.ts +++ b/test/integration/routes.chats.test.ts @@ -223,30 +223,6 @@ describe('POST /chats/:id/delete', () => { }); }); -describe('POST /chats/:id/reasoning', () => { - it('mirrors the requested mode (on/off/stream)', async () => { - const c = chats.create('openclaw/default'); - for (const mode of ['on', 'stream', 'off'] as const) { - const res = await request(app) - .post(`/chats/${c.id}/reasoning`) - .set('content-type', 'application/json') - .send({ mode }); - expect(res.status).toBe(200); - expect(chats.get(c.id)!.reasoning_mode).toBe(mode); - } - }); - - it('falls back to "off" for unknown modes', async () => { - const c = chats.create('openclaw/default'); - const res = await request(app) - .post(`/chats/${c.id}/reasoning`) - .set('content-type', 'application/json') - .send({ mode: 'gibberish' }); - expect(res.status).toBe(200); - expect(chats.get(c.id)!.reasoning_mode).toBe('off'); - }); -}); - describe('Composer queue routes', () => { it('POST /chats/:id/queue creates + GET lists + DELETE removes', async () => { const c = chats.create('openclaw/default'); diff --git a/test/unit/secureExport.test.ts b/test/unit/secureExport.test.ts new file mode 100644 index 0000000..b3b5820 --- /dev/null +++ b/test/unit/secureExport.test.ts @@ -0,0 +1,90 @@ +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { + mkdtempSync, mkdirSync, writeFileSync, rmSync, readFileSync, existsSync, +} from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; + +import { ingestSources } from '../../packages/iclaw-runtime/src/secure-ingest'; +import { + exportWorkspace, applyChanges, +} from '../../packages/iclaw-runtime/src/secure-export'; + +let root: string; +let workspace: string; + +beforeEach(() => { + root = mkdtempSync(join(tmpdir(), 'iclaw-export-test-')); + workspace = join(root, 'workspace'); + mkdirSync(workspace, { recursive: true }); +}); +afterEach(() => { + rmSync(root, { recursive: true, force: true }); +}); + +describe('exportWorkspace', () => { + it('copies the sandbox out, excluding workspace internals', () => { + writeFileSync(join(workspace, 'result.txt'), 'output'); + mkdirSync(join(workspace, '.tools'), { recursive: true }); + writeFileSync(join(workspace, '.tools', 'bin'), 'junk'); + writeFileSync(join(workspace, '.iclaw-ingest.json'), '{}'); + + const dest = join(root, 'out'); + const r = exportWorkspace(workspace, dest); + expect(r.ok).toBe(true); + expect(r.path!.startsWith(dest)).toBe(true); + expect(readFileSync(join(r.path!, 'result.txt'), 'utf-8')).toBe('output'); + // Internals are not exported. + expect(existsSync(join(r.path!, '.tools'))).toBe(false); + expect(existsSync(join(r.path!, '.iclaw-ingest.json'))).toBe(false); + }); +}); + +describe('applyChanges', () => { + it('copies new and modified files back to the original, without deleting', async () => { + // Original project. + const src = join(root, 'project'); + mkdirSync(join(src, 'src'), { recursive: true }); + writeFileSync(join(src, 'src', 'a.ts'), 'A\n'); + writeFileSync(join(src, 'keep.txt'), 'KEEP\n'); + + // Ingest it (records the source→target mapping in .iclaw-ingest.json). + await ingestSources(workspace, [{ kind: 'folder', path: src }]); + const sandbox = join(workspace, 'project'); + + // Agent edits a.ts, adds b.ts, and "deletes" keep.txt inside the sandbox. + writeFileSync(join(sandbox, 'src', 'a.ts'), 'A changed\n'); + writeFileSync(join(sandbox, 'new.ts'), 'NEW\n'); + rmSync(join(sandbox, 'keep.txt'), { force: true }); + + const results = applyChanges(workspace); + expect(results).toHaveLength(1); + expect(results[0].ok).toBe(true); + const applied = results[0].applied!.map((c) => `${c.kind}:${c.path}`).sort(); + + // Modified + created are applied back to the original. + expect(readFileSync(join(src, 'src', 'a.ts'), 'utf-8')).toBe('A changed\n'); + expect(readFileSync(join(src, 'new.ts'), 'utf-8')).toBe('NEW\n'); + expect(applied.some((s) => s.includes('a.ts'))).toBe(true); + expect(applied.some((s) => s.includes('new.ts'))).toBe(true); + + // Deletion in the sandbox does NOT delete the user's original file. + expect(existsSync(join(src, 'keep.txt'))).toBe(true); + expect(readFileSync(join(src, 'keep.txt'), 'utf-8')).toBe('KEEP\n'); + }); + + it('applies nothing when the sandbox is unchanged', async () => { + const src = join(root, 'p2'); + mkdirSync(src, { recursive: true }); + writeFileSync(join(src, 'f.txt'), 'same\n'); + await ingestSources(workspace, [{ kind: 'folder', path: src }]); + + const results = applyChanges(workspace); + expect(results[0].ok).toBe(true); + expect(results[0].applied).toEqual([]); + }); + + it('returns empty when nothing was ingested', () => { + expect(applyChanges(workspace)).toEqual([]); + }); +}); diff --git a/test/unit/secureIngest.test.ts b/test/unit/secureIngest.test.ts new file mode 100644 index 0000000..03e39fb --- /dev/null +++ b/test/unit/secureIngest.test.ts @@ -0,0 +1,110 @@ +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import { mkdtempSync, mkdirSync, writeFileSync, rmSync, readFileSync, existsSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; + +import { ingestSources, describeIngest } from '../../packages/iclaw-runtime/src/secure-ingest'; + +let root: string; +let workspace: string; + +beforeEach(() => { + root = mkdtempSync(join(tmpdir(), 'iclaw-ingest-test-')); + workspace = join(root, 'workspace'); + mkdirSync(workspace, { recursive: true }); +}); +afterEach(() => { + rmSync(root, { recursive: true, force: true }); +}); + +function makeProject(name: string): string { + const dir = join(root, name); + mkdirSync(join(dir, 'src'), { recursive: true }); + writeFileSync(join(dir, 'src', 'index.ts'), 'export const x = 1;\n'); + writeFileSync(join(dir, 'README.md'), '# hi\n'); + // noise that should be skipped + mkdirSync(join(dir, 'node_modules', 'pkg'), { recursive: true }); + writeFileSync(join(dir, 'node_modules', 'pkg', 'big.js'), 'junk'); + mkdirSync(join(dir, '.git'), { recursive: true }); + writeFileSync(join(dir, '.git', 'HEAD'), 'ref: x'); + return dir; +} + +describe('ingestSources — folder', () => { + it('copies a folder into the workspace and leaves the original untouched', async () => { + const src = makeProject('my-app'); + const results = await ingestSources(workspace, [{ kind: 'folder', path: src }]); + + expect(results).toHaveLength(1); + expect(results[0]).toMatchObject({ kind: 'folder', ok: true, target: 'my-app' }); + + // Copy landed in the sandbox. + const copied = join(workspace, 'my-app'); + expect(readFileSync(join(copied, 'src', 'index.ts'), 'utf-8')).toContain('export const x'); + expect(existsSync(join(copied, 'README.md'))).toBe(true); + + // Original is byte-for-byte intact (Safe Mode never touches it). + expect(readFileSync(join(src, 'src', 'index.ts'), 'utf-8')).toBe('export const x = 1;\n'); + expect(existsSync(join(src, 'node_modules', 'pkg', 'big.js'))).toBe(true); + }); + + it('skips node_modules and .git when copying', async () => { + const src = makeProject('app2'); + await ingestSources(workspace, [{ kind: 'folder', path: src }]); + const copied = join(workspace, 'app2'); + expect(existsSync(join(copied, 'node_modules'))).toBe(false); + expect(existsSync(join(copied, '.git'))).toBe(false); + // file count reflects only the real source files (index.ts + README.md). + const log = JSON.parse(readFileSync(join(workspace, '.iclaw-ingest.json'), 'utf-8')); + expect(log.results[0].files).toBe(2); + }); + + it('gives colliding basenames distinct targets', async () => { + const a = join(root, 'a', 'project'); + const b = join(root, 'b', 'project'); + mkdirSync(a, { recursive: true }); + mkdirSync(b, { recursive: true }); + writeFileSync(join(a, 'f.txt'), 'A'); + writeFileSync(join(b, 'f.txt'), 'B'); + const results = await ingestSources(workspace, [ + { kind: 'folder', path: a }, + { kind: 'folder', path: b }, + ]); + expect(results[0].target).toBe('project'); + expect(results[1].target).toBe('project-1'); + expect(readFileSync(join(workspace, 'project', 'f.txt'), 'utf-8')).toBe('A'); + expect(readFileSync(join(workspace, 'project-1', 'f.txt'), 'utf-8')).toBe('B'); + }); + + it('refuses a secret-bearing folder root (e.g. .ssh) without throwing', async () => { + const secret = join(root, '.ssh'); + mkdirSync(secret, { recursive: true }); + writeFileSync(join(secret, 'id_rsa'), 'PRIVATE'); + const results = await ingestSources(workspace, [{ kind: 'folder', path: secret }]); + expect(results[0].ok).toBe(false); + expect(results[0].error).toBeTruthy(); + // Nothing leaked into the sandbox. + expect(existsSync(join(workspace, '.ssh'))).toBe(false); + }); + + it('reports a missing folder as a failed (not thrown) result', async () => { + const results = await ingestSources(workspace, [ + { kind: 'folder', path: join(root, 'does-not-exist') }, + ]); + expect(results[0].ok).toBe(false); + }); +}); + +describe('describeIngest', () => { + it('summarizes copies and reassures about originals', () => { + const text = describeIngest([ + { kind: 'folder', source: '/home/me/app', ok: true, target: 'app', files: 12 }, + ]); + expect(text).toContain('original files are unchanged'); + expect(text).toContain('/workspace/app'); + }); + + it('returns empty for no sources', () => { + expect(describeIngest([])).toBe(''); + }); +}); diff --git a/test/unit/store.chats.test.ts b/test/unit/store.chats.test.ts index daf0111..5f51c07 100644 --- a/test/unit/store.chats.test.ts +++ b/test/unit/store.chats.test.ts @@ -15,7 +15,6 @@ describe('store.chats', () => { expect(typeof c.openclaw_session_id).toBe('string'); expect(c.openclaw_session_id.length).toBeGreaterThan(8); expect(c.shares_to_project).toBe(1); - expect(c.reasoning_mode).toBe('off'); }); it('list() returns chats newest-updated first', () => { @@ -104,16 +103,6 @@ describe('store.chats', () => { expect(chats.get(b.id)!.unread).toBe(1); }); - it('setReasoningMode persists allowed values', () => { - const c = chats.create('openclaw/default'); - chats.setReasoningMode(c.id, 'on'); - expect(chats.get(c.id)!.reasoning_mode).toBe('on'); - chats.setReasoningMode(c.id, 'stream'); - expect(chats.get(c.id)!.reasoning_mode).toBe('stream'); - chats.setReasoningMode(c.id, 'off'); - expect(chats.get(c.id)!.reasoning_mode).toBe('off'); - }); - it('SQLite trigger bumps chats.updated_at on message insert', () => { const c = chats.create('openclaw/default'); db.prepare("UPDATE chats SET updated_at = '2020-01-01 00:00:00' WHERE id = ?").run(c.id); diff --git a/test/unit/store.projectSkills.test.ts b/test/unit/store.projectSkills.test.ts new file mode 100644 index 0000000..0d4bf13 --- /dev/null +++ b/test/unit/store.projectSkills.test.ts @@ -0,0 +1,174 @@ +import { afterEach, describe, expect, it } from 'vitest'; +import { resetTestDb } from '../helpers/db'; +import { + projects, + projectSkills, + projectSkillSuggestions, + chats, + enrichSkillWithSourceChatTitle, +} from '../../src/services/store'; +import { + extractJsonObject, + parseReviewedSkills, +} from '../../src/services/projectSkills'; + +afterEach(() => resetTestDb()); + +describe('store.projectSkills', () => { + it('create() persists a project skill and serializes tags', () => { + const p = projects.create('P'); + const s = projectSkills.create({ + projectId: p.id, + name: 'deploy-flow', + description: 'How to deploy the service', + body: '# Deploy\nSteps...', + tags: ['ops', 'deploy'], + }); + expect(s.name).toBe('deploy-flow'); + expect(s.project_id).toBe(p.id); + expect(s.version).toBe(1); + expect(JSON.parse(s.tags!)).toEqual(['ops', 'deploy']); + }); + + it('create() supports global skills (project_id null) and listForProject merges them', () => { + const p = projects.create('P'); + projectSkills.create({ projectId: p.id, name: 'local-skill', description: 'd', body: 'b' }); + projectSkills.create({ projectId: null, name: 'global-skill', description: 'd', body: 'b' }); + const forProject = projectSkills.listForProject(p.id); + const names = forProject.map((s) => s.name).sort(); + expect(names).toEqual(['global-skill', 'local-skill']); + expect(projectSkills.listGlobal().map((s) => s.name)).toEqual(['global-skill']); + }); + + it('getByName() distinguishes project scope from global scope', () => { + const p = projects.create('P'); + projectSkills.create({ projectId: p.id, name: 'dup', description: 'proj', body: 'b' }); + projectSkills.create({ projectId: null, name: 'dup', description: 'glob', body: 'b' }); + expect(projectSkills.getByName(p.id, 'dup')?.description).toBe('proj'); + expect(projectSkills.getByName(null, 'dup')?.description).toBe('glob'); + }); + + it('update() bumps version + updated_at and can rewrite tags', () => { + const p = projects.create('P'); + const s = projectSkills.create({ projectId: p.id, name: 'n', description: 'd', body: 'b' }); + projectSkills.update(s.id, { description: 'd2', body: 'b2', tags: ['x'] }); + const after = projectSkills.get(s.id)!; + expect(after.version).toBe(2); + expect(after.description).toBe('d2'); + expect(after.body).toBe('b2'); + expect(JSON.parse(after.tags!)).toEqual(['x']); + }); + + it('listIndex() returns id/name/description only', () => { + const p = projects.create('P'); + projectSkills.create({ projectId: p.id, name: 'n', description: 'summary', body: 'long body' }); + const idx = projectSkills.listIndex(p.id); + expect(idx).toHaveLength(1); + expect(idx[0]).toMatchObject({ name: 'n', description: 'summary' }); + expect(idx[0]).not.toHaveProperty('body'); + }); + + it('remove() deletes the skill; project removal cascades skills', () => { + const p = projects.create('P'); + const s = projectSkills.create({ projectId: p.id, name: 'n', description: 'd', body: 'b' }); + projectSkills.remove(s.id); + expect(projectSkills.get(s.id)).toBeUndefined(); + + const p2 = projects.create('P2'); + projectSkills.create({ projectId: p2.id, name: 'n', description: 'd', body: 'b' }); + projects.remove(p2.id); + expect(projectSkills.listByProject(p2.id)).toHaveLength(0); + }); + + it('enrichSkillWithSourceChatTitle pulls the source chat title', () => { + const p = projects.create('P'); + const c = chats.create('openclaw/default', p.id); + chats.rename(c.id, 'My Chat'); + const s = projectSkills.create({ + projectId: p.id, + name: 'n', + description: 'd', + body: 'b', + sourceChatId: c.id, + }); + expect(enrichSkillWithSourceChatTitle(s).source_chat_title).toBe('My Chat'); + }); +}); + +describe('store.projectSkillSuggestions', () => { + it('insert() persists a suggestion with kind + untrusted flag', () => { + const p = projects.create('P'); + const c = chats.create('openclaw/default', p.id); + const sug = projectSkillSuggestions.insert({ + projectId: p.id, + chatId: c.id, + kind: 'new', + name: 'n', + description: 'd', + body: 'b', + untrusted: true, + assistantMessageId: null, + }); + expect(sug.kind).toBe('new'); + expect(sug.untrusted).toBe(1); + expect(projectSkillSuggestions.listByChat(c.id)).toHaveLength(1); + expect(projectSkillSuggestions.listByProject(p.id)).toHaveLength(1); + projectSkillSuggestions.remove(sug.id); + expect(projectSkillSuggestions.get(sug.id)).toBeUndefined(); + }); +}); + +describe('projectSkills reviewer parsing', () => { + it('extractJsonObject pulls a balanced object from fenced output', () => { + const raw = 'Sure!\n```json\n{"skills":[{"name":"a"}]}\n```\nDone'; + expect(extractJsonObject(raw)).toBe('{"skills":[{"name":"a"}]}'); + }); + + it('extractJsonObject returns null for NONE / no object', () => { + expect(extractJsonObject('NONE')).toBeNull(); + expect(extractJsonObject('no json here')).toBeNull(); + }); + + it('parseReviewedSkills accepts well-formed skills and kebab-cases names', () => { + const raw = JSON.stringify({ + skills: [ + { + action: 'new', + name: 'Deploy Flow', + description: 'How to deploy', + tags: ['ops'], + body: '# Deploy\nstep', + }, + ], + }); + const out = parseReviewedSkills(raw); + expect(out).toHaveLength(1); + expect(out[0].name).toBe('deploy-flow'); + expect(out[0].action).toBe('new'); + expect(out[0].tags).toEqual(['ops']); + }); + + it('parseReviewedSkills drops entries missing name/description/body', () => { + const raw = JSON.stringify({ + skills: [ + { action: 'new', name: 'ok', description: '', body: 'b' }, + { action: 'new', name: 'good', description: 'd', body: 'b' }, + ], + }); + const out = parseReviewedSkills(raw); + expect(out.map((s) => s.name)).toEqual(['good']); + }); + + it('parseReviewedSkills returns [] for empty skills array', () => { + expect(parseReviewedSkills('{"skills":[]}')).toEqual([]); + }); + + it('parseReviewedSkills keeps patch action + target', () => { + const raw = JSON.stringify({ + skills: [{ action: 'patch', target: 'Existing Skill', name: 'existing-skill', description: 'd', body: 'b' }], + }); + const out = parseReviewedSkills(raw); + expect(out[0].action).toBe('patch'); + expect(out[0].target).toBe('existing-skill'); + }); +}); diff --git a/test/unit/subtaskLlm.test.ts b/test/unit/subtaskLlm.test.ts new file mode 100644 index 0000000..593dca9 --- /dev/null +++ b/test/unit/subtaskLlm.test.ts @@ -0,0 +1,84 @@ +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; + +// Mock the three collaborators so we can assert routing without real network. +vi.mock('../../src/services/openRouter', () => ({ + openRouterEnabled: vi.fn(), + complete: vi.fn(), + isOpenRouterFailure: (e: unknown) => + /^openrouter:/i.test(e instanceof Error ? e.message : String(e)), +})); +vi.mock('../../src/services/config', () => ({ + loadOpenRouterConfig: () => ({ summaryModel: 'cheap/model' }), +})); +vi.mock('../../src/services/openclawWs', () => { + const runTurn = vi.fn(async (opts: { onEvent: (ev: { type: string; text: string }) => void }) => { + opts.onEvent({ type: 'text-final', text: 'openclaw-result' }); + }); + return { + openclawWs: { + createSession: vi.fn(async () => ({ key: 'k1' })), + runTurn, + deleteSession: vi.fn(async () => {}), + }, + }; +}); + +import { runSubtaskTurn } from '../../src/services/subtaskLlm'; +import { openRouterEnabled, complete } from '../../src/services/openRouter'; +import { openclawWs } from '../../src/services/openclawWs'; + +const mockEnabled = openRouterEnabled as unknown as ReturnType<typeof vi.fn>; +const mockComplete = complete as unknown as ReturnType<typeof vi.fn>; + +afterEach(() => vi.clearAllMocks()); + +describe('runSubtaskTurn routing', () => { + it('uses OpenRouter (cheap model) when configured', async () => { + mockEnabled.mockReturnValue(true); + mockComplete.mockResolvedValue('openrouter-result'); + + const out = await runSubtaskTurn('hi', { maxTokens: 100 }); + + expect(out).toBe('openrouter-result'); + expect(mockComplete).toHaveBeenCalledTimes(1); + expect(mockComplete.mock.calls[0][0]).toMatchObject({ + model: 'cheap/model', + maxTokens: 100, + }); + expect(openclawWs.createSession).not.toHaveBeenCalled(); + }); + + it('passes an optional system prompt through to OpenRouter', async () => { + mockEnabled.mockReturnValue(true); + mockComplete.mockResolvedValue('ok'); + await runSubtaskTurn('user text', { system: 'be terse' }); + const messages = mockComplete.mock.calls[0][0].messages; + expect(messages[0]).toEqual({ role: 'system', content: 'be terse' }); + expect(messages[1]).toEqual({ role: 'user', content: 'user text' }); + }); + + it('falls back to OpenClaw when OpenRouter is not configured', async () => { + mockEnabled.mockReturnValue(false); + const out = await runSubtaskTurn('hi'); + expect(out).toBe('openclaw-result'); + expect(mockComplete).not.toHaveBeenCalled(); + expect(openclawWs.createSession).toHaveBeenCalledTimes(1); + expect(openclawWs.deleteSession).toHaveBeenCalledTimes(1); + }); + + it('falls back to OpenClaw when the OpenRouter call throws', async () => { + mockEnabled.mockReturnValue(true); + mockComplete.mockRejectedValue(new Error('openrouter: HTTP 500')); + const out = await runSubtaskTurn('hi'); + expect(out).toBe('openclaw-result'); + expect(openclawWs.createSession).toHaveBeenCalledTimes(1); + }); + + it('returns OpenRouter output verbatim even when empty (no double-spend fallback)', async () => { + mockEnabled.mockReturnValue(true); + mockComplete.mockResolvedValue(''); + const out = await runSubtaskTurn('hi'); + expect(out).toBe(''); + expect(openclawWs.createSession).not.toHaveBeenCalled(); + }); +}); diff --git a/test/unit/toolLabels.test.ts b/test/unit/toolLabels.test.ts index bebf5b9..f448f74 100644 --- a/test/unit/toolLabels.test.ts +++ b/test/unit/toolLabels.test.ts @@ -1,26 +1,29 @@ import { describe, expect, it } from 'vitest'; import { toolActivityLabel, + toolActivityDetail, lifecycleActivityLabel, } from '../../src/services/toolLabels'; describe('toolActivityLabel', () => { + // Generic heuristics for gateway tool names (use names NOT in the exact + // iClaw-runtime switch, which is covered separately below). it('maps search-ish names', () => { - expect(toolActivityLabel('web_search')).toBe('Searching…'); + expect(toolActivityLabel('code_search')).toBe('Searching…'); expect(toolActivityLabel('grep')).toBe('Searching…'); expect(toolActivityLabel('lookup_lib')).toBe('Searching…'); expect(toolActivityLabel('BrowsePages')).toBe('Searching…'); }); it('maps edit-ish names', () => { - expect(toolActivityLabel('edit_file')).toBe('Editing file…'); + expect(toolActivityLabel('apply_changes')).toBe('Editing file…'); expect(toolActivityLabel('writeText')).toBe('Editing file…'); expect(toolActivityLabel('patch')).toBe('Editing file…'); expect(toolActivityLabel('save')).toBe('Editing file…'); }); it('maps read-ish names', () => { - expect(toolActivityLabel('read_file')).toBe('Reading file…'); + expect(toolActivityLabel('tail_log')).toBe('Reading file…'); expect(toolActivityLabel('cat')).toBe('Reading file…'); expect(toolActivityLabel('head')).toBe('Reading file…'); expect(toolActivityLabel('view_file')).toBe('Reading file…'); @@ -30,7 +33,24 @@ describe('toolActivityLabel', () => { expect(toolActivityLabel('bash')).toBe('Running command…'); expect(toolActivityLabel('exec')).toBe('Running command…'); expect(toolActivityLabel('shell')).toBe('Running command…'); - expect(toolActivityLabel('run_command')).toBe('Running command…'); + expect(toolActivityLabel('terminal_run')).toBe('Running command…'); + }); + + // iClaw runtime tools get specific labels (exact match, checked before the + // generic heuristics — so social_search doesn't collapse to "Searching…"). + it('maps iClaw runtime tools to specific labels', () => { + expect(toolActivityLabel('social_search')).toBe('Searching social media…'); + expect(toolActivityLabel('web_search')).toBe('Searching the web…'); + expect(toolActivityLabel('web_fetch')).toBe('Reading the web…'); + expect(toolActivityLabel('read_summary')).toBe('Skimming a file…'); + expect(toolActivityLabel('analyze_link')).toBe('Analyzing the link…'); + expect(toolActivityLabel('show_image')).toBe('Sharing an image…'); + expect(toolActivityLabel('search_files')).toBe('Searching files…'); + expect(toolActivityLabel('list_files')).toBe('Listing files…'); + expect(toolActivityLabel('read_file')).toBe('Reading a file…'); + expect(toolActivityLabel('write_file')).toBe('Writing a file…'); + expect(toolActivityLabel('edit_file')).toBe('Editing a file…'); + expect(toolActivityLabel('run_command')).toBe('Running a command…'); }); it('maps thinking-ish names', () => { @@ -52,6 +72,33 @@ describe('toolActivityLabel', () => { }); }); +describe('toolActivityDetail', () => { + it('pulls the query / command / path', () => { + expect(toolActivityDetail('web_search', { query: 'openclaw traffic', count: 5 })).toBe('openclaw traffic'); + expect(toolActivityDetail('run_command', { command: 'ls -la' })).toBe('ls -la'); + expect(toolActivityDetail('read_file', { path: '/a/b/c.ts' })).toBe('/a/b/c.ts'); + }); + + it('reduces a URL to host + path (trailing slash trimmed)', () => { + expect(toolActivityDetail('web_fetch', { url: 'https://github.com/Zijian-Ni/awesome-ai-agents-2026' })) + .toBe('github.com/Zijian-Ni/awesome-ai-agents-2026'); + expect(toolActivityDetail('web_fetch', { url: 'https://x.com/' })).toBe('x.com'); + }); + + it('returns undefined when there is nothing useful', () => { + expect(toolActivityDetail('show_image', {})).toBeUndefined(); + expect(toolActivityDetail('web_search', { query: ' ' })).toBeUndefined(); + expect(toolActivityDetail('web_fetch', null)).toBeUndefined(); + expect(toolActivityDetail('web_fetch', 'oops')).toBeUndefined(); + }); + + it('caps long detail at 70 chars', () => { + const d = toolActivityDetail('web_search', { query: 'a'.repeat(120) }); + expect(d).toHaveLength(70); + expect(d?.endsWith('…')).toBe(true); + }); +}); + describe('lifecycleActivityLabel', () => { it('maps known phases', () => { expect(lifecycleActivityLabel('thinking')).toBe('Thinking…'); diff --git a/test/unit/turnReply.test.ts b/test/unit/turnReply.test.ts index 933cdee..cb71cc8 100644 --- a/test/unit/turnReply.test.ts +++ b/test/unit/turnReply.test.ts @@ -10,6 +10,7 @@ import { describe, expect, it } from 'vitest'; import { extractAssistantText, extractSourceReplyFromMessageToolResult, + extractTurnUsage, resolveFromHistorySlice, sliceFromLastUser, type HistoryMessageLike, @@ -267,3 +268,45 @@ describe('resolveFromHistorySlice — chat #23 regression', () => { expect(resolveFromHistorySlice(slice)).toBe(samari); }); }); + +describe('extractTurnUsage', () => { + it('returns nulls when no assistant row carries usage', () => { + const slice: HistoryMessageLike[] = [ + { role: 'assistant', content: 'hi' }, + { role: 'toolResult', content: [] }, + ]; + expect(extractTurnUsage(slice)).toEqual({ tokens: null, cached: null }); + }); + + it('prefers an explicit total and reads cache-read tokens', () => { + const slice: HistoryMessageLike[] = [ + { role: 'user', content: 'q' }, + { role: 'assistant', content: 'a', usage: { total_tokens: 1234, cache_read_input_tokens: 200 } }, + ]; + expect(extractTurnUsage(slice)).toEqual({ tokens: 1234, cached: 200 }); + }); + + it('falls back to input + output when no total is present (camelCase)', () => { + const slice: HistoryMessageLike[] = [ + { role: 'assistant', content: 'a', usage: { inputTokens: 100, outputTokens: 23, cacheRead: 10 } }, + ]; + expect(extractTurnUsage(slice)).toEqual({ tokens: 123, cached: 10 }); + }); + + it('sums usage across the multiple assistant segments of a tool-loop turn', () => { + const slice: HistoryMessageLike[] = [ + { role: 'assistant', content: 'preamble', usage: { total_tokens: 100, cacheRead: 5 } }, + { role: 'toolResult', content: [] }, + { role: 'assistant', content: 'final', usage: { input_tokens: 40, output_tokens: 10 } }, + ]; + expect(extractTurnUsage(slice)).toEqual({ tokens: 150, cached: 5 }); + }); + + it('ignores non-object / malformed usage', () => { + const slice: HistoryMessageLike[] = [ + { role: 'assistant', content: 'a', usage: 'nope' }, + { role: 'assistant', content: 'b', usage: { foo: 'bar' } }, + ]; + expect(extractTurnUsage(slice)).toEqual({ tokens: null, cached: null }); + }); +}); diff --git a/test/unit/workContainerPaths.test.ts b/test/unit/workContainerPaths.test.ts new file mode 100644 index 0000000..77f359b --- /dev/null +++ b/test/unit/workContainerPaths.test.ts @@ -0,0 +1,119 @@ +import { describe, it, expect } from 'vitest'; + +import { + toWorkMounts, + hostToContainer, + containerToHost, + translateCommandPaths, +} from '../../packages/iclaw-runtime/src/work-container'; + +const WIN = 'win32' as NodeJS.Platform; +const POSIX = 'linux' as NodeJS.Platform; + +describe('toWorkMounts', () => { + it('assigns normalized /work/<n> paths and basename labels', () => { + const mounts = toWorkMounts([ + { path: '/Users/me/project', readonly: false }, + { path: '/Users/me/Downloads', readonly: true }, + ]); + expect(mounts[0]).toMatchObject({ + path: '/Users/me/project', + containerPath: '/work/0', + readonly: false, + label: 'project', + }); + expect(mounts[1]).toMatchObject({ containerPath: '/work/1', readonly: true, label: 'Downloads' }); + }); + + it('derives a label from a Windows path', () => { + const [m] = toWorkMounts([{ path: 'C:\\Users\\me\\app', readonly: false }]); + expect(m.label).toBe('app'); + expect(m.containerPath).toBe('/work/0'); + }); +}); + +describe('hostToContainer', () => { + const mounts = toWorkMounts([ + { path: '/Users/me/project', readonly: false }, + { path: '/Users/me/Downloads', readonly: true }, + ]); + + it('maps the folder root and nested paths', () => { + expect(hostToContainer('/Users/me/project', mounts, POSIX)).toBe('/work/0'); + expect(hostToContainer('/Users/me/project/src/a.ts', mounts, POSIX)).toBe('/work/0/src/a.ts'); + expect(hostToContainer('/Users/me/Downloads/x', mounts, POSIX)).toBe('/work/1/x'); + }); + + it('returns null for a path outside every mount', () => { + expect(hostToContainer('/etc/passwd', mounts, POSIX)).toBeNull(); + expect(hostToContainer('/Users/me/projectX/y', mounts, POSIX)).toBeNull(); + }); + + it('prefers the longest (nested) mount root', () => { + const nested = toWorkMounts([ + { path: '/a', readonly: false }, + { path: '/a/b', readonly: false }, + ]); + expect(hostToContainer('/a/b/c', nested, POSIX)).toBe('/work/1/c'); + }); + + it('maps Windows drive paths case-insensitively', () => { + const win = toWorkMounts([{ path: 'C:\\Users\\me\\app', readonly: false }]); + expect(hostToContainer('C:\\Users\\me\\app\\src\\x.ts', win, WIN)).toBe('/work/0/src/x.ts'); + expect(hostToContainer('c:/users/me/app/y', win, WIN)).toBe('/work/0/y'); + }); +}); + +describe('containerToHost', () => { + it('reverses the mapping with the host separator', () => { + const posix = toWorkMounts([{ path: '/Users/me/project', readonly: false }]); + expect(containerToHost('/work/0/src/a.ts', posix)).toBe('/Users/me/project/src/a.ts'); + expect(containerToHost('/work/0', posix)).toBe('/Users/me/project'); + + const win = toWorkMounts([{ path: 'C:\\Users\\me\\app', readonly: false }]); + expect(containerToHost('/work/0/src/x.ts', win)).toBe('C:\\Users\\me\\app\\src\\x.ts'); + }); +}); + +describe('translateCommandPaths', () => { + const mounts = toWorkMounts([{ path: '/Users/me/project', readonly: false }]); + + it('rewrites a host root + tail to its container path', () => { + expect(translateCommandPaths('cat /Users/me/project/a.txt', mounts, POSIX)).toBe( + 'cat /work/0/a.txt', + ); + expect(translateCommandPaths('cd /Users/me/project && ls', mounts, POSIX)).toBe( + 'cd /work/0 && ls', + ); + }); + + it('rewrites Windows roots whole, in both slash forms', () => { + const win = toWorkMounts([{ path: 'C:\\Users\\me\\app', readonly: false }]); + expect(translateCommandPaths('cat C:\\Users\\me\\app\\a.txt', win, WIN)).toBe('cat /work/0/a.txt'); + expect(translateCommandPaths('cat C:/Users/me/app/a.txt', win, WIN)).toBe('cat /work/0/a.txt'); + }); + + it('stops at shell metacharacters and leaves unrelated paths alone', () => { + expect(translateCommandPaths('cat /Users/me/project/a|wc -l', mounts, POSIX)).toBe( + 'cat /work/0/a|wc -l', + ); + expect(translateCommandPaths('ls /etc/hosts', mounts, POSIX)).toBe('ls /etc/hosts'); + }); + + it('does not match a root inside a longer sibling path', () => { + const ms = toWorkMounts([{ path: '/a', readonly: false }]); + // /abc is a different folder — must stay untouched. + expect(translateCommandPaths('ls /abc/x', ms, POSIX)).toBe('ls /abc/x'); + // The exact root (and nested) still map. + expect(translateCommandPaths('ls /a/x', ms, POSIX)).toBe('ls /work/0/x'); + expect(translateCommandPaths('cd /a && ls', ms, POSIX)).toBe('cd /work/0 && ls'); + }); + + it('prefers the longest mount root', () => { + const nested = toWorkMounts([ + { path: '/a', readonly: false }, + { path: '/a/b', readonly: false }, + ]); + expect(translateCommandPaths('cd /a/b/c', nested, POSIX)).toBe('cd /work/1/c'); + }); +}); diff --git a/views/chat.ejs b/views/chat.ejs index 4892503..49d9ba8 100644 --- a/views/chat.ejs +++ b/views/chat.ejs @@ -14,11 +14,15 @@ aria-label="Chat title" /> </form> + <% if (typeof devMode !== 'undefined' && devMode) { const _tokTotal = (chatMessages || []).reduce((a, m) => a + (Number(m.tokens) || 0), 0); %> + <span class="chat-token-total" id="chat-token-total" data-total="<%= _tokTotal %>" + title="Total tokens spent in this chat (dev mode)" <%= _tokTotal > 0 ? '' : 'hidden' %>><%= _tokTotal.toLocaleString() %> tok total</span> + <% } %> <div class="chat-header-tools"> <form class="agent-form header-field" method="post" action="/chats/<%= activeChat.id %>/agent"> <label class="header-field-label" for="chat-agent-select">Agent</label> - <select name="agent" id="chat-agent-select" onchange="this.form.submit()" aria-label="Agent"> + <select name="agent" id="chat-agent-select" onchange="this.form.requestSubmit()" aria-label="Agent"> <% agents.forEach(function (a) { %> <option value="<%= a.id %>" <%= a.id === activeChat.agent ? 'selected' : '' %>> <%= a.id %> @@ -30,83 +34,7 @@ </select> </form> - <label - class="reasoning-toggle header-field" - title="Show the model's reasoning / thinking inline. Type /reasoning on in chat to actually enable it on OpenClaw side." - aria-label="Reasoning" - > - <input - type="checkbox" - id="chat-reasoning-toggle" - <%= activeChat.reasoning_mode && activeChat.reasoning_mode !== 'off' ? 'checked' : '' %> - /> - <%- include('partials/header-tool-icon', { icon: 'reasoning' }) %> - <span class="header-tool-label">Reasoning</span> - </label> - - <% if (activeChat.project_id) { %> - <form - method="post" - action="/chats/<%= activeChat.id %>/shares" - class="share-form" - title="When on, after each assistant reply you get suggested facts to add to the project (you confirm each)." - > - <label - class="share-toggle" - title="Suggest facts for project" - aria-label="Suggest facts for project" - > - <input - type="checkbox" - name="shares" - value="1" - <%= activeChat.shares_to_project ? 'checked' : '' %> - onchange="this.form.submit()" - /> - <%- include('partials/header-tool-icon', { icon: 'facts' }) %> - <span class="header-tool-label">Suggest facts for project</span> - </label> - </form> - <% } %> - - <button - type="button" - class="stop-btn btn btn--ghost btn--sm header-action-btn" - id="stop-btn" - aria-label="Stop generation" - title="Stop this turn" - <%= isWorking ? '' : 'hidden' %> - > - <%- include('partials/header-tool-icon', { icon: 'stop' }) %> - <span class="header-tool-label">Stop</span> - </button> - - <% if (cloudShareBaseUrl) { %> - <button - type="button" - class="share-btn btn btn--ghost btn--sm header-action-btn" - id="share-btn" - data-cloud-base-url="<%= cloudShareBaseUrl %>" - title="Encrypted in your browser; the key is in the # part of the link. Optional password." - aria-label="Share this chat" - > - <%- include('partials/header-tool-icon', { icon: 'share' }) %> - <span class="header-tool-label">Share</span> - </button> - <% } %> - - <form method="post" action="/chats/<%= activeChat.id %>/delete" class="delete-form"> - <button - type="submit" - class="btn btn--ghost btn--sm header-action-btn chat-delete-btn" - onclick="return confirm('Delete this chat?')" - aria-label="Delete chat" - title="Delete this chat" - > - <%- include('partials/header-tool-icon', { icon: 'delete' }) %> - <span class="header-tool-label">Delete</span> - </button> - </form> + <%- include('partials/header-chat-tools') %> </div> </header> @@ -169,7 +97,11 @@ > <div class="messages-thread"> <% if (chatMessages.length === 0) { %> - <p class="muted empty-state">No messages yet. Say something.</p> + <% if (!Array.isArray(allProjects) || allProjects.length === 0) { %> + <%- include('partials/welcomeCard') %> + <% } else { %> + <p class="muted empty-state">iClaw - the safer way to work with powerful AI</p> + <% } %> <% } else { %> <% chatMessages.forEach(function (m) { %> <div class="msg <%= m.role %>" data-msg-id="<%= m.id %>"> @@ -220,6 +152,9 @@ <% }); %> </div> <% } %> + <% if (typeof devMode !== 'undefined' && devMode && m.tokens) { %> + <span class="msg-tokens" title="Tokens spent on this reply"><%= Number(m.tokens).toLocaleString() %> tok<% if (m.cached_tokens) { %> · <%= Number(m.cached_tokens).toLocaleString() %> cached<% } %></span> + <% } %> </div> <% }); %> <% } %> @@ -237,6 +172,7 @@ %> <div class="msg assistant streaming <%= _waitingClass %>" id="reload-placeholder"> <div class="role">assistant</div> + <div class="msg-body stream-body"></div> <div class="stream-status<%= _detail ? ' has-detail' : '' %>" <% if (_detail) { %> @@ -245,7 +181,6 @@ title="<%= _detail %>" <% } %> ><%= _act.label %></div> - <div class="msg-body stream-body"></div> </div> <% } %> </div> diff --git a/views/index.ejs b/views/index.ejs index bdbd1d9..6e77fd2 100644 --- a/views/index.ejs +++ b/views/index.ejs @@ -32,37 +32,22 @@ </select> </form> - <button - type="button" - class="stop-btn btn btn--ghost btn--sm header-action-btn" - id="stop-btn" - aria-label="Stop generation" - title="Stop this turn" - hidden - > - <%- include('partials/header-tool-icon', { icon: 'stop' }) %> - <span class="header-tool-label">Stop</span> - </button> + <%- include('partials/header-chat-tools', { draftHeader: true }) %> - <span - class="gateway <%= gatewayStatus %>" - id="gateway-badge" - data-base-url="<%= openclawBaseUrl %>" - title="<%= openclawBaseUrl %>" - > - OpenClaw: <%= gatewayStatus === 'ok' ? 'connected' : (gatewayStatus === 'degraded' ? 'unreachable' : 'off') %> - </span> </div> </header> + <% const _firstChat = typeof isFirstChat !== 'undefined' && isFirstChat; %> + <% const _noProjects = !Array.isArray(allProjects) || allProjects.length === 0; %> <div class="draft-body is-picking" id="draft-body" + <%- (_firstChat || _noProjects) ? 'data-auto-none="1"' : '' %> > <div class="draft-pick-stage" id="draft-pick-stage"> - <% if (agentsError) { %> - <%- include('partials/gatewayError', { agentsError, gatewayUp, openclawBaseUrl }) %> - <% } else { %> + <%# No model connected (skipped onboarding, no OpenClaw) is no longer a + dead end here — we show the normal picker + composer and re-offer the + connect choice when the user actually tries to send (connectModal). %> <section class="project-pick" id="project-pick" @@ -71,6 +56,7 @@ data-initial-project-id="<%= preselectedProject ? String(preselectedProject.id) : '' %>" > <h2 class="project-pick-title" id="project-pick-title">Choose a project</h2> + <p class="project-pick-hint">Press <kbd>Space</kbd> to select No project</p> <div class="project-pick-grid"> <button type="button" @@ -101,27 +87,35 @@ <% }); %> </div> </section> - <% } %> </div> <section class="messages" id="messages" data-chat-id="" data-draft="1"> <div class="messages-thread"> + <% if (_firstChat) { %> + <%- include('partials/welcomeCard') %> + <% } else { %> <p class="muted empty-state" id="draft-empty-hint" hidden>Write your first message.</p> + <% } %> </div> </section> <section class="queue" id="queue" aria-live="polite"></section> - <% if (!agentsError) { %> <div id="draft-composer-wrap" class="draft-composer-wrap" hidden> - <%- include('partials/composer') %> + <%- include('partials/composer', { gatewayOk: gatewayStatus === 'ok' }) %> </div> - <% } %> </div> + <% if (cloudShareBaseUrl) { %> + <%- include('partials/share-modal') %> + <% } %> + <script src="/js/vendor/marked.min.js"></script> <script src="/js/vendor/highlight.min.js"></script> <script src="/js/iclaw.js?v=1"></script> + <% if (cloudShareBaseUrl) { %> + <script src="/js/share.js" defer></script> + <% } %> </main> </div> diff --git a/views/partials/composer.ejs b/views/partials/composer.ejs index dea5d7c..5b9611e 100644 --- a/views/partials/composer.ejs +++ b/views/partials/composer.ejs @@ -37,7 +37,36 @@ }) %> <% }); %></div> -<form id="send-form" class="composer"> +<% + // Mode selector data. Server passes the enabled modes (config-driven) and + // the default. Fall back gracefully if a render path forgot the locals so + // the composer never breaks — the selector just hides. + const _modes = + typeof chatModes !== 'undefined' && Array.isArray(chatModes) ? chatModes : []; + const _defaultMode = + typeof defaultChatMode !== 'undefined' && defaultChatMode ? defaultChatMode : 'execute'; + const _defaultModeDef = + _modes.find(function (m) { return m.id === _defaultMode; }) || _modes[0] || null; + // The mode this chat was last used in (server-derived from its last message). + // Empty for new chats. Used to pre-select the picker so switching chats keeps + // each chat's own mode instead of resetting to the default. + const _chatCurrentMode = + typeof chatCurrentMode !== 'undefined' && chatCurrentMode ? chatCurrentMode : ''; + const _curModeDef = _modes.find(function (m) { return m.id === _chatCurrentMode; }); + // Don't pre-select a locked mode (e.g. a chat last used in Work after the key + // was removed) — fall back to the default, which is always usable. + const _initialMode = + _curModeDef && _curModeDef.available !== false ? _chatCurrentMode : _defaultMode; + const _initialModeDef = + _modes.find(function (m) { return m.id === _initialMode; }) || _defaultModeDef; +%> +<% + // Whether Full Power (Execute → OpenClaw) is usable right now. Stamped here so + // the JS gating works on every composer surface (the gateway badge only exists + // on the new-chat header). Defaults to available when a render path omits it. + const _gatewayOk = typeof gatewayOk !== 'undefined' ? !!gatewayOk : true; +%> +<form id="send-form" class="composer" data-gateway-ok="<%= _gatewayOk ? '1' : '0' %>"> <div id="composer-reply-bar" class="composer-reply-bar" hidden> <div class="composer-reply-bar-inner"> <span class="composer-reply-accent" aria-hidden="true"></span> @@ -63,66 +92,172 @@ </svg> <span>Drop file here</span> </div> - <input - type="file" - id="composer-file-input" - class="composer-file-input" - multiple - hidden - /> - <div - id="composer-attach-menu" - class="menu composer-attach-menu" - role="menu" - aria-label="Attachments" - hidden - > - <button type="button" class="menu-item composer-attach-menu-item" data-attach-pick="file" role="menuitem"> - <span class="menu-item__title">File</span> - </button> - <button type="button" class="menu-item composer-attach-menu-item" data-attach-pick="secret" role="menuitem"> - <span class="menu-item__title">Secret</span> - </button> + <input type="file" id="composer-file-input" class="composer-file-input" multiple hidden /> + <!-- menus stay here so JS can position them relative to the field --> + <div id="composer-attach-menu" class="menu composer-attach-menu" role="menu" aria-label="Attachments" hidden> + <button type="button" class="menu-item composer-attach-menu-item" data-attach-pick="file" role="menuitem"><span class="menu-item__title">File</span></button> + <button type="button" class="menu-item composer-attach-menu-item" data-attach-pick="secret" role="menuitem"><span class="menu-item__title">Secret</span></button> </div> - <div - id="composer-secret-pick-menu" - class="menu composer-secret-pick-menu" - role="menu" - aria-label="Choose secret" - hidden - ></div> - <button - type="button" - class="composer-attach" - id="composer-attach-btn" - aria-label="Add attachment" - title="File or secret (also drag & drop and paste)" - aria-haspopup="menu" - aria-expanded="false" - aria-controls="composer-attach-menu" - > - <svg xmlns="http://www.w3.org/2000/svg" width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.25" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"> - <path d="M21.44 11.05l-9.19 9.19a6 6 0 01-8.49-8.49l8.57-8.57A4 4 0 1117.95 8.83l-8.59 8.57a2 2 0 01-2.83-2.83l8.49-8.49"/> - </svg> - </button> + <div id="composer-secret-pick-menu" class="menu composer-secret-pick-menu" role="menu" aria-label="Choose secret" hidden></div> + + <!-- Textarea — full width, top --> <div class="composer-input-stack"> <textarea id="composer-input" name="content" - placeholder="Message…" + placeholder="Ask anything" rows="1" ></textarea> + <!-- Replaces the textarea when Full Power is selected but OpenClaw is off. + The toolbar (incl. the mode switcher) stays visible so the user can + switch mode right below this message. --> + <div class="composer-exec-msg" id="composer-exec-msg" aria-hidden="true"> + OpenClaw is off. Turn it on, or switch mode below. + </div> + <!-- Replaces the textarea when a Docker-required mode (Safe work) is + selected but Docker isn't running. The Install/Start button text and + action are set by syncDockerAvailability in iclaw.js. --> + <div class="composer-docker-msg" id="composer-docker-msg" aria-hidden="true"> + <span id="composer-docker-msg-text">Safe work needs Docker.</span> + <button type="button" class="composer-docker-btn" id="composer-docker-action">Install Docker</button> + <span class="composer-docker-size" id="composer-docker-size"></span> + <span class="composer-docker-or">or switch mode below.</span> + </div> + <% if (typeof sttEnabled !== 'undefined' && sttEnabled) { %> + <!-- Voice recording bar — replaces the textarea while the mic is held. + Pipeline after release is unchanged (audio → /api/stt → transcript). --> + <div class="composer-recording" id="composer-recording" hidden aria-hidden="true"> + <span class="composer-recording__dot" aria-hidden="true"></span> + <span class="composer-recording__time" id="composer-recording-time">00:00.00</span> + <canvas class="composer-recording__wave" id="composer-recording-wave" aria-hidden="true"></canvas> + <button type="button" class="composer-recording__cancel" id="composer-recording-cancel">Cancel</button> + <span class="composer-recording__hint" id="composer-recording-hint">‹ slide to cancel · slide up to lock</span> + </div> + <% } %> </div> - <button - type="submit" - class="composer-send" - id="composer-send-btn" - aria-label="Send (hold to schedule)" - > - <svg xmlns="http://www.w3.org/2000/svg" width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.25" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"> - <path d="M12 19V5M5 12l7-7 7 7"/> + + <!-- Toolbar — bottom row --> + <div class="composer-toolbar"> + <div class="composer-toolbar__left"> + <!-- Attach --> + <button + type="button" + class="composer-toolbar-btn composer-attach" + id="composer-attach-btn" + aria-label="Add attachment" + title="File or secret (also drag & drop and paste)" + aria-haspopup="menu" + aria-expanded="false" + aria-controls="composer-attach-menu" + > + <svg xmlns="http://www.w3.org/2000/svg" width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.25" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"> + <path d="M21.44 11.05l-9.19 9.19a6 6 0 01-8.49-8.49l8.57-8.57A4 4 0 1117.95 8.83l-8.59 8.57a2 2 0 01-2.83-2.83l8.49-8.49"/> + </svg> + </button> + + <!-- Mode selector --> + <% if (_modes.length > 1 && _defaultModeDef) { %> + <div class="composer-modes composer-toolbar-modes" id="composer-modes" data-default-mode="<%= _defaultMode %>" data-chat-mode="<%= _chatCurrentMode %>"> + <button + type="button" + class="composer-toolbar-btn composer-mode-btn" + id="composer-mode-btn" + data-mode="<%= _initialMode %>" + aria-haspopup="menu" + aria-expanded="false" + aria-controls="composer-mode-menu" + title="<%= _initialModeDef.description %>" + > + <span class="composer-mode-btn__label" id="composer-mode-label"><%= _initialModeDef.label %></span> + <svg xmlns="http://www.w3.org/2000/svg" width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"> + <polyline points="6 9 12 15 18 9"/> + </svg> + </button> + <div id="composer-mode-menu" class="menu composer-mode-menu" role="menu" aria-label="Message mode" hidden> + <% _modes.forEach(function (m) { %> + <% if (m.ephemeral) { %><div class="composer-mode-menu-sep" role="separator" aria-hidden="true"></div><% } %> + <button type="button" class="menu-item composer-mode-menu-item<%= m.available === false ? ' is-unavailable' : '' %>" role="menuitemradio" data-mode="<%= m.id %>" data-requires-docker="<%= m.requiresDocker ? '1' : '0' %>" data-requires-key="<%= m.available === false ? '1' : '0' %>" aria-checked="<%= m.id === _initialMode ? 'true' : 'false' %>"> + <span class="menu-item__title"><%= m.label %></span> + <span class="composer-mode-menu-item__desc"><%= m.description %></span> + </button> + <% }); %> + </div> + </div> + <% } %> + + <!-- Secure Mode network toggle --> + <button + type="button" + class="composer-toolbar-btn composer-network-toggle" + id="composer-network-toggle-btn" + aria-label="Toggle network access" + title="Network is off - click to enable" + data-network="off" + hidden + > + <svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.25" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"> + <circle cx="12" cy="12" r="10"/> + <line x1="2" y1="12" x2="22" y2="12"/> + <path d="M12 2a15.3 15.3 0 0 1 4 10 15.3 15.3 0 0 1-4 10 15.3 15.3 0 0 1-4-10 15.3 15.3 0 0 1 4-10z"/> + </svg> + </button> + + <!-- Work Mode folders --> + <button + type="button" + class="composer-toolbar-btn composer-work-folders" + id="composer-work-folders-btn" + aria-label="Work Mode folders" + title="Configure allowed folders for Work Mode" + hidden + > + <svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.25" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"> + <path d="M22 19a2 2 0 0 1-2 2H4a2 2 0 0 1-2-2V5a2 2 0 0 1 2-2h5l2 3h9a2 2 0 0 1 2 2z"/> + </svg> + <span class="composer-work-folders__count" id="composer-work-folders-count"></span> + </button> + </div> + + <div class="composer-toolbar__right"> + <!-- Mic --> + <% if (typeof sttEnabled !== 'undefined' && sttEnabled) { %> + <button type="button" class="composer-mic" id="composer-mic-btn" data-state="idle" aria-label="Record voice" aria-pressed="false" title="Record voice"> + <svg class="composer-mic__idle" xmlns="http://www.w3.org/2000/svg" width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.25" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"> + <path d="M12 2a3 3 0 0 0-3 3v7a3 3 0 0 0 6 0V5a3 3 0 0 0-3-3z"/> + <path d="M19 10v2a7 7 0 0 1-14 0v-2"/> + <line x1="12" y1="19" x2="12" y2="22"/> + </svg> + <svg class="composer-mic__busy" xmlns="http://www.w3.org/2000/svg" width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5" stroke-linecap="round" aria-hidden="true"> + <path d="M21 12a9 9 0 1 1-6.219-8.56"/> + </svg> + <svg class="composer-mic__send" xmlns="http://www.w3.org/2000/svg" width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"> + <path d="M12 19V5M5 12l7-7 7 7"/> + </svg> + </button> + <% } %> + + <!-- Send --> + <button type="submit" class="composer-send" id="composer-send-btn" aria-label="Send (hold to schedule)"> + <svg xmlns="http://www.w3.org/2000/svg" width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.25" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"> + <path d="M12 19V5M5 12l7-7 7 7"/> + </svg> + </button> + </div> + </div> + + <% if (typeof sttEnabled !== 'undefined' && sttEnabled) { %> + <!-- Floating lock indicator — shown while recording; fills as you drag up. + Positioned over the mic by JS. --> + <div class="composer-lock-hint" id="composer-lock-hint" hidden aria-hidden="true"> + <svg class="composer-lock-hint__chev" xmlns="http://www.w3.org/2000/svg" width="13" height="13" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"> + <polyline points="18 15 12 9 6 15"/> + </svg> + <svg class="composer-lock-hint__lock" xmlns="http://www.w3.org/2000/svg" width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"> + <rect x="3" y="11" width="18" height="11" rx="2"/> + <path d="M7 11V7a5 5 0 0 1 10 0v4"/> </svg> - </button> + </div> + <% } %> <% if (typeof sendHintShow !== 'undefined' && sendHintShow) { %> <!-- Discovery pill — surfaces the long-press menu (scheduled msg / create @@ -210,6 +345,24 @@ </div> </div> </div> + + <!-- Secure Mode workspace indicator — shares the bottom row with the secret UI --> + <div class="secure-workspace-bar" id="secure-workspace-bar" hidden> + <span class="secure-workspace-bar__label">Secure workspace</span> + <span class="secure-workspace-bar__size" id="secure-workspace-size" hidden></span> + <span class="secure-workspace-bar__sep" id="secure-workspace-sep">·</span> + <span class="secure-workspace-bar__ttl" id="secure-workspace-ttl"></span> + <button type="button" class="secure-workspace-bar__change" id="secure-workspace-change">change</button> + <button type="button" class="secure-workspace-bar__action" id="secure-workspace-export" title="Copy the sandbox out to a folder on your computer">Export</button> + <button type="button" class="secure-workspace-bar__action" id="secure-workspace-apply" title="Copy the sandbox's new/changed files back to your original folders">Apply changes</button> + <button type="button" class="secure-workspace-bar__destroy" id="secure-workspace-destroy" title="Delete the sandbox copy and everything in it">Destroy</button> + <!-- Anchored to the bar so it pops up in place instead of shifting layout --> + <div class="secure-ttl-menu menu" id="secure-ttl-menu" role="menu" hidden> + <button type="button" class="menu-item" data-ttl="1" role="menuitem">1 day</button> + <button type="button" class="menu-item" data-ttl="7" role="menuitem">7 days</button> + <button type="button" class="menu-item" data-ttl="30" role="menuitem">30 days</button> + </div> + </div> <div class="composer-secret-modal" id="composer-secret-modal" hidden> <div class="composer-secret-modal__backdrop" id="composer-secret-modal-backdrop"></div> <div class="composer-secret-modal__panel" role="dialog" aria-modal="true" aria-labelledby="composer-secret-modal-title"> @@ -289,3 +442,25 @@ </div> </div> </div> + +<!-- Work Mode folders modal --> +<div class="work-folders-modal" id="work-folders-modal" hidden> + <div class="work-folders-modal__backdrop" id="work-folders-backdrop"></div> + <div class="work-folders-modal__panel" role="dialog" aria-modal="true" aria-labelledby="work-folders-title"> + <header class="work-folders-modal__head"> + <div class="work-folders-modal__head-row"> + <div> + <h3 class="work-folders-modal__title" id="work-folders-title">Work Mode folders</h3> + <p class="work-folders-modal__hint muted">AI can only access these folders. New folders are read-only — click the access pill to allow writing.</p> + </div> + <button type="button" class="btn btn--primary btn--sm" id="work-folders-browse-btn"> + <svg xmlns="http://www.w3.org/2000/svg" width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.25" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M22 19a2 2 0 0 1-2 2H4a2 2 0 0 1-2-2V5a2 2 0 0 1 2-2h5l2 3h9a2 2 0 0 1 2 2z"/></svg> + Browse + </button> + </div> + </header> + <ul class="work-folders-list" id="work-folders-list"></ul> + </div> +</div> + +<%- include('connectModal') %> diff --git a/views/partials/connectModal.ejs b/views/partials/connectModal.ejs new file mode 100644 index 0000000..5e916ad --- /dev/null +++ b/views/partials/connectModal.ejs @@ -0,0 +1,47 @@ +<%# Connect-a-model chooser. Shown when someone who skipped onboarding (no + OpenRouter key, no reachable OpenClaw) actually tries to send — instead of a + dead-end error we re-offer the onboarding choice. Toggled by iclaw.js + (openConnectChooser) from the composer submit guard. %> +<div + class="connect-modal" + id="connect-modal" + hidden + role="dialog" + aria-modal="true" + aria-labelledby="connect-modal-title" +> + <div class="connect-modal__backdrop" data-connect-close aria-hidden="true"></div> + <div class="connect-modal__card onb-card-shell" role="document"> + <button type="button" class="connect-modal__close" data-connect-close aria-label="Close">×</button> + + <h2 class="onb-title" id="connect-modal-title">Connect a model to start</h2> + <p class="onb-sub">iClaw needs an AI model to reply — connect one to begin.</p> + + <button type="button" class="onb-option" id="connect-pick-openrouter"> + <span class="onb-option-main"> + <span class="onb-option-name">Connect OpenRouter</span> + <span class="onb-option-desc"> + Pay as you go - about $5 lasts a long time, and we’ll guide you + </span> + </span> + <span class="onb-option-chevron" aria-hidden="true">›</span> + </button> + + <button type="button" class="onb-option onb-option--disabled" disabled> + <span class="onb-option-main"> + <span class="onb-option-name"> + Use iClaw credits + <span class="onb-badge">Coming soon</span> + </span> + <span class="onb-option-desc"> + No setup - we handle the AI for you + </span> + </span> + </button> + + <p class="connect-modal__note"> + Already running OpenClaw? Switch to <strong>Full Power</strong> and start it + from the sidebar. + </p> + </div> +</div> diff --git a/views/partials/head.ejs b/views/partials/head.ejs index 0e8286c..074eb91 100644 --- a/views/partials/head.ejs +++ b/views/partials/head.ejs @@ -17,7 +17,11 @@ <meta name="iclaw-ra-relay-binding" content="<%= typeof raRelayBindingB64url !== 'undefined' ? raRelayBindingB64url : '' %>" /> <script type="module"> window.__iclawRaE2eBoot = import('/js/ra-e2e-transport.mjs?v=ra-gate-9').then(function (m) { - return m.installRaE2eTransport(); + window.iclawE2eNavigate = m.e2eNavigate; + return Promise.resolve(m.installRaE2eTransport()).then(function (ok) { + m.setupE2eSpaNavigation(); + return ok; + }); }); </script> <% } %> @@ -26,6 +30,7 @@ window.__ICLAW_PROJECT_LOGO_EMOJIS__ = <%- JSON.stringify(projectLogoEmojis || []) %>; window.__ICLAW_PROJECTS__ = <%- JSON.stringify(typeof projectsMini !== 'undefined' ? projectsMini : []) %>; window.__ICLAW_SCHEDULED_CHAT_COUNTS__ = <%- JSON.stringify(typeof scheduledChatCounts !== 'undefined' ? scheduledChatCounts : {}) %>; + window.__ICLAW_DEV__ = <%- JSON.stringify(typeof devMode !== 'undefined' ? !!devMode : false) %>; </script> </head> <body> diff --git a/views/partials/header-chat-tools.ejs b/views/partials/header-chat-tools.ejs new file mode 100644 index 0000000..1c414aa --- /dev/null +++ b/views/partials/header-chat-tools.ejs @@ -0,0 +1,84 @@ +<% + // Shared chat-header tool cluster (suggest-facts / stop / share / + // delete). Rendered live in chat.ejs and, in a dormant "draft" form, in + // index.ejs (the new-chat screen). On the draft the chat row doesn't exist + // yet, so the id-bound controls render hidden + action-less; adoptDraftChat() + // in iclaw.js reveals them and fills in the /chats/:id actions the moment the + // first message creates the row. Keeping ONE copy of this markup is what stops + // a header tool from silently going missing on freshly-started chats. + const _draft = typeof draftHeader !== 'undefined' && draftHeader; + const _cid = _draft ? '' : activeChat.id; + const _hasProject = !_draft && Boolean(activeChat.project_id); + const _sharesOn = !_draft && activeChat.shares_to_project; + const _working = typeof isWorking !== 'undefined' && isWorking; +%> + <% if (_draft || _hasProject) { %> + <form + method="post" + action="<%= _draft ? '' : '/chats/' + _cid + '/shares' %>" + class="share-form" + title="When on, after each assistant reply you get suggested facts to add to the project (you confirm each)." + <%= _draft ? 'hidden' : '' %> + > + <label + class="share-toggle" + title="Suggest facts for project" + aria-label="Suggest facts for project" + > + <input + type="checkbox" + name="shares" + value="1" + <%= _sharesOn ? 'checked' : '' %> + onchange="this.form.requestSubmit()" + /> + <%- include('header-tool-icon', { icon: 'facts' }) %> + <span class="header-tool-label">Suggest facts for project</span> + </label> + </form> + <% } %> + + <button + type="button" + class="stop-btn btn btn--ghost btn--sm header-action-btn" + id="stop-btn" + aria-label="Stop generation" + title="Stop this turn" + <%= _working ? '' : 'hidden' %> + > + <%- include('header-tool-icon', { icon: 'stop' }) %> + <span class="header-tool-label">Stop</span> + </button> + + <% if (cloudShareBaseUrl) { %> + <button + type="button" + class="share-btn btn btn--ghost btn--sm header-action-btn" + id="share-btn" + data-cloud-base-url="<%= cloudShareBaseUrl %>" + title="Encrypted in your browser; the key is in the # part of the link. Optional password." + aria-label="Share this chat" + <%= _draft ? 'hidden' : '' %> + > + <%- include('header-tool-icon', { icon: 'share' }) %> + <span class="header-tool-label">Share</span> + </button> + <% } %> + + <form + method="post" + action="<%= _draft ? '' : '/chats/' + _cid + '/delete' %>" + class="delete-form" + <%= _draft ? 'hidden' : '' %> + > + <button + type="submit" + class="btn btn--ghost btn--sm header-action-btn chat-delete-btn" + onclick="return confirm('Delete this chat?')" + aria-label="Delete chat" + title="Delete this chat" + > + <%- include('header-tool-icon', { icon: 'delete' }) %> + <span class="header-tool-label">Delete</span> + </button> + </form> diff --git a/views/partials/header-tool-icon.ejs b/views/partials/header-tool-icon.ejs index 9f55f86..28d2d4e 100644 --- a/views/partials/header-tool-icon.ejs +++ b/views/partials/header-tool-icon.ejs @@ -15,11 +15,6 @@ <line x1="10" y1="11" x2="10" y2="17" /> <line x1="14" y1="11" x2="14" y2="17" /> </svg> -<% } else if (_icon === 'reasoning') { %> -<svg class="header-tool-icon" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true" focusable="false"> - <path d="M12 2a7 7 0 0 1 7 7c0 2.5-1.2 4.7-3 6.1V18a2 2 0 0 1-2 2h-4a2 2 0 0 1-2-2v-2.9A7 7 0 0 1 5 9a7 7 0 0 1 7-7z" /> - <line x1="9" y1="22" x2="15" y2="22" /> -</svg> <% } else if (_icon === 'facts') { %> <svg class="header-tool-icon" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true" focusable="false"> <path d="M14 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V8z" /> diff --git a/views/partials/sidebar.ejs b/views/partials/sidebar.ejs index 93560fb..7bba4b6 100644 --- a/views/partials/sidebar.ejs +++ b/views/partials/sidebar.ejs @@ -11,7 +11,7 @@ <div class="sidebar-top"> <div class="sidebar-toolbar"> <a - href="/settings" + href="/settings/voice-ask" class="sidebar-settings-btn<%= typeof activeSettings !== 'undefined' && activeSettings ? ' active' : '' %>" aria-label="Settings" title="Settings" @@ -22,6 +22,8 @@ </svg> </a> <a href="/" class="new-chat-btn <%= !activeChat && !activeProject && !(typeof activeProjectsList !== 'undefined' && activeProjectsList) ? 'active' : '' %>">New chat</a> + <% /* Search only earns its place once there are many chats — declutter for new users. */ %> + <% if (chats.length >= 15) { %> <div class="sidebar-search-slot"> <button type="button" @@ -53,13 +55,17 @@ </button> </div> </div> + <% } %> </div> <p id="sidebar-search-status" class="sidebar-search-status" hidden aria-live="polite"></p> <div class="sidebar-projects-nav"> + <% /* Projects surfaces once a user is established (>=5 chats) or already has one. */ %> + <% if (chats.length >= 5 || _projectsList.length > 0) { %> <a href="/projects" class="sidebar-projects-btn<%= typeof activeProjectsList !== 'undefined' && activeProjectsList ? ' active' : '' %>" >Projects</a> + <% } %> <% if (typeof hasAnyTasks !== 'undefined' && hasAnyTasks) { const _taskSignals = typeof taskStatusSignals !== 'undefined' ? taskStatusSignals : {}; const _taskDotHuman = _taskSignals.needsHuman; @@ -142,9 +148,9 @@ <% } %> <% if (chats.length === 0 && _projectsList.length === 0) { %> - <p class="muted empty-list">No projects or chats yet.</p> + <p class="muted empty-list">Your chats will appear here</p> <% } else if (chats.length === 0) { %> - <p class="muted empty-list">No chats yet. Click <strong>+ New chat</strong> and pick a project on the home screen.</p> + <p class="muted empty-list">Your chats will appear here — hit <strong>+ New chat</strong> to start</p> <% } %> <div id="sidebar-list-tail" class="sidebar-list-tail" aria-hidden="true"></div> diff --git a/views/partials/welcomeCard.ejs b/views/partials/welcomeCard.ejs new file mode 100644 index 0000000..529328c --- /dev/null +++ b/views/partials/welcomeCard.ejs @@ -0,0 +1,22 @@ +<%# Conversational welcome + scenario chips. Shown to users who haven't set up + any project yet (first chat, or any empty chat while no project exists) in + place of the terse one-liner. Carries `empty-state` so clearEmptyState() + removes it on the first message; the chips are wired by initWelcomeChips. %> +<div class="welcome-card empty-state" id="welcome-card"> + <div class="welcome-avatar" aria-hidden="true">iC</div> + <div class="welcome-bubble"> + <p class="welcome-greeting">Hi - I’m iClaw</p> + <p class="welcome-lead">From now on, you don’t have to choose between convenience, power and safety</p> + <ul class="welcome-list"> + <li>Unlike other agents, I go online inside an isolated space - so a dangerous site can’t do any harm</li> + <li>Other agents want access to your whole computer. I only work where you let me</li> + <li>And our developers are 70% coffee - so we built a really efficient system that saves your tokens and your money</li> + </ul> + <div class="welcome-chips" role="group" aria-label="Suggestions to get started"> + <button type="button" class="welcome-chip" data-prompt="Open an unknown website safely">Open an unknown website safely</button> + <button type="button" class="welcome-chip" data-prompt="Check a ZIP file safely, isolated from my computer">Check a ZIP file safely, isolated from my computer</button> + <button type="button" class="welcome-chip" data-prompt="Check this GitHub repo in a sandbox">Check this GitHub repo in a sandbox</button> + <button type="button" class="welcome-chip" data-prompt="Summarize this 2-hour YouTube video for $0.0005">Summarize this 2-hour YouTube video for $0.0005</button> + </div> + </div> +</div> diff --git a/views/project.ejs b/views/project.ejs index 8556143..b84956f 100644 --- a/views/project.ejs +++ b/views/project.ejs @@ -119,6 +119,18 @@ data-tab-base="Memory" data-tab-count="<%= facts.length %>" >Memory</button> + <button + type="button" + class="project-tab" + role="tab" + id="project-tab-skills" + aria-controls="project-panel-skills" + aria-selected="false" + tabindex="-1" + data-project-tab="skills" + data-tab-base="Skills" + data-tab-count="<%= (skills || []).length %>" + >Skills</button> <button type="button" class="project-tab" @@ -231,6 +243,58 @@ </div> </div> + <div + id="project-panel-skills" + class="project-panel" + role="tabpanel" + aria-labelledby="project-tab-skills" + hidden + > + <div class="project-sheet project-sheet--flush project-sheet--stacked"> + <ul id="skills-list" class="project-chats"> + <% (skills || []).forEach(function (s) { %> + <li class="skill" data-skill-id="<%= s.id %>" data-skill-version="<%= s.version %>"> + <div class="project-row-head muted"> + <% if (s.project_id === null) { %> + <span class="skill-scope-badge" title="Available to every project">Global</span> + <% } %> + <% if (s.source_chat_id) { %> + <a href="/chats/<%= s.source_chat_id %>" class="project-chat-source"><%= (s.source_chat_title || '').trim() || 'Chat' %></a> + <% } else { %> + <span>—</span> + <% } %> + </div> + <input + class="skill-name" + aria-label="Skill name" + value="<%= s.name %>" + spellcheck="false" + /> + <textarea + class="skill-description" + aria-label="Skill summary" + rows="2" + ><%= s.description %></textarea> + <details class="skill-body-details"> + <summary>View / edit procedure</summary> + <textarea + class="skill-body" + aria-label="Skill procedure (SKILL.md)" + rows="10" + ><%= s.body %></textarea> + </details> + <div class="fact-meta"> + <button type="button" class="fact-delete skill-delete" aria-label="Remove skill">Remove</button> + </div> + </li> + <% }); %> + <% if (!(skills || []).length) { %> + <li class="project-chats-empty muted">No skills yet. Accept a skill suggestion in a chat for this project.</li> + <% } %> + </ul> + </div> + </div> + <div id="project-panel-links" class="project-panel" diff --git a/views/settings.ejs b/views/settings.ejs index 80934a5..eec4197 100644 --- a/views/settings.ejs +++ b/views/settings.ejs @@ -10,6 +10,76 @@ <h1 class="settings-title">Settings</h1> </header> + <nav class="settings-tabs" aria-label="Settings sections"> + <a class="settings-tab<%= settingsTab === 'voice-ask' ? ' is-active' : '' %>" href="/settings/voice-ask"<%- settingsTab === 'voice-ask' ? ' aria-current="page"' : '' %>>Voice & Ask</a> + <a class="settings-tab<%= settingsTab === 'remote-access' ? ' is-active' : '' %>" href="/settings/remote-access"<%- settingsTab === 'remote-access' ? ' aria-current="page"' : '' %>>Remote Access</a> + </nav> + + <% if (settingsTab === 'voice-ask') { %> + <!-- ─────────── Voice & Ask (OpenRouter) ─────────── --> + <section class="settings-section"> + <header class="settings-section-head"> + <div class="settings-section-head-text"> + <h2 class="settings-section-title"> + <svg class="settings-section-icon" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"> + <path d="m12 3-1.9 5.8a2 2 0 0 1-1.3 1.3L3 12l5.8 1.9a2 2 0 0 1 1.3 1.3L12 21l1.9-5.8a2 2 0 0 1 1.3-1.3L21 12l-5.8-1.9a2 2 0 0 1-1.3-1.3z"/><path d="M5 3v4"/><path d="M3 5h4"/> + </svg> + Voice & Ask + </h2> + <p class="settings-section-desc"> + Paste your OpenRouter key to unlock: + </p> + <ul class="settings-section-feature-list"> + <li>🎙 <strong>Voice messages</strong> — speak instead of type</li> + <li>💬 <strong>Ask mode</strong> — quick answers without agent execution</li> + <li>🗂 <strong>Work mode</strong> — AI works inside your selected folders only, file writes need your approval</li> + <li>✏️ <strong>Auto-named chats</strong> — titles generated automatically</li> + </ul> + <p class="settings-section-desc settings-section-desc--sub">The key stays on this device.</p> + </div> + </header> + + <% if (openRouter.hasKey) { %> + <div class="or-card" id="or-card"> + <div class="or-status-row"> + <span class="or-dot" aria-hidden="true"></span> + <div class="or-status-text"> + <div class="or-status-title">Connected</div> + <code class="or-key"><%= openRouter.maskedKey %></code> + </div> + <button type="button" class="btn btn--ghost btn--sm btn--danger" id="or-disconnect">Disconnect</button> + </div> + <div class="or-usage" id="or-usage"> + <div class="or-usage-loading"> + <span class="ra-spinner" aria-hidden="true"></span><span>Loading usage…</span> + </div> + </div> + </div> + <% } else { %> + <div class="or-card or-connect" id="or-card"> + <form class="or-form" id="or-form"> + <input + type="password" + id="or-key-input" + class="or-input" + placeholder="Paste your key (sk-or-…)" + autocomplete="off" + spellcheck="false" + aria-label="OpenRouter API key" + /> + <button type="submit" class="btn btn--primary btn--sm" id="or-connect">Connect</button> + </form> + <p class="or-err" id="or-err" role="alert" hidden></p> + <a class="or-help" href="https://openrouter.ai/keys" target="_blank" rel="noopener noreferrer"> + Don’t have a key? Get one at openrouter.ai/keys + <svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="1.8" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M7 17 17 7"/><path d="M7 7h10v10"/></svg> + </a> + </div> + <% } %> + </section> + <% } %> + + <% if (settingsTab === 'remote-access') { %> <!-- ─────────── Remote Access ─────────── --> <section class="settings-section"> <header class="settings-section-head"> @@ -126,10 +196,12 @@ <% } %> </div> </section> + <% } %> </div> </main> </div> +<% if (settingsTab === 'remote-access') { %> <!-- ─────────── New-tunnel modal ─────────── --> <dialog id="ra-modal" class="ra-modal" aria-labelledby="ra-modal-title" aria-describedby="ra-modal-desc"> <form method="dialog" class="ra-modal-form" id="ra-modal-form"> @@ -191,6 +263,7 @@ </footer> </form> </dialog> +<% } %> <style> /* ───────────── Page shell ───────────── */ @@ -203,7 +276,7 @@ overflow-y: auto; } .settings-inner { width: 100%; } -.settings-head { margin-bottom: var(--space-7); } +.settings-head { margin-bottom: var(--space-6); } .settings-title { margin: 0; font-size: 1.5rem; @@ -213,6 +286,37 @@ color: var(--text); } +/* ───────────── Section tabs (each its own page) ───────────── */ +.settings-tabs { + display: flex; + flex-wrap: wrap; + gap: var(--space-2); + margin: 0 0 var(--space-8); +} +.settings-tab { + display: inline-flex; + align-items: center; + padding: var(--space-3) var(--space-6); + border-radius: var(--radius-pill); + border: 1px solid color-mix(in srgb, var(--border) 80%, transparent); + background: var(--surface); + color: var(--muted); + font-size: var(--text-sm); + font-weight: 600; + letter-spacing: -0.01em; + text-decoration: none; + transition: color 0.15s ease, border-color 0.15s ease, background 0.15s ease; +} +.settings-tab:hover { + color: var(--text); + border-color: color-mix(in srgb, var(--text) 22%, var(--border)); +} +.settings-tab.is-active { + color: var(--btn-fg); + background: var(--btn-bg); + border-color: var(--btn-bg); +} + /* ───────────── Section ───────────── */ .settings-section { margin: 0 0 var(--space-9); } .settings-section-head { margin: 0 0 var(--space-5); } @@ -246,6 +350,23 @@ color: var(--muted); max-width: 26rem; } +.settings-section-desc--sub { + margin-top: var(--space-2); +} +.settings-section-feature-list { + margin: var(--space-2) 0 0; + padding-left: 0; + list-style: none; + display: flex; + flex-direction: column; + gap: var(--space-1); + font-size: var(--text-sm); + color: var(--muted); + max-width: 26rem; +} +.settings-section-feature-list li { + line-height: 1.45; +} .ra-new-btn { flex-shrink: 0; gap: var(--space-2); @@ -686,14 +807,121 @@ padding: 0 var(--space-5) var(--space-5); } +/* ───────────── Voice & Ask (OpenRouter) ───────────── */ +.or-card { + background: var(--surface); + border: 1px solid color-mix(in srgb, var(--border) 80%, transparent); + border-radius: var(--radius-lg); + padding: var(--space-5); +} + +/* Connected — status row */ +.or-status-row { display: flex; align-items: center; gap: var(--space-3); } +.or-dot { + flex-shrink: 0; + width: 8px; height: 8px; + border-radius: 50%; + background: var(--approve, #2da44e); + box-shadow: 0 0 0 3px color-mix(in srgb, var(--approve, #2da44e) 22%, transparent); +} +.or-status-text { flex: 1; min-width: 0; } +.or-status-title { + font-size: var(--text-sm); + font-weight: 600; + color: var(--text); + letter-spacing: -0.01em; +} +.or-key { + display: block; + margin-top: 2px; + font-family: ui-monospace, SFMono-Regular, Menlo, monospace; + font-size: var(--text-xs); + color: var(--muted); +} + +/* Connected — usage readout */ +.or-usage { + margin-top: var(--space-5); + padding-top: var(--space-5); + border-top: 1px solid color-mix(in srgb, var(--border) 55%, transparent); +} +.or-usage-loading { + display: flex; align-items: center; gap: var(--space-2); + font-size: var(--text-sm); color: var(--muted); font-style: italic; +} +.or-usage-grid { display: flex; gap: var(--space-10); } +.or-usage-value { + font-size: 1.5rem; + font-weight: 600; + letter-spacing: -0.02em; + color: var(--text); + font-variant-numeric: tabular-nums; + font-feature-settings: "tnum"; + line-height: 1.1; +} +.or-usage-label { margin-top: 3px; font-size: var(--text-xs); color: var(--muted); } +.or-bar { + margin-top: var(--space-5); + height: 5px; + border-radius: var(--radius-pill); + background: color-mix(in srgb, var(--text) 9%, transparent); + overflow: hidden; +} +.or-bar-fill { + height: 100%; + border-radius: var(--radius-pill); + background: var(--accent, var(--apple-blue)); + transition: width 0.4s ease; +} +.or-usage-muted { font-size: var(--text-sm); color: var(--muted); } + +/* Not connected — connect form */ +.or-form { display: flex; gap: var(--space-3); align-items: center; } +.or-input { + flex: 1; + min-width: 0; + margin: 0; + padding: var(--space-4) var(--space-5); + border: 1px solid var(--border); + border-radius: var(--radius-md); + background: color-mix(in srgb, var(--text) 3%, var(--surface)); + color: var(--text); + font-family: ui-monospace, SFMono-Regular, Menlo, monospace; + font-size: var(--text-sm); + transition: border-color 0.15s ease, box-shadow 0.15s ease; +} +.or-input::placeholder { color: color-mix(in srgb, var(--muted) 75%, transparent); } +.or-input:focus { + outline: none; + border-color: color-mix(in srgb, var(--text) 22%, var(--border)); + box-shadow: var(--focus-ring); +} +.or-err { margin: var(--space-3) 0 0; font-size: var(--text-sm); color: var(--danger); } +.or-err[hidden] { display: none; } +.or-help { + display: inline-flex; + align-items: center; + gap: var(--space-2); + margin-top: var(--space-5); + font-size: var(--text-sm); + color: var(--muted); + text-decoration: none; +} +.or-help > svg { width: 13px; height: 13px; flex-shrink: 0; } +.or-help:hover { color: var(--text); } +.or-help:hover { text-decoration: underline; text-underline-offset: 2px; } + @media (max-width: 520px) { .settings-page { padding: var(--space-7) var(--space-4) var(--space-8); } .settings-section-head-row { flex-direction: column; align-items: stretch; } .ra-new-btn { justify-content: center; } .ra-tunnel-head { align-items: flex-start; } + .or-form { flex-direction: column; align-items: stretch; } + .or-usage-grid { gap: var(--space-8); } } </style> +<% if (settingsTab === 'remote-access') { %> <script> (function () { const list = document.getElementById('ra-list'); @@ -1227,6 +1455,113 @@ tickRemaining(); })(); </script> +<% } %> + +<% if (settingsTab === 'voice-ask') { %> +<script> +(function () { + const card = document.getElementById('or-card'); + if (!card) return; + + function usd(n) { + if (!isFinite(n) || n <= 0) return '$0'; + if (n < 0.01) return '$' + n.toFixed(4); + if (n < 1) return '$' + n.toFixed(3); + return '$' + n.toFixed(2); + } + + /* ─── connect ─── */ + const form = document.getElementById('or-form'); + if (form) { + const input = document.getElementById('or-key-input'); + const btn = document.getElementById('or-connect'); + const err = document.getElementById('or-err'); + form.addEventListener('submit', async function (e) { + e.preventDefault(); + const key = ((input && input.value) || '').trim(); + if (err) err.hidden = true; + if (!key) { if (input) input.focus(); return; } + btn.disabled = true; + const orig = btn.textContent; + btn.textContent = 'Connecting…'; + try { + const res = await fetch('/api/openrouter/key', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ key }), + }); + const data = await res.json().catch(function () { return {}; }); + if (!res.ok) { + if (err) { err.textContent = data.error || ('Could not save key (HTTP ' + res.status + ')'); err.hidden = false; } + btn.disabled = false; + btn.textContent = orig; + return; + } + // Reload so the connected state + composer (mic, Ask) pick up the key. + location.reload(); + } catch (e2) { + if (err) { err.textContent = 'Could not save key: ' + e2.message; err.hidden = false; } + btn.disabled = false; + btn.textContent = orig; + } + }); + } + + /* ─── disconnect ─── */ + const disconnectBtn = document.getElementById('or-disconnect'); + if (disconnectBtn) { + disconnectBtn.addEventListener('click', async function () { + if (!confirm('Disconnect OpenRouter? Voice messages, Ask mode, and Work mode turn off until you add a key again.')) return; + disconnectBtn.disabled = true; + const orig = disconnectBtn.textContent; + disconnectBtn.textContent = 'Disconnecting…'; + try { + const res = await fetch('/api/openrouter/key', { method: 'DELETE' }); + if (!res.ok) { alert('Failed to disconnect'); disconnectBtn.disabled = false; disconnectBtn.textContent = orig; return; } + location.reload(); + } catch (e3) { + alert('Failed to disconnect: ' + e3.message); + disconnectBtn.disabled = false; + disconnectBtn.textContent = orig; + } + }); + } + + /* ─── usage ─── */ + const usageEl = document.getElementById('or-usage'); + if (usageEl) { + (async function () { + try { + const res = await fetch('/api/openrouter/usage'); + const data = await res.json().catch(function () { return {}; }); + if (!res.ok || !data.usage) { + usageEl.innerHTML = '<div class="or-usage-muted">Couldn’t load usage right now.</div>'; + return; + } + const u = data.usage; + const used = Number(u.usage || 0); + const remaining = u.remaining == null ? null : Number(u.remaining); + const limit = u.limit == null ? null : Number(u.limit); + + let html = '<div class="or-usage-grid">'; + html += '<div><div class="or-usage-value">' + usd(used) + '</div><div class="or-usage-label">spent</div></div>'; + if (remaining != null) { + html += '<div><div class="or-usage-value">' + usd(remaining) + '</div><div class="or-usage-label">remaining</div></div>'; + } + html += '</div>'; + if (limit != null && limit > 0) { + const pct = Math.max(0, Math.min(100, (used / limit) * 100)); + html += '<div class="or-bar"><div class="or-bar-fill" style="width:' + pct.toFixed(1) + '%"></div></div>'; + } + usageEl.innerHTML = html; + } catch (e4) { + usageEl.innerHTML = '<div class="or-usage-muted">Couldn’t load usage right now.</div>'; + } + })(); + } +})(); +</script> +<% } %> <script src="/js/iclaw.js?v=1"></script> diff --git a/views/welcome.ejs b/views/welcome.ejs new file mode 100644 index 0000000..32a3ed2 --- /dev/null +++ b/views/welcome.ejs @@ -0,0 +1,227 @@ +<%- include('partials/head', { title: 'Welcome to iClaw' }) %> + +<div class="onb"> + <div class="onb-card-shell" role="dialog" aria-labelledby="onb-title"> + + <!-- Step 1 — pick a power source --> + <section class="onb-step" id="onb-choose"> + <h1 class="onb-title" id="onb-title">Welcome to iClaw</h1> + <p class="onb-sub">iClaw works with the world’s best AI models - connect one to begin</p> + + <button type="button" class="onb-option" id="onb-pick-openrouter"> + <span class="onb-option-main"> + <span class="onb-option-name">Connect OpenRouter</span> + <span class="onb-option-desc"> + Pay as you go - about $5 lasts a long time, and we’ll guide you + </span> + </span> + <span class="onb-option-chevron" aria-hidden="true">›</span> + </button> + + <button type="button" class="onb-option onb-option--disabled" disabled> + <span class="onb-option-main"> + <span class="onb-option-name"> + Use iClaw credits + <span class="onb-badge">Coming soon</span> + </span> + <span class="onb-option-desc"> + No setup - we handle the AI for you + </span> + </span> + </button> + + <button type="button" class="onb-skip" id="onb-skip">Skip for now</button> + </section> + + <!-- Step 2 — paste the OpenRouter key --> + <section class="onb-step" id="onb-key" hidden> + <button type="button" class="onb-back" id="onb-back" aria-label="Back">‹ Back</button> + <h1 class="onb-title">Connect OpenRouter</h1> + + <ol class="onb-steps"> + <li> + Sign in at + <a href="https://openrouter.ai/keys" target="_blank" rel="noopener noreferrer">openrouter.ai/keys</a> + </li> + <li>Add about $5 - it goes a long way</li> + <li>Click <strong>Create Key</strong>, then copy and paste it below</li> + </ol> + + <form class="onb-key-form" id="onb-key-form" autocomplete="off"> + <input + type="password" + id="onb-key-input" + class="onb-key-input" + placeholder="sk-or-..." + aria-label="OpenRouter API key" + spellcheck="false" + autocapitalize="off" + /> + <button type="submit" class="btn btn--primary onb-key-submit" id="onb-key-submit"> + Connect + </button> + </form> + <p class="onb-key-error" id="onb-key-error" role="alert" hidden></p> + <p class="onb-key-note">Your key stays on this computer only</p> + </section> + + <!-- Honest background-prep line, shared by both steps --> + <div class="onb-prep" id="onb-prep" aria-live="polite" hidden> + <span class="onb-prep-dot" id="onb-prep-dot"></span> + <span class="onb-prep-text" id="onb-prep-text"></span> + </div> + </div> +</div> + +<script> +(function () { + var stepChoose = document.getElementById('onb-choose'); + var stepKey = document.getElementById('onb-key'); + var prepText = document.getElementById('onb-prep-text'); + var prepDot = document.getElementById('onb-prep-dot'); + + function show(step) { + stepChoose.hidden = step !== 'choose'; + stepKey.hidden = step !== 'key'; + if (step === 'key') { + var input = document.getElementById('onb-key-input'); + if (input) input.focus(); + } + } + + document.getElementById('onb-pick-openrouter').addEventListener('click', function () { + show('key'); + }); + document.getElementById('onb-back').addEventListener('click', function () { + show('choose'); + }); + + function complete() { + return fetch('/api/onboarding/complete', { method: 'POST' }); + } + + document.getElementById('onb-skip').addEventListener('click', function () { + complete().finally(function () { window.location.href = '/'; }); + }); + + var form = document.getElementById('onb-key-form'); + var errorEl = document.getElementById('onb-key-error'); + var submitBtn = document.getElementById('onb-key-submit'); + var keyInput = document.getElementById('onb-key-input'); + var noCreditOkFor = null; // key the user confirmed despite a $0 balance + + function resetBtn() { + submitBtn.disabled = false; + submitBtn.textContent = 'Connect'; + } + function showError(msg, warn) { + errorEl.textContent = msg; + errorEl.classList.toggle('is-warn', !!warn); + errorEl.hidden = false; + } + + var INVALID = { + empty: 'Paste your OpenRouter key first.', + unauthorized: 'That key was rejected by OpenRouter. Double-check you copied it in full.', + http: 'OpenRouter couldn’t verify that key. Try again in a moment.', + network: 'Couldn’t reach OpenRouter to verify the key — check your connection.', + }; + + // Save the (already-validated) key, mark onboarding done, enter the app. + function saveAndEnter(key) { + return fetch('/api/openrouter/key', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ key: key }), + }) + .then(function (res) { return res.json().then(function (b) { return { ok: res.ok, body: b }; }); }) + .then(function (r) { + if (!r.ok) { + showError((r.body && r.body.error) || 'Could not save the key.'); + resetBtn(); + return; + } + return complete().then(function () { window.location.href = '/'; }); + }); + } + + form.addEventListener('submit', function (e) { + e.preventDefault(); + var key = keyInput.value.trim(); + errorEl.hidden = true; + if (!key) { showError(INVALID.empty); return; } + + // Second click after a "no credit" warning → proceed without re-checking. + if (noCreditOkFor === key) { submitBtn.disabled = true; submitBtn.textContent = 'Connecting…'; saveAndEnter(key); return; } + + submitBtn.disabled = true; + submitBtn.textContent = 'Verifying…'; + fetch('/api/openrouter/validate', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ key: key }), + }) + .then(function (res) { return res.json().catch(function () { return {}; }); }) + .then(function (v) { + if (!v || !v.valid) { + showError(INVALID[(v && v.reason)] || INVALID.http); + resetBtn(); + return; + } + // Valid but no credit: warn once, let them continue anyway. + if (typeof v.remaining === 'number' && v.remaining <= 0) { + noCreditOkFor = key; + showError('Key works, but it has no credit yet — paid models won’t run until you top up. Click Connect again to continue anyway.', true); + resetBtn(); + submitBtn.textContent = 'Continue anyway'; + return; + } + submitBtn.textContent = 'Connecting…'; + return saveAndEnter(key); + }) + .catch(function () { showError(INVALID.network); resetBtn(); }); + }); + // Editing the key clears a prior "continue anyway" confirmation. + keyInput.addEventListener('input', function () { noCreditOkFor = null; }); + + // Background-prep poller: an honest line about Docker / OpenClaw readiness. + // Never blocks anything — purely informational. + // Only worth a line when something is actually happening or needs attention. + // When everything's fine (or we're still checking) we say nothing — a green + // "all good" badge is just noise for a non-technical user. + var prep = document.getElementById('onb-prep'); + function describe(env) { + // Keep onboarding jargon-free for non-technical users: only surface a + // neutral "setting up" line while the sandbox image downloads. Docker's + // state (stopped/missing) is NOT shown here — it's handled automatically + // (started on demand) and surfaces in-context later, with an Install + // button, only if the user actually picks Safe work. + if (env.docker === 'pulling') return 'Setting up your workspace…'; + return null; + } + function settled(env) { + return env.docker === 'ready' || env.docker === 'missing' || env.docker === 'stopped'; + } + function poll() { + fetch('/api/onboarding/status') + .then(function (r) { return r.json(); }) + .then(function (env) { + var msg = describe(env); + if (msg) { + prepText.textContent = msg; + if (prep) prep.hidden = false; + } else if (prep) { + prep.hidden = true; + } + // Green dot only means "all set". Attention states (missing/stopped) + // keep the amber pulsing dot; 'ready' is hidden anyway. + prepDot.classList.toggle('is-done', env.docker === 'ready'); + if (!settled(env)) setTimeout(poll, 1500); + }) + .catch(function () { setTimeout(poll, 3000); }); + } + poll(); +})(); +</script> + +<%- include('partials/foot') %>