Built-in Harper Agent Component

[kriszyp]

Proposal/Motivation

Add an AI agent that ships inside the Harper server process (vs. the external
harper-agent CLI tool at /home/kzyp/dev/harper-agent). The motivation:

• Users can interact with their running Harper instance through a single LLM-backed
  operation API, instead of running a separate CLI on their dev machine.
• The agent has zero-friction access to live server state: it calls operation()
  directly in-process (no auth round-trips, no HTTP), reads logs straight from the
  log files, watches/edits component files, and can attach to the inspector that
  threadServer.js already opens.
• Autonomous mode lets it set timers and continue iterating on a goal (e.g.
  "build a recommendations component, then test it, then refine") without a human
  in the loop on each turn.
• Can work in harmony with harper-agent or AI chat in studio, as it can be treated like an effective sub-agent to carry out tasks. Likely that this may also be used to work on git branches to implement a feature in isol

Inherits model/provider choices, the @openai/agents loop, and Zod-typed tool
patterns from harper-agent so the two share DNA.

Architecture overview

Where it runs. Built-in component registered via HARPER_BUILTIN_COMPONENTS,
loaded by core/components/componentLoader.ts:121. Exports startOnMainThread
(legacy API, like replication/subscriptionManager.ts:55). Lives on the main
thread alongside the operations API server (core/server/operationsServer.ts:40,
default port 9925). Worker threads handle application/REST traffic; the agent
stays out of their way.

Request flow (simple — no bridge needed since the ops API is main-thread):

1. Component's startOnMainThread registers agent_prompt (and friends) via
   server.registerOperation. These handlers run directly on the main thread,
   in the same runtime as the agent loop.
2. agent_prompt appends the user message to the named session and kicks the
   agent loop (or attaches to the in-flight loop for that session). Returns
   { session_id, message_id } immediately — the loop runs asynchronously on
   the JS event loop.
3. Clients poll get_agent_session to see transcript/status updates. SSE
   subscription can come later.
4. Multiple concurrent sessions run interleaved on the event loop — agent
   turns are mostly awaits on LLM/HTTP, so they cooperate cleanly. Per-session
   serialization prevents two prompts in the same session from racing.

Component structure

Add a new top-level directory at the harper-pro root, mirroring replication/,
licensing/, analytics/:

agent/
  agent.ts              // entry: exports startOnMainThread, handleApplication
  operations.ts         // registerOperation calls (runs on all threads)
  agentLoop.ts          // wraps @openai/agents Agent + run loop
  session.ts            // CombinedSession backed by hdb_agent_session table
  tools/
    operationTool.ts    // execute any registered operation (allowlist-gated)
    readLogTool.ts      // wraps utility/logging/readLog
    fsTools.ts          // read/write/list files (scoped to components root + log dir + config)
    configTool.ts       // get/set via configUtils
    httpFetchTool.ts    // outbound HTTP (web research + self-test)
    debugTool.ts        // attach to inspector port, set logpoints, profile
    timerTool.ts        // schedule follow-up callback (setTimeout/setInterval w/ .unref)
    componentTool.ts    // package/deploy/drop components via existing ops
  model.ts              // copied/lifted from harper-agent/lifecycle/getModel.ts

Wire it up in bin/harper.js:5-7:

process.env.HARPER_BUILTIN_COMPONENTS = ... + ',agent=@/dist/agent/agent.js';

Operations to register

All gated on superuser role. Registered on the main thread in
startOnMainThread, same pattern as install_usage_license in
licensing/usageLicensing.ts:79-89:

Name	Purpose	Returns
agent_prompt	Append a user message to a session and start/resume the loop	{ session_id, message_id }
get_agent_session	Read a session's transcript + status (polled by clients)	conversation items + status
list_agent_sessions	Enumerate sessions	array
cancel_agent_run	Abort an in-progress run	ack
approve_agent_action	Resolve a pending approval (option b from the issues list)	ack
set_agent_config	Update model/maxTurns/allowed-tools at runtime	ack

Tool inventory (separate tools, per user's preference)

Models guide their behavior much better with named, scoped tools than with one
"do anything" tool. Each tool below is a separate tool() definition. @openai/agents
accepts a Zod schema today; if we want to avoid a new dependency we can pass a
plain JSON Schema object instead (the SDK supports parameters as JSON Schema).
Pick one and stay consistent.

Database / operations

• run_operation — calls operation(body, ctx, true) from serverUtilities.ts:215.
  Authorizes as the agent's configured user. Body is any registered operation.
• describe_schema, describe_table — pre-bound conveniences (already callable
  via run_operation, but separate tools instruct the model better).
• search, sql — same rationale.

Filesystem

• read_file, write_file, apply_patch, list_dir, grep_files —
  scoped to: componentsRoot (CONFIG_PARAMS.COMPONENTSROOT), the log dir
  (harper_logger.getLogPath()), and the config file dir (configUtils.getConfigFilePath()).
  No access elsewhere by default.

Logs / observability

• read_log — wraps utility/logging/readLog.
• get_analytics — already a registered operation; exposed for autonomous tuning.

Configuration

• get_config, set_config — wrap configUtils.getConfiguration / setConfiguration.
  Gated behind allowDestructive: true (set_config only).
• restart_service — wraps existing restart op for hot-reload after config edits.
  Gated behind allowDestructive: true, AND requires approval even when
  autoApprove=true (a misfire here bricks a node).

Components / build

• add_component, package_component, deploy_component, drop_component,
  install_node_modules — wrap the existing ops.

HTTP

• http_fetch — outbound fetch for web research and self-testing apps the
  agent built (hits its own server via localhost:<port>). One tool covers both.

Debugging (worker threads only — main runs the agent itself; pausing its own
isolate would deadlock)

• inspector_attach({ workerIndex }) — connects to the CDP endpoint at
  THREADS_DEBUG_STARTINGPORT + workerIndex (only available when
  threads_debug=true, see threadServer.js:25-54). Application code runs in
  workers anyway, so this covers the actual use case.
• inspector_evaluate, inspector_set_breakpoint, inspector_set_logpoint,
  inspector_profile_cpu — CDP wrappers via a WS client to the worker's debug port.
  All reject workerIndex < 0.

Scheduling

• schedule_followup({ delayMs, prompt }) — setTimeout(...).unref(); on
  fire, appends a synthetic user message to the session, kicking the loop. This
  is what enables periodic/autonomous behavior. Track scheduled timers in-memory
  so they can be cancelled.

Configuration schema

New agent block in harperdb-config.yaml, validated against
core/config-root.schema.json:

agent:
  enabled: true                # default false; opt-in to avoid surprise LLM bills
  provider: anthropic          # openai | anthropic | google | ollama
  model: claude-opus-4-7
  # API keys come from env vars only (ANTHROPIC_API_KEY, OPENAI_API_KEY, etc.)
  # Never stored in config — avoids leakage via get_configuration.
  maxTurns: 50
  maxCostUsd: 5.00
  allowedTools: ['read_file','write_file','run_operation', ...]  # default: all safe tools
  autoApprove: false           # if true, agent runs without approval gates (still gated by allowDestructive)
  allowDestructive: false      # required to enable set_config, restart_service, drop_component, etc.
  user: hdb_admin              # role/user the agent acts as (default: superuser)
  componentsScope: ./components

API keys: env vars only, read at startup. Never logged. Model-provider imports
are lazy (only the chosen provider is loaded) so the rest stay
dead-code-eliminated.

Session storage

New system table system.hdb_agent_session keyed by session_id, holding an
ordered array of items conforming to @openai/agents's AgentInputItem, plus
a pendingApprovals: ApprovalRequest[] field used by the approval flow
(approve_agent_action resolves entries here). Mirrors the structure of
harper-agent's DiskSession (JSON file) but uses Harper's own LMDB/Rocks
storage. Implement CombinedSession interface from
harper-agent/lifecycle/session.ts against the table so the existing
runAgentForOnePass.ts flow drops in unchanged.

Permission / auth model

• Operation agent_prompt (and friends) require superuser — enforced by
  the same pattern licensing uses (the operations layer's auth covers it; we
  just don't set bypass_auth).
• Tool calls execute under the agent's configured user. Default: a system
  hdb_agent superuser created at startup if missing. Operators can swap to a
  restricted role to limit blast radius (e.g. read-only role for an analytics
  agent).
• Tools that touch the FS reject paths outside the scoped roots.

Self-debugging support

threadServer.js:30-62 opens the V8 inspector on THREADS_DEBUG_PORT (main)
or THREADS_DEBUG_STARTINGPORT + workerIndex (workers) when
THREADS_DEBUG=true. The agent's inspector_* tools speak CDP over WS to
those ports — same as how Chrome DevTools attaches. The user should be told to
set threads_debug: true in config (and a sensible starting port) before
asking the agent to debug.

Key dependencies to add

To harper-pro/package.json:

• @openai/agents (core loop, interruptions, session abstraction)
• ai + @ai-sdk/openai + @ai-sdk/anthropic + @ai-sdk/google (provider abstraction)
• ollama-ai-provider-v2 (optional, can be optionalDependency)
• zod (tool schemas)

These add ~several MB to the install. Consider making the whole agent component
an optional dependency or a separate npm package (@harperfast/server-agent)
loaded only when agent.enabled=true. Recommended: separate package, loaded
dynamically. That keeps the Harper baseline lean and aligns with the
licensing/replication style.

Resolved decisions

• Threading. Ops API is main-thread (operationsServer.ts:40); register
  directly there, no bridge.
• API keys. Env vars only, never in config.
• Status updates. Clients poll get_agent_session (SSE/streaming is a later add).
• Approvals. Stored on the session; resolved via approve_agent_action
  operation (option b).
• Concurrency. Multiple concurrent sessions interleave fine on the event
  loop. Per-session serialization only.
• Destructive ops. allowDestructive: true flag required to enable
  config-mutating / restart tools.
• Inspector. Workers only; tools reject workerIndex < 0.
• harper-agent code reuse. Lift sources directly into harper-pro/agent/.
  Extract a shared package later if it earns its weight.

Remaining open questions

1. Cost controls. Reuse harper-agent/utils/sessions/cost.ts for hard caps
   via maxCostUsd; emit notify-level log when approaching the cap. Confirm
   that's enough vs. needing a per-session cap too.
2. Distribution. Adding @openai/agents + @ai-sdk/* to harper-pro is a
   meaningful install-size hit. Recommend shipping as a separate package
   (@harperfast/server-agent) loaded only when agent.enabled=true. Confirm
   or override.
3. Tool schema flavor. Zod is idiomatic for @openai/agents and gives
   us free runtime validation; plain JSON Schema avoids the dep. Mild preference
   for Zod since it's already transitively present via harper-agent. Confirm.
4. First-run UX. Server-side only — no REPL. Docs page with curl examples
   • Studio panel later.

Critical files to read / modify

Modify:

• /home/kzyp/dev/harper-pro/bin/harper.js — add agent=@/dist/agent/agent.js
  to HARPER_BUILTIN_COMPONENTS.
• /home/kzyp/dev/harper-pro/package.json — add dependencies.
• /home/kzyp/dev/harper-pro/core/config-root.schema.json — add agent block schema.
• /home/kzyp/dev/harper-pro/core/utility/hdbTerms.ts — add config param constants
  • new OPERATIONS_ENUM entries for agent_prompt, get_agent_session, etc.

Create: /home/kzyp/dev/harper-pro/agent/ (full new directory).

Read & reuse:

• harper-agent/agent/runAgentForOnePass.ts — the loop. Largely copyable.
• harper-agent/lifecycle/getModel.ts — provider wrapping. Copyable.
• harper-agent/utils/sessions/cost.ts, rateLimits.ts — cost/rate tracking. Copyable.
• harper-agent/tools/files/*, harper-agent/tools/general/shellTool.ts — base FS/shell tools.
• harper-pro/licensing/usageLicensing.ts — exemplar for the component
  shape (handleApplication, registerOperation, system table reads).

Verification

1. Build & boot. npm run build in harper-pro, start a server with the
   new env var; check logs for Agent component initialized. No-op if
   agent.enabled=false.
2. Round-trip prompt. curl -X POST /agent_prompt -d '{"message":"describe the system schema"}'
   → returns session_id → curl /get_agent_session/<id> shows the agent
   called describe_schema and produced a response.
3. Autonomous build. Prompt "create a Resource called Hello that returns
   'hi'", confirm a component file appears, then call localhost:9926/Hello
   to confirm the agent's app works.
4. Self-test loop. Prompt "build X, then call it on localhost:,
   debug any failures, repeat until working" — verifies http_fetch + tool
   chaining.
5. Debugging. With threads_debug=true, prompt "find why the Foo resource
   throws on POST" — should attach to inspector, set a logpoint, observe a
   request, report root cause.
6. Scheduled work. Prompt "every 5 minutes for the next hour, check
   cluster status and alert me if any node is down" — verifies
   schedule_followup + persistence.
7. Boundary checks. Confirm read_file('/etc/passwd') is rejected;
   confirm non-superuser callers of agent_prompt get 403.
8. Integration test at harper-pro/integrationTests/agent/ with a
   stub provider (or recorded fixtures) so CI doesn't burn LLM credits.

[richardchen874-sys]

The opt-in + maxCostUsd design is a good instinct. Built-in agents can become expensive in surprising ways because tool loops, retries, and long context are usually what drive the bill, not a single model call.

One addition I’d consider is model routing per agent phase:

• cheaper model for file search, classification, and simple edits
• stronger model for planning or risky changes
• hard per-run budget with visible token/cost traces
• fallback provider if latency or pricing changes

I'm building a private-beta OpenAI-compatible API aggregation layer focused on Chinese models like DeepSeek / Qwen / GLM-style providers, using official-provider keys. If you ever want to benchmark lower-cost models for Harper agent runs, I’d be happy to share some test credits.

Curious: do you expect the biggest cost driver to be long context, tool-call loops, or higher-end planning models?