Factory agent pulls skills on demand instead of front-loading them

packages/software-factory/.claude/CLAUDE.md

@@ -13,6 +13,10 @@ pnpm factory:go --brief-url <url> --target-realm <url>
 - `--debug` — verbose logs.
 - `--agent openrouter` — use the opencode-OpenRouter passthrough agent
   (routes via the realm-server's `/_openrouter/chat/completions` proxy).
+- `--skills-dir <path>` — replace the on-demand skill library with this
+  directory (bundled plugin skills are not read; workflow skills still
+  front-load from the factory package). Fails fast if it yields no
+  skills. Repeatable.
 
 ## Running tests
 
@@ -25,32 +29,29 @@ pnpm factory:go --brief-url <url> --target-realm <url>
 Two parallel skill paths exist, one per factory run mode:
 
 - **SDK orchestrator** (`pnpm factory:go`): the loader at
-  `src/factory-skill-loader.ts` reads from
-  **`.agents/skills-orchestrator/`** first. Those skills describe the
-  factory-MCP-tool surface (`signal_done`, `get_card_schema`,
-  `run_lint`, …) that `ToolUseFactoryAgent` actually provides at
-  runtime.
+  `src/factory-skill-loader.ts` searches exactly two directories —
+  **`.agents/skills-orchestrator/`** (the factory workflow skills,
+  which describe the factory-MCP-tool surface: `signal_done`,
+  `get_card_schema`, `run_lint`, …) and
+  **`packages/boxel-cli/plugin/skills/`** (the domain-skill library:
+  skills bundled from `cardstack/boxel-skills` plus CLI-native skills
+  like `boxel-api` / `boxel-command`). Only the workflow skill is
+  front-loaded into the system prompt; every other skill appears as a
+  one-line index entry and is fetched on demand via the `read_skill`
+  factory tool. A small exclusion list in the loader keeps
+  realm-lifecycle skills (`realm-sync`, `file-ops`, …) out of the
+  index — the orchestrator owns workspace→realm sync.
 - **Interactive Claude Code** (paste the prompt from
   `docs/runbook.md`): Claude Code reads
   **`.agents/skills/`** via the `.claude/skills` symlink. Those
   skills describe the `boxel` CLI surface and the agent-owned
   status lifecycle. The interactive flow has no orchestrator
   process; the agent drives the loop directly.
 
-Fallback dirs for both modes (skills that aren't software-factory
-specific):
-
-1. `packages/boxel-cli/plugin/skills/` — boxel-cli Claude Code
-   plugin skills (`boxel-api`, `boxel-command`); same directory
-   the plugin distributes to end users.
-2. monorepo root `.agents/skills/` — general domain skills
-   (`boxel-development`, `boxel-file-structure`,
-   `ember-best-practices`).
-
-The two software-factory skill sets diverged during CS-11149. They
-stay separated until the SDK orchestrator is retired; at that
-point the orchestrator code and `.agents/skills-orchestrator/` get deleted
-together.
+The two software-factory skill sets diverged during CS-11149 and stay
+separated permanently — each run mode has its own tool surface, so
+neither skill set can serve the other. Keep edits to workflow guidance
+in sync across both when the change isn't surface-specific.
 
 ## Architectural principle
 

packages/software-factory/prompts/system.md

@@ -24,6 +24,10 @@ Factory tools (call by name):
 - **`get_card_schema({ module, name })`** — fetch the live JSON Schema
   of a card definition. Required before writing a tracker
   (Project / Issue / KnowledgeArticle) or Spec card.
+- **`read_skill({ skill, reference? })`** — load a skill from the skill
+  library (see the Skill library section below). Returns the skill text
+  plus its reference filenames; pass `reference` to fetch one of those
+  documents.
 - **`run_lint`** / **`run_parse`** / **`run_evaluate`** /
   **`run_instantiate`** / **`run_tests`** — mid-turn validators. Optional:
   the orchestrator runs these automatically after `signal_done`.
@@ -43,14 +47,15 @@ Inspect existing state before making changes; do not guess.
 
 # Rules
 
-- **Stay in your target realm.** The loaded skills + the issue
-  description contain everything you need to implement the card. Do
+- **Stay in your target realm.** The loaded skills, the skill library
+  (via `read_skill`), and the issue description contain everything you
+  need to implement the card. Do
   NOT run `boxel file ls` / `boxel search` / `boxel read-transpiled`
   against any realm other than the target realm shown below — not the
   base realm, not the software-factory realm, not experiments{{#if enableBoxelUiDiscovery}}{{else}}, not catalog{{/if}}. Cross-realm
   exploration burns tokens and time without helping. If a pattern isn't
-  covered by your skills, write the card using your own knowledge and let
-  validation tell you what to fix.{{#if enableBoxelUiDiscovery}}
+  covered by the skill library, write the card using your own knowledge
+  and let validation tell you what to fix.{{#if enableBoxelUiDiscovery}}
   - **Exception — catalog component specs (mandatory).** The catalog
     realm (`@cardstack/catalog/`) publishes one Spec card per boxel-ui
     component, indexed and searchable. Before writing **any** UI in a
@@ -110,6 +115,34 @@ tool introspects the real `CardDef` at runtime, so the shape stays
 correct as the tracker schema evolves. Schemas are cached per-process,
 so repeat calls are cheap.
 
+{{#if skillIndex}}
+
+# Skill library (load on demand)
+
+Beyond the skills included in full below, a library of skills is
+available on demand. Each entry is one skill; load it with
+`read_skill({ skill: '<name>' })` **before** doing work its description
+covers — do not guess at conventions a skill documents. The result
+lists the skill's reference documents; fetch one with
+`read_skill({ skill: '<name>', reference: '<filename>' })`. Load skills
+selectively — only what the current issue actually needs.
+
+Entries marked **(suggested)** were matched to this issue's text; treat
+them as a starting point, not a limit.
+
+These skills document the `boxel` CLI for several environments, not
+just this one. In this factory session, read-only commands they mention
+(`boxel search`, `boxel read-transpiled`) are fine via Bash, but
+**never** run realm-mutating commands they mention — `boxel realm
+push` / `sync` / `create`, `boxel file write` / `touch` / `delete`. You
+write workspace files with `Write` / `Edit`; the orchestrator owns
+syncing them to the realm.
+
+{{#each skillIndex}}
+- `{{name}}`{{#if suggested}} **(suggested)**{{/if}} — {{description}}
+{{/each}}
+{{/if}}
+
 {{#each skills}}
 
 # Skill: {{name}}

packages/software-factory/scripts/smoke-tests/factory-skill-smoke.ts

@@ -33,7 +33,7 @@ const SAMPLE_ISSUES: { label: string; issue: IssueData }[] = [
     },
   },
   {
-    label: '.gts component work (triggers ember-best-practices)',
+    label: '.gts component work (triggers UI skill suggestions)',
     issue: {
       id: 'Issues/gts-component',
       title: 'Build a dashboard component',
@@ -117,11 +117,12 @@ async function main(): Promise<void> {
     log.info(`  Issue: ${issue.title}`);
 
     // 1. Resolve
-    let skillNames = resolver.resolve(issue, project);
-    log.info(`  Resolved skills: [${skillNames.join(', ')}]`);
+    let { load: skillNames, suggested } = resolver.resolve(issue, project);
+    log.info(`  Front-loaded skills: [${skillNames.join(', ')}]`);
+    log.info(`  Suggested (on demand): [${suggested.join(', ')}]`);
 
-    // 2. Load (with issue context for reference filtering)
-    let skills = await loader.loadAll(skillNames, issue);
+    // 2. Load the front-loaded set
+    let skills = await loader.loadAll(skillNames);
     log.info(`  Loaded: ${skills.length}/${skillNames.length} skills`);
 
     for (let skill of skills) {
@@ -150,33 +151,22 @@ async function main(): Promise<void> {
     }
   }
 
-  // Summary: list all discoverable skills
-  log.info('--- All discoverable skills ---');
-  let allSkillNames = [
-    'boxel-development',
-    'boxel-file-structure',
-    'boxel-api',
-    'boxel-command',
-    'ember-best-practices',
-    'software-factory-operations',
-  ];
-
-  let allSkills = await loader.loadAll(allSkillNames);
+  // Summary: the on-demand skill index (what read_skill exposes)
+  log.info('--- On-demand skill index ---');
+  let index = await loader.buildIndex();
   let grandTotal = 0;
-  for (let skill of allSkills) {
+  for (let entry of index) {
+    let skill = await loader.load(entry.name);
     let tokens = estimateTokens(skill);
     grandTotal += tokens;
     let refCount = skill.references?.length ?? 0;
     let refNote = refCount > 0 ? ` (${refCount} refs)` : '';
-    log.info(`  ${skill.name}: ~${tokens} tokens${refNote}`);
+    log.info(`  ${entry.name}: ~${tokens} tokens${refNote}`);
   }
-  let missing = allSkillNames.filter(
-    (n) => !allSkills.find((s) => s.name === n),
+  log.info(
+    `  Total: ~${grandTotal} tokens across ${index.length} indexed skills ` +
+      '(loaded on demand, never all at once)',
   );
-  if (missing.length > 0) {
-    log.info(`  Not found: [${missing.join(', ')}]`);
-  }
-  log.info(`  Total: ~${grandTotal} tokens across ${allSkills.length} skills`);
 
   log.info('\nSmoke test passed.');
 }

packages/software-factory/src/factory-agent/claude-code.ts

@@ -295,6 +295,7 @@ export class ClaudeCodeFactoryAgent implements LoopAgent {
       darkfactoryModuleUrl: requireDarkfactoryModuleUrl(context),
       enableBoxelUiDiscovery: context.enableBoxelUiDiscovery === true,
       skills,
+      skillIndex: context.skillIndex ?? [],
     });
 
     // Two tool surfaces are visible to the model on the Claude backend:

packages/software-factory/src/factory-agent/opencode.ts

@@ -473,6 +473,7 @@ export class OpencodeFactoryAgent implements LoopAgent {
       darkfactoryModuleUrl: requireDarkfactoryModuleUrl(context),
       enableBoxelUiDiscovery: context.enableBoxelUiDiscovery === true,
       skills,
+      skillIndex: context.skillIndex ?? [],
     });
   }
 

packages/software-factory/src/factory-agent/types.ts

@@ -114,6 +114,22 @@ export interface ResolvedSkill {
   references?: string[];
 }
 
+/**
+ * One row of the on-demand skill index rendered into the system prompt.
+ * The agent reads the description to decide relevance, then loads the full
+ * skill via the `read_skill` factory tool.
+ */
+export interface SkillIndexEntry {
+  name: string;
+  description: string;
+  /**
+   * Set when the resolver flagged this skill as likely relevant to the
+   * current issue. A hint for the agent, not a gate — any indexed skill is
+   * loadable.
+   */
+  suggested?: boolean;
+}
+
 export interface ToolArg {
   name: string;
   type: string;
@@ -216,7 +232,14 @@ export interface AgentContext {
   project: ProjectData;
   issue: IssueData;
   knowledge: KnowledgeArticleData[];
+  /** Front-loaded skills, rendered in full into the system prompt. */
   skills: ResolvedSkill[];
+  /**
+   * On-demand skill index: every skill the agent may load via `read_skill`,
+   * one line each in the system prompt. Entries the resolver considers
+   * relevant to the current issue carry `suggested: true`.
+   */
+  skillIndex?: SkillIndexEntry[];
   /** @deprecated Tools are now provided separately as FactoryTool[] to agent.run(). */
   tools?: ToolManifest[];
   /** @deprecated Use validationResults/validationContext instead. */

packages/software-factory/src/factory-context-builder.ts

@@ -3,6 +3,7 @@ import type {
   IssueData,
   KnowledgeArticleData,
   ProjectData,
+  SkillIndexEntry,
   TestResult,
   ValidationResults,
 } from './factory-agent/index.ts';
@@ -71,14 +72,42 @@ export class ContextBuilder {
     this.enableBoxelUiDiscovery = config.enableBoxelUiDiscovery === true;
   }
 
+  /**
+   * Resolve, load, and index skills for one issue.
+   *
+   * Front-loaded skills (`resolution.load`) are read in full and budgeted;
+   * everything else in the library becomes an on-demand index entry, with
+   * the resolver's `suggested` picks marked. Skills that made it into the
+   * front-loaded set are dropped from the index — their full text is
+   * already in the prompt.
+   */
+  private async resolveSkills(
+    issue: IssueData,
+    project: ProjectData,
+  ): Promise<{ skills: ResolvedSkill[]; skillIndex: SkillIndexEntry[] }> {
+    let resolution = this.skillResolver.resolve(issue, project);
+
+    let skills = await this.skillLoader.loadAll(resolution.load);
+    skills = enforceSkillBudget(skills, this.maxSkillTokens);
+
+    let loadedNames = new Set(skills.map((s) => s.name));
+    let suggestedNames = new Set(resolution.suggested);
+    let skillIndex = (await this.skillLoader.buildIndex())
+      .filter((entry) => !loadedNames.has(entry.name))
+      .map((entry) =>
+        suggestedNames.has(entry.name) ? { ...entry, suggested: true } : entry,
+      );
+
+    return { skills, skillIndex };
+  }
+
   /**
    * Assemble a complete AgentContext for one iteration of the execution loop.
    *
    * Steps:
-   * 1. Resolve skill names from issue + project context
-   * 2. Load all resolved skills from disk
-   * 3. Apply skill budget if configured
-   * 4. Return AgentContext (tools are provided separately as FactoryTool[])
+   * 1. Resolve front-loaded skill names + suggestions from issue/project
+   * 2. Load front-loaded skills from disk, build the on-demand skill index
+   * 3. Return AgentContext (tools are provided separately as FactoryTool[])
    */
   async build(params: {
     project: ProjectData;
@@ -92,24 +121,14 @@ export class ContextBuilder {
     let { project, issue, knowledge, targetRealm, darkfactoryModuleUrl } =
       params;
 
-    // Step 1: Resolve which skills are needed for this issue
-    let skillNames = this.skillResolver.resolve(issue, project);
-
-    // Step 2: Load skill content from disk
-    let skills: ResolvedSkill[] = await this.skillLoader.loadAll(
-      skillNames,
-      issue,
-    );
-
-    // Step 3: Enforce token budget if configured
-    skills = enforceSkillBudget(skills, this.maxSkillTokens);
+    let { skills, skillIndex } = await this.resolveSkills(issue, project);
 
-    // Step 4: Assemble the context
     let context: AgentContext = {
       project,
       issue,
       knowledge,
       skills,
+      skillIndex,
       targetRealm,
       enableBoxelUiDiscovery: this.enableBoxelUiDiscovery,
       ...(darkfactoryModuleUrl ? { darkfactoryModuleUrl } : {}),
@@ -170,22 +189,16 @@ export class ContextBuilder {
       }
     }
 
-    // Step 2: Resolve and load skills
-    let skillNames = this.skillResolver.resolve(issue, project);
-    let skills: ResolvedSkill[] = await this.skillLoader.loadAll(
-      skillNames,
-      issue,
-    );
-
-    // Step 3: Enforce token budget if configured
-    skills = enforceSkillBudget(skills, this.maxSkillTokens);
+    // Step 2: Resolve and load front-loaded skills + on-demand index
+    let { skills, skillIndex } = await this.resolveSkills(issue, project);
 
-    // Step 4: Assemble the context
+    // Step 3: Assemble the context
     let context: AgentContext = {
       project,
       issue,
       knowledge,
       skills,
+      skillIndex,
       targetRealm,
       enableBoxelUiDiscovery: this.enableBoxelUiDiscovery,
       ...(darkfactoryModuleUrl ? { darkfactoryModuleUrl } : {}),