Software factory v2/v3: design-first loop, on-demand skills, control/product realm split#5549
Draft
jurgenwerk wants to merge 38 commits into
Draft
Software factory v2/v3: design-first loop, on-demand skills, control/product realm split#5549jurgenwerk wants to merge 38 commits into
jurgenwerk wants to merge 38 commits into
Conversation
…-origin instantiate skip, cardinal-rules skill Four pragmatic fixes discovered running the BSL v0 multi-brief pass, each tracked upstream: CS-12192 (seed path hardcoded realm-wide; now overwrites with the current brief), CS-12185 (bootstrap can never pass instantiate; use NoOpValidator), CS-12197 (skip prerender instantiation for Specs referencing cross-origin base modules), plus an always-loaded skill of silent-failure traps (CS-12194 context). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
boxel-ui's compiled dist imports from 'lodash' but package.json only declared 'lodash-es'; embroider's strict resolver fails any fresh host build. Lockfile updated surgically for just this addition. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three new agent tools for the V2 loop: - list_skills: catalog of every skill resolvable from the loader's search dirs (28 currently, incl. the full design stack the front-loaded resolver never ships), with one-line descriptions and reference file names. - read_skill: progressive disclosure — SKILL.md first, individual reference files on request. Complements CS-12194. - screenshot_html: renders a workspace HTML file in headless Chromium (@playwright/test, already a dep) and writes a PNG into the workspace; the agent then Reads the PNG with its image-capable native Read tool. Powers the HTML-first design/crit loop: mockup -> screenshot -> critique -> revise, seconds per iteration, before any .gts is written. Input and output paths are constrained inside the workspace. Smoke-tested: catalog lists 28 skills, reference reads work, screenshot round-trip 687ms with escape/missing-file guards verified. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
--v2 changes the loop three ways: 1. Lean skill core: implementation issues front-load only software-factory-operations-v2 (new ~90-line design-first contract with a when-you-need-X-read-Y pointer table) + boxel-file-structure + boxel-workspace-cardinal-rules — ~4.1k tokens vs V1's ~88k (95% cut). Everything else loads on demand via list_skills / read_skill. 2. Design-first implement prompt (issue-implement-v2.md): mandatory HTML+CSS mockup with real sample copy -> screenshot_html -> Read the PNG -> named-defect critique -> revise, BEFORE any .gts; then the .gts is a translation of the accepted mockup. No .test.gts — tests belong to a separate hardening phase. 3. Validation pipeline drops the QUnit step (includeTestStep=false); parse/lint/eval/instantiate remain. boxel-ui discovery defaults ON. Bootstrap issues are unchanged in this commit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
First pilot run finding: the slimmed operations skill dropped the old skill's 'display title and short description' requirement and the agent left both empty (README + linkedExamples were fine). Restore it as an explicit MUST in both the skill and the implement prompt. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
RunLog: the factory writes run-log.gts + Runs/<slug>.json into the TARGET realm and appends entries on real events (issue picked, design screenshots, validation results, card-ready, run done). Screenshots embed as images the moment they exist; the REAL card embeds via linksTo the moment its first instance lands. Entries append with stable entries.N.card relationship indexes; display is newest-first via column-reverse. All writer failures are swallowed — the log can never take down a run. Also: v2 prompt now mandates ONE mockup page per card (was one file per surface — saves ~3-4 min/card of generation), and the v2 skill forbids curl-polling the realm for unsynced files (verify via run_* which sync first). Deployed retroactively to chris/jaraoke/Runs/jaraoke-player as the demo. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Context forking for the V2 loop. Before the first implementation issue, a priming turn (prompts/prime.md) reads skills, design language, and precedent once and writes design/DESIGN-NOTES.md; its SDK session id is captured (init event) and every subsequent implementation turn runs with resume + forkSession:true, inheriting the primed conversation as a shared provider-cached prefix instead of rebuilding context per issue. - AgentContext.resumeSession / primeTurn; AgentRunResult.sessionId - claude-code backend: session capture + resume/forkSession query options - issue-loop: prime state machine (undefined -> primed | null=disabled on failure), resumeSession attached to non-bootstrap contexts - --fork-context CLI flag threaded through entrypoint and wiring Failure-safe: a failed prime disables forking and the run proceeds as plain --v2. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… status Full design pass per the workspace design playbook (mockup -> crit -> port): - Workspace (card-grid v2) DNA: stage/surface paper, hairline rules, mono micro-labels (IBM Plex), signage chip masthead, Boxel-teal accent used in exactly two places (scan line, shipped/stat accents) - ALIVE under status=running: teal scan line across the masthead, pulsing live dot, one-shot teal arrival wash on the newest feed entry — all motion gated off when the run completes - NOW/NEXT band with dramatic type scale; asymmetric stat rail with big light tabular counters (cards ready / design rounds / validations green / issues done) computed from entries - Wire-feed entries: mono time rail + kind-colored micro-chips; design screenshots now embed via a linksTo'd PngDef FILE CARD (auth-safe, replaces raw <img> which 401s on private realms; imageUrl kept as fallback); shipped cards render live in a parent-owned CardContainer (delegated-render-control :deep overrides) - Container-query responsive isolated (rail collapses under 700px) and a proper CQ fitted template (badge/strip/tall variants) Writer: designEntriesFromToolCalls links the screenshot's file card (entries.N.image); lint-clean against the realm; redeployed to chris/jaraoke with the demo instance upgraded. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… clock Loop now reports every state change to the live blog: issue picked (with backlog→in-progress note), iteration N-of-M revision turns, agent blocked (with the clarification message), validation-exhausted blocks (with the orchestrator comment context), and the in-progress→done transition. Issue-ish entries carry a linksTo of the Issue card rendered as a clickable 'show me' atom chip; card-ready keeps the full embedded render. Isolated view: a 1s-tick elapsed clock (HH:MM:SS, live while running, frozen at last-entry time after) and a three-bar equalizer throbber next to the NOW item while the run is live. New kind colors: blocked=red, status=teal-green, iteration/comment=quiet. Lint clean; module redeployed to chris/jaraoke (status: ready, no indexing errors). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Runtime field validation rejected the linked PngDef screenshot cards: file cards descend from FileDef (base/file-api default export), a separate hierarchy from CardDef. linksTo(() => FileDef) accepts every file-card subtype (PngDef, JpegDef, ...). Redeployed to chris/jaraoke; module ready, indexing clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…, fix — literal Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
/_atomic?waitForIndex=true rewrites card sources with containsMany
FieldDef attributes emptied to {}. The writer now re-writes its instance
via a raw card-source write after every workspace sync.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- AgentContext.onToolCall hook: MCP tool results stream as they land; native Write/Edit tool_use blocks stream from the message loop - createRunLogStreamHandler: design rounds (with screenshot), first .gts write, failed checks (rate-limited), and post_update commentary land on the run-log card mid-turn; post_update also becomes an issue comment - post_update factory tool: the agent's social-style commentary channel - RunLogWriter: serialized write chain; stream appends skip the full workspace sync (raw instance write only); entries carry who (orchestrator/executor); entry template v2 (glyphs, relative time, who chip, fitted card embeds instead of atom chips) - v2 prompt + skill: live-blogging is an explicit duty Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ites Native Write calls never reach result.toolCalls (MCP-only), so card-ready entries shipped linkless. The per-issue stream handler now records instance JSON writes (infra dirs excluded) and the loop links the first three as clickable embeds on issue completion. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ps demanding tests - issue-loop: validation entries carry cardPath to the most informative Validations/<step>_<issueId>-<iteration> card; template renders validation/status/iteration card links as compact atom chips - bootstrap-implement-v2.md: v2 bootstrap prompt variant with the QUnit requirements stripped — the stale 'pipeline enforces run_tests' belief was baking .test.gts into every issue description, overriding the brief and the skill's no-tests hard rule Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Agent-triggered run_* tool syncs — and conflict re-uploads after out-of-band raw writes — also push the run-log instance through the stripping atomic path. The shared syncWorkspace now raw re-writes the instance after every successful sync via RunLogWriter.healInstance(). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Turn type — not the issue — decides the budget: prime/bootstrap/build/fix
map to {model, effort} via IssueLoopConfig.modelPolicy, applied to the
Agent SDK options per turn. v2 default: fix iterations (>= 2, mechanical
lint/parse fix-ups) run claude-sonnet-5 at medium effort; --fix-model /
--fix-effort override, --fix-model inherit disables. Note: provider cache
is per-model, so budget-switched fork turns re-ingest the prime prefix
uncached (still cheaper on the smaller model).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…pencode honors per-turn model Provider cache is per-model, so the default fix budget stays in-family (inherit model, effort=medium) — cache hit preserved, thinking cut. Cross-family switching (claude-sonnet-5, or OpenRouter ids like openai/gpt-5.2 / moonshotai/kimi-k2 on the opencode backend) is an explicit --fix-model opt-in that pays only when turn output is large relative to the primed prefix. Opencode backend now applies context.modelBudget.model per prompt. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…, forked) --phase-split runs each implementation issue's first iteration as two turns: a design turn (mockups + critique + design/<slug>-NOTES.md handoff contract, session-default flagship budget) and a build turn (pure translation, forked from the design session, default claude-sonnet-5 @ medium — the one turn type where output is large enough that a family switch beats the cache re-ingest). New prompts issue-design-v2.md / issue-build-v2.md; AgentContext.phase selects them. --build-model/--build-effort override; fix iterations unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
From the wardrobe drop-zone failure: agent imported the nonexistent @cardstack/boxel-host/commands/ path (runtime-only failure), and Photos.app drags arrive with an empty FileList — dataTransfer.items fallback + always-visible drop feedback are now skill rules. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
From the wardrobe walkthrough: a plain Query self-closed renders nothing silently — the component needs searchEntryWireQueryFromQuery + realms array + htmlQuery format, block form with entry.component. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Stall narration (timer reset by stream events; posts what the model is doing when the log goes quiet), per-turn telemetry (budget, duration, tool events, files touched, token usage captured from the SDK result message), scheduler queue notes, and watchdog routing (sync failures at normal level, heals at verbose). Level-gated via --monitor-level quiet|normal|verbose, rate-limited per event class, and posted raw-write-only so the monitor never triggers a workspace sync. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One shared workspace, two realm destinations. Control-plane paths (Issues/Projects/Boards/Knowledge Articles/Spec/Validations/Runs + run-log.gts) are excluded from the product atomic sync via a factory-managed .boxelignore and pushed to the control realm by ControlPlaneSync — individual hash-gated raw writes, which makes the entire control plane structurally immune to the /_atomic FieldDef strip (no heal needed under the split). Run-log entry links become absolute URLs with per-entry realm targeting (ship moments → product, issues/validations → control); the index card's board link goes absolute cross-realm. Product .gts updates no longer invalidate the run log the operator watches, and control churn no longer re-runs product-realm live queries (the flashing-tiles failure). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Derive the valid @cardstack/boxel-host/tools/* catalogue from the host source at run start; inject it as a generated skill in every agent context (never budget-trimmed) and add an in-process 'imports' validation step that fails any workspace .gts importing a phantom host tool — legacy commands/ paths get the rename fix, unknown tools get a near-match suggestion. Root cause of two of five wardrobe-run field failures: all static gates passed, the import exploded at runtime in the browser. Also fixes the pipeline's step typing (ValidationStepRunner vs the ValidationStep string union) and the issue-closure narrowing — the package now typechecks clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Render gate (P0): after each non-bootstrap issue completes, capture real host renders of its shipped cards via POST /_screenshot-card (isolated + embedded; fitted is an upstream ask), flag suspected blank renders by size, attach the PNGs to the run log, and push them through the product sync. Acceptance walkthrough: a fresh verifier turn (default budget claude-sonnet-5 @ medium — unforked context, no cache-family concern) reads the PNGs with the image-capable Read tool and verdicts each acceptance criterion — a criterion passes only on visible evidence, never on 'the class exists'. FAILs auto-file defect issues the scheduler picks up next cycle; interactive criteria become NEEDS-HUMAN-VERIFY decisions on the run log. Best-effort: gate failures never un-do a done issue. Defaults (v3): phase-split ON under --v2 (--no-phase-split opts out), render gate + walkthrough ON under --v2 (--no-render-gate opts out), monitor level normal. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Four new list_skills/read_skill-discoverable skills, adapted from the battle-tested workspace docs (workspace-only libraries and realm URLs stripped): boxel-runtime-surfaces (import-path catalogue + capability cheatsheet), boxel-live-surfaces (the searchResultsComponent SearchEntryWireQuery contract that fixed the wardrobe gallery, live-feed recipe, churn + unhydrated-linksTo traps), boxel-delegated-render-control (format choice + child-card chrome traps), and boxel-fetch-contexts (who carries auth where; why raw fetch 401s; request-forward proxy). The v2 operations skill's pointer table routes to all four, and the build prompts now warn that the render gate judges visible affordances. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
--brief-url now accepts a github.com repo URL: the brief is synthesized from the repo's metadata + README, and the seed step writes a PORT-0 analysis issue (issueType 'analysis', NoOp validation) that the bootstrap seed is blockedBy. The analysis turn — a research turn on the session flagship model — pulls the repo tree, README, and every referenced screenshot/GIF/video into a sync-ignored .factory-scratch/ dir, READS the media (frames extracted for video where possible), and writes the port-background Knowledge Article: feature inventory, screen catalogue, inferred data model, UX flows, a 'better than the original' rubric, and the proposed Boxel card-family mapping — then links it onto the bootstrap issue via relatedKnowledge so planning starts from what the app actually looks like, not just its README. Analysis issues are excluded from phase-split, prime forking, budget downgrades, and the render gate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Briefs are authored content; a GitHub repo is source material for an inspired-by port — so the repo gets its own flag. Exactly one of --brief-url / --repo-url is required (a github URL passed as a brief now errors with a pointer instead of a JSON parse failure). The analysis protocol deepens from media-only to code-level: shallow clone into .factory-scratch/, dependency map (each meaningful dep → role → Boxel-side answer or NEEDS-HUMAN-DECISION), specialized-logic inventory with AI prompt templates captured verbatim, the data model as implemented, and a PORTED TEST CONTRACT — assertions mined from the original's tests per feature plus fixture data that seeds the port's sample instances (card-level checks vs walkthrough-level acceptance distinguished). Deliverables become two Knowledge Articles (port-background + port-code-analysis), both linked to bootstrap, whose seed now instructs planning from them: rubric → Project success criteria, test contract → issue acceptance criteria, fixtures → sample instances. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…-log activity feed Batch of software-factory improvements on top of the v2/v3 base. Grouped by area: Restart & scheduling - Seed resume guard: on restart, a bootstrap/analysis/design seed already `done` for the CURRENT brief is left intact instead of re-armed to backlog, so a resumed run stops re-running the repo study / bootstrap. A changed brief still re-arms (multi-brief pass). (factory-seed) - Loop reconciles picked issues against the AUTHORITATIVE local workspace file (readLocalStatus) before running — skips an issue the index still reports in_progress/backlog but the workspace has marked done, closing the control-plane index-lag race. (issue-scheduler, issue-loop) Agent session reuse - Inner fix loop chains sessions: after iteration 1, a fix turn resumes the prior iteration's session (files/edits stay in context, prompt-cached) so the agent fixes instead of re-reading. The orchestrator decides continue-vs-fresh per turn (decideSessionStrategy): continue while converging; fresh (re-fork prime) on a stalled/repeating failure or after a chain cap. (issue-loop) Bug-fix path - Defect/bug issues skip the design round: no phase-split DESIGN turn, and a dedicated diagnose-and-fix prompt (issue-fix-v2) instead of the design-first implement prompt. (issue-loop, factory-prompt-loader, prompts/issue-fix-v2) Run-log activity feed (query-backed model) - Reveal-on-scroll feed (workspace.gts pattern): one 100-cap query, client reveals 20 at a time; non-shifting live updates buffer new arrivals behind a subtle "N new" pill instead of pushing the reader's content down. - Query-backed facet counts; entries link their issue via an atom chip; comment + status-change entries so issue comments and state transitions show in the feed (the containsMany "everything is an entry" behavior, restored in the Posts model); BFM card directives in post_update bodies resolved to live card links with realm routing; design HTML mockup sources preserved in design-history alongside the PNG. (run-log, factory-tool-builder, issue-loop) Infrastructure - Content-hash workspace fingerprint (was mtime) to stop bidirectional-pull sync churn. (validation-run-cache) - Per-turn agent tool telemetry with waste flags. (agent-tool-telemetry) - Control-plane .boxelignore refreshes a stale managed block. (control-plane-sync) - Transient-agent-error classification; instance discovery; workspace skills. All touched suites green (issue-loop, factory-seed, issue-scheduler, run-log, factory-prompt-loader, agent-tool-telemetry, control-plane-sync via node:test, validation-run-cache, instance-discovery, transient-agent-error, factory-context-builder, factory-agent-claude-code). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
What the branch adds over main, why, and how each piece works — the v2 design-first loop, the query-backed run-log activity feed, per-turn model budgets + phase-split, context forking + inner-loop session reuse, the v3 control/product realm split, GitHub-port flow, render gate, resume-safe restarts, the bug-fix fast path, and infra hardening. Written for a reviewer and for operators. Includes a test-suite table and a file map. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The overview undersold how the executor discovers and loads skills. Document both paths properly: the list_skills/read_skill MCP tools, and the `.claude/skills` materialization (workspace-skills.ts) that puts the factory skill catalog on disk so the agent explores it natively with Glob/Grep/Read — the same convention as boxel-workspaces — targeting the unknown-unknown failure mode a bespoke lookup tool can't reach. Added file-map and test-table rows. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…rface The parse tests still asserted the pre-repo-url shape: no controlRealm field and the old "Missing required --brief-url" message. Match the current parser, and round-trip the parsed options through JSON so the shape assertion pins defined defaults instead of enumerating every optional flag. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mechanical eslint --fix / prettier --write; no logic changes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
Preview deploymentsHost Test Results 1 files ±0 1 suites ±0 2h 43m 39s ⏱️ + 1m 52s Results for commit 580b9f2. ± Comparison against earlier commit da9bec4. Realm Server Test Results 1 files ±0 1 suites ±0 17m 6s ⏱️ +54s Results for commit 580b9f2. ± Comparison against earlier commit da9bec4. |
createSeedIssue re-arms a pending seed with the current brief and only reports 'existing' for a done seed belonging to the same brief. The spec still asserted the pre-resume-guard behavior where any existing seed short-circuited to 'existing'. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two bugs surfaced by a live factory run: Spec/ was listed in CONTROL_DIRS, routing Catalog Spec cards to the control realm while the instantiate validation's spec discovery searches the product realm — so whole-realm instantiate could never find a spec, stalling every implementation issue in a 30s-timeout retry loop until the agent hand-wrote a product copy, which the next product sync then deleted as unowned. Spec cards are product artifacts; route them to the product realm and resolve run-log Spec links against it. The run log's captureFeed modifier wrote tracked state (anchorId, newCount, feedFetched) synchronously inside the render transaction after the template had consumed it, tripping Glimmer's backtracking assertion and error-tiling the prerendered card. Defer the reconcile to a microtask outside the render, change-guard every write so re-runs settle, and skip the write entirely once the component is destroyed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
boxel-ui declaring lodash pulls full CJS lodash into the host test build graph, which shifts GC attribution enough to trip the per-module memory baseline check (+249MB on suites whose baseline samples are negative). No boxel-ui source imports lodash; the compiled-dist resolver concern is a separate change that should land with its own baseline refresh. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lands the
factory-v2-experimentbranch: a design-first rework of the software factory build loop plus operational hardening. The full reviewer guide, area by area, is inpackages/software-factory/docs/factory-v2-v3-overview.mdon this branch.In brief:
--v2build loop: HTML mockup → screenshot → critique → translate to.gts; a DESIGN turn on the flagship model, a BUILD turn on a cheaper one forked from the design sessionlist_skills/read_skilltools plus native.claude/skillsmaterialization into the workspace so the agent discovers skills with Glob/Grep/ReadRunLogEntrycards, reveal-on-scroll, buffered live updates)--control-realm: one workspace syncing to a product realm (deliverable cards) and a control realm (issues, run log, validation artifacts), so run-log churn can't invalidate the product index--repo-urlport flow: a PORT-ANALYSIS issue studies a real GitHub app (code included) before bootstrap; render gate judges acceptance criteria against host screenshotsOn top of the original branch:
origin/mainmerged in (no conflicts), the factory-entrypoint tests aligned with the current--repo-url/--control-realmarg surface, and a mechanical prettier pass.Note for running tests:
control-plane-sync.test.tsuses thenode:testrunner (node --test), not the qunit harness. The port-allocator dual-stack test expects Linux socket semantics and fails locally on macOS — on main too, unrelated to this branch.