Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions agent/instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,14 @@ Always begin with `read_run_stage_context`. After planner delegation, call
`apply_implementation_result`. A subagent response alone never changes durable
state.

Route by the returned `run.loopKey` before considering the stage. The declared
development path remains `planning → test-writing → development → validation →
code-review → commit → pr → done`. The research skeleton declares
`planning → researching → authoring → done` for `research-loop`, but its research planner,
researcher fan-out, and research author are intentionally undeclared until
issues #44, #45, and #46. Fail closed for every research stage: do not delegate
to a development sibling, advance durable state, or fabricate an artifact.

After validation-reviewer delegation, call `apply_validation_review_result`.
Only that root tool may apply the review recommendation: `commit` advances;
`development` requeues development, validation, and review; `test-writing`
Expand Down
27 changes: 21 additions & 6 deletions agent/tools/read_run_stage_context.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,13 @@ import { db } from "@/db/client";
import { agentPlans, approvals, artifacts, loopRuns, runSteps } from "@/db/schema";
import { createImplementationFixtureHandoff } from "../implementation-fixture";
import { createPlanningAgentSeedPlan } from "../planning-agent";
import { createPrPreparationFixtureContext } from "../pr-preparation-fixture";
import { resolveImplementerFixtureMode } from "../subagents/implementer/lib/fixture-mode";
import { resolvePrPreparerFixtureMode } from "../subagents/pr-preparer/lib/fixture-mode";
import { resolveTestWriterFixtureMode } from "../subagents/test-writer/lib/fixture-mode";
import { resolveValidationReviewerFixtureMode } from "../subagents/validation-reviewer/lib/fixture-mode";
import { computeTestPlanDigest } from "../test-writing-agent";
import { createValidationReviewFixtureContext } from "../validation-review-fixture";
import { createPrPreparationFixtureContext } from "../pr-preparation-fixture";

export default defineTool({
description: "Read durable run, plan, approval, step, and artifact context for stage routing.",
Expand All @@ -28,7 +28,7 @@ export default defineTool({
uri: artifact.uri,
})),
plans: [{ id: context.planId, plan: context.plan, status: context.planStatus }],
run: context.run,
run: { ...context.run, loopKey: "development-loop" },
steps: [
{ ...context.validationStep, stage: "validation" },
{ ...context.reviewStep, stage: "code-review" },
Expand Down Expand Up @@ -64,7 +64,7 @@ export default defineTool({
},
],
plans: [{ id: context.plan.identity.id, plan: context.plan, status: context.planStatus }],
run: context.run,
run: { ...context.run, loopKey: "development-loop" },
steps: [context.validationStep, { ...context.reviewStep, stage: "code-review" }],
};
}
Expand Down Expand Up @@ -101,7 +101,12 @@ export default defineTool({
},
],
plans: [{ id: planId, plan, status: "approved" }],
run: { currentStage: "development", id: runId, status: "running" },
run: {
currentStage: "development",
id: runId,
loopKey: "development-loop",
status: "running",
},
steps: [
{
id: "00000000-0000-4000-8000-000000000347",
Expand Down Expand Up @@ -143,7 +148,12 @@ export default defineTool({
],
artifacts: [],
plans: [{ id: planId, plan, status: "approved" }],
run: { currentStage: "test-writing", id: runId, status: "running" },
run: {
currentStage: "test-writing",
id: runId,
loopKey: "development-loop",
status: "running",
},
steps: [
{
id: "00000000-0000-4000-8000-000000000347",
Expand All @@ -165,7 +175,12 @@ export default defineTool({
db.select().from(artifacts).where(eq(artifacts.runId, runId)),
]);
return {
run: { currentStage: run.currentStage, id: run.id, status: run.status },
run: {
currentStage: run.currentStage,
id: run.id,
loopKey: run.loopKey,
status: run.status,
},
steps: steps.map(({ id, stage, status }) => ({ id, stage, status })),
plans: plans.map(({ id, plan, status }) => ({ id, plan, status })),
approvals: planApprovals.map(({ id, metadata, status }) => ({ id, metadata, status })),
Expand Down
4 changes: 2 additions & 2 deletions docs/adr/0012-telemetry-backend-and-metric-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ Run cancellations are counted as `loopworks.run.completed` with `status="cancell

### Relationship to `observability_events`

The `observability_events.metric_name` column keeps its snake_case event vocabulary (for example `development_loop_run_created`): those rows are durable control-plane records for audit and replay, per the "logs are not the event store" rule. OTel meters are operational telemetry. Where both exist, one instrumentation helper emits both from a single call site, and the DB event name maps to a meter plus attributes (`development_loop_run_created` → `loopworks.run.started{loop.key="development-loop"}`). Neither vocabulary may be written ad hoc outside `src/lib/observability/` helpers.
The `observability_events.metric_name` column keeps its snake_case event vocabulary (for example `development_loop_run_created` and `research_loop_run_created`): those rows are durable control-plane records for audit and replay, per the "logs are not the event store" rule. OTel meters are operational telemetry. Where both exist, one instrumentation helper emits both from a single call site. Development creation maps to `loopworks.run.started{loop.key="development-loop", trigger.label="agent-ready"}` and research creation maps to `loopworks.run.started{loop.key="research-loop", trigger.label="spike"}`. Neither vocabulary may be written ad hoc outside `src/lib/observability/` helpers.

### Correlation alignment

Expand All @@ -71,7 +71,7 @@ The `observability_events.metric_name` column keeps its snake_case event vocabul
| `repository` | `repositories` | `loopworks.repository` |
| `traceId` | `loop_runs.trace_id`, `run_steps.trace_id`, `observability_events.trace_id` | W3C trace id from active span context |

The active W3C trace id is persisted into the `trace_id` columns at run and step creation, closing the loop between spans, logs, and durable records. Artifacts carry no `trace_id` of their own: `artifacts.run_id` and `artifacts.step_id` correlate them transitively through the owning run and step.
The active W3C trace id is persisted into the `trace_id` columns at development- and research-run creation, their steps, and their durable creation events, closing the loop between spans, logs, and durable records. Artifacts carry no `trace_id` of their own: `artifacts.run_id` and `artifacts.step_id` correlate them transitively through the owning run and step.

### Rollout

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@ Date: 2026-07-13
Driving issues: [#47](https://github.com/ncolesummers/loopworks/issues/47),
[#48](https://github.com/ncolesummers/loopworks/issues/48), and
[#49](https://github.com/ncolesummers/loopworks/issues/49), and
[#50](https://github.com/ncolesummers/loopworks/issues/50)
[#50](https://github.com/ncolesummers/loopworks/issues/50), with the
research-loop generality probe in [#43](https://github.com/ncolesummers/loopworks/issues/43)

## Context

Expand All @@ -31,6 +32,16 @@ implementer, validation-reviewer, and PR-preparer use `openai/gpt-5.6-terra`. Ea
independent `xhigh` reasoning configuration so model routing can evolve per
stage without changing the shared topology.

The same neutral root routes by durable `loopKey`. Issue #43 declares the
research sequence `planning → researching → authoring → done`, actors
`research-planner`, `researcher`, `research-author`, and `loopworks`, and the
artifact boundaries for a research plan, per-subquestion findings index,
research document, and completion summary. It intentionally does not declare
research Eve subagents, live fan-out, provider evals, or versioned research
payload schemas. Until issues #44-#46 declare those siblings and transitions,
the root fails closed rather than routing research stages to development actors
or advancing durable state.

Planner, test-writer, and implementer receive repository-scoped discovery, text
search, and line-range read tools against their isolated commit-pinned
checkouts. The tools
Expand Down Expand Up @@ -129,6 +140,9 @@ deterministic control-plane behavior rather than model judgment.
7. PR-preparation tests cover exact evidence binding, non-UI empty manifests,
idempotent persistence, conflicting replay, and guarded-writer approval.
8. `bun run validate` and `bun run build` pass before review.
9. Research-routing discovery proves `loopKey` is returned, the root fails
closed for undeclared research siblings, and no research subagent directories
or versioned research payload schemas are introduced by #43.

## Follow-Ups

Expand Down
9 changes: 8 additions & 1 deletion docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,8 @@ stage subagent, validates the typed result, and invokes deterministic
control-plane transitions. Stage subagents are siblings with independent model,
tool, and isolated sandbox contracts; they do not own GitHub writes or workflow
state transitions.
Routing begins with the durable `loopKey`, so development and research stage
names cannot accidentally share actors.

The planner subagent should:

Expand All @@ -188,7 +190,12 @@ The planner subagent should:

Stage specialists use typed handoffs and independent capability boundaries:

1. Research loop skeleton, research planner, researcher, and research author agents — the `spike` plus `agent-ready` research loop, run in parallel to the development loop.
1. Research loop skeleton — issue #43 persists and projects `planning →
researching → authoring → done` with actor and artifact placeholders. The
findings contract is `one_per_subquestion` from isolated child sessions, but
no child sessions run yet. Research planner, fan-out, authoring agent, live
eval, and versioned payload work stays in #44-#46; the root fails closed until
those siblings exist.
2. Test-writer subagent — test-writing stage, red test evidence plus a reusable automated test plan, explicit seed data, and a bounded test-only patch.
3. Implementation subagent — development stage, digest-bound production patch
plus signed focused and aggregate green evidence. It reuses the exact
Expand Down
17 changes: 17 additions & 0 deletions docs/loop-manifest.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,23 @@ skipped/no-op reason such as `loop_disabled` so operators can explain why an
`agent-ready` issue did not start. Research-loop disabled evidence is tracked
separately from the development-loop skeleton.

The enabled `research-loop` is a parallel, fixture-backed generality probe. It
requires both `spike` and `agent-ready`, uses a separate
`repo:{repo}:loop:research` concurrency group, and permits only repository read,
browser, and validation tool categories. GitHub writeback is disabled; only
manifest rollout requires approval. Its deterministic sequence and persisted
placeholder contract are:

1. Planning by `research-planner`: one research plan (`plan`).
2. Researching by `researcher`: findings (`other`), indexed
`one_per_subquestion` from isolated child sessions.
3. Authoring by `research-author`: one research document (`other`).
4. Done by `loopworks`: one completion summary (`other`).

Issue #43 creates only the four steps and four placeholders. It does not create
an agent plan or approval row, run live fan-out, or define versioned research
payload schemas; those behaviors remain in #44-#46.

The planning-to-test-writing boundary requires an `approved` `plan-review`
record tied to the exact run, plan row, and canonical plan digest. The
test-writing stage succeeds only when every approved-plan acceptance criterion
Expand Down
13 changes: 8 additions & 5 deletions docs/observability.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,14 +77,17 @@ The internal control plane should eventually expose:

ADR 0012 defines the concrete metric-name contract. The exported metric names
live in `src/lib/observability/metrics.ts`; code should import helpers from that
module instead of naming OTel meters ad hoc. The first wired metric maps the
durable `development_loop_run_created` control-plane event to
`loopworks.run.started` with `loop.key`, `repository`, and `trigger.label`
attributes.
module instead of naming OTel meters ad hoc. The run-creation helpers map
durable `development_loop_run_created` and `research_loop_run_created`
control-plane events to `loopworks.run.started` with `loop.key`, `repository`,
and `trigger.label` attributes. Research creation uses
`loop.key=research-loop` and `trigger.label=spike`; disabled research triggers
persist `research_loop_noop` without fabricating a run or emitting a
run-started metric.

## Tracing Direction

The MVP logger is compatible with trace context. Development-loop run creation
The MVP logger is compatible with trace context. Development- and research-loop run creation
persists the active W3C trace id into `loop_runs.trace_id`,
`run_steps.trace_id`, and `observability_events.trace_id`, allowing Axiom traces,
stdout logs, and durable run records to be correlated.
Expand Down
4 changes: 2 additions & 2 deletions docs/personas-and-test-scenarios.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,9 +93,9 @@ Risks:
| P03 | Product Operator | A durable decision from planning links to an ADR proposal or accepted ADR. | Integration, docs review |
| P04 | Product Operator | An operator switches between light and dark mode from the app shell; the choice persists across reloads and both themes meet contrast. | Playwright, a11y |
| M01 | Maintainer | Catalog rows show owner, framework, CI commands, docs, observability, design-system, enabled loops, Vercel project links, and search/filter controls. | Playwright, Storybook |
| M02 | Maintainer | Turning a loop off prevents trigger execution and records a skipped reason. | Unit, Playwright |
| M02 | Maintainer | Turning either development or research routing off prevents trigger execution and records the loop-specific skipped reason without fabricating a run. | Unit, integration, Playwright |
| M03 | Maintainer | Missing Vercel credentials in dev returns explicit fixture fallback metadata; production does not silently return fixtures. | Unit, integration |
| A01 | Agent Supervisor | Run detail shows planning, test-writing, development, validation, code review, commit, PR, and done stages, including separate red-evidence and automated-test-plan artifacts. | Unit, Storybook, Playwright |
| A01 | Agent Supervisor | Run detail shows the exact loop sequence and artifacts: all development stages with separate red/test-plan evidence, or research planning, researching, authoring, and done with four placeholder contracts. | Unit, Storybook, Playwright |
| A02 | Agent Supervisor | Approval gates show requested, approved, rejected, bypassed, and expired states with actor and evidence; test writing requires an exact approved plan review. | Unit, Storybook, Playwright |
| A03 | Agent Supervisor | AC-mapped expected-red evidence appears before implementation, and green deterministic validation appears before LLM review or judgment. | Integration, Playwright |
| R01 | Reviewer | A PR intent or created PR links to the source issue, run, validation artifacts, and Vercel preview. | Integration, Playwright |
Expand Down
21 changes: 10 additions & 11 deletions scripts/github-webhook-fixture.ts
Original file line number Diff line number Diff line change
Expand Up @@ -51,26 +51,25 @@ function createSignature(secret: string, payloadText: string): string {
}

function createPayload(kind: GithubWebhookFixtureKind): GithubWebhookFixturePayload {
const labels =
kind === "spike-agent-ready"
? ["agent-ready", "spike", "area:loops", "area:agents", "loop:development", "priority:p0"]
: ["agent-ready", "area:loops", "area:agents", "loop:development", "priority:p0"];
const isResearch = kind === "spike-agent-ready";
const labels = isResearch
? ["agent-ready", "spike", "area:loops", "area:agents", "loop:research", "priority:p2"]
: ["agent-ready", "area:loops", "area:agents", "loop:development", "priority:p0"];

return {
action: "labeled",
issue: {
body: "Implement the first durable loop skeleton for issues labeled agent-ready.",
html_url: "https://github.com/ncolesummers/loopworks/issues/11",
body: isResearch
? "Implement a durable, fixture-backed research loop skeleton."
: "Implement the first durable loop skeleton for issues labeled agent-ready.",
html_url: `https://github.com/ncolesummers/loopworks/issues/${isResearch ? 43 : 11}`,
labels: labels.map((name) => ({ name })),
milestone: {
title: "M3 Durable Loop MVP",
},
number: 11,
number: isResearch ? 43 : 11,
state: "open",
title:
kind === "spike-agent-ready"
? "Research agent-ready development loop skeleton"
: "Agent-ready development loop skeleton",
title: isResearch ? "Research loop skeleton" : "Agent-ready development loop skeleton",
},
repository: {
full_name: defaultRepository,
Expand Down
Loading
Loading