From 92e38757ecbf8d1bee9a403fcd7356d9c0bdc36a Mon Sep 17 00:00:00 2001 From: Sarthak Agrawal Date: Sat, 15 Aug 2026 20:54:59 +0530 Subject: [PATCH] docs: archive shipped performance specs batch 2 --- .../.openspec.yaml | 0 .../design.md | 0 .../proposal.md | 0 .../qualification.md | 0 .../specs/flow-optimization-campaigns/spec.md | 0 .../tasks.md | 0 .../.openspec.yaml | 0 .../design.md | 0 .../proposal.md | 0 .../qualification.md | 0 .../specs/local-flow-runtime-tools/spec.md | 0 .../tasks.md | 0 .../.openspec.yaml | 0 .../design.md | 0 .../proposal.md | 0 .../specs/runtime-failure-capsules/spec.md | 0 .../tasks.md | 0 .../specs/flow-optimization-campaigns/spec.md | 84 ++++++++++++ .../specs/local-flow-runtime-tools/spec.md | 102 +++++++++++++++ .../specs/runtime-failure-capsules/spec.md | 121 ++++++++++++++++++ 20 files changed, 307 insertions(+) rename openspec/changes/{add-flow-optimization-campaigns => archive/2026-08-15-add-flow-optimization-campaigns}/.openspec.yaml (100%) rename openspec/changes/{add-flow-optimization-campaigns => archive/2026-08-15-add-flow-optimization-campaigns}/design.md (100%) rename openspec/changes/{add-flow-optimization-campaigns => archive/2026-08-15-add-flow-optimization-campaigns}/proposal.md (100%) rename openspec/changes/{add-flow-optimization-campaigns => archive/2026-08-15-add-flow-optimization-campaigns}/qualification.md (100%) rename openspec/changes/{add-flow-optimization-campaigns => archive/2026-08-15-add-flow-optimization-campaigns}/specs/flow-optimization-campaigns/spec.md (100%) rename openspec/changes/{add-flow-optimization-campaigns => archive/2026-08-15-add-flow-optimization-campaigns}/tasks.md (100%) rename openspec/changes/{add-local-flow-runtime-tools => archive/2026-08-15-add-local-flow-runtime-tools}/.openspec.yaml (100%) rename openspec/changes/{add-local-flow-runtime-tools => archive/2026-08-15-add-local-flow-runtime-tools}/design.md (100%) rename openspec/changes/{add-local-flow-runtime-tools => archive/2026-08-15-add-local-flow-runtime-tools}/proposal.md (100%) rename openspec/changes/{add-local-flow-runtime-tools => archive/2026-08-15-add-local-flow-runtime-tools}/qualification.md (100%) rename openspec/changes/{add-local-flow-runtime-tools => archive/2026-08-15-add-local-flow-runtime-tools}/specs/local-flow-runtime-tools/spec.md (100%) rename openspec/changes/{add-local-flow-runtime-tools => archive/2026-08-15-add-local-flow-runtime-tools}/tasks.md (100%) rename openspec/changes/{add-runtime-failure-capsules => archive/2026-08-15-add-runtime-failure-capsules}/.openspec.yaml (100%) rename openspec/changes/{add-runtime-failure-capsules => archive/2026-08-15-add-runtime-failure-capsules}/design.md (100%) rename openspec/changes/{add-runtime-failure-capsules => archive/2026-08-15-add-runtime-failure-capsules}/proposal.md (100%) rename openspec/changes/{add-runtime-failure-capsules => archive/2026-08-15-add-runtime-failure-capsules}/specs/runtime-failure-capsules/spec.md (100%) rename openspec/changes/{add-runtime-failure-capsules => archive/2026-08-15-add-runtime-failure-capsules}/tasks.md (100%) create mode 100644 openspec/specs/flow-optimization-campaigns/spec.md create mode 100644 openspec/specs/local-flow-runtime-tools/spec.md create mode 100644 openspec/specs/runtime-failure-capsules/spec.md diff --git a/openspec/changes/add-flow-optimization-campaigns/.openspec.yaml b/openspec/changes/archive/2026-08-15-add-flow-optimization-campaigns/.openspec.yaml similarity index 100% rename from openspec/changes/add-flow-optimization-campaigns/.openspec.yaml rename to openspec/changes/archive/2026-08-15-add-flow-optimization-campaigns/.openspec.yaml diff --git a/openspec/changes/add-flow-optimization-campaigns/design.md b/openspec/changes/archive/2026-08-15-add-flow-optimization-campaigns/design.md similarity index 100% rename from openspec/changes/add-flow-optimization-campaigns/design.md rename to openspec/changes/archive/2026-08-15-add-flow-optimization-campaigns/design.md diff --git a/openspec/changes/add-flow-optimization-campaigns/proposal.md b/openspec/changes/archive/2026-08-15-add-flow-optimization-campaigns/proposal.md similarity index 100% rename from openspec/changes/add-flow-optimization-campaigns/proposal.md rename to openspec/changes/archive/2026-08-15-add-flow-optimization-campaigns/proposal.md diff --git a/openspec/changes/add-flow-optimization-campaigns/qualification.md b/openspec/changes/archive/2026-08-15-add-flow-optimization-campaigns/qualification.md similarity index 100% rename from openspec/changes/add-flow-optimization-campaigns/qualification.md rename to openspec/changes/archive/2026-08-15-add-flow-optimization-campaigns/qualification.md diff --git a/openspec/changes/add-flow-optimization-campaigns/specs/flow-optimization-campaigns/spec.md b/openspec/changes/archive/2026-08-15-add-flow-optimization-campaigns/specs/flow-optimization-campaigns/spec.md similarity index 100% rename from openspec/changes/add-flow-optimization-campaigns/specs/flow-optimization-campaigns/spec.md rename to openspec/changes/archive/2026-08-15-add-flow-optimization-campaigns/specs/flow-optimization-campaigns/spec.md diff --git a/openspec/changes/add-flow-optimization-campaigns/tasks.md b/openspec/changes/archive/2026-08-15-add-flow-optimization-campaigns/tasks.md similarity index 100% rename from openspec/changes/add-flow-optimization-campaigns/tasks.md rename to openspec/changes/archive/2026-08-15-add-flow-optimization-campaigns/tasks.md diff --git a/openspec/changes/add-local-flow-runtime-tools/.openspec.yaml b/openspec/changes/archive/2026-08-15-add-local-flow-runtime-tools/.openspec.yaml similarity index 100% rename from openspec/changes/add-local-flow-runtime-tools/.openspec.yaml rename to openspec/changes/archive/2026-08-15-add-local-flow-runtime-tools/.openspec.yaml diff --git a/openspec/changes/add-local-flow-runtime-tools/design.md b/openspec/changes/archive/2026-08-15-add-local-flow-runtime-tools/design.md similarity index 100% rename from openspec/changes/add-local-flow-runtime-tools/design.md rename to openspec/changes/archive/2026-08-15-add-local-flow-runtime-tools/design.md diff --git a/openspec/changes/add-local-flow-runtime-tools/proposal.md b/openspec/changes/archive/2026-08-15-add-local-flow-runtime-tools/proposal.md similarity index 100% rename from openspec/changes/add-local-flow-runtime-tools/proposal.md rename to openspec/changes/archive/2026-08-15-add-local-flow-runtime-tools/proposal.md diff --git a/openspec/changes/add-local-flow-runtime-tools/qualification.md b/openspec/changes/archive/2026-08-15-add-local-flow-runtime-tools/qualification.md similarity index 100% rename from openspec/changes/add-local-flow-runtime-tools/qualification.md rename to openspec/changes/archive/2026-08-15-add-local-flow-runtime-tools/qualification.md diff --git a/openspec/changes/add-local-flow-runtime-tools/specs/local-flow-runtime-tools/spec.md b/openspec/changes/archive/2026-08-15-add-local-flow-runtime-tools/specs/local-flow-runtime-tools/spec.md similarity index 100% rename from openspec/changes/add-local-flow-runtime-tools/specs/local-flow-runtime-tools/spec.md rename to openspec/changes/archive/2026-08-15-add-local-flow-runtime-tools/specs/local-flow-runtime-tools/spec.md diff --git a/openspec/changes/add-local-flow-runtime-tools/tasks.md b/openspec/changes/archive/2026-08-15-add-local-flow-runtime-tools/tasks.md similarity index 100% rename from openspec/changes/add-local-flow-runtime-tools/tasks.md rename to openspec/changes/archive/2026-08-15-add-local-flow-runtime-tools/tasks.md diff --git a/openspec/changes/add-runtime-failure-capsules/.openspec.yaml b/openspec/changes/archive/2026-08-15-add-runtime-failure-capsules/.openspec.yaml similarity index 100% rename from openspec/changes/add-runtime-failure-capsules/.openspec.yaml rename to openspec/changes/archive/2026-08-15-add-runtime-failure-capsules/.openspec.yaml diff --git a/openspec/changes/add-runtime-failure-capsules/design.md b/openspec/changes/archive/2026-08-15-add-runtime-failure-capsules/design.md similarity index 100% rename from openspec/changes/add-runtime-failure-capsules/design.md rename to openspec/changes/archive/2026-08-15-add-runtime-failure-capsules/design.md diff --git a/openspec/changes/add-runtime-failure-capsules/proposal.md b/openspec/changes/archive/2026-08-15-add-runtime-failure-capsules/proposal.md similarity index 100% rename from openspec/changes/add-runtime-failure-capsules/proposal.md rename to openspec/changes/archive/2026-08-15-add-runtime-failure-capsules/proposal.md diff --git a/openspec/changes/add-runtime-failure-capsules/specs/runtime-failure-capsules/spec.md b/openspec/changes/archive/2026-08-15-add-runtime-failure-capsules/specs/runtime-failure-capsules/spec.md similarity index 100% rename from openspec/changes/add-runtime-failure-capsules/specs/runtime-failure-capsules/spec.md rename to openspec/changes/archive/2026-08-15-add-runtime-failure-capsules/specs/runtime-failure-capsules/spec.md diff --git a/openspec/changes/add-runtime-failure-capsules/tasks.md b/openspec/changes/archive/2026-08-15-add-runtime-failure-capsules/tasks.md similarity index 100% rename from openspec/changes/add-runtime-failure-capsules/tasks.md rename to openspec/changes/archive/2026-08-15-add-runtime-failure-capsules/tasks.md diff --git a/openspec/specs/flow-optimization-campaigns/spec.md b/openspec/specs/flow-optimization-campaigns/spec.md new file mode 100644 index 00000000..ec177d3e --- /dev/null +++ b/openspec/specs/flow-optimization-campaigns/spec.md @@ -0,0 +1,84 @@ +# flow-optimization-campaigns Specification + +## Purpose +Let an AI coding agent discover and prioritize bounded local performance flows +before entering CodeVetter's existing correctness-gated optimization loop. +## Requirements +### Requirement: Repository flows are discovered without execution first +CodeVetter SHALL use bounded repository qualification to discover exact local +performance workloads before running application code. Only supported adapters, +contained targets, direct timing evidence, and candidates without unsafe safety +flags MUST be eligible for automatic screening. A loopback-only local service +signal MAY remain eligible for the supported Node flow adapters. + +#### Scenario: Safe measured workloads exist +- **WHEN** a repository contains exact Node, Vitest, or Go benchmark workloads with direct timing evidence +- **THEN** CodeVetter returns a bounded deterministic inventory ordered by qualification evidence + +#### Scenario: Candidate may access external state +- **WHEN** qualification detects network, integration, secret, production, or escaping-path evidence +- **THEN** the candidate remains visible as excluded and MUST NOT execute automatically + +#### Scenario: URL text is only local fixture data +- **WHEN** a measured workload contains URL strings but does not invoke a network client +- **THEN** CodeVetter does not classify those strings alone as external execution evidence + +### Requirement: Screening uses existing runtime evidence +CodeVetter SHALL screen at most the caller's bounded flow limit using existing +performance capsules and deterministic diagnosis. Every screened flow MUST +retain exact scope identity, measurement provenance, diagnosis, limitations, +and cleanup state. + +#### Scenario: Exact workload completes +- **WHEN** an eligible discovered workload executes successfully +- **THEN** its flow-campaign entry references its measured supported-scale cost and deterministic diagnosis + +#### Scenario: Workload is incomplete or startup dominated +- **WHEN** a workload fails, times out, lacks a comparable domain metric, or is dominated by runner startup +- **THEN** CodeVetter does not assign an optimization priority and returns the missing evidence as its next action + +### Requirement: Priority combines measured cost with explicit product context +CodeVetter SHALL rank actionable flows by measured supported-scale milliseconds +multiplied by bounded frequency and user-impact weights. Optional project-owned +weights MUST bind to exact candidate identity; absent weights MUST default to +neutral values and remain disclosed as unverified product context. + +#### Scenario: Product weights are supplied +- **WHEN** a valid priority manifest supplies frequency and user-impact weights for a discovered candidate +- **THEN** the result records both weights, their provenance, and the resulting deterministic priority score + +#### Scenario: Product weights are absent +- **WHEN** no matching priority entry exists +- **THEN** CodeVetter uses neutral weights, reports that production frequency and user impact are unknown, and MUST NOT claim production impact + +#### Scenario: Flow is already cheap +- **WHEN** deterministic diagnosis classifies a flow as already fast at its supported scale +- **THEN** the flow is retained as a regression guardrail but MUST rank below actionable optimization flows + +### Requirement: One next action controls the campaign handoff +CodeVetter SHALL return one deterministic next action for the complete plan. It +MUST select the highest-priority actionable flow, request a better workload when +evidence is inadequate, or recommend another repository when no material local +flow remains. + +#### Scenario: Actionable flow leads the plan +- **WHEN** at least one screened flow has actionable diagnosis and comparable supported-scale cost +- **THEN** the next action identifies its exact adapter, target, name, and the manifest inputs required by the existing optimization campaign + +#### Scenario: No actionable flow remains +- **WHEN** every screened flow is already cheap or non-actionable +- **THEN** the next action preserves useful guardrails and recommends profiling a different product flow + +### Requirement: Planner operations remain closed, local, and bounded +The planner SHALL be exposed through machine-readable CLI and MCP operations +with closed arguments. It MUST NOT edit product source, install dependencies, +invoke a model, contact production, infer credentials, or retain raw profiler +artifacts. + +#### Scenario: Agent starts local planning +- **WHEN** an agent calls the planner with a repository, bounded flow count, sample policy, and optional contained priority manifest +- **THEN** CodeVetter performs only the declared local screening work and returns one validated portable result + +#### Scenario: Unknown or unsafe input +- **WHEN** a caller supplies unknown fields, an escaping manifest path, unsupported weights, or an excessive flow count +- **THEN** the operation fails closed before executing a workload diff --git a/openspec/specs/local-flow-runtime-tools/spec.md b/openspec/specs/local-flow-runtime-tools/spec.md new file mode 100644 index 00000000..91595673 --- /dev/null +++ b/openspec/specs/local-flow-runtime-tools/spec.md @@ -0,0 +1,102 @@ +# local-flow-runtime-tools Specification + +## Purpose +Give coding agents bounded machine operations that capture and interrogate local application flows using runtime evidence rather than prompt-only profiling instructions. +## Requirements +### Requirement: Exact local flow capture +CodeVetter SHALL capture one exact supported local workload as a root flow without requiring source-code modification, arbitrary shell execution, or a hosted service. The capture result MUST record the adapter, exact target and name, revision identity, capture policy, executions, limitations, and cleanup outcome. + +#### Scenario: Agent captures a Node HTTP test +- **WHEN** an agent requests an exact repository-contained Node test through the local flow capture tool +- **THEN** CodeVetter runs only that bounded test scope and returns an opaque capture identifier plus a compact root-flow summary + +#### Scenario: Unsupported or incomplete capture +- **WHEN** the exact workload is unsupported, fails, times out, escapes the repository, or produces incomplete required evidence +- **THEN** CodeVetter returns `no_confidence` and MUST NOT create an actionable optimization claim + +### Requirement: Recursive flow evidence +CodeVetter SHALL represent a captured workload as recursively related flows with stable capture-local identifiers. Each flow MUST distinguish observed elapsed time from inferred or unaccounted time and MUST cite the runtime evidence that created it. + +#### Scenario: Local HTTP client and server activity +- **WHEN** a captured Node workload performs loopback HTTP requests handled in the same diagnostic execution +- **THEN** the result includes bounded client and server child flows with method, normalized route, status, elapsed time, and causal relationships where correlation is supported + +#### Scenario: Request-scoped built-in SQLite activity +- **WHEN** an observed Node HTTP handler executes built-in `node:sqlite` statements +- **THEN** each execution is nested beneath that server flow with its operation, normalized value-free statement shape, outcome, and elapsed time + +#### Scenario: Same-execution child accounting +- **WHEN** a diagnostic parent and its children have comparable timestamps from the same execution +- **THEN** CodeVetter reports interval-union accounted time and remaining unaccounted time without exceeding the parent duration + +#### Scenario: Flow detail is unavailable +- **WHEN** runtime instrumentation cannot account for a portion of root-flow elapsed time +- **THEN** CodeVetter reports the portion as unaccounted or unavailable rather than assigning it to a source location + +### Requirement: Progressive machine queries +CodeVetter SHALL expose closed-schema machine operations to capture, inspect, explain, and verify local flows. Query operations MUST accept opaque identifiers returned by prior operations and MUST return bounded structured content without requiring an agent skill to parse raw profile formats. + +#### Scenario: Untrained agent inspects a capture +- **WHEN** an MCP client lists tools and calls the inspection operation with a valid capture identifier +- **THEN** the client receives the flow hierarchy, evidence coverage, materiality, and limitations without reading a V8 or pprof artifact + +#### Scenario: Unknown identifier or argument +- **WHEN** a client supplies an unknown tool argument, flow identifier, or capture identifier +- **THEN** the tool fails closed with a sanitized bounded error + +### Requirement: Evidence, inference, and actionability remain separate +CodeVetter MUST keep direct observations, deterministic interpretations, unverified hypotheses, and verified comparisons separate. A sampled source location SHALL be actionable only when it satisfies the recorded materiality policy and repeats across independent diagnostic profiles or is supported by a compatible deterministic domain metric. + +#### Scenario: Low-sample unstable hotspot +- **WHEN** independent diagnostic profiles disagree on the leading source candidate or the candidate is immaterial to the root flow +- **THEN** CodeVetter returns `no_confidence` with the missing evidence and MUST NOT recommend editing that source location + +#### Scenario: Stable material candidate +- **WHEN** independent profiles agree on a repository-owned candidate and the candidate passes the recorded sample, duration, and share thresholds +- **THEN** CodeVetter may return an unverified actionable hypothesis with an explicit falsification experiment + +#### Scenario: Repeated application function intersects CPU evidence +- **WHEN** bounded V8 coverage records a named repository application function repeatedly and CPU evidence selects the same file/function family +- **THEN** CodeVetter reports a repeated-work hypothesis with both evidence references and an explicit identical-scope verification experiment + +#### Scenario: Function frequency lacks timing support +- **WHEN** a repository function executes frequently but does not intersect material CPU evidence +- **THEN** CodeVetter reports observed frequency only and MUST NOT call the function slow or actionable + +### Requirement: Identical-scope optimization verification +CodeVetter SHALL compare compatible baseline and candidate captures using identical workload identity and unprofiled measurements. Verification MUST distinguish mechanical improvement from material product impact. + +#### Scenario: Agent verifies a candidate change +- **WHEN** an agent captures the same flow before and after one candidate change and requests verification +- **THEN** CodeVetter reports capture identifiers, compatibility, measured movement, statistical limitations, mechanical confirmation, material usefulness, and whether shipping is recommended without embedding either complete source capsule + +#### Scenario: Incompatible flows +- **WHEN** the adapter, target, exact name, or required measurement identity differs +- **THEN** verification returns `no_confidence` and MUST NOT confirm the optimization + +### Requirement: Local privacy, containment, and cost bounds +CodeVetter MUST redact captured output before normalization, retain no raw profile by default, remove owned temporary artifacts, bound executions and stored captures, and avoid hosted-service or production configuration access. + +#### Scenario: Capture completes successfully +- **WHEN** a local capture finishes +- **THEN** raw owned diagnostic artifacts are removed and the result records redaction, truncation, and temporary-artifact retention state + +#### Scenario: Runtime flow contains request data +- **WHEN** HTTP instrumentation observes a URL containing query values or variable-looking path segments +- **THEN** CodeVetter omits query values and normalizes sensitive-looking segments before returning or storing the flow + +#### Scenario: SQL execution contains application values +- **WHEN** request-scoped SQLite execution uses literals or bound arguments +- **THEN** CodeVetter captures neither arguments nor rows and replaces SQL literals with placeholders before returning or storing the statement shape + +#### Scenario: Function coverage capture completes +- **WHEN** the application-frequency diagnostic pass completes +- **THEN** CodeVetter retains only bounded repository-relative function names, source anchors, and counts and removes the raw V8 coverage documents + +#### Scenario: Nested Vitest assertion is selected by leaf name +- **WHEN** an agent supplies a leaf test name inside one or more Vitest `describe` blocks +- **THEN** CodeVetter executes exactly one matching assertion, or returns no confidence when the leaf name is absent or ambiguous + +#### Scenario: Vitest records transformed TypeScript execution +- **WHEN** the repository-local Vitest V8 coverage provider is available +- **THEN** CodeVetter writes its JSON report only to an owned temporary directory and normalizes positive named functions against original TypeScript locations diff --git a/openspec/specs/runtime-failure-capsules/spec.md b/openspec/specs/runtime-failure-capsules/spec.md new file mode 100644 index 00000000..e151c61e --- /dev/null +++ b/openspec/specs/runtime-failure-capsules/spec.md @@ -0,0 +1,121 @@ +# runtime-failure-capsules Specification + +## Purpose +Define a bounded machine-readable diagnosis for common Node, browser, Cloudflare Worker, and Go verification failures without requiring a universal debugger or turning incomplete evidence into proof. +## Requirements +### Requirement: Supported runtime lanes are detected from repository evidence +CodeVetter SHALL detect Node test, browser test, Cloudflare Worker test, and Go +test lanes only from bounded repository manifests and configuration. Detection +MUST report the evidence, supported adapters, and limitations and MUST NOT claim +that a detected lane has executed successfully. + +#### Scenario: Worker Vitest repository is detected +- **WHEN** a repository contains a package manifest, Vitest configuration, and a Wrangler configuration +- **THEN** CodeVetter reports Node test and Cloudflare Worker test lanes with the evidence paths that established them + +#### Scenario: Unsupported repository is inspected +- **WHEN** no supported manifest or test configuration is found +- **THEN** CodeVetter returns an empty support set and explicit limitations without running a guessed command + +### Requirement: Diagnostic execution is exact, bounded, and shell-free +CodeVetter SHALL execute only a closed adapter with one repository-relative test +target and optional exact test-name selector. Supported executable adapters in +the first slice SHALL be Node test, Vitest, Playwright, and Go test. Each run +MUST use separated program arguments, a declared timeout, bounded output, +minimal inherited environment, and owned process termination. + +#### Scenario: Exact failing test is rerun +- **WHEN** the selected adapter, test target, and optional test name are valid and available +- **THEN** CodeVetter runs only that declared diagnostic scope and records the exact executable identity and arguments without invoking a shell + +#### Scenario: Target escapes the repository +- **WHEN** a test target is absolute, traverses outside the repository, resolves through an escaping symlink, or is not a regular file +- **THEN** CodeVetter rejects the run before starting a process + +#### Scenario: Diagnostic execution times out +- **WHEN** the owned diagnostic process exceeds its declared timeout +- **THEN** CodeVetter terminates it, records the timeout as an operational limitation, and returns `no_confidence` + +### Requirement: Failure capsules separate evidence from interpretation +Each diagnostic run SHALL return one versioned Runtime Failure Capsule with +subject identity, adapter identity, exact scope, terminal state, observations, +source frames, relevant changes, limitations, capture coverage, and a verdict. +Directly captured observations MUST remain separate from deterministic +relationships and unverified hypotheses. The first slice MUST NOT ask a model +to create evidence or a verdict. + +#### Scenario: Test fails with a changed source frame +- **WHEN** a diagnostic test reproduces a failure and its stack contains a source frame intersecting the selected Git diff +- **THEN** the capsule records the exception or panic as observed evidence and the frame-to-change match as a deterministic relationship + +#### Scenario: Diagnostic rerun does not reproduce +- **WHEN** the selected diagnostic scope exits successfully or contains no qualifying failure +- **THEN** the capsule returns `no_confidence`, states that the failure did not reproduce, and does not invent a likely cause + +### Requirement: Source and diff correlation is deterministic and bounded +CodeVetter SHALL normalize repository-contained source frames, inspect one +explicit Git diff range or the local diff, and rank relevant changed files and +lines using deterministic rules. Changed-frame intersection SHALL outrank +same-file proximity, and absent matches SHALL remain an explicit evidence gap. + +#### Scenario: Stack line is changed +- **WHEN** an observed frame points to a line added or modified by the selected diff +- **THEN** that file and line rank first with reason `changed_frame_intersection` + +#### Scenario: Failure has only dependency frames +- **WHEN** every observed frame is outside the repository or under excluded dependency/generated roots +- **THEN** CodeVetter records no relevant source match and preserves the source-attribution limitation + +### Requirement: Captured data is redacted and bounded before output +CodeVetter MUST redact credential-shaped keys and values, authorization and +cookie material, configured sensitive fields, environment values, URL query +values, and repository-absolute path prefixes before evidence enters a capsule. +Collections, strings, stack frames, output bytes, and artifacts MUST have hard +bounds, and truncation MUST be disclosed. + +#### Scenario: Failure output contains a token +- **WHEN** stdout, stderr, or an imported receipt contains credential-shaped material +- **THEN** the capsule contains a redaction marker instead of the material and records that redaction occurred + +#### Scenario: Runner emits oversized output +- **WHEN** captured output exceeds its byte bound +- **THEN** CodeVetter retains only the bounded prefix or suffix required by policy and records truncation without treating capture as complete + +### Requirement: Existing browser and Worker evidence is normalized, not rerun by a new engine +CodeVetter SHALL accept bounded existing T-Rex, warm-verification, Playwright, +and Worker-test result documents as imported observations. The original receipt +identity, verdict, limitations, and provenance MUST remain authoritative; the +capsule MUST NOT upgrade `failed` or `no_confidence` evidence to a pass. + +#### Scenario: Failed Playwright receipt is imported +- **WHEN** a bounded receipt contains a failed browser test, page exception, console failure, or network failure +- **THEN** CodeVetter maps those facts into normalized observations while retaining the source receipt identity and limitations + +#### Scenario: Worker receipt is incomplete +- **WHEN** a Worker test result lacks terminal or source identity +- **THEN** the capsule reports incomplete imported evidence and returns `no_confidence` + +### Requirement: Machine interface has stable outcomes +The repository CLI SHALL expose lane detection and diagnostic execution with +JSON output. It SHALL exit `1` for a reproduced executable failure, `2` for +`no_confidence`, invalid input, or operational failure, and `0` only for a +successful detection request. A diagnostic failure capsule MUST NOT be treated +as proof that the overall change fails beyond its exact scope. + +#### Scenario: JSON diagnostic reproduces a failure +- **WHEN** the CLI runs a selected diagnostic scope with `--json` and captures a qualifying failure +- **THEN** stdout contains exactly one capsule document and the process exits `1` + +#### Scenario: Detection succeeds +- **WHEN** the CLI inspects a valid repository without executing tests +- **THEN** stdout contains one support report and the process exits `0` + +### Requirement: Power-law coverage is measured against owned fixtures +CodeVetter SHALL maintain a small owned corpus spanning Node exceptions, +asynchronous failures, browser or Worker receipt failures, Go panics, redaction, +non-reproduction, timeout, and changed-line attribution. Published coverage +claims MUST derive from executed corpus results rather than stack inventory. + +#### Scenario: Corpus qualification runs +- **WHEN** the focused qualification command executes the owned fixtures +- **THEN** it reports per-lane reproduction, relevant-file attribution, redaction, and no-confidence outcomes with no provider, network, browser download, or production dependency