Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
35f2670
chore(opi-implement): checkpoint task 16.15.2 ledger (passing — ci.ym…
13luiz Aug 4, 2026
5227257
refactor(opi-coding-agent): remove core native sandbox and enforce mi…
13luiz Aug 4, 2026
e872aa6
chore(opi-implement): checkpoint task 16.16.1 ledger (passing — core …
13luiz Aug 4, 2026
0bf07e7
test(opi-coding-agent): prove install-to-execute and cross-surface di…
13luiz Aug 4, 2026
bc09c65
chore(opi-implement): checkpoint task 16.16.2 ledger (passing — insta…
13luiz Aug 4, 2026
f8aff02
docs(workspace): synchronize Phase 16 docs and close repository gates
13luiz Aug 4, 2026
8bb4b45
chore(opi-implement): checkpoint task 16.16.3 ledger (passing — docs …
13luiz Aug 4, 2026
eb7bed8
chore: archive opi-implement phase 16 ledger snapshot
13luiz Aug 4, 2026
2b23010
fix(execution): remediate phase 16 audit findings
13luiz Aug 5, 2026
2c48c85
docs(phase16): record remediation plan
13luiz Aug 5, 2026
edd8d91
fix(execution): remediate phase 16 audit findings
13luiz Aug 6, 2026
8b547da
chore: reconcile opi-implement harness files with opi-spec.md changes
13luiz Aug 6, 2026
9c1617b
docs(phase16): record independent audit results
13luiz Aug 6, 2026
e3565bb
feat(opi-implement): add grilling step and tier-scoped verification
13luiz Aug 8, 2026
458736f
fix(execution): remediate phase 16 audit findings
13luiz Aug 9, 2026
00da540
chore: reconcile opi-implement harness files with opi-spec.md changes
13luiz Aug 9, 2026
8992025
Merge branch 'main' into codex/phase16-remediation
13luiz Aug 9, 2026
26613ac
refactor(workflow): streamline skills and verification
13luiz Aug 9, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
605 changes: 161 additions & 444 deletions .claude/skills/README.md

Large diffs are not rendered by default.

482 changes: 140 additions & 342 deletions .claude/skills/README.zh.md

Large diffs are not rendered by default.

83 changes: 83 additions & 0 deletions .claude/skills/_shared/references/finding-contract.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# Normalized Finding Contract

Audit and runtime eval artifacts use this interchange when their findings may
enter `opi-remediate`. Narrative reports remain human-readable; each actionable
finding also carries these fields.

```yaml
id: <source-stable identifier>
source_kind: audit | eval
source_path: <repo-relative artifact path>
source_model: <reported reviewer/evaluator identity>
independence: independent-family | fresh-context-same-family | unknown
axis: standards | spec | security | test-quality | invariants | integration | residuals | runtime-fidelity
severity: Blocker | Major | Minor | Info
title: <concise title>
claim: <falsifiable problem statement>
evidence:
- location: <file:line, trace event, artifact, or command>
detail: <observed evidence>
criterion_source: <spec/rule citation or null>
reproduction:
- <command or eval case>
confidence: high | medium | low
status: unverified
```

## Field rules

- `id` is stable within `source_path`. Consumers identify the source finding by
`(source_path, id)`; they do not assume IDs are globally unique.
- `source_kind` distinguishes static/code review evidence from runtime fidelity
evidence. It does not imply either source is more trustworthy.
- `source_model` reports the identity claimed by the producer. Never invent a
model ID.
- `independence` reports the actual relationship to the implementation or model
under evaluation. A fresh context on the same family is degraded independence,
not `independent-family`.
- `axis` preserves Standards and Spec as separate Matt review axes. Opi-specific
audit dimensions and eval use the remaining values.
- `severity` uses the four-tier scale below. Foreign labels are normalized at
ingestion while preserving the original label in the narrative report.
- `claim` must be independently checkable. Recommendations belong outside the
claim.
- `evidence` cites observed facts, not reviewer confidence or conclusions.
- `criterion_source` is `null` only when no normative criterion applies.
- `reproduction` may name an eval case when a direct command is unavailable.
- `status` is always `unverified` at production time.

## Remediation ownership

`opi-remediate` preserves every source field unchanged and records its own
verification status separately as `Confirmed`, `Partially confirmed`, `Cannot
confirm`, or `Refuted`. Consensus clustering may select a candidate severity,
but it never silently reranks an individual source finding. Any final severity
change is recorded with code/trace evidence and rationale.

Malformed finding blocks remain visible in the source report but are not
silently repaired. Remediation reports the missing fields and asks for a source
correction or treats the narrative as an explicitly degraded legacy input.

## Severity scale

| Tier | Meaning |
|---|---|
| **Blocker** | Cannot ship safely: normal-path data loss, credential/user-data exposure, crash or panic on expected input, or a common-path deadlock/infinite loop. |
| **Major** | Incorrect behavior or a significant gap that must be fixed before the next phase: wrong output for valid input, silent edge-case corruption, material spec deviation, cascading error handling failure, or a critical-path test gap. |
| **Minor** | Quality or completeness gap without incorrect behavior: a non-critical test gap, documentation drift, duplicate logic, naming inconsistency, or an unsynchronized localized counterpart. |
| **Info** | Improvement or future consideration rather than a defect: performance opportunity, API ergonomics, scale observation, or documented design trade-off. |

Normalize foreign labels as follows while retaining the producer's original
label in narrative evidence:

| Canonical | Common foreign labels |
|---|---|
| Blocker | P0, Critical |
| Major | P1, High |
| Minor | P2, Medium, Warning |
| Info | P3, Low, Note |

When a label is unfamiliar, map by described impact rather than spelling:
security/data-loss/crash to Blocker; wrong behavior/spec deviation to Major;
quality/test/doc gap to Minor; suggestion/style/future work to Info. A healthy
review usually has few Blockers; do not inflate severity to create urgency.
73 changes: 57 additions & 16 deletions .claude/skills/opi-audit/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
name: opi-audit
disable-model-invocation: true
description: >-
Perform an independent code audit of a specific opi implementation phase.
Given a phase number, automatically extract the task graph, design spec, and
Expand Down Expand Up @@ -48,7 +49,9 @@ rules below exist to protect this independence.

3. Extract the task graph from `tasks[]`:
- Task IDs, titles, crates, `definition_of_done`
- `verified_at_commit` values (first and last define the commit range)
- `verified_at_commit` values. Resolve the first task commit's parent as the
fixed point and the last task commit as HEAD; verify the three-dot diff is
non-empty before review
- `depends_on` relationships

4. If `phase_exit` exists for this phase, extract:
Expand All @@ -61,14 +64,22 @@ rules below exist to protect this independence.

### Phase B: Dimension inference and interview

Seven audit dimensions are available. Not all apply to every phase.
Matt `code-review` supplies two mandatory, separate axes at the ledger-derived
fixed commit range:

| Axis | Question |
|---|---|
| Standards | Does the committed phase diff follow `AGENTS.md` / `CLAUDE.md`, other documented repository standards, and the Matt Fowler-smell baseline? |
| Spec | Does the diff implement the registered source without omissions, incorrect behavior, or scope expansion? |

Do not merge or rerank these axes; a phase may pass one and fail the other.
Opi then adds the applicable phase-wide dimensions below.

| Dimension | When it applies |
|-----------|----------------|
| Correctness | Always |
| Security / redaction | Tasks involving export, user data, credentials, network I/O |
| Test quality | Always (but depth varies) |
| Spec compliance | Always |
| Invariants | When the spec defines explicit invariants or contracts |
| Cross-task integration | Phases with 4+ tasks or multi-crate changes |
| Residuals | Always (catch-all for issues outside other dimensions) |
Expand All @@ -84,7 +95,7 @@ After inferring, briefly confirm with the user:
- Whether any dimensions should be added or dropped

If the user provided a `focus` parameter, weight those dimensions higher but
still cover the basics (correctness, test quality, spec compliance).
still cover the basics (Standards, Spec, correctness, and test quality).

### Phase C: Deep read

Expand All @@ -110,19 +121,26 @@ judgments.

### Phase D: Audit execution

Work through each active dimension. For each finding, follow the template in
`references/finding-template.md` (read it now if you haven't). The template is
a guide, not a straitjacket -- adapt field names or add fields when a finding
needs different structure to be clear.
First open Matt `code-review`. When the phase head is current `HEAD`, invoke it
with the resolved fixed point and registered sources. For a historical phase,
apply its exact two-axis prompts to `git diff <fixed>...<phase-head>` and record
`adapted-historical-range`; do not check out or rewrite the user's working tree.
Its Standards and Spec agents receive this restriction:

**Severity levels** (four-tier):
```text
Do not invoke code-review, opi-audit, or spawn additional agents.
```

| Level | Meaning |
|-------|---------|
| Blocker | Data loss, security vulnerability, crash on normal path, cannot ship |
| Major | Incorrect behavior, unhandled edge case, significant spec deviation |
| Minor | Code quality gap, missing edge-case test, doc inconsistency |
| Info | Improvement suggestion, style preference, future consideration |
Preserve their results under separate `Standards` and `Spec` report headings.
Then run the applicable opi dimensions over the complete phase, including
unchanged production paths needed to verify the committed diff's integration.

Work through each active dimension. For each finding, follow the template in
`references/finding-template.md` (read it now if you haven't). The template is
a guide for narrative clarity. Every actionable finding also emits the exact
normalized block from `../_shared/references/finding-contract.md`, using the
canonical severity definitions in
`../_shared/references/finding-contract.md`.

**Correctness audit**:
- Trace each task's DoD claims against the actual code
Expand All @@ -143,7 +161,7 @@ needs different structure to be clear.
- Verify isolation: temp directories, no shared state, no test ordering deps
- Look for missing negative tests (error paths, rejection paths)

**Spec compliance audit**:
**Spec axis follow-through**:
- Map each Success Criterion from the spec to code evidence
- Verify each Non-Goal is not accidentally implemented
- Check priority tiers (P0/P1/P2) against actual completion
Expand Down Expand Up @@ -215,6 +233,25 @@ uncertain, ask the user.
**Impact:** <consequences if unfixed>
**Fix:** <specific suggested remediation>

```yaml
id: <source-stable identifier>
source_kind: audit
source_path: docs/snapshots/phase<N>/audit.<model-id>.md
source_model: <model-id>
independence: <independent-family | fresh-context-same-family | unknown>
axis: <standards | spec | security | test-quality | invariants | integration | residuals>
severity: <Blocker | Major | Minor | Info>
title: <short title>
claim: <falsifiable problem statement>
evidence:
- location: <file:line or command>
detail: <observed evidence>
criterion_source: <spec/rule citation or null>
reproduction: [<command>]
confidence: <high | medium | low>
status: unverified
```

---

## N+1. Invariant Verification (if applicable)
Expand Down Expand Up @@ -272,3 +309,7 @@ proportional to its independence from prior reviews.
- Read `references/finding-template.md` for the finding format, severity
definitions with examples, and a complete finding example drawn from a real
audit.
- Read `../_shared/references/finding-contract.md` for the machine-stable
interchange consumed by `opi-remediate`.
- Open Matt `code-review` before running the Standards/Spec axes; this skill's
summary is not a substitute for the real subskill.
6 changes: 6 additions & 0 deletions .claude/skills/opi-audit/agents/openai.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
interface:
display_name: "Opi Audit"
short_description: "Audit one implementation phase against its specs"
default_prompt: "Use $opi-audit to independently audit phase=<N> against its committed specification and implementation range."
policy:
allow_implicit_invocation: false
17 changes: 15 additions & 2 deletions .claude/skills/opi-audit/references/finding-template.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@

## Severity definitions

Canonical four-tier definitions and auditor-scale unification live in
`../../_shared/references/finding-contract.md`. The per-tier guidance below is
audit-specific.

### Blocker

The implementation cannot ship safely. Examples:
Expand Down Expand Up @@ -46,8 +50,9 @@ Improvement opportunity or future consideration. Not a defect. Examples:
## Finding format

Each finding should include enough context that a developer can locate and
understand the issue without re-reading the full source. Adapt fields as needed
-- the goal is clarity, not rigid conformance.
understand the issue without re-reading the full source. Narrative fields may be
adapted for clarity, but every actionable finding also includes the normalized
block from `../../_shared/references/finding-contract.md`.

### Recommended fields

Expand All @@ -67,6 +72,14 @@ understand the issue without re-reading the full source. Adapt fields as needed
- **Test gap:** when the finding includes a missing test observation
- **Related:** when multiple findings are connected

### Normalized block

Append a YAML block with the exact fields from the shared finding contract.
For audit output, `source_kind` is `audit`; `axis` preserves `standards` and
`spec` separately or names the applicable opi audit dimension. Set
`status: unverified` even when the auditor is confident: `opi-remediate` owns
independent verification.

## Complete example

This example is drawn from a real Phase 12 audit finding:
Expand Down
Loading
Loading