Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,11 @@ summary: Chronological history of repository and skill changes.

# Changelog

## 2026-07-28 — Required passing validation and current-head lens evidence for a clean review verdict
## 2026-07-28 — Added consumer/impact-traversal evidence and required passing validation and current-head lens evidence for a clean review verdict

- feat: add consumer/impact-traversal evidence to the shared review contract
- feat: require passing validation and current-head lens evidence for a clean
review verdict
review verdict (`b1e51979628652e4ef60adad44089bf54f4551e7`)

## 2026-07-27 — Made database comparison output ephemeral, enforced untrusted-content boundaries, bound epic delegation, hardened command execution, populated the solution-simplicity and code-simplicity strata, enforced acceptance-gated closeout, populated the correctness stratum, recovered carved suffixes, folded owner adjudications, and ran the frozen v1 baseline

Expand Down
43 changes: 43 additions & 0 deletions review-suite/CONTRACT.md
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,49 @@ after a correctness fix, or only code simplicity and correctness after a
code-simplicity fix) cannot produce a valid `clean` aggregate. This child
defines no selective-reuse exception across different heads.

### Consumer/impact evidence

A `correctness` or `aggregate` result may record `consumer_impact_evidence`: one
entry per changed shared symbol or contract whose other call sites/consumers
were traversed, each naming the `changed_symbol`, its defining `location`, the
`consumer_search_evidence` inspected (one or more `location` + `detail` pairs
describing what was found), and a `disposition` of `all_consumers_consistent`,
`inconsistency_found`, or `no_other_consumers`.

This makes a reviewer's consumer/impact traversal machine-checkable instead of
an unenforced expectation a reviewer can silently skip. The validator enforces
structure and non-emptiness; it does not determine which changed symbols require
an entry — that judgment belongs to the lens performing the traversal (a later
child consumes this schema to populate it). Given a supplied entry:

- only `correctness` or `aggregate` results may include this evidence;
- every entry requires at least one concrete `consumer_search_evidence` item,
mirroring the existing "empty impact is valid only with concrete search
evidence" principle used elsewhere in this contract family — a disposition is
never accepted on the strength of an unevidenced claim; and
- `all_consumers_consistent` and `inconsistency_found` describe at least one
other consumer by definition, so each requires search evidence covering more
than the changed symbol's own location (two or more entries);
`no_other_consumers` requires only the one concrete search that found nothing
else.

This validator deliberately cannot decide *whether* a given changed symbol
needed an entry at all, and an aggregate `clean` result that omits
`consumer_impact_evidence` entirely is schema-valid. That is not an oversight:
`scripts/validate.py` takes only a packet and a result as input and has no
repository checkout to search, so it cannot itself determine whether a changed
symbol has other call sites — the real baseline miss this evidence exists to
surface involved a sibling call site the diff never touched, which only live
repository access (available to the reviewing agent, not to this validator) can
find. Building that determination into the validator would be exactly the
independent correctness explorer and static call-graph tooling this contract
family's non-goals rule out. Completeness of a given traversal — did the lens
find every consumer that mattered — is judged by the lens performing the
traversal and by forward-testing its output against a fixture's expected result,
the same way this contract family already judges any other lens-specific finding
(a duplicated-policy or behavior-bug miss is likewise never something this
schema-and-structure validator can detect on its own).

## Simplification proposal dispositions

When an orchestrator asks correctness to assess a validated simplification
Expand Down
39 changes: 38 additions & 1 deletion review-suite/contracts/review-result.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
"blocking_reasons"
],
"properties": {
"schema_version": {"const": "1.1"},
"schema_version": {"const": "1.2"},
"lens": {
"enum": [
"correctness",
Expand Down Expand Up @@ -108,6 +108,43 @@
"type": "array",
"items": {"type": "string", "minLength": 1}
},
"consumer_impact_evidence": {
"type": "array",
"items": {
"type": "object",
"additionalProperties": false,
"required": [
"changed_symbol",
"location",
"consumer_search_evidence",
"disposition"
],
"properties": {
"changed_symbol": {"type": "string", "minLength": 1},
"location": {"type": "string", "minLength": 1},
"consumer_search_evidence": {
"type": "array",
"minItems": 1,
"items": {
"type": "object",
"additionalProperties": false,
"required": ["location", "detail"],
"properties": {
"location": {"type": "string", "minLength": 1},
"detail": {"type": "string", "minLength": 1}
}
}
},
"disposition": {
"enum": [
"all_consumers_consistent",
"inconsistency_found",
"no_other_consumers"
]
}
}
}
},
"proposal_dispositions": {
"type": "array",
"items": {
Expand Down
2 changes: 1 addition & 1 deletion review-suite/fixtures/auth-regression/expected.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"schema_version": "1.1",
"schema_version": "1.2",
"lens": "correctness",
"candidate": {
"head_sha": "8181818181818181818181818181818181818181",
Expand Down
2 changes: 1 addition & 1 deletion review-suite/fixtures/behavior-bug/expected.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"schema_version": "1.1",
"schema_version": "1.2",
"lens": "correctness",
"candidate": {
"head_sha": "1111111111111111111111111111111111111111",
Expand Down
2 changes: 1 addition & 1 deletion review-suite/fixtures/clean-change/expected.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"schema_version": "1.1",
"schema_version": "1.2",
"lens": "aggregate",
"candidate": {
"head_sha": "5555555555555555555555555555555555555555",
Expand Down
2 changes: 1 addition & 1 deletion review-suite/fixtures/code-simplicity-clean/expected.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"schema_version": "1.1",
"schema_version": "1.2",
"lens": "code_simplicity",
"candidate": {
"head_sha": "8c8c8c8c8c8c8c8c8c8c8c8c8c8c8c8c8c8c8c8c",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"schema_version": "1.1",
"schema_version": "1.2",
"lens": "code_simplicity",
"candidate": {
"head_sha": "8989898989898989898989898989898989898989",
Expand Down
63 changes: 63 additions & 0 deletions review-suite/fixtures/consumer-impact-traversal/expected.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
{
"schema_version": "1.2",
"lens": "aggregate",
"candidate": {
"head_sha": "6262626262626262626262626262626262626262",
"comparison_base_sha": "d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2"
},
"verdict": "clean",
"findings": [],
"blocking_reasons": [],
"lens_executions": [
{
"lens": "solution_simplicity",
"head_sha": "6262626262626262626262626262626262626262",
"comparison_base_sha": "d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2",
"verdict": "clean",
"freshly_executed": true
},
{
"lens": "correctness",
"head_sha": "6262626262626262626262626262626262626262",
"comparison_base_sha": "d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2",
"verdict": "clean",
"freshly_executed": true
},
{
"lens": "code_simplicity",
"head_sha": "6262626262626262626262626262626262626262",
"comparison_base_sha": "d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2",
"verdict": "clean",
"freshly_executed": true
}
],
"consumer_impact_evidence": [
{
"changed_symbol": "dependency_finalized",
"location": "lib/dependency_finalized.py:1",
"consumer_search_evidence": [
{
"location": "lib/orchestrator.py:10",
"detail": "Call site updated in this diff to pass strict=True explicitly, matching the new default."
},
{
"location": "lib/worker.py:9",
"detail": "Repository-wide search found this existing call site already passes strict=True; confirmed consistent with the hardened default, no change required."
}
],
"disposition": "all_consumers_consistent"
},
{
"changed_symbol": "_format_summary",
"location": "lib/summary.py:4",
"consumer_search_evidence": [
{
"location": "lib/summary.py:2",
"detail": "Repository-wide search found exactly one caller, render_summary in the same file; no other module imports _format_summary."
}
],
"disposition": "no_other_consumers"
}
],
"next_action": "No material review action is required."
}
24 changes: 24 additions & 0 deletions review-suite/fixtures/consumer-impact-traversal/packet.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
{
"schema_version": "1.0",
"repository": {"identity": "example/pipeline", "base_branch": "main"},
"candidate": {
"head_sha": "6262626262626262626262626262626262626262",
"comparison_base_sha": "d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2",
"diff": {
"format": "unified_diff",
"complete": true,
"content": "diff --git a/lib/dependency_finalized.py b/lib/dependency_finalized.py\n--- a/lib/dependency_finalized.py\n+++ b/lib/dependency_finalized.py\n@@ -1,4 +1,4 @@\n-def dependency_finalized(status, strict=False):\n+def dependency_finalized(status, strict=True):\n if strict and status != \"ready\":\n raise DependencyError(status)\n return status == \"ready\"\ndiff --git a/lib/orchestrator.py b/lib/orchestrator.py\n--- a/lib/orchestrator.py\n+++ b/lib/orchestrator.py\n@@ -9,2 +9,2 @@\n def finalize_stage(status):\n- return dependency_finalized(status)\n+ return dependency_finalized(status, strict=True)\ndiff --git a/lib/summary.py b/lib/summary.py\n--- a/lib/summary.py\n+++ b/lib/summary.py\n@@ -1,7 +1,7 @@\n def render_summary(counts):\n- return _format_summary(counts, verbose=False)\n+ return _format_summary(counts, verbose=True)\n \n-def _format_summary(counts, verbose=False):\n+def _format_summary(counts, verbose=True):\n if verbose:\n return \", \".join(f\"{key}={value}\" for key, value in counts.items())\n return str(len(counts))\n"
}
},
"change_contract": {
"goal": "Make dependency finalization strict by default and render summaries verbosely by default.",
"acceptance_criteria": ["dependency_finalized defaults to strict verification.", "Every existing call site of dependency_finalized behaves consistently under the new default.", "Summary rendering defaults to verbose output."],
"non_goals": ["Change the summary output format itself."],
"preserved_behaviors": ["dependency_finalized still raises DependencyError for a non-ready status under strict mode."]
},
"sources": {"repository_instructions": [], "named_documents": [], "nearby_patterns": []},
"validation": [
{"name": "pipeline tests", "command": "pytest tests/test_pipeline.py", "scope": "focused", "status": "passed", "result": "6 passed"},
{"name": "full tests", "command": "pytest", "scope": "full", "status": "passed", "result": "48 passed"}
]
}
2 changes: 1 addition & 1 deletion review-suite/fixtures/duplicated-policy/expected.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"schema_version": "1.1",
"schema_version": "1.2",
"lens": "code_simplicity",
"candidate": {
"head_sha": "2222222222222222222222222222222222222222",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"schema_version": "1.1",
"schema_version": "1.2",
"lens": "code_simplicity",
"candidate": {
"head_sha": "8b8b8b8b8b8b8b8b8b8b8b8b8b8b8b8b8b8b8b8b",
Expand Down
2 changes: 1 addition & 1 deletion review-suite/fixtures/imagined-machinery/expected.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"schema_version": "1.1",
"schema_version": "1.2",
"lens": "solution_simplicity",
"candidate": {
"head_sha": "3333333333333333333333333333333333333333",
Expand Down
1 change: 1 addition & 0 deletions review-suite/fixtures/manifest.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
{"name": "speculative-backfill", "packet_valid": true},
{"name": "missing-simplification-requirements", "packet_valid": true},
{"name": "clean-change", "packet_valid": true},
{"name": "consumer-impact-traversal", "packet_valid": true},
{"name": "missing-evidence", "packet_valid": false},
{"name": "unrelated-base-drift", "packet_valid": true}
]
2 changes: 1 addition & 1 deletion review-suite/fixtures/missing-evidence/expected.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"schema_version": "1.1",
"schema_version": "1.2",
"lens": "aggregate",
"candidate": {
"head_sha": "6666666666666666666666666666666666666666",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"schema_version": "1.1",
"schema_version": "1.2",
"lens": "solution_simplicity",
"candidate": {
"head_sha": "8686868686868686868686868686868686868686",
Expand Down
2 changes: 1 addition & 1 deletion review-suite/fixtures/missing-test/expected.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"schema_version": "1.1",
"schema_version": "1.2",
"lens": "correctness",
"candidate": {
"head_sha": "8282828282828282828282828282828282828282",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"schema_version": "1.1",
"schema_version": "1.2",
"lens": "code_simplicity",
"candidate": {
"head_sha": "8888888888888888888888888888888888888888",
Expand Down
2 changes: 1 addition & 1 deletion review-suite/fixtures/necessary-complexity/expected.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"schema_version": "1.1",
"schema_version": "1.2",
"lens": "solution_simplicity",
"candidate": {
"head_sha": "4444444444444444444444444444444444444444",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"schema_version": "1.1",
"schema_version": "1.2",
"lens": "correctness",
"candidate": {
"head_sha": "8383838383838383838383838383838383838383",
Expand Down
2 changes: 1 addition & 1 deletion review-suite/fixtures/shared-test-setup/expected.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"schema_version": "1.1",
"schema_version": "1.2",
"lens": "code_simplicity",
"candidate": {
"head_sha": "8a8a8a8a8a8a8a8a8a8a8a8a8a8a8a8a8a8a8a8a",
Expand Down
2 changes: 1 addition & 1 deletion review-suite/fixtures/speculative-backfill/expected.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"schema_version": "1.1",
"schema_version": "1.2",
"lens": "solution_simplicity",
"candidate": {
"head_sha": "8585858585858585858585858585858585858585",
Expand Down
2 changes: 1 addition & 1 deletion review-suite/fixtures/unrelated-base-drift/expected.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"schema_version": "1.1",
"schema_version": "1.2",
"lens": "aggregate",
"candidate": {
"head_sha": "7777777777777777777777777777777777777777",
Expand Down
2 changes: 1 addition & 1 deletion review-suite/scripts/evals/calibration.py
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ def kinds(self) -> frozenset[str]:
def probe_result(probe: dict[str, Any], candidate: dict[str, Any]) -> dict[str, Any]:
"""Render one probe as a review result bound to the case's candidate."""
return {
"schema_version": "1.1",
"schema_version": "1.2",
"lens": "aggregate",
"candidate": candidate,
"verdict": probe["verdict"],
Expand Down
2 changes: 1 addition & 1 deletion review-suite/scripts/evals/fixture_executor.py
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ def _clean_lens_executions(candidate: dict[str, Any]) -> list[dict[str, Any]]:

def _review(candidate: dict[str, Any], verdict: str, findings: list[dict[str, Any]]):
result = {
"schema_version": "1.1",
"schema_version": "1.2",
"lens": "aggregate",
"candidate": dict(candidate),
"verdict": verdict,
Expand Down
Loading
Loading