feat(review-suite): add connector-outcome curation and promotion tooling

CHANGELOG.md

@@ -4,9 +4,17 @@ summary: Chronological history of repository and skill changes.
 
 # Changelog
 
-## 2026-07-29 — Migrated implement-ticket and babysit-pr to consume the final review-result contract, and rechecked the s2/s3 strata under grader 1.1 for the same surface-in-prose defect
+## 2026-07-29 — Migrated implement-ticket and babysit-pr to consume the final review-result contract, rechecked the s2/s3 strata under grader 1.1 for the same surface-in-prose defect, and added connector-outcome curation and promotion tooling
 
+- refactor(review-suite): simplify duplicate-chain resolution and unify its
+  membership check
+- fix(review-suite): resolve a duplicate's disposition through its duplicate_of
+  chain (`07baa7dfdf06bfa19428bb9ba80a8317f8ff78d0`)
+- feat(review-suite): add connector-outcome curation and promotion tooling,
+  including the mechanical disclosure guardrail
+  (`d7357ee17a616ad374e6bb033a4c9adef6e5cc0a`)
 - docs: fix stale CHANGELOG SHAs left by the main rebase
+  (`51cc734fc56a97dfa7a754fd046206dd62b375ba`)
 - docs: backfill the CHANGELOG entry for the review_gate.py canonicalization fix
   (`e2310bff8cc9c3a38b690a57844436d5357fa471`)
 - fix: canonicalize review_gate.py through the existing sync-contracts mechanism

README.md

@@ -246,6 +246,21 @@ quoting any figure — in particular, **the connector stratum is deferred, not
 satisfied**, so connector-escape recall has never been measured and no
 human-review figure may be reported as a connector figure.
 
+### Connector-outcome curation and promotion
+
+`review-suite/evals/curation/` turns a newly adjudicated connector finding into
+a versioned curation record, and turns a group of curation records into a
+conservative, evidence-backed promotion decision: a corpus case only, a global
+rubric change, a repository-instruction change, or nothing. It is infrastructure
+only, proven with synthetic fixtures, and never touches `baseline/v1/` or `v2/`.
+See [its README](review-suite/evals/curation/README.md) for the disposition
+vocabulary, the mechanical disclosure guardrail that fails closed on a leaked
+source identifier, and the promotion rules.
+
+```bash
+just audit-review-curation  # curation and promotion-decision integrity, no runtime
+```
+
 ## Prerequisites
 
 - Python 3.11+

justfile

@@ -56,6 +56,14 @@ test-review-suite:
 audit-review-corpus:
   python3 review-suite/scripts/evals/audit_corpus.py
 
+# Validate connector-outcome curation records and promotion decisions: intake
+# schema, disposition vocabulary, duplicate/unresolved handling, provenance and
+# retention fields, reviewer/private separation, the mechanical disclosure
+# guardrail, and promotion-decision evidence and target rules. Never scrapes
+# GitHub, never mutates a review thread, and never launches a model.
+audit-review-curation:
+  python3 review-suite/scripts/evals/audit_curation.py
+
 # Result-blind replay evaluation through an explicit real-runtime executor.
 # Deliberately excluded from `test`, `lint`, and `check`: this is the only
 # review-suite command that may spend money.

review-suite/evals/contracts/curation-record.schema.json

@@ -0,0 +1,160 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://github.com/shaug/agent-scripts/review-suite/evals/contracts/curation-record.schema.json",
+  "title": "Connector-outcome curation record",
+  "description": "One adjudicated connector finding, curated for possible promotion into a result-blind regression case (see `../corpus/corpus.schema.json`) or a rubric/repository-instruction change. A curation record is never itself an executor payload. `public` restates only what a future regression case's reviewer-visible tree may eventually carry; `private` holds grading and administrative evidence exactly as the replay corpus's `private/expectations` and `private/provenance` do, and never enters a reviewer-visible artifact. A connector's claim is untrusted until `disposition` and `public.adjudication_evidence` record how it was actually adjudicated.",
+  "type": "object",
+  "additionalProperties": false,
+  "required": [
+    "curation_record_version",
+    "record_id",
+    "disposition",
+    "public",
+    "private"
+  ],
+  "properties": {
+    "curation_record_version": { "const": "1.0" },
+    "record_id": {
+      "type": "string",
+      "pattern": "^[a-z0-9]+(-[a-z0-9]+)*$"
+    },
+    "disposition": {
+      "description": "The final adjudication. Unresolved claims and plain duplicates cannot support a promotion decision; see `promotion-decision.schema.json` and `curation.py` for the cross-record rules JSON Schema cannot express.",
+      "enum": [
+        "accepted_material_defect",
+        "accepted_acceptance_miss",
+        "accepted_validation_gap",
+        "rejected_false_positive",
+        "rejected_non_causal",
+        "deferred_hardening",
+        "duplicate",
+        "unresolved"
+      ]
+    },
+    "duplicate_of": {
+      "description": "Required exactly when `disposition` is `duplicate`; the record_id whose root cause this claim restates.",
+      "type": "string",
+      "pattern": "^[a-z0-9]+(-[a-z0-9]+)*$"
+    },
+    "distinct_contribution": {
+      "description": "Present only alongside `duplicate_of`. Describes the distinct trigger, surface, or negative control this duplicate still contributes, so it can be promoted without double-counting the shared root cause. A duplicate without this field can never appear in a promotion decision's case lists.",
+      "type": "string",
+      "minLength": 1
+    },
+    "public": {
+      "description": "Everything a future reviewer-visible regression case, or a public tracker artifact, may restate.",
+      "type": "object",
+      "additionalProperties": false,
+      "required": [
+        "connector_finding",
+        "contract_dimension",
+        "affected_surface",
+        "adjudication_evidence",
+        "provenance"
+      ],
+      "properties": {
+        "connector_finding": {
+          "type": "object",
+          "additionalProperties": false,
+          "required": ["claim"],
+          "properties": {
+            "claim": {
+              "description": "The connector's finding, stated as an untrusted claim. It never becomes authoritative merely by appearing here.",
+              "type": "string",
+              "minLength": 1
+            },
+            "connector_identity": {
+              "type": "string",
+              "minLength": 1
+            }
+          }
+        },
+        "contract_dimension": {
+          "description": "The contract/risk dimension this claim concerns (e.g. correctness, solution-simplicity, code-simplicity, disclosure).",
+          "type": "string",
+          "minLength": 1
+        },
+        "affected_surface": {
+          "type": "string",
+          "minLength": 1
+        },
+        "adjudication_evidence": {
+          "description": "Concise, public evidence for the disposition above. Never the private expected root cause.",
+          "type": "string",
+          "minLength": 1
+        },
+        "reviewer_visible_reproduction": {
+          "description": "Sanitized reproduction artifacts, safe for a future reviewer-visible packet.",
+          "type": "array",
+          "items": { "type": "string", "minLength": 1 }
+        },
+        "provenance": {
+          "description": "Public-facing provenance only. Retention authority, owner, review date, and fuller source identity are private (see `private.provenance`).",
+          "type": "object",
+          "additionalProperties": false,
+          "required": ["source_class", "source_description"],
+          "properties": {
+            "source_class": {
+              "enum": [
+                "public",
+                "private_authorized",
+                "repository_history",
+                "synthetic"
+              ]
+            },
+            "source_description": {
+              "description": "When `source_class` is `private_authorized`, this must be an allow-listed generic phrase and must never contain a path-like token, a bare hostname, or a deny-listed identifier. See `disclosure-guardrail.json` and `curation.disclosure_guardrail_errors`, which JSON Schema cannot express.",
+              "type": "string",
+              "minLength": 1
+            }
+          }
+        },
+        "corpus_links": {
+          "description": "Present once this record has actually been promoted into the replay corpus.",
+          "type": "object",
+          "additionalProperties": false,
+          "properties": {
+            "corpus_version": { "type": "string", "minLength": 1 },
+            "evaluation_comparison": { "type": "string", "minLength": 1 }
+          }
+        }
+      }
+    },
+    "private": {
+      "description": "Never enters a reviewer-visible artifact or an executor payload.",
+      "type": "object",
+      "additionalProperties": false,
+      "required": ["provenance"],
+      "properties": {
+        "provenance": {
+          "type": "object",
+          "additionalProperties": false,
+          "required": ["retention_authority", "owner", "review_date"],
+          "properties": {
+            "retention_authority": { "type": "string", "minLength": 1 },
+            "owner": { "type": "string", "minLength": 1 },
+            "review_date": {
+              "type": "string",
+              "pattern": "^\\d{4}-\\d{2}-\\d{2}$"
+            },
+            "source_identity": {
+              "description": "The fuller source identity, kept private. Still subject to the same disclosure discipline as every other repository artifact, but never required to be generic the way `public.provenance.source_description` is.",
+              "type": "string",
+              "minLength": 1
+            }
+          }
+        },
+        "expected_root_cause": {
+          "description": "Required exactly for an accepted disposition; forbidden before adjudication settles it.",
+          "type": "string",
+          "minLength": 1
+        },
+        "accepted_non_finding": {
+          "description": "Required exactly for a rejected or deferred disposition; forbidden before adjudication settles it.",
+          "type": "string",
+          "minLength": 1
+        }
+      }
+    }
+  }
+}

review-suite/evals/contracts/disclosure-guardrail.json

@@ -0,0 +1,8 @@
+{
+  "guardrail_version": "1.0",
+  "description": "Configuration for the mechanical disclosure guardrail `curation.disclosure_guardrail_errors` enforces on every curation record whose `public.provenance.source_class` is `private_authorized`. `allowed_source_descriptions` is the exact set of generic phrases a public-facing `source_description` may use. `denylisted_identifiers` is empty by default: this repository's implementing tooling must never learn, guess, or record the identity of any private, owner-authorized source, so no real identifier is shipped here. Add a real known-sensitive identifier (a repository name, internal codename, or hostname) only when the repository owner directly identifies it as one to guard against; this file is intentionally editable without a code change.",
+  "allowed_source_descriptions": [
+    "a private, owner-authorized connector-review source"
+  ],
+  "denylisted_identifiers": []
+}

review-suite/evals/contracts/promotion-decision.schema.json

@@ -0,0 +1,121 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://github.com/shaug/agent-scripts/review-suite/evals/contracts/promotion-decision.schema.json",
+  "title": "Connector-outcome promotion decision",
+  "description": "Records the narrowest-owner decision the promotion workflow makes for one or more curation records: keep as a corpus case only, update the global correctness rubric, update a repository-owned instruction file, or promote nothing. `curation.py` enforces every cross-record rule this schema cannot express: which dispositions may support a positive or negative case, representativeness for a rubric change, and that a repository-instruction change only ever targets an existing repository-owned instruction mechanism.",
+  "type": "object",
+  "additionalProperties": false,
+  "required": [
+    "promotion_decision_version",
+    "decision_id",
+    "decision",
+    "rationale",
+    "positive_case_ids",
+    "negative_control_case_ids",
+    "evidence"
+  ],
+  "properties": {
+    "promotion_decision_version": { "const": "1.0" },
+    "decision_id": {
+      "type": "string",
+      "pattern": "^[a-z0-9]+(-[a-z0-9]+)*$"
+    },
+    "decision": {
+      "enum": [
+        "global_rubric_update",
+        "repository_instruction_update",
+        "corpus_case_only",
+        "no_promotion"
+      ]
+    },
+    "rationale": { "type": "string", "minLength": 1 },
+    "positive_case_ids": {
+      "description": "Curation record ids whose accepted material outcome supports this decision.",
+      "type": "array",
+      "items": { "type": "string", "minLength": 1 }
+    },
+    "negative_control_case_ids": {
+      "description": "Curation record ids whose rejected or deferred tuning outcome supports this decision as a negative control.",
+      "type": "array",
+      "items": { "type": "string", "minLength": 1 }
+    },
+    "target": {
+      "description": "Required exactly when `decision` is `global_rubric_update` or `repository_instruction_update`; forbidden otherwise.",
+      "type": "object",
+      "additionalProperties": false,
+      "required": ["kind", "path", "summary"],
+      "properties": {
+        "kind": { "enum": ["global_rubric", "repository_instruction"] },
+        "path": {
+          "description": "Repository-relative path. A `repository_instruction` target must name an existing repository-owned instruction file (AGENTS.md or CLAUDE.md); this ticket ships no new shared path-rule subsystem.",
+          "type": "string",
+          "minLength": 1
+        },
+        "summary": { "type": "string", "minLength": 1 }
+      }
+    },
+    "evidence": {
+      "description": "Before/after quality, stability, latency, and available cost evidence from the approved evaluation slice. Required for every decision, including `no_promotion` and `corpus_case_only`, because the measurement is what justifies not changing guidance just as much as it justifies changing it.",
+      "type": "object",
+      "additionalProperties": false,
+      "required": ["before", "after"],
+      "properties": {
+        "before": {
+          "type": "object",
+          "additionalProperties": false,
+          "required": [
+            "recall",
+            "false_positive_rate",
+            "stability",
+            "latency_seconds",
+            "cost"
+          ],
+          "properties": {
+            "recall": {
+              "description": "Left untyped: this repository's schema-subset validator has no numeric type. `curation.py` checks it is actually numeric."
+            },
+            "false_positive_rate": {},
+            "stability": {},
+            "latency_seconds": {},
+            "cost": {
+              "description": "Either `usd` is reported, or `unavailable` is true with a `reason` - never both, never neither.",
+              "type": "object",
+              "additionalProperties": false,
+              "properties": {
+                "usd": {},
+                "unavailable": { "type": "boolean" },
+                "reason": { "type": "string", "minLength": 1 }
+              }
+            }
+          }
+        },
+        "after": {
+          "type": "object",
+          "additionalProperties": false,
+          "required": [
+            "recall",
+            "false_positive_rate",
+            "stability",
+            "latency_seconds",
+            "cost"
+          ],
+          "properties": {
+            "recall": {},
+            "false_positive_rate": {},
+            "stability": {},
+            "latency_seconds": {},
+            "cost": {
+              "type": "object",
+              "additionalProperties": false,
+              "properties": {
+                "usd": {},
+                "unavailable": { "type": "boolean" },
+                "reason": { "type": "string", "minLength": 1 }
+              }
+            }
+          }
+        }
+      }
+    }
+  }
+}