feat(proof): fail-closed evidence battery for Codex 2x cold-resume fixture - #226
Conversation
…xture Add evidence_battery rows to proof manifests so fixture, human-reviewer replication, and controlled-ablation claims are source-labeled and fail-closed. osc prove compare now blocks boundedProof when a required_for_pass item is not demonstrated or reproduced, and renders a proof-battery table. Evidence-battery source_refs follow the same local/public-safe rules as metric refs. Strengthen the Codex cold-resume fixture: - required rows: codex-2x-cold-resume-replicates (demonstrated) and cold-resume-packet-contract (demonstrated) - disclosure-only non-claims: human-reviewer-replication (not_demonstrated) and controlled-ablations (mixed_not_proven) - cold-resume packet contract recorded in aggregate receipt - fixture-local boundary files under evidence/ The bounded Codex 2x claim stays source-labeled to committed receipts and does not claim broad workload dominance, human replication, or release/merge/publish/deployment/compliance approval.
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 29055385e7
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Codex review (P2): a manifest with no evidence_battery rows computed evidenceBatteryPass=true because the blocking list was empty, so osc prove compare claimed "passes required evidence-battery items" for fixtures that supplied no battery at all — overstating the fail-closed gate. Distinguish missing/empty batteries from demonstrated/reproduced ones: - evidenceBatteryPresent: boolean (length > 0) - evidenceBatteryPass: boolean | null (null = not evaluated when empty) - evidenceBatteryStatus: 'pass' | 'fail' | 'not_evaluated' - boundedProof = metricProof && evidenceBatteryPass !== false (empty does not block, but no longer claims to pass) - verdict/report show "not evaluated" and never claim "passes required evidence-battery items" when no battery rows were supplied Adds a regression test against the legacy scaffold-vs-naked-codex fixture.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d259fc492f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Codex review (P2): if a manifest accidentally drops a required
evidence_battery row (e.g. cold-resume-packet-contract), the gate
only inspected remaining rows, so deletion bypassed the fail-closed
contract — deletion was treated differently from marking a row
not_demonstrated.
Add a manifest-level required_evidence array of evidence IDs. Every
listed ID must be present in evidence_battery (validated as
missing-required-evidence-row, a structural failure that makes
osc prove check fail and osc prove compare throw) and demonstrated/
reproduced (caught by the existing per-row required_for_pass gate).
Omission and status-downgrade are now both fail-closed.
The Codex fixture declares:
required_evidence: ["codex-2x-cold-resume-replicates",
"cold-resume-packet-contract"]
so dropping either required row now fails the manifest structurally
rather than silently passing on the surviving row.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1eac205e3c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…atus Codex review (P2): a manifest could list an ID in required_evidence while leaving that row required_for_pass:false with a non-passing status (e.g. not_demonstrated), and the gate still passed because blocking only checked the per-row required_for_pass flag. The docs described required_evidence as fail-closed unless demonstrated, but the code did not enforce that. The gate now blocks when a row's id is in required_evidence OR its required_for_pass is true, and its status is not demonstrated/ reproduced. So required_evidence membership implies required-for-pass at the gate level, matching the documented contract.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5250152e66
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Codex review (P2): proofRequiredEvidence() (validation) trims IDs, but the compare path rebuilt requiredEvidence with proofStrings(), which does not trim. A whitespace-padded required_evidence ID therefore passed validation (trimmed match) while the compare path's requiredEvidenceSet held the untrimmed string, so requiredEvidenceSet.has(item.id) missed the row and a not_demonstrated required row would not block. Trim required_evidence IDs in the compare path so both paths use normalized IDs.
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b2592735a2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…y table Codex review (P2): when a manifest uses required_evidence to make a row fail-closed while leaving required_for_pass false/omitted, the gate blocked correctly via requiredEvidenceSet, but the rendered "Required" column still labeled that row "disclosure-only" because it only checked item.requiredForPass. The report contradicted the actual PASS/FAIL gate and hid which evidence claims were required. The "Required" cell now reads "required" when item.requiredForPass OR the id is in summary.evidenceBatteryRequired.
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
@codex review |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
Codex Review: Didn't find any major issues. Keep it up! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
…sync Bump package version 0.32.1 -> 0.33.0 (package.json + package-lock.json root and packages[""] fields). Add v0.33.0 CHANGELOG entry (candidate state until trusted publishing completes) and keep v0.32.1 as current Latest. Add release-sync plan 174 (prep done; publication is owner-approved follow-through, not an AC) and candidate evidence note. Update section-parser live-corpus hashes for the new plan/evidence/changelog. Publishes the fail-closed evidence battery from PR #226 to the public npm and GitHub release surfaces via owner-approved trusted publishing.
…sync Bump package version 0.32.1 -> 0.33.0 (package.json + package-lock.json root and packages[""] fields). Add v0.33.0 CHANGELOG entry (candidate state until trusted publishing completes) and keep v0.32.1 as current Latest. Add release-sync plan 174 (prep done; publication is owner-approved follow-through, not an AC) and candidate evidence note. Update section-parser live-corpus hashes for the new plan/evidence/changelog. Publishes the fail-closed evidence battery from PR #226 to the public npm and GitHub release surfaces via owner-approved trusted publishing.
…sync Bump package version 0.32.1 -> 0.33.0 (package.json + package-lock.json root and packages[""] fields). Add v0.33.0 CHANGELOG entry (candidate state until trusted publishing completes) and keep v0.32.1 as current Latest. Add release-sync plan 174 (prep done; publication is owner-approved follow-through, not an AC) and candidate evidence note. Update section-parser live-corpus hashes for the new plan/evidence/changelog. Publishes the fail-closed evidence battery from PR #226 to the public npm and GitHub release surfaces via owner-approved trusted publishing.
…sync Bump package version 0.32.1 -> 0.33.0 (package.json + package-lock.json root and packages[""] fields). Add v0.33.0 CHANGELOG entry (candidate state until trusted publishing completes) and keep v0.32.1 as current Latest. Add release-sync plan 174 (prep done; publication is owner-approved follow-through, not an AC) and candidate evidence note. Update section-parser live-corpus hashes for the new plan/evidence/changelog. Publishes the fail-closed evidence battery from PR #226 to the public npm and GitHub release surfaces via owner-approved trusted publishing.
…sync (#228) Bump package version 0.32.1 -> 0.33.0 (package.json + package-lock.json root and packages[""] fields). Add v0.33.0 CHANGELOG entry (candidate state until trusted publishing completes) and keep v0.32.1 as current Latest. Add release-sync plan 174 (prep done; publication is owner-approved follow-through, not an AC) and candidate evidence note. Update section-parser live-corpus hashes for the new plan/evidence/changelog. Publishes the fail-closed evidence battery from PR #226 to the public npm and GitHub release surfaces via owner-approved trusted publishing. Co-authored-by: Graphanov <graphanov@users.noreply.github.com>
1 participant
Summary
Strengthen the bounded proof battery for the Codex 2x cold-resume fixture so adjacent non-claims cannot be accidentally promoted, while keeping every claim source-labeled, bounded, and fail-closed.
src/compare.ts):osc prove comparenow blocksboundedProofwhen anyrequired_for_passevidence-battery item is notdemonstratedorreproduced. Evidence-batterysource_refsfollow the same local/public-safe rules as metric refs (no URLs, absolute paths, parent escapes, or private/runtime directories). The report renders a proof-battery table.examples/proof/codex-token-efficient-resume/):codex-2x-cold-resume-replicates(demonstrated) andcold-resume-packet-contract(demonstrated).human-reviewer-replication(not_demonstrated) andcontrolled-ablations(mixed_not_proven).receipts/aggregate.json.evidence/.The bounded Codex 2x claim remains source-labeled to committed receipts and does not claim broad workload dominance, human replication, or release/merge/publish/deployment/compliance approval.
Verification
npm test— 48 files, 558 tests passed.npm run build— passed.osc prove check examples/proof/codex-token-efficient-resume/manifest.json— PASS, 0 warnings.osc prove compare examples/proof/codex-token-efficient-resume/manifest.json --format json—boundedProof: true,evidenceBatteryPass: true,blocking: 0.osc plan validate 173-codex-token-efficiency-proof --strict— 0 issues.osc verify --evidence-chain --plan 173-codex-token-efficiency-proof --strict— 26/26 links intact, strict pass../verify.sh --strict— 9 pass, 0 fail, 1 warn (worktree immutability skip).osc doctor --check secret-scan— PASS.Risk / Gates
evidence_batterybehave exactly as before (pass/fail on metrics + thresholds only)..osc/plans/done/173-codex-token-efficiency-proof.md(closed with checked acceptance criteria and evidence note).