fix(verify): fail closed on unsupported safety invariants (ARN-213) - #391
fix(verify): fail closed on unsupported safety invariants (ARN-213)#391rita-aga wants to merge 31 commits into
Conversation
|
Independent GPT-5.6 review of the open GitHub diff at
Verdict: FAIL |
|
Remediated the independent-review findings from #391 (comment) in commit da2a371: failed pristine creation now exits before SSE/query projection and is removed from collection discovery; the first successful initializing action restores collection visibility; successful transitions to Deleted remove the ID instead of reindexing it; simulation now follows the same action-backed initialization contract. Focused HTTP and simulation regressions are rerunning on the pushed head. |
|
Independent GPT-5.6 review of the open GitHub diff at The previous simulation/production initialization mismatch is resolved:
Current GitHub checks are still running; the completed verification, compile/lint, integrity, and platform-random jobs pass. The finding above is independent of those pending results. Verdict: FAIL |
|
Remediated the current-head publication race from #391 (comment) in commit 50869c6. Generic actor spawn no longer publishes collection visibility; validated create/action success is now the single publication boundary, with Deleted results removed. The regression explicitly spawns the pristine actor, performs a direct OData GET at that pre-validation interleaving, and requires 404 before the existing create/Issue flow proves post-initialization visibility. |
|
Independent GPT-5.6 review of the open GitHub diff at
Current checks at review time: compile/lint, integrity, instrumentation, spec verification, and verification-contract checks pass; Verdict: FAIL |
|
Remediated both findings from #391 (comment) in f512ffd. Publication now occurs at the actor response boundary when either the mutation succeeded (including no-journal actors) or prior durable sequence exists (including valid bootstrap + rejected first action); rejected zero-event shells remain hidden. Unsupported-safety classification now shares one declaration/type/capability contract across temper-spec, temper-verify, runtime artifact compilation, and direct simulation. Missing named counters fail closed, unsupported assertions trigger even with invalid |
|
Independent GPT-5.6 final review of the open GitHub diff at
CI at review time is not green: Verdict: FAIL |
|
Independent GPT-5.6 review of the open GitHub diff at The earlier publication, initialization, passivation, hydration-eviction, registry, replay-parameter, and callback-stack remediations are present. Two safety gaps remain:
Focused corpus validation passes: 110 specs, 120 declarations, 120 typed. At review time GitHub compile/lint, verification contract, integrity/DST, spec verification, instrumentation, and platform-random checks pass; core, platform-boot, platform-consistency, and Tests remain pending. Verdict: FAIL |
|
Remediation for the current-head independent review FAIL at #391 (comment) is now pushed in 2df8ef3 (on top of af9b5d6). Addressed all findings and the push-gate compatibility regression:
Validation on current head:
Awaiting a fresh independent GitHub-diff review on current head before any Greptile request. |
|
Independent GPT-5.6 review of the open GitHub diff at The prior registry-admission, invalid-shell publication/passivation, production/simulation initialization, legacy-constructor, direct protected-variable mutation, and snapshot/replay blockers are materially addressed in the current diff. One safety gap remains:
At review time Compile & Lint, Integrity/DST, Verification Contract, Spec Verification, Instrumentation, and platform-random pass; core, platform-boot, platform-consistency, and Tests remain pending. This finding is independent of those pending checks. Verdict: FAIL |
ARN-213 live local E2E — current head
|
|
Independent GPT-5.6 review of the complete open GitHub diff at da5431d. The current head closes the original guard-only injection for newly constructed actors: model-proved invariants now protect every declared boolean/counter, and the regression covers create, PATCH, action, and replay. One live-registry admission gap remains.
All live GitHub checks for this exact head are green; this finding is independent of CI. Verdict: FAIL |
|
ARN-213 current-head remediation evidence GitHub head: Remediation commits since the prior independent FAIL:
The current head closes the live-registry admission gap and preserves fail-closed behavior across registration, startup restore, and hot swap. Model compatibility now compares invariant signatures, states/actions, and every resolved effect field/variant; safe additive actions are restricted to unique effect-free transitions from existing nonterminal states. Recursive terminal-leaf detection covers compound assertions. Durability coverage includes pre/post-swap journal restart replay and deterministic forced snapshot-failure retention. Exact final push command: Final local gate result: The isolated rerun of the one load-sensitive WASM restore timeout from the preceding attempt passed 1/1; it then passed 6/6 in the final full gate without a code change. Fresh independent GitHub-head review is now in progress. Greptile has not been requested yet, preserving the required gate order. |
|
Independent review of the full current GitHub PR diff. Reviewed head SHA: c05c4b1 Blocking finding: [P1] Model-proved counter safety can still be mutated by effects omitted from the model. In crates/temper-spec/src/automaton/translate.rs:100-113, ResolvedEffect::is_verifiable excludes SetCounterFromParam, IncrementCounterByParam, and DecrementCounterByParam. crates/temper-verify/src/model/builder.rs:83-97 consequently removes those effects from every verification transition, and lines 148-157 explicitly classify them as runtime-only. However, the new model-protection contract in crates/temper-spec/src/automaton/runtime_assert.rs:110-139 treats counter-to-literal assertions as model-proved and says protected values change only through modeled transition effects. Production still executes SetCounterFromParam at crates/temper-server/src/entity_actor/effects.rs:882-905, while runtime_invariant_failure only evaluates the new StringNonEmpty and CounterVarCompare runtime subset. Therefore a spec with budget initially 0, an input action that sets budget from a parameter, and invariant budget <= 10 can pass the cascade because the model sees a no-op, then accept budget=100 live and durably replay it with no invariant rejection. The added test modeled_effect_can_consume_protected_action_param_during_replay demonstrates that this unmodeled effect is intentionally allowed through the protection boundary, but it only uses a safe value and does not test the false-proof case. Exact hot-swap effect comparison does not repair this initial admission unsoundness. Please either model parameter-driven counter effects conservatively, reject them as a verification capability error whenever they can affect a model-proved claim or reachability, or atomically enforce the affected model invariant at runtime. Add a regression that proves cascade admission, live execution, and replay cannot accept an out-of-range parameter value. I also traced unsupported-invariant preflight through cached/bootstrap/registry activation, runtime-contract rollback/publication, snapshot and journal hydration, contract-preserving hot swaps, additive action checks, and terminal-invariant handling. No additional blocking finding was identified in those scoped paths. Verdict: FAIL |
ARN-213 current-head validation and live E2E evidenceGitHub head: Full unbypassed push gateExact command: cd /Users/seshendranalla/Development/temper/agent-worktrees/arn165/codex-arn-213-unsupported-safety-invariants
export CARGO_INCREMENTAL=0; git push -u origin HEADExact terminal result: The unbypassed hook ran Behavioral RED/GREEN evidenceRED commit cargo test -p temper-server --all-features --lib parameter_counter_effect_cannot_escape_verified_limit_live_or_on_replay -- --nocaptureCurrent-head GREEN: Compatibility/dependency regressions: Live local startup E2EThe exact live startup commands and full output are preserved in the earlier E2E evidence comment. Observed behavior: The current full gate revalidated the same startup/registry capability path and the new parameter-derived live/replay path at exact head Review markersFinal local code-quality review: PASS, zero findings. Final DST review: PASS, zero change-specific findings. The PR-wide diff touches |
|
Independent GPT-5.6 review of the full open GitHub PR diff. Reviewed head SHA:
Focused validation on this exact detached head: Verdict: FAIL |
|
Fresh independent GPT-5.6 review of the open GitHub diff. Reviewed SHA: 0c36145 Blocking finding:
The remaining reviewed paths are coherent: verifier backends fail closed at zero budget; cached and fresh deployment paths preserve capability checks and runtime-only disclosures; declaration/source-span matching is exact and ordered; runtime-only assertions are checked on tentative state with rollback; snapshot contract binding, full replay validation, tombstones, passivation, and hot-swap rejection address the durability/replay boundary. Several GitHub CI jobs were still pending at review time, while Verification Contract, Integrity/DST Patterns, and Spec Verification had passed. Verdict: FAIL |
|
Fresh independent GPT-5.6 review of the OPEN GitHub PR diff. Reviewed exact GitHub head: PR state verified: open, unmerged. Findings: no actionable correctness, durability, regression, or DST findings on the current diff. Review notes:
Per reviewer-session constraints, I reviewed GitHub-hosted metadata, changed-file patches, and exact-head file contents and did not edit, build, test, or create a worktree. Verdict: PASS |
|
@greptile review |
ARN-213 final exact-head E2E and validation evidenceGitHub head: Full unbypassed push gateExact command: CARGO_INCREMENTAL=0 git push -u origin HEADTerminal result: Greptile remediation regressionCARGO_INCREMENTAL=0 cargo test -p temper-server registry::model_contract_tests -- --nocaptureThis proves a string-initial change under an unchanged safety invariant is rejected before registry mutation and preserves the prior source, swap-controller identity, and live table. Live unsupported startup: fails closedTURSO_URL=file:/tmp/arn213-e2e-ace07b4e/unsupported-acfd7c43.db target/debug/temper serve --no-observe --specs-dir /tmp/arn213-e2e-ace07b4e/unsupported --tenant arn213-final --port 3215Live supported control: starts and serves healthTURSO_URL=file:/tmp/arn213-e2e-ace07b4e/supported-acfd7c43.db target/debug/temper serve --no-observe --specs-dir /tmp/arn213-e2e-ace07b4e/supported --tenant arn213-final --port 3216
curl -sS -i http://127.0.0.1:3216/healthzThe live server was stopped with Ctrl-C after the successful request. Contributor gateThis PR touches |
|
Independent GPT-5.6 final review of the complete open GitHub PR diff at No actionable findings.
Verdict: PASS |
|
ARENA SHIPPABLE · GPT-5.6 · 2026-07-20T19:50:13Z Exact current GitHub head: Mandatory gate audit, in order:
Validation summary:
Thread-aware final audit: Greptile review completed; its sole actionable thread has a concrete diff/test reply and is resolved. Contributor gate: this PR touches |
Objective
Make unsupported safety-invariant expressions a hard verification failure instead of allowing a specification to pass with warnings.
Tracks ARN-213.
Current branch state
This draft opens immediately with ADR 0171, which defines the fail-closed capability boundary and structured source diagnostics. The mandated isolated RED regression and GREEN implementation will follow as separate commits.
Tradeoff
Specifications using invariant syntax that the verification backends cannot prove will be rejected at the deployment gate. This intentionally replaces warning-only acceptance; supported safety guarantees remain available, and unsupported forms must be modeled explicitly before deployment.
Validation plan
cargo fmt --check,git diff --check, strict clippy for touched crates,cargo test --workspaceDo not merge: ARN-165 arena submission.
Greptile Summary
This PR makes unsupported safety invariants fail closed across verification and runtime paths. The main changes are:
Confidence Score: 4/5
The hotswap safety-contract path needs a fix before merging.
crates/temper-server/src/registry/model_contract.rs
Important Files Changed
Prompt To Fix All With AI
Reviews (1): Last reviewed commit: "fix(safety): close runtime enforcement r..." | Re-trigger Greptile
Context used (6)