fix(harness): enforce real verifier gates#233
Conversation
|
@greptile-apps please review |
Greptile SummaryThis PR hardens the conformance harness by requiring runners that declare a verifier mode to actually execute it, and by extending the managed-signer transfer guard to cover multisig signer tails and Token-2022 ATA derivation. It also replaces several inline CI scripts with testable modules and tightens a number of accounting gates.
Confidence Score: 5/5Safe to merge; the logic changes are well-tested and fail-closed, with only minor diagnostic-quality issues in a new shell script and a gate error message. Every new enforcement path has a corresponding unit or integration test. The
Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Runner manifest JSON] -->|isRunnerManifest| B{Has modesByIntent?}
B -->|No| C[Default: unsupported-mode allowed for all modes]
B -->|Yes| D[isModeCapabilities validation\n- known intents only\n- non-empty arrays\n- no duplicates]
D -->|Has strictModesByIntent?| E[strictModesByIntent must subset modesByIntent\n- only verifier modes allowed]
D -->|No| F[Mode declared: unsupported-mode is a conformance error]
E --> F
F --> G[Per-vector: assertDeclaredModeWasExecuted]
G -->|unsupported-mode returned| H[Test FAILS immediately]
G -->|accept / reject| I[recordModeExecution]
I --> J[afterAll: assertStrictModeCoverage]
J -->|strict mode missing accept OR reject| K[Suite FAILS]
J -->|both outcomes covered| L[Strict mode gate passes]
C --> M[afterAll: UNSUPPORTED_MODE_EXEMPTIONS check]
M -->|no exemption and 0 executions| N[Orphaned mode error]
M -->|exemption present| O[OK]
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
A[Runner manifest JSON] -->|isRunnerManifest| B{Has modesByIntent?}
B -->|No| C[Default: unsupported-mode allowed for all modes]
B -->|Yes| D[isModeCapabilities validation\n- known intents only\n- non-empty arrays\n- no duplicates]
D -->|Has strictModesByIntent?| E[strictModesByIntent must subset modesByIntent\n- only verifier modes allowed]
D -->|No| F[Mode declared: unsupported-mode is a conformance error]
E --> F
F --> G[Per-vector: assertDeclaredModeWasExecuted]
G -->|unsupported-mode returned| H[Test FAILS immediately]
G -->|accept / reject| I[recordModeExecution]
I --> J[afterAll: assertStrictModeCoverage]
J -->|strict mode missing accept OR reject| K[Suite FAILS]
J -->|both outcomes covered| L[Strict mode gate passes]
C --> M[afterAll: UNSUPPORTED_MODE_EXEMPTIONS check]
M -->|no exemption and 0 executions| N[Orphaned mode error]
M -->|exemption present| O[OK]
Reviews (10): Last reviewed commit: "Merge remote-tracking branch 'origin/fix..." | Re-trigger Greptile |
|
@greptile-apps please re-review. The router now classifies both sides of renames, validates every classifier output, requires selected workflows XOR explicit docs-only, and exercises missing/contradictory/skip cases in a blocking gate test; adversarial Sol review is clean. |
|
@greptile-apps please review |
Route x402 and session conformance through the real SDK verifiers and boot probes instead of accepting a fixture echo, so a verifier that never runs can no longer report a false pass. Adds the typescript adapter-boot fixture and expands the conformance, protocol, x402, session-voucher, value-binding, boot-policy and e2e tests to cover the routed path.
Add assertDeclaredModeWasExecuted and assertStrictModeCoverage: an SDK that declares support for a mode must execute both an accept and a reject vector for it, otherwise the run fails. Blocks an adapter from declaring a mode and silently returning unsupported-mode for every vector.
…ectors Each runner manifest now declares the modes it supports per intent (for example go.json declares charge and session canonical-bytes and x402-exact verify-x402-transaction) so the mode-support gate can hold it to that contract. Adds the canonical-json-rejects-lone-surrogate-value and canonical-json-es6-number-boundaries vectors plus a session-voucher reject vector.
assert-run-count.mjs forces the settlement pair-test floor to >= 1 and fails when zero pair-tests execute or when vitest reports success=false, closing the passWithNoTests false-green. Covers it plus the coverage-shape, matrix-coverage, ci-coverage and vector-accounting gate tests; ci-coverage-gate.test.ts pulls in the yaml dev dependency.
verify-routed-sdk-gate.mjs fails when any routed SDK job finishes non-success and rejects the vacuous nothing-selected-nothing-ran case via docs_only === (selected.length > 0). Pins workflow selection through select-pr-workflows with a fixture-backed test and wires the gate into pr-routing.yml.
check-rust-coverage.py validates the llvm-cov JSON with a default floor of 90 on lines and regions, aggregate and per-file, over non-empty mpp and x402 scopes, and exits non-zero on any shortfall or malformed report. Ships an adversarial self-test.
failopen-default-store.yaml flags a replay or session store that silently falls back to an in-memory default on the go and python SDK server sources, while the good fixtures stay clean. Tightens the gated-optional-field rule and the semgrep runner and makes findings blocking.
Add the pr216 delivery-accounting ledger with validate, generate and reconcile scripts, a payment-channels revision gate, and publish-workflow and repo-hygiene guards. Remove the two vendored solana-mpp tarballs and reject tracked package tarballs going forward.
Invoke the run-count, Rust-coverage, delivery-ledger and payment-channels revision gates from ci.yml, pin the payment-channels checkout ref and add the focused Python session fault-injection leg in harness.yml, and build harness verifiers with cargo --locked in setup-harness. Bounds the lua coverage leg and refreshes the go and python legs.
Rebasing this leaf onto the harness base tip pulls in PR #214 (solana-go v2), which rewrites the four go dependency manifests (harness/go-{client,server}/go.{mod,sum}). Their delivery-head blob no longer matches the 216 source, so the ledger reconciles them from integrated to open_pr owned by #214 and repoints the surviving integrated evidence at the current delivery head. Produced by scripts/reconcile-pr216-open-deliveries.mjs; validate-pr216-ledger passes.
The native source-contract probes asserted Rust markers PR #227 never ships (is_shared legacy capability, config.allow_unsafe_memory_store, the "atomic durable shared replay store is required" message) and PHP error text the php leaf never emits ("MPP requires an injected atomic durable/shared replay store", "does not affirm durable/shared capability"). Those probes were permanently red against the real SDK sources. Replace them with the same mechanism the #238 finalize agent uses: an asserted-skip roster for SDKs that do not implement the shared PAY_KIT_ALLOW_INMEMORY_REPLAY_STORE fail-closed contract, plus a git-grep honest-roster test that REDs the moment any rostered SDK gains an opt-in reference, forcing promotion to a live probe instead of a silent skip. No probe now asserts a contract no leaf delivers.
This leaf shipped the identical missing-ATA blocking change (remove continue-on-error from the session no-ATA leg, "This regression is blocking" comment) that #228 already delivers, and #228 owns python.yml. Revert python.yml to base so only #228 delivers it and the two leaves do not conflict at integration.
…o boot-policy roster cc93323 dropped the php/ruby/lua source-contract probes and mislabeled all three in unimplementedProbes as having "no off-localnet fail-closed boot guard". That is false: each enforces the same money-path policy natively. - php src/Protocols/Mpp/Adapter.php + Server/SolanaChargeHandler.php reject an omitted or non-durable/non-shared replay store off localnet - ruby lib/pay_kit/protocols/mpp/runtime.rb raises unless localnet or a durable replay_store is supplied - lua protocols/mpp/init.lua + server/init.lua error unless localnet or a shared replay store is configured Remove php/ruby/lua from unimplementedProbes and cover them with native source-contract probes, gated by the same git-grep detection #238 uses (sdkFilesReferencingOptIn kept byte-identical; a parallel marker helper carries each SDK's native rejection string). A probe auto-requires when its guard is in-tree and asserts-skip with an honest reason otherwise. Patterns are verified against current source, unlike the stale strings cc93323 removed. rust/kotlin/swift stay asserted-skip (verified: no such guard in-tree).
The go/ts/python fail-closed probes required their SDK guard, but those remediations land in sibling leaves (#227/#238/#228), so in this harness leaf they hard-failed. Resolve each covered probe through the same sdkImplementsGuard git-grep mechanism the native php/ruby/lua probes already use: a probe is required only when its SDK source references the opt-in marker in-tree, else it asserts-skip with a loud PENDING note and auto-promotes to required at integration. Moves the REPO_ROOT const above the probe resolution to avoid a module-eval temporal-dead-zone error.
cda6f96 to
66964c3
Compare
|
@greptileai review |
python.yml checked out Moonsong-Labs/solana-payment-channels while go.yml, harness.yml, and the build-payment-channels action all use solana-foundation/payment-channels at the same ref. The check-payment-channels-revision gate this leaf adds correctly flagged the drift: a mismatched program checkout decodes the epoch-aware openSlot bytes as a recipient count. Pin python.yml to the same repository so every payment-channels build uses one program.
|
@greptileai review |
…216 equivalence ledger Those five redelivery PRs landed on main rather than the #219 line, so their open_pr followUps are corrected from 'Land PR #X' to note the merge and the revalidation that follows when #219 rebases onto main. Validator + self-test still pass (113 commits, 237 paths accounted for).
Rebuilds rehearsal/pr216-integration purely from the current leaf heads in the canonical merge order. Conflicts abort loudly: the resolution belongs in the later leaf of the conflicting pair, never in the rehearsal branch, so a green rehearsal is green by construction.
…into fix/harness-adversarial-hardening
…into fix/harness-adversarial-hardening # Conflicts: # harness/test/boot-policy.test.ts
…nto fix/harness-adversarial-hardening # Conflicts: # .github/workflows/ci.yml # .github/workflows/harness.yml # harness/test/boot-policy.test.ts
… into fix/harness-adversarial-hardening # Conflicts: # harness/test/boot-policy.test.ts # harness/test/value-binding-verify.test.ts
…fix/harness-adversarial-hardening
…compare rule typeof secret !== 'string' inspects the value's runtime type tag, not its bytes, so it cannot leak match progress; the blocking scan flagged the pay-kit challengeBindingSecret validator for it. Pin the class with a known-good fixture so the self-test keeps the exclusion honest.
…fix/harness-adversarial-hardening
…into fix/harness-adversarial-hardening
…o fix/harness-adversarial-hardening # Conflicts: # scripts/check-rust-coverage.py # scripts/check-rust-coverage_test.py
…' into fix/harness-adversarial-hardening
…osed cascade Re-run the reconcile against the tree that now carries every redelivery leaf (the same tree #240 rebuilds from the leaf heads): two harness paths a sibling leaf superseded move from integrated to open_pr evidence, and the delivery baseline advances. Validator and self-test pass; the blocking radar scan on this tree reports zero findings.
…hygiene gates The ruby native-guard probe follows #235's durable_shared_replay_store? predicate rename (the guard itself is unchanged and stronger), and the missing-ATA hygiene gate matches the reconciled deliver-or-reject step name while still asserting the step stays blocking with the red-fault env.
… fixtures The session verifier now requires a valid context slot on confirmation responses (it pins follow-up account reads via min_context_slot), so the bound-tx mock reports one; and the close settle+distribute composer decodes the signed wire to bind the broadcast signature before submitting, so the fixture compiles the composed instructions into a real signed transaction and asserts the reported signature is wire-derived, never the RPC response value.
… union Order the mpp-session leaf (#239) ahead of the python leaf (#228) so their overlapping edits to the shared ci.yml + boot-policy.test.ts interleave without a phantom 3-way conflict (both are ultimately reconciled by the harness meta leaf, which owns those paths and merges last). Add a pure-union assertion after the merges: every non-merge commit the rehearsal introduces over base must be contained in some leaf head, else it is a hand-authored patch on #240 and the script refuses to push. This makes 'green by construction' mechanically checked, not asserted.
… replay store
The mainnet-fork settlement suite injected `{ ...Store.memory(), isShared: true }`
into createPayKit. Once the MPP replay-store policy requires an atomic
putIfAbsent (a spread of the legacy get/put store has none), createPayKit fails
closed with a ConfigurationError before any settlement runs, so every gated
settlement assertion errored in beforeAll. This surfaces only in the full
cascade: the leaf that tightened the policy does not run the on-chain leg, and
the on-chain leg's leaves predate the tightening. Build a real single-process
store (Map-backed get/put/delete + atomic putIfAbsent, isShared+isDurable) and
declare it production, matching the SDK's own createSharedTestReplayStore
fixture. Drop mpp.allowUnsafeMemoryStore: a production store needs no unsafe
escape hatch, and keeping it would be a false 'allow unsafe' marker on a
money-path test.
…me rationale mpp/client/subscription.rs and mpp/server/session.rs fall under the per-file 90% floor for the same reason the three existing exemptions do: their fail-closed validation and reject paths are unit-covered (mock RPC), but the residual gap is the on-chain RpcClient success paths (SubscriptionAuthority init broadcast, fetch_channel_account + settle_and_seal assembly, get_transaction top-up fetch) plus async state-machine desugaring regions that stay count-0 under llvm-cov, none of which a unit run can exercise. Each exemption names its concrete surfpool-integration follow-up. The session entry also records a real, still-open reject-coverage gap (on-chain channel status/mint/payee/rent-payer mismatch rejects are reachable via the SessionAccountRpc mock but not yet asserted) so it is documented rather than hidden. Non-exempt aggregate stays 96.97%/96.66% (mpp) and 94.46%/94.02% (x402), well above the 90% floor.
…e import) The pre-merge of the shared blockhash cache left two artifacts that only the integrated Rust gate catches (the leaf's own core/Rust job stops earlier): two consecutive blank lines that cargo fmt --check rejects, and a second, redundant `use super::*;` inside the test module (module-level import at the top already covers it) that clippy -D warnings rejects as unused. Collapse the blanks and drop the duplicate import; behavior is unchanged.
9b1b881 to
b9fe46b
Compare
…' into fix/harness-adversarial-hardening # Conflicts: # harness/test/ci-coverage-gate.test.ts
…cascade root Add the end-state seal: the cascade root lets the Rust conformance/coverage steps and the python session leg wait on their subjects, so this leaf (the last in the chain) asserts every such subject exists in the integrated tree — a pending gate cannot silently outlive its purpose. Track the restructured cascade in the rebuild script: the gate-activation root (#244) enters first and the python leaf precedes the mpp-session leaf, mirroring the telescoped PR chain.
…' into fix/harness-adversarial-hardening
…' into fix/harness-adversarial-hardening
…' into fix/harness-adversarial-hardening
…' into fix/harness-adversarial-hardening
Summary
Verification
Stack
Targets #219 as requested. This PR should merge after the language fixes it exercises, or together with them, so the blocking gates never encode unavailable behavior.