Dev Refresh

README.md

@@ -1,35 +1,36 @@
-# SIR: Signal Integrity Resolver Version 2.0
+# SIR: Signal Integrity Resolver Version 2.1
 
 [![SIR Real Governance Audit](https://github.com/SDL-HQ/sir-firewall/actions/workflows/audit-and-sign.yml/badge.svg)](https://github.com/SDL-HQ/sir-firewall/actions/workflows/audit-and-sign.yml)
 
 Deterministic pre-inference governance gate · rules-only · cryptographically signed proof
 
-Plain language: SIR sits in front of an AI model (or agent) and inspects a prompt before it reaches inference. It either lets the prompt through (PASS) or blocks it (BLOCK) using deterministic, versioned rules.
+Plain language: SIR sits in front of an AI model or agent and inspects a prompt before it reaches inference. It either lets the prompt through (`PASS`) or blocks it (`BLOCK`) using deterministic, versioned rules.
 
 Models provide capability. SIR makes governance enforceable and provable. It does not claim model alignment. It claims deterministic enforcement and verifiable evidence for a given policy and test suite.
 
-SIR is built for high-stakes AI systems that touch real money, real data, or real-world decisions. The goal is simple: produce verifiable evidence that a given governance configuration actually enforces what it claims, without relying on “trust us”.
+SIR is built for high-stakes AI systems that touch real money, real data, or real-world decisions. The goal is simple: produce verifiable evidence that a given governance configuration actually enforces what it claims, without relying on "trust us".
 
-Terminology note: in public/operator wording we prefer **governance gate**. Stable technical identifiers remain unchanged (`sir-firewall`, `sir_firewall`, proof class names, commands, URLs, and paths). See `docs/terminology.md`.
+Terminology note: in public and operator wording we prefer **governance gate**. Stable technical identifiers remain unchanged (`sir-firewall`, `sir_firewall`, proof class names, commands, URLs, and paths). See `docs/terminology.md`.
 
 ---
 
 ## Live proof (GitHub Pages)
 
-These are the served pages (human trust surface). Use these links. Do not click the `.html` files in the repo browser (GitHub will show source instead of serving it).
+These are the served pages (human trust surface). Use these links. Do not click the `.html` files in the repo browser because GitHub will show source instead of serving it.
 
 - Latest passing audit (human page): https://sdl-hq.github.io/sir-firewall/latest-audit.html
 - Latest run status (PASS / FAIL / INCONCLUSIVE): https://sdl-hq.github.io/sir-firewall/latest-run.json
 - Run archive (passes + failures): https://sdl-hq.github.io/sir-firewall/runs/index.html
 
 Important semantics:
 
-- `latest-audit.*` means latest PASSING audit (last known good proof).
+- `latest-audit.*` means latest passing audit (last known good proof).
 - `latest-run.json` means most recent run status, including failures or inconclusive runs.
 - The run archive always contains per-run artefacts for both passes and failures.
-- Gate outcome (`PASS`/`BLOCK`) is distinct from run/publication status (`PASS`/`FAIL`/`INCONCLUSIVE`).
+- Gate outcome (`PASS` / `BLOCK`) is distinct from run/publication status (`PASS` / `FAIL` / `INCONCLUSIVE`).
 - `latest-audit.*` and `latest-run.*` are single-run truth surfaces, not paired benchmark claims.
-- Canonical operator/reviewer method: `docs/minimal-pilot-runbook.md`.
+- Procedural cold-start path: `docs/minimal-pilot-runbook.md`
+- Evaluation and interpretation path: `docs/evaluator-technical-explainer.md`
 
 ---
 
@@ -42,15 +43,15 @@ git clone https://github.com/SDL-HQ/sir-firewall.git && cd sir-firewall && \
 python3 -m venv .venv && source .venv/bin/activate && \
 python3 -m pip install -U pip && python3 -m pip install -e . && \
 curl -s https://raw.githubusercontent.com/SDL-HQ/sir-firewall/main/proofs/latest-audit.json | python3 tools/verify_certificate.py -
-````
+```
 
 Expected:
 
 `OK: payload_hash matches reconstructed signed payload and signature verifies against ...; this proves payload integrity + signature validity only (not policy correctness, model safety, or broader trust guarantees).`
 
 Verification scope note: certificate verification is cryptographic integrity checking of signed payload bytes against the relevant public key material (`signing_key_id` via registry when resolvable, otherwise explicit `--pubkey`). It does not prove policy correctness, model safety, deployment completeness, or broader organizational trust posture.
 
-Note on the trailing `-`: it explicitly means “read JSON from stdin” (the pipe). This is the explicit/portable form we standardise on here.
+Note on the trailing `-`: it explicitly means "read JSON from stdin" (the pipe). This is the explicit and portable form we standardise on here.
 
 If you downloaded the file instead of piping:
 
@@ -83,7 +84,7 @@ python3 -m pip install -e .
 sir run --mode audit --pack generic_safety
 ```
 
-Source-tree bootstrap fallback (no editable install; useful for restricted/offline environments):
+Source-tree bootstrap fallback (no editable install; useful for restricted or offline environments):
 
 ```bash
 PYTHONPATH=src python3 red_team_suite.py --suite tests/domain_packs/generic_safety.csv --no-model-calls
@@ -95,20 +96,21 @@ Live gating check (PASS prompts call provider):
 
 ```bash
 python3 -m pip install -e ".[live]"
+
 # xAI example
 export XAI_API_KEY=your_xai_api_key_here
 sir run --mode live --pack generic_safety --provider xai --model grok-4-1-fast
 
 # OpenAI example
 export OPENAI_API_KEY=your_openai_api_key_here
-sir run --mode live --pack generic_safety --provider openai --model gpt-4.1-mini
+sir run --mode live --pack generic_safety --provider openai --model gpt-5.4-mini
 ```
 
-Windows note: if `pip install -e ".[live]"` fails due to long paths, run from a short path (e.g. `C:\sir\...`) or enable Windows long path support.
+Windows note: if `pip install -e ".[live]"` fails due to long paths, run from a short path such as `C:\sir\...` or enable Windows long path support.
 
-Current supported provider/model selection is documented in `docs/model-selection.md`.
+Current supported provider and model selection is documented in `docs/model-selection.md`.
 
-`publish_run.py` produces signed archive receipts and requires `SDL_PRIVATE_KEY_PEM`; not required for basic evaluation.
+`publish_run.py` produces signed archive receipts and requires `SDL_PRIVATE_KEY_PEM`; this is not required for basic evaluation.
 
 Low-level `python3 tools/...` commands remain available for debugging and CI internals, but operators should start with `sir ...`.
 
@@ -119,27 +121,24 @@ Low-level `python3 tools/...` commands remain available for debugging and CI int
 SIR is:
 
 * A deterministic pre-inference governance gate that runs before an LLM sees the text
-* Text-first and request-level in current capability
+* Primarily text-first at the request path, with bounded first-wave support for structured and tool-result ingress
 * Structured-envelope aware around that request path
-* Pack/scenario evaluation against that request path
+* Pack and scenario evaluation against that request path
 * Deterministic and explainable (rules-only; no embeddings, no hidden scoring)
-* A proof-producing system (signed certificate + fingerprint + ITGL hash chain + per-run archives)
+* A proof-producing system (signed certificate, fingerprint, ITGL hash chain, and per-run archives)
 
 SIR is not:
 
 * A post-hoc moderation layer that reacts after the model already saw the input
 * A probabilistic trust score or black-box classifier
 * A general alignment or ethics solution
+* Native multimodal governance
+* Deep stateful conversational governance
+* Native full tool or function-call governance
+* Internal model reasoning visibility
+* Post-inference model behavior governance
 
-Current boundary summary:
-
-* No native multimodal gating
-* No deep stateful conversational governance
-* No native tool/function-call governance
-* No internal model reasoning visibility
-* No post-inference model behavior governance
-
-For full scope boundary, failure modes, and residual-risk semantics, use `docs/assurance-kit.md` (canonical).
+For current scope boundary, failure modes, and residual-risk semantics, use `docs/assurance-kit.md`.
 
 ---
 
@@ -153,16 +152,16 @@ Evidence is defined by the versioned contract:
 Key fields:
 
 * `proof_class` is explicit: `FIREWALL_ONLY_AUDIT`, `LIVE_GATING_CHECK`, `SCENARIO_AUDIT`
-* `provider_call_attempts` counts attempted downstream calls (including retries/timeouts)
+* `provider_call_attempts` counts attempted downstream calls, including retries and timeouts
 * `provider_call_successes` is informational
 * `model_calls_made` is a legacy alias equal to `provider_call_attempts`
-* `trust_fingerprint` is canonical; `safety_fingerprint` is retained as legacy alias
+* `trust_fingerprint` is canonical; `safety_fingerprint` is retained as a legacy alias
 
 ---
 
 ## Why this exists
 
-Most “governance”, “safety”, and “compliance” claims are unverifiable. SIR exists to turn them into auditable evidence that security review, compliance, and (where applicable) underwriting can actually consume.
+Most "governance", "safety", and "compliance" claims are unverifiable. SIR exists to turn them into auditable evidence that security review, compliance, and, where applicable, underwriting can actually consume.
 
 Accountability sits in two versioned, auditable boxes:
 
@@ -173,10 +172,10 @@ Questions SIR answers with evidence:
 
 * What suite was tested?
 * What policy and configuration was enforced?
-* What happened during the run (including failures)?
+* What happened during the run, including failures?
 * Can an independent party verify the claim offline?
 
-SIR’s job is simple: enforce policy before inference, then prove what happened without relying on “trust us”.
+SIR’s job is simple: enforce policy before inference, then prove what happened without relying on "trust us".
 
 ---
 
@@ -185,12 +184,12 @@ SIR’s job is simple: enforce policy before inference, then prove what happened
 * Gate core: `src/sir_firewall/`
 * Domain pack suites (CSV): `tests/domain_packs/`
 * Scenario packs: `tests/scenario_packs/`
-* Runner: `red_team_suite.py` (writes run logs + summary + ITGL)
+* Runner: `red_team_suite.py` (writes run logs, summary, and ITGL)
 * Proofs (repo artefacts):
 
   * Signed cert (latest pointer): `proofs/latest-audit.json`
   * Human page (backed by JSON): `proofs/latest-audit.html`
-  * ITGL ledger + final hash: `proofs/itgl_ledger.jsonl`, `proofs/itgl_final_hash.txt`
+  * ITGL ledger and final hash: `proofs/itgl_ledger.jsonl`, `proofs/itgl_final_hash.txt`
   * Run archive (passes + failures): `proofs/runs/<run_id>/...`
 * Offline verification:
 
@@ -202,14 +201,15 @@ SIR’s job is simple: enforce policy before inference, then prove what happened
 
 ## Guides
 
-* [Assurance kit](docs/assurance-kit.md) (canonical evaluation and verification path)
+* [Minimal pilot runbook](docs/minimal-pilot-runbook.md) (procedural cold-start path)
+* [Evaluator technical explainer](docs/evaluator-technical-explainer.md) (evaluation and interpretation path)
+* [Assurance kit](docs/assurance-kit.md) (supporting evaluation and verification reference)
 * [Evidence perimeter note](docs/evidence-perimeter.v2.md) (current bounded benchmark perimeter)
 * [External technical review preparation](docs/external-technical-review-prep.md)
-* [Evaluator technical explainer](docs/evaluator-technical-explainer.md)
 * [Engineer guide](docs/engineer-guide.md) (local runs, signing, serving)
 * [Trial guide](docs/trial-guide.md) (auditors, insurers, evidence capture)
-* [Key governance readiness](docs/key-governance-readiness.md) (authority map and CRYPTO_ENFORCED checklist)
-* [Release notes](docs/release-notes-2.0.md) (2.0 closeout)
+* [Key governance readiness](docs/key-governance-readiness.md) (authority map and `CRYPTO_ENFORCED` checklist)
+* [Release notes](docs/release-notes-2.1.md) (2.1 closeout)
 * [Retention / Tier B export](RETENTION.md)
 
 ---
@@ -235,3 +235,6 @@ MIT Licensed © 2025 Structural Design Labs
 ## Contact
 
 [https://www.structuraldesignlabs.com](https://www.structuraldesignlabs.com) · [info@structuraldesignlabs.com](mailto:info@structuraldesignlabs.com) · @SDL_HQ
+
+```
+```

docs/comparison-discipline.v2.md

@@ -1,4 +1,4 @@
-# S2.3 Comparison Discipline v2 (Internal)
+# S2.3 Comparison Discipline v2 (Transparent)
 
 Purpose: keep benchmark-cycle and pack-result comparisons boring, evidence-linked, and reproducible.
 

docs/evidence-perimeter.v3.md

@@ -0,0 +1,135 @@
+# SIR current evidence perimeter v3
+
+Date (UTC): 2026-05-03
+
+## Method note
+Paired benchmark means an ungated baseline run and an SIR-gated run executed against the same prompt set, then compared at result and prompt ID level.
+
+## Models included in current evidence state
+Primary cross-provider comparison set:
+- `gpt-4o-mini`
+- `xai/grok-4-1-fast`
+- `xai/grok-4.20-0309-reasoning`
+- `xai/grok-4.20-0309-non-reasoning`
+- `gpt-4.1-mini`
+- `gpt-5.4-mini`
+
+## Packs included in current perimeter state
+Core gate-effect comparison packs:
+- `generic_safety`
+- `data_exfiltration_pressure`
+- `support_operator_override`
+
+Extended governance pressure pack now in scope:
+- `eu_ai_act_compliance_pressure`
+
+## Evidence classes
+- paired benchmark evidence
+- selected comparisons used latest `valid_complete` pairs
+- core comparison packs remain coherent live/provider-path evidence
+- `eu_ai_act_compliance_pressure` now has both:
+  - paired governance-gate-only benchmark evidence
+  - recent live paired evidence confirming the same directional effect
+
+## Observed paired outcomes for the core comparison packs
+For the four-model primary comparison set, on all three core packs:
+- ungated baseline: `AUDIT FAILED`
+- SIR-gated run: `AUDIT PASSED`
+
+## Core pack-level results
+- `generic_safety`: ungated leaks 100, SIR-gated leaks 0, gated benign/pass denominator 50
+- `data_exfiltration_pressure`: ungated leaks 27, SIR-gated leaks 0, gated benign/pass denominator 23
+- `support_operator_override`: ungated leaks 26, SIR-gated leaks 0, gated benign/pass denominator 24
+
+## Core benign/pass denominator summary
+- total gated benign/pass denominator across compared live pairs: `97`
+- total harmless blocked across compared live pairs: `0`
+
+## Core ID-level discrimination review
+- across all four compared models, ungated leaked prompt ID sets were identical for each pack
+- across all four compared models, SIR-gated blocked/passed prompt ID sets were identical for each pack
+
+## Extended cross-model ID-level finding
+ID-level discrimination review was subsequently extended to all six models in the current evidence state across the three core packs:
+- ungated leaked prompt ID sets were identical across all six models for each pack
+- reasoning vs non-reasoning variants produced identical outcomes
+- three generations of GPT (`gpt-4o-mini`, `gpt-4.1-mini`, `gpt-5.4-mini`) produced identical outcomes
+- SIR-gated blocked/passed prompt ID sets were identical across all six models
+
+This finding was published: [Seven models, two providers, identical failure sets](https://x.com/SDL_HQ/status/2049438575574364279) - 2026-05-03
+
+What this supports:
+- common-mode prompt-layer vulnerability across the tested model set
+- model selection, including generation level and reasoning capability, did not change attack outcomes on the tested packs
+
+What this does not support:
+- universal claim across all models or all attack categories
+- model-vs-model discrimination within the tested set
+
+## `eu_ai_act_compliance_pressure` current position
+Current paired governance-gate-only benchmark position on `xai/grok-4-1-fast`:
+- ungated baseline leaks: `100`
+- SIR-gated leaks: `26`
+- delta leaks: `-74`
+- delta harmless blocked: `0`
+
+Recent live paired position on `xai/grok-4-1-fast`:
+- `AUDIT FAILED` to `AUDIT FAILED`
+- delta leaks: `-74`
+- delta provider calls: `-74`
+- delta harmless blocked: `0`
+
+## `eu_ai_act_compliance_pressure` progression
+Observed paired benchmark progression for this pack on the current bounded rule path:
+- `100 -> 100`
+- `100 -> 75`
+- `100 -> 57`
+- `100 -> 32`
+- `100 -> 26`
+
+This progression reflects bounded deterministic expansion inside the SIR rule layer, not benchmark semantic changes.
+
+## `eu_ai_act_compliance_pressure` category state
+Current gated-state interpretation:
+- `transparency_bypass`: cleared
+- `prohibited_manipulation`: cleared
+- `oversight_circumvention`: reduced to one intentional residual row
+- `data_boundary`: dominant remaining leakage class
+
+## What current evidence supports
+- the three core packs still strongly distinguish SIR off vs on
+- across the tested model set, the current three packs show common-mode prompt-layer susceptibility across six models and two providers
+- on those packs, in that tested set, model selection did not change attack outcomes; SIR did
+- `eu_ai_act_compliance_pressure` now shows that SIR can materially reduce governance-pressure leakage through bounded deterministic extensions
+- the current bounded EU-pack work reduced leaks from `100` to `26` in paired governance-gate-only benchmark mode with `0` harmless blocked
+- the same `-74` leak reduction was also observed in a recent live paired run, with `-74` provider calls and `0` harmless blocked
+
+## What current evidence does not support
+- it does not show that all models are equivalent generally
+- it does not show that all attack classes are common-mode generally
+- it does not show model-vs-model discrimination within the current tested set
+- it does not establish full coverage or residual-risk claims beyond the packs, models, and evidence classes listed here
+- it does not show that `eu_ai_act_compliance_pressure` is fully covered; the pack still fails overall
+- it does not support broad legal or compliance claims from the EU-pack work
+
+## What this means for liability exposure
+- prompt-layer attack success is not just a model-quality issue; it is an exposure pathway where unsafe or deceptive requests can reach inference and create downstream operational, regulatory, or liability exposure
+- SIR reduces that exposure by blocking classes of prompt-layer requests before inference and by preserving a signed audit trail showing what was attempted, what was blocked, and what was allowed
+- for review, dispute, or claims handling, the resulting artefacts provide replayable evidence of gate behavior and decision lineage rather than post-hoc narrative alone
+- `0` harmless blocked in the measured reductions means the current bounded rule gains were not achieved by broadly degrading normal use in the tested benign slice
+- this does not eliminate liability or certify compliance, but it does improve the evidence position and control position around prompt-layer failure modes
+
+## Current perimeter limit
+- the core benchmark set remains validated as a gate-effect evidence set for the three listed core packs
+- it is still not a model-separation benchmark set
+- `eu_ai_act_compliance_pressure` is now a bounded partial-coverage evidence pack, not a full-pass pack
+- remaining EU-pack leakage is concentrated almost entirely in `data_boundary`, plus one intentionally unimplemented oversight residual
+
+## Current intentional pause
+- `data_boundary` work is intentionally paused
+- current judgment is that `data_boundary` is the least attractive remaining deterministic target and carries higher complexity and ambiguity risk
+- the final oversight residual was intentionally not implemented for safety-to-complexity reasons
+
+## Next perimeter expansion
+- next perimeter expansion, if needed, will be determined by the next forcing-function audience and evidence requirement
+- current default recommendation is to stop EU-pack expansion here for now, unless a specific requirement justifies revisiting the paused remainder

docs/latest-audit.html

@@ -212,4 +212,4 @@ <h2>Run provenance</h2>
   </script>
 </body>
 </html>
-<!-- SIR_BUILD: date=2026-04-25T01:05:58Z payload_hash=sha256:59b5eaeccc79a6a5f8608b26539bdef79dc429f7e9f35ddee07435fc4ae5638f -->
+<!-- SIR_BUILD: date=2026-04-29T08:29:12Z payload_hash=sha256:b4750021e787fa2fe888576d19c517c0311ff8a9977b58eddb7852548c50a736 -->