feat(benchmarks): add pinned external evaluation runner

[VibhorGautam]

what

adds a benchmark runner that sends pinned external payloads through the Python Firewall and a live Go sidecar

the current runnable set is InjecAgent base and Lakera's public 8-case PINT format sample. InjecAgent tool outputs use on_context because v1 has no on_tool_result, PINT uses on_prompt. each dataset runs scanner OFF, TF-IDF ON and MiniLM ON

the runner fetches only the selected dataset and license, checks pinned SHA256s, records the runner, ACF config, model and package versions, and writes full plus exact-overlap-excluded results. datasets and raw result JSON stay out of git

why

we didn't have a reproducible way to run external payloads through the actual SDK and sidecar path

the main split here is SDK semantic signals vs final enforced verdicts. the Go integration harness sends directly to the sidecar, so it can't measure the Python semantic scanner. this runner keeps the live sidecar path the same across all 3 modes and reports both the signals gained and whether the final verdict changed

current results

• InjecAgent: OFF caught 0/1054. TF-IDF signaled 19/1054 and MiniLM signaled 15/1054, but both changed 0 final verdicts
• PINT format sample: 1/2 attacks caught, 0/6 benign false positives, and 0/1 attacks caught after exact-overlap exclusion
• the semantic runs also expose the current signal contract gap. encoding_evasion, role_hijack and tool_abuse reached the context hook without a sidecar weight or direct policy rule
• P50/P90/P95/P99 are included as local descriptive measurements, not portable performance claims

limits

this is detector evaluation, not agent attack success rate

the public PINT file is only an 8-case format sample, not the full benchmark. the semantic library also names PINT as a source, so the overlap-excluded result is not a held-out set

on_context is a proxy for injected tool output until ACF has an on_tool_result hook. AgentDojo is pinned in the manifest but its framework attack extraction stays a follow-up

testing

• 26 benchmark tests pass
• full Go suite and go vet pass
• OPA 57/57
• all 6 full modes rerun from a clean runner commit
• committed report matches the renderer output and all decision outcome hashes stayed stable

AI tool disclosure

Codex helped draft the runner and test scaffolding, and reviewed the provenance checks and report wording. I set the benchmark scope, hook mapping, contamination method and claim boundaries after the team discussion. The final branch was rerun locally in all 6 modes and checked against the raw outputs, outcome hashes, Go, OPA and Python tests. I understand the code and can defend the decisions in this PR