wire up the adversarial integration harness - #57
Conversation
efeffcf to
43691cd
Compare
|
In harnessConfig(), the signal weight key is shell_metachar: 0.75 but the actual signal the scanner emits is shell_metacharacter. Because the YAML load replaces the whole map, the correct key from the Go defaults gets wiped out. Double-check if I got this correctly and fix it. The other thing is that the test opens a socket connection to the sidecar but never sets a read deadline. If the sidecar hangs on any payload, the whole test run stalls silently until the 120s process timeout kills everything; no per-test error, just a wall of tests that never ran. Potential fix: add conn.SetDeadline(time.Now().Add(5 * time.Second)) right after net.Dial. |
|
good catch on both - fixed in the next commit. renamed shell_metachar to shell_metacharacter so the YAML map carries the right key, and added a 5s read deadline on the socket so a stalled payload surfaces a proper test error instead of silently hanging |
draft, opening early so it can grow in the open
this fills in the tests/integration stub. harness_test.go was just a doc comment and the payload file was empty, so it now builds the sidecar, starts it against a temp socket and config, and fires each payload from adversarial_payloads.json through the real process over uds, checking the decision
starting with the on_prompt hook (jailbreaks block, benign allows), green. i'll add the other three hooks and more categories from here, plus a coverage summary once there's enough to summarise
one structural call worth your eyes: tests/integration sits outside the sidecar go module, so the
make integrationtarget never actually ran. i gave it its own go.mod (path under the sidecar so it can reuse the real frame codec and signer via a replace) and pointed the makefile target at it. happy to move it inside the sidecar module instead if you'd rather that layoutotel correlation (a run emitting the same spans as prod) is a later phase and leans on #41, so i'll add it once that merges