Skip to content

wire up the adversarial integration harness - #57

Merged
tharindupr merged 12 commits into
c2siorg:mainfrom
VibhorGautam:feat/integration-harness
Jul 6, 2026
Merged

wire up the adversarial integration harness#57
tharindupr merged 12 commits into
c2siorg:mainfrom
VibhorGautam:feat/integration-harness

Conversation

@VibhorGautam

Copy link
Copy Markdown
Contributor

draft, opening early so it can grow in the open

this fills in the tests/integration stub. harness_test.go was just a doc comment and the payload file was empty, so it now builds the sidecar, starts it against a temp socket and config, and fires each payload from adversarial_payloads.json through the real process over uds, checking the decision

starting with the on_prompt hook (jailbreaks block, benign allows), green. i'll add the other three hooks and more categories from here, plus a coverage summary once there's enough to summarise

one structural call worth your eyes: tests/integration sits outside the sidecar go module, so the make integration target never actually ran. i gave it its own go.mod (path under the sidecar so it can reuse the real frame codec and signer via a replace) and pointed the makefile target at it. happy to move it inside the sidecar module instead if you'd rather that layout

otel correlation (a run emitting the same spans as prod) is a later phase and leans on #41, so i'll add it once that merges

@VibhorGautam
VibhorGautam force-pushed the feat/integration-harness branch from efeffcf to 43691cd Compare June 8, 2026 04:44
@VibhorGautam
VibhorGautam marked this pull request as ready for review July 1, 2026 18:27
@tharindupr

Copy link
Copy Markdown
Collaborator

In harnessConfig(), the signal weight key is shell_metachar: 0.75 but the actual signal the scanner emits is shell_metacharacter. Because the YAML load replaces the whole map, the correct key from the Go defaults gets wiped out. Double-check if I got this correctly and fix it.

The other thing is that the test opens a socket connection to the sidecar but never sets a read deadline. If the sidecar hangs on any payload, the whole test run stalls silently until the 120s process timeout kills everything; no per-test error, just a wall of tests that never ran. Potential fix: add conn.SetDeadline(time.Now().Add(5 * time.Second)) right after net.Dial.

@VibhorGautam

Copy link
Copy Markdown
Contributor Author

good catch on both - fixed in the next commit. renamed shell_metachar to shell_metacharacter so the YAML map carries the right key, and added a 5s read deadline on the socket so a stalled payload surfaces a proper test error instead of silently hanging

@tharindupr
tharindupr merged commit 69b651a into c2siorg:main Jul 6, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants