feat: adversarial restraint eval (Inspect) under evals/ - #176
Merged
Conversation
An Inspect eval that red-teams empathySync's restraint: a generator writes hard prompts, the real pipeline answers them headless (storage redirected to a throwaway dir), and a stronger, different-family judge grades whether the answer held its restraint rather than its helpfulness. Fail-closed judge parsing; a memory-guard preflight refuses to start without model presence and RAM headroom; findings only - it never edits the pipeline, prompts, or corpus. inspect_ai/openai are an optional evals extra, so a normal install stays lean and the tests importorskip when it is absent (CI stays green without it). Rewrites docs/adversarial-eval-loop.md from the pre-Inspect draft to match the built design, and adds the checklist rows the new evals/ dir needs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an adversarial restraint eval under
evals/empathysync_restraint/, built on Inspect. It red-teams empathySync's core promise - restraint on sensitive topics - and produces findings for a human to review. It never edits the pipeline, prompts, or corpus.How it works, per sample:
./datais never read or written.NOANSWER, never a silent pass.Model roles are chosen deliberately: the engine under test is the small model empathySync actually ships (test reality), and the judge is a stronger, different-family model (a model is a lenient grader of its own output).
inspect_ai/openaiare an optionalevalsextra, not core deps - a normal install stays lean, and the testsimportorskipthe extra so CI stays green without it.Changes
evals/empathysync_restraint/- the eval: config, dataset loader + frozen starter set, offline generator, pipeline solver, restraint scorer, preflight guard, task, runner, README.tests/test_restraint_eval.py- pure-logic + mocked tests for the risky bits (fail-closed judge parsing, scorer verdict mapping, dataset loader, preflight).docs/adversarial-eval-loop.md- rewritten from the pre-Inspect draft to match the built design; now the rationale companion to the module README.pyproject.toml- new optionalevalsextra.CHANGELOG.md,TESTING_CHECKLIST.md,MERGE_CHECKLIST.md- entries/rows for the new eval and the newevals/top-level dir.Not touched:
docs/architecture.md. That doc maps the runtime safety pipeline insidesrc/; this harness exercises the pipeline from outside and is documented by its own README + design doc, so adding it to the component table would be noise.Testing
pytest tests/test_restraint_eval.py -q- 15 passedpytest tests/ -m "not conversation"- 1143 passed, 23 deselected, no new failuresblack --check --line-length=100 evals/ tests/test_restraint_eval.py- cleanflake8(project args) onevals/and the test - cleanevalsextra (CI installs core only, soimportorskipguards them)