Skip to content

feat: adversarial restraint eval (Inspect) under evals/ - #176

Merged
Olawoyin007 merged 1 commit into
mainfrom
feat/adversarial-eval-loop
Jul 23, 2026
Merged

feat: adversarial restraint eval (Inspect) under evals/#176
Olawoyin007 merged 1 commit into
mainfrom
feat/adversarial-eval-loop

Conversation

@Olawoyin007

Copy link
Copy Markdown
Owner

Summary

Adds an adversarial restraint eval under evals/empathysync_restraint/, built on Inspect. It red-teams empathySync's core promise - restraint on sensitive topics - and produces findings for a human to review. It never edits the pipeline, prompts, or corpus.

How it works, per sample:

  1. Solver runs the adversarial prompt through the real empathySync pipeline, headless, with storage redirected to a throwaway dir so ./data is never read or written.
  2. Scorer is a model-graded judge whose rubric rewards restraint (deferring, handing off, steering toward exit), not helpfulness. It is fail-closed: an unparseable judge verdict is NOANSWER, never a silent pass.
  3. Preflight refuses to start unless the models exist and there is real RAM headroom - unified-memory over-commit reboots the target box, so it checks rather than assumes.

Model roles are chosen deliberately: the engine under test is the small model empathySync actually ships (test reality), and the judge is a stronger, different-family model (a model is a lenient grader of its own output).

inspect_ai/openai are an optional evals extra, not core deps - a normal install stays lean, and the tests importorskip the extra so CI stays green without it.

Changes

  • evals/empathysync_restraint/ - the eval: config, dataset loader + frozen starter set, offline generator, pipeline solver, restraint scorer, preflight guard, task, runner, README.
  • tests/test_restraint_eval.py - pure-logic + mocked tests for the risky bits (fail-closed judge parsing, scorer verdict mapping, dataset loader, preflight).
  • docs/adversarial-eval-loop.md - rewritten from the pre-Inspect draft to match the built design; now the rationale companion to the module README.
  • pyproject.toml - new optional evals extra.
  • CHANGELOG.md, TESTING_CHECKLIST.md, MERGE_CHECKLIST.md - entries/rows for the new eval and the new evals/ top-level dir.

Not touched: docs/architecture.md. That doc maps the runtime safety pipeline inside src/; this harness exercises the pipeline from outside and is documented by its own README + design doc, so adding it to the component table would be noise.

Testing

  • pytest tests/test_restraint_eval.py -q - 15 passed
  • pytest tests/ -m "not conversation" - 1143 passed, 23 deselected, no new failures
  • black --check --line-length=100 evals/ tests/test_restraint_eval.py - clean
  • flake8 (project args) on evals/ and the test - clean
  • Verified the eval tests skip without the evals extra (CI installs core only, so importorskip guards them)
  • Live eval run - deliberately not run here; it loads the 65 GB judge and produces findings for human review, which is a you-in-the-loop action, not part of landing the code.

An Inspect eval that red-teams empathySync's restraint: a generator writes
hard prompts, the real pipeline answers them headless (storage redirected to
a throwaway dir), and a stronger, different-family judge grades whether the
answer held its restraint rather than its helpfulness. Fail-closed judge
parsing; a memory-guard preflight refuses to start without model presence and
RAM headroom; findings only - it never edits the pipeline, prompts, or corpus.

inspect_ai/openai are an optional evals extra, so a normal install stays lean
and the tests importorskip when it is absent (CI stays green without it).

Rewrites docs/adversarial-eval-loop.md from the pre-Inspect draft to match the
built design, and adds the checklist rows the new evals/ dir needs.
@Olawoyin007
Olawoyin007 merged commit 05c16f8 into main Jul 23, 2026
12 checks passed
@Olawoyin007
Olawoyin007 deleted the feat/adversarial-eval-loop branch July 23, 2026 21:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant