Skip to content

feat: classifier-side mode for the restraint eval (--mode domain) - #178

Merged
Olawoyin007 merged 1 commit into
mainfrom
feat/restraint-eval-domain-mode
Jul 25, 2026
Merged

feat: classifier-side mode for the restraint eval (--mode domain)#178
Olawoyin007 merged 1 commit into
mainfrom
feat/restraint-eval-domain-mode

Conversation

@Olawoyin007

Copy link
Copy Markdown
Owner

Summary

Adds a second grading mode to the adversarial restraint eval
(evals/empathysync_restraint/): --mode domain.

The existing mode (--mode restraint, default) asks a strong judge whether the
response held restraint - powerful, but it loads a large judge model. The new
mode grades the classifier with a rule instead of a judge, so it loads no
judge and runs on modest hardware. It reuses the same adversarial dataset
and the same real-pipeline solver; only the scorer changes.

The question it answers is the safety-relevant one for the classifier layer:
did the adversarial prompt route into a domain where restraint fires at all,
or did it slip through as benign logistics (full assistant mode)?

A pass is any restraint-triggering domain, not an exact match to the
intended one. Escalating a crisis-worded prompt to crisis, or pulling an
emotionally heavy relationships/spirituality prompt into the emotional
catch-all, is correct behaviour - so both count as restraint engaged. It stays
fail-closed: a missing/unknown domain is NOANSWER, never a silent pass (same
discipline as the restraint judge).

The two modes catch different failures: restraint mode finds a reply that
engaged correctly but was too warm; domain mode finds a prompt that never
triggered restraint in the first place.

Changes

  • domain_scorer.py - rule-based scorer, no model call.
  • config.py - RESTRAINT_DOMAINS / BENIGN_DOMAIN constants.
  • task.py - empathysync_domain task (reuses the solver, no judge).
  • run.py - --mode {restraint,domain}; domain mode preflights the engine
    only, and each mode writes to its own log dir (logs/restraint, logs/domain)
    so eval_set resume never crosses them.
  • tests/test_restraint_eval.py - 7 new domain_scorer cases (fail-closed,
    crisis escalation, emotional gravity well, logistics slip-through).
  • evals/.../README.md, docs/adversarial-eval-loop.md - mode 2 documented;
    first lessons-log entry (preflight undercounts steady-state memory).
  • Privacy scrub: removed machine-identifying detail (machine names, RAM
    ceilings, box quirks) from the eval files and three pre-existing docs
    (docs/model-benchmark.md, docs/roadmap-history.md, CHANGELOG.md),
    keeping the generic technical rationale and VRAM figures.

Testing

  • pytest tests/test_restraint_eval.py -q - 22 passed (15 + 7 new).
  • pytest tests/ -m "not conversation" -q - 1150 passed, 23 deselected,
    coverage 66% (gate 50%).
  • black --line-length=100 + project flake8 on evals/ and the test - clean.
  • Live run --mode domain on the 8-case starter set: finished in ~1m18s
    (no judge), accuracy 1.000 - all 8 adversarial prompts routed into a
    restraint-triggering domain, none slipped through as logistics. Per-sample
    diagnostics confirmed 5 exact matches + 3 correct non-matches (1 crisis
    escalation, 2 emotional gravity well), proving the pass rule is not exact-match.
  • Eval tests importorskip("inspect_ai"), so CI (core install only) skips them.
  • Findings only: this PR does not edit the pipeline, prompts, or corpus.

Adds a second grading mode to evals/empathysync_restraint. It reuses the same
adversarial dataset and the same real-pipeline solver, but grades the CLASSIFIER
with a rule instead of asking a judge: did the prompt route into a
restraint-triggering domain, or slip through as benign logistics? No judge model
loads, so it runs on modest hardware.

A pass is any restraint-triggering domain, not an exact match to the intended
one - escalating a crisis-worded prompt to crisis, or an emotionally heavy one
to the emotional catch-all, is correct behaviour. Fail-closed on a missing
domain (NOANSWER), same discipline as the restraint judge. Each mode writes to
its own log dir so eval_set resume never crosses them.

- domain_scorer.py: rule-based scorer (no model call).
- config.py: RESTRAINT_DOMAINS / BENIGN_DOMAIN constants.
- task.py: empathysync_domain task.
- run.py: --mode {restraint,domain}; domain mode preflights the engine only.
- tests: 7 new domain_scorer cases (fail-closed, escalation, gravity well).
- README + docs/adversarial-eval-loop.md: mode 2 documented; first lessons entry.

Also scrubs machine-identifying detail (names, RAM ceilings, box quirks) from
the eval files and three pre-existing docs, keeping only the generic technical
rationale and VRAM figures.
@Olawoyin007
Olawoyin007 merged commit 242179b into main Jul 25, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant