security-adversarial: classify evaluator entrypoint liveness (redblue silent no-op)

[ruvnet]

Nightly Dream Cycle, 2026-08-13. DEEP=security-adversarial, SCAN=redblue,supply-chain. Full report: docs/dream-cycle/2026-08-13-security-adversarial-report.md. Issue: #6.

Hypothesis

Given a nightly pipeline that shells out to third-party evaluator entrypoints and trusts their exit code as a pass/fail signal, when a deterministic classifyEntrypointResult function is interposed between the raw exec result and the pipeline's EVALUATED=yes/no/blocked decision, then it should correctly separate LIVE (real output, exit 0) from BLOCKED (nonzero exit, has a reason) from SUSPICIOUS-SILENT (exit 0, zero output on both streams) — a distinction the raw exit code alone cannot make. Frozen before implementation.

Candidate

+584/-0 across 8 files (2 new source files, 1 new test file, 1 new ADR, 1 new report, 3 small edits). One conceptual change: packages/cli/src/entrypoint.ts (pure classifier) + verify-entrypoint CLI command wired in index.ts/bin.ts.

Evaluation Receipt

Real evaluator: npm test (vitest, this repo's own bench entrypoint).

	Baseline (3de0107)	Candidate
Tests	85	96 (+11, 0 removed/modified)
Result	85 passed	96 passed

Baseline

Parent commit 3de01079abe64e4a1f2d3fe3b758523705b3bf47, 85/85 tests passing, clean build.

Darwin Lineage

Not run — DARWIN=not-applicable. No evolvable population for a single pure classification function with an exhaustive 3-branch structure.

Evidence

Reproduced live tonight (grade A, first-hand): npx -y @metaharness/redblue <any subcommand> → 0-byte stdout, 0-byte stderr, exit 0, every time. Root-caused to the installed package's isMain guard (import.meta.url === \file://${process.argv[1]}`), a known npm-bin-symlink ESM footgun (nodejs/node#57616, grade B; native fix import.meta.mainshipped, grade A). This repo's owndream.config.json:17documents the broken invocation as the live SCAN=redblue evaluator entrypoint. Live receipt against all four realevaluatorEntrypointsin the report:bench→live, flywheel→blocked, redblue→suspicious-silent, darwin --version`→blocked.

Reward-Hack Check

Independent critic (separate agent): CLEAR of reward-hacking — only intended files changed, all additive, no test/gold/threshold tampering. Found + fixed before finalizing: (1) an unquoted multi-word --cmd silently truncated and produced a false live verdict — the exact failure class this tool exists to catch — fixed with a hard usage-error guard on stray positionals, regression test added; (2) error.code can be non-numeric on maxBuffer overflow with no override — fixed with explicit 10MiB buffer + defensive coercion. One risk flagged and correctly not fixed tonight (out of scope): io.exec shells an unsanitized string; safe today (human-typed CLI only), but automating evaluatorEntrypoints → verify-entrypoint later needs an allowlist or execFile first — tracked in issue #6.

Security Review

child_process.exec in bin.ts runs via /bin/sh -c; cmd today only comes from a trusted local CLI flag, no new remote attack surface, no credentials, no LLM calls. IO.exec is optional — no existing call site gains new capability.

Regression Analysis

0 pre-existing tests modified or removed. All 85 baseline tests still pass unchanged; 11 new tests added (5 pure-classifier, 6 CLI-wiring incl. the post-critique regression test).

ADR

ADR-0002 (Proposed) — extends ADR-0001 §2.3/§2.4's "evaluators stay optional" principle to "never silently trusted either."

Gist

No gh gist create available this session (no gist tool, no gh CLI in this environment). Report committed at docs/dream-cycle/2026-08-13-security-adversarial-report.md instead. GIST=LOCAL.

Issue

#6

Witness

report_sha256 : 8984e0cf5bcd850e6122aaec2831b16b74307a2a51e4e0b708118513fe24d64a
session_commit: 3de01079abe64e4a1f2d3fe3b758523705b3bf47
witness       : ec2052aa3ce57068bba6f723ab0a00f9cde7b7a42e5abed71ea4388a43e4a4a4

Verify: sha256sum docs/dream-cycle/2026-08-13-security-adversarial-report.md, then printf '%s%s' "<hash>" "3de01079abe64e4a1f2d3fe3b758523705b3bf47" | sha256sum must equal the witness above. Confirmed tonight via dream-machine witness verify (✓ VALID).

Merge Policy

Draft — human review required. This PR does not carry the automerge-safe label: it adds new CLI surface and a shell-exec wiring path, which per this repo's own guarded-auto-merge policy (.github/workflows/automerge.yml) is exactly the kind of change that stays human-review-only. The session never merges and never applies that label itself.

---

Generated by Claude Code