Skip to content

Add dual-pool-review: adversarial review methodology with dual-pool rotation#118

Closed
YuhaoLin2005 wants to merge 3 commits into
daymade:mainfrom
YuhaoLin2005:add-dual-pool-review
Closed

Add dual-pool-review: adversarial review methodology with dual-pool rotation#118
YuhaoLin2005 wants to merge 3 commits into
daymade:mainfrom
YuhaoLin2005:add-dual-pool-review

Conversation

@YuhaoLin2005

Copy link
Copy Markdown

Problem

The marketplace has excellent QA tools (qa-expert for testing infrastructure, marketplace-health-check for repo audits, code-reviewer for single-pass review), but no adversarial review methodology that prevents both reviewer blind spots and confirmation bias. A single reviewer — no matter how skilled — has domains they don't know, failure modes they're blind to, and assumptions they share with the author. Running the same reviewer multiple times doesn't help: they miss the same things every pass.

Solution

dual-pool-review fills this gap with a structured multi-round adversarial review system:

  • Dual-pool rotation: Fixed pool (4-6 curated personas that learn your patterns) + Random pool (1 fresh reviewer per round, recruited via web search with verifiable expertise)
  • Quote-First Rule: Every reviewer extracts verbatim quotes from web search BEFORE seeing the target — findings without traceable quotes are rejected
  • Symmetric Burden: Reporting zero findings is as expensive as reporting one — must cite 3+ quotes the output successfully satisfies (prevents lazy "LGTM")
  • Evidence Table: Mandatory per-round table linking every finding to a specific, searchable source quote — the non-negotiable quality gate
  • Adaptive Stop Check: Each round asks "did this find issues the previous round missed?" — self-terminates when value stops
  • Concurrence Promotion with Independence Requirement: 2+ reviewers finding the same issue promotes severity, but at least one must be from the random pool (prevents model-bias corroboration)
  • Lightweight Path: Single-file low-risk changes get 1 reviewer + web search; escalate to full dual-pool only if CRITICAL/WARNING found

How it complements existing marketplace skills

Existing Skill dual-pool-review adds
qa-expert Adversarial methodology for test plan review, multi-persona blind-spot coverage
code-reviewer Multi-round rotation with fresh perspectives, evidence-backed findings
marketplace-health-check Same adversarial spirit, but reusable for any output (code, docs, config, identity)
fact-checker Structural review methodology with rotating expertise lenses

Changes

  • dual-pool-review/SKILL.md — 152 lines, follows daymade format conventions (YAML frontmatter with keywords, ## When to Use This Skill section, imperative style)
  • .claude-plugin/marketplace.json — Added plugin entry (category: productivity, 14 keywords), bumped version to 1.79.0
  • README.md / README.zh-CN.md — Added listing as skill feat: Add standalone zero-dependency validator to skill-reviewer #83 with install instructions, use cases, and key features

Prior validation

The methodology was reviewed by 3 adversarial expert personas (Torvalds, Carmack, Cagan) and all 6 findings were incorporated before this submission.

…otation

- Multi-round review with fixed (curated) + random (web-search recruited) reviewer pools
- Quote-First Rule with mandatory evidence tables
- Symmetric Burden: zero findings costs as much as a finding
- Adaptive stop check for self-terminating rounds
- Concurrence promotion requiring random-pool independence
- Lightweight path for single-file low-risk changes
@YuhaoLin2005

Copy link
Copy Markdown
Author

✅ Validation Results

Ran the full marketplace validation pipeline from CLAUDE.md §Validation:

quick_validate.py — PASS ✅

Skill is valid!

Had to remove keywords from SKILL.md YAML frontmatter — quick_validate rejects it as an invalid frontmatter field. The allowed fields are: agent, allowed-tools, argument-hint, compatibility, context, description, disable-model-invocation, hooks, license, metadata, model, name, user-invocable. Keywords live in marketplace.json and README entries only.

check_marketplace.sh — PASS ✅ (2 warn-only items)

Check Result
[1/4] JSON syntax PASS — marketplace.json is valid JSON
[2/4] claude plugin validate SKIP — claude CLI not installed (Windows)
[3/4] source+skills resolution PASS — all plugin source paths resolve to real SKILL.md
[4/4] reverse sync WARN — 2 pre-existing unregistered skills (daymade-claude-code/claude-migrate-memory-to-doc, design-style-picker), not introduced by this PR

check_doc_skill_lists.py — PASS ✅

Authoritative skills in marketplace.json: 81
CLAUDE.md: 81 listed — OK
README.md: 81 listed — OK
README.zh-CN.md: 81 listed — OK
README.md version badge: 1.79.0 — OK
README.zh-CN.md version badge: 1.79.0 — OK
Result: all doc skill lists are in sync with marketplace.json.

security_scan.py — BLOCKED ⚠️

gitleaks not available on Windows. This skill is a pure methodology (SKILL.md only, no scripts, no dependencies, no bundled resources), so the attack surface is minimal.


Summary: 4/4 applicable checks pass. The 2 warnings are pre-existing issues in the repo, not introduced by this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant