Eval-Driven Skill Development at Scale
Transform your TDD/PDCA skill iteration from sequential single-scenario reruns into parallel eval swarms where dozens of prompt variants are tested simultaneously against your scenario suite. Claude can autonomously propose prompt mutations, run them in isolated worktrees, score results, and only surface the winning variants for human review. This would have eliminated the 10-minute timeout pain and the do-beads-addon.md vs do-prompts.md rediscovery loop entirely.
Getting started: Use git worktrees with the Task tool spawning parallel sub-agents, each running a different prompt variant against your eval harness, with results aggregated to a scoreboard.
Paste into Claude Code:
I want to set up an autonomous eval-driven skill improvement loop for my PDCA/beads skill. Please: (1) Create a script that spawns N parallel git worktrees, each with a different prompt variant I specify; (2) Run the full eval suite in each worktree concurrently with a per-scenario timeout to avoid the 10-minute hang issue; (3) Aggregate pass/fail/score results into a markdown leaderboard with diffs showing what changed in each variant; (4) Auto-commit the winning variant to a candidate branch and open a PR with the eval evidence in the description. Then demonstrate by generating 5 prompt mutations addressing the 'vacuous green' anti-pattern and running them against the called-shot scenarios.
Copy
Eval-Driven Skill Development at Scale
Transform your TDD/PDCA skill iteration from sequential single-scenario reruns into parallel eval swarms where dozens of prompt variants are tested simultaneously against your scenario suite. Claude can autonomously propose prompt mutations, run them in isolated worktrees, score results, and only surface the winning variants for human review. This would have eliminated the 10-minute timeout pain and the do-beads-addon.md vs do-prompts.md rediscovery loop entirely.
Getting started: Use git worktrees with the Task tool spawning parallel sub-agents, each running a different prompt variant against your eval harness, with results aggregated to a scoreboard.
Paste into Claude Code:
I want to set up an autonomous eval-driven skill improvement loop for my PDCA/beads skill. Please: (1) Create a script that spawns N parallel git worktrees, each with a different prompt variant I specify; (2) Run the full eval suite in each worktree concurrently with a per-scenario timeout to avoid the 10-minute hang issue; (3) Aggregate pass/fail/score results into a markdown leaderboard with diffs showing what changed in each variant; (4) Auto-commit the winning variant to a candidate branch and open a PR with the eval evidence in the description. Then demonstrate by generating 5 prompt mutations addressing the 'vacuous green' anti-pattern and running them against the called-shot scenarios.
Copy