From ad17a9798a0df86245f13b010e71ee1de1b9c00d Mon Sep 17 00:00:00 2001
From: Claude <noreply@anthropic.com>
Date: Wed, 15 Jul 2026 03:39:02 +0000
Subject: [PATCH 1/2] docs: design exploration for a model-performance eval
 suite

Captures the interactively-decided design for an in-repo evals/ suite:
model-only diff-out tasks mined from git history across four domains
(bugfix, feature, UI snapshot, balance/simulator), deterministic graders
with an LLM judge on top, fast (~15 min) and slow (simulator) tiers.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BxnJXgu63wex2CCtfuPy7G
---
 .../2026-07-15-model-eval-suite.md            | 254 ++++++++++++++++++
 1 file changed, 254 insertions(+)
 create mode 100644 docs/explorations/2026-07-15-model-eval-suite.md

diff --git a/docs/explorations/2026-07-15-model-eval-suite.md b/docs/explorations/2026-07-15-model-eval-suite.md
new file mode 100644
index 00000000..37a6ba37
--- /dev/null
+++ b/docs/explorations/2026-07-15-model-eval-suite.md
@@ -0,0 +1,254 @@
+# Exploring: an eval suite for measuring model performance on Quest
+
+**Status**: pre-commitment design, crystallized through an interactive Q&A
+session (decisions logged below). Nothing here is implemented — graduate it
+into an `/opsx:propose` change (or just build `evals/` directly, since this
+is dev tooling, not game behavior) when someone is ready to build.
+
+**Prompted by**: "I would like to create an eval suite for measuring
+performance of a model working on various aspects of quest."
+
+## Decisions locked in with the owner
+
+| Axis | Decision |
+|---|---|
+| Subject | **Model-only, scoped tasks** — not the full agent loop. Fixed context in, single answer out, deterministic grading. |
+| Coverage | **Bug fixing**, **feature implementation**, **UI/TUI rendering**, **balance & simulation reasoning** (all four). |
+| Task source | **Mined from git history** (real merged PRs, reverted to create tasks with known-good reference solutions). |
+| Grading | **Deterministic gates + LLM judge** — tests/snapshots/simulators decide pass/fail; a rubric judge scores quality on top. |
+| Task format | **Files in, diff out** — prompt + selected source files + failing output; model returns a unified diff. |
+| Runtime budget | **Fast suite ≈ 15 min** for the main tier; simulator-graded tasks in a separate slow tier. |
+| Location | **In-repo, `evals/`** directory. |
+| Contamination | **Not a concern** — relative comparison between models/prompts is the use case. |
+
+## Why Quest is a good eval substrate
+
+The repo already contains the hard part of any coding eval — trustworthy,
+deterministic graders:
+
+- **`cargo test`** with a large unit/integration surface, including the
+  save-compat corpus (`tests/fixtures/saves/`) that catches serde breakage.
+- **UI snapshot tests** (insta + ratatui `TestBackend`) — committed `.snap`
+  files are pixel-exact ground truth for rendering tasks, and
+  `snapshot_rendering_is_deterministic` catches wall-clock/RNG leaks.
+- **Seeded simulators** (`simulator --check-progression`, `deep_simulator`,
+  `voyage_simulator`) — objective graders for balance reasoning, a domain
+  most coding evals can't touch at all.
+- **~754 merged PRs** of real history to mine (the working clone is shallow
+  at 50 commits; mining requires `git fetch --unshallow`).
+
+## Architecture
+
+```
+evals/
+├── README.md              # how to run, how to add tasks
+├── harness/               # Python runner (uv script, anthropic SDK)
+│   ├── run.py             # materialize → prompt → apply diff → grade → report
+│   ├── judge.py           # LLM judge pass over passing/failing diffs
+│   └── mine.py            # task-mining pipeline (see below)
+├── tasks/
+│   ├── bugfix/qb-001-crit-double-dip/
+│   │   ├── task.yaml      # metadata + grader spec (schema below)
+│   │   ├── prompt.md      # what the model is told
+│   │   ├── bug.patch      # applied to the eval base commit to create the task state
+│   │   └── reference.patch# the known-good solution (never shown to the model)
+│   ├── feature/ …
+│   ├── ui/ …
+│   └── balance/ …         # slow tier
+└── results/               # gitignored; JSON + markdown reports per run
+```
+
+### `task.yaml` schema
+
+```yaml
+id: qb-001-crit-double-dip
+domain: bugfix            # bugfix | feature | ui | balance
+tier: fast                # fast | slow
+source_pr: 412            # provenance, for auditing
+context_files:            # exactly what the model sees, in order
+  - src/combat/damage.rs
+  - src/core/constants.rs
+show_failing_output: true # include grader output in the prompt
+graders:                  # hard gates, all must pass
+  - cargo test --test combat_tests crit_
+  - cargo test --lib combat::
+edit_allowlist:           # diff may only touch these paths
+  - src/combat/**
+judge_rubric: bugfix      # which rubric judge.py applies
+```
+
+### One base commit, one warm build
+
+All fast-tier tasks are pinned to a **single eval snapshot commit** (a tag,
+e.g. `eval-base-2026-07`), with each task's `bug.patch` rebased onto it
+during mining. This is the key to the 15-minute budget: the harness builds
+the snapshot once (warm `target/`), then each task is
+apply-patch → incremental rebuild (~10–30 s) → targeted test filter. Tasks
+that can't be rebased onto the snapshot are dropped or hand-adapted rather
+than letting the suite fragment across base commits.
+
+The snapshot is re-cut deliberately (e.g. quarterly, or after a big system
+lands like Act 2) and tasks re-validated against it — the mining pipeline
+makes regeneration cheap.
+
+### Run flow per task
+
+1. `git worktree add` at the snapshot tag, apply `bug.patch`.
+2. Render the prompt: `prompt.md` + full contents of `context_files` +
+   (optionally) the failing grader output, captured once at mining time so
+   the run doesn't pay for a red build.
+3. One model call. The model must return a single fenced unified diff.
+4. Apply with `git apply --3way`. If it fails to apply, **one** repair round
+   (the model sees only the apply error, not grader feedback), then fail as
+   `apply_error`. A `--strict` flag disables the repair round for pure
+   single-shot measurement.
+5. Enforce the `edit_allowlist` — a diff touching tests, `.snap` files,
+   `evals/`, or anything outside the allowlist is an automatic
+   `illegal_edit` failure. This is what keeps graders trustworthy:
+   *the model can never re-bless a snapshot, edit a test, or touch the
+   harness.*
+6. Run the graders. All green → **pass**; record which gate failed otherwise.
+7. Judge pass (async, batched): rubric-scored 0–4 regardless of pass/fail.
+
+## The four domains
+
+### 1. Bug fixing (`bugfix/`, fast tier, ~25 tasks)
+
+Mined by reverse-applying real fix commits onto the snapshot. Grader = the
+tests that shipped with (or already covered) the fix. Quest-specific
+sub-flavors worth deliberately over-sampling because the repo's conventions
+make them distinctive:
+
+- combat/damage-pipeline math (ordering of Giant's Might → Haven →
+  prestige → ascension → defense → crit)
+- serde/save-compat regressions (grader: `save_compat_tests` on the corpus)
+- RNG determinism leaks (grader: `snapshot_rendering_is_deterministic`)
+- tick-loop timing (grader: `game_tick_tests`)
+
+### 2. Feature implementation (`feature/`, fast tier, ~15 tasks)
+
+Small, scoped features mined from history (a new title, a new fishing rank
+tier, a new input keybinding routed through `handle_game_input`, a new
+`TickEvent`). Grader = the tests that shipped with the feature. The prompt
+states the required behavior precisely (derived from the PR description +
+test names); the test *source* is withheld but expected test names are
+listed, so the task is implementation, not spec-guessing. Full
+add-a-minigame (15 integration points) is out of scope for model-only
+single-shot — that belongs in a future agentic tier.
+
+### 3. UI / TUI rendering (`ui/`, fast tier, ~10 tasks)
+
+Perturb render code (layout math, style/color selection, responsive-tier
+branching, overlay composition) and grade against the committed `.snap`
+ground truth via `cargo test snapshot` filters. The edit allowlist
+categorically excludes `src/ui/snapshots/` — an intentional-looking
+re-bless is an `illegal_edit`. Determinism rules (freezable `ui/clock.rs`)
+give a second grader for free on several tasks.
+
+### 4. Balance & simulation reasoning (`balance/`, **slow tier**, ~10 tasks)
+
+The distinctive domain. Two shapes:
+
+- **Restore**: revert a real balance retune (e.g. part of the era retune in
+  PR #731) and ask the model to restore progression, given the simulator's
+  failing assertion output. Grader: `simulator --check-progression`
+  (~2–5 min).
+- **Hit a target**: "Ascension VII is reached too early at P200 speedrun;
+  adjust costs so it lands in tick range X–Y without breaking the other
+  scenarios." Grader: progression check + a task-specific assertion run.
+
+Slow tier runs on demand / nightly, not per-iteration. Where possible each
+balance task also gets a *fast proxy grader* (a unit test asserting the
+derived curve, e.g. cumulative ascension cost) so the fast suite still
+carries a thin balance signal.
+
+## Mining pipeline (`mine.py`)
+
+1. `git fetch --unshallow`; list merged PRs via the GitHub API.
+2. **Filter**: 5–300 changed lines in `src/`; must touch test files or be
+   covered by existing tests; exclude the heavy audit-log noise (the
+   majority of recent commits are 1-line `meta-audit` history appends —
+   subject-line and path filters remove them).
+3. **Classify** domain by path (`src/combat|core` → bugfix/balance,
+   `src/ui` → ui, etc.) + subject keywords; human curates the final label.
+4. **Validate** the task property that makes grading meaningful:
+   - graders **red** after `bug.patch` (task state),
+   - graders **green** after `reference.patch` (reference solves it),
+   - reference diff stays inside the `edit_allowlist`.
+   Tasks failing any leg are auto-rejected.
+5. Emit the task directory; a human pass writes/edits `prompt.md` (mined
+   prompts start from the PR title/description, rewritten to describe the
+   *symptom*, not the solution).
+
+## LLM judge
+
+Runs after deterministic grading, never overrides it. Per-domain rubrics
+scoring 0–4 on:
+
+- **Convention compliance** — the CLAUDE.md rules a test can't always catch:
+  seeded-RNG cleanliness in the tick loop, Haven-bonus-as-parameter (no
+  globals), `serde(default)` on new `GameState` fields, no wall-clock reads
+  in render code, comment discipline.
+- **Minimality** — did the diff change only what the task required?
+- **Solution match** — semantic proximity to `reference.patch` (informative,
+  not gating; legitimate alternative fixes score full marks if conventions
+  hold).
+
+Judge model pinned per suite version so scores are comparable across runs.
+
+## Metrics & reporting
+
+- **Primary**: pass@1 overall and per domain (all hard gates green).
+- **Diagnostic funnel** per task: `apply_error` → `illegal_edit` →
+  `build_error` → `test_fail` → `pass`. Separating these matters: a model
+  that can't emit applicable diffs needs different work than one that
+  writes plausible-but-wrong fixes.
+- **Secondary**: mean judge score (reported separately, never blended into
+  pass rate); pass@k optional via `--samples k`.
+- Output: `results/<run-id>/report.{json,md}` — per-task rows plus domain
+  aggregates; markdown report designed for pasting into a PR/issue when
+  comparing two models or prompts.
+
+## Suite size and runtime (targets)
+
+| Tier | Tasks | Grading cost | Wall clock |
+|---|---|---|---|
+| Fast | ~50 (25 bugfix, 15 feature, 10 ui) | warm build + ~10–30 s incremental each | ≤ 15 min sequential after one-time snapshot build |
+| Slow | ~10 balance | 2–5 min sim each | ~30–50 min, nightly/on-demand |
+
+Model-call latency overlaps with grading (call task N+1 while N grades), so
+the model side shouldn't extend wall clock.
+
+## Deliberately out of scope (for now)
+
+- **Agentic tier** — full Claude Code sessions graded on final repo state
+  (where add-a-minigame, spec-workflow adherence, and `drive-game` visual
+  verification would live). The task schema above is designed so an agentic
+  runner could reuse the same task directories later (same patches, same
+  graders; just drop the `context_files` spoon-feeding).
+- **Doc/wiki tasks** — gradeable only by judge, weakest signal; the audit
+  skills already cover this ground operationally.
+- **Contamination mitigation** — explicitly waived; if the use case ever
+  shifts to absolute cross-lab claims, add post-cutoff-only mining and
+  symbol-renaming mutation.
+
+## Open questions
+
+1. **Snapshot cadence** — re-cut `eval-base` per quarter, or freeze it for
+   longitudinal comparability and accept growing drift from HEAD? (Leaning:
+   freeze per suite version `v1`, `v2`, …; never mutate a shipped version.)
+2. **Feature-task test visibility** — names-only (proposed) vs. full test
+   source in the prompt. Names-only measures more, but a badly named test
+   makes the task unfair; curation has to catch that.
+3. **Harness language** — Python + anthropic SDK is the fast path; a Rust
+   harness in-repo would dodge a toolchain but slow iteration. (Leaning
+   Python under `evals/harness/`, `uv` single-file scripts, repo stays
+   pure-Rust outside `evals/`.)
+
+## History
+
+- **2026-07-15** — initial design session. Eight design decisions taken
+  interactively (table at top); repo groundwork checked: shallow clone
+  (50 commits, `--unshallow` needed for mining), ~754 merged PRs upstream,
+  recent history dominated by 1-line audit-log commits (mining filter
+  required).

From af69a84bdc3d4a7b1d464359e08888dd6c7d7b7d Mon Sep 17 00:00:00 2001
From: Claude <noreply@anthropic.com>
Date: Wed, 15 Jul 2026 04:21:23 +0000
Subject: [PATCH 2/2] =?UTF-8?q?evals:=20v1=20model-eval=20suite=20?=
 =?UTF-8?q?=E2=80=94=20harness,=205=20validated=20seed=20tasks,=20miner?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Implements the eval-suite design (docs/explorations/2026-07-15-model-eval-suite.md):

- evals/harness/: stdlib-only Python runner. run.py materializes each task
  in a scratch worktree (bug.patch on the pinned base commit), prompts the
  model (files in, one unified diff out), enforces per-task edit allowlists
  + a global denylist (no tests, no .snap re-bless, no harness edits),
  grades with the repo's own tests/snapshots/simulators, and reports an
  apply_error -> illegal_edit -> build_error -> test_fail -> pass funnel.
  validate mode proves every task honest: graders red on the bug state,
  green after reference.patch. judge.py adds rubric quality scores on top;
  mine.py lists/extracts task skeletons from git history.
- evals/tasks/: 5 hand-authored seed tasks, all red/green-validated —
  2 bugfix (damage-percent stacking, serde save-wipe), 1 ui (prestige
  gauge vs snapshots), 1 feature (shift-hotkey via input replay), and
  1 slow-tier balance task graded by the progression simulator.
- make eval-validate target; evals/results/ + __pycache__/ gitignored;
  CLAUDE.md pointer and design-doc history entry.

Reference-mode end-to-end run passes 4/4 on the fast tier; the balance
task validates red/green against the release simulator.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BxnJXgu63wex2CCtfuPy7G
---
 .gitignore                                    |   4 +
 CLAUDE.md                                     |  10 +
 Makefile                                      |   7 +-
 .../2026-07-15-model-eval-suite.md            |  39 +-
 evals/README.md                               | 127 +++++++
 evals/config.toml                             |  38 ++
 evals/harness/judge.py                        |  95 +++++
 evals/harness/lib.py                          | 347 ++++++++++++++++++
 evals/harness/mine.py                         | 189 ++++++++++
 evals/harness/rubrics/balance.md              |  18 +
 evals/harness/rubrics/bugfix.md               |  18 +
 evals/harness/rubrics/feature.md              |  17 +
 evals/harness/rubrics/ui.md                   |  17 +
 evals/harness/run.py                          | 268 ++++++++++++++
 .../balance/qbal-001-meadow-wall/bug.patch    |  17 +
 .../qbal-001-meadow-wall/failing_output.txt   |  69 ++++
 .../balance/qbal-001-meadow-wall/prompt.md    |  20 +
 .../qbal-001-meadow-wall/reference.patch      |  17 +
 .../balance/qbal-001-meadow-wall/task.toml    |   9 +
 .../bugfix/qb-001-armory-stacking/bug.patch   |  21 ++
 .../qb-001-armory-stacking/failing_output.txt |  36 ++
 .../bugfix/qb-001-armory-stacking/prompt.md   |  18 +
 .../qb-001-armory-stacking/reference.patch    |  21 ++
 .../bugfix/qb-001-armory-stacking/task.toml   |   9 +
 evals/tasks/bugfix/qb-002-save-wipe/bug.patch |  12 +
 .../qb-002-save-wipe/failing_output.txt       |  67 ++++
 evals/tasks/bugfix/qb-002-save-wipe/prompt.md |  20 +
 .../bugfix/qb-002-save-wipe/reference.patch   |  12 +
 evals/tasks/bugfix/qb-002-save-wipe/task.toml |   9 +
 .../feature/qf-001-shift-hotkeys/bug.patch    |  13 +
 .../qf-001-shift-hotkeys/failing_output.txt   |  36 ++
 .../feature/qf-001-shift-hotkeys/prompt.md    |  17 +
 .../qf-001-shift-hotkeys/reference.patch      |  13 +
 .../feature/qf-001-shift-hotkeys/task.toml    |   9 +
 .../tasks/ui/qu-001-prestige-gauge/bug.patch  |  16 +
 .../qu-001-prestige-gauge/failing_output.txt  |  99 +++++
 .../tasks/ui/qu-001-prestige-gauge/prompt.md  |  17 +
 .../ui/qu-001-prestige-gauge/reference.patch  |  16 +
 .../tasks/ui/qu-001-prestige-gauge/task.toml  |   9 +
 39 files changed, 1778 insertions(+), 18 deletions(-)
 create mode 100644 evals/README.md
 create mode 100644 evals/config.toml
 create mode 100644 evals/harness/judge.py
 create mode 100644 evals/harness/lib.py
 create mode 100644 evals/harness/mine.py
 create mode 100644 evals/harness/rubrics/balance.md
 create mode 100644 evals/harness/rubrics/bugfix.md
 create mode 100644 evals/harness/rubrics/feature.md
 create mode 100644 evals/harness/rubrics/ui.md
 create mode 100644 evals/harness/run.py
 create mode 100644 evals/tasks/balance/qbal-001-meadow-wall/bug.patch
 create mode 100644 evals/tasks/balance/qbal-001-meadow-wall/failing_output.txt
 create mode 100644 evals/tasks/balance/qbal-001-meadow-wall/prompt.md
 create mode 100644 evals/tasks/balance/qbal-001-meadow-wall/reference.patch
 create mode 100644 evals/tasks/balance/qbal-001-meadow-wall/task.toml
 create mode 100644 evals/tasks/bugfix/qb-001-armory-stacking/bug.patch
 create mode 100644 evals/tasks/bugfix/qb-001-armory-stacking/failing_output.txt
 create mode 100644 evals/tasks/bugfix/qb-001-armory-stacking/prompt.md
 create mode 100644 evals/tasks/bugfix/qb-001-armory-stacking/reference.patch
 create mode 100644 evals/tasks/bugfix/qb-001-armory-stacking/task.toml
 create mode 100644 evals/tasks/bugfix/qb-002-save-wipe/bug.patch
 create mode 100644 evals/tasks/bugfix/qb-002-save-wipe/failing_output.txt
 create mode 100644 evals/tasks/bugfix/qb-002-save-wipe/prompt.md
 create mode 100644 evals/tasks/bugfix/qb-002-save-wipe/reference.patch
 create mode 100644 evals/tasks/bugfix/qb-002-save-wipe/task.toml
 create mode 100644 evals/tasks/feature/qf-001-shift-hotkeys/bug.patch
 create mode 100644 evals/tasks/feature/qf-001-shift-hotkeys/failing_output.txt
 create mode 100644 evals/tasks/feature/qf-001-shift-hotkeys/prompt.md
 create mode 100644 evals/tasks/feature/qf-001-shift-hotkeys/reference.patch
 create mode 100644 evals/tasks/feature/qf-001-shift-hotkeys/task.toml
 create mode 100644 evals/tasks/ui/qu-001-prestige-gauge/bug.patch
 create mode 100644 evals/tasks/ui/qu-001-prestige-gauge/failing_output.txt
 create mode 100644 evals/tasks/ui/qu-001-prestige-gauge/prompt.md
 create mode 100644 evals/tasks/ui/qu-001-prestige-gauge/reference.patch
 create mode 100644 evals/tasks/ui/qu-001-prestige-gauge/task.toml

diff --git a/.gitignore b/.gitignore
index 1c48a5fb..0d413eeb 100644
--- a/.gitignore
+++ b/.gitignore
@@ -21,3 +21,7 @@ Thumbs.db
 # insta pending snapshots (accepted snapshots in src/**/snapshots/ ARE committed)
 *.snap.new
 *.pending-snap
+
+# model-eval run artifacts (evals/tasks/ and the harness ARE committed)
+evals/results/
+__pycache__/
diff --git a/CLAUDE.md b/CLAUDE.md
index 8b838931..60aecb06 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -141,6 +141,16 @@ Larger modules have their own `CLAUDE.md` with implementation patterns, integrat
 | Vessel | `src/vessel/` | [CLAUDE.md](src/vessel/CLAUDE.md) | Act 2: Vessel launch gate + Voyage engine, dark behind the `ACT2_ENABLED` kill-switch (`QUEST_ACT2=1` to preview) |
 | Main Helpers | `src/main_helpers/` | [CLAUDE.md](src/main_helpers/CLAUDE.md) | Orchestration between main.rs and domain modules |
 
+### Model Evals (`evals/`)
+
+A model-performance eval suite over this codebase: scoped "files in, diff
+out" tasks graded by the repo's own tests/snapshots/simulators. Tasks are
+pinned to `evals/config.toml`'s `base_commit`; `make eval-validate` proves
+every fast-tier task is still honest (graders red on the bug state, green on
+the reference fix) — run it when touching `evals/` or re-cutting the base
+commit. Design rationale: `docs/explorations/2026-07-15-model-eval-suite.md`;
+operational docs: [evals/README.md](evals/README.md).
+
 ### Simulators
 
 **Game Simulator** (`src/bin/simulator/`): Headless game balance simulator calling `game_tick_with_context()` with no UI/delay. Supports `--ticks`, `--seed`, `--prestige`, `--runs`, `--strategy <profile>` (casual/optimal/speedrun), `--stormbreaker`, `--assertions`, `--check-progression`. Strategy profiles inject challenge wins, enhancement, sigils, ascension, and auto-prestige.
diff --git a/Makefile b/Makefile
index e760bb17..6b7f935b 100644
--- a/Makefile
+++ b/Makefile
@@ -5,7 +5,7 @@
 # `/opsx:*` skills' bare `openspec ...` commands behave predictably.
 OPENSPEC_VERSION := 1.5.0
 
-.PHONY: check fmt lint test build audit all clean install setup openspec-setup coverage coverage-html coverage-check
+.PHONY: check fmt lint test build audit all clean install setup openspec-setup coverage coverage-html coverage-check eval-validate
 
 # Run all PR checks locally (uses same script as CI)
 check:
@@ -62,6 +62,11 @@ coverage-check:
 		--ignore-filename-regex "(ui/|utils/updater|utils/build_info|tick_events)" \
 		--fail-under-lines 90
 
+# Integrity-check the model-eval task suite (red on bug, green on reference).
+# Fast tier only; add "--tier all" manually to validate simulator-graded tasks.
+eval-validate:
+	@python3 evals/harness/run.py validate
+
 # Clean build artifacts
 clean:
 	@cargo clean
diff --git a/docs/explorations/2026-07-15-model-eval-suite.md b/docs/explorations/2026-07-15-model-eval-suite.md
index 37a6ba37..99d48997 100644
--- a/docs/explorations/2026-07-15-model-eval-suite.md
+++ b/docs/explorations/2026-07-15-model-eval-suite.md
@@ -58,23 +58,21 @@ evals/
 └── results/               # gitignored; JSON + markdown reports per run
 ```
 
-### `task.yaml` schema
-
-```yaml
-id: qb-001-crit-double-dip
-domain: bugfix            # bugfix | feature | ui | balance
-tier: fast                # fast | slow
-source_pr: 412            # provenance, for auditing
-context_files:            # exactly what the model sees, in order
-  - src/combat/damage.rs
-  - src/core/constants.rs
-show_failing_output: true # include grader output in the prompt
-graders:                  # hard gates, all must pass
-  - cargo test --test combat_tests crit_
-  - cargo test --lib combat::
-edit_allowlist:           # diff may only touch these paths
-  - src/combat/**
-judge_rubric: bugfix      # which rubric judge.py applies
+### `task.toml` schema
+
+(TOML rather than YAML: Python 3.11's stdlib `tomllib` keeps the harness
+zero-dependency.)
+
+```toml
+id = "qb-001-armory-stacking"
+domain = "bugfix"              # bugfix | feature | ui | balance
+tier = "fast"                  # fast | slow
+source = "hand-authored"       # or "mined:<sha>" for provenance
+context_files = ["src/combat/player_attack.rs", "src/combat/events.rs"]
+show_failing_output = true     # include cached grader output in the prompt
+graders = ["cargo test --test combat_tests"]   # hard gates, all must pass
+edit_allowlist = ["src/combat/"]               # diff may only touch these
+judge_rubric = "bugfix"        # which rubric judge.py applies
 ```
 
 ### One base commit, one warm build
@@ -252,3 +250,10 @@ the model side shouldn't extend wall clock.
   (50 commits, `--unshallow` needed for mining), ~754 merged PRs upstream,
   recent history dominated by 1-line audit-log commits (mining filter
   required).
+- **2026-07-15 (same day)** — v1 implemented in `evals/` on this design:
+  stdlib-only Python harness (`run.py` / `mine.py` / `judge.py`), TOML task
+  metadata (deviation from the yaml sketch above, noted inline), and five
+  hand-authored seed tasks (2 bugfix, 1 ui, 1 feature, 1 slow-tier
+  balance), each red/green-validated against the real graders. See
+  `evals/README.md` for the operational docs; this exploration stays as
+  the rationale record.
diff --git a/evals/README.md b/evals/README.md
new file mode 100644
index 00000000..8d6886a0
--- /dev/null
+++ b/evals/README.md
@@ -0,0 +1,127 @@
+# Quest model-eval suite
+
+Measures how well a model works on Quest's codebase using scoped,
+deterministic tasks: the model gets a symptom description + selected source
+files + failing check output, and must return one unified diff. The harness
+applies the diff in an isolated worktree and grades it with the repo's own
+graders (tests, snapshot tests, seeded simulators).
+
+Design rationale: [docs/explorations/2026-07-15-model-eval-suite.md](../docs/explorations/2026-07-15-model-eval-suite.md).
+
+## Quick start
+
+```bash
+# Integrity-check every fast task (no API key needed):
+python3 evals/harness/run.py validate
+
+# Dry-run the full pipeline using the reference solutions as "the model":
+python3 evals/harness/run.py run --reference
+
+# Evaluate a real model (needs ANTHROPIC_API_KEY + `pip install anthropic`):
+python3 evals/harness/run.py run --model claude-sonnet-5
+
+# Include the slow tier (simulator-graded balance tasks):
+python3 evals/harness/run.py run --tier all --model claude-sonnet-5
+
+# Judge pass over a finished run (quality scores on top of pass/fail):
+python3 evals/harness/judge.py evals/results/<run-dir>
+```
+
+Reports land in `evals/results/<timestamp>-<label>/report.{json,md}`
+(gitignored). Per-task artifacts (prompt, raw response, extracted patch,
+grader logs) sit alongside for debugging.
+
+## How a task works
+
+```
+evals/tasks/<domain>/<task-id>/
+├── task.toml            # metadata: graders, context files, edit allowlist
+├── prompt.md            # the symptom description the model sees
+├── bug.patch            # applied to the eval base commit -> the task state
+├── reference.patch      # known-good fix (never shown to the model)
+└── failing_output.txt   # grader output on the bug state (shown if enabled)
+```
+
+Every task is pinned to one shared **eval base commit**
+(`config.toml [suite] base_commit`), so a single warm build serves the whole
+fast tier. The runner reuses one scratch worktree (`.worktrees/eval-scratch`)
+and the main `target/` dir; between tasks it hard-resets to base.
+
+**Outcome funnel** (each task lands in exactly one):
+`apply_error` → `illegal_edit` → `build_error` → `test_fail` → `pass`.
+The distinction matters: a model that can't emit applicable diffs needs
+different work than one whose fixes are plausible but wrong.
+
+**Anti-gaming**: diffs may only touch the task's `edit_allowlist`, and never
+the global denylist (`tests/`, `*.snap`, `evals/`, build files — see
+`config.toml`). A model can never re-bless a snapshot, edit a test, or
+patch the harness.
+
+**Repair round**: if a diff fails to *apply* (malformed patch), the model
+gets exactly one retry seeing only the apply error — never grader feedback.
+`--strict` disables even that.
+
+## Validation is the contract
+
+`run.py validate` proves every task is honest:
+
+1. `bug.patch` applies cleanly to the base commit,
+2. at least one grader is **red** on the bug state,
+3. `reference.patch` applies on top and turns everything **green**,
+4. the reference stays inside the edit allowlist.
+
+Run it after authoring or mining tasks, after re-cutting `base_commit`, and
+in any PR that touches `evals/tasks/`. `--refresh-failing-output` rewrites
+each task's cached `failing_output.txt` from the live red run.
+
+## Adding tasks
+
+**Mined (preferred):**
+
+```bash
+git fetch --unshallow                      # once; the CI clone is shallow
+python3 evals/harness/mine.py candidates --limit 40
+python3 evals/harness/mine.py extract <sha> --id qb-0xx-short-name
+```
+
+Skeletons land in `evals/tasks/_incoming/` with the red leg pre-checked.
+Then the human pass: rewrite `prompt.md` to describe the **symptom** (what a
+player or CI sees — never the fix), tighten `context_files` and
+`edit_allowlist`, move the directory to `evals/tasks/<domain>/`, and run
+`run.py validate <task-id>`.
+
+**Hand-authored:** create the same file set by hand; author `bug.patch`
+against the base commit and make `reference.patch` its exact reverse (the
+fix as a model would write it). Validate the same way.
+
+Task ID prefixes: `qb-` bugfix, `qf-` feature, `qu-` ui, `qbal-` balance.
+
+## Prompt-writing rules
+
+- Describe the observable symptom, the affected flow, and the acceptance
+  bar. Name the failing tests if `show_failing_output` is off.
+- Never name the culprit line, constant, or the shape of the fix.
+- Keep `context_files` to what a competent engineer would actually need —
+  spoon-feeding exactly one file with one suspicious line makes the task
+  trivial; dumping 20 files makes it a context-length test instead.
+
+## Scoring
+
+- **Primary**: pass@1 per domain and overall (deterministic gates only).
+- **Diagnostic**: the outcome funnel distribution.
+- **Secondary**: judge scores 0-4 (convention / minimality / solution_match)
+  from `judge.py`, reported separately and never blended into pass rate.
+- pass@k: repeat `run.py run` k times (runs are cheap once warm) and
+  aggregate the per-run `report.json` files; a built-in `--samples` flag is
+  deliberately deferred until someone actually needs it.
+
+Comparing two models = two `run.py run` invocations + diffing `report.md`.
+The suite version + base commit are stamped into every report; results are
+only comparable within the same suite version.
+
+## Tiers
+
+| Tier | Graders | Cost | When |
+|---|---|---|---|
+| `fast` (default) | unit/integration tests, snapshot tests | seconds/task after one warm build | every iteration |
+| `slow` | `simulator --check-progression` (release build) | minutes/task | nightly / pre-release / on demand |
diff --git a/evals/config.toml b/evals/config.toml
new file mode 100644
index 00000000..9d5a35c9
--- /dev/null
+++ b/evals/config.toml
@@ -0,0 +1,38 @@
+# Quest model-eval suite configuration.
+#
+# base_commit is the "eval snapshot" every task's bug.patch is authored
+# against. Re-cutting it is a deliberate act (new suite version): re-validate
+# every task with `run.py validate` after changing it.
+
+[suite]
+version = "v1"
+base_commit = "5f5e3adc47fbb2909e630632e47e0a41b259089b" # main @ 2026-07-14
+
+# Paths a model diff may NEVER touch, regardless of the per-task allowlist.
+# This is what keeps the graders trustworthy: no editing tests, no
+# re-blessing snapshots, no touching the harness or the build definition.
+denied_prefixes = [
+    "tests/",
+    "evals/",
+    "benches/",
+    "scripts/",
+    ".github/",
+    "openspec/",
+    "docs/",
+    "quest.wiki/",
+    "src/ui/snapshots/",
+]
+denied_files = ["Cargo.toml", "Cargo.lock", "Makefile", "build.rs"]
+denied_suffixes = [".snap"]
+
+[run]
+# Default model under evaluation; override with --model.
+model = "claude-sonnet-5"
+max_tokens = 16384
+# Grader timeouts (seconds).
+fast_grader_timeout = 1200
+slow_grader_timeout = 3600
+
+[judge]
+model = "claude-opus-4-8"
+max_tokens = 2048
diff --git a/evals/harness/judge.py b/evals/harness/judge.py
new file mode 100644
index 00000000..c8bbe328
--- /dev/null
+++ b/evals/harness/judge.py
@@ -0,0 +1,95 @@
+#!/usr/bin/env python3
+"""LLM-judge pass over a completed eval run.
+
+Usage:
+    judge.py <results-run-dir> [--model MODEL]
+
+For every task in the run that produced a diff, scores it 0-4 on the
+domain rubric (evals/harness/rubrics/<rubric>.md). Judge scores are
+QUALITY signal only — they never override the deterministic outcome.
+Writes judge.json next to each task's result.json and a judge summary
+into the run's report.json.
+"""
+
+from __future__ import annotations
+
+import json
+import re
+import sys
+from pathlib import Path
+
+import lib
+from lib import log
+
+RUBRICS_DIR = Path(__file__).resolve().parent / "rubrics"
+
+JUDGE_JSON_RE = re.compile(r"\{.*\}", re.S)
+
+
+def judge_prompt(task: lib.Task, model_patch: str, outcome: str) -> str:
+    rubric = (RUBRICS_DIR / f"{task.judge_rubric}.md").read_text()
+    return "\n".join(
+        [
+            "You are grading a model-produced patch for a Rust game codebase (Quest).",
+            "Score strictly per the rubric. The deterministic grader outcome is given;",
+            "do not re-litigate correctness — judge quality dimensions only.",
+            "",
+            "## Rubric",
+            rubric.strip(),
+            "",
+            "## Task given to the model",
+            task.prompt_text.strip(),
+            "",
+            f"## Deterministic outcome: {outcome}",
+            "",
+            "## Model patch",
+            "```diff",
+            model_patch.strip(),
+            "```",
+            "",
+            "## Reference patch (a known-good solution; alternatives may be equally valid)",
+            "```diff",
+            task.reference_patch.read_text().strip(),
+            "```",
+            "",
+            "Respond with ONLY a JSON object:",
+            '{"convention": 0-4, "minimality": 0-4, "solution_match": 0-4, "rationale": "<2 sentences>"}',
+        ]
+    )
+
+
+def main() -> int:
+    args = sys.argv[1:]
+    if not args:
+        log(__doc__)
+        return 0
+    run_dir = Path(args[0])
+    model = lib.CONFIG["judge"]["model"]
+    if "--model" in args:
+        model = args[args.index("--model") + 1]
+
+    report_path = run_dir / "report.json"
+    report = json.loads(report_path.read_text())
+    tasks_by_id = {t.id: t for t in lib.discover_tasks(tier="all")}
+
+    for row in report["tasks"]:
+        task = tasks_by_id.get(row["task"])
+        patch_file = run_dir / row["task"] / "model.patch"
+        if task is None or not patch_file.exists():
+            continue
+        prompt = judge_prompt(task, patch_file.read_text(), row["outcome"])
+        response = lib.call_model(prompt, model, lib.CONFIG["judge"]["max_tokens"])
+        m = JUDGE_JSON_RE.search(response)
+        scores = json.loads(m.group(0)) if m else {"error": "unparseable judge response"}
+        (run_dir / row["task"] / "judge.json").write_text(json.dumps(scores, indent=2))
+        row["judge"] = scores
+        log(f"{row['task']}: {scores}")
+
+    report["meta"]["judge_model"] = model
+    report_path.write_text(json.dumps(report, indent=2))
+    log(f"\njudge scores merged into {report_path}")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())
diff --git a/evals/harness/lib.py b/evals/harness/lib.py
new file mode 100644
index 00000000..b09c5cfa
--- /dev/null
+++ b/evals/harness/lib.py
@@ -0,0 +1,347 @@
+"""Shared plumbing for the Quest model-eval harness.
+
+Zero third-party dependencies: stdlib only (tomllib needs Python 3.11+).
+The optional `anthropic` package is imported lazily and only when a real
+model call is requested (run.py without --reference, or judge.py).
+"""
+
+from __future__ import annotations
+
+import datetime
+import json
+import os
+import re
+import subprocess
+import sys
+import tomllib
+from dataclasses import dataclass, field
+from pathlib import Path
+
+EVALS_DIR = Path(__file__).resolve().parent.parent
+REPO = EVALS_DIR.parent
+TASKS_DIR = EVALS_DIR / "tasks"
+RESULTS_DIR = EVALS_DIR / "results"
+SCRATCH_WT = REPO / ".worktrees" / "eval-scratch"
+
+CONFIG = tomllib.loads((EVALS_DIR / "config.toml").read_text())
+
+# Funnel outcomes, ordered from "never even applied" to "solved".
+OUTCOMES = ["apply_error", "illegal_edit", "build_error", "test_fail", "pass"]
+
+BUILD_ERROR_RE = re.compile(r"error\[E\d+\]|error: could not compile|error: linking")
+
+
+def log(msg: str) -> None:
+    print(msg, flush=True)
+
+
+def sh(
+    cmd: list[str] | str,
+    cwd: Path = REPO,
+    env: dict | None = None,
+    check: bool = True,
+    timeout: int = 1800,
+) -> subprocess.CompletedProcess:
+    """Run a command, capturing combined output as text."""
+    shell = isinstance(cmd, str)
+    full_env = dict(os.environ)
+    if env:
+        full_env.update(env)
+    proc = subprocess.run(
+        cmd,
+        cwd=cwd,
+        shell=shell,
+        env=full_env,
+        stdout=subprocess.PIPE,
+        stderr=subprocess.STDOUT,
+        text=True,
+        timeout=timeout,
+    )
+    if check and proc.returncode != 0:
+        raise RuntimeError(
+            f"command failed ({proc.returncode}): {cmd}\n{proc.stdout[-4000:]}"
+        )
+    return proc
+
+
+# ── Tasks ────────────────────────────────────────────────────────────────
+
+
+@dataclass
+class Task:
+    id: str
+    domain: str
+    tier: str
+    dir: Path
+    context_files: list[str]
+    graders: list[str]
+    edit_allowlist: list[str]
+    judge_rubric: str
+    show_failing_output: bool = True
+    source: str = "hand-authored"
+
+    @property
+    def prompt_text(self) -> str:
+        return (self.dir / "prompt.md").read_text()
+
+    @property
+    def bug_patch(self) -> Path:
+        return self.dir / "bug.patch"
+
+    @property
+    def reference_patch(self) -> Path:
+        return self.dir / "reference.patch"
+
+    @property
+    def failing_output(self) -> str | None:
+        p = self.dir / "failing_output.txt"
+        return p.read_text() if p.exists() else None
+
+    @classmethod
+    def load(cls, tdir: Path) -> "Task":
+        meta = tomllib.loads((tdir / "task.toml").read_text())
+        return cls(
+            id=meta["id"],
+            domain=meta["domain"],
+            tier=meta.get("tier", "fast"),
+            dir=tdir,
+            context_files=meta["context_files"],
+            graders=meta["graders"],
+            edit_allowlist=meta["edit_allowlist"],
+            judge_rubric=meta.get("judge_rubric", meta["domain"]),
+            show_failing_output=meta.get("show_failing_output", True),
+            source=meta.get("source", "hand-authored"),
+        )
+
+
+def discover_tasks(ids: list[str] | None = None, tier: str = "fast") -> list[Task]:
+    tasks = []
+    for toml_path in sorted(TASKS_DIR.glob("*/*/task.toml")):
+        if "_incoming" in toml_path.parts:
+            continue
+        task = Task.load(toml_path.parent)
+        if ids and task.id not in ids:
+            continue
+        if tier != "all" and not ids and task.tier != tier:
+            continue
+        tasks.append(task)
+    if ids:
+        missing = set(ids) - {t.id for t in tasks}
+        if missing:
+            sys.exit(f"unknown task ids: {', '.join(sorted(missing))}")
+    return tasks
+
+
+# ── Worktree management ──────────────────────────────────────────────────
+
+
+def base_commit() -> str:
+    return CONFIG["suite"]["base_commit"]
+
+
+def ensure_worktree() -> Path:
+    """Create (or reset) the shared scratch worktree at the eval base commit.
+
+    One worktree is reused for every task, reset between tasks, so cargo's
+    incremental cache in the shared target dir stays warm.
+    """
+    base = base_commit()
+    if not SCRATCH_WT.exists():
+        sh(["git", "worktree", "prune"])
+        sh(["git", "worktree", "add", "--detach", str(SCRATCH_WT), base])
+    reset_worktree()
+    return SCRATCH_WT
+
+
+def reset_worktree() -> None:
+    sh(["git", "-C", str(SCRATCH_WT), "reset", "--hard", base_commit()])
+    sh(["git", "-C", str(SCRATCH_WT), "clean", "-fdq"])
+
+
+# ── Patches ──────────────────────────────────────────────────────────────
+
+
+def apply_patch(wt: Path, patch: str | Path) -> tuple[bool, str]:
+    """Apply a unified diff in the worktree. Returns (ok, error_output)."""
+    if isinstance(patch, Path):
+        patch_text = patch.read_text()
+    else:
+        patch_text = patch
+    if not patch_text.endswith("\n"):
+        patch_text += "\n"
+    for extra in ([], ["--recount", "-C1"]):
+        proc = subprocess.run(
+            ["git", "apply", "--whitespace=nowarn", *extra, "-"],
+            cwd=wt,
+            input=patch_text,
+            text=True,
+            stdout=subprocess.PIPE,
+            stderr=subprocess.STDOUT,
+        )
+        if proc.returncode == 0:
+            return True, ""
+    return False, proc.stdout
+
+
+def patch_paths(patch_text: str) -> list[str]:
+    """Every repo-relative path a diff touches (old and new sides)."""
+    paths: set[str] = set()
+    for m in re.finditer(r"^diff --git a/(\S+) b/(\S+)", patch_text, re.M):
+        paths.update(m.groups())
+    for m in re.finditer(r"^(?:---|\+\+\+) [ab]/(\S+)", patch_text, re.M):
+        paths.add(m.group(1))
+    paths.discard("/dev/null")
+    return sorted(paths)
+
+
+def path_violations(paths: list[str], allowlist: list[str]) -> list[str]:
+    """Paths that fall outside the task allowlist or inside the global denylist."""
+    suite = CONFIG["suite"]
+    bad = []
+    for p in paths:
+        denied = (
+            any(p.startswith(pre) for pre in suite["denied_prefixes"])
+            or p in suite["denied_files"]
+            or any(p.endswith(suf) for suf in suite["denied_suffixes"])
+        )
+        allowed = any(p.startswith(a) for a in allowlist)
+        if denied or not allowed:
+            bad.append(p)
+    return bad
+
+
+# ── Graders ──────────────────────────────────────────────────────────────
+
+
+@dataclass
+class GraderResult:
+    cmd: str
+    ok: bool
+    output: str
+    build_error: bool = False
+
+
+def run_graders(task: Task, wt: Path) -> list[GraderResult]:
+    timeout_key = "slow_grader_timeout" if task.tier == "slow" else "fast_grader_timeout"
+    timeout = CONFIG["run"][timeout_key]
+    env = {
+        "CARGO_TARGET_DIR": str(REPO / "target"),
+        "INSTA_UPDATE": "no",
+    }
+    results = []
+    for cmd in task.graders:
+        try:
+            proc = sh(cmd, cwd=wt, env=env, check=False, timeout=timeout)
+            out, code = proc.stdout, proc.returncode
+        except subprocess.TimeoutExpired as e:
+            out, code = (e.output or "") + "\n[grader timed out]", 124
+        results.append(
+            GraderResult(
+                cmd=cmd,
+                ok=code == 0,
+                output=out,
+                build_error=code != 0 and bool(BUILD_ERROR_RE.search(out)),
+            )
+        )
+    return results
+
+
+def classify(graders: list[GraderResult]) -> str:
+    if all(g.ok for g in graders):
+        return "pass"
+    if any(g.build_error for g in graders):
+        return "build_error"
+    return "test_fail"
+
+
+def failing_excerpt(graders: list[GraderResult], limit: int = 6000) -> str:
+    failed = [g for g in graders if not g.ok]
+    per = limit // max(1, len(failed))
+    chunks = []
+    for g in failed:
+        # Drop passing-test noise so the excerpt is all signal.
+        kept = "\n".join(
+            line for line in g.output.splitlines() if not line.endswith("... ok")
+        )
+        chunks.append(f"$ {g.cmd}\n{kept[-per:]}")
+    return "\n\n".join(chunks)
+
+
+# ── Model calls ──────────────────────────────────────────────────────────
+
+
+def call_model(prompt: str, model: str, max_tokens: int) -> str:
+    try:
+        import anthropic
+    except ImportError:
+        sys.exit(
+            "the `anthropic` package is required for live model calls "
+            "(pip install anthropic), or use --reference for a dry run"
+        )
+    client = anthropic.Anthropic()
+    msg = client.messages.create(
+        model=model,
+        max_tokens=max_tokens,
+        messages=[{"role": "user", "content": prompt}],
+    )
+    return "".join(b.text for b in msg.content if b.type == "text")
+
+
+DIFF_BLOCK_RE = re.compile(r"```(?:diff|patch)?\n(.*?)```", re.S)
+
+
+def extract_diff(response: str) -> str | None:
+    """Pull the last plausible unified diff out of a model response."""
+    candidates = [
+        m.group(1)
+        for m in DIFF_BLOCK_RE.finditer(response)
+        if re.search(r"^(--- |\+\+\+ |diff --git )", m.group(1), re.M)
+    ]
+    if candidates:
+        return candidates[-1]
+    # Unfenced fallback: the whole response looks like a diff.
+    if re.match(r"^(diff --git |--- )", response.strip()):
+        return response.strip() + "\n"
+    return None
+
+
+# ── Reporting ────────────────────────────────────────────────────────────
+
+
+def new_run_dir(label: str) -> Path:
+    stamp = datetime.datetime.now(datetime.UTC).strftime("%Y%m%d-%H%M%S")
+    run_dir = RESULTS_DIR / f"{stamp}-{label}"
+    run_dir.mkdir(parents=True, exist_ok=False)
+    return run_dir
+
+
+def write_report(run_dir: Path, rows: list[dict], meta: dict) -> None:
+    report = {"meta": meta, "tasks": rows}
+    (run_dir / "report.json").write_text(json.dumps(report, indent=2))
+
+    by_domain: dict[str, list[dict]] = {}
+    for r in rows:
+        by_domain.setdefault(r["domain"], []).append(r)
+
+    lines = [
+        f"# Eval run: {run_dir.name}",
+        "",
+        f"- suite: {CONFIG['suite']['version']} @ `{base_commit()[:12]}`",
+        *[f"- {k}: {v}" for k, v in meta.items()],
+        "",
+        "| task | domain | outcome | failed gate |",
+        "|---|---|---|---|",
+    ]
+    for r in rows:
+        lines.append(
+            f"| {r['task']} | {r['domain']} | {r['outcome']} | {r.get('failed_gate', '')} |"
+        )
+    lines += ["", "## By domain", ""]
+    for domain, rs in sorted(by_domain.items()):
+        passed = sum(1 for r in rs if r["outcome"] == "pass")
+        lines.append(f"- **{domain}**: {passed}/{len(rs)} pass")
+    total_pass = sum(1 for r in rows if r["outcome"] == "pass")
+    lines += ["", f"**Total: {total_pass}/{len(rows)} pass**", ""]
+    (run_dir / "report.md").write_text("\n".join(lines))
+    log("\n".join(lines[-(len(by_domain) + 4) :]))
+    log(f"\nreport: {run_dir / 'report.md'}")
diff --git a/evals/harness/mine.py b/evals/harness/mine.py
new file mode 100644
index 00000000..fee6b349
--- /dev/null
+++ b/evals/harness/mine.py
@@ -0,0 +1,189 @@
+#!/usr/bin/env python3
+"""Task-mining pipeline: turn real Quest history into eval tasks.
+
+Usage:
+    mine.py candidates [--limit N] [--since REV]
+        List commits that look like viable task material, scored and
+        classified by domain. Requires full history:
+        `git fetch --unshallow` first if the clone is shallow.
+
+    mine.py extract <commit> --id <task-id> [--domain D]
+        Emit a task skeleton into evals/tasks/_incoming/<task-id>/:
+        bug.patch (reverse of the commit, src/ only), reference.patch
+        (forward), guessed graders/context, and a prompt.md stub from the
+        commit message. Then validates the red leg (bug applies to the
+        eval base and at least one grader fails). Curate by hand, move it
+        into evals/tasks/<domain>/, rewrite prompt.md to describe the
+        SYMPTOM (never the solution), and run `run.py validate <task-id>`.
+
+Mined tasks are raw material, not finished tasks — every one needs the
+human pass before it counts.
+"""
+
+from __future__ import annotations
+
+import re
+import subprocess
+import sys
+
+import lib
+from lib import log, sh
+
+# Subjects that are almost never task material.
+NOISE_RE = re.compile(
+    r"^(Log |meta-audit|doc-audit|wiki-audit|test-audit|dependency-audit|perf-audit"
+    r"|docs?:|chore|style:|Merge |Revert )",
+    re.I,
+)
+
+# Path prefix -> (domain, grader commands) guesses, mirroring the
+# "How to Verify Your Change" table in CLAUDE.md.
+DOMAIN_MAP = [
+    ("src/ui/", "ui", ["cargo test --lib snapshot"]),
+    ("src/combat/", "bugfix", ["cargo test --test combat_tests"]),
+    ("src/core/tick", "bugfix", ["cargo test --test game_tick_tests"]),
+    ("src/core/constants.rs", "balance", ["cargo run --release --bin simulator -- --check-progression"]),
+    ("src/items/", "bugfix", ["cargo test --test item_tests"]),
+    ("src/input/", "feature", ["cargo test --bin quest input::replay_tests"]),
+    ("src/character/", "bugfix", ["cargo test --test save_compat_tests", "cargo test --test character_tests"]),
+    ("src/deep/", "bugfix", ["cargo test --test deep_tests"]),
+    ("src/loom/", "bugfix", ["cargo test --test loom_tests"]),
+    ("src/zones/", "bugfix", ["cargo test --test zone_tests"]),
+    ("src/fishing/", "bugfix", ["cargo test --test fishing_tests"]),
+    ("src/challenges/", "feature", ["cargo test --lib challenges"]),
+]
+
+
+def is_task_source(path: str) -> bool:
+    """Rust source under src/ — excludes docs, snapshots, and test-only code."""
+    return (
+        path.startswith("src/")
+        and path.endswith(".rs")
+        and not path.startswith("src/ui/snapshots/")
+    )
+
+
+def numstat(rev: str) -> list[tuple[int, int, str]]:
+    out = sh(["git", "show", "--numstat", "--format=", rev], check=True).stdout
+    rows = []
+    for line in out.splitlines():
+        parts = line.split("\t")
+        if len(parts) == 3 and parts[0].isdigit():
+            rows.append((int(parts[0]), int(parts[1]), parts[2]))
+    return rows
+
+
+def classify(paths: list[str]) -> tuple[str, list[str]]:
+    for prefix, domain, graders in DOMAIN_MAP:
+        if any(p.startswith(prefix) for p in paths):
+            return domain, graders
+    return "bugfix", ["cargo test"]
+
+
+def cmd_candidates(args: list[str]) -> int:
+    limit = int(args[args.index("--limit") + 1]) if "--limit" in args else 40
+    since = args[args.index("--since") + 1] if "--since" in args else None
+
+    if sh(["git", "rev-parse", "--is-shallow-repository"]).stdout.strip() == "true":
+        log("NOTE: shallow clone — run `git fetch --unshallow` to mine full history\n")
+
+    rev_range = f"{since}..HEAD" if since else "HEAD"
+    commits = sh(
+        ["git", "log", "--no-merges", "--format=%H|%s", rev_range], check=True
+    ).stdout.splitlines()
+
+    found = 0
+    for line in commits:
+        if found >= limit:
+            break
+        sha, subject = line.split("|", 1)
+        if NOISE_RE.search(subject):
+            continue
+        rows = numstat(sha)
+        src_rows = [r for r in rows if is_task_source(r[2])]
+        touched_tests = [r[2] for r in rows if r[2].startswith("tests/") or "test" in r[2]]
+        src_lines = sum(a + d for a, d, _ in src_rows)
+        if not src_rows or not (5 <= src_lines <= 300):
+            continue
+        domain, graders = classify([r[2] for r in src_rows])
+        covered = "tests✓" if touched_tests else "tests?"
+        log(f"{sha[:10]}  {domain:8} {src_lines:4}L {covered}  {subject[:80]}")
+        found += 1
+    log(f"\n{found} candidate(s). Extract one with: mine.py extract <sha> --id <task-id>")
+    return 0
+
+
+def cmd_extract(args: list[str]) -> int:
+    sha = args[0]
+    if "--id" not in args:
+        sys.exit("--id <task-id> is required")
+    task_id = args[args.index("--id") + 1]
+
+    rows = numstat(sha)
+    src_paths = [p for _, _, p in rows if is_task_source(p)]
+    if not src_paths:
+        sys.exit(f"{sha}: no src/ changes to build a task from")
+    domain, graders = classify(src_paths)
+    if "--domain" in args:
+        domain = args[args.index("--domain") + 1]
+
+    forward = sh(["git", "diff", f"{sha}^", sha, "--", *src_paths], check=True).stdout
+    reverse = sh(["git", "diff", sha, f"{sha}^", "--", *src_paths], check=True).stdout
+    subject_body = sh(["git", "log", "-1", "--format=%s%n%n%b", sha], check=True).stdout
+
+    tdir = lib.TASKS_DIR / "_incoming" / task_id
+    tdir.mkdir(parents=True, exist_ok=True)
+    (tdir / "bug.patch").write_text(reverse)
+    (tdir / "reference.patch").write_text(forward)
+    (tdir / "prompt.md").write_text(
+        "<!-- CURATE ME: describe the SYMPTOM a player/CI sees, never the fix.\n"
+        f"     Mined from {sha[:12]}. Original message:\n{subject_body.strip()}\n-->\n"
+    )
+    dirs = sorted({p.rsplit("/", 1)[0] + "/" for p in src_paths})
+    (tdir / "task.toml").write_text(
+        f'id = "{task_id}"\n'
+        f'domain = "{domain}"\n'
+        f'tier = "{"slow" if domain == "balance" else "fast"}"\n'
+        f'source = "mined:{sha[:12]}"\n'
+        f"context_files = {src_paths!r}\n".replace("'", '"')
+        + f"graders = {graders!r}\n".replace("'", '"')
+        + f"edit_allowlist = {dirs!r}\n".replace("'", '"')
+        + f'judge_rubric = "{domain}"\n'
+    )
+
+    # Red-leg check: does the bug patch apply to the eval base, and does
+    # at least one guessed grader actually fail there?
+    wt = lib.ensure_worktree()
+    ok, err = lib.apply_patch(wt, tdir / "bug.patch")
+    if not ok:
+        log(f"WARNING: bug.patch does not apply to base {lib.base_commit()[:12]} — "
+            f"needs manual rebase:\n{err[:800]}")
+        return 1
+    task = lib.Task.load(tdir)
+    red = lib.run_graders(task, wt)
+    red_fail = [g for g in red if not g.ok]
+    if red_fail:
+        (tdir / "failing_output.txt").write_text(lib.failing_excerpt(red))
+        log(f"red leg OK: {len(red_fail)}/{len(red)} graders fail on the bug state")
+    else:
+        log("WARNING: no guessed grader fails on the bug state — find the real "
+            "covering test or discard this candidate")
+    log(f"\nskeleton written to {tdir}\nNext: curate prompt.md + task.toml, move to "
+        f"evals/tasks/{domain}/{task_id}/, then run.py validate {task_id}")
+    return 0
+
+
+def main() -> int:
+    args = sys.argv[1:]
+    if not args or args[0] in {"-h", "--help"}:
+        log(__doc__)
+        return 0
+    if args[0] == "candidates":
+        return cmd_candidates(args[1:])
+    if args[0] == "extract":
+        return cmd_extract(args[1:])
+    sys.exit(f"unknown command: {args[0]}")
+
+
+if __name__ == "__main__":
+    sys.exit(main())
diff --git a/evals/harness/rubrics/balance.md b/evals/harness/rubrics/balance.md
new file mode 100644
index 00000000..b7ba0924
--- /dev/null
+++ b/evals/harness/rubrics/balance.md
@@ -0,0 +1,18 @@
+# Balance rubric
+
+Score each dimension 0-4 (0 = badly violated, 4 = exemplary).
+
+**convention** — Balance changes live in the documented constants
+(`core/constants.rs` tables and named multipliers), keep the documented
+curve shapes (e.g. fracture 1.6x, Loom 1.25x scaling), and stay consistent
+with the numbers CLAUDE.md documents. Penalize magic numbers scattered into
+logic code instead of the constants table.
+
+**minimality** — Adjusts the smallest set of knobs that restores the
+progression target. Penalize shotgun retunes of unrelated zones/systems.
+
+**solution_match** — Semantic proximity to the reference retune. A different
+knob that legitimately restores pacing (and would plausibly survive the full
+progression gate) can score 3-4; compensating hacks that mask the symptom
+(e.g. buffing player XP globally to outrun an enemy-stat inflation) score
+0-1 even when the simulator passes.
diff --git a/evals/harness/rubrics/bugfix.md b/evals/harness/rubrics/bugfix.md
new file mode 100644
index 00000000..28bcdc7f
--- /dev/null
+++ b/evals/harness/rubrics/bugfix.md
@@ -0,0 +1,18 @@
+# Bugfix rubric
+
+Score each dimension 0-4 (0 = badly violated, 4 = exemplary).
+
+**convention** — Quest/CLAUDE.md conventions a test can't always catch:
+seeded-RNG cleanliness in tick-path code (no wall-clock or thread RNG),
+Haven bonuses passed as parameters (never globals), `#[serde(default)]` /
+`alias` / migration for save-format changes (never editing the corpus),
+comment discipline (comments state constraints, not narration of the change).
+
+**minimality** — The diff changes only what the defect requires. Penalize
+drive-by refactors, renamed variables, reformatted untouched lines, added
+comments explaining the fix to a reviewer.
+
+**solution_match** — Semantic proximity to the reference fix. A different
+but equally sound fix that respects the pipeline's documented ordering
+scores 4; a fix that happens to pass the graders while distorting adjacent
+behavior (e.g. compensating constants) scores 0-1.
diff --git a/evals/harness/rubrics/feature.md b/evals/harness/rubrics/feature.md
new file mode 100644
index 00000000..f6061252
--- /dev/null
+++ b/evals/harness/rubrics/feature.md
@@ -0,0 +1,17 @@
+# Feature rubric
+
+Score each dimension 0-4 (0 = badly violated, 4 = exemplary).
+
+**convention** — The implementation matches Quest's established patterns for
+the touched module: input routed through the existing `handle_game_input`
+dispatch shape, `InputResult` variants chosen correctly (a save-worthy action
+returns `NeedsSaveWithEvent`), module layout (types/logic split) respected,
+difficulty-tier and forfeit patterns followed where relevant.
+
+**minimality** — Implements exactly the described behavior. Penalize
+speculative extensions, config knobs nobody asked for, and copy-paste that
+duplicates an existing helper instead of reusing it.
+
+**solution_match** — Semantic proximity to the reference implementation.
+Equally idiomatic alternates score 4; implementations that satisfy the tests
+by special-casing the tested inputs score 0.
diff --git a/evals/harness/rubrics/ui.md b/evals/harness/rubrics/ui.md
new file mode 100644
index 00000000..1058c003
--- /dev/null
+++ b/evals/harness/rubrics/ui.md
@@ -0,0 +1,17 @@
+# UI rubric
+
+Score each dimension 0-4 (0 = badly violated, 4 = exemplary).
+
+**convention** — Quest UI determinism rules: no wall-clock reads in render
+code (only the freezable `ui/clock.rs`), no RNG in render paths, responsive
+size-tier branching kept intact, styles built with the same idioms as the
+surrounding code (ratatui `Style`/`Span` composition, not ad-hoc escape
+codes).
+
+**minimality** — The rendering change touches only the affected widget's
+math or content. Penalize layout reshuffles, constraint changes, or style
+churn on lines the task didn't require.
+
+**solution_match** — Semantic proximity to the reference. Producing the
+expected frame by hardcoding literal strings/widths that happen to match the
+snapshot scores 0; restoring the general computation scores 4.
diff --git a/evals/harness/run.py b/evals/harness/run.py
new file mode 100644
index 00000000..68bbfee0
--- /dev/null
+++ b/evals/harness/run.py
@@ -0,0 +1,268 @@
+#!/usr/bin/env python3
+"""Quest model-eval runner.
+
+Usage:
+    run.py list                          # show all tasks
+    run.py validate [ids...] [options]   # integrity-check tasks (no model)
+    run.py run [ids...] [options]        # evaluate a model on tasks
+
+Options:
+    --tier fast|slow|all      task tier to include (default: fast)
+    --model MODEL             model under evaluation (default: config.toml)
+    --reference               answer with reference.patch instead of a model
+                              (dry-runs the whole pipeline, no API key needed)
+    --strict                  no repair round on malformed/unappliable diffs
+    --refresh-failing-output  (validate) rewrite each task's failing_output.txt
+
+`validate` proves each task is honest: graders RED on the bug state, GREEN
+after reference.patch, and the reference stays inside the edit allowlist.
+Run it after authoring or mining any task, and after moving base_commit.
+"""
+
+from __future__ import annotations
+
+import json
+import sys
+
+import lib
+from lib import log
+
+
+def build_prompt(task: lib.Task, wt) -> str:
+    parts = [
+        "You are fixing an issue in Quest, a terminal-based idle RPG written in Rust.",
+        "Work only from the information below; you cannot run commands or open other files.",
+        "",
+        "## Task",
+        "",
+        task.prompt_text.strip(),
+        "",
+        "## Files",
+    ]
+    for rel in task.context_files:
+        content = (wt / rel).read_text()
+        parts += ["", f"### {rel}", "", "```rust", content.rstrip("\n"), "```"]
+    if task.show_failing_output and task.failing_output:
+        parts += [
+            "",
+            "## Failing check output",
+            "",
+            "```",
+            task.failing_output.strip(),
+            "```",
+        ]
+    allow = ", ".join(f"`{a}`" for a in task.edit_allowlist)
+    parts += [
+        "",
+        "## Response format (required)",
+        "",
+        "Reply with exactly ONE unified diff in a fenced ```diff block, using",
+        "`--- a/<path>` / `+++ b/<path>` headers with paths relative to the repo root.",
+        f"You may only modify files under: {allow}.",
+        "Do not modify tests, snapshots, or build files. Keep the change minimal.",
+        "Any text outside the fenced diff block is ignored.",
+    ]
+    return "\n".join(parts)
+
+
+def grade_diff(task: lib.Task, wt, diff_text: str) -> dict:
+    """Apply a candidate diff to the bugged worktree and run the graders."""
+    paths = lib.patch_paths(diff_text)
+    if not paths:
+        return {"outcome": "apply_error", "failed_gate": "no file paths found in diff"}
+    violations = lib.path_violations(paths, task.edit_allowlist)
+    if violations:
+        return {
+            "outcome": "illegal_edit",
+            "failed_gate": f"touches disallowed paths: {', '.join(violations)}",
+        }
+    ok, err = lib.apply_patch(wt, diff_text)
+    if not ok:
+        return {"outcome": "apply_error", "failed_gate": "git apply failed", "apply_error": err}
+    graders = lib.run_graders(task, wt)
+    outcome = lib.classify(graders)
+    failed = next((g.cmd for g in graders if not g.ok), "")
+    return {
+        "outcome": outcome,
+        "failed_gate": failed,
+        "grader_log": "\n\n".join(f"$ {g.cmd}\n{g.output[-4000:]}" for g in graders),
+    }
+
+
+def cmd_run(tasks: list[lib.Task], opts: dict) -> int:
+    model = opts.get("model") or lib.CONFIG["run"]["model"]
+    reference_mode = opts.get("reference", False)
+    label = "reference" if reference_mode else model.replace("/", "-")
+    run_dir = lib.new_run_dir(label)
+    rows = []
+
+    for task in tasks:
+        log(f"\n=== {task.id} ({task.domain}/{task.tier}) ===")
+        wt = lib.ensure_worktree()
+        ok, err = lib.apply_patch(wt, task.bug_patch)
+        if not ok:
+            sys.exit(f"{task.id}: bug.patch no longer applies to base — re-validate the suite:\n{err}")
+
+        task_dir = run_dir / task.id
+        task_dir.mkdir(parents=True)
+        prompt = build_prompt(task, wt)
+        (task_dir / "prompt.md").write_text(prompt)
+
+        if reference_mode:
+            diff_text = task.reference_patch.read_text()
+            response = "(reference mode: reference.patch used as the response)"
+        else:
+            response = lib.call_model(prompt, model, lib.CONFIG["run"]["max_tokens"])
+            diff_text = lib.extract_diff(response)
+        (task_dir / "response.txt").write_text(response)
+
+        result: dict
+        if diff_text is None:
+            result = {"outcome": "apply_error", "failed_gate": "no diff block in response"}
+        else:
+            result = grade_diff(task, wt, diff_text)
+        # One repair round: only for responses that never applied (malformed
+        # or missing diff), never for grader feedback — that would turn
+        # single-shot into iteration.
+        if (
+            result["outcome"] == "apply_error"
+            and not opts.get("strict")
+            and not reference_mode
+        ):
+            repair_prompt = (
+                prompt
+                + "\n\n## Your previous response could not be applied\n\n```\n"
+                + result.get("apply_error", result["failed_gate"])[-2000:]
+                + "\n```\n\nResend the complete corrected diff in one ```diff block."
+            )
+            response = lib.call_model(repair_prompt, model, lib.CONFIG["run"]["max_tokens"])
+            (task_dir / "response_repair.txt").write_text(response)
+            diff_text = lib.extract_diff(response)
+            if diff_text is not None:
+                lib.reset_worktree()
+                lib.apply_patch(wt, task.bug_patch)
+                result = grade_diff(task, wt, diff_text)
+                result["repaired"] = True
+
+        if diff_text is not None:
+            (task_dir / "model.patch").write_text(diff_text)
+        if "grader_log" in result:
+            (task_dir / "graders.txt").write_text(result.pop("grader_log"))
+
+        row = {"task": task.id, "domain": task.domain, "tier": task.tier, **result}
+        (task_dir / "result.json").write_text(json.dumps(row, indent=2))
+        rows.append(row)
+        log(f"    -> {row['outcome']}" + (f" ({row['failed_gate']})" if row["outcome"] != "pass" else ""))
+
+    lib.write_report(
+        run_dir,
+        rows,
+        {"mode": "reference" if reference_mode else "model", "model": None if reference_mode else model},
+    )
+    return 0 if all(r["outcome"] == "pass" for r in rows) else 1
+
+
+def cmd_validate(tasks: list[lib.Task], opts: dict) -> int:
+    failures = []
+    for task in tasks:
+        log(f"\n=== validate {task.id} ({task.domain}/{task.tier}) ===")
+
+        ref_paths = lib.patch_paths(task.reference_patch.read_text())
+        violations = lib.path_violations(ref_paths, task.edit_allowlist)
+        if violations:
+            failures.append(f"{task.id}: reference.patch outside allowlist: {violations}")
+            log("    FAIL: reference outside allowlist")
+            continue
+
+        wt = lib.ensure_worktree()
+        ok, err = lib.apply_patch(wt, task.bug_patch)
+        if not ok:
+            failures.append(f"{task.id}: bug.patch does not apply: {err[:500]}")
+            log("    FAIL: bug.patch does not apply")
+            continue
+
+        red = lib.run_graders(task, wt)
+        red_fail = [g for g in red if not g.ok]
+        if not red_fail:
+            failures.append(f"{task.id}: graders all GREEN on bug state — task detects nothing")
+            log("    FAIL: no grader is red on the bug state")
+            continue
+        log(f"    red OK: {len(red_fail)}/{len(red)} graders fail on bug state")
+        if opts.get("refresh_failing_output"):
+            (task.dir / "failing_output.txt").write_text(lib.failing_excerpt(red))
+            log("    wrote failing_output.txt")
+
+        ok, err = lib.apply_patch(wt, task.reference_patch)
+        if not ok:
+            failures.append(f"{task.id}: reference.patch does not apply on bug state: {err[:500]}")
+            log("    FAIL: reference.patch does not apply")
+            continue
+
+        green = lib.run_graders(task, wt)
+        green_fail = [g for g in green if not g.ok]
+        if green_fail:
+            failures.append(
+                f"{task.id}: graders still RED after reference: {[g.cmd for g in green_fail]}"
+            )
+            log(f"    FAIL: still red after reference: {[g.cmd for g in green_fail]}")
+            continue
+        log("    green OK: all graders pass after reference.patch")
+
+    log("\n" + "=" * 60)
+    if failures:
+        log(f"VALIDATE FAILED ({len(failures)}):")
+        for f in failures:
+            log(f"  - {f}")
+        return 1
+    log(f"VALIDATE OK: {len(tasks)} task(s) honest (red on bug, green on reference)")
+    return 0
+
+
+def cmd_list(tasks: list[lib.Task], _opts: dict) -> int:
+    for t in tasks:
+        log(f"{t.id:28} {t.domain:8} {t.tier:5} graders={len(t.graders)}  {t.dir.relative_to(lib.EVALS_DIR)}")
+    log(f"\n{len(tasks)} task(s)")
+    return 0
+
+
+def main() -> int:
+    args = sys.argv[1:]
+    if not args or args[0] in {"-h", "--help"}:
+        log(__doc__)
+        return 0
+    cmd, rest = args[0], args[1:]
+
+    opts: dict = {"tier": "fast"}
+    ids = []
+    i = 0
+    while i < len(rest):
+        a = rest[i]
+        if a == "--tier":
+            opts["tier"] = rest[i + 1]; i += 2
+        elif a == "--model":
+            opts["model"] = rest[i + 1]; i += 2
+        elif a == "--reference":
+            opts["reference"] = True; i += 1
+        elif a == "--strict":
+            opts["strict"] = True; i += 1
+        elif a == "--refresh-failing-output":
+            opts["refresh_failing_output"] = True; i += 1
+        else:
+            ids.append(a); i += 1
+
+    tier = "all" if cmd == "list" and "--tier" not in sys.argv else opts["tier"]
+    tasks = lib.discover_tasks(ids or None, tier=tier)
+    if not tasks:
+        sys.exit("no tasks matched")
+
+    if cmd == "list":
+        return cmd_list(tasks, opts)
+    if cmd == "validate":
+        return cmd_validate(tasks, opts)
+    if cmd == "run":
+        return cmd_run(tasks, opts)
+    sys.exit(f"unknown command: {cmd}")
+
+
+if __name__ == "__main__":
+    sys.exit(main())
diff --git a/evals/tasks/balance/qbal-001-meadow-wall/bug.patch b/evals/tasks/balance/qbal-001-meadow-wall/bug.patch
new file mode 100644
index 00000000..b22f4829
--- /dev/null
+++ b/evals/tasks/balance/qbal-001-meadow-wall/bug.patch
@@ -0,0 +1,17 @@
+diff --git a/src/core/constants.rs b/src/core/constants.rs
+index b3e8927..ad79eed 100644
+--- a/src/core/constants.rs
++++ b/src/core/constants.rs
+@@ -111,9 +111,9 @@ pub const FRONTIER_BACKOFF_MAX_CYCLES: u32 = 8;
+ // Index 0 = Zone 1, Index 10 = Zone 11 (The Expanse)
+ // hp_step/dmg_step/def_step are per-subzone depth increments above depth 1
+ pub const ZONE_ENEMY_STATS: [(u64, u64, u64, u64, u64, u64); 50] = [
+-    (55, 9, 7, 2, 0, 0),           // Zone 1: Meadow
+-    (90, 14, 13, 3, 2, 1),         // Zone 2: Dark Forest
+-    (160, 22, 22, 4, 6, 2),        // Zone 3: Mountain Pass
++    (550, 90, 40, 12, 0, 0),       // Zone 1: Meadow
++    (900, 140, 75, 18, 6, 3),      // Zone 2: Dark Forest
++    (1600, 220, 130, 24, 18, 6),   // Zone 3: Mountain Pass
+     (215, 27, 31, 6, 10, 3),       // Zone 4: Ancient Ruins
+     (305, 32, 42, 7, 16, 3),       // Zone 5: Volcanic Wastes
+     (380, 40, 53, 8, 22, 4),       // Zone 6: Frozen Tundra
diff --git a/evals/tasks/balance/qbal-001-meadow-wall/failing_output.txt b/evals/tasks/balance/qbal-001-meadow-wall/failing_output.txt
new file mode 100644
index 00000000..a8cce5b5
--- /dev/null
+++ b/evals/tasks/balance/qbal-001-meadow-wall/failing_output.txt
@@ -0,0 +1,69 @@
+$ cargo run --release --bin simulator -- --check-progression
+h (observed 0-1) FAILED: actual=1200, expected LessOrEqual 20
+  ✘ [seed  1] At least 5 achievements in 2h (observed 10-13) FAILED: actual=0, expected GreaterOrEqual 5
+  ✘ [seed  4] Zone 2 reached within 45min (observed ~10-22min) FAILED: actual=18446744073709551616, expected LessOrEqual 27000
+  ✘ [seed  4] Level 20 reached within 1h (observed ~19-23min) FAILED: actual=18446744073709551616, expected LessOrEqual 36000
+  ✘ [seed  4] At least 300 kills in 2h (observed ~650-740) FAILED: actual=0, expected GreaterOrEqual 300
+  ✘ [seed  4] At least 15 boss kills in 2h (observed ~41-48) FAILED: actual=0, expected GreaterOrEqual 15
+  ✘ [seed  4] At least 50 item drops in 2h (observed ~147-179) FAILED: actual=0, expected GreaterOrEqual 50
+  ✘ [seed  4] At most 20 deaths in 2h (observed 0-1) FAILED: actual=1200, expected LessOrEqual 20
+  ✘ [seed  4] At least 5 achievements in 2h (observed 10-13) FAILED: actual=0, expected GreaterOrEqual 5
+  ✘ [seed  7] Zone 2 reached within 45min (observed ~10-22min) FAILED: actual=18446744073709551616, expected LessOrEqual 27000
+  ✘ [seed  7] Level 20 reached within 1h (observed ~19-23min) FAILED: actual=18446744073709551616, expected LessOrEqual 36000
+  ✘ [seed  7] At least 300 kills in 2h (observed ~650-740) FAILED: actual=0, expected GreaterOrEqual 300
+  ✘ [seed  7] At least 15 boss kills in 2h (observed ~41-48) FAILED: actual=0, expected GreaterOrEqual 15
+  ✘ [seed  7] At least 50 item drops in 2h (observed ~147-179) FAILED: actual=0, expected GreaterOrEqual 50
+  ✘ [seed  7] At most 20 deaths in 2h (observed 0-1) FAILED: actual=1200, expected LessOrEqual 20
+  ✘ [seed  7] At least 5 achievements in 2h (observed 10-13) FAILED: actual=0, expected GreaterOrEqual 5
+  ✘ [seed 10] Zone 2 reached within 45min (observed ~10-22min) FAILED: actual=18446744073709551616, expected LessOrEqual 27000
+  ✘ [seed 10] Level 20 reached within 1h (observed ~19-23min) FAILED: actual=18446744073709551616, expected LessOrEqual 36000
+  ✘ [seed 10] At least 300 kills in 2h (observed ~650-740) FAILED: actual=0, expected GreaterOrEqual 300
+  ✘ [seed 10] At least 15 boss kills in 2h (observed ~41-48) FAILED: actual=0, expected GreaterOrEqual 15
+  ✘ [seed 10] At least 50 item drops in 2h (observed ~147-179) FAILED: actual=0, expected GreaterOrEqual 50
+  ✘ [seed 10] At most 20 deaths in 2h (observed 0-1) FAILED: actual=1200, expected LessOrEqual 20
+  ✘ [seed 10] At least 5 achievements in 2h (observed 10-13) FAILED: actual=0, expected GreaterOrEqual 5
+  ✘ [seed 13] Zone 2 reached within 45min (observed ~10-22min) FAILED: actual=18446744073709551616, expected LessOrEqual 27000
+  ✘ [seed 13] Level 20 reached within 1h (observed ~19-23min) FAILED: actual=18446744073709551616, expected LessOrEqual 36000
+  ✘ [seed 13] At least 300 kills in 2h (observed ~650-740) FAILED: actual=0, expected GreaterOrEqual 300
+  ✘ [seed 13] At least 15 boss kills in 2h (observed ~41-48) FAILED: actual=0, expected GreaterOrEqual 15
+  ✘ [seed 13] At least 50 item drops in 2h (observed ~147-179) FAILED: actual=0, expected GreaterOrEqual 50
+  ✘ [seed 13] At most 20 deaths in 2h (observed 0-1) FAILED: actual=1200, expected LessOrEqual 20
+  ✘ [seed 13] At least 5 achievements in 2h (observed 10-13) FAILED: actual=0, expected GreaterOrEqual 5
+
+--- prestige-economy (P0 hero, optimal strategy, 6h, 216000 ticks = 6h 00m 00s, 3 seed(s)) ---
+  ✘ [seed  1] Auto-prestige fires 3+ times in 6h (observed ~14-15) FAILED: actual=0, expected GreaterOrEqual 3
+  ✔ [seed  1] At least 15 PR earned in 6h (observed 28-46) (actual=17)
+  ✔ [seed  1] PR income covers PR spending from P0 (actual=0)
+  ✔ [seed  1] At least 8 challenges won in 6h (observed ~17) (actual=17)
+  ✔ [seed  1] At least 6 Haven rooms built in 6h (observed ~15) (actual=10)
+  ✘ [seed  1] At least 12 achievements in 6h (observed ~20) FAILED: actual=5, expected GreaterOrEqual 12
+  ✘ [seed  5] Auto-prestige fires 3+ times in 6h (observed ~14-15) FAILED: actual=0, expected GreaterOrEqual 3
+  ✔ [seed  5] At least 15 PR earned in 6h (observed 28-46) (actual=17)
+  ✔ [seed  5] PR income covers PR spending from P0 (actual=0)
+  ✔ [seed  5] At least 8 challenges won in 6h (observed ~17) (actual=17)
+  ✔ [seed  5] At least 6 Haven rooms built in 6h (observed ~15) (actual=10)
+  ✘ [seed  5] At least 12 achievements in 6h (observed ~20) FAILED: actual=5, expected GreaterOrEqual 12
+  ✘ [seed  9] Auto-prestige fires 3+ times in 6h (observed ~14-15) FAILED: actual=0, expected GreaterOrEqual 3
+  ✔ [seed  9] At least 15 PR earned in 6h (observed 28-46) (actual=17)
+  ✔ [seed  9] PR income covers PR spending from P0 (actual=0)
+  ✔ [seed  9] At least 8 challenges won in 6h (observed ~17) (actual=17)
+  ✔ [seed  9] At least 6 Haven rooms built in 6h (observed ~15) (actual=10)
+  ✘ [seed  9] At least 12 achievements in 6h (observed ~20) FAILED: actual=5, expected GreaterOrEqual 12
+
+--- endgame-systems (P200 hero, speedrun strategy, 30h, 1080000 ticks = 30h 00m 00s, 2 seed(s)) ---
+  ✔ [seed  1] Deep reaches layer 25+ in 30h (observed 30) (actual=30)
+  ✔ [seed  1] Fracture zone cap reaches Z27+ (observed Z30) (actual=30)
+  ✔ [seed  1] At least 4 Loom patterns completed (observed 6) (actual=6)
+  ✔ [seed  1] Loom zone cap reaches Z31+ (observed Z34) (actual=34)
+  ✔ [seed  1] Ascension I+ reached (observed II) (actual=2)
+  ✔ [seed  1] Pushes into fracture zones Z13+ (observed Z16-17) (actual=17)
+  ✔ [seed  1] Reaches level 100+ (observed ~370-420) (actual=1092)
+  ✔ [seed  9] Deep reaches layer 25+ in 30h (observed 30) (actual=30)
+  ✔ [seed  9] Fracture zone cap reaches Z27+ (observed Z30) (actual=30)
+  ✔ [seed  9] At least 4 Loom patterns completed (observed 6) (actual=6)
+  ✔ [seed  9] Loom zone cap reaches Z31+ (observed Z34) (actual=34)
+  ✔ [seed  9] Ascension I+ reached (observed II) (actual=2)
+  ✔ [seed  9] Pushes into fracture zones Z13+ (observed Z16-17) (actual=17)
+  ✔ [seed  9] Reaches level 100+ (observed ~370-420) (actual=1021)
+
+Progression check FAILED — the game no longer progresses as expected.
\ No newline at end of file
diff --git a/evals/tasks/balance/qbal-001-meadow-wall/prompt.md b/evals/tasks/balance/qbal-001-meadow-wall/prompt.md
new file mode 100644
index 00000000..e8ca5b99
--- /dev/null
+++ b/evals/tasks/balance/qbal-001-meadow-wall/prompt.md
@@ -0,0 +1,20 @@
+An early-game difficulty retune overshot badly. New players report being
+stuck in the first zones: ordinary Meadow mobs take dozens of hits to
+kill while two-shotting a fresh hero, so first-session progress has
+effectively stopped. The CI progression gate is red on its early-game
+scenario (a fresh P0 hero simulated for 2 hours across several seeds) —
+output attached.
+
+Intended early-game pacing, per the design baseline:
+
+- A fresh hero clears into Zone 2 well within the first 45 minutes and
+  reaches Level 20 within the first hour.
+- Early zones are a gentle on-ramp; the first real difficulty wall is
+  Zone 11 (The Expanse), which must stay a wall (~6x the stats of
+  Zone 10) for endgame pacing.
+- Mid zones (4-10) were not part of the retune and their pacing is
+  currently considered correct.
+
+Restore early-game pacing so the progression gate passes again, keeping
+the documented shape of the zone difficulty curve (smooth ramp through
+Zones 1-10, hard wall at 11).
diff --git a/evals/tasks/balance/qbal-001-meadow-wall/reference.patch b/evals/tasks/balance/qbal-001-meadow-wall/reference.patch
new file mode 100644
index 00000000..0cf5e6c3
--- /dev/null
+++ b/evals/tasks/balance/qbal-001-meadow-wall/reference.patch
@@ -0,0 +1,17 @@
+diff --git b/src/core/constants.rs a/src/core/constants.rs
+index ad79eed..b3e8927 100644
+--- b/src/core/constants.rs
++++ a/src/core/constants.rs
+@@ -111,9 +111,9 @@ pub const FRONTIER_BACKOFF_MAX_CYCLES: u32 = 8;
+ // Index 0 = Zone 1, Index 10 = Zone 11 (The Expanse)
+ // hp_step/dmg_step/def_step are per-subzone depth increments above depth 1
+ pub const ZONE_ENEMY_STATS: [(u64, u64, u64, u64, u64, u64); 50] = [
+-    (550, 90, 40, 12, 0, 0),       // Zone 1: Meadow
+-    (900, 140, 75, 18, 6, 3),      // Zone 2: Dark Forest
+-    (1600, 220, 130, 24, 18, 6),   // Zone 3: Mountain Pass
++    (55, 9, 7, 2, 0, 0),           // Zone 1: Meadow
++    (90, 14, 13, 3, 2, 1),         // Zone 2: Dark Forest
++    (160, 22, 22, 4, 6, 2),        // Zone 3: Mountain Pass
+     (215, 27, 31, 6, 10, 3),       // Zone 4: Ancient Ruins
+     (305, 32, 42, 7, 16, 3),       // Zone 5: Volcanic Wastes
+     (380, 40, 53, 8, 22, 4),       // Zone 6: Frozen Tundra
diff --git a/evals/tasks/balance/qbal-001-meadow-wall/task.toml b/evals/tasks/balance/qbal-001-meadow-wall/task.toml
new file mode 100644
index 00000000..0db7fb72
--- /dev/null
+++ b/evals/tasks/balance/qbal-001-meadow-wall/task.toml
@@ -0,0 +1,9 @@
+id = "qbal-001-meadow-wall"
+domain = "balance"
+tier = "slow"
+source = "hand-authored"
+context_files = ["src/core/constants.rs"]
+graders = ["cargo run --release --bin simulator -- --check-progression"]
+edit_allowlist = ["src/core/constants.rs"]
+judge_rubric = "balance"
+show_failing_output = true
diff --git a/evals/tasks/bugfix/qb-001-armory-stacking/bug.patch b/evals/tasks/bugfix/qb-001-armory-stacking/bug.patch
new file mode 100644
index 00000000..3e85f7af
--- /dev/null
+++ b/evals/tasks/bugfix/qb-001-armory-stacking/bug.patch
@@ -0,0 +1,21 @@
+diff --git a/src/combat/player_attack.rs b/src/combat/player_attack.rs
+index f794535..5f76bf9 100644
+--- a/src/combat/player_attack.rs
++++ b/src/combat/player_attack.rs
+@@ -45,12 +45,10 @@ pub(crate) fn resolve_player_attack<R: Rng>(
+     // Player attacks normally
+     // 1. Base damage from DerivedStats (STR/INT + equipment)
+     let base_damage = derived.total_damage();
+-    // 1b. Apply early damage percent (e.g. Giant's Might): +% base damage
+-    let early_boosted_damage =
+-        (base_damage as f64 * (1.0 + bonuses.early_damage_percent / 100.0)) as u64;
+-    // 2. Apply damage percent multiplier (e.g. Haven Armory): +% damage
+-    let boosted_damage =
+-        (early_boosted_damage as f64 * (1.0 + bonuses.damage_percent / 100.0)) as u64;
++    // 2. Apply damage percent bonuses (e.g. Giant's Might, Haven Armory): +% damage
++    let boosted_damage = (base_damage as f64
++        * (1.0 + (bonuses.early_damage_percent + bonuses.damage_percent) / 100.0))
++        as u64;
+     // 3. Apply flat damage bonus (e.g. prestige), added after % bonuses, before defense
+     let pre_defense_damage = boosted_damage.saturating_add(bonuses.flat_damage);
+     // 4. Apply Ascension multiplier to damage
diff --git a/evals/tasks/bugfix/qb-001-armory-stacking/failing_output.txt b/evals/tasks/bugfix/qb-001-armory-stacking/failing_output.txt
new file mode 100644
index 00000000..3d56250f
--- /dev/null
+++ b/evals/tasks/bugfix/qb-001-armory-stacking/failing_output.txt
@@ -0,0 +1,36 @@
+$ cargo test --test combat_tests
+   Compiling quest v0.1.0 (/home/user/quest/.worktrees/eval-scratch)
+    Finished `test` profile [unoptimized + debuginfo] target(s) in 9.27s
+     Running tests/combat_tests.rs (/home/user/quest/target/debug/deps/combat_tests-45df4f082fe8f91d)
+
+running 101 tests
+test combat_tests::combat_damage_pipeline_test::giants_might_and_armory_stack_multiplicatively ... FAILED
+
+failures:
+
+---- combat_tests::combat_damage_pipeline_test::giants_might_and_armory_stack_multiplicatively stdout ----
+
+thread 'combat_tests::combat_damage_pipeline_test::giants_might_and_armory_stack_multiplicatively' (17266) panicked at tests/combat_tests/combat_damage_pipeline_test.rs:337:5:
+Combined Giants Might + Armory should be ~5x, got ratio 3.50
+stack backtrace:
+   0: __rustc::rust_begin_unwind
+             at /rustc/e408947bfd200af42db322daf0fadfe7e26d3bd1/library/std/src/panicking.rs:689:5
+   1: core::panicking::panic_fmt
+             at /rustc/e408947bfd200af42db322daf0fadfe7e26d3bd1/library/core/src/panicking.rs:80:14
+   2: combat_tests::combat_tests::combat_damage_pipeline_test::giants_might_and_armory_stack_multiplicatively
+             at ./tests/combat_tests/combat_damage_pipeline_test.rs:337:5
+   3: combat_tests::combat_tests::combat_damage_pipeline_test::giants_might_and_armory_stack_multiplicatively::{{closure}}
+             at ./tests/combat_tests/combat_damage_pipeline_test.rs:310:52
+   4: core::ops::function::FnOnce::call_once
+             at /rustc/e408947bfd200af42db322daf0fadfe7e26d3bd1/library/core/src/ops/function.rs:250:5
+   5: core::ops::function::FnOnce::call_once
+             at /rustc/e408947bfd200af42db322daf0fadfe7e26d3bd1/library/core/src/ops/function.rs:250:5
+note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
+
+
+failures:
+    combat_tests::combat_damage_pipeline_test::giants_might_and_armory_stack_multiplicatively
+
+test result: FAILED. 100 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.24s
+
+error: test failed, to rerun pass `--test combat_tests`
\ No newline at end of file
diff --git a/evals/tasks/bugfix/qb-001-armory-stacking/prompt.md b/evals/tasks/bugfix/qb-001-armory-stacking/prompt.md
new file mode 100644
index 00000000..0d55a58c
--- /dev/null
+++ b/evals/tasks/bugfix/qb-001-armory-stacking/prompt.md
@@ -0,0 +1,18 @@
+Players who have both Giant's Might (a god item granting +150% early damage)
+and a maxed Haven Armory (+100% damage) report that their damage dropped
+after a recent refactor. With both bonuses active they now hit for about
+3.5x base damage; before the refactor — and per the documented combat
+pipeline — the two bonuses are supposed to compound to 5x.
+
+The documented player damage pipeline is:
+
+    base damage -> Giant's Might % (early_damage_percent)
+                -> Haven Armory % (damage_percent)
+                -> prestige flat damage -> ascension multiplier
+                -> enemy defense -> min 1 -> crit (2x)
+
+Each bonus on its own appears to work (150% alone gives 2.5x, 100% alone
+gives 2x). Only the combination is wrong.
+
+Fix the damage calculation so the two percentage bonuses compound the way
+the pipeline documents.
diff --git a/evals/tasks/bugfix/qb-001-armory-stacking/reference.patch b/evals/tasks/bugfix/qb-001-armory-stacking/reference.patch
new file mode 100644
index 00000000..35b37b45
--- /dev/null
+++ b/evals/tasks/bugfix/qb-001-armory-stacking/reference.patch
@@ -0,0 +1,21 @@
+diff --git b/src/combat/player_attack.rs a/src/combat/player_attack.rs
+index 5f76bf9..f794535 100644
+--- b/src/combat/player_attack.rs
++++ a/src/combat/player_attack.rs
+@@ -45,10 +45,12 @@ pub(crate) fn resolve_player_attack<R: Rng>(
+     // Player attacks normally
+     // 1. Base damage from DerivedStats (STR/INT + equipment)
+     let base_damage = derived.total_damage();
+-    // 2. Apply damage percent bonuses (e.g. Giant's Might, Haven Armory): +% damage
+-    let boosted_damage = (base_damage as f64
+-        * (1.0 + (bonuses.early_damage_percent + bonuses.damage_percent) / 100.0))
+-        as u64;
++    // 1b. Apply early damage percent (e.g. Giant's Might): +% base damage
++    let early_boosted_damage =
++        (base_damage as f64 * (1.0 + bonuses.early_damage_percent / 100.0)) as u64;
++    // 2. Apply damage percent multiplier (e.g. Haven Armory): +% damage
++    let boosted_damage =
++        (early_boosted_damage as f64 * (1.0 + bonuses.damage_percent / 100.0)) as u64;
+     // 3. Apply flat damage bonus (e.g. prestige), added after % bonuses, before defense
+     let pre_defense_damage = boosted_damage.saturating_add(bonuses.flat_damage);
+     // 4. Apply Ascension multiplier to damage
diff --git a/evals/tasks/bugfix/qb-001-armory-stacking/task.toml b/evals/tasks/bugfix/qb-001-armory-stacking/task.toml
new file mode 100644
index 00000000..f344e32f
--- /dev/null
+++ b/evals/tasks/bugfix/qb-001-armory-stacking/task.toml
@@ -0,0 +1,9 @@
+id = "qb-001-armory-stacking"
+domain = "bugfix"
+tier = "fast"
+source = "hand-authored"
+context_files = ["src/combat/player_attack.rs", "src/combat/events.rs"]
+graders = ["cargo test --test combat_tests"]
+edit_allowlist = ["src/combat/"]
+judge_rubric = "bugfix"
+show_failing_output = true
diff --git a/evals/tasks/bugfix/qb-002-save-wipe/bug.patch b/evals/tasks/bugfix/qb-002-save-wipe/bug.patch
new file mode 100644
index 00000000..c10db663
--- /dev/null
+++ b/evals/tasks/bugfix/qb-002-save-wipe/bug.patch
@@ -0,0 +1,12 @@
+diff --git a/src/character/manager.rs b/src/character/manager.rs
+index 27d90e6..5d25436 100644
+--- a/src/character/manager.rs
++++ b/src/character/manager.rs
+@@ -37,7 +37,6 @@ pub(super) struct CharacterSaveData {
+     pub(super) ascension_level: u32,
+     #[serde(default)]
+     pub(super) vessel_signal_discovered: bool,
+-    #[serde(default)]
+     pub(super) vessel_launched: bool,
+     #[serde(default)]
+     pub(super) vessel_arrived: bool,
diff --git a/evals/tasks/bugfix/qb-002-save-wipe/failing_output.txt b/evals/tasks/bugfix/qb-002-save-wipe/failing_output.txt
new file mode 100644
index 00000000..d2471a55
--- /dev/null
+++ b/evals/tasks/bugfix/qb-002-save-wipe/failing_output.txt
@@ -0,0 +1,67 @@
+$ cargo test --test save_compat_tests
+   Compiling quest v0.1.0 (/home/user/quest/.worktrees/eval-scratch)
+    Finished `test` profile [unoptimized + debuginfo] target(s) in 9.91s
+     Running tests/save_compat_tests.rs (/home/user/quest/target/debug/deps/save_compat_tests-2fe3456fee849e6e)
+
+running 13 tests
+test generate_vessel_corpus_v1_addition ... ignored
+test regenerate_save_corpus ... ignored
+test character_corpus_loads ... FAILED
+test character_corpus_round_trips ... FAILED
+
+failures:
+
+---- character_corpus_loads stdout ----
+
+thread 'character_corpus_loads' (20398) panicked at tests/save_compat_tests.rs:59:54:
+called `Result::unwrap()` on an `Err` value: Custom { kind: InvalidData, error: Error("missing field `vessel_launched`", line: 70, column: 1) }
+stack backtrace:
+   0: __rustc::rust_begin_unwind
+             at /rustc/e408947bfd200af42db322daf0fadfe7e26d3bd1/library/std/src/panicking.rs:689:5
+   1: core::panicking::panic_fmt
+             at /rustc/e408947bfd200af42db322daf0fadfe7e26d3bd1/library/core/src/panicking.rs:80:14
+   2: core::result::unwrap_failed
+             at /rustc/e408947bfd200af42db322daf0fadfe7e26d3bd1/library/core/src/result.rs:1867:5
+   3: unwrap<quest::core::game_state::GameState, std::io::error::Error>
+             at /rustc/e408947bfd200af42db322daf0fadfe7e26d3bd1/library/core/src/result.rs:1233:23
+   4: character_corpus_loads
+             at ./tests/save_compat_tests.rs:59:54
+   5: save_compat_tests::character_corpus_loads::{{closure}}
+             at ./tests/save_compat_tests.rs:56:28
+   6: core::ops::function::FnOnce::call_once
+             at /rustc/e408947bfd200af42db322daf0fadfe7e26d3bd1/library/core/src/ops/function.rs:250:5
+   7: core::ops::function::FnOnce::call_once
+             at /rustc/e408947bfd200af42db322daf0fadfe7e26d3bd1/library/core/src/ops/function.rs:250:5
+note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
+
+---- character_corpus_round_trips stdout ----
+
+thread 'character_corpus_round_trips' (20399) panicked at tests/save_compat_tests.rs:132:53:
+called `Result::unwrap()` on an `Err` value: Custom { kind: InvalidData, error: Error("missing field `vessel_launched`", line: 70, column: 1) }
+stack backtrace:
+   0: __rustc::rust_begin_unwind
+             at /rustc/e408947bfd200af42db322daf0fadfe7e26d3bd1/library/std/src/panicking.rs:689:5
+   1: core::panicking::panic_fmt
+             at /rustc/e408947bfd200af42db322daf0fadfe7e26d3bd1/library/core/src/panicking.rs:80:14
+   2: core::result::unwrap_failed
+             at /rustc/e408947bfd200af42db322daf0fadfe7e26d3bd1/library/core/src/result.rs:1867:5
+   3: unwrap<quest::core::game_state::GameState, std::io::error::Error>
+             at /rustc/e408947bfd200af42db322daf0fadfe7e26d3bd1/library/core/src/result.rs:1233:23
+   4: character_corpus_round_trips
+             at ./tests/save_compat_tests.rs:132:53
+   5: save_compat_tests::character_corpus_round_trips::{{closure}}
+             at ./tests/save_compat_tests.rs:120:34
+   6: core::ops::function::FnOnce::call_once
+             at /rustc/e408947bfd200af42db322daf0fadfe7e26d3bd1/library/core/src/ops/function.rs:250:5
+   7: core::ops::function::FnOnce::call_once
+             at /rustc/e408947bfd200af42db322daf0fadfe7e26d3bd1/library/core/src/ops/function.rs:250:5
+note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
+
+
+failures:
+    character_corpus_loads
+    character_corpus_round_trips
+
+test result: FAILED. 9 passed; 2 failed; 2 ignored; 0 measured; 0 filtered out; finished in 0.22s
+
+error: test failed, to rerun pass `--test save_compat_tests`
\ No newline at end of file
diff --git a/evals/tasks/bugfix/qb-002-save-wipe/prompt.md b/evals/tasks/bugfix/qb-002-save-wipe/prompt.md
new file mode 100644
index 00000000..8e25acb5
--- /dev/null
+++ b/evals/tasks/bugfix/qb-002-save-wipe/prompt.md
@@ -0,0 +1,20 @@
+A player upgraded Quest from a version released before Act 2 shipped and
+their high-prestige character was silently replaced by a fresh Level 1
+character. Nothing crashed; the character just came up empty.
+
+Context on how loading works in this codebase: the account/character
+loaders fall back to a default state when JSON parsing fails, so a
+deserialization error never surfaces as a crash — it silently wipes the
+player's progress. The committed save corpus (`tests/fixtures/saves/v1/`)
+exists precisely to catch this: those files are real saves written by
+older versions of the game, frozen forever, and a corpus test failure
+means existing player saves no longer load.
+
+QA bisected the regression to a recent "attribute cleanup" in the
+character persistence code. Old save files — which predate several fields
+that newer versions write — no longer deserialize.
+
+Fix the persistence code so that saves written by pre-Act 2 versions load
+correctly again. Per this repo's conventions, save-format compatibility is
+fixed in the type definitions (defaults/aliases/migrations), never by
+editing the frozen corpus files.
diff --git a/evals/tasks/bugfix/qb-002-save-wipe/reference.patch b/evals/tasks/bugfix/qb-002-save-wipe/reference.patch
new file mode 100644
index 00000000..7375dc85
--- /dev/null
+++ b/evals/tasks/bugfix/qb-002-save-wipe/reference.patch
@@ -0,0 +1,12 @@
+diff --git b/src/character/manager.rs a/src/character/manager.rs
+index 5d25436..27d90e6 100644
+--- b/src/character/manager.rs
++++ a/src/character/manager.rs
+@@ -37,6 +37,7 @@ pub(super) struct CharacterSaveData {
+     pub(super) ascension_level: u32,
+     #[serde(default)]
+     pub(super) vessel_signal_discovered: bool,
++    #[serde(default)]
+     pub(super) vessel_launched: bool,
+     #[serde(default)]
+     pub(super) vessel_arrived: bool,
diff --git a/evals/tasks/bugfix/qb-002-save-wipe/task.toml b/evals/tasks/bugfix/qb-002-save-wipe/task.toml
new file mode 100644
index 00000000..cec09be3
--- /dev/null
+++ b/evals/tasks/bugfix/qb-002-save-wipe/task.toml
@@ -0,0 +1,9 @@
+id = "qb-002-save-wipe"
+domain = "bugfix"
+tier = "fast"
+source = "hand-authored"
+context_files = ["src/character/manager.rs"]
+graders = ["cargo test --test save_compat_tests"]
+edit_allowlist = ["src/character/"]
+judge_rubric = "bugfix"
+show_failing_output = true
diff --git a/evals/tasks/feature/qf-001-shift-hotkeys/bug.patch b/evals/tasks/feature/qf-001-shift-hotkeys/bug.patch
new file mode 100644
index 00000000..8b1efb69
--- /dev/null
+++ b/evals/tasks/feature/qf-001-shift-hotkeys/bug.patch
@@ -0,0 +1,13 @@
+diff --git a/src/input/mod.rs b/src/input/mod.rs
+index ee6a374..c2c0b65 100644
+--- a/src/input/mod.rs
++++ b/src/input/mod.rs
+@@ -678,7 +678,7 @@ fn handle_base_game(
+             }
+             InputResult::Continue
+         }
+-        KeyCode::Char('h') | KeyCode::Char('H') => {
++        KeyCode::Char('h') => {
+             if haven.discovered {
+                 haven_ui.open();
+             }
diff --git a/evals/tasks/feature/qf-001-shift-hotkeys/failing_output.txt b/evals/tasks/feature/qf-001-shift-hotkeys/failing_output.txt
new file mode 100644
index 00000000..c66c1ef1
--- /dev/null
+++ b/evals/tasks/feature/qf-001-shift-hotkeys/failing_output.txt
@@ -0,0 +1,36 @@
+$ cargo test --bin quest input::replay_tests
+   Compiling quest v0.1.0 (/home/user/quest/.worktrees/eval-scratch)
+    Finished `test` profile [unoptimized + debuginfo] target(s) in 39.52s
+     Running unittests src/main.rs (/home/user/quest/target/debug/deps/quest-161316338b202bbd)
+
+running 46 tests
+test input::replay_tests::uppercase_hotkey_variant_also_opens_overlay ... FAILED
+
+failures:
+
+---- input::replay_tests::uppercase_hotkey_variant_also_opens_overlay stdout ----
+
+thread 'input::replay_tests::uppercase_hotkey_variant_also_opens_overlay' (22160) panicked at src/input/replay_tests.rs:55:5:
+[Shift+H] should open the Haven too
+stack backtrace:
+   0: __rustc::rust_begin_unwind
+             at /rustc/e408947bfd200af42db322daf0fadfe7e26d3bd1/library/std/src/panicking.rs:689:5
+   1: core::panicking::panic_fmt
+             at /rustc/e408947bfd200af42db322daf0fadfe7e26d3bd1/library/core/src/panicking.rs:80:14
+   2: quest::input::replay_tests::uppercase_hotkey_variant_also_opens_overlay
+             at ./src/input/replay_tests.rs:55:5
+   3: quest::input::replay_tests::uppercase_hotkey_variant_also_opens_overlay::{{closure}}
+             at ./src/input/replay_tests.rs:52:49
+   4: core::ops::function::FnOnce::call_once
+             at /rustc/e408947bfd200af42db322daf0fadfe7e26d3bd1/library/core/src/ops/function.rs:250:5
+   5: core::ops::function::FnOnce::call_once
+             at /rustc/e408947bfd200af42db322daf0fadfe7e26d3bd1/library/core/src/ops/function.rs:250:5
+note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
+
+
+failures:
+    input::replay_tests::uppercase_hotkey_variant_also_opens_overlay
+
+test result: FAILED. 45 passed; 1 failed; 0 ignored; 0 measured; 3027 filtered out; finished in 0.20s
+
+error: test failed, to rerun pass `--bin quest`
\ No newline at end of file
diff --git a/evals/tasks/feature/qf-001-shift-hotkeys/prompt.md b/evals/tasks/feature/qf-001-shift-hotkeys/prompt.md
new file mode 100644
index 00000000..1a03bd7b
--- /dev/null
+++ b/evals/tasks/feature/qf-001-shift-hotkeys/prompt.md
@@ -0,0 +1,17 @@
+Quest's base-game hotkeys are letters ([P]restige, [H]aven, [S]oulforge,
+[G] Stormglass, [D]eep, and so on). The UX rule for this screen is that
+every letter hotkey must also accept its shifted variant — players with
+Caps Lock on, or who habitually hold Shift, should get the same behavior
+as the lowercase press.
+
+Currently, pressing Shift+H with the Haven discovered does nothing, while
+`h` opens the Haven overlay as expected. Every other overlay hotkey on the
+base screen honors its uppercase variant; the Haven hotkey is the odd one
+out.
+
+Make the Haven hotkey accept the shifted variant like the rest, following
+the same pattern the neighboring hotkeys use. The behavior must remain
+gated on Haven discovery exactly as the lowercase path is.
+
+The input replay test `uppercase_hotkey_variant_also_opens_overlay`
+(headless harness driving the real input dispatcher) covers this behavior.
diff --git a/evals/tasks/feature/qf-001-shift-hotkeys/reference.patch b/evals/tasks/feature/qf-001-shift-hotkeys/reference.patch
new file mode 100644
index 00000000..93560bdf
--- /dev/null
+++ b/evals/tasks/feature/qf-001-shift-hotkeys/reference.patch
@@ -0,0 +1,13 @@
+diff --git b/src/input/mod.rs a/src/input/mod.rs
+index c2c0b65..ee6a374 100644
+--- b/src/input/mod.rs
++++ a/src/input/mod.rs
+@@ -678,7 +678,7 @@ fn handle_base_game(
+             }
+             InputResult::Continue
+         }
+-        KeyCode::Char('h') => {
++        KeyCode::Char('h') | KeyCode::Char('H') => {
+             if haven.discovered {
+                 haven_ui.open();
+             }
diff --git a/evals/tasks/feature/qf-001-shift-hotkeys/task.toml b/evals/tasks/feature/qf-001-shift-hotkeys/task.toml
new file mode 100644
index 00000000..2e439dcb
--- /dev/null
+++ b/evals/tasks/feature/qf-001-shift-hotkeys/task.toml
@@ -0,0 +1,9 @@
+id = "qf-001-shift-hotkeys"
+domain = "feature"
+tier = "fast"
+source = "hand-authored"
+context_files = ["src/input/mod.rs"]
+graders = ["cargo test --bin quest input::replay_tests"]
+edit_allowlist = ["src/input/"]
+judge_rubric = "feature"
+show_failing_output = true
diff --git a/evals/tasks/ui/qu-001-prestige-gauge/bug.patch b/evals/tasks/ui/qu-001-prestige-gauge/bug.patch
new file mode 100644
index 00000000..2a8d0829
--- /dev/null
+++ b/evals/tasks/ui/qu-001-prestige-gauge/bug.patch
@@ -0,0 +1,16 @@
+diff --git a/src/ui/stats_panel.rs b/src/ui/stats_panel.rs
+index b43a61f..c28006d 100644
+--- a/src/ui/stats_panel.rs
++++ b/src/ui/stats_panel.rs
+@@ -416,8 +416,9 @@ fn draw_hero_panel(
+     frame.render_widget(Paragraph::new(prestige_lines), chunks[2]);
+ 
+     // Prestige gauge
+-    let prestige_ratio =
+-        (game_state.character_level as f64 / next_prestige.required_level as f64).min(1.0);
++    let prestige_ratio = (game_state.character_level.saturating_sub(1) as f64
++        / next_prestige.required_level as f64)
++        .min(1.0);
+     let prestige_eta = match game_state.xp_per_hour() {
+         Some(rate) if rate > 0 && game_state.character_level < next_prestige.required_level => {
+             let xp_remaining_current = xp_for_next_level(game_state.character_level)
diff --git a/evals/tasks/ui/qu-001-prestige-gauge/failing_output.txt b/evals/tasks/ui/qu-001-prestige-gauge/failing_output.txt
new file mode 100644
index 00000000..ba3f1026
--- /dev/null
+++ b/evals/tasks/ui/qu-001-prestige-gauge/failing_output.txt
@@ -0,0 +1,99 @@
+$ cargo test --lib snapshot
+: Reset, modifier: NONE,
+  190   190 │         x: 133, y: 8, fg: DarkGray, bg: Reset, underline: Reset, modifier: NONE,
+────────────┴───────────────────────────────────────────────────────────────────
+Stopped on the first failure. Run `cargo insta test` to run all snapshots.
+
+thread 'ui::snapshot_tests::snapshot_all_minigames' (23802) panicked at /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/insta-1.48.0/src/runtime.rs:719:13:
+snapshot assertion for 'minigame_chess_xl_160x45' failed in line 113
+stack backtrace:
+   0: __rustc::rust_begin_unwind
+             at /rustc/e408947bfd200af42db322daf0fadfe7e26d3bd1/library/std/src/panicking.rs:689:5
+   1: core::panicking::panic_fmt
+             at /rustc/e408947bfd200af42db322daf0fadfe7e26d3bd1/library/core/src/panicking.rs:80:14
+   2: finalize
+             at /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/insta-1.48.0/src/runtime.rs:719:13
+   3: assert_snapshot
+             at /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/insta-1.48.0/src/runtime.rs:947:13
+   4: {closure#0}
+             at ./src/ui/snapshot_tests.rs:113:22
+   5: insta::settings::Settings::bind
+             at /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/insta-1.48.0/src/settings.rs:617:9
+   6: quest::ui::snapshot_tests::assert_scene_snapshot
+             at ./src/ui/snapshot_tests.rs:113:14
+   7: quest::ui::snapshot_tests::assert_frame_snapshot
+             at ./src/ui/snapshot_tests.rs:100:5
+   8: snapshot_all_minigames
+             at ./src/ui/snapshot_tests.rs:187:9
+   9: quest::ui::snapshot_tests::snapshot_all_minigames::{{closure}}
+             at ./src/ui/snapshot_tests.rs:183:28
+  10: core::ops::function::FnOnce::call_once
+             at /rustc/e408947bfd200af42db322daf0fadfe7e26d3bd1/library/core/src/ops/function.rs:250:5
+  11: core::ops::function::FnOnce::call_once
+             at /rustc/e408947bfd200af42db322daf0fadfe7e26d3bd1/library/core/src/ops/function.rs:250:5
+note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
+
+---- ui::snapshot_tests::snapshot_midgame_all_tiers stdout ----
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Snapshot Summary ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+Snapshot file: src/ui/snapshots/midgame_xl_160x45.snap
+Snapshot: midgame_xl_160x45
+Source: src/ui/snapshot_tests.rs:113
+────────────────────────────────────────────────────────────────────────────────
+Expression: frame
+────────────────────────────────────────────────────────────────────────────────
+-old snapshot
++new results
+────────────┬───────────────────────────────────────────────────────────────────
+  213   213 │         x: 80, y: 7, fg: Magenta, bg: Reset, underline: Reset, modifier: NONE,
+  214   214 │         x: 0, y: 8, fg: Reset, bg: Reset, underline: Reset, modifier: NONE,
+  215   215 │         x: 1, y: 8, fg: Rgb(180, 100, 255), bg: Rgb(20, 10, 30), underline: Reset, modifier: BOLD,
+  216   216 │         x: 28, y: 8, fg: Rgb(20, 10, 30), bg: Rgb(180, 100, 255), underline: Reset, modifier: BOLD,
+  217       │-        x: 40, y: 8, fg: Rgb(180, 100, 255), bg: Rgb(20, 10, 30), underline: Reset, modifier: BOLD,
+        217 │+        x: 39, y: 8, fg: Rgb(180, 100, 255), bg: Rgb(20, 10, 30), underline: Reset, modifier: BOLD,
+  218   218 │         x: 79, y: 8, fg: Reset, bg: Reset, underline: Reset, modifier: NONE,
+  219   219 │         x: 80, y: 8, fg: Magenta, bg: Reset, underline: Reset, modifier: NONE,
+  220   220 │         x: 81, y: 8, fg: Reset, bg: Rgb(10, 30, 80), underline: Reset, modifier: NONE,
+  221   221 │         x: 90, y: 8, fg: Reset, bg: Rgb(9, 29, 79), underline: Reset, modifier: NONE,
+────────────┴───────────────────────────────────────────────────────────────────
+Stopped on the first failure. Run `cargo insta test` to run all snapshots.
+
+thread 'ui::snapshot_tests::snapshot_midgame_all_tiers' (23829) panicked at /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/insta-1.48.0/src/runtime.rs:719:13:
+snapshot assertion for 'midgame_xl_160x45' failed in line 113
+stack backtrace:
+   0: __rustc::rust_begin_unwind
+             at /rustc/e408947bfd200af42db322daf0fadfe7e26d3bd1/library/std/src/panicking.rs:689:5
+   1: core::panicking::panic_fmt
+             at /rustc/e408947bfd200af42db322daf0fadfe7e26d3bd1/library/core/src/panicking.rs:80:14
+   2: finalize
+             at /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/insta-1.48.0/src/runtime.rs:719:13
+   3: assert_snapshot
+             at /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/insta-1.48.0/src/runtime.rs:947:13
+   4: {closure#0}
+             at ./src/ui/snapshot_tests.rs:113:22
+   5: insta::settings::Settings::bind
+             at /root/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/insta-1.48.0/src/settings.rs:617:9
+   6: quest::ui::snapshot_tests::assert_scene_snapshot
+             at ./src/ui/snapshot_tests.rs:113:14
+   7: quest::ui::snapshot_tests::assert_frame_snapshot
+             at ./src/ui/snapshot_tests.rs:100:5
+   8: quest::ui::snapshot_tests::snapshot_midgame_all_tiers
+             at ./src/ui/snapshot_tests.rs:133:5
+   9: quest::ui::snapshot_tests::snapshot_midgame_all_tiers::{{closure}}
+             at ./src/ui/snapshot_tests.rs:131:32
+  10: core::ops::function::FnOnce::call_once
+             at /rustc/e408947bfd200af42db322daf0fadfe7e26d3bd1/library/core/src/ops/function.rs:250:5
+  11: core::ops::function::FnOnce::call_once
+             at /rustc/e408947bfd200af42db322daf0fadfe7e26d3bd1/library/core/src/ops/function.rs:250:5
+note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
+
+
+failures:
+    ui::snapshot_tests::snapshot_all_minigames
+    ui::snapshot_tests::snapshot_boss_ready
+    ui::snapshot_tests::snapshot_combat_active
+    ui::snapshot_tests::snapshot_fresh_all_tiers
+    ui::snapshot_tests::snapshot_midgame_all_tiers
+
+test result: FAILED. 40 passed; 5 failed; 0 ignored; 0 measured; 2862 filtered out; finished in 0.52s
+
+error: test failed, to rerun pass `--lib`
\ No newline at end of file
diff --git a/evals/tasks/ui/qu-001-prestige-gauge/prompt.md b/evals/tasks/ui/qu-001-prestige-gauge/prompt.md
new file mode 100644
index 00000000..677013ee
--- /dev/null
+++ b/evals/tasks/ui/qu-001-prestige-gauge/prompt.md
@@ -0,0 +1,17 @@
+The prestige progress gauge in the hero panel (the bar labelled
+`Lv X/Y to <Tier> (P<rank>)`) renders shorter than it should at every
+level. A brand-new Level 1 character shows a completely empty gauge, where
+it used to show a small amount of fill (level 1 of 10 toward Bronze). The
+regression is visible across all terminal size tiers that render the hero
+panel.
+
+The gauge is meant to show plain progress toward the next prestige tier's
+required level: at Level 1 with a Level-10 requirement the bar should be
+10% full, and it should reach 100% exactly when the requirement is met.
+
+The UI snapshot tests (which compare full rendered frames against
+committed reference snapshots) caught the regression — excerpt attached.
+In the diffs, the lines starting with `-` are the expected (correct)
+frames and `+` is what currently renders.
+
+Fix the gauge so it fills to the documented ratio again.
diff --git a/evals/tasks/ui/qu-001-prestige-gauge/reference.patch b/evals/tasks/ui/qu-001-prestige-gauge/reference.patch
new file mode 100644
index 00000000..19fd3d45
--- /dev/null
+++ b/evals/tasks/ui/qu-001-prestige-gauge/reference.patch
@@ -0,0 +1,16 @@
+diff --git b/src/ui/stats_panel.rs a/src/ui/stats_panel.rs
+index c28006d..b43a61f 100644
+--- b/src/ui/stats_panel.rs
++++ a/src/ui/stats_panel.rs
+@@ -416,9 +416,8 @@ fn draw_hero_panel(
+     frame.render_widget(Paragraph::new(prestige_lines), chunks[2]);
+ 
+     // Prestige gauge
+-    let prestige_ratio = (game_state.character_level.saturating_sub(1) as f64
+-        / next_prestige.required_level as f64)
+-        .min(1.0);
++    let prestige_ratio =
++        (game_state.character_level as f64 / next_prestige.required_level as f64).min(1.0);
+     let prestige_eta = match game_state.xp_per_hour() {
+         Some(rate) if rate > 0 && game_state.character_level < next_prestige.required_level => {
+             let xp_remaining_current = xp_for_next_level(game_state.character_level)
diff --git a/evals/tasks/ui/qu-001-prestige-gauge/task.toml b/evals/tasks/ui/qu-001-prestige-gauge/task.toml
new file mode 100644
index 00000000..452378b4
--- /dev/null
+++ b/evals/tasks/ui/qu-001-prestige-gauge/task.toml
@@ -0,0 +1,9 @@
+id = "qu-001-prestige-gauge"
+domain = "ui"
+tier = "fast"
+source = "hand-authored"
+context_files = ["src/ui/stats_panel.rs"]
+graders = ["cargo test --lib snapshot"]
+edit_allowlist = ["src/ui/"]
+judge_rubric = "ui"
+show_failing_output = true