M34 — EfficientCube: shorter solutions via search-width retune (measured, not more training)#26
Open
PieterjanDeClippel wants to merge 5 commits into
Open
M34 — EfficientCube: shorter solutions via search-width retune (measured, not more training)#26PieterjanDeClippel wants to merge 5 commits into
PieterjanDeClippel wants to merge 5 commits into
Conversation
… not training) Solve-rate is saturated (100% beam d4→d26); the honest levers are solution length (optimality) and search cost. Adds CUBE_SOLVER_IMPROVE_PRD.md + PLAN.md M34 stub. Gated, zero-retraining-first: W1 instrument length, W2 beam-width sweep, W3 value-guided beam, W4 (optional) more training. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…y probe
Solve-rate is saturated, so the metric that can move is solution length. The
eval already computed beamLen but only printed it; surface it as CSV metrics
and add a ground-truth optimality probe.
- Evaluate(): emit d{depth}_beamlen + d{depth}_slack (beamlen - scramble depth)
- CubePolicyLab: route --eval-only output to cube-policy-eval.csv so the wider
metric set can't misalign the training log's header
- --optimal-probe (TryRunStandaloneEval): beam vs BreadthFirstPlanner optimum
at BFS-tractable depths 1-6; reports provably-optimal fraction (criterion C)
- extract BuildBeamForward() helper (shared by eval + probe)
Baseline of the shipped 346.8M net (beam 2000, RTX 3060): 10/10 solve d4->d26;
100% provably QTM-optimal d1-6; slack ~0 through d12 then +4-6 qt at d14-d18 —
the mid-depth band is the concrete W2/W3 headroom.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ost) Add --beam-sweep w1,w2,... eval-only mode: re-runs the depth eval per width and writes a self-describing cube-policy-sweep.csv. Extract a shared EvaluateDepths(logits, width, includeGreedy) helper that also tallies mean expansions (net-forward count = machine-independent search-cost proxy). Sweep of the shipped net (10 cubes/depth): - width >=1000 already solves 100% d4->d26; beam 2000 (shipped) is conservative -> dropping web to 1000 ~halves cost at tiny length cost (criterion B, latency) - wider = shorter mid-depth: d14 17.8->14.4qt at b5000 (~optimal), d16-d22 -1-2qt, at ~2.4x expansions (criterion A, quality) Frames W3: can value-guidance at beam 2000 buy beam-5000 lengths at beam-2000 cost? Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The trained value head is dropped by PolicyAsMlp() and unused by the beam. Wire it in and test whether it shortens solutions. - CubePolicyNet.PolicyAndValueAsMlp(): combined [324,h,h,13] head so one resident forward yields logits + value (index 12) - CubePolicySearch.BeamSearchValueGuided(...): rank by cumLogProb - lambda*relu(value); forwards all candidate children (value heuristic needs each child's own value), reuses their logits next step. lambda=0 reduces to pure-policy ranking. - --value-sweep lambda1,lambda2,... eval mode -> cube-policy-value.csv - tests: combined head reproduces net logits+value; lambda=0 == pure-policy length Result: value guidance shortens solutions monotonically (lambda=8: d18 -2.4, d22 -2.6qt; fixed d16 9/10->10/10) BUT loses at matched compute — the ~10x child-forwarding overhead means plain beam-widening is 3-11x more efficient for the same length (d18 24.0qt: vg-500-l8 @110k exp vs pure-2000 @39k). Signal real but weak -> not worth shipping. The web win is W2's beam-width knob. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…solutions) The M34 W2 sweep showed width 5000 near-optimalizes mid-depth solutions (d14 17.8->14.4qt, d16-d22 -1-2qt) over the shipped 2000, at ~2.4x the search. Owner chose quality over latency. Beam stays pure-policy (W3 value-guidance was worse per-compute). Endpoint contract + Kociemba reference unchanged. Also marks M34 complete: W4 (more training) skipped — not indicated (model is optimal wherever verifiable; gap is search-bound, loss sample-bound). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PieterjanDeClippel
added a commit
that referenced
this pull request
Jul 13, 2026
… (fixed upstream) MintPlayer.Polyglot.MSBuild 0.6.0 (PR #26) ships the stamp-Outputs + RemoveDir fix for the multi-.pg incremental-transpile bug that was diagnosed and verified here during M40.1. With it upstream, the temporary local _PolyglotForceFullRetranspile target and the MintPlayer.Polyglot.MSBuild.targets.FIXED drop-in are no longer needed — both removed. Verified on 0.6.0 with NO local workaround: clean build green; the incremental single-.pg touch that used to break (CS0101/CS0260) now rebuilds clean; no-op rebuild up-to-date; full suite 362/362, chess perft 25/25. Docs reconciled: CLAUDE.md's "known build failure" note marked fixed in 0.6.0; PLAN M40.1 and the polyglot-pilot handoff marked resolved (kept for root-cause history). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PieterjanDeClippel
added a commit
that referenced
this pull request
Jul 13, 2026
* docs(M39): PRD + plan for self-play training (Connect-4 → chess)
A three-agent investigation established that ~70% of the outer loop already
exists (PolicyValueNet, the soft-CE+value train step, ITrainingCampaign/
CampaignRunner, model store + checkpoint format, the M38 AdamState/TrainWindow/
PolicyGrowth plumbing, action masking, RNG streams, --viz, the A/B harness).
Adds docs/prd/CHESS_SELFPLAY_PRD.md and an M39 milestone in PLAN.md:
AlphaZero-style self-play (MCTS + the two-headed net) on ONE new deep seam,
IZeroSumGame<TState> in Core/Planning (a sibling to IDeterministicModel).
Phased to de-risk the novel machinery on Connect-4 first (cheap CPU
convergence + a negamax oracle), then land chess as the seam's second consumer
(perft-gated legal movegen). Honest scope: MLP-only net → legal, steadily-
improving play, not engine strength; self-play is CPU-bound. Docs only.
Stacked on the M38 branch (reuses its Lab plumbing).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* M39.1: self-play rails (IZeroSumGame + MCTS + SelfPlayCampaign) on Connect-4
The reusable self-play stack, proven on Connect-4 before chess's rules
surface (PLAN M39.1). New and self-contained:
- Core/Planning/IZeroSumGame<TState> — the two-player zero-sum game seam
(side-to-move + win/loss/draw + per-state legal moves), a sibling to
IDeterministicModel that MCTS and the self-play trainer both consume.
- Core/Planning/Mcts.cs — AlphaZero PUCT: select (Q+U) → expand-with-priors →
net-leaf eval → value backup negated every ply (zero-sum), Dirichlet root
noise, returns the root visit-count π. Composes over the seam + an Evaluate
delegate exactly as ValueGuidedSearch composes over IDeterministicModel.
- Environments/Connect4/{Connect4Game : IZeroSumGame, Connect4Solver} — the
cheap first consumer + a depth-limited negamax test oracle.
- Lab/PolicyValueTraining.TrainStep — soft-CE(π) + MSE(tanh value, z), the
AlphaZero loss (generalized from CubePolicyTraining to raw arrays).
- Lab/SelfPlayCampaign<TState> : ITrainingCampaign — plays K MCTS self-play
games/chunk → (obs, π, z) rolling window → trains PolicyValueNet; Evaluate =
win-rate vs a random-legal opponent; reuses AdamState + TrainWindow +
telemetry + the model store/checkpoint format. --game connect4 dispatch.
REUSED unchanged: PolicyValueNet (the AZ net), Adam, the autograd ops,
ITrainingCampaign/CampaignRunner/AIHost, IModelStore + checkpoint format,
AdamState/TrainWindow, SeedSequence streams, --viz telemetry. (Growth via
PolicyGrowth deferred — it needs an IGrowableTrunkNet wrapper.)
Gate met: 5 fast Connect-4/MCTS tests green (incl. MCTS finding a forced win
purely by search, agreeing with negamax) + the Slow self-play resume-roundtrip
contract; 331 fast tests total green. A 4-minute `--game connect4` run from
random init reaches 100% vs a random opponent with a falling policy loss
(1.37→1.22) — the pipeline learns end-to-end. (A discriminating frozen-baseline
arena lands with chess in M39.2.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* M39.2: chess as the self-play stack's second consumer (perft-verified)
Chess plugs into the M39.1 rails — SelfPlayCampaign<TState>, MCTS,
PolicyValueNet, and the training step are REUSED UNCHANGED. Only chess-specific
code is new:
- Environments/Chess/ChessBoard.cs (ChessRules + ChessState) — full legal move
generation (castling through/into check, en passant, promotion, pins/checks)
+ make-move + attack detection + terminal detection (checkmate, stalemate,
50-move, insufficient material). Correctness-first mailbox engine.
- ChessFen.cs — FEN parse/render for test positions.
- ChessMoveEncoding.cs — the AlphaZero 64×73 = 4672 action space (queen/knight/
underpromotion planes); encode + decode (queen-promo inferred on apply).
- ChessGame : IZeroSumGame<ChessState> — 18-plane observation, legal-moves→
indices, apply-by-index.
- Lab/ChessLab.cs + --game chess dispatch.
Gates met:
- PERFT (the hard movegen gate): 25/25 published node counts match, incl.
startpos depth 5 = 4,865,609 and Kiwipete depth 4 = 4,085,603 (3s Release).
- Move encoding: encode→decode→Apply round-trips every legal move on positions
rich in castling/en-passant/all promotion flavours, no index collisions;
Result detects fool's-mate and stalemate.
- End-to-end: a 2-minute `--game chess` run from random init plays legal chess
and reaches 62.5% vs a random-legal opponent (16 sims/move), reusing the rails.
- 355 fast tests green (+24 chess).
Threefold-repetition, perspective canonicalization, batched-leaf MCTS, and the
anti-exploitation league/opening-diversity levers (see the PRD robustness note
added here) are M39.3. Honest scope: MLP-over-flattened-planes → legal,
improving play, not engine strength.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* M39.3 (robustness slice): opponent-diversity self-play
The direct code answer to "two self-trained AIs co-adapt into a narrow style,
so a novel/weak human move disorients the net and it loses to a beginner"
(the exploitability failure — real even at superhuman level, cf. the KataGo
cyclic-group exploit).
SelfPlayCampaign gains --opponent-random <frac>: that fraction of games are
learner(MCTS)-vs-random-legal instead of pure self-play, so the net trains on
the off-distribution positions a blunder/unexpected move reaches and learns to
punish them. Only the learner's positions are recorded, with a constant
learner-perspective outcome z (distinct from self-play's alternating z).
Default 0 → behaviour unchanged. Wired into --game connect4 and --game chess.
(Primary robustness is still search-from-the-actual-position; Dirichlet noise +
temperature already broaden self-play. League/opening-diversity/adversarial
fine-tune remain M39.3 future work — see the PRD robustness note. NoisyNets is
NOT the tool here — it perturbs the policy, not position coverage.)
Behaviour-preserving at the default; 356 fast tests green + the new
learner-vs-random contract test (frac 1.0). PLAN M39 marked M39.1+M39.2 shipped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* M39: chess --demo — play/print one AI game (net+MCTS vs random) as FENs
A small dev helper to WATCH the self-taught net play: `--game chess --demo`
loads the trained checkpoint and plays the net (White, MCTS) vs a random-legal
opponent, printing the FEN after each ply. Used to capture a game for a
browser replay; not part of training.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(M40): plan — play the chess AI in the browser via Polyglot single-source
The FruitCake pattern for chess: write the inference path once in a .pg,
transpile to C# (training) + TypeScript (browser); the browser downloads and
parses the .ckpt and runs net+MCTS client-side — zero server inference.
Feasibility VERIFIED by transpiling a probe: Polyglot supports Math.exp/tanh/
log/sqrt + bitwise ops (transcendentals aren't bit-exact C#/JS, but chess
inference doesn't need that). Adds docs/prd/CHESS_WEB_POLYGLOT_PRD.md (incl. a
§8 reference appendix: files to port, the exact .ckpt byte-format for the TS
parser, store ids, commands) + an M40 milestone in PLAN.md. Phased: M40.1
single-source the engine (re-validate perft) → M40.2 net-forward + MCTS in the
.pg + a TS ckpt parser → M40.3 Angular chess page (client-side play). Docs only.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* M40.1 — single-source the chess engine via Polyglot (perft 25/25 on generated core)
Port the perft-verified chess engine (ChessBoard: ChessState+ChessRules, and
ChessMoveEncoding) into one Polyglot source, chess_solver.pg, transpiled to C#
(obj/, build-time) — the same source the browser client will run in M40.2/M40.3.
- chess_solver.pg: internal Pg-prefixed core (PgChessState + PgChessMove). Board is
List<i32>[64], promotion/castling are i32, perft returns i32 (fits every tested
depth; widened to long in C#). Rays are bounded for+continue (no while); delta
tables are List<(i32,i32)> iterated with tuple destructuring — every construct is
one proven by fruitcake_solver.pg.
- ChessState / ChessRules / ChessMoveEncoding are now thin C# facades over the
generated core; PieceType, ChessMove, and all public signatures are unchanged, so
ChessGame, ChessFen, SelfPlayCampaign, and both chess test files are untouched.
- ChessGame's seam (LegalMoves/Apply/Result) delegates to the core's
legalMoveIndices/applyIndex/result, single-sourcing the index logic training and
the browser both use. Perft recurses entirely in-core (no per-node marshalling).
Gate: perft 25/25 on the GENERATED engine (incl. startpos d5 = 4,865,609, Kiwipete
d4 = 4,085,603, ~10s), encoding round-trip + terminal detection green, SelfPlay
chess contract green, full fast suite 355/355 (no FruitCake/Snake/MountainCar
regression from the shared re-transpile). PLAN M40.1 marked shipped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* build(polyglot): bump MSBuild transpiler 0.3.1 → 0.5.3 + fix multi-.pg incremental bug
0.5.3 bundles win-x64/linux-x64/linux-arm64/osx-x64/osx-arm64 (adds macOS, so dev
there no longer needs a manual $(PolyglotTool)). CI (ubuntu) already worked on 0.3.1
— it too bundled the linux binaries.
The version bump does NOT fix the multi-.pg stale-prelude codegen bug (verified: an
incremental touch of one .pg still produces CS0101/CS0260 under 0.5.3). Root cause is
in the package .targets, not the CLI: PolyglotTranspile uses per-file Inputs/Outputs,
so MSBuild's partial-incremental build invokes the CLI with only the changed .pg — but
the CLI must see the whole set in one invocation to emit a single shared prelude +
`partial class PolyglotProgram`. A single-file run re-emits an inline, non-partial
prelude that clashes with the other solvers' generated files.
Fix it at our project layer: a _PolyglotForceFullRetranspile target wipes the generated
polyglot dir whenever any .pg is newer than the shared prelude, forcing PolyglotTranspile
to regenerate the full set in one CLI call. Its single static Output (not a per-file
transform) prevents MSBuild from partially batching it. Verified: the incremental touch
that reliably broke now rebuilds clean in both Release and Debug; no-op rebuilds stay
up-to-date; full suite 362/362, perft 25/25. Durable fix belongs upstream in the
.targets (tracked for a MintPlayer.Polyglot PR).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(polyglot): sharpen the multi-.pg incremental-bug handoff + verified .targets fix
Update POLYGLOT_TOPLEVEL_RECORD_BUG.md (the existing Snake-M34 handoff for this bug)
with what M40.1 pinned down:
- Root cause is the MSBuild .targets, not the CLI — bumping 0.3.1 → 0.5.3 does NOT
fix it (verified). PolyglotTranspile's per-file Inputs/Outputs let MSBuild's partial
incremental build invoke the CLI with only the CHANGED .pg; the CLI then emits that
unit standalone (inline prelude, non-partial PolyglotProgram) and it collides with
the other solvers' generated files. The stale prelude is a symptom, not the cause.
- Flags that the earlier "clean the --out dir" idea is insufficient and harmful on its
own (would delete the other solvers' .cs under a subset invocation).
- Gives the recommended upstream fix: a single stamp Output (un-batchable) + RemoveDir,
so any staleness re-runs the FULL set in one CLI call.
- Documents the consumer-side _PolyglotForceFullRetranspile mitigation already shipped
in the Environments csproj, and points the csproj comment at this handoff.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(polyglot): verified drop-in .targets fix for the multi-.pg incremental bug
Direct writes to C:\Repos\MintPlayer.Polyglot are hook-blocked, so package the fix as
a ready-to-apply handoff instead of a direct PR:
- Add docs/prd/polyglot-pilot/MintPlayer.Polyglot.MSBuild.targets.FIXED — the upstream
build/MintPlayer.Polyglot.MSBuild.targets with the minimal fix applied (stamp Outputs
+ RemoveDir + Touch + FileWrites stamp, plus the rationale comment). A session inside
the Polyglot repo copies it over the source file, branches, and PRs.
- Strengthen the handoff doc: the patch is VERIFIED, not proposed. Built a 2-.pg repro
harness that imports the SOURCE .targets directly (no NuGet-cache mutation): the
unpatched target fails the touch-one-.pg rebuild (a.cs standalone → CS0101/CS0260/
CS8863); the patched target makes all four states green (recover, incremental touch,
no-op up-to-date skip, clean build), with incrementality preserved via the stamp.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* build(polyglot): 0.5.3 → 0.6.0, drop local incremental-bug workaround (fixed upstream)
MintPlayer.Polyglot.MSBuild 0.6.0 (PR #26) ships the stamp-Outputs + RemoveDir fix for
the multi-.pg incremental-transpile bug that was diagnosed and verified here during
M40.1. With it upstream, the temporary local _PolyglotForceFullRetranspile target and
the MintPlayer.Polyglot.MSBuild.targets.FIXED drop-in are no longer needed — both removed.
Verified on 0.6.0 with NO local workaround: clean build green; the incremental single-.pg
touch that used to break (CS0101/CS0260) now rebuilds clean; no-op rebuild up-to-date;
full suite 362/362, chess perft 25/25.
Docs reconciled: CLAUDE.md's "known build failure" note marked fixed in 0.6.0; PLAN M40.1
and the polyglot-pilot handoff marked resolved (kept for root-cause history).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* M40.2 — single-source the chess inference math (net forward + MCTS) + TS .ckpt parser
Add the browser inference path to chess_solver.pg (transpiled to C# + committed TS):
- writeObservation(): the 18-plane × 64 = 1152 observation, matching ChessGame.WriteObservation
exactly (verified by a parity test over startpos / Kiwipete / an en-passant position).
- PgPolicyValueNet.forward(): ReLU trunk (flat-array weights) → policy logits + linear value,
mirroring Core/Nn/PolicyValueNet on one row (same shape as fruitcake's PgDuelingNet).
- PgChessMcts: inference-only AlphaZero PUCT (masked-softmax priors, sqrt exploration, value
negated per ply, NO Dirichlet) — the browser/serving search, single-sourced so C# and TS share it.
- Math cleanup: use std.math Math.abs/Math.max (type-preserving on i32), keep isign (no std sign).
C# side (Environments):
- ChessNetParityTests (the M40.2 gate): Save a PolicyValueNet through its real .ckpt path, parse the
bytes into the generated PgPolicyValueNet exactly as chess-net.ts will, and assert forward (logits +
value) agrees within an f32 tolerance on the start position. Plus writeObservation parity and an MCTS
runtime smoke (valid legal-move distribution summing to 1; chooseMove legal). 358/358 fast tests green.
Browser side (RLDemo.Web ClientApp):
- chess/chess_solver.ts — committed TS twin (regenerated from the .pg; compiles under the app's tsconfig).
- chess/chess-net.ts — the ONE non-single-sourced piece: parses the "selfplay-pv" .ckpt (magic RLNC,
trunk widths, per-layer W/b in Parameters() order; inputSize=1152/actions=4672 supplied) into the
generated PgPolicyValueNet. Mirrors fruitcake-net.ts.
Note: the generated TS is loosely typed in spots (emitter drops local List annotations; renders List<T?>
as `T | null[]`) — compiles under the app's non-strict tsconfig and is runtime-correct, but the emitter
gaps will be handed off to MintPlayer.Polyglot. M40.3 (the interactive board) is next.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(M40.2): mark shipped; note Polyglot TS-emitter issue #27 (verified fix filed)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* build(polyglot): 0.6.0 → 0.7.0 — TS emitter fix (issue #27/PR #28); chess_solver.ts now strictly typed
0.7.0 ships the TypeScript-emitter fix diagnosed + verified during M40.2 (typed local
List declarations; parenthesized List<T?>). Regenerated the committed chess_solver.ts:
- `let d: [number, number][] = []` (was `let d = []`)
- `let moves: PgChessMove[] = []`
- `children: (PgMctsNode | null)[]` (was `PgMctsNode | null[]`)
Verified strict-tsc-clean (strict + noImplicitAny) on chess_solver.ts + chess-net.ts —
the M40.2 "loosely typed generated TS" caveat is resolved. Full build green, 365/365
tests (incl. deep chess perft) — no regression across the four solvers from the
re-transpile. PLAN M40.2 note updated.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* M40.3 — interactive chess page (play the AI / watch AI-vs-AI), client-side
The browser plays chess against the self-taught net with zero server inference — the
engine, net forward, and PUCT search are the transpiled chess_solver.ts single source.
- chess-director.ts: wraps PgChessState + PgChessMcts + the .ckpt-loaded net. Resolves a
human (from,to) click to a legal action index (auto-queen on promotion), applies moves,
and plays the AI's move for the side to move (Black in play mode, either side in watch).
Falls back to a random legal move if no checkpoint is present. Yields to the UI before
the (synchronous) search so the board paints + shows "thinking".
- chess.ts: standalone Angular component — an 8x8 board (White at the bottom), click-to-move
with legal-target dots + last-move/selected highlights, check/checkmate/stalemate/draw
status, and two modes: "Play the AI" and "Watch AI vs AI" (a self-restarting loop). Signals
drive reactivity; the watch loop is cancellable on mode change / navigation.
- Route /chess + a Home tile.
Verified: Angular builds the chess chunk cleanly (51.75 kB); /chess served; no /api/chess
endpoint (404); director/net/solver strict-tsc clean; and the transpiled engine+net+MCTS
plays a full legal game in Node over the real checkpoint (all moves legal). The shipped
weights (wwwroot/models/chess.az.ckpt, LFS) land in a follow-up once training converges.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* M40.3 UX: square rows, orange move highlights, AI-vs-AI speed slider
Feedback from trying the page:
- Board rows no longer collapse on empty squares — added grid-template-rows: repeat(8, 1fr)
so all 8 rows are equal height (square cells regardless of contents).
- Move/selection/target highlights recoloured blue → orange (rgba(232,145,42,…)) for
much clearer visibility against the board.
- Watch mode gets a Speed slider (input type=range) controlling the pause between moves
(left = slower … right = faster); the watch loop reads the signal each move so it
responds live.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* M40.3 UX: captured-pieces tray (red ✕ over each taken piece)
Show material that's been captured off the board, in the controls row (next to the
speed slider in watch mode / New game in play mode). ChessDirector.capturedPieces()
derives the missing set from the board vs the starting material; the component renders
each as its glyph with a red ✕ overlay, updating after every move via the tick signal.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(M40.4): plan chess difficulty — team investigation → PRD §9 + PLAN M40.4
Three read-only agents investigated the checkpoint/training machinery, the web
surfaces, and difficulty-composition design. Synthesis (PRD §9, PLAN M40.4):
- Difficulty spine = MCTS `sims` on ONE net + a browser-side visit-count TEMPERATURE
for human-like variety at the low end. PgChessMcts.search already returns the visit
distribution π (chooseMove = argmax), so temperature/sampling live caller-side in
chess-director.ts — zero .pg changes, no RNG added to the Polyglot core.
- Net-ladder (the owner's idea) → optional "watch it learn" novelty only, not the
backbone: a small briefly-trained MLP has no reliably-rankable intermediate
checkpoints and each is ~5–6 MB.
- Delivery: a committed chess-difficulties.json manifest ({label, ckpt, sims, temp,
cpuct}) — shippable now on the single net (tiers differ by sims/temp), upgradeable to
multi-checkpoint later with zero code change.
- Honest scope: top tier labelled "Full strength", never "Grandmaster".
- Capture-ladder + net-vs-net Elo notes for the optional novelty/per-side tiers.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* M40.4a — auto-captured difficulty ladder in the Lab (net-vs-net arena, hands-off)
Training is offline-only via the Lab; this makes it produce the web app's difficulty
ladder with no manual steps. Enable with `--game chess --ladder` and walk away: as the
net improves, new difficulty checkpoints + a manifest are written straight into the web
models dir.
SelfPlayCampaign<TState> (generic):
- ArenaVsNet(challenger, champion, games): full net-vs-net games (eval MCTS argmax),
diversified by a short randomized opening drawn from a dedicated _arenaRng seeded
independently of the self-play/eval streams → training trajectory is unaffected (the
arena is inference-only; a --ladder run just spends some eval-time, so a time-bounded
run does marginally fewer games).
- MaybePromoteDifficulty (after each Checkpoint): promote a new tier when the live net
beats the last champion by EITHER a rise in winRate-vs-random ≥ --promote-margin (the
discriminating signal while nets are weak — two weak nets draw each other head-to-head,
so net-vs-net alone stays ~50% and can't rank them) OR head-to-head arena ≥ --arena-margin
(once winRate-vs-random saturates). PromoteTier writes {env}.az.d{K}.ckpt + rewrites
{env}-difficulties.json; champion := a frozen (save→reload) snapshot; ladder-resume
adopts the highest tier on disk.
- ChessLab flags: --ladder, --difficulty-dir (default the web models dir), --promote-margin
(0.08), --arena-margin (0.60), --arena-games (20), --difficulty-sims (128), --opening-plies (6);
plus --first-eval/--eval-every (configurable cadence, threaded through LabHost).
Verified: a forced-promotion run produced an ordered 5-tier ladder (chess.az.d1..d5.ckpt +
a 5-entry manifest); the gate correctly declines when neither signal is met; 40/40 chess +
self-play tests pass. PRD §9.6 / PLAN M40.4a.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* M40.4b — chess difficulty selector + adopt @mintplayer/ng-bootstrap (dark theme preserved)
Web difficulty picker (reads the Lab-written manifest) + first adoption of the
@mintplayer/ng-bootstrap component library across the app.
- chess-net.ts: loadDifficulties() parses /models/chess-difficulties.json ({label, ckpt,
sims, temperature, cpuct, winRateVsRandom}), with a single-net fallback.
- chess-director.ts: difficulties[] + current + setDifficulty(d) (loads the tier's ckpt,
cached by url so sims-only switches don't refetch); aiStep now samples ∝ π^(1/T) when a
tier's temperature > 0 (else argmax). Defaults to the strongest tier.
- chess.ts: difficulty <bs-select> (shown when >1 tier), mode tabs as <bs-button-group>,
New game as a themed bootstrap button (BsButtonTypeDirective), speed slider as <bs-range>.
- Adoption: `npm i @mintplayer/ng-bootstrap` (auto-installs peers; aligned the app to a
consistent Angular 22.0.6). angular.json loads @mintplayer/ng-bootstrap/bootstrap.scss.
Dark-blue theme PRESERVED by re-pointing Bootstrap 5.3 CSS variables (--bs-primary #6ea8fe,
--bs-body-bg #14171f, …) in styles.scss; the app's global button rule scoped to :not(.btn)
so Bootstrap buttons keep their themed styling. (Bootstrap doesn't import card/forms, so
Home's .card is untouched.)
CLAUDE.md: hardened the rule that the .NET host serving Angular is already running and
user-owned — never start/stop/kill/restart it; for new npm deps, ask the user to restart.
Verified: Angular builds clean (chess chunk 64.33 kB, no TS/SCSS errors); /chess + the JSON
manifest serve; difficulty tiers populate from the running --ladder training.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* web: Chess nav item + roll ng-bootstrap themed buttons across all game pages
- app.html: add the Chess nav menu link (after FruitCake).
- Convert each game page's ACTION buttons to Bootstrap themed buttons via
BsButtonTypeDirective (`<button [color]="colors.primary|secondary">`, `colors = Color`):
snake, mountaincar, fruit-cake (mode toggles + fullscreen), cube (move-notation /
scramble / solve / reset / playback), game-2048 (CTAs + playback), rush-hour (toolbar +
playback + move steps). Two-state toggles bind active→primary, inactive→secondary.
- Left the playing surfaces untouched: snake/mountaincar/fruit-cake/cube/rush-hour boards
are <canvas>; 2048's grid cells are <div>s; rush-hour's drawing-tool palette; and 2048's
retro "Try again" overlay button keeps its intentional Cirulli brown.
- Removed now-dead per-component button padding/skin styles; kept layout-only button rules
(flex sizing). Global styles.scss `button:not(.btn)` still styles any unconverted button.
- angular.json: drop the obsolete `mixed-decls` sass silence key.
Verified: full Angular build recompiles clean (no NG8113 / TS / SCSS errors); all six pages
carry [color] bindings on their action buttons.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* web: tweak button color roles — snake/mountaincar mode buttons primary; cube Scramble+Reset secondary
Per review of the rendered pages: snake & mountaincar 'Watch AI' / 'Play yourself'|'Drive yourself'
now always primary (the active mode stays [disabled], so the active cue is preserved); cube
'Scramble (full)' and 'Reset' -> secondary (Solve remains the primary call-to-action).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* train(chess): material-shaped value target + material-based ladder ranking (anti-plateau)
Self-play stalled at ~random strength: a net that can't force mate draws every game
(ply cap) → outcome z ≈ 0 → the value head learns "predict draw" and gets no gradient
toward "a pawn up is better". Fix per the owner's insight — reward material.
- Core: new optional IMaterialScore<TState> capability (a game may expose a dense,
side-to-move-relative material advantage). Games without material don't implement it.
- ChessGame implements it: Σ piece values (P1 N3 B3 R5 Q9; king 0 = mate is the ±1
outcome), white−black, signed by side to move.
- SelfPlayCampaign: value target is now a blend (1−α)·outcome + α·tanh(material/5) per
position (α = --material-weight, default 0.5) when the game supports IMaterialScore —
a DENSE signal on every capture. Pure outcome (α ignored) when it doesn't, so Connect4
is unchanged. Early on (all draws) the target is ~pure material; as decisive games
appear the outcome term kicks in — a self-adjusting curriculum, no schedule.
- Ladder gate: the net-vs-net arena now also returns the challenger's average end-of-game
material margin (pawns); promotion's PRIMARY signal is material ≥ --promote-material
(default 0.75) — non-saturating, so it separates nets that only ever draw (the old
head-to-head sat at 50%). winRate-vs-random delta + head-to-head kept as fallbacks.
Verified: Lab builds, 40/40 chess + self-play tests pass (α defaults keep existing
tests' pure-outcome behaviour via games that don't implement IMaterialScore... chess now
does, but the contract tests still pass). Training restarted with --material-weight 0.5.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(M41): plan reusable deterministic CPU-parallel self-play (investigation + PRD)
Two read-only agents investigated the owner's question (why isn't the cube's CPU
parallelization in the reusable Core library, and should self-play get it?):
- Cube data-gen parallelism is hand-rolled + DUPLICATED in two Lab files
(CubeImitationCampaign / CubeEfficientCampaign: Parallel.For + per-worker lists +
per-worker seeded RNG); nothing reusable in Core. Self-play + DQN campaigns are
single-threaded (only implicit backend GEMM). No principled reason — organic growth.
- Self-play IS safe to parallelize: shared read-only net inference is concurrent-safe
(fresh buffers, no static cache, [ThreadStatic] NoGrad, batch-1 forwards don't
nest-parallelize); the only blockers are the shared sample window/counters and the
shared mutable RNGs. Bottleneck is CPU-bound chess movegen inside MCTS.
- Bitwise reproducibility (M25/M26/M36-SHA) is preservable exactly as Core already does
it (VectorEnv per-unit RNG; GEMM disjoint rows → DOP-invariant).
Decision: extract a reusable `DeterministicParallel.Generate` into Core (per-index-derived
RNG + ordered merge → bitwise-identical at any DOP); self-play adopts it for ~N-core speedup
on the movegen bottleneck; cube dedups onto it. Full design + phases + reproducibility gate
in PARALLEL_SELFPLAY_PRD.md; milestone PLAN.md M41.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(chess): plan M42 — conv residual net (the plateau fix), after M41
Fresh 2-agent analysis of the net situation, per owner request:
- chess net is a flat [256,256] MLP (PolicyValueNet) — the real bottleneck
behind the self-play plateau, not search depth or reward shaping
- a residual net (ResidualMlp) is ALREADY in Core but wrong shape (single
scalar head, residual MLP not conv, cube-DAVI-only) → nothing to "move"
- NO convolution anywhere (no Conv2D/im2col/pool/BatchNorm; LayerNorm exists)
- SelfPlayCampaign/PolicyValueTraining hardcode concrete PolicyValueNet →
no two-headed-net interface exists
- obs reshapes cleanly to [18,8,8]; browser twin (chess_solver.pg) is flat-MLP
and needs a conv forward too
Owner chose a true Conv2D residual net. New RESIDUAL_CONV_NET_PRD.md + PLAN M42
(M42.1 Conv2D via im2col+GEMM+col2im; M42.2 IPolicyValueNet + ConvResidualNet;
M42.3 train chess; M42.4 browser conv forward + parity). M41 (parallel
self-play) lands first to make the heavier training affordable.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(core): M41.1 — DeterministicParallel.Generate (bitwise DOP-invariant)
The reusable form of the data-generation loop the cube campaigns hand-roll and
that parallel self-play (M41.2) needs. Generate<TItem>(count, seeds, stream,
baseIndex, makeItem, parallel, dop) runs makeItem over [0,count), each with a
per-index RNG derived from (seeds, stream, baseIndex+i) via the golden-ratio
stride used everywhere else (VectorEnv), writes disjoint ordered slots, and
returns results in index order. No shared mutable state, no reduction — so the
output is bitwise-identical whether run parallel or sequential, at any DOP.
Tests prove: parallel == sequential bitwise; DOP-invariant across 1/2/4/8/16;
ascending-index ordering; distinct per-index streams; baseIndex shifts streams;
seed-sensitivity; empty/negative-count edges. Verified green in isolation
against Core (the full suite can't build here while the web host holds the
RLDemo.Web DLLs locked; the test lives in the normal test project).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(lab): M41.2 — parallel self-play generation (DOP-invariant)
SelfPlayCampaign now fans a chunk's games across cores via
DeterministicParallel.Generate instead of a single-threaded loop. Each game is
a pure function of its own index-derived RNG over the stable read-only net and
RETURNS its samples; the owner thread merges them into the window in ascending
game index, then trains. Removes the three shared-state blockers from the hot
path: the games no longer touch _window, the game counter, or a shared RNG.
- per-game color keys off the global game index (not a shared counter)
- the window shuffle moved to the Buffer stream so it can't collide with the
Policy stream game 0 derives its RNG from
- MCTS/SelectMove/random-move now take a per-game rng parameter
- eval/arena stay sequential (not the bottleneck; can't affect trained weights)
- ChessLab: --parallel (fan out) + --dop (cap), default sequential
Gate (SelfPlayCampaignTests.ParallelGeneration_...AtAnyDop): a short Connect-4
run produces a byte-identical net+optimizer checkpoint for sequential vs
parallel-dop-1 vs parallel-dop-8. Plus the 12 DeterministicParallel unit tests.
Full solution builds; targeted tests green (rest of the Slow suite skipped).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(core): M42.1 — Conv2D autograd op (im2col + GEMM, no new kernels)
Tensor.Conv2D over an NCHW input carried as rank-2 [N, C·H·W] → [N, outC·oH·oW]
(so it never trips the rank-2 assertions). Implemented the standard im2col way:
gather receptive fields into [M, inC·kH·kW], one GEMM against weight [inC·kH·kW,
outC] through Backend.Current (GPU-routable), permute + bias. Backward reuses
the backend's transposed GEMMs (dWeight = colsᵀ·dOut, dCols = dOut·Wᵀ) plus a
col2im scatter for dInput and a column-sum for dBias. No IComputeBackend/ILGPU
change — conv is pure data-movement around the existing tuned GEMM.
Gate: three finite-difference gradient checks (GradCheckTests.Conv2D_*) —
3×3 SAME, stride-2 valid, and 1×1 — all green. This is the spatial primitive
for the AlphaZero-style convolutional residual net (M42.2).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: M42.2 — IPolicyValueNet + conv residual net + arch-agnostic campaign
Introduce IPolicyValueNet (Forward/Parameters/LayerActivations/Save/Describe) so
the self-play stack no longer hardcodes the concrete flat net. PolicyValueNet
implements it verbatim (zero behaviour change — the self-play determinism gate
still produces a byte-identical checkpoint). New ConvResidualPolicyValueNet: an
AlphaZero-style tower over the 18×8×8 board — 3×3 conv stem, N residual blocks
(relu(x + LN(conv(relu(LN(conv(x))))))), a 1×1-conv policy head → linear to the
flat action space, and a 1×1-conv value head → linear→relu→linear scalar. Uses
the M42.1 Conv2D op and LayerNorm (not BatchNorm), value left linear (trainer
tanh's it).
SelfPlayCampaign/PolicyValueTraining now work through IPolicyValueNet; net
construction + checkpoint reload go through an IPolicyValueNetBuilder (MlpNetBuilder
default, back-compat kind "selfplay-pv"; ConvNetBuilder kind "selfplay-pv-conv").
ChessLab: --arch conv|mlp + --filters/--blocks.
Gate (ConvResidualNetTests): correct head shapes, exact save/load round-trip, and
a fixed batch's AlphaZero loss falls under Adam. MLP self-play tests unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(plan): mark M41.1/M41.2 + M42.1/M42.2 shipped; M42.3 training, M42.4 gated
M41.1 (DeterministicParallel), M41.2 (parallel self-play, byte-identical
dop-invariant gate), M42.1 (Conv2D op), M42.2 (IPolicyValueNet + conv residual
net + arch-agnostic campaign) all shipped with green targeted tests. M41.3
(cube dedup) skipped as optional. M42.3 conv training running in background;
M42.4 (browser conv port) deliberately gated on M42.3 beating the baseline.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(web): ship the chess demo net — commit MLP tier-1 checkpoint (LFS) + manifest
The /chess page (client-side inference, play-vs-AI + watch-AI, difficulty
picker) was already built (M40); it just had no net committed, so a fresh
deploy had nothing to load. Commit the trained tier-1 net (chess.az.d1.ckpt,
6.2 MB via Git LFS) + chess-difficulties.json (the manifest the page fetches).
This is the flat-MLP net — plays legal chess client-side with zero server
inference, but weak (the pre-conv-net plateau). The stronger conv net (M42.3
training) will replace it later with no page changes. Verified headless: the
checkpoint loads through the inference path and plays a full legal game.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(lab): M41.3 — cube data-gen reuses DeterministicParallel (Core)
Both cube campaigns (imitation + efficient) dropped their hand-rolled
Parallel.For + per-worker-list + per-worker-seeded-RNG idiom and now call the
Core primitive. Added a raw-ulong-baseSeed overload to
DeterministicParallel.Generate (the SeedSequence overload delegates to it) for
callers that manage their own seed; passing (baseSeed: roundBase, baseIndex: 1)
reproduces the old `roundBase + φ·(worker+1)` per-worker seeding byte-for-byte,
so cube training output is unchanged. The shared Interlocked solve-counter is
gone — each generator returns its own count, summed on the owner thread.
Gate: DeterministicParallelTests.RawSeedOverload_ReproducesTheCubeHandRolled…
locks the byte-identical seeding; Lab builds; 13 primitive tests green. This
completes M41 (the parallelism is now fully in the class library, cube included).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: sync PRDs + PLAN to shipped reality (M40 net, M41 complete, M42.1/2)
- PARALLEL_SELFPLAY_PRD: Planned → SHIPPED; note the raw-baseSeed overload and
that M41.3 cube dedup preserved output byte-for-byte (better than the doc's
"outputs may change" assumption); eval/arena left sequential.
- RESIDUAL_CONV_NET_PRD: Planned → M42.1/M42.2 shipped, M42.3 training, M42.4
gated; Conv2D needed no backend/ILGPU change; de-risk residual-MLP skipped.
- PLAN: M40 header + M40.3/M40.4 marked shipped (demo net committed at
chess.az.d1.ckpt); M41 fully shipped incl. M41.3.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
PieterjanDeClippel
added a commit
that referenced
this pull request
Jul 13, 2026
* docs(M38): PRD + plan to reduce per-game boilerplate (supersedes PR #27) A three-agent audit mapped the remaining copy-paste across three layers and the staleness of PR #27 (M36 INetworkTelemetrySource + M37 DqnGrowth edited exactly the campaign bodies that PR relocates, so a rebase drops net-growth). Adds docs/prd/BOILERPLATE_REDUCTION_PRD.md and a phased, revert-friendly M38 milestone in PLAN.md. Behaviour-preserving refactor (SHA256-bitwise-identical training); re-cuts DqnScoreCampaign to absorb the M36/M37 duplication rather than fight it, keeps PR #27's still-valid web RefreshingCheckpoint<T> + checkpoint README, and fixes two drift bugs the duplication caused (frontend poller try/catch, RushHour puzzle validator). Docs only — no code change yet. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * M38 B0: checkpoint docs + de-dup drift bugs (no behaviour change) Lowest-risk first step of the boilerplate-reduction plan. - Core/Checkpoints/README.md: when-to-use-which-checkpoint decision table (from PR #27, still accurate). - RushHour: one shared RushHourBoardDto.TryBuildPuzzle used by both the controller (analyze/solve) and RushHourDeckStore.Upsert — kills the duplicated validator that could drift so the deck store accepts a board the solver rejects. - Frontend: shared pollModelStatus() with try/catch built in, replacing 3 hand-rolled pollers — two of which (2048, rush-hour) lacked the catch and could raise an unhandled rejection on a backend blip. - Collapse Status2048Response into the shared StatusResponse DTO (moved to its own file); identical wire shape, so the API tests are unaffected. Build 0 errors; 320 fast tests green (incl. Cube/RushHour API + service). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * M38 B1: deep web checkpoint modules (RefreshingCheckpoint + StartupCheckpoint) Extract the two copy-paste families in the web model-service layer into ModelServiceInfrastructure.cs and apply them by composition (no shallow ModelService<T> base): - RefreshingCheckpoint<T>: "re-read the checkpoint on a cadence, double-checked lock, keep the previous net on a corrupt/mid-write read" — was copy-pasted 4x (Cube policy/value/efficient + RushHour policy). Corrupt-read error defined out of existence: callers only ever see a usable net or null. Cube's resident-GPU forward rebuild rides the onReload hook, sharing the service lock so it stays serialized against Dispose. - StartupCheckpoint<T>: load-at-startup readiness (Status/Error + lazy Value + Initialize), was hand-rolled 3x. CubeModelService 235->~160, RushHour 109->~50, 2048 43->~24; ModelStatus / IModelStartupService / ModelStartupHostedService relocated into the infra file. Behaviour-preserving (only the model-unavailable warning wording changed). Build 0 errors; 320 fast tests green (Cube/RushHour API + service suites). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * M38 B2: DqnScoreCampaign base — absorb the M36/M37 duplication Re-cut the shared score-maximizing DQN spine (the change PR #27 predated). Beyond the resume / per-chunk-train / keep-best / checkpoint lifecycle, the base now OWNS the two features that landed after PR #27 and had been copy- pasted per game: - Progressive net growth (M37): grow/growEvery + _growRng + DqnGrowth.Maybe inside the base TrainChunk — removed from both subclasses. - Live telemetry (M36): the base implements INetworkTelemetrySource itself (SnapshotParameters/Sample/SampleIo/SampleActivations), reading the current net + gate + targetSteps it already owns — the ~90-line twin is gone from both subclasses, which now only supply ObservationSize + input/output labels. Snake 213->110, FruitCake 236->110; base is 190 lines. Subclasses supply env, BaseOptions, EvaluateNet, labels, and (FruitCake) AdaptWarmNet (plain->noisy + GrowInput) + noisy StateId. Behaviour-preserving, PROVEN bitwise-identical: fresh fixed-seed `--game snake` and `--game fruitcake` runs (6000 steps, seed 1) produce SHA256-equal snake.dqn.ckpt / snake.dqn-state.ckpt / fruitcake.dqn.ckpt / fruitcake.dqn-state.ckpt vs the pre-refactor build (deployable net AND full resume state). 320 fast tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * M38 B3: LabHost.Run — one shared Lab bootstrap tail Every game Lab hand-rolled the same ~15-line tail: build the AIHost (+ AddGpuBackend for the cube labs), resolve store/runner, construct the campaign, attach the --viz viewer, runner.Run(CampaignOptions), then keep the process alive for the viewer. Extracted into LabHost.Run(args, dataDir, hours, evalOnly, useGpu, build, onEval): the caller supplies only the campaign factory (pulling AdaptiveBackend from the provider for GPU labs) and the eval IO hook. WaitForViewer moves into LabHost (off SnakeLab — every other lab was calling it cross-type, a smell). Applied to all six labs (snake, fruitcake, rushhour, cube, cube-policy, cube-davi). Also fixed the two CS9107 warnings B2 introduced by exposing the base's LearningRate as a property instead of re-capturing the ctor param. Behaviour-preserving. Smoke-tested eval-only: snake/fruitcake (CPU DQN), rushhour (CPU imitation) print the expected placeholders/eval; cube-policy boots the GPU path (AddGpuBackend → AdaptiveBackend → real RTX 3060 detected) and evaluates. Build 0 errors / no new warnings; 320 fast tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * M38 B4: extract SupervisedTraining plumbing (AdamState + TrainWindow) The three imitation/policy campaigns (RushHour, Cube, EfficientCube) each copy-pasted two self-contained bits: the Adam moment load-or-init/save dance (BinaryReader/Writer + UTF-8 + leaveOpen + AdamCheckpoint.Read/Write; the "no stored optimizer" branch) and the rolling window-mean of the (CE, Huber, acc) batch losses (four fields + guarded divide + reset). Extracted into two deep helpers: - AdamState.LoadOrInit / .Save — hides the stream dance; defines the no-optimizer case out of existence (returns a fresh Adam). - TrainWindow struct — Add(ce, huber, acc) + MeanAndReset(). The campaigns stay SEPARATE (their Evaluate and data-generation are different algorithms — merging them would be a shallow/leaky base); only the shared plumbing moved. Net-load + PolicyGrowth wiring stay per-campaign (net types differ). CubeEfficient's own progress blob (samples+round) is untouched. Behaviour-preserving (pure code movement — identical arithmetic + stream I/O, no RNG/logic change). 320 fast tests green — incl. RushHourCampaign_Checkpoints_AndResumes, which saves policy-adam via the new helper and resumes it. RushHour eval-only smoke is byte-identical to pre-B4. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * M38 B5: AtomicFile.Write — one temp-then-rename helper GalleryStore.Add and RushHourDeckStore.Write both hand-rolled the same "write to <path>.tmp, then File.Move overwrite" so a concurrent reader never sees a half-written JSON file. Extracted into AtomicFile.Write(path, contents) (also creates the directory). Both call sites collapse to one line. Behaviour-preserving. Build 0 errors; 320 fast tests green (gallery + deck API/store tests exercise the write path). Note: the frontend watch-scaffolding dedup (wake-lock re-acquire / director-loop, PRD B5 P7/P8) is deliberately left for now — the wake lock is already a deep shared service (ScreenWakeLock owns its own visibility re-acquire), the residual per-component wiring is ~3 lines, and this repo's rules forbid running the Angular build/test locally, so those changes can't be verified here. Not worth a blind, behaviour-touching edit for a cosmetic win. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(M38): mark B0–B5 landed; record deferred B3 CliArgs + B5 frontend Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * M38 B3 (cont.): CliArgs — one flag reader; fix locale-dependent parsing The six game Labs each hand-rolled a forward `for` loop that repeated, per flag, the `i+1<Length` bounds guard and `args[++i]` step — and parsed int/long/ulong WITHOUT InvariantCulture while double/float used it. On a comma-decimal machine locale that inconsistency could misparse a dot-decimal integer/grouping. Extracted `CliArgs`: typed, bounds-safe, culture-invariant reads (`Dbl/Flt/Int/Long/ULong/Str/Ints/Has`), last-occurrence-wins to match the old loop. Migrated snake, fruitcake, rushhour, cube, cube-policy (both of Snake's loops). CubeDaviLab's 46-flag config-precedence block is left as-is (its appsettings→CLI merge is deliberately bespoke). Verified: new CliArgsTests (6, incl. a nl-BE comma-decimal culture test that proves `0.997`/`0.0005` still parse correctly) — 326 fast tests green; and end-to-end smoke (snake honours `--steps 4000`, cube-policy `--beam 40`, fruitcake `--nstep 3 --shape`). Build 0 errors / no new warnings. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * M38 B5 (cont.): remove the redundant per-component wake-lock visibility handler snake / mountaincar / fruit-cake each carried a byte-identical `onVisibilityChange()` (+ a `(document:visibilitychange)` host binding) that re-acquired the screen wake lock on foreground. But `ScreenWakeLock` ALREADY does this itself: `acquire()` registers its own `visibilitychange` listener that re-requests while desired, and `release()` removes it. So the component handlers were a redundant no-op (acquire() early-returns when already desired). Removed all three — the re-acquire concern lives solely in the deep service (information hiding), and the triplicated method is gone. Verified live (ran the host, Playwright): the Angular bundle compiles clean; Snake "Watch AI" plays (food 17→20), dispatching visibilitychange hidden→visible throws nothing and the AI keeps ticking, and the console has zero errors/warnings across the flow. (Left alone: the director watch-loop — its shared part is a 3-line setInterval wrapper around game-specific render/status, i.e. a shallow abstraction not worth extracting.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(M38): mark B3 CliArgs + B5 P7 complete and live-verified Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * M39: self-play training (Connect-4 rails → perft-verified chess) (#32) * docs(M39): PRD + plan for self-play training (Connect-4 → chess) A three-agent investigation established that ~70% of the outer loop already exists (PolicyValueNet, the soft-CE+value train step, ITrainingCampaign/ CampaignRunner, model store + checkpoint format, the M38 AdamState/TrainWindow/ PolicyGrowth plumbing, action masking, RNG streams, --viz, the A/B harness). Adds docs/prd/CHESS_SELFPLAY_PRD.md and an M39 milestone in PLAN.md: AlphaZero-style self-play (MCTS + the two-headed net) on ONE new deep seam, IZeroSumGame<TState> in Core/Planning (a sibling to IDeterministicModel). Phased to de-risk the novel machinery on Connect-4 first (cheap CPU convergence + a negamax oracle), then land chess as the seam's second consumer (perft-gated legal movegen). Honest scope: MLP-only net → legal, steadily- improving play, not engine strength; self-play is CPU-bound. Docs only. Stacked on the M38 branch (reuses its Lab plumbing). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * M39.1: self-play rails (IZeroSumGame + MCTS + SelfPlayCampaign) on Connect-4 The reusable self-play stack, proven on Connect-4 before chess's rules surface (PLAN M39.1). New and self-contained: - Core/Planning/IZeroSumGame<TState> — the two-player zero-sum game seam (side-to-move + win/loss/draw + per-state legal moves), a sibling to IDeterministicModel that MCTS and the self-play trainer both consume. - Core/Planning/Mcts.cs — AlphaZero PUCT: select (Q+U) → expand-with-priors → net-leaf eval → value backup negated every ply (zero-sum), Dirichlet root noise, returns the root visit-count π. Composes over the seam + an Evaluate delegate exactly as ValueGuidedSearch composes over IDeterministicModel. - Environments/Connect4/{Connect4Game : IZeroSumGame, Connect4Solver} — the cheap first consumer + a depth-limited negamax test oracle. - Lab/PolicyValueTraining.TrainStep — soft-CE(π) + MSE(tanh value, z), the AlphaZero loss (generalized from CubePolicyTraining to raw arrays). - Lab/SelfPlayCampaign<TState> : ITrainingCampaign — plays K MCTS self-play games/chunk → (obs, π, z) rolling window → trains PolicyValueNet; Evaluate = win-rate vs a random-legal opponent; reuses AdamState + TrainWindow + telemetry + the model store/checkpoint format. --game connect4 dispatch. REUSED unchanged: PolicyValueNet (the AZ net), Adam, the autograd ops, ITrainingCampaign/CampaignRunner/AIHost, IModelStore + checkpoint format, AdamState/TrainWindow, SeedSequence streams, --viz telemetry. (Growth via PolicyGrowth deferred — it needs an IGrowableTrunkNet wrapper.) Gate met: 5 fast Connect-4/MCTS tests green (incl. MCTS finding a forced win purely by search, agreeing with negamax) + the Slow self-play resume-roundtrip contract; 331 fast tests total green. A 4-minute `--game connect4` run from random init reaches 100% vs a random opponent with a falling policy loss (1.37→1.22) — the pipeline learns end-to-end. (A discriminating frozen-baseline arena lands with chess in M39.2.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * M39.2: chess as the self-play stack's second consumer (perft-verified) Chess plugs into the M39.1 rails — SelfPlayCampaign<TState>, MCTS, PolicyValueNet, and the training step are REUSED UNCHANGED. Only chess-specific code is new: - Environments/Chess/ChessBoard.cs (ChessRules + ChessState) — full legal move generation (castling through/into check, en passant, promotion, pins/checks) + make-move + attack detection + terminal detection (checkmate, stalemate, 50-move, insufficient material). Correctness-first mailbox engine. - ChessFen.cs — FEN parse/render for test positions. - ChessMoveEncoding.cs — the AlphaZero 64×73 = 4672 action space (queen/knight/ underpromotion planes); encode + decode (queen-promo inferred on apply). - ChessGame : IZeroSumGame<ChessState> — 18-plane observation, legal-moves→ indices, apply-by-index. - Lab/ChessLab.cs + --game chess dispatch. Gates met: - PERFT (the hard movegen gate): 25/25 published node counts match, incl. startpos depth 5 = 4,865,609 and Kiwipete depth 4 = 4,085,603 (3s Release). - Move encoding: encode→decode→Apply round-trips every legal move on positions rich in castling/en-passant/all promotion flavours, no index collisions; Result detects fool's-mate and stalemate. - End-to-end: a 2-minute `--game chess` run from random init plays legal chess and reaches 62.5% vs a random-legal opponent (16 sims/move), reusing the rails. - 355 fast tests green (+24 chess). Threefold-repetition, perspective canonicalization, batched-leaf MCTS, and the anti-exploitation league/opening-diversity levers (see the PRD robustness note added here) are M39.3. Honest scope: MLP-over-flattened-planes → legal, improving play, not engine strength. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * M39.3 (robustness slice): opponent-diversity self-play The direct code answer to "two self-trained AIs co-adapt into a narrow style, so a novel/weak human move disorients the net and it loses to a beginner" (the exploitability failure — real even at superhuman level, cf. the KataGo cyclic-group exploit). SelfPlayCampaign gains --opponent-random <frac>: that fraction of games are learner(MCTS)-vs-random-legal instead of pure self-play, so the net trains on the off-distribution positions a blunder/unexpected move reaches and learns to punish them. Only the learner's positions are recorded, with a constant learner-perspective outcome z (distinct from self-play's alternating z). Default 0 → behaviour unchanged. Wired into --game connect4 and --game chess. (Primary robustness is still search-from-the-actual-position; Dirichlet noise + temperature already broaden self-play. League/opening-diversity/adversarial fine-tune remain M39.3 future work — see the PRD robustness note. NoisyNets is NOT the tool here — it perturbs the policy, not position coverage.) Behaviour-preserving at the default; 356 fast tests green + the new learner-vs-random contract test (frac 1.0). PLAN M39 marked M39.1+M39.2 shipped. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * M39: chess --demo — play/print one AI game (net+MCTS vs random) as FENs A small dev helper to WATCH the self-taught net play: `--game chess --demo` loads the trained checkpoint and plays the net (White, MCTS) vs a random-legal opponent, printing the FEN after each ply. Used to capture a game for a browser replay; not part of training. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(M40): plan — play the chess AI in the browser via Polyglot single-source The FruitCake pattern for chess: write the inference path once in a .pg, transpile to C# (training) + TypeScript (browser); the browser downloads and parses the .ckpt and runs net+MCTS client-side — zero server inference. Feasibility VERIFIED by transpiling a probe: Polyglot supports Math.exp/tanh/ log/sqrt + bitwise ops (transcendentals aren't bit-exact C#/JS, but chess inference doesn't need that). Adds docs/prd/CHESS_WEB_POLYGLOT_PRD.md (incl. a §8 reference appendix: files to port, the exact .ckpt byte-format for the TS parser, store ids, commands) + an M40 milestone in PLAN.md. Phased: M40.1 single-source the engine (re-validate perft) → M40.2 net-forward + MCTS in the .pg + a TS ckpt parser → M40.3 Angular chess page (client-side play). Docs only. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * M40.1 — single-source the chess engine via Polyglot (perft 25/25 on generated core) Port the perft-verified chess engine (ChessBoard: ChessState+ChessRules, and ChessMoveEncoding) into one Polyglot source, chess_solver.pg, transpiled to C# (obj/, build-time) — the same source the browser client will run in M40.2/M40.3. - chess_solver.pg: internal Pg-prefixed core (PgChessState + PgChessMove). Board is List<i32>[64], promotion/castling are i32, perft returns i32 (fits every tested depth; widened to long in C#). Rays are bounded for+continue (no while); delta tables are List<(i32,i32)> iterated with tuple destructuring — every construct is one proven by fruitcake_solver.pg. - ChessState / ChessRules / ChessMoveEncoding are now thin C# facades over the generated core; PieceType, ChessMove, and all public signatures are unchanged, so ChessGame, ChessFen, SelfPlayCampaign, and both chess test files are untouched. - ChessGame's seam (LegalMoves/Apply/Result) delegates to the core's legalMoveIndices/applyIndex/result, single-sourcing the index logic training and the browser both use. Perft recurses entirely in-core (no per-node marshalling). Gate: perft 25/25 on the GENERATED engine (incl. startpos d5 = 4,865,609, Kiwipete d4 = 4,085,603, ~10s), encoding round-trip + terminal detection green, SelfPlay chess contract green, full fast suite 355/355 (no FruitCake/Snake/MountainCar regression from the shared re-transpile). PLAN M40.1 marked shipped. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * build(polyglot): bump MSBuild transpiler 0.3.1 → 0.5.3 + fix multi-.pg incremental bug 0.5.3 bundles win-x64/linux-x64/linux-arm64/osx-x64/osx-arm64 (adds macOS, so dev there no longer needs a manual $(PolyglotTool)). CI (ubuntu) already worked on 0.3.1 — it too bundled the linux binaries. The version bump does NOT fix the multi-.pg stale-prelude codegen bug (verified: an incremental touch of one .pg still produces CS0101/CS0260 under 0.5.3). Root cause is in the package .targets, not the CLI: PolyglotTranspile uses per-file Inputs/Outputs, so MSBuild's partial-incremental build invokes the CLI with only the changed .pg — but the CLI must see the whole set in one invocation to emit a single shared prelude + `partial class PolyglotProgram`. A single-file run re-emits an inline, non-partial prelude that clashes with the other solvers' generated files. Fix it at our project layer: a _PolyglotForceFullRetranspile target wipes the generated polyglot dir whenever any .pg is newer than the shared prelude, forcing PolyglotTranspile to regenerate the full set in one CLI call. Its single static Output (not a per-file transform) prevents MSBuild from partially batching it. Verified: the incremental touch that reliably broke now rebuilds clean in both Release and Debug; no-op rebuilds stay up-to-date; full suite 362/362, perft 25/25. Durable fix belongs upstream in the .targets (tracked for a MintPlayer.Polyglot PR). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(polyglot): sharpen the multi-.pg incremental-bug handoff + verified .targets fix Update POLYGLOT_TOPLEVEL_RECORD_BUG.md (the existing Snake-M34 handoff for this bug) with what M40.1 pinned down: - Root cause is the MSBuild .targets, not the CLI — bumping 0.3.1 → 0.5.3 does NOT fix it (verified). PolyglotTranspile's per-file Inputs/Outputs let MSBuild's partial incremental build invoke the CLI with only the CHANGED .pg; the CLI then emits that unit standalone (inline prelude, non-partial PolyglotProgram) and it collides with the other solvers' generated files. The stale prelude is a symptom, not the cause. - Flags that the earlier "clean the --out dir" idea is insufficient and harmful on its own (would delete the other solvers' .cs under a subset invocation). - Gives the recommended upstream fix: a single stamp Output (un-batchable) + RemoveDir, so any staleness re-runs the FULL set in one CLI call. - Documents the consumer-side _PolyglotForceFullRetranspile mitigation already shipped in the Environments csproj, and points the csproj comment at this handoff. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(polyglot): verified drop-in .targets fix for the multi-.pg incremental bug Direct writes to C:\Repos\MintPlayer.Polyglot are hook-blocked, so package the fix as a ready-to-apply handoff instead of a direct PR: - Add docs/prd/polyglot-pilot/MintPlayer.Polyglot.MSBuild.targets.FIXED — the upstream build/MintPlayer.Polyglot.MSBuild.targets with the minimal fix applied (stamp Outputs + RemoveDir + Touch + FileWrites stamp, plus the rationale comment). A session inside the Polyglot repo copies it over the source file, branches, and PRs. - Strengthen the handoff doc: the patch is VERIFIED, not proposed. Built a 2-.pg repro harness that imports the SOURCE .targets directly (no NuGet-cache mutation): the unpatched target fails the touch-one-.pg rebuild (a.cs standalone → CS0101/CS0260/ CS8863); the patched target makes all four states green (recover, incremental touch, no-op up-to-date skip, clean build), with incrementality preserved via the stamp. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * build(polyglot): 0.5.3 → 0.6.0, drop local incremental-bug workaround (fixed upstream) MintPlayer.Polyglot.MSBuild 0.6.0 (PR #26) ships the stamp-Outputs + RemoveDir fix for the multi-.pg incremental-transpile bug that was diagnosed and verified here during M40.1. With it upstream, the temporary local _PolyglotForceFullRetranspile target and the MintPlayer.Polyglot.MSBuild.targets.FIXED drop-in are no longer needed — both removed. Verified on 0.6.0 with NO local workaround: clean build green; the incremental single-.pg touch that used to break (CS0101/CS0260) now rebuilds clean; no-op rebuild up-to-date; full suite 362/362, chess perft 25/25. Docs reconciled: CLAUDE.md's "known build failure" note marked fixed in 0.6.0; PLAN M40.1 and the polyglot-pilot handoff marked resolved (kept for root-cause history). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * M40.2 — single-source the chess inference math (net forward + MCTS) + TS .ckpt parser Add the browser inference path to chess_solver.pg (transpiled to C# + committed TS): - writeObservation(): the 18-plane × 64 = 1152 observation, matching ChessGame.WriteObservation exactly (verified by a parity test over startpos / Kiwipete / an en-passant position). - PgPolicyValueNet.forward(): ReLU trunk (flat-array weights) → policy logits + linear value, mirroring Core/Nn/PolicyValueNet on one row (same shape as fruitcake's PgDuelingNet). - PgChessMcts: inference-only AlphaZero PUCT (masked-softmax priors, sqrt exploration, value negated per ply, NO Dirichlet) — the browser/serving search, single-sourced so C# and TS share it. - Math cleanup: use std.math Math.abs/Math.max (type-preserving on i32), keep isign (no std sign). C# side (Environments): - ChessNetParityTests (the M40.2 gate): Save a PolicyValueNet through its real .ckpt path, parse the bytes into the generated PgPolicyValueNet exactly as chess-net.ts will, and assert forward (logits + value) agrees within an f32 tolerance on the start position. Plus writeObservation parity and an MCTS runtime smoke (valid legal-move distribution summing to 1; chooseMove legal). 358/358 fast tests green. Browser side (RLDemo.Web ClientApp): - chess/chess_solver.ts — committed TS twin (regenerated from the .pg; compiles under the app's tsconfig). - chess/chess-net.ts — the ONE non-single-sourced piece: parses the "selfplay-pv" .ckpt (magic RLNC, trunk widths, per-layer W/b in Parameters() order; inputSize=1152/actions=4672 supplied) into the generated PgPolicyValueNet. Mirrors fruitcake-net.ts. Note: the generated TS is loosely typed in spots (emitter drops local List annotations; renders List<T?> as `T | null[]`) — compiles under the app's non-strict tsconfig and is runtime-correct, but the emitter gaps will be handed off to MintPlayer.Polyglot. M40.3 (the interactive board) is next. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(M40.2): mark shipped; note Polyglot TS-emitter issue #27 (verified fix filed) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * build(polyglot): 0.6.0 → 0.7.0 — TS emitter fix (issue #27/PR #28); chess_solver.ts now strictly typed 0.7.0 ships the TypeScript-emitter fix diagnosed + verified during M40.2 (typed local List declarations; parenthesized List<T?>). Regenerated the committed chess_solver.ts: - `let d: [number, number][] = []` (was `let d = []`) - `let moves: PgChessMove[] = []` - `children: (PgMctsNode | null)[]` (was `PgMctsNode | null[]`) Verified strict-tsc-clean (strict + noImplicitAny) on chess_solver.ts + chess-net.ts — the M40.2 "loosely typed generated TS" caveat is resolved. Full build green, 365/365 tests (incl. deep chess perft) — no regression across the four solvers from the re-transpile. PLAN M40.2 note updated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * M40.3 — interactive chess page (play the AI / watch AI-vs-AI), client-side The browser plays chess against the self-taught net with zero server inference — the engine, net forward, and PUCT search are the transpiled chess_solver.ts single source. - chess-director.ts: wraps PgChessState + PgChessMcts + the .ckpt-loaded net. Resolves a human (from,to) click to a legal action index (auto-queen on promotion), applies moves, and plays the AI's move for the side to move (Black in play mode, either side in watch). Falls back to a random legal move if no checkpoint is present. Yields to the UI before the (synchronous) search so the board paints + shows "thinking". - chess.ts: standalone Angular component — an 8x8 board (White at the bottom), click-to-move with legal-target dots + last-move/selected highlights, check/checkmate/stalemate/draw status, and two modes: "Play the AI" and "Watch AI vs AI" (a self-restarting loop). Signals drive reactivity; the watch loop is cancellable on mode change / navigation. - Route /chess + a Home tile. Verified: Angular builds the chess chunk cleanly (51.75 kB); /chess served; no /api/chess endpoint (404); director/net/solver strict-tsc clean; and the transpiled engine+net+MCTS plays a full legal game in Node over the real checkpoint (all moves legal). The shipped weights (wwwroot/models/chess.az.ckpt, LFS) land in a follow-up once training converges. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * M40.3 UX: square rows, orange move highlights, AI-vs-AI speed slider Feedback from trying the page: - Board rows no longer collapse on empty squares — added grid-template-rows: repeat(8, 1fr) so all 8 rows are equal height (square cells regardless of contents). - Move/selection/target highlights recoloured blue → orange (rgba(232,145,42,…)) for much clearer visibility against the board. - Watch mode gets a Speed slider (input type=range) controlling the pause between moves (left = slower … right = faster); the watch loop reads the signal each move so it responds live. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * M40.3 UX: captured-pieces tray (red ✕ over each taken piece) Show material that's been captured off the board, in the controls row (next to the speed slider in watch mode / New game in play mode). ChessDirector.capturedPieces() derives the missing set from the board vs the starting material; the component renders each as its glyph with a red ✕ overlay, updating after every move via the tick signal. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(M40.4): plan chess difficulty — team investigation → PRD §9 + PLAN M40.4 Three read-only agents investigated the checkpoint/training machinery, the web surfaces, and difficulty-composition design. Synthesis (PRD §9, PLAN M40.4): - Difficulty spine = MCTS `sims` on ONE net + a browser-side visit-count TEMPERATURE for human-like variety at the low end. PgChessMcts.search already returns the visit distribution π (chooseMove = argmax), so temperature/sampling live caller-side in chess-director.ts — zero .pg changes, no RNG added to the Polyglot core. - Net-ladder (the owner's idea) → optional "watch it learn" novelty only, not the backbone: a small briefly-trained MLP has no reliably-rankable intermediate checkpoints and each is ~5–6 MB. - Delivery: a committed chess-difficulties.json manifest ({label, ckpt, sims, temp, cpuct}) — shippable now on the single net (tiers differ by sims/temp), upgradeable to multi-checkpoint later with zero code change. - Honest scope: top tier labelled "Full strength", never "Grandmaster". - Capture-ladder + net-vs-net Elo notes for the optional novelty/per-side tiers. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * M40.4a — auto-captured difficulty ladder in the Lab (net-vs-net arena, hands-off) Training is offline-only via the Lab; this makes it produce the web app's difficulty ladder with no manual steps. Enable with `--game chess --ladder` and walk away: as the net improves, new difficulty checkpoints + a manifest are written straight into the web models dir. SelfPlayCampaign<TState> (generic): - ArenaVsNet(challenger, champion, games): full net-vs-net games (eval MCTS argmax), diversified by a short randomized opening drawn from a dedicated _arenaRng seeded independently of the self-play/eval streams → training trajectory is unaffected (the arena is inference-only; a --ladder run just spends some eval-time, so a time-bounded run does marginally fewer games). - MaybePromoteDifficulty (after each Checkpoint): promote a new tier when the live net beats the last champion by EITHER a rise in winRate-vs-random ≥ --promote-margin (the discriminating signal while nets are weak — two weak nets draw each other head-to-head, so net-vs-net alone stays ~50% and can't rank them) OR head-to-head arena ≥ --arena-margin (once winRate-vs-random saturates). PromoteTier writes {env}.az.d{K}.ckpt + rewrites {env}-difficulties.json; champion := a frozen (save→reload) snapshot; ladder-resume adopts the highest tier on disk. - ChessLab flags: --ladder, --difficulty-dir (default the web models dir), --promote-margin (0.08), --arena-margin (0.60), --arena-games (20), --difficulty-sims (128), --opening-plies (6); plus --first-eval/--eval-every (configurable cadence, threaded through LabHost). Verified: a forced-promotion run produced an ordered 5-tier ladder (chess.az.d1..d5.ckpt + a 5-entry manifest); the gate correctly declines when neither signal is met; 40/40 chess + self-play tests pass. PRD §9.6 / PLAN M40.4a. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * M40.4b — chess difficulty selector + adopt @mintplayer/ng-bootstrap (dark theme preserved) Web difficulty picker (reads the Lab-written manifest) + first adoption of the @mintplayer/ng-bootstrap component library across the app. - chess-net.ts: loadDifficulties() parses /models/chess-difficulties.json ({label, ckpt, sims, temperature, cpuct, winRateVsRandom}), with a single-net fallback. - chess-director.ts: difficulties[] + current + setDifficulty(d) (loads the tier's ckpt, cached by url so sims-only switches don't refetch); aiStep now samples ∝ π^(1/T) when a tier's temperature > 0 (else argmax). Defaults to the strongest tier. - chess.ts: difficulty <bs-select> (shown when >1 tier), mode tabs as <bs-button-group>, New game as a themed bootstrap button (BsButtonTypeDirective), speed slider as <bs-range>. - Adoption: `npm i @mintplayer/ng-bootstrap` (auto-installs peers; aligned the app to a consistent Angular 22.0.6). angular.json loads @mintplayer/ng-bootstrap/bootstrap.scss. Dark-blue theme PRESERVED by re-pointing Bootstrap 5.3 CSS variables (--bs-primary #6ea8fe, --bs-body-bg #14171f, …) in styles.scss; the app's global button rule scoped to :not(.btn) so Bootstrap buttons keep their themed styling. (Bootstrap doesn't import card/forms, so Home's .card is untouched.) CLAUDE.md: hardened the rule that the .NET host serving Angular is already running and user-owned — never start/stop/kill/restart it; for new npm deps, ask the user to restart. Verified: Angular builds clean (chess chunk 64.33 kB, no TS/SCSS errors); /chess + the JSON manifest serve; difficulty tiers populate from the running --ladder training. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * web: Chess nav item + roll ng-bootstrap themed buttons across all game pages - app.html: add the Chess nav menu link (after FruitCake). - Convert each game page's ACTION buttons to Bootstrap themed buttons via BsButtonTypeDirective (`<button [color]="colors.primary|secondary">`, `colors = Color`): snake, mountaincar, fruit-cake (mode toggles + fullscreen), cube (move-notation / scramble / solve / reset / playback), game-2048 (CTAs + playback), rush-hour (toolbar + playback + move steps). Two-state toggles bind active→primary, inactive→secondary. - Left the playing surfaces untouched: snake/mountaincar/fruit-cake/cube/rush-hour boards are <canvas>; 2048's grid cells are <div>s; rush-hour's drawing-tool palette; and 2048's retro "Try again" overlay button keeps its intentional Cirulli brown. - Removed now-dead per-component button padding/skin styles; kept layout-only button rules (flex sizing). Global styles.scss `button:not(.btn)` still styles any unconverted button. - angular.json: drop the obsolete `mixed-decls` sass silence key. Verified: full Angular build recompiles clean (no NG8113 / TS / SCSS errors); all six pages carry [color] bindings on their action buttons. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * web: tweak button color roles — snake/mountaincar mode buttons primary; cube Scramble+Reset secondary Per review of the rendered pages: snake & mountaincar 'Watch AI' / 'Play yourself'|'Drive yourself' now always primary (the active mode stays [disabled], so the active cue is preserved); cube 'Scramble (full)' and 'Reset' -> secondary (Solve remains the primary call-to-action). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * train(chess): material-shaped value target + material-based ladder ranking (anti-plateau) Self-play stalled at ~random strength: a net that can't force mate draws every game (ply cap) → outcome z ≈ 0 → the value head learns "predict draw" and gets no gradient toward "a pawn up is better". Fix per the owner's insight — reward material. - Core: new optional IMaterialScore<TState> capability (a game may expose a dense, side-to-move-relative material advantage). Games without material don't implement it. - ChessGame implements it: Σ piece values (P1 N3 B3 R5 Q9; king 0 = mate is the ±1 outcome), white−black, signed by side to move. - SelfPlayCampaign: value target is now a blend (1−α)·outcome + α·tanh(material/5) per position (α = --material-weight, default 0.5) when the game supports IMaterialScore — a DENSE signal on every capture. Pure outcome (α ignored) when it doesn't, so Connect4 is unchanged. Early on (all draws) the target is ~pure material; as decisive games appear the outcome term kicks in — a self-adjusting curriculum, no schedule. - Ladder gate: the net-vs-net arena now also returns the challenger's average end-of-game material margin (pawns); promotion's PRIMARY signal is material ≥ --promote-material (default 0.75) — non-saturating, so it separates nets that only ever draw (the old head-to-head sat at 50%). winRate-vs-random delta + head-to-head kept as fallbacks. Verified: Lab builds, 40/40 chess + self-play tests pass (α defaults keep existing tests' pure-outcome behaviour via games that don't implement IMaterialScore... chess now does, but the contract tests still pass). Training restarted with --material-weight 0.5. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(M41): plan reusable deterministic CPU-parallel self-play (investigation + PRD) Two read-only agents investigated the owner's question (why isn't the cube's CPU parallelization in the reusable Core library, and should self-play get it?): - Cube data-gen parallelism is hand-rolled + DUPLICATED in two Lab files (CubeImitationCampaign / CubeEfficientCampaign: Parallel.For + per-worker lists + per-worker seeded RNG); nothing reusable in Core. Self-play + DQN campaigns are single-threaded (only implicit backend GEMM). No principled reason — organic growth. - Self-play IS safe to parallelize: shared read-only net inference is concurrent-safe (fresh buffers, no static cache, [ThreadStatic] NoGrad, batch-1 forwards don't nest-parallelize); the only blockers are the shared sample window/counters and the shared mutable RNGs. Bottleneck is CPU-bound chess movegen inside MCTS. - Bitwise reproducibility (M25/M26/M36-SHA) is preservable exactly as Core already does it (VectorEnv per-unit RNG; GEMM disjoint rows → DOP-invariant). Decision: extract a reusable `DeterministicParallel.Generate` into Core (per-index-derived RNG + ordered merge → bitwise-identical at any DOP); self-play adopts it for ~N-core speedup on the movegen bottleneck; cube dedups onto it. Full design + phases + reproducibility gate in PARALLEL_SELFPLAY_PRD.md; milestone PLAN.md M41. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(chess): plan M42 — conv residual net (the plateau fix), after M41 Fresh 2-agent analysis of the net situation, per owner request: - chess net is a flat [256,256] MLP (PolicyValueNet) — the real bottleneck behind the self-play plateau, not search depth or reward shaping - a residual net (ResidualMlp) is ALREADY in Core but wrong shape (single scalar head, residual MLP not conv, cube-DAVI-only) → nothing to "move" - NO convolution anywhere (no Conv2D/im2col/pool/BatchNorm; LayerNorm exists) - SelfPlayCampaign/PolicyValueTraining hardcode concrete PolicyValueNet → no two-headed-net interface exists - obs reshapes cleanly to [18,8,8]; browser twin (chess_solver.pg) is flat-MLP and needs a conv forward too Owner chose a true Conv2D residual net. New RESIDUAL_CONV_NET_PRD.md + PLAN M42 (M42.1 Conv2D via im2col+GEMM+col2im; M42.2 IPolicyValueNet + ConvResidualNet; M42.3 train chess; M42.4 browser conv forward + parity). M41 (parallel self-play) lands first to make the heavier training affordable. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(core): M41.1 — DeterministicParallel.Generate (bitwise DOP-invariant) The reusable form of the data-generation loop the cube campaigns hand-roll and that parallel self-play (M41.2) needs. Generate<TItem>(count, seeds, stream, baseIndex, makeItem, parallel, dop) runs makeItem over [0,count), each with a per-index RNG derived from (seeds, stream, baseIndex+i) via the golden-ratio stride used everywhere else (VectorEnv), writes disjoint ordered slots, and returns results in index order. No shared mutable state, no reduction — so the output is bitwise-identical whether run parallel or sequential, at any DOP. Tests prove: parallel == sequential bitwise; DOP-invariant across 1/2/4/8/16; ascending-index ordering; distinct per-index streams; baseIndex shifts streams; seed-sensitivity; empty/negative-count edges. Verified green in isolation against Core (the full suite can't build here while the web host holds the RLDemo.Web DLLs locked; the test lives in the normal test project). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(lab): M41.2 — parallel self-play generation (DOP-invariant) SelfPlayCampaign now fans a chunk's games across cores via DeterministicParallel.Generate instead of a single-threaded loop. Each game is a pure function of its own index-derived RNG over the stable read-only net and RETURNS its samples; the owner thread merges them into the window in ascending game index, then trains. Removes the three shared-state blockers from the hot path: the games no longer touch _window, the game counter, or a shared RNG. - per-game color keys off the global game index (not a shared counter) - the window shuffle moved to the Buffer stream so it can't collide with the Policy stream game 0 derives its RNG from - MCTS/SelectMove/random-move now take a per-game rng parameter - eval/arena stay sequential (not the bottleneck; can't affect trained weights) - ChessLab: --parallel (fan out) + --dop (cap), default sequential Gate (SelfPlayCampaignTests.ParallelGeneration_...AtAnyDop): a short Connect-4 run produces a byte-identical net+optimizer checkpoint for sequential vs parallel-dop-1 vs parallel-dop-8. Plus the 12 DeterministicParallel unit tests. Full solution builds; targeted tests green (rest of the Slow suite skipped). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(core): M42.1 — Conv2D autograd op (im2col + GEMM, no new kernels) Tensor.Conv2D over an NCHW input carried as rank-2 [N, C·H·W] → [N, outC·oH·oW] (so it never trips the rank-2 assertions). Implemented the standard im2col way: gather receptive fields into [M, inC·kH·kW], one GEMM against weight [inC·kH·kW, outC] through Backend.Current (GPU-routable), permute + bias. Backward reuses the backend's transposed GEMMs (dWeight = colsᵀ·dOut, dCols = dOut·Wᵀ) plus a col2im scatter for dInput and a column-sum for dBias. No IComputeBackend/ILGPU change — conv is pure data-movement around the existing tuned GEMM. Gate: three finite-difference gradient checks (GradCheckTests.Conv2D_*) — 3×3 SAME, stride-2 valid, and 1×1 — all green. This is the spatial primitive for the AlphaZero-style convolutional residual net (M42.2). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat: M42.2 — IPolicyValueNet + conv residual net + arch-agnostic campaign Introduce IPolicyValueNet (Forward/Parameters/LayerActivations/Save/Describe) so the self-play stack no longer hardcodes the concrete flat net. PolicyValueNet implements it verbatim (zero behaviour change — the self-play determinism gate still produces a byte-identical checkpoint). New ConvResidualPolicyValueNet: an AlphaZero-style tower over the 18×8×8 board — 3×3 conv stem, N residual blocks (relu(x + LN(conv(relu(LN(conv(x))))))), a 1×1-conv policy head → linear to the flat action space, and a 1×1-conv value head → linear→relu→linear scalar. Uses the M42.1 Conv2D op and LayerNorm (not BatchNorm), value left linear (trainer tanh's it). SelfPlayCampaign/PolicyValueTraining now work through IPolicyValueNet; net construction + checkpoint reload go through an IPolicyValueNetBuilder (MlpNetBuilder default, back-compat kind "selfplay-pv"; ConvNetBuilder kind "selfplay-pv-conv"). ChessLab: --arch conv|mlp + --filters/--blocks. Gate (ConvResidualNetTests): correct head shapes, exact save/load round-trip, and a fixed batch's AlphaZero loss falls under Adam. MLP self-play tests unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(plan): mark M41.1/M41.2 + M42.1/M42.2 shipped; M42.3 training, M42.4 gated M41.1 (DeterministicParallel), M41.2 (parallel self-play, byte-identical dop-invariant gate), M42.1 (Conv2D op), M42.2 (IPolicyValueNet + conv residual net + arch-agnostic campaign) all shipped with green targeted tests. M41.3 (cube dedup) skipped as optional. M42.3 conv training running in background; M42.4 (browser conv port) deliberately gated on M42.3 beating the baseline. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(web): ship the chess demo net — commit MLP tier-1 checkpoint (LFS) + manifest The /chess page (client-side inference, play-vs-AI + watch-AI, difficulty picker) was already built (M40); it just had no net committed, so a fresh deploy had nothing to load. Commit the trained tier-1 net (chess.az.d1.ckpt, 6.2 MB via Git LFS) + chess-difficulties.json (the manifest the page fetches). This is the flat-MLP net — plays legal chess client-side with zero server inference, but weak (the pre-conv-net plateau). The stronger conv net (M42.3 training) will replace it later with no page changes. Verified headless: the checkpoint loads through the inference path and plays a full legal game. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(lab): M41.3 — cube data-gen reuses DeterministicParallel (Core) Both cube campaigns (imitation + efficient) dropped their hand-rolled Parallel.For + per-worker-list + per-worker-seeded-RNG idiom and now call the Core primitive. Added a raw-ulong-baseSeed overload to DeterministicParallel.Generate (the SeedSequence overload delegates to it) for callers that manage their own seed; passing (baseSeed: roundBase, baseIndex: 1) reproduces the old `roundBase + φ·(worker+1)` per-worker seeding byte-for-byte, so cube training output is unchanged. The shared Interlocked solve-counter is gone — each generator returns its own count, summed on the owner thread. Gate: DeterministicParallelTests.RawSeedOverload_ReproducesTheCubeHandRolled… locks the byte-identical seeding; Lab builds; 13 primitive tests green. This completes M41 (the parallelism is now fully in the class library, cube included). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: sync PRDs + PLAN to shipped reality (M40 net, M41 complete, M42.1/2) - PARALLEL_SELFPLAY_PRD: Planned → SHIPPED; note the raw-baseSeed overload and that M41.3 cube dedup preserved output byte-for-byte (better than the doc's "outputs may change" assumption); eval/arena left sequential. - RESIDUAL_CONV_NET_PRD: Planned → M42.1/M42.2 shipped, M42.3 training, M42.4 gated; Conv2D needed no backend/ILGPU change; de-risk residual-MLP skipped. - PLAN: M40 header + M40.3/M40.4 marked shipped (demo net committed at chess.az.d1.ckpt); M41 fully shipped incl. M41.3. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
M34 — "Can we improve the Rubik's Cube AI?"
The shipped EfficientCube solver already solved 100% (beam) at every eval depth d4→d26, so solve-rate was saturated. This work established — by measurement — that the only real levers were solution length and search cost, then acted on them. Full write-up:
docs/prd/CUBE_SOLVER_IMPROVE_PRD.md(PLAN.md M34).Done phase-by-phase, one commit per phase, gated so each phase's evidence justified the next.
Outcome
The model was already optimal wherever verifiable; the real improvement is a search-width retune, not retraining.
--eval-only) + a BFS provable-optimality probe (--optimal-probe). Baseline of the shipped 346.8M net: 100% provably QTM-optimal at d1–6, optimal through d12, gap only at d14–d18.--beam-sweep: width ≥1000 already solves 100% (2000 was conservative); wider = shorter mid-depth (d14 17.8→14.4qt at b5000 ≈ optimal, d16–d22 −1–2qt), ~linear expansions cost.BeamSearchValueGuided+--value-sweep(+ 2 unit tests). Null result: value guidance shortens per-fixed-width but loses 3–11× on compute vs simply widening the pure beam. Kept λ=0.OPTIMIZATIONS.md).CubeController(quality over latency, owner's call): d14→14.4qt, d16–d22 −1–2qt at ~2.4× search. Beam stays pure-policy.What ships
--eval-onlylength metrics,--optimal-probe,--beam-sweep,--value-sweep).Notes for review
--value-sweeptooling and the combined head) but λ stays 0.🤖 Generated with Claude Code