M36 — Network visualizer: watch any net evolve live during training

README.md

@@ -24,7 +24,7 @@ See also:
 | `src/RLDemo.Console` | Console demo — watch agents learn and play (`--save`/`--load` persist trained models) |
 | `src/RLDemo.Web` | **MintPlayer.AI.ReinforcementLearning Playground** — ASP.NET Core + Angular web app: three games (Rush Hour, classic-feel 2048, a 3D Rubik's Cube), each playable yourself and solvable by the trained AI with step-through playback, plus a public gallery of every submitted board |
 | `tests/MintPlayer.AI.ReinforcementLearning.Tests` | xUnit suite incl. solved-threshold gates, determinism tests and web API integration tests |
-| `tools/MintPlayer.AI.ReinforcementLearning.Lab` | Long-running imitation-learning campaigns (Rush Hour from the BFS oracle, Rubik's Cube from Kociemba) — resumable, checkpointing into the model store |
+| `tools/MintPlayer.AI.ReinforcementLearning.Lab` | Long-running training campaigns (Rush Hour, Rubik's Cube, Snake, FruitCake) — resumable, checkpointing into the model store. Add `--viz` for a live in-browser **network visualizer** (see below) |
 
 ## Run the playground
 
@@ -120,6 +120,36 @@ checkpoints live (it re-reads them every few minutes), or at `models/` to refres
 committed seeds. The DQN fallbacks can be retrained from the console:
 `dotnet run --project src/RLDemo.Console -c Release -- rushhour|cube --save --data models`.
 
+### Training flags
+
+Every campaign is launched by `tools/MintPlayer.AI.ReinforcementLearning.Lab` and shares a common set of flags,
+plus a few per-game knobs.
+
+**Common (all `--game`s):**
+
+| Flag | Meaning |
+|---|---|
+| `--game <name>` | `rushhour` (default), `snake`, `fruitcake`, `cube`, `cube-policy`, `cube-davi` |
+| `--hours H` | wall-clock training budget |
+| `--data DIR` | model-store directory to read/write checkpoints (e.g. `src/RLDemo.Web/data` or `models`) |
+| `--seed S` | RNG seed (runs are reproducible) |
+| `--lr LR` | learning rate |
+| `--eval-only` | evaluate the stored net and exit — no training |
+| `--viz [port]` | live [network visualizer](#watch-the-network-train-live-visualizer) (Development only; bare `--viz` = 5250) |
+| `--grow` / `--grow-every N` | progressively grow the net wider+deeper (all games **except** `cube-davi`, which grows via `--auto-widen`) |
+
+**Per-game knobs:**
+
+| Game | Flags |
+|---|---|
+| `snake`, `fruitcake` (DQN) | `--steps N` (absolute step cap), `--chunk-steps N`, `--episodes N` (eval games), `--explore E` (ε-start), `--hidden a,b` (trunk widths), `--gamma G` |
+| `snake` (extra) | `--train-grid`, `--eval-grid`, `--step-penalty`, `--safe-mask`; `--search` evaluates the look-ahead planner (with `--depth`/`--beam`/`--w-*`/`--net`) instead of training |
+| `fruitcake` (extra) | `--nstep N`, `--shape` (reward shaping), `--noisy` (NoisyNets); offline comparison modes `--ab` (`--baseline`, `--ab-episodes`) and `--search-eval` (`--depth`/`--topk`/`--topk2`/`--leaf`) |
+| `cube` (imitation) | `--width W` (trunk width) |
+| `cube-policy` (EfficientCube) | `--width W`, `--max-scramble D`, `--beam W`, `--episodes N` |
+| `rushhour` (imitation) | common flags only |
+| `cube-davi` (self-taught value net) | its own set — see the section below (`--net residual`, `--width`, `--max-depth`, `--samples`, `--auto-widen`, `--grow-to`, and the `--eval-only --search --batched …` measurement mode) |
+
 ### Train / further-train the self-taught cube solver (DAVI)
 
 The cube's strongest net is trained *teacher-free* by deep approximate value iteration
@@ -153,6 +183,50 @@ dotnet run --project tools/MintPlayer.AI.ReinforcementLearning.Lab -c Release --
 Full recipe + expected single-GPU wall-clock:
 [`tools/…/Lab/CUBE_CAMPAIGN.md`](tools/MintPlayer.AI.ReinforcementLearning.Lab/CUBE_CAMPAIGN.md).
 
+## Watch the network train (live visualizer)
+
+Add `--viz` to **any** Lab training run to open a live, in-browser view of the neural network as it learns:
+
+```
+dotnet run --project tools/MintPlayer.AI.ReinforcementLearning.Lab -c Release -- --game snake --viz
+```
+
+Open the printed URL (default `http://localhost:5250`). The page draws the network as a node-link graph
+with per-layer weight heatmaps and a loss sparkline, all repainting live — you watch the weights move
+from random init toward a policy. **Hover any neuron, connection, or heatmap** for a plain-language
+explanation: each input names the observation feature it is (and its current value), each output the
+action it controls (and its live Q-value/score), and hidden neurons show their current activation. Works
+for every game — `snake`, `fruitcake`, `rushhour`, `cube`, `cube-policy`, `cube-davi`.
+
+![Live network visualizer](docs/screenshots/m36-network-visualizer.png)
+
+**Watch the net grow, too.** The DQN games can grow their network *wider and deeper* mid-training
+(function-preserving Net2WiderNet / Net2DeeperNet). Add `--grow` and watch the graph add units and layers live:
+
+```
+dotnet run --project tools/MintPlayer.AI.ReinforcementLearning.Lab -c Release -- --game fruitcake --viz --grow
+```
+
+It starts from a tiny net and grows through `[16] → [32] → [32,32] → … → [128,128,128]` with no loss spike (each
+step preserves the function exactly). `--grow-every N` sets how many training steps between growth steps — slow it
+down (and open the page immediately) to catch every stage from the start:
+
+```
+dotnet run --project tools/MintPlayer.AI.ReinforcementLearning.Lab -c Release -- \
+  --game fruitcake --viz --grow --grow-every 5000
+```
+
+`--grow` works for **every trainable game** — the DQN nets (Snake, FruitCake) and the two-headed imitation policy
+nets (`--game cube|rushhour --grow`); the cube's self-taught DAVI value net already grows its width during its
+campaign (`--auto-widen`).
+
+![The network after growing wider and deeper](docs/screenshots/m36-network-grows.png)
+
+It is a **development-only** tool: the socket only starts in a Development host environment (the Lab
+defaults to it; set `DOTNET_ENVIRONMENT=Production` to disable) and is never part of the deployed web app.
+Telemetry is read-only — a watched run trains bitwise-identically to an unwatched one. Design notes:
+[docs/prd/NETWORK_VISUALIZER_PRD.md](docs/prd/NETWORK_VISUALIZER_PRD.md).
+
 ## Run the tests
 
 ```

docs/ARCHITECTURE.md

@@ -333,6 +333,8 @@ stops early) and **score-maximizing** (eval = mean return; runs to the wall-cloc
 | [`tools/…Lab/FruitCakeAb.cs`](../tools/MintPlayer.AI.ReinforcementLearning.Lab/FruitCakeAb.cs) | Paired-seed A/B of two nets → mean±SD, paired Δ±SE, verdict. |
 | [`tools/…Lab/FruitCakeSearchEval.cs`](../tools/MintPlayer.AI.ReinforcementLearning.Lab/FruitCakeSearchEval.cs) | Same net, **search vs greedy** on paired seeds → score + max-tier distribution + watermelon count. |
 | [`tools/…Lab/CampaignCli.cs`](../tools/MintPlayer.AI.ReinforcementLearning.Lab/CampaignCli.cs) | `ConsoleAndCsv(path)` — the `OnEval` IO bridge (console + CSV). |
+| [`Core/Telemetry/NetworkTelemetry.cs`](../src/MintPlayer.AI.ReinforcementLearning.Core/Telemetry/NetworkTelemetry.cs) | Live-network telemetry seam (**pull** model): `INetworkTelemetrySource` (`NetKind`/`SnapshotParameters()`/`Sample()`) + `NetworkInspector` (describes/snapshots any net from its parameter tensors; bounded ≤24² weight heatmap). Read-only — training stays bitwise-identical. Every campaign implements the source. |
+| [`tools/…Lab/VizServer.cs`](../tools/MintPlayer.AI.ReinforcementLearning.Lab/VizServer.cs) + [`VizLauncher.cs`](../tools/MintPlayer.AI.ReinforcementLearning.Lab/VizLauncher.cs) | `--viz [port]` live viewer: an `HttpListener` on localhost serving one self-contained Canvas 2D page (node-link graph + heatmaps + **beginner hover tooltips**) + a **WebSocket** (`/ws`). Fully async (per-viewer `Channel` + async pump + async sample loop). `VizLauncher` is the shared flag handler, **gated to a Development environment**. Bidirectional-ready for future viewer→trainer controls. |
 
 ```bash
 # Train (score-maximizing, resumable): writes <data>/<env>.dqn.ckpt (deployable, save-best) + .dqn-state.ckpt (resume)
@@ -349,6 +351,30 @@ dotnet run -c Release --project tools/MintPlayer.AI.ReinforcementLearning.Lab --
   --game fruitcake --search-eval --data ./data --depth 3 --topk 5 --topk2 2
 ```
 
+**Grow the net mid-training (M37).** Function-preserving growth (Net2WiderNet/Net2DeeperNet) is factored into
+`Core/Nn/Net2Net.cs` (`WidenTrunk` splits duplicated units' outgoing weights; `SetIdentity` inserts an identity layer)
+and shared by every trainable net. `--grow` (`[16]→…→[128,128,128]` schedule, watch it in `--viz`) works for: **DQN**
+games via `DuelingQNet.WidenTo`/`.Deepen` + `DqnGrowth` (rebuilds state via `DqnTrainingState.WithNetwork`); the
+**imitation/EfficientCube policy nets** — `CubePolicyNet`/`RushHourPolicyNet` were refactored onto a shared
+variable-depth `PolicyValueNet` core — via `PolicyGrowth` (their checkpoint is now v2 with a trunk-widths array; v1
+shipped files still load). DAVI's `ResidualMlp` already grows width (`--auto-widen`/`--grow-to`). *Change it:* schedule
+in `DqnGrowth.Stages`; operators in `Net2Net`.
+
+**Watch a net train (M36).** Add `--viz [port]` to **any** game (`--game snake --viz`, bare = 5250; works for all six
+`--game`s): the Lab hosts a localhost page that shows the net's topology + a live weight snapshot over a **WebSocket**
+(`/ws`), so you watch the weights move from random init toward a policy in-browser — with hover tooltips explaining
+each part. It's a **pull** model: each campaign implements `INetworkTelemetrySource`, and `VizServer`'s async sample
+loop reads `SnapshotParameters()`/`Sample()` on a cadence. A source may also expose `InputLabels`/`OutputLabels` +
+`SampleIo()` + `SampleActivations()` (default null) to name each neuron and stream its live value. Labels live on the
+envs (`SnakeEnv`, `FruitCakeEnv`, `RubiksCubeEnv`, `RushHourBoard`); every neuron — input, hidden (each net exposes
+`LayerActivations`), and output — shows a live value. The DQN games forward their actual `CurrentObs`; the
+batch-trained nets (Cube/Rush Hour/DAVI) forward a fixed probe state (see `CubeViz`). The viewer attaches output
+labels to the column matching their count (an action head can precede a value head) and draws labeled columns in full. Reading the parameter `float[]`s never mutates anything, so
+a viz run is byte-identical to a no-viz one (SHA256-verified). The socket is **gated to a Development environment**
+(`VizLauncher`); dev-only, never in the deployed web app. See `prd/NETWORK_VISUALIZER_PRD.md`. *Change it:* frame
+contents/heatmap in `NetworkInspector`; each campaign's `SnapshotParameters()`/`Sample()`; page/tooltips/transport in
+`VizServer`; the `--viz` gate in `VizLauncher`.
+
 **Resume contract:** a campaign saves a *deployable* net (`<env>.dqn.ckpt`, **save-best guarded** for noisy
 DQN eval) and a *full resume state* (`<env>.dqn-state.ckpt`: optimizer + replay buffer + RNG + env snapshot).
 Re-running continues bitwise-identically. The cube's self-taught DAVI campaign has its own recipe in