M36 — Network visualizer: watch any net evolve live during training#30
Merged
Conversation
A dev-only tool to see a neural network and watch it change as it trains. - Pull-based telemetry seam in Core (INetworkTelemetrySource + NetworkInspector), keyed off a net's parameter tensors so it covers every architecture (DQN dueling, imitation/EfficientCube policy nets, DAVI value net) with no per-trainer code. - Every ITrainingCampaign implements the source; a shared VizLauncher wires --viz into all six games (snake, fruitcake, rushhour, cube, cube-policy, cube-davi). - Lab-hosted, fully-async WebSocket viewer (VizServer): a self-contained Canvas page with a live node-link graph, per-layer weight heatmaps, a loss sparkline, and beginner hover tooltips explaining each neuron/connection. - Gated to a Development host environment; never part of the deployed web app. - Read-only sampling: training stays bitwise-identical (SHA256-verified viz vs no-viz checkpoints). 314 fast tests green. Docs: docs/prd/NETWORK_VISUALIZER_PRD.md, PLAN.md M36, PRD.md §17, ARCHITECTURE.md §8. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make the hover tooltips meaningful instead of generic "Input/Output neuron": each input names the exact observation feature it represents plus its current value, and each output names the action it controls plus its live Q-value. - Optional semantics on INetworkTelemetrySource (default-null DIM so only opting games change): InputLabels, OutputLabels, and SampleIo() (current observation + the net's forward output for it). Threaded into NetworkTopology (labels) and NetworkFrame (live values). - FruitCakeEnv publishes ObservationLabels (all 89 features, mirroring fruitcake_solver.pg) + ActionLabels (14 drop columns); FruitCakeDqnCampaign overrides the members (SampleIo forwards the latest observation to per-column Q-values). Read-only forward — verified SHA256-identical viz-vs-no-viz WITH a viewer connected during training. - Viewer draws labeled input/output columns in full (uncapped; canvas grows) so each neuron is individually hoverable, showing its meaning and live value; unlabeled/hidden columns stay capped with generic tooltips. Other environments fall back to generic tooltips until they add labels. 314 fast tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…tions Extend the environment-aware tooltips to every game and show each neuron's live value, not just input/output. - Hidden-neuron activations: DuelingQNet.LayerActivations(input) returns every layer's post-activation output; the source exposes SampleActivations() and the frame carries them, so hovering a hidden neuron shows its current activation. Wired for the DQN games (Snake, FruitCake), which have a running observation. - Labels for all games: SnakeEnv (177 egocentric-vision features + 4 directions), RubiksCubeEnv.ActionLabels (12 quarter-turns), RushHourBoard.ActionLabels (32 vehicle×dir moves). Snake also gets live input/output/hidden values; the batch-trained policy/DAVI nets (Cube, Rush Hour) get labels only (no single current observation to forward). - Viewer: attaches output labels to the column whose size matches the label count (a policy net's action head precedes its scalar value head — Describe no longer forces output labels to match the final layer), renders every neuron's live value (input / hidden activation / output Q), and treats column 0 as the input. All forwards are read-only — verified SHA256-identical viz-vs-no-viz with a viewer connected. 314 fast tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Extend live per-neuron values (input / hidden activation / output) to the batch-trained nets (Cube, Rush Hour, DAVI), which have no running observation. - LayerActivations added to CubePolicyNet, RushHourPolicyNet, and ResidualMlp (the residual variant captures each block's inner ReLU + post-skip sum, aligned to the layers the telemetry inspector recovers). - The batch campaigns forward a FIXED probe state each frame — a constant-seed depth-8 scramble (shared CubeViz helper) or the level-1 Rush Hour puzzle — so a viewer watches the net's move preferences / cost-to-go + hidden activations for one specific board evolve as it learns. DAVI's single output is labeled "Estimated distance to solved". - All forwards are read-only (no Backward) and, for a single row, stay on the CPU even under the GPU backend (below its MAC threshold), so they never contend with a GPU training step — verified live on cube-davi with an RTX 3060, and SHA256-identical viz-vs-no-viz remains. Verified in-browser: Snake, FruitCake, Rush Hour, and DAVI all show real, varied hidden activations and live output values. 314 fast tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ne plan - README: new "Watch the network train (live visualizer)" section (--viz on any game, what it shows, dev-only/Development-gated, read-only) + screenshot, and note --viz on the Lab layout row. - NETWORK_VISUALIZER_PRD §5: M36.1 now records all-games labels + per-neuron live values (input/output/hidden activations); dropped the now-shipped "activation tracing" item from M36.3. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ames Grow a DQN net wider and deeper mid-training, function-preservingly, so you can watch it grow live in the visualizer. - DuelingQNet.WidenTo (Net2WiderNet: duplicate a random unit, split the next layer's incoming weights across copies) and .Deepen (Net2DeeperNet: an extra trunk layer initialized to identity — exact after a ReLU). Both compute the same function; two forward-equality unit tests assert it. - DqnTrainingState.WithNetwork swaps in the grown net + a fresh Adam (moments are keyed to the parameter set) while carrying the replay buffer, RNG streams, n-step accumulator and step count forward (obs/action dims unchanged → still valid). - Shared DqnGrowth helper applies a staged schedule ([16]->[32]->[32,32]->[64,64]->[64,64,64]->[128,128,128], alternating wider/deeper) on a step cadence. Wired into BOTH DQN games: --game snake|fruitcake --grow [--grow-every N] (starts from the tiny stage). Coverage of "grow wherever possible": the DQN games grow wider+deeper (new); the DAVI cube value net (ResidualMlp) already grew width (pre-existing --auto-widen / --grow-to); the two-headed imitation policy nets have a fixed 2-layer trunk, so widening is feasible but deepening needs a variable-depth-trunk refactor (deferred). Demoed: fruitcake --viz --grow grew [16] -> [128,128,128] live with no loss spike (function-preserving). 316 fast tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make progressive growth available for the imitation/EfficientCube policy nets, so cube and rush hour can grow wider AND deeper like the DQN games. - Refactor CubePolicyNet and RushHourPolicyNet onto a shared, variable-depth Core/Nn/PolicyValueNet (trunk Linear[] + policy head + value head); the two become thin wrappers preserving their public API. Adds WidenTo/Deepen/Trunk + LayerActivations. - Factor the growth math into Core/Nn/Net2Net (WidenTrunk splits duplicated units' outgoing weights; SetIdentity inserts an identity layer); DuelingQNet now uses it too. New IGrowableTrunkNet<T> lets one PolicyGrowth helper drive any policy net on the shared DqnGrowth schedule. - Checkpoint format for the policy nets → v2 (stores the trunk-widths array). v1 shipped files still load (one hidden width -> a two-layer trunk) — guarded by a test, since the web cube/rushhour solvers load v1 checkpoints. - Wire --grow / --grow-every into the cube, cube-policy and rushhour campaigns (start from the tiny stage, grow on a sample cadence, rebuild Adam). Verified: rushhour --viz --grow grows the policy net [16] -> [128,128,128] live, function-preserving (identity diagonals visible in the deepened layers). 320 fast tests green (+4: widen/deepen forward-equality for DuelingQNet & PolicyValueNet, v1-checkpoint load, grown round-trip). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
--grow was documented but --grow-every (steps between growth steps) was only in PLAN.md; add it to the README growth section with a reproducible command so you can slow the schedule down and watch every stage from the start. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Document the common Lab flags (--game/--hours/--data/--seed/--lr/--eval-only/ --viz/--grow) and the per-game training knobs (snake/fruitcake DQN, cube & cube-policy, rushhour), cross-linking the visualizer section and the cube-davi flag set. Fills the gap where flags like --chunk-steps/--hidden/--width were undocumented. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The PR adds new public API to the published libraries (Core telemetry seam + Net2Net/PolicyValueNet/growth operators; Environments label tables + policy-net growth) — all additive, no breaking changes (existing signatures preserved, policy checkpoint v2 loads v1). Bump RLNetVersion 0.3.0 -> 0.4.0 so master's --skip-duplicate NuGet publish actually ships them. Flip M36/M37 to done. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A dev-only tool to watch a neural network evolve while it trains — and to make the picture readable by people new to reinforcement learning. Run any game's training with
--vizand open the printedhttp://localhost:<port>/:The page shows a live node-link graph, per-layer weight heatmaps, and a loss sparkline that repaint as the weights move from random init toward a policy. Hover any neuron, connection, or heatmap for a plain-language explanation of what it is and what training is doing to it.
How
Core/Telemetry/NetworkTelemetry.cs):INetworkTelemetrySource(NetKind/SnapshotParameters()/Sample()) +NetworkInspector, which turns any net's parameter tensors into a topology + a bounded (≤24²) magnitude heatmap. Keying offParameters()(notIModule) means it covers the non-IModuletwo-headed policy nets too.ITrainingCampaignimplements the source in ~4 lines; a sharedVizLauncherwires--viz [port]intosnake, fruitcake, rushhour, cube, cube-policy, cube-davi. No trainer changes.tools/…Lab/VizServer.cs): anHttpListenerserving one self-contained HTML/Canvas page + a WebSocket (/ws). Fully async — per-viewer boundedChannel+ async send pump + async sample loop; no blocking I/O; costs nothing while no browser is connected. WebSocket (over one-way SSE) is deliberate so the channel can later carry viewer→trainer controls without swapping transport.VizLauncheronly starts the socket when the host environment is Development (the Lab defaults to Development;DOTNET_ENVIRONMENT=Productiondisables it).Safety / verification
snake.dqn.ckpt+snake.dqn-state.ckptare SHA256-identical.Docs
docs/prd/NETWORK_VISUALIZER_PRD.md(new),PLAN.mdM36,PRD.md§17,ARCHITECTURE.md§8.Follow-ups (not in this PR)
.ckptinspection as an Angular/networkpage (reuses the browser.ckptparsers, client-side).🤖 Generated with Claude Code