olmoe: adopt route_trace.h — the last engine on the shared history - #754
Conversation
Fourth and final engine for JustVugg#700, after colibri.c (JustVugg#716), kimi_k3.c (JustVugg#719) and inkling.c (JustVugg#742). Every engine that routes experts now writes the same format. m->freq moves from one flat [n_layers * n_experts] block to route_trace.h's row per layer, so the three read sites -- the hot-pin ranking, the bump in moe() and the LFRU eviction guard -- change shape but not meaning. last_access stays flat; it is a separate array route_trace.h does not own. Every olmoe layer routes and there is no MTP layer, so the only row dropped is the spare one rt_init leaves. The history is opt-in behind COLI_USAGE, exactly as kimi_k3 does it: with the variable unset nothing is loaded and nothing is written, so the default path is unchanged. PPL=1 deliberately does not save, so a loss sweep cannot fold its own tokens into the persisted ranking. Verified against the real OLMoE-1B-7B-0125-Instruct converted with tools/convert_olmoe_merged.py. Output is byte-identical to dev on the same run, down to hit=1216 miss=832 -- the eviction path decides exactly as before, which is the property the layout change could have broken. The COLI_USAGE round trip writes the -1/-2 header with the olmoe identity, leaves no row for the dropped layer, and accumulates 2048 -> 4096 across two runs. make check green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Fourth and final engine for JustVugg#700, after colibri.c (JustVugg#716), kimi_k3.c (JustVugg#719) and inkling.c (JustVugg#742). Every engine that routes experts now writes the same format. m->freq moves from one flat [n_layers * n_experts] block to route_trace.h's row per layer, so the three read sites -- the hot-pin ranking, the bump in moe() and the LFRU eviction guard -- change shape but not meaning. last_access stays flat; it is a separate array route_trace.h does not own. Every olmoe layer routes and there is no MTP layer, so the only row dropped is the spare one rt_init leaves. The history is opt-in behind COLI_USAGE, exactly as kimi_k3 does it: with the variable unset nothing is loaded and nothing is written, so the default path is unchanged. PPL=1 deliberately does not save, so a loss sweep cannot fold its own tokens into the persisted ranking. Verified against the real OLMoE-1B-7B-0125-Instruct converted with tools/convert_olmoe_merged.py. Output is byte-identical to dev on the same run, down to hit=1216 miss=832 -- the eviction path decides exactly as before, which is the property the layout change could have broken. The COLI_USAGE round trip writes the -1/-2 header with the olmoe identity, leaves no row for the dropped layer, and accumulates 2048 -> 4096 across two runs. make check green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Merging — that closes the loop from #700. Four engines, one history format, six weeks after you filed the issue that pointed out there were two mutually unreadable writers under the same filename and two engines producing nothing at all. Verified before merging: 13/13 CI including all three
The round-trip evidence is the useful part: The two things you found and deliberately left alone are the right call, and both deserve to be picked up rather than forgotten:
Separating them from the move was correct: they are behaviour questions, and bundling them would have made a +24/-10 refactor into an argument. Thank you for seeing this through all four engines. #700 started as "I hit this while instrumenting K3 and would rather propose the shared facility than send a K3-only patch", and that is exactly what it turned into. |
The fourth and last engine for #700, after
colibri.c(#716),kimi_k3.c(#719) andinkling.c(#742). With this, every engine that routes experts writes the same history.m->freqmoves from one flat[n_layers * n_experts]block toroute_trace.h's row perlayer, so each read site — the hot-pin ranking, the bump in
moe(), and the LFRU evictionguard — changes shape but not meaning.
last_accessstays flat; it is a separate array androute_trace.hdoes not own it.The history follows
kimi_k3's convention exactly: opt-in behindCOLI_USAGE. With thevariable unset nothing is loaded and nothing is written, so the default path is unchanged.
PPL=1deliberately does not save — a loss sweep should not fold its own tokens into thepersisted ranking.
Verified against the real OLMoE-1B-7B-0125-Instruct, converted with
tools/convert_olmoe_merged.py. The patched engine is byte-identical todevon the samerun — same tokens and the same
hit=1216 miss=832, so the eviction path decides exactly asit did. The
COLI_USAGEround trip writes-1 16 64/-2 1 3841822045, has no row for thedropped layer 16, and accumulates 2048 → 4096 across two runs.
make checkgreen.route_trace.hitself needs no change —"olmoe"was already inrt_engine_names[]from#716, so the refusal message can name it on both sides.
Two things I found on the way and deliberately did not touch —
hot_pinned.binloads with noheader or identity check, and
ref_olmoe_real.jsonno longer matches a fresh conversion — arewritten up in #700, since both are behaviour questions rather than part of this move.
🤖 Generated with Claude Code