Skip to content

olmoe: adopt route_trace.h — the last engine on the shared history - #754

Merged
JustVugg merged 2 commits into
JustVugg:devfrom
terrizoaguimor:feat/olmoe-route-trace
Aug 1, 2026
Merged

olmoe: adopt route_trace.h — the last engine on the shared history#754
JustVugg merged 2 commits into
JustVugg:devfrom
terrizoaguimor:feat/olmoe-route-trace

Conversation

@terrizoaguimor

Copy link
Copy Markdown
Contributor

The fourth and last engine for #700, after colibri.c (#716), kimi_k3.c (#719) and
inkling.c (#742). With this, every engine that routes experts writes the same history.

m->freq moves from one flat [n_layers * n_experts] block to route_trace.h's row per
layer, so each read site — the hot-pin ranking, the bump in moe(), and the LFRU eviction
guard — changes shape but not meaning. last_access stays flat; it is a separate array and
route_trace.h does not own it.

The history follows kimi_k3's convention exactly: opt-in behind COLI_USAGE. With the
variable unset nothing is loaded and nothing is written, so the default path is unchanged.
PPL=1 deliberately does not save — a loss sweep should not fold its own tokens into the
persisted ranking.

Verified against the real OLMoE-1B-7B-0125-Instruct, converted with
tools/convert_olmoe_merged.py. The patched engine is byte-identical to dev on the same
run — same tokens and the same hit=1216 miss=832, so the eviction path decides exactly as
it did. The COLI_USAGE round trip writes -1 16 64 / -2 1 3841822045, has no row for the
dropped layer 16, and accumulates 2048 → 4096 across two runs. make check green.

route_trace.h itself needs no change — "olmoe" was already in rt_engine_names[] from
#716, so the refusal message can name it on both sides.

Two things I found on the way and deliberately did not touch — hot_pinned.bin loads with no
header or identity check, and ref_olmoe_real.json no longer matches a fresh conversion — are
written up in #700, since both are behaviour questions rather than part of this move.

🤖 Generated with Claude Code

terrizoaguimor and others added 2 commits August 1, 2026 08:24
Fourth and final engine for JustVugg#700, after colibri.c (JustVugg#716), kimi_k3.c (JustVugg#719) and
inkling.c (JustVugg#742). Every engine that routes experts now writes the same format.

m->freq moves from one flat [n_layers * n_experts] block to route_trace.h's row
per layer, so the three read sites -- the hot-pin ranking, the bump in moe() and
the LFRU eviction guard -- change shape but not meaning. last_access stays flat;
it is a separate array route_trace.h does not own. Every olmoe layer routes and
there is no MTP layer, so the only row dropped is the spare one rt_init leaves.

The history is opt-in behind COLI_USAGE, exactly as kimi_k3 does it: with the
variable unset nothing is loaded and nothing is written, so the default path is
unchanged. PPL=1 deliberately does not save, so a loss sweep cannot fold its own
tokens into the persisted ranking.

Verified against the real OLMoE-1B-7B-0125-Instruct converted with
tools/convert_olmoe_merged.py. Output is byte-identical to dev on the same run,
down to hit=1216 miss=832 -- the eviction path decides exactly as before, which
is the property the layout change could have broken. The COLI_USAGE round trip
writes the -1/-2 header with the olmoe identity, leaves no row for the dropped
layer, and accumulates 2048 -> 4096 across two runs. make check green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Fourth and final engine for JustVugg#700, after colibri.c (JustVugg#716), kimi_k3.c (JustVugg#719) and
inkling.c (JustVugg#742). Every engine that routes experts now writes the same format.

m->freq moves from one flat [n_layers * n_experts] block to route_trace.h's row
per layer, so the three read sites -- the hot-pin ranking, the bump in moe() and
the LFRU eviction guard -- change shape but not meaning. last_access stays flat;
it is a separate array route_trace.h does not own. Every olmoe layer routes and
there is no MTP layer, so the only row dropped is the spare one rt_init leaves.

The history is opt-in behind COLI_USAGE, exactly as kimi_k3 does it: with the
variable unset nothing is loaded and nothing is written, so the default path is
unchanged. PPL=1 deliberately does not save, so a loss sweep cannot fold its own
tokens into the persisted ranking.

Verified against the real OLMoE-1B-7B-0125-Instruct converted with
tools/convert_olmoe_merged.py. Output is byte-identical to dev on the same run,
down to hit=1216 miss=832 -- the eviction path decides exactly as before, which
is the property the layout change could have broken. The COLI_USAGE round trip
writes the -1/-2 header with the olmoe identity, leaves no row for the dropped
layer, and accumulates 2048 -> 4096 across two runs. make check green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@JustVugg

JustVugg commented Aug 1, 2026

Copy link
Copy Markdown
Owner

Merging — that closes the loop from #700. Four engines, one history format, six weeks after you filed the issue that pointed out there were two mutually unreadable writers under the same filename and two engines producing nothing at all.

Verified before merging: 13/13 CI including all three All engines platforms, and the change is genuinely small at +24/-10 — which is the signature of the facility being right rather than the port being shallow. m->freq going from a flat [n_layers * n_experts] block to a row per layer touches three read sites and changes shape without changing meaning; that is exactly the kind of thing that gets subtly wrong, and your note that last_access stays flat because route_trace.h does not own it is the sentence that tells me you looked.

COLI_USAGE-gated with the default path untouched, matching kimi_k3, and PPL=1 deliberately not saving — a loss sweep should not fold its own tokens into a persisted ranking. Agreed, and worth having written down.

The round-trip evidence is the useful part: -1 16 64 / -2 1 3841822045, no row for the dropped layer, and 2048 → 4096 accumulating across two runs. That is the header, the drop, and the accumulation all shown at once.

The two things you found and deliberately left alone are the right call, and both deserve to be picked up rather than forgotten:

  • hot_pinned.bin loading with no header or identity check is the same class of problem this whole issue was about — a file trusted because it is where it is expected to be. It should probably get the same refuse-by-name treatment.
  • ref_olmoe_real.json no longer matching a fresh conversion is the more worrying of the two, because an oracle that has quietly drifted from what the converter produces is an oracle that stops catching things.

Separating them from the move was correct: they are behaviour questions, and bundling them would have made a +24/-10 refactor into an argument.

Thank you for seeing this through all four engines. #700 started as "I hit this while instrumenting K3 and would rather propose the shared facility than send a K3-only patch", and that is exactly what it turned into.

@JustVugg
JustVugg merged commit fe308c9 into JustVugg:dev Aug 1, 2026
13 checks passed
@JustVugg JustVugg added the enhancement New feature or request label Aug 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants