feat(kimi_k3): adopt the shared routing telemetry (#700) — stacked on #716 - #719
Merged
Merged
Conversation
JustVugg
added
enhancement
terrizoaguimor
force-pushed
the
kimi-k3-telemetry
branch
from
July 31, 2026 21:52
79c2181 to
f52f36e
Compare
.coli_usage had two incompatible writers and two engines that could not
produce it at all: colibri.c wrote sparse text, inkling.c wrote a dense
IKU1 binary block, kimi_k3.c and olmoe.c wrote nothing, and both writers
defaulted to the same filename. A model directory shared between two
engines ended up holding a history the other one refuses.
route_trace.h now owns the format and the ROUTE_TRACE stream, with no
Model or Cfg dependency, so any engine can write and read the same bytes.
colibri.c and telemetry.h move onto it; the duplicated writer and
pin_load duplicated parser are gone.
The header records are encoded as triples with a negative layer so that
readers built before this change parse them, discard them, and continue
into the data:
-1 <n_layers> <n_experts>
-2 <format_version> <engine_id>
That forces three rules, each of which is asserted in the tests against a
literal copy of the pre-existing reader loop rather than against
expectations of it:
- every field must be numeric, or fscanf returns < 3 and the reader
silently drops every record after that point;
- the engine id must be the THIRD field, because old readers parse the
second with %d and glm_moe_dsa hashes to 3815245270, above INT_MAX;
- exactly three fields, never more. A fourth leaves one value
unconsumed, which desynchronises an old reader and makes it fabricate
an admitted record from that leftover plus the start of the next line.
A history whose identity does not match is refused, by name, on every path
the engine chooses for itself: the .coli_usage next to the weights,
PIN=auto, and the AUTOPIN seed. A path the user typed is honoured and says
so once, because pins decide which bytes are resident and never what the
model computes, so the worst case is self-inflicted and visible in the hit
rate. Format version and IKU1 geometry are parse correctness rather than
attribution and are never relaxed.
An empty history stays a zero-byte file, since PIN=auto tests file size to
decide whether to fall back to stats.txt.
rt_drop_row releases the counter row of a layer that does not route, which
is the load admission rule and not an optimisation: every reader treats a
NULL row as "no expert here to attribute a count to", the shape eusage had
before, and which the Vulkan expert-selection path also relies on.
Verified against the unmodified tree: byte-identical ROUTE_TRACE dumps,
history data, [STATS] line, [PIN] placement and dashboard EMAP, plus an
unmodified binary reading a new-format file and this build reading a
legacy one. Confirmed on GLM-5.2 (78x256) as well as the small fixture,
and the unit test passes cross-compiled with MinGW-w64 under wine, where
the CRT rename() shim in compat.h turns out to be load-bearing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Phase B of JustVugg#700. kimi_k3.c gains the expert history and the ROUTE_TRACE stream by calling into route_trace.h, so it writes the same bytes as every other engine and PIN=auto works on it for the first time. Five call sites, on a Model and Cfg that share nothing with GLM's, which is the engine-agnostic claim demonstrated rather than asserted: rt_init dimensions and identity, after model_init settles them rt_drop_row dense layers and the absent MTP row get no counter row rt_load seed from an existing history when COLI_USAGE is set rt_route one call per routing decision: traces and counts rt_save persist on exit The drops are not housekeeping, they are the load admission rule. K3 has dense layers (l->sparse = i >= c->first_dense) and no MTP row, and every reader treats a NULL counter row as "no expert here to attribute a count to". Without them a history record naming a dense layer is silently accumulated, added to the reported total, and written back out. rt_route sits after the top-p renormalisation, so the gates it traces are the ones the layer actually applies. Adds no new diagnostics: the one warning in kimi_k3.c (g_k3_vk unused) is present in the unmodified file at the same site, checked by building both with identical flags. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
terrizoaguimor
force-pushed
the
kimi-k3-telemetry
branch
from
July 31, 2026 22:22
cc4b8ca to
116f65a
Compare
5 tasks
This was referenced Aug 1, 2026
terrizoaguimor
added a commit
to terrizoaguimor/colibri
that referenced
this pull request
Aug 1, 2026
Third engine onto the shared telemetry from JustVugg#700, after colibri.c (JustVugg#716) and kimi_k3.c (JustVugg#719). The counters, the load and the save move to route_trace.h; the bump sites are untouched, since m->eusage now aliases rt_counts_all(). The transition is the point: rt_load reads both the IKU1 block every previous inkling wrote and the shared text format, so an existing .coli_usage keeps working and is rewritten in the new form. The header checks the old pins_load did inline -- magic, n_layers, n_experts against the live config -- all still fire, now in the reader, which additionally refuses a history written by a different engine. Narrower than before, never wider; and parse geometry does not bend to a trusted path the way identity does. Fixes a latent bug on the way: pins_load documents PIN_N=0 as "seeds the ranking from the history but pins nothing", but the npin guard ran before the memcpy, so the counters started at zero and the next usage_save replaced the accumulated ranking with that run's counts alone. Verified on the tiny fixture: rebuild clean, oracle token-exact (36/36 teacher-forced, 24/24 generated), make check 273 passed / 34 skipped, and the built engine reading a planted legacy history pins the right 9 experts while dropping the dense layer's counts. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
6 tasks
terrizoaguimor
added a commit
to terrizoaguimor/colibri
that referenced
this pull request
Aug 1, 2026
Fourth and final engine for JustVugg#700, after colibri.c (JustVugg#716), kimi_k3.c (JustVugg#719) and inkling.c (JustVugg#742). Every engine that routes experts now writes the same format. m->freq moves from one flat [n_layers * n_experts] block to route_trace.h's row per layer, so the three read sites -- the hot-pin ranking, the bump in moe() and the LFRU eviction guard -- change shape but not meaning. last_access stays flat; it is a separate array route_trace.h does not own. Every olmoe layer routes and there is no MTP layer, so the only row dropped is the spare one rt_init leaves. The history is opt-in behind COLI_USAGE, exactly as kimi_k3 does it: with the variable unset nothing is loaded and nothing is written, so the default path is unchanged. PPL=1 deliberately does not save, so a loss sweep cannot fold its own tokens into the persisted ranking. Verified against the real OLMoE-1B-7B-0125-Instruct converted with tools/convert_olmoe_merged.py. Output is byte-identical to dev on the same run, down to hit=1216 miss=832 -- the eviction path decides exactly as before, which is the property the layout change could have broken. The COLI_USAGE round trip writes the -1/-2 header with the olmoe identity, leaves no row for the dropped layer, and accumulates 2048 -> 4096 across two runs. make check green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
terrizoaguimor
added a commit
to terrizoaguimor/colibri
that referenced
this pull request
Aug 1, 2026
Fourth and final engine for JustVugg#700, after colibri.c (JustVugg#716), kimi_k3.c (JustVugg#719) and inkling.c (JustVugg#742). Every engine that routes experts now writes the same format. m->freq moves from one flat [n_layers * n_experts] block to route_trace.h's row per layer, so the three read sites -- the hot-pin ranking, the bump in moe() and the LFRU eviction guard -- change shape but not meaning. last_access stays flat; it is a separate array route_trace.h does not own. Every olmoe layer routes and there is no MTP layer, so the only row dropped is the spare one rt_init leaves. The history is opt-in behind COLI_USAGE, exactly as kimi_k3 does it: with the variable unset nothing is loaded and nothing is written, so the default path is unchanged. PPL=1 deliberately does not save, so a loss sweep cannot fold its own tokens into the persisted ranking. Verified against the real OLMoE-1B-7B-0125-Instruct converted with tools/convert_olmoe_merged.py. Output is byte-identical to dev on the same run, down to hit=1216 miss=832 -- the eviction path decides exactly as before, which is the property the layout change could have broken. The COLI_USAGE round trip writes the -1/-2 header with the olmoe identity, leaves no row for the dropped layer, and accumulates 2048 -> 4096 across two runs. make check green. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase B of #700. Stacked on #716 — do not merge this first.
kimi_k3.cgains the expert history and theROUTE_TRACEstream by calling intoroute_trace.h, so it writes the same bytes as every other engine andPIN=autoworks on itfor the first time.
/cc @steve-m — this touches the router loop in the engine you contributed in #676, so you should
have eyes on it. One line inside
moe_forward, placed after the top-p renormalisation so thegates it records are the ones the layer actually applies. Nothing else in your forward path
changes.
The five call sites
That is the whole diff, on a
ModelandCfgthat share nothing with GLM's — which is theengine-agnostic claim from #700 demonstrated rather than asserted.
Why the drops are not optional
They are the load admission rule, not housekeeping. K3 has dense layers
(
l->sparse = (i >= c->first_dense)) and no MTP row, and every reader treats a NULL counter rowas "there is no expert here to attribute a count to". Skip them and a history record naming a
dense layer is silently accumulated, added to the reported total, and written back out on the
next save — a record no engine would ever emit.
This is the more interesting result of the exercise, and I would not have predicted it: the
row-shape problem is not a GLM quirk. It recurred in the second engine immediately, which is
also why
dev's Vulkan expert-selection path (if(m->eusage[i])) keeps working unchanged.Verified
kimi_k3.cfrom Kimi K3 engine: native-MXFP4 expert streaming, KDA + gated NoPE MLA + AttnRes + LatentMoE (#658) #676,-O3 -march=native -fopenmp.g_k3_vkdefined but not used) ispresent in the unmodified file at the same site — checked by compiling both with identical
flags rather than assuming.
Not yet run against real K3 weights: I do not currently hold the 1.5 TB checkout, and @JustVugg
indicated in #700 that re-fetching it to strengthen a compile-level claim was not a cost worth
absorbing. If either of you would rather have a runtime trace before merging, say so and I will
arrange it.
🤖 Generated with Claude Code