Found during a tree-wide activation-dtype sweep. This is a correctness defect, not a width one, and it invalidates a previously recorded measurement.
The defect
src/vllm/model_executor/models/kimi_linear_device.cpp:
:1126 — row_bytes is computed from the stream dtype: vt::SizeOf(dh.dtype).:984 — the scatter's SOURCE y is unconditionally f32.:1115 — the destination expert_out is unconditionally f32.:1139-1146 — the scatter copies row_bytes per row.
With VT_KIMI_BF16_STREAM=1, dh.dtype is bf16, so row_bytes halves while the source and destination are still f32. Every routed-expert row is copied at half length and half stride, on all 26 MoE layers.
The f32 default is correct only by the coincidence that SizeOf(f32) == 4 matches the buffers it is copying.
Why it matters beyond the bug
.agents/specs/ records a negative result for the Kimi-Linear bf16 residual stream -- "bf16 residual stream REFUTED, 4/128". That measurement was taken with VT_KIMI_BF16_STREAM=1, i.e. through this defect. The arm under test was corrupting expert outputs while it was being judged.
That negative result is confounded and should be reopened. A lever was rejected on evidence produced by a bug in the lever's own enabling path. This project has a standing lesson that negative results are regime-dependent; this is a sharper version -- the regime was broken.
Fix shape
Either size row_bytes from the buffers actually being copied, or make y/expert_out follow the stream dtype. The second is the direction the rest of the sweep argues for (see the parent dtype work), but the first is the minimal correctness repair and should not wait for it.
Gates
VT_KIMI_BF16_STREAM=1 is not the default, so no shipped path is currently wrong -- this is a latent defect on an opt-in lever. But the lever is how the bf16-stream question gets answered, so it blocks that row.
A RED-first test should set the stream to bf16 and assert the scattered expert output is byte-identical to the f32-stream arm's dequantized equivalent, or at minimum that every destination row is fully written. A mutation that halves row_bytes must fail it.
Provenance
Source-read and confirmed by re-reading the three anchors; no runtime confirmation (read-only session, no GPU). Before acting, confirm dh.dtype is genuinely bf16 under that env on a live run -- this campaign has four recorded instances of a confident source reading contradicted by a runtime log.
Found during a tree-wide activation-dtype sweep. This is a correctness defect, not a width one, and it invalidates a previously recorded measurement.
The defect
src/vllm/model_executor/models/kimi_linear_device.cpp::1126—row_bytesis computed from the stream dtype:vt::SizeOf(dh.dtype).:984— the scatter's SOURCEyis unconditionally f32.:1115— the destinationexpert_outis unconditionally f32.:1139-1146— the scatter copiesrow_bytesper row.With
VT_KIMI_BF16_STREAM=1,dh.dtypeis bf16, sorow_byteshalves while the source and destination are still f32. Every routed-expert row is copied at half length and half stride, on all 26 MoE layers.The f32 default is correct only by the coincidence that
SizeOf(f32) == 4matches the buffers it is copying.Why it matters beyond the bug
.agents/specs/records a negative result for the Kimi-Linear bf16 residual stream -- "bf16 residual stream REFUTED, 4/128". That measurement was taken withVT_KIMI_BF16_STREAM=1, i.e. through this defect. The arm under test was corrupting expert outputs while it was being judged.That negative result is confounded and should be reopened. A lever was rejected on evidence produced by a bug in the lever's own enabling path. This project has a standing lesson that negative results are regime-dependent; this is a sharper version -- the regime was broken.
Fix shape
Either size
row_bytesfrom the buffers actually being copied, or makey/expert_outfollow the stream dtype. The second is the direction the rest of the sweep argues for (see the parent dtype work), but the first is the minimal correctness repair and should not wait for it.Gates
VT_KIMI_BF16_STREAM=1is not the default, so no shipped path is currently wrong -- this is a latent defect on an opt-in lever. But the lever is how the bf16-stream question gets answered, so it blocks that row.A RED-first test should set the stream to bf16 and assert the scattered expert output is byte-identical to the f32-stream arm's dequantized equivalent, or at minimum that every destination row is fully written. A mutation that halves
row_bytesmust fail it.Provenance
Source-read and confirmed by re-reading the three anchors; no runtime confirmation (read-only session, no GPU). Before acting, confirm
dh.dtypeis genuinely bf16 under that env on a live run -- this campaign has four recorded instances of a confident source reading contradicted by a runtime log.