The defect class is reachability, not arithmetic. #417 and #339 each report a
site that emits f32 where vLLM emits bf16. The reason those sites exist is
structural: eight functions in the tree decide an activation width, and
six of them have internal linkage — they live in the anonymous namespace of
src/vllm/model_executor/models/qwen3_5.cpp and cannot be called by any other
translation unit:
| Resolver |
Where |
Linkage |
Bf16GemmOutEnabled() |
qwen3_5.cpp:1779 |
internal |
GdnActDType() |
qwen3_5.cpp:2990 |
internal |
GdnInDType() |
qwen3_5.cpp:3015 |
internal |
GdnOutDType(bool) |
qwen3_5.cpp:3042 |
internal |
ResidualDType() |
qwen3_5.cpp:3062 |
internal |
MergedGdnBaOutputDType(bool) |
qwen3_5.cpp:3094 |
internal |
ResolveKvCacheDType() |
include/vllm/v1/kv_cache_dtype.h:28 |
shared |
ResolveMambaSsmCacheDType() |
shared header |
shared |
A ninth decision — the attention-block activation dtype — is computed inline as
an unnamed lambda at include/vllm/model_executor/models/dense_attn_block.h:362.
It is in a shared header but has no name, so it can only be inherited by
including the whole block.
The consequence is exactly what the two policy sections added in AGENTS.md
("Inherit its defaults, do not re-invent them") and .agents/porting.md
("Mirror the memory format, not just the math") describe: a new model author
cannot reach the decision, so they write a literal. There are 1255
DType::k{F32,BF16,F16} literals across src/vllm/model_executor/models/
(736 bf16 / 554 f32 / 6 f16), 784 of them activation-buffer allocations.
kimi_linear_device.cpp alone contains 158 of the narrow
DBuf x(d, DType::kF32, { form.
The correlation is the argument: ~30 models route through
dense_attn_block.h's single adt and have zero findings; every model with a
dtype defect hand-rolled its own forward.
Related: include/vllm/model_executor/layers/linear.h:78 gives
LinearMethodBase::Apply an out_dtype parameter that vLLM does not have —
upstream's apply(layer, x, bias) leaves the decision to the METHOD. Its
sibling MlpGateUpMethodBase::Apply (linear.h:151) has no such parameter,
decides bf16 internally, and has had none of these bugs.
Ask
A behaviour-preserving consistency refactor: promote the width resolvers to a
shared header beside dense_attn_block.h, keeping env names and defaults
EXACTLY as they are, so a future port inherits the decision instead of
re-deriving it. Any site whose literal DISAGREES with what the resolver would
return is reported as a separate finding — repairing it is a behaviour change
and owes its own gated row (#417, #339, #401 are those rows).
This issue is the consistency half only. It does not change a single resolved
dtype.
The defect class is reachability, not arithmetic. #417 and #339 each report a
site that emits f32 where vLLM emits bf16. The reason those sites exist is
structural: eight functions in the tree decide an activation width, and
six of them have internal linkage — they live in the anonymous namespace of
src/vllm/model_executor/models/qwen3_5.cppand cannot be called by any othertranslation unit:
Bf16GemmOutEnabled()qwen3_5.cpp:1779GdnActDType()qwen3_5.cpp:2990GdnInDType()qwen3_5.cpp:3015GdnOutDType(bool)qwen3_5.cpp:3042ResidualDType()qwen3_5.cpp:3062MergedGdnBaOutputDType(bool)qwen3_5.cpp:3094ResolveKvCacheDType()include/vllm/v1/kv_cache_dtype.h:28ResolveMambaSsmCacheDType()A ninth decision — the attention-block activation dtype — is computed inline as
an unnamed lambda at
include/vllm/model_executor/models/dense_attn_block.h:362.It is in a shared header but has no name, so it can only be inherited by
including the whole block.
The consequence is exactly what the two policy sections added in AGENTS.md
("Inherit its defaults, do not re-invent them") and
.agents/porting.md("Mirror the memory format, not just the math") describe: a new model author
cannot reach the decision, so they write a literal. There are 1255
DType::k{F32,BF16,F16}literals acrosssrc/vllm/model_executor/models/(736 bf16 / 554 f32 / 6 f16), 784 of them activation-buffer allocations.
kimi_linear_device.cppalone contains 158 of the narrowDBuf x(d, DType::kF32, {form.The correlation is the argument: ~30 models route through
dense_attn_block.h's singleadtand have zero findings; every model with adtype defect hand-rolled its own forward.
Related:
include/vllm/model_executor/layers/linear.h:78givesLinearMethodBase::Applyanout_dtypeparameter that vLLM does not have —upstream's
apply(layer, x, bias)leaves the decision to the METHOD. Itssibling
MlpGateUpMethodBase::Apply(linear.h:151) has no such parameter,decides bf16 internally, and has had none of these bugs.
Ask
A behaviour-preserving consistency refactor: promote the width resolvers to a
shared header beside
dense_attn_block.h, keeping env names and defaultsEXACTLY as they are, so a future port inherits the decision instead of
re-deriving it. Any site whose literal DISAGREES with what the resolver would
return is reported as a separate finding — repairing it is a behaviour change
and owes its own gated row (#417, #339, #401 are those rows).
This issue is the consistency half only. It does not change a single resolved
dtype.