[Bug] DSpark spec decoding: V1 runner passes dspark_target_layer_ids raw, but the capture rule is idx+1 — draft receives aux hidden states shifted by one layer

[fdaluiso]

Target repository: https://github.com/jasl/vllm (fork). Verified against tree at commit b5c0d43b.

Environment

• vLLM: jasl fork (github.com/jasl/vllm), DeepSeek-V4 / DSpark enablement branch, tree verified at b5c0d43b.
• Model: hybrid DeepSeek-V4 checkpoint, num_hidden_layers = 43 (target), with a DSpark speculative drafter of 3 layers (runtime layer ids 43/44/45).
• Draft config: dspark_target_layer_ids = (40, 41, 42) (the aux hidden states the drafter was trained on).
• Speculative decoding: DSpark (EAGLE3-style aux-hidden-state drafter), vLLM V1 engine, gpu_model_runner path.
• Hardware: NVIDIA GB10 (Grace Blackwell, sm_121a), ARM64, 128 GB UMA — but the bug is hardware-independent.

Summary

The V1 model runner (vllm/v1/worker/gpu_model_runner.py, _get_eagle3_aux_layers_from_config) forwards the DSpark draft config's dspark_target_layer_ids tuple unmodified to the target model. But the target model's native capture rule treats each entry as "the layer whose input should be captured", i.e. the output of layer L-1. The net effect: the drafter receives the outputs of layers 39/40/41 instead of the outputs of 40/41/42 it was trained on, silently depressing acceptance rate.

The same tree already contains the correct conversion in a different runner path — the two paths contradict each other, and only one can match the checkpoint's training semantics.

Details, with file:line references (verified in-tree)

1. The capture rule — vllm/models/deepseek_v4/nvidia/model.py:1254:

if idx + 1 in self.aux_hidden_state_layers:
    # Reconstruct the aux hidden state for draft models
    aux_recon = mhc_post_tilelang(hidden_states, residual, post_mix, res_mix)
    aux_hidden_states.append(aux_recon.mean(dim=1))

An entry L in aux_hidden_state_layers captures the state at idx = L - 1, i.e. the output of layer L-1 (equivalently: the input of layer L). So the tuple's semantics is "input-of-layer-L".

2. The buggy path — vllm/v1/worker/gpu_model_runner.py:5441 (inside _get_eagle3_aux_layers_from_config):

layer_ids = getattr(hf_config, "eagle_aux_hidden_state_layer_ids", None)
if not layer_ids:
    layer_ids = getattr(hf_config, "dspark_target_layer_ids", None)   # <-- raw, no +1

The tuple (40, 41, 42) is passed through raw, so under the capture rule the model collects the outputs of layers 39/40/41.

Notably, the very same function does perform the conversion — but only for DFlash (gpu_model_runner.py:5446-5450):

if dflash_config and isinstance(dflash_config, dict):
    # Add 1 to convert DFlash's aux layer id semantics
    layer_ids = [i + 1 for i in (dflash_config.get("target_layer_ids") or [])]

3. The self-contradiction — the other runner path in the same tree, vllm/v1/worker/gpu/spec_decode/eagle/eagle3_utils.py:47-50, applies the +1 for DSpark:

if not layer_ids:
    dspark_layer_ids = getattr(hf_config, "dspark_target_layer_ids", None)
    if dspark_layer_ids:
        layer_ids = [i + 1 for i in dspark_layer_ids]

Two opposite conventions for the same config key in the same repo; only the +1 variant matches the checkpoint's training (the DSpark drafter is trained on the outputs of the listed target layers).

4. Runtime evidence — boot log (gpu_model_runner.py:5416):

Using auxiliary layers from speculative config: (40, 41, 42)

confirming the raw tuple reaches the model, where the idx + 1 rule then captures outputs of 39/40/41.

Root cause

Inconsistent aux-layer-id semantics between the draft checkpoint config ("capture the outputs of these target layers") and the target model's capture rule ("entry = layer whose input is captured"), with the conversion applied in one runner path (eagle3_utils.py) but missing in the other (gpu_model_runner.py). The bug is masked because adjacent-layer residual streams are highly correlated (~0.95): the drafter still works, just measurably worse — a silent quality bug rather than a crash.

This is part of a family: DeepSeekV4DSparkLayer reads the target's hf_config (vllm/models/deepseek_v4/nvidia/dspark.py:137, and runtime_layer_idx = config.num_hidden_layers + dspark_layer_idx at :146), which lacks the dspark_* keys. This is at least the third bug from the same root we have found and patched locally: (1) dspark_block_size resolving to 0 from the target config, (2) the KV zeroer using a packed upstream stride (zeroing the wrong, live KV record), and now (3) this aux off-by-one — plus (4) the companion rope/YaRN issue filed separately.

Proposed fix (minimal)

Align gpu_model_runner.py with eagle3_utils.py — apply the same +1 conversion for DSpark:

--- a/vllm/v1/worker/gpu_model_runner.py
+++ b/vllm/v1/worker/gpu_model_runner.py
@@ (in _get_eagle3_aux_layers_from_config)
         layer_ids = getattr(hf_config, "eagle_aux_hidden_state_layer_ids", None)
         if not layer_ids:
-            layer_ids = getattr(hf_config, "dspark_target_layer_ids", None)
+            dspark_layer_ids = getattr(hf_config, "dspark_target_layer_ids", None)
+            if dspark_layer_ids:
+                # Same conversion as eagle3_utils.py: the capture rule in
+                # models/deepseek_v4/nvidia/model.py is
+                # `if idx + 1 in self.aux_hidden_state_layers`, i.e. an entry L
+                # means "input of layer L" (= output of L-1). The checkpoint's
+                # dspark_target_layer_ids means "capture the OUTPUT of layer i".
+                layer_ids = [i + 1 for i in dspark_layer_ids]

The change is inert for checkpoints without dspark_target_layer_ids (branch not taken). Alternatively, unify both runner paths on a single shared helper so the semantics cannot diverge again.

We applied this fix locally as an env-gated site-patch on the installed wheel and measured the impact below.

Measured impact

Three-arm A/B on the same probes, same day, same checkpoint (single boot per arm):

Arm	acceptance pos0	mean accepted length
native (no fix)	0.756	2.97
with +1 fix	0.780	3.19

A consistent gain at zero runtime cost. The magnitude matches the mechanism: adjacent residual streams are highly correlated, so the drafter degrades rather than breaks.

Reproducibility

1. Static: read the three code sites above — the contradiction between gpu_model_runner.py:5441 (raw) and eagle3_utils.py:47-50 (+1) is visible by inspection; the capture rule at nvidia/model.py:1254 disambiguates which one is correct.
2. Runtime: boot any DSpark checkpoint through the V1 gpu_model_runner path and observe the log line Using auxiliary layers from speculative config: (40, 41, 42) — then note the capture rule collects idx = 39, 40, 41.
3. A CPU-only unit test encoding the tuple semantics, the capture rule, an anti-vacuity arm (without the fix the captured set MUST be 39/40/41) and a non-regression sweep passes 11/11 — available on request.
4. End-to-end: compare acceptance pos0 / mean accepted length with and without the +1 on the same probe set.

Related

• Companion report: DSpark draft layers beyond the target's compress_ratios silently fall back to yarn-scaled rope (filed separately).
• Same-root previously found: dspark_block_size read as 0 from the target hf_config; KV zeroer packed-stride corruption.

---

This report was prepared with AI assistance (Claude, Anthropic); all file:line references were verified by hand against the tree at commit b5c0d43b.