record(ROAD-V1-C3): punch-list item 17 calls DSpark unspiked three days after its W1-W8 landed, and files TLI under the wrong row - #539
Open
localai-bot wants to merge 2 commits into
Open
Conversation
…k tail (#536) Punch-list item 17 of .agents/specs/roadmap-v1-completion.md reads "`ROAD-V1-C3` DSpark + TLI (core spec-decode done; overlaps D3). Size M", and the §2 row it summarises calls both halves "unspiked". Re-verifying against the tree, as AGENTS.md requires before claiming, found that no clause of it survives. DSPARK. The spike spec landed 2b34262 on 2026-08-09, `SPEC-DSPARK` has been `ACTIVE` in the engine matrix ever since, and W1-W8 are implemented and GPU-gated: the Markov head, the sequential sampler, native AND Speculators-format loading, the `d2t` reduced draft vocab, the runner/one-surface wiring, the device sequential sample (#436) and the T=1+k verify capture (#442, mirroring vLLM's `uniform_decode_query_len = 1 + num_speculative_tokens`). Roughly twenty `measure(SPEC-DSPARK)` commits sit behind that, two of them landing today. Under pinned clocks the 35B-A3B MoE lane reads 0.975x on the code cell and 1.012x on the prose cell against the pinned graphed oracle -- not parity, not claimed as such -- with the residual localised to `marlin_moe_wna16::Marlin` and attributed to a 12.9% effective-DRAM-bandwidth gap on byte-equivalent machine code. What remains is a perf tail plus owed gates, not a port. TLI. Genuinely untouched -- no commit, code, spec or issue -- and it is NOT a DSpark tail. Upstream's `use_heterogeneous_vocab` (config/speculative.py:150) and `VocabMapping` (v1/spec_decode/vocab_mapping.py:68) are consumed only by `SpecDecodeBaseProposer` (v1/spec_decode/llm_base_proposer.py) and v1/spec_decode/draft_model.py:19. The V2-runner speculators our DFlash/DSpark port mirrors carry no heterogeneous-vocab path at all, so nothing DSpark lands moves TLI: its host row is `SPEC-DRAFT-MODEL`, locally a CPU propose brick with no runner construction, and TLI is prerequisite-blocked behind that row's W3. DSpark's `d2t` does not cover it either -- `d2t` offsets ids inside ONE tokenizer's vocabulary, while `VocabMapping` builds a string-level intersection ACROSS tokenizer families. Filing the two as one item hid a real dependency. D3. The overlap runs the other way: spec-decode-breadth-d3.md puts DSpark and TLI explicitly out of `ROAD-V1-D3` and back under C3, so D3's landing covers no part of this tail. What DSpark reused is C3's own MTP/DFlash verify/reject loop. Reconciled surfaces: roadmap-v1-completion.md §2 and §3 item 17; roadmap_v1.md row 3, the C3 area row and the DSpark grounding note (which still said `INVENTORIED` and "future spike"); spec-decode-inventory.md's `dspark` row, lifecycle summary, TLI axis and the "no `speculators`-format adapter" paragraph that W3 refutes; dspark-spec-decode.md's Status field, stale at "W1-W5 ... W6 PARTIAL ... ~2% BEHIND spec-off"; and docs/STATUS.md, which listed `dspark` among the INVENTORIED methods while its own DSpark paragraph three hundred lines later described the shipped lane. Issues #436, #442 and #513 were tracking DSpark work while absent from roadmap_v1.md's intake table, so the "three places must agree" rule had only two; they are listed now. Every superseded sentence is struck through and kept in place. It is history, and a reader has to be able to see what the record used to claim -- which is also the diagnosis: the drift is entirely in statements ABOUT the row stored away from the row, where nothing fails when they go stale. Records only. No src/, include/ or tests/ file is touched, no measurement is asserted, and no GPU was used. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: ClaudeCode:claude-opus-5 [ClaudeCode]
FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: ClaudeCode:claude-opus-5 [ClaudeCode]
localai-bot
added a commit
that referenced
this pull request
Aug 12, 2026
…_topk -- remove the literal bounds (#505) Lands the #505 fix: DsaTopkKernel sized `bool chosen[512]` and `int64_t picked[64]` by literal while index_topk is 512 (V4-Flash) / 1024 (V4-Pro). MEASURED on dgx.casa (GB10, sm_121a): the pre-fix kernel at Flash's own width takes `cudaStreamDestroy: an illegal memory access` and SIGABRTs; after the two-pass threshold rewrite the suite is 23/23 with 83913/83913 assertions and 0 skipped, re-verified from the merged tree with CUTLASS + FlashAttention-2 hard-verified in that run's own configure log. The fix removes the bounds rather than asserting them, so there is no configurable limit left to outgrow, and it drops the O(topk^2) emit sort. FRESH REVIEW: PASS. An independent reviewer reproduced both the defect and the fix on real sm_121a hardware, fuzzed 3,000,081 shapes across three independent implementations (host reference, kernel transcription, and its own O(n^2) rank-count oracle) with ZERO divergence, and ran a 12-row device mutation table. It found no blocking issue and 6 non-blocking findings, addressed in a follow-up. Notably it confirmed the tie-heavy case is uniquely load-bearing: it is the only case that catches a tie-break inversion or a value-only threshold. CI: windows-msvc-cpu and windows-msvc-vulkan are red at the repo-wide BASELINE, not from this change -- the same two jobs fail on #539, #541 and on #511 which already merged, and this PR's own log shows every target building with no `error C####`, failing instead in the release-packaging PowerShell step. Merged on direct developer instruction after the requested review pass. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
localai-bot
added a commit
that referenced
this pull request
Aug 13, 2026
…opk, and correct two guard-comment overclaims (#552) Lands the six findings from the fresh review of #505, which returned PASS after failing to find any input where the two-pass threshold selection diverges from `DsaTopkSelect` -- 3,000,081 fuzzed shapes across three independent implementations, zero divergence. CODE: `DsaTopkLaunch` now checks `cudaGetLastError()` after the launch and refuses non-positive `topk` to match the host reference, which previously asserted while the device arm silently returned an empty vector. TESTS: two new device cases. One drives the window clamps -- under-run, over-run, both at once, and an in-range control -- which nothing exercised before. One asserts BOTH arms refuse topk=0 and topk=-1. PROSE: the `w < topk` guard comment claimed to defend against a NaN row writing past its own row. That cannot occur: pass 2's predicate is satisfied by exactly rank(th) <= topk elements and a NaN never satisfies it either. Corrected to belt-and-braces, bounded by construction. "Strictly cheaper" is now scoped to a comparison with the old kernel, not fitness for the real geometry. REFUTED IN FLIGHT: finding 4 held that the launch check would make the next fault attributable to this launch. The arm built to demonstrate it disproved it -- the pre-fix kernel WITH the check still reports `cudaStreamDestroy: an illegal memory access`, because a stack-overflow fault is asynchronous while `cudaGetLastError()` covers launch-configuration errors. The check is kept and its comment now says what it actually does; claiming otherwise would have repeated finding 1's defect inside the change that exists to correct one. DEVICE ARMS on dgx.casa (GB10 sm_121a), each a fresh nvcc rebuild from a pristine kernel with the binary mtime verified to advance, scoped with `-tc=` never `-ts=`, under `flock $HOME/gpu.lock`: baseline 6/6 SUCCESS; no_topk_guard FAILURE; no_ws_clamp exit=134 CRASHED; no_we_clamp 1052 failed assertions; no_launch_check SUCCESS (unobservable by construction); prefix_with_check exit=134; restored full suite 25/25 with 90062 assertions, 0 skipped, SUCCESS. Both new cases have teeth against the mutation each was written for. CPU 25/25 and 13/13, preflight fully clean including the contention-sensitive floor harness. Windows MSVC red is the repo-wide baseline (same two jobs fail on #539, #541 and on #511 which already merged; no `error C####`, the failure is the release-packaging PowerShell step). Merged on direct developer instruction. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #536. Owning row:
ROAD-V1-C3(roadmap row 3), which staysACTIVE.I was dispatched as a fresh implementer on punch-list item 17 of
.agents/specs/roadmap-v1-completion.md§3 — "ROAD-V1-C3DSpark + TLI (corespec-decode done; overlaps D3). Size M." Re-verifying the gap against the tree
first, as AGENTS.md requires, found that no clause of it survives. This is a
reconciliation, not an implementation: no
src/,include/ortests/file istouched, no measurement is asserted, and the GPU lock was never taken.
What git shows
2b342620e(2026-08-09);SPEC-DSPARKACTIVEsincemeasure(SPEC-DSPARK)commits, two of them landing on main todaySPEC-DRAFT-MODELROAD-V1-D3"spec-decode-breadth-d3.md§Scope puts DSpark and TLI out of D3DSpark. The lane ships
qwen3_dspark.{h,cpp}+_weights.cpp,v1/worker/gpu/spec_decode/dspark/speculator.{h,cpp}, five test files, and aSpeculators-format config adapter. W7 (#436) moved the sequential Markov sample
on device; W8 (#442) captured the T=1+k verify by mirroring vLLM's
uniform_decode_query_len = 1 + num_speculative_tokens(
v1/cudagraph_dispatcher.py:37) — same-binary A/B +12.2%/+3.5%, textbyte-identical, four e2e spec suites green with capture ON and OFF. En route it
fixed an engine-wide defect no landed gate could see:
EngineCoreProcneverthreaded
check_for_draft_tokens, so every speculator's drafts were droppedon the CLI and server paths. Cross-engine under pinned clocks the 35B-A3B MoE
lane is 0.975x (code cell, non-overlapping) / 1.012x (prose cell) — not
parity, and the row does not claim it; the residual is localised to
marlin_moe_wna16::Marlinand attributed to a 12.9% effective-DRAM-bandwidthgap on byte-equivalent machine code (94 registers / 3664 SASS both sides, with
upstream doing 4.4% more work per launch). No ceiling is declared: the next
lever is
cudaMemAdvise/placement on the expert slab, since upstreamncuisblocked in both replay modes and a standalone
moe_wna16_marlin_gemmharness isthe only remaining route.
TLI is filed under the wrong row, and that is the useful half
Upstream TLI is
use_heterogeneous_vocab(config/speculative.py:150) plusVocabMapping(v1/spec_decode/vocab_mapping.py:68). Both are consumed onlyby
v1/spec_decode/llm_base_proposer.py(SpecDecodeBaseProposer) andv1/spec_decode/draft_model.py:19. The V2-runner speculators our DFlash/DSparkport mirrors (
v1/worker/gpu/spec_decode/{dflash,dspark}/) have noheterogeneous-vocab path at all — so nothing DSpark lands moves TLI, and its host
row is
SPEC-DRAFT-MODEL, locally a CPU propose brick with no runnerconstruction. TLI is prerequisite-blocked behind that row's W3, not merely
unspiked.
In particular DSpark's
d2tdoes not cover it.d2toffsets ids inside ONEtokenizer's vocabulary (
draft_id + d2t[draft_id]);VocabMappingbuilds astring-level intersection ACROSS tokenizer families, probing the space prefix at
init so a BPE draft (
Ġ) can face a SentencePiece target (▁). Filing them asone item hid a real dependency.
The surfaces reconciled
.agents/specs/roadmap-v1-completion.md— §2 C3 row and §3 item 17.agents/roadmap_v1.md— row 3, the C3 area row, the DSpark grounding note,and intake-table rows for SPEC-DSPARK W7: the sequential Markov sample loop is HOST-side — 28% of the draft step, 14.9 MB downloaded per step #436, SPEC-DSPARK W8: capture the T=1+k speculative VERIFY shape (the measured 0.870x-0.981x gap) #442, DFlash/DSpark draft loading hardcodes the VL weight prefix, so a text-only Qwen3.5 target cannot share its embedding #513 and ROAD-V1-C3's named tail records DSpark as unspiked three days after it landed, and files TLI under the wrong lane #536
.agents/specs/spec-decode-inventory.md— thedsparkrow, the lifecyclesummary, the TLI config axis, and the "no
speculators-format adapter"paragraph (W3 shipped that adapter)
.agents/specs/dspark-spec-decode.md— theStatusfield (stale at"W1-W5 … W6 PARTIAL … ~2% BEHIND spec-off", 2026-08-10) plus a new §8
recording the drift and carrying the issue link
docs/STATUS.md— the method-surface line listeddsparkamong theINVENTORIED methods, contradicting the same page's own DSpark paragraph
Every superseded sentence is struck through and kept in place, not deleted:
it is history, and a reader must be able to see what the record used to claim.
Why the drift happened
The punch-list is written from row summaries, and those summaries were the thing
not updated — the spike went from "planned" straight into twenty-odd
measure(SPEC-DSPARK)commits while the inventory's lifecycle list andSTATUS.md's one-line method surface stayed put. Both are facts about a rowstored away from the row, which is the shape AGENTS.md's Records section warns
about: nothing fails when they drift.
Three issues (#436, #442, #513) were tracking DSpark work while absent from
roadmap_v1.md's intake table, so the "three places must agree" rule had onlytwo. They are listed now. Neither #436 nor #442 is closed here: #436's
device-returning block-forward half is explicitly deferred and sized in the spec
§6k, and #442 remains the tracker for the unclosed 2.5% residual.
Gates
scripts/agent-preflight.sh --staged: green, includingcheck-agent-record,check-doc-checkpoint,check-public-doc-tables(thedocs/STATUS.mdedit isin-place and shortens the line, so no ratchet moves),
check-now-current,audit-live-rowsand the mutation suites.Not done
No GPU work and no new measurement — there was nothing to measure, and the
existing DSpark numbers are already recorded in
docs/BENCHMARKS.mdand thespec.
local-ai-workerwas never touched and$HOME/gpu.lockwas never taken.Needs a fresh reviewer who is not me. Merge authority is not mine.