fix(cuda): dispatch grouped int4 (fmt=4) in the async expert-group path

[kreuzzelg]

Summary

coli_cuda_expert_group_issue — the async decode path that MoE decode rides (colibri.c issues it per device each layer) — never dispatches grouped int4. Groups containing an fmt=4 member fall through to the per-expert quant_matmul(..., gs=0, ng=1) fallback, which applies a single per-row scale to a per-group-scaled container: silently wrong output, no error. The sync path (coli_cuda_expert_group) has handled this correctly since #334; the gap is only in the async split. This is the CUDA counterpart of the fmt=4 decode work tracked for Metal in #585/#587.

Affected today: any grouped-int4 (gs) container decoding through the CUDA backend — e.g. GLM gs64 — whenever the async expert-group path is taken.

The fix

Route all-int4 groups with a grouped member through the existing #334 kernels, exactly like the sync path does:

• grouped_hidden_g4_dual + grouped_down_g4, one launch pair per group;
• silu is fused in the dual kernel's epilogue (a03c79e), so there is no separate silu_mul launch — gate[] already holds silu(g)*u and up[] is never written;
• per-row (fmt=2) members ride along as the ng=1 special case, per-row-only groups keep their existing w4 branch, odd-gs groups keep falling through (and the generic path still refuses to decode fmt=4 as int2, fmt=4 CUDA attention crash — FIXED (#298); benchmark tracker + follow-ups (PROFILE accounting, cache warmth) #334).

Net change in backend_cuda.cu: one dispatch branch in the async function, mirroring the sync path.

Test: repaired oracle + new API phase

tests/test_grouped_g4_cuda.cu had gone stale: it was written against the pre-fusion kernels and still checked gate[] against the raw matmul and read up[]. On current dev it fails with 14,400 mismatches — it seems no CI runs
it (needs a GPU). This PR:

1. repairs the oracle for the fused epilogue (expectation is now silu(g)*u, up[] unchecked), and
2. adds a public-API phase: coli_cuda_tensor_upload_g + sync coli_cuda_expert_group vs async issue/take on mixed fmt=4/fmt=2 groups with mixed row counts. The async result must match the sync result bit for bit, and both must match the f64 CPU oracle.

The API phase fails on dev without the fix (776 mismatches) and passes with it — it guards exactly this regression.

nvcc -O2 -std=c++17 -arch=native tests/test_grouped_g4_cuda.cu -o tests/test_grouped_g4
./tests/test_grouped_g4

Validation

Check	Result
Oracle phase (gs=64 + tail group + per-row member, 50 trials)	0 mismatches
API phase: sync vs f64 oracle	0 mismatches
API phase: async vs sync	bit-identical
Same test on unfixed dev	FAIL (as intended)
End-to-end, downstream: Qwen3.6-35B-A3B gs64 int4 container, CPU vs CUDA VRAM expert tier (2 GPUs, async path, ~10k expert launches)	logits cosine 0.9999999 (1 token), 1.0000000 after 24 greedy tokens, token-identical text

Hardware: RTX 3070 (sm_86) + Quadro RTX 4000 (sm_75), CUDA 12.x, Linux.

Notes

• coli_cuda_expert_group_resident_issue is untouched: it cleanly rejects
  fmt=4 (if(!all_s4) return 0) and callers fall back, so there is no wrong
  output there — extending it to the g4 kernels is a natural follow-up if
  wanted (same pattern, silu already fused).
• Context: this also unblocks grouped-int4 containers on the qwen36 CUDA VRAM
  expert tier proposed in feat(qwen36): CUDA VRAM expert tier — heat-based placement across GPUs via the shared CUDA backend #713 (the tier only uploads per-row today, precisely
  because of this gap).

Thanks on Claude Fable 5 :-)

[kreuzzelg]

Sharing the measurements behind this fix, so the numbers are in one place. All on the same box (Threadripper PRO 3945WX, 12C, 94 GB RAM, RTX 3070 8 GB + Quadro RTX 4000 8 GB), same prompt, 200-token decode. colibri = qwen36 engine + CUDA VRAM expert tier (#713 / fork branch gs64-gpu, which includes this PR's fix — required for correct gs64 output on the async decode path). Cold = fresh heat file, warm = second run with learned heat. Ollama baseline = Qwen3.6-35B-A3B Q4_K_M (note: different quantization scheme than colibri's int4, so treat cross-system rows as system-level, not quant-level comparisons).

tok/s cold / warm	Ollama (both GPUs)	colibri, Quadro RTX 4000 only	colibri, RTX 3070 only	colibri, both GPUs
per-row int4 (Qwen3.6)	10.31 / 10.56	9.17 / 9.62	9.32 / 10.90	11.20 / 11.32
gs64 int4 (Qwen3.6)	–	9.16 / 10.40	9.28 / 11.55	10.23 / 11.09
gs64 int4 (KAT-Coder-V2.5-Dev)	–	9.28 / 10.36	9.55 / 11.72	10.37 / 11.33
experts resident in VRAM (gs64)	–	3,935 (38 %)	3,618 (35 %)	7,502 (73 %)
VRAM hit rate cold / warm (gs64)	–	39 % / 93–97 %	36 % / 91–96 %	75 % / 100 %
VRAM used	14.4 GiB	6.6 GiB	6.6 GiB	13.2 GiB
peak RAM	40 GB (system)	~40 GB RSS	~41 GB RSS	~30 GB RSS
TTFT warm (49-token prompt)	~0.9 s	2.1 s	1.8 s	1.9 s

Takeaways:

• With this fix, gs64 runs at full tier speed — warm it matches or beats per-row on every configuration, while cutting first-token logit error by ~44 % vs per-row (A/B against an int8 anchor). Before the fix the async path silently produced wrong output for fmt=4.
• A single fast 8 GB card is enough to beat the dual-GPU Ollama baseline: the RTX 3070 alone decodes gs64 at 11.55–11.72 tok/s warm vs 10.56 for Ollama using both GPUs — despite holding only 35 % of the experts (misses are cheap: overlapped CPU AVX2 work).
• The single 3070 even edges out colibri's own dual-GPU setup (11.55 vs 11.09 warm): the slower Turing card sets the pace in the 2-GPU group. Two GPUs still win on cold-start hit rate and RAM footprint (30 vs 41 GB RSS).
• Containers: qwen36-35b-a3b-colibri-i4-gs64, kat-coder-v2.5-dev-colibri-i4-gs64. Raw logs and methodology in the fork's benchmark workspace; happy to share details on request.

Thanks on Claude Fable 5 :-)

[JustVugg]

Rebase request — and this is the one I most want landed of anything currently open.

applies a single per-row scale to a per-group-scaled container: silently wrong output, no error

That hits today, on the container the README actively recommends. GLM-5.2 gs64 through the CUDA async expert-group path produces wrong numbers with nothing to indicate it. The sync path has been correct since #334, so the failure is invisible to anyone comparing against a sync run — and it is exactly the shape we spent this week finding elsewhere: a fallback that silently does the wrong arithmetic instead of refusing.

dev has moved a lot since you opened this (v1.4.0 plus roughly twenty merges since), so it now conflicts. What changed on our side that makes the rebase cheaper than it was:

• c/Makefile no longer holds a hand-written TEST_BINS line — gates derive from the build rules (build: derive TEST_BINS from the build rules, not the .c files (corrects #731) #733) — so if your only Makefile conflict was appending a test, just drop the entry.
• Merge order is oldest-clean-first now. Once this is green it goes ahead of anything opened after it, which for this PR means straight away.

If the conflict turns out to be in the CUDA backend rather than mechanical, say so and I will look at it with you — for a silent-wrong-output bug I would rather do the work than let it queue.

One question while you are in there, and it may already be answered by your patch: does the same gap exist for fmt=5 (int3-g64) and fmt=8 (fp8-e4m3, block scales) on that async path? Both are group- or block-scaled and would be wrong in the same way under a gs=0, ng=1 fallback. If the dispatch is a whitelist rather than a refusal, the next format to arrive inherits the bug — and fmt=8 just gained a UE8M0 scale variant in #779, so there is now one more way to reach it.

Separately: #712 is still waiting on the front-end wiring — model_arch, engine_for, and a serve loop — before Qwen3.6 can be driven by coli chat/web/serve. Details are in my comment there. No rush, but this one is the more urgent of your two.

[kreuzzelg]

Rebased onto current dev (72ddb67). The conflict was only the locals block in coli_cuda_expert_group_issue colliding with the new E8 handling — the dispatch branch itself applied cleanly, and this PR never touched the Makefile. Re-ran locally on sm_86: the grouped-g4 oracle (kernel phase + the public-API sync-vs-async phase), make cuda-test, and the ragged-attention test — all green.

On fmt=5 / fmt=8: today neither can reach the async path (or the sync one), because they never become CUDA tensors at all — qt_cuda_upload in colibri.c refuses both at the gate, and any format row_bytes() doesn't know fails the upload on a zero allocation. So no wrong output ships today.

But you're right about the shape. The async fallback refused nothing: anything that did arrive would run quant_matmul with gs=0, ng=1 — per-row-scale semantics — and be silently wrong exactly the way fmt=4 was. The sync generic branch refused fmt=4 by name, a whitelist of one. I've added a second commit that turns both into refusals: fmt>3 in either fallback returns 0 and the caller falls back to CPU — slower, never wrong. When fmt=5 or fmt=8 (or the #779 UE8M0 variant) gain CUDA tensors, they inherit the refusal instead of the bug, and whoever adds the kernel has to route them deliberately, the same way fmt=4 is routed now.