Request for SGLang code you may have for using b12x for DeekSeek v4 Flash inference

[arcticjoe]

Hello,
I'm trying to run DeepSeek-V4-Flash-NVFP4 on a 2× DGX Spark (GB10, sm_121, ARM64) using your b12x SGLang stack. I built from local-inference-lab/sglang@main (c662a9fd) + b12x@f9226c99 (master) but hit a crash at weight load: the public fork calls prepare_w4a16_mxfp4_native_weights which was renamed to prepare_w4a16_modelopt_native_weights in b12x on May 25.
I wonder if your private lukealonso/sglang fork presumably has the updated glue, if yes, would you be able to share access, or point me to the sglang overlay commit that pairs with current b12x master?
Also, many thanks for all you work on helping us squeeze more performance out of our sm12x chips!
Cheers,
Gus

[SeedSource]

+1 on this from another Spark corner — we're attempting GLM-5.1 (REAP NVFP4) on 4x GB10 and hit the same wall: b12x vets clean and installs fine, but the sglang-side registration/adapter layer only exists in the voipmonitor x86_64 images. @lukealonso if there's a private sglang overlay or any published patch-set, we're happy to test/port on aarch64 and report back — 4-node rig, same-day turnaround. Meanwhile we're trying the vLLM route from PR #40082. Will share whatever we get working either way.

[milizhang]

@SeedSource Have you checked the repo?

[SeedSource]

@milizhang — yes, thanks for the pointer, and I went back and re-checked it today since it's clearly moving fast (8 commits since my first look, including the new Lucifer DS4-Flash/CUTLASS image).

Honest read for our situation: the images target SM120 + x86_64 — every Dockerfile pins TORCH_CUDA_ARCH_LIST=12.0a / CMAKE_CUDA_ARCHITECTURES=120a (the sglang one is even labelled "RTX Blackwell (sm_120a)"), and the cuBLAS/CUDA/NCCL paths are all x86_64-linux. Our boxes are GB10 / SM121 / aarch64, so the prebuilt images won't run for us as-is — two gaps, the arch (12.0a vs 12.1a) and the platform.

That said, the recipe is exactly the reference we were missing for the DeepSeek-v4-Flash path — the Lucifer branch, the b12x↔SGLang integration, and the specific cherry-pick set (CUTLASS NVFP4 GEMM #21314, FP4 KV #21601, Triton MLA FP8 KV #20479, the arch-suffix fix #19963, FlashInfer #2913) plus the patches/ overlay. That's portable; the work on our side is rebuilding the stack for sm_121a/aarch64 rather than reinventing the integration.

We'll attempt the port on our 4-node rig and report back what does/doesn't carry over to SM121 — could be a useful aarch64 data point for the repo. Appreciate you flagging it.

[SeedSource]

@milizhang — thanks again, that pointer paid off. We pulled b12x (PR #11) onto a DGX Spark GB10 / SM121 / aarch64 node and the kernels run correctly as-is — no port needed:

• NVFP4 dense GEMM: numerically exact vs the FlashInfer CUTLASS FP4 reference (cos=1.0, max_abs=0), ~1.08x its time on the Nemotron shared-expert shape.
• Compressed sparse-MLA (DSA, indexed + top-k): all 10 cases correct (cos≈0.99999), decode-shape latency actually beats the in-repo target ratio (~0.93); prefill is slower (GB10 autotune configs not present yet, only RTX-PRO-6000).

Setup was just pip install nvidia-cutlass-dsl==4.5.2 + b12x with torch 2.12 in a CUDA-13 aarch64 container; cap (12, 1) detected, zero kernel changes. (pyproject still says "SM120-only" but the README's "SM120/SM121" is the accurate one — confirmed on real GB10.)

@lukealonso — if a GB10/SM121 datapoint is useful for the repo, happy to share fuller numbers or run specific shapes. Two bench scripts at PR#11 head have arg/default snags unrelated to arch (benchmark_nsa_indexer.py references args.contiguous_q_lens; benchmark_mla.py defaults to 8 heads but unified_sm120 prefill needs heads % 16 == 0) — minor, can file if wanted.