Update dependency flashinfer-python to v0.6.14

[konflux-internal-p02]

ℹ️ Note

This PR body was truncated due to platform limits.

This PR contains the following updates:

Package	Change	Age	Confidence
flashinfer-python	==0.2.11 → ==0.6.14

---

Release Notes

flashinfer-ai/flashinfer (flashinfer-python)

v0.6.14

Compare Source

Manual Intervention Needed ⚠️

Please modify your install instructions as you bump the version to 0.6.14.

pip install flashinfer-python
 
pip install flashinfer-cubin --index-url https://flashinfer.ai/whl     # this is the difference
 
pip install flashinfer-jit-cache --index-url https://flashinfer.ai/whl/cu129

# OR 
pip install flashinfer-jit-cache --index-url https://flashinfer.ai/whl/cu130

See also #​3808

Highlights

This release pushes FlashInfer's coverage onto Blackwell RTX PRO and DGX Spark silicon, lands the CuTe-DSL rewrite of the gated-delta-rule (GDN) kernels behind Qwen3.5/3.6, and makes those kernels production-ready inside vLLM.

Gemma 4 and W4A16 on Blackwell RTX Pro and DGX Spark

Blackwell SM12x parts (RTX PRO 6000, DGX Spark / GB10) previously trailed Blackwell GB200 NVL72 (and B300 SM103) on attention shape and quantization coverage. This release closes the gap for Gemma 4 and weight-only-quantized inference: head_dim=512 attention for Gemma 4's global layers now runs on SM120/121, the FMHAv2 prefill path gains head_dim=256/512 and sliding-window masking on SM120, the new mm_bf16_fp4 W4A16 GEMM is tuned for DGX Spark (completing W4A16 across both dense GEMM and MoE), and gated tanh-GELU brings Gemma 4 MoE onto the CUTLASS backend.

• #​3576
• #​3518
• #​3597
• #​3501

GDN / gated delta rule: CuTe-DSL overhaul (Qwen3.5/3.6)

The gated-delta-rule kernels behind the Qwen3.5/3.6 family were rewritten from CUTLASS C++ to CuTe-DSL (#​3491). The rewrite eliminates the C++ JIT compilation pain reported by customers and establishes the base for context-parallel delta-rule kernels — covering SM90 prefill and its context-parallel variant, SM120 prefill, and a ~20–25% GDN prefill speedup from mainloop efficiency work.

• #​3477
• #​3481
• #​3479
• #​3742

GDN production-ready in vLLM

Two gaps blocked GDN serving in vLLM (#​3602); both are now resolved. GDN kernels are compilation batch-size agnostic, so a single compiled cubin is reused across batch shapes instead of recompiling on every new batch size at inference time, and a new BF16 state recovery/decode kernel writes SSM state into preallocated space to supply the MTP-compatible spec-decode path vLLM needs.

• #​3649
• #​3502

DeepSeek-class sparse MLA on Blackwell, FP8 KV on Hopper

New sparse-MLA paged-attention kernels extend the DeepSeek-V4 (d_qk=512) and DeepSeek-V3.2 / GLM-5.1 (d_qk=576) families onto SM120/121 through the existing flashinfer.mla APIs, with DSv4 coverage broadened to 8/16/32 head counts. On Hopper SM90, native FP8 KV cache support eliminates SGLang's per-layer cast workaround, saving an HBM round-trip per layer on DeepSeek-V3/V4 while staying bit-identical to the BF16 path.

• #​3395
• #​3545
• #​3694

This release pushes FlashInfer's coverage onto Blackwell RTX PRO and DGX Spark silicon, lands the CuTe-DSL rewrite of the gated-delta-rule (GDN) kernels behind Qwen3.5/3.6, and makes those kernels production-ready inside vLLM.

What's Changed

• fix: use explicit .ptr on scalar struct fields in monolithic MLA decode to get cleaner logs by @​jdebache in #​3458
• fix: One sided MOE A2A warp token policy hangs in some cases, disable… by @​djns99 in #​3371
• test: correct XQA NVFP4 skip reason to SM120/SM121 by @​deng451e in #​3510
• fix XQA NVFP4 head dim by @​Njuapp in #​3534
• tests: split test_trtllm_gen_attention.py into prefill / decode / decode-xqa shards by @​bkryu in #​3162
• docs(misc): expose DiT/RoPE norms; new RSTs for GDN decode/prefill/Mamba by @​kangbintNV in #​3446
• perf: cache cudaGetDeviceProperties in CudaDevice (fmha_v2) by @​aws-jiadingg in #​3522
• FMHAv2 on SM120 for head_dim 256/512 + sliding-window masks by @​dbari in #​3518
• feat: add FlashInfer Trace Apply by @​yongwww in #​3240
• Fix smem race in FilteredTopK overflow refinement by @​awgu in #​3529
• Extend Require Workspace Size and Disable Overlapping Criritcal Workspace Sections w/ 'auto' backend + autotuning for MLA decode by @​Vinnie6167 in #​3465
• perf(sampling): Optimize top_k_top_p_sampling_from_logits/from_probs for large-vocab small-k sampling by @​bkryu in #​3461
• Use an environment variable to control the number of reserved SMs for overlapping in TRT-LLM fused MoE by @​jinyangyuan-nvidia in #​3483
• Remove excessive CuteDslMoEWrapper memory allocation by @​nvjullin in #​3404
• docs: close v0.6.13 doc-check gaps + fix(moe) misleading topk_indices ICHECK message by @​kangbintNV in #​3546
• Unified MoE API: MoELayer with cross-backend NVFP4 autotune by @​aleozlx in #​3093
• Support smaller DSv4 sparse MLA head counts by @​PerkzZheng in #​3545
• feat(bench): add GDN routines to flashinfer_benchmark.py by @​bkryu in #​3572
• Add BF16 MoE SwiGLU OA params by @​ruoqianguo in #​3532
• perf(gemm): update mm_fp4 b12x SM120 NVFP4 dense GEMM kernel by @​yichengj0 in #​3560
• feat: add sm90 delta rule dsl prefill by @​guangyunh-nv in #​3477
• fix: headDim=512 GQA decode (#​3343) by @​djmmoss in #​3393
• Add Qidi Sang to CODEOWNERS by @​saltyminty in #​3593
• [feat] Add gated tanh-GELU (GeluTanh) activation to CUTLASS fused MoE (GEMMA 4) by @​jhaotingc in #​3501
• [chore] Add jiahanc to GDN commiter by @​jiahanc in #​3600
• fix(ci): skip TRT-LLM Gen BF16 MoE SwiGLU test outside SM10.x by @​ruoqianguo in #​3580
• fix(mla): warn when backend='auto' falls back to non-Blackwell kernel on SM>=100 by @​elwhyjay in #​3405
• [fix] Fix SM100 GDN prefill hang by @​jiahanc in #​3581
• feat: improve gated delta rule benchmark script by @​guangyunh-nv in #​3616
• feat: delta rule work with zero length sequence by @​guangyunh-nv in #​3536
• feat(cutile): introduce cuTile backend (mm_bf16 + bmm_bf16 + gemm_fp8_nt_groupwise) by @​yifeis-nv in #​3426
• chore: remove leftover cpp srcs from #​3477 by @​guangyunh-nv in #​3613
• feat: add mxfp8 quant to moe a2a combine by @​IwakuraRein in #​3376
• fix: scope trtllm-gen block_size guard to its own backend by @​elwhyjay in #​3428
• feat(attention): add SM120 sparse MLA kernels by @​lucifer1004 in #​3395
• test: match test output scaling with cutedsl kernel (scale before bf16 cast) by @​saltyminty in #​3596
• fix(sampling): fix shared-memory race and out-of-range token id in SamplingFromLogitsKernel by @​kahyunnam in #​3624
• feat(attention): head_dim=512 support for attention prefill & decode for Gemma 4 on SM120/121 by @​bkryu in #​3576
• [fix] explicitly validate grouped_gemm_nt_masked layout contract to prevent NaN contamination by @​kahyunnam in #​3574
• fix(gemm): relax b12x FP4 K constraint from 128 to 32 (TMA alignment) by @​yichengj0 in #​3646
• Add MXFP8 MoE SwiGLU OA parameters by @​ruoqianguo in #​3504
• Add support for non-multiple VEC_COLS in fused_rmsnorm_silu for bf16/fp8 by @​xueweilnvidia in #​3417
• fix(topk): eliminate multi-CTA radix top-k stream hangs on SM120/SM121 by @​waynehacking8 in #​3615
• feat: add sm120 delta rule dsl prefill by @​guangyunh-nv in #​3479
• Add SM120 NVFP4 attention JIT path by @​tiffany940107 in #​3640
• Add MXFP8 MoE GEMM entry (cute SM120 backend) by @​CarstyYou in #​3562
• feat: Add BF16_FP4 GEMM with cuDNN and CuTe-DSL backends for SM120/121 for W4A16 workloads by @​bkryu in #​3597
• perf(gdn): make GDN kernels compilation batch-size agnostic (support dynamic batch shapes for vLLM integration) by @​kahyunnam in #​3649
• feat: add sm90 cp delta rule dsl by @​guangyunh-nv in #​3481
• fix(sampling): terminate top-p search at adjacent float bounds by @​djmmoss in #​3623
• fix: fix tinygemm barrier bug by @​yweng0828 in #​3630
• feat(gdn): BF16 state recovery/decode kernel with per-request K and f… by @​ameynaik-hub in #​3502
• feat(benchmark): add MLA --mla_is_var_seq / --mla_cute_dsl_impl knobs by @​lunarz-dev in #​3695
• Enable SwapAB in mm_fp4 cute-dsl backend when M is not a multiple of 8. by @​b8zhong in #​3667
• fix(moe): Fix unbounded weight-cache growth in b12x MoE by @​bkryu in #​3709
• Add Relu2 + ungated MoE to CuteDSL MoE by @​b8zhong in #​3642
• trtllm_batch_decode_with_kv_cache_mla trtllm-gen backend cum_seq_lens_q support by @​saltyminty in #​3238
• fix: pass XQA NVFP4 scale-factor strides by @​Njuapp in #​3608
• feat(attention): FP8 KV cache support for Hopper SM90 MLA by @​qsang-nv in #​3694
• [GDN] sm100: support more state dtype by @​Observer007 in #​3715
• feat: cuTile Grouped MXFP8 Quantization by @​philipphack in #​3657
• Accept uint8 workspaces in CUTE DSL MLA decode by @​leejnau in #​3599
• tests: split test_trtllm_gen_fused_moe.py into shards by @​feih-nv in #​3635
• perf(moe): Enhance CuteDSL NVF4 MOE Perf by @​liyuhannnnn in #​3564
• fix(mamba): reject SM120/SM121 in SSDCombined with a clear error by @​waynehacking8 in #​3668
• docs(gemm): add missing .rst entries for mm_bf16_fp4 and prepare_bf16_fp4_weights by @​kangbintNV in #​3710
• docs: add missing parameter entries to docstrings and env vars by @​kangbintNV in #​3627
• rename back by @​aleozlx in #​3730
• docs: document scale_major_mode param and FLASHINFER_AUTOTUNE_DIR env var by @​kangbintNV in #​3696
• feat: add mxfp4/nvfp4 quant to moe a2a combine by @​IwakuraRein in #​3643
• chore: fix enable_pdl for trtllm-gen routing and finalize kernel by @​IwakuraRein in #​3588
• Bug fix (gdn): Layout contract fix from #​3649 by @​kahyunnam in #​3693
• Ameyn/fix fp32 mtp pool out indices by @​ameynaik-hub in #​3490
• Prune moe tests by @​aleozlx in #​3733
• Fix the output allocation consistency of trtllm-gen MoE APIs by @​b8zhong in #​3678
• feat(comm): Support per-token LoRA Info in MoE a2a comm payloads by @​JyChang012 in #​3375
• docs(gdn): document missing ssm_state_indices param in gated_delta_rule_mtp by @​kangbintNV in #​3725
• feat(moe): enable DSFp8 + LoRA delta path by @​zetacat in #​3708
• Improve GDN prefill perf by ~20-25% (mainloop efficiency) by @​jhjpark in #​3742
• test: fix CUDA OOM in batch-prefill custom-mask test on 24GB CI GPUs by @​waynehacking8 in #​3609
• Remove pagesize 16/32 assertion from xqa nvfp4 sm120 by @​Njuapp in #​3724

New Contributors

• @​deng451e made their first contribution in #​3510
• @​Njuapp made their first contribution in #​3534
• @​aws-jiadingg made their first contribution in #​3522
• @​awgu made their first contribution in #​3529
• @​ruoqianguo made their first contribution in #​3532
• @​yichengj0 made their first contribution in #​3560
• @​jhaotingc made their first contribution in #​3501
• @​elwhyjay made their first contribution in #​3405
• @​yifeis-nv made their first contribution in #​3426
• @​lucifer1004 made their first contribution in #​3395
• @​waynehacking8 made their first contribution in #​3615
• @​tiffany940107 made their first contribution in #​3640
• @​CarstyYou made their first contribution in #​3562
• @​feih-nv made their first contribution in #​3635
• @​liyuhannnnn made their first contribution in #​3564
• @​JyChang012 made their first contribution in #​3375
• @​zetacat made their first contribution in #​3708

Full Changelog: flashinfer-ai/flashinfer@v0.6.13rc2...v0.6.14

v0.6.13

Compare Source

What's Changed

• Run high-likelihood OOM culprits separately, record memory usage and test duration for analysis by @​dierksen in #​2961
• fix(autotuner): differentiate file cache entries by runner specific kernel parameters by @​qiching in #​3367
• feat: integrate cute-dsl Blackwell GQA decode into BatchDecodeWithPagedKVCacheWrapper by @​richardmcai in #​3360
• Fix returning reference to temporary in moe gemm by @​benbarsdell in #​3332
• Enable compression of GPU device binaries by @​benbarsdell in #​2949
• feat: MNNVL Allreduce quant fusion and performance optimization by @​timlee0212 in #​3385
• fix(norm): widen address arithmetic to int64 for large contiguous inputs > 2**31 elements by @​bkryu in #​3392
• MLA Decode Autotuning Across TRTLLM-Gen and CuTe Backends by @​Vinnie6167 in #​3355
• profiler: group perfetto traces by SM, one row per block by @​Edenzzzz in #​3038
• Make cute dsl mxfp8/nvfp4 quantizer bitwise exact by @​zianglih in #​3387
• Extend autotuner delay kernel length by @​yanqinz2 in #​3373
• Enable smaller tile N for SM100 Cute-DSL NVFP4 GEMM by @​b8zhong in #​3403
• test: align test_fmha_v2_prefill SM gating with is_sm12x_supported by @​leonardHONG in #​3182
• feat(autotuner): enable per-op autotune bypass for faster framework warmup by @​qiching in #​3396
• bench: Unify PDL behavior, add missing norm routines, and misc improvements by @​bkryu in #​3435
• feat(kda): add recurrent KDA decode kernel with per-K gating by @​djmmoss in #​2572
• Add mHC post mapping and pre big-fuse kernels by @​jmydurant in #​3285
• [Feat] Add num_heads < 128 support for mla decode kernel by @​Observer007 in #​3309
• Fix cross-warp race in checkpointing SSU kernel (mamba) by @​ishovkun in #​3439
• feat(trace): add check callbacks to trace templates by @​yyihuang in #​3330
• Add BGMV MoE CUDA kernels for multi-LoRA by @​taehokim20 in #​3249
• replace deprecated APIs: cute.make_fragment and cute.core.ThrMma by @​brandon-yujie-sun in #​3430
• [ci feat] Support /bot run tests/ to scope CI test runs by @​kahyunnam in #​3422
• perf(attention): Speed up FP8 KV-cache prefill (FA2 BatchPrefill) by repacking K/V to BF16 in shared memory by @​bkryu in #​3485
• Ep api design -- Adding the actual code and tests by @​Anerudhan in #​3453
• Optimize mxfp8 quantization on sm100 by @​IwakuraRein in #​3289
• Fix silent no-op autotuning for cuBLAS bmm_fp8 and cuDNN bmm_fp8/mm_fp4 by @​bkryu in #​3437
• fix(quantization): nvfp4_quantize(backend='cuda') silently corrupts scale factors when global_scale is not float32 by @​bkryu in #​3497
• feat(moe): add SWIGLUSTEP activation to CUTLASS fused MoE by @​bkryu in #​3492
• Support LSE buffers in TRTLLM API by @​saltyminty in #​3410
• NFC: replace deprecated API: cute.make_fragment by @​brandon-yujie-sun in #​3473
• feat(moe): write routing_replay_out from custom routing kernels by @​jdebache in #​3382
• Add CuTe DSL NVFP4 quantization with 4over6 FP16 scoring by @​zianglih in #​3448
• fix intermittent exit 141 (SIGPIPE) in test resource summary by @​yongwww in #​3498
• add_cudnn_mxfp8 by @​yanqinz2 in #​3489
• fix: use routedScalingFactor to initialize mRouteScale by @​yweng0828 in #​3499
• fix: Simple approach to restore support for bias for fp4 block scale types by @​djns99 in #​3416
• docs(infra): document env vars, refresh SM list, fix stale paths in CLAUDE.md by @​kangbintNV in #​3440
• docs(quant/sampling/activation): canonicalize quantization.rst, stub fp4_quantization.rst by @​kangbintNV in #​3447
• docs(gemm): document GEMM + grouped_mm public surface; canonicalize aliases by @​kangbintNV in #​3442
• Optimize MoE routing top-k reduction and non-power-of-two sorting by @​jiahanc in #​3476
• [bugfix] TorchDistBackend.bcast uses global rank instead of local rank by @​xuanyu-mistral in #​3418
• docs(comm): structural RST refactor for MoeAlltoAll/DCP/Mixed comm by @​kangbintNV in #​3445
• [docs] Backfill missing docstrings and decorators across kernels by @​kangbintNV in #​3456
• docs(attention): backfill missing/stale Attention/POD/cuDNN/CuteDSL APIs; restore single_prefill_with_kv_cache_return_lse by @​kangbintNV in #​3441
• docs(moe): close fused_moe / trtllm_*_moe / CuteDSL MoE doc gaps by @​kangbintNV in #​3443
• docs(comm): NumPy-style docstrings + Deprecated leads for 21 STALE comm APIs (no decorator changes) by @​kangbintNV in #​3444
• bump version to 0.6.13 by @​aleozlx in #​3513

New Contributors

• @​richardmcai made their first contribution in #​3360
• @​jmydurant made their first contribution in #​3285
• @​taehokim20 made their first contribution in #​3249
• @​brandon-yujie-sun made their first contribution in #​3430
• @​xuanyu-mistral made their first contribution in #​3418

Full Changelog: flashinfer-ai/flashinfer@v0.6.12rc3...v0.6.13

v0.6.12

Compare Source

What's Changed

• Loosened trtllm_ragged_attention_deepseek shape assertion by @​nvjullin in #​3064
• Update moe gemm by @​IwakuraRein in #​3239
• perf: optimize per-token nvfp4 quantization kernel. by @​IwakuraRein in #​3237
• build: add sccache-backed jit-cache builds and AOT diagnostics by @​dierksen in #​3205
• non-override tactic control by @​yanqinz2 in #​3260
• ci(jit-cache): limit sm110 builds to aarch64 by @​dierksen in #​3275
• feat(moe): add SM120 W4A16 b12x kernels by @​lukealonso in #​3271
• Add dynamic tokens-per-page TRTLLM-GEN GQA kernels by @​PerkzZheng in #​3259
• fix(cute_dsl/moe): unbias autotuner profiling for tile_size enumeration by @​leejnau in #​3252
• Support Kimi K2.5 H64 CuTe DSL MLA decode by @​saltyminty in #​3235
• feat: FP8 output support for CUTLASS MLA paged attention by @​carlyou in #​2779
• fix(jit): propagate -DNDEBUG to host-side cflags by @​arpera in #​3278
• feat: add SM120 fmha_v2 kernels to AOT pip wheel builds by @​blake-snc in #​2885
• bench(moe_deepseek): fix moe benchmark (supersedes #​2886) by @​leejnau in #​3292
• fix(gdn_decode): widen pool indices to Int64 to prevent int32 element-offset overflow by @​vadiklyutiy in #​3230
• [chore] Add guard to blackwell GDN prefill by @​jiahanc in #​3267
• fix: remove over-strict K%4 assert in get_shuffle_matrix_sf_a_row_indices by @​jimmyzho in #​3163
• ci: isolate nightly package tests from source tree by @​dierksen in #​3274
• Fix [Spark unit test CI]: defer torch._dynamo.disable to avoid import-time crash in CI by @​kahyunnam in #​3290
• bench(moe_deepseek): scope autotune(True) to pre-warm only by @​leejnau in #​3301
• Improved simple mamba SSU kernel by @​ishovkun in #​2962
• add cuda tile dependency for cuda 13.0 by @​nv-yunzheq in #​3305
• [Fix] Fix XQA V tile reading from wrong page when nbVItersPerXIter > 1 by @​qsang-nv in #​3022
• fix: MNNVL Allreduce uses bitwise sentinel checking to avoid subnormal value issue (#​3053) by @​timlee0212 in #​3304
• Fix: remove nvfp4 llama4 blocker by @​IwakuraRein in #​3313
• [chore] add mamba codeowners list by @​jimmyzho in #​3318
• Modify release deletion command in workflow by @​aleozlx in #​3307
• Add to code owners by @​dhiraj113 in #​3326
• feat: Add CuTe DSL grouped-gemm + combine fusion support by @​nvcastet in #​2944
• fix(gdn): allow importing gdn_decode without a CUDA device by @​kahyunnam in #​3293
• feat: enable glm5 router gemm by @​b8zhong in #​3185
• fix(fmha_v2): fix FP8 V-scratch pipeline and varlen scheduler on SM90 by @​jimmyzho in #​3276
• fix typo llama routing issue in trtllm-gen moe by @​IwakuraRein in #​3303
• feat(logging,trace): cuda-graph-compatible level-5/10 logging + fi_trace template additions/fixes by @​yyihuang in #​3172
• Use cudnn 9.23 new API to query workspace with override shape by @​yanqinz2 in #​3291
• feat: Expose unpacked topk weights for routed moe (fp4) by @​aleozlx in #​2425
• Reland support lse in trtllm paged attn kernels by @​murphymatt in #​3116
• fix(CI unit tests, cute_dsl, spark): set USER env var before torch._dynamo import for unmapped UIDs by @​kahyunnam in #​3314
• feat(trace): embed runnable init() in every TraceTemplate by @​yyihuang in #​3221
• feat(cute_dsl/moe): deterministic balanced autotune profile inputs by @​leejnau in #​3286
• feat(cute_dsl/moe): add moe_output_memset_inplace dense memset wrapper by @​leejnau in #​3328
• Fix/3170 dense blockscaled sm12x by @​leonardHONG in #​3180
• test: enable bmm_mxfp8 cutlass backend coverage on SM12x by @​leonardHONG in #​3183
• Ep api design - Build Infra dependencies by @​Anerudhan in #​3315
• [feat] Add gemma RMS AR fusion by @​jiahanc in #​3322
• checkpointing_ssu kernel: fused replay + conditional state-write for Mamba2 by @​ishovkun in #​3324
• Ameyn/gdn bf16 dispatcher and 4d pool by @​ameynaik-hub in #​3268
• Update trtllm FMHA cubins by @​djmmoss in #​3317
• fix(trace): repair TGV and XQA MLA reference tests by @​yyihuang in #​3365
• feat: Add 8x4 swizzle layout support to MXFP4 and MXFP8 CuTe-DSL kernels by @​bkryu in #​3357
• Add AGENTS.md shim by @​aleozlx in #​3342
• Add list_api script by @​aleozlx in #​3341
• Support 4over6 nvfp4 for quantizer and fused MoE by @​zianglih in #​3264
• Add DeepSeek V4 sparse MLA TRTLLM-GEN kernels by @​PerkzZheng in #​3269
• Reject EP configurations in b12x MoE with a clear error by @​kahyunnam in #​3302
• fix(cute_dsl): avoid MoE wrapper runner reference cycle by @​leejnau in #​3340
• feat: Add support for LoRa delta in MOE mxint4 x bf16, MXFP8 & BF16 to trtllm backend by @​djns99 in #​3153
• Restore monolithic CuTe-DSL MLA decode alongside modular, gated by cute_dsl_impl= by @​pgera in #​3296
• feat: RMSNorm + RoPE fusion for WAN: flashinfer.diffusion_ops.fused_qk_rmsnorm_rope by @​kahyunnam in #​3148
• fix deprecation warnings from cute-dsl by @​b8zhong in #​3333
• feat(cute_dsl/moe): re-enable use_cold_l2_cache in CuteDslMoEWrapper TuningConfig by @​leejnau in #​3384
• Add torch.compile-compatible custom op for fp4_quantize by @​Kh4L in #​3081
• Replace SM120 W4A16 MoE kernels by @​lukealonso in #​3336
• bump version to 0.6.12 by @​aleozlx in #​3388

New Contributors

• @​carlyou made their first contribution in #​2779
• @​nvcastet made their first contribution in #​2944
• @​Kh4L made their first contribution in #​3081

Full Changelog: flashinfer-ai/flashinfer@v0.6.11rc1...v0.6.12

v0.6.11.post3

Compare Source

Full Changelog: flashinfer-ai/flashinfer@v0.6.11.post2...v0.6.11.post3

v0.6.11.post2

Compare Source

Full Changelog: flashinfer-ai/flashinfer@v0.6.11.post1...v0.6.11.post2

v0.6.11.post1

Compare Source

Full Changelog: flashinfer-ai/flashinfer@v0.6.11...v0.6.11.post1

v0.6.11

Compare Source

What's Changed

• trying this one character fix for main branch by @​aleozlx in #​3213
• Add git submodule update to build_backend.py by @​kahyunnam in #​3190
• fix(cute_dsl/moe): correct tile_size=256 gemm2 tactic enumeration by @​leejnau in #​3171
• Fix trace-bmm-fp8 test: B should be K-major for subword types by @​xrq-phys in #​3184
• feat: Add DiT-oriented kernels where Qk (Bmm1) type can be reinterpreted into Int8 or BFloat16 by @​xrq-phys in #​2711
• [fmha-v2] Support HND and NHD paged KV cache layouts with conditional stride handling by @​zhou-yuxin in #​2799
• [feat] Trtllm-gen Per-token Nvfp4 MoE by @​IwakuraRein in #​3027
• feat: Add cuBLASLt backend for mm_bf16 and enable multi-tactic autotuning for FP8/MXFP8 runners by @​vadiklyutiy in #​2914
• trtllm non causal support by @​saltyminty in #​3020
• feat: DiT layer norm fusions for WAN: flashinfer.diffusion_ops by @​kahyunnam in #​3157
• Refactor Part 3- Add block-per-token feature in the customized routing method by @​ChristinaZ in #​3166
• fix(cute_dsl/moe): correct off-by-one in get_max_num_tiles to match TRT-LLM by @​leejnau in #​3198
• Yanqinz/fix cudnn sm120 nan by @​yanqinz2 in #​3192
• bump version to 0.6.10 by @​aleozlx in #​3179
• fix: align is_sm120f_supported with SM12x family semantics by @​leonardHONG in #​3175
• fix: add sm_121 to TMEM column fallback map by @​leonardHONG in #​3173
• Include TinyGEMM into BF16 autotuner by @​askliar in #​3203
• fix(dcp_alltoall): require MNNVL workspace, drop broken plain-memory path by @​davidjpyu in #​3210
• Integrate CUTLASS Small Tile N Blockscaled GEMMs/Grouped GEMMs for SM120 and SM121 by @​depaulmillz in #​3152
• Fix bf16 cudnn override-shape test call signature by @​Vinnie6167 in #​3215
• Ameyn/wide vec t1 by @​ameynaik-hub in #​3147
• [Perf] Add FMHAv2 to flashinfer_benchmark.py and eliminate unnecessary H2D by @​jimmyzho in #​2841
• Fix multi-instances using same random seed by @​guyuankan in #​3102
• add_grouped_mm_operation_directory by @​yanqinz2 in #​3052
• Support Allreduce + Norm + Per-token Group Fp8 Quant Fusion by @​wzhao18 in #​3059
• [Bugfix] Fix fused MoE autotuning correctness issues by filtering clusterDimZ by @​wzhao18 in #​3227
• fix: add jitter to cubin download backoff by @​pluh-nv in #​3169
• cute_dsl/moe: drop redundant Python-side moe_sort buffer init by @​leejnau in [#​3226](https://redirect.github.co

✂ Note

PR body was truncated to here.

---

Configuration

📅 Schedule: (UTC)

• Branch creation
  • At any time (no schedule defined)
• Automerge
  • At any time (no schedule defined)

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

---

• If you want to rebase/retry this PR, check this box

---

To execute skipped test pipelines write comment /ok-to-test.

---

Documentation

Find out how to configure dependency updates in MintMaker documentation or see all available configuration options in Renovate documentation.

[konflux-internal-p02]

Autoclosing Skipped

This PR has been flagged for autoclosing. However, it is being skipped due to the branch being already modified. Please close/delete it manually or report a bug if you think this is in error.