Reject FP8/MXFP8 SDPA forward combinations exposed to a cuDNN 9.24 split-KV bug

[vedaanta]

Problem

cuDNN 9.24 on Blackwell can select a split-KV prefill kernel for FP8/MXFP8 SDPA forward graphs whose combine step encodes the result as FP16 regardless of the requested output data type. A BF16 output tensor then holds FP16 bit patterns, which reinterpret as ~1e-21 denormals — effectively a silent all-zeros output. The kernel is preferred exactly when the Q-tile grid under-fills the GPU while s_kv is long, e.g. cross-attention with Q=(1, 1024, h, 128) against K/V=(1, 62208, h, 128).

cuDNN 9.25.0 ships the kernel fix. Versions before 9.24 do not have the kernel. Since 9.24 binaries are already public, the frontend should refuse the exposed combination there rather than let it corrupt silently.

Change

• sdpa_support_surface.h: reject, on cuDNN 9.24 only (92400 <= version < 92500), FP8/MXFP8 forward graphs with BF16 output, s_q > 1, s_kv >= 4096, no diagonal band bounds, non-paged / non-ragged / non-dropout — the exact envelope in which the backend can select the split-KV prefill kernel. The error message names the bug and the fixed version. FP16 and FP8 outputs are unaffected by the bug and stay allowed, as do causal/banded, short-s_kv, paged, ragged and dropout graphs (the kernel is never selected for those).
• test/python/test_sdpa_fp8_split_kv_guard.py: build-only tests (no execution, no quantization deps) asserting the combination raises cudnnGraphNotSupportedError on 9.24 and validates cleanly on other versions, plus the neighboring allowed combinations (short s_kv, causal).

Verification (B200)

• cuDNN 9.24.0.43: the failing shape now raises cudnnGraphNotSupportedError with the actionable message (previously returned all-zeros); s_kv=512 and causal variants still build and produce correct results; BF16 (non-FP8) SDPA unaffected.
• cuDNN 9.25 / 9.26 pre-release builds with the kernel fix: guard is inert, all three tests pass, and the split-KV plan produces correct results.

🤖 Generated with Claude Code

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 496b7948-5409-472d-a37e-ca327530c746

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}