The cuDNN frontend has added grouped GEMM fusions, such as grouped GEMM + swiglu + MXFP8 quantize. However, these are only accessible through the public API here, which depends on torch and torch Tensors.
https://github.com/NVIDIA/cudnn-frontend/blob/97f6cb3b88cacff507cca1280db5650a457d92b3/python/cudnn/grouped_gemm/grouped_gemm_swiglu/api.py
Since this depends on torch Tensors, we cannot call this from JAX/XLA. So we’d like one of two options:
- CuTeDSL kernel is made public directly and a stable direct CuTeDSL kernel interface is committed to from cuDNN frontend team. JAX can call CuTeDSL kernels directly, it’s the public API torch Tensor wrapper that is problematic for us.
- Expose this functionality through the C++ cuDNN frontend API. We can call the C++ cuDNN frontend API from JAX/XLA, but not the cuDNN frontend Python API as it depends on torch Tensors
The cuDNN frontend has added grouped GEMM fusions, such as grouped GEMM + swiglu + MXFP8 quantize. However, these are only accessible through the public API here, which depends on torch and torch Tensors.
https://github.com/NVIDIA/cudnn-frontend/blob/97f6cb3b88cacff507cca1280db5650a457d92b3/python/cudnn/grouped_gemm/grouped_gemm_swiglu/api.py
Since this depends on torch Tensors, we cannot call this from JAX/XLA. So we’d like one of two options: