Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
c8d1bbd
Add optional JAX support to FE OSS APIs
mgoldfarb-nvidia Jun 30, 2026
ccf4841
Align Torch and JAX FE operation names
mgoldfarb-nvidia Jun 30, 2026
713841f
Add Torch and JAX documentation tabs
mgoldfarb-nvidia Jun 30, 2026
598b033
Simplify optional JAX API policy
mgoldfarb-nvidia Jun 30, 2026
1439267
Share JAX static value validation
mgoldfarb-nvidia Jun 30, 2026
2130e23
Support validating multiple JAX dimensions
mgoldfarb-nvidia Jun 30, 2026
340f37d
Rename shared JAX utilities module
mgoldfarb-nvidia Jul 1, 2026
2ca5acb
Use native CUTLASS TensorSpec in JAX adapter
mgoldfarb-nvidia Jul 1, 2026
cd068a3
Add JAX DSA kernel examples
mgoldfarb-nvidia Jul 1, 2026
f09767f
Delegate TensorSpec validation to CUTLASS
mgoldfarb-nvidia Jul 1, 2026
48fc53f
Simplify JAX buffer initialization
mgoldfarb-nvidia Jul 1, 2026
909cbad
Remove unused JAX caller alias support
mgoldfarb-nvidia Jul 1, 2026
7611435
Restore existing DSA architecture documentation
mgoldfarb-nvidia Jul 1, 2026
2a8ca36
Rely on JAX static shape specialization
mgoldfarb-nvidia Jul 1, 2026
fdfb4a6
Remove redundant JAX static validators
mgoldfarb-nvidia Jul 1, 2026
1b87d29
Centralize JAX optional dependency imports
mgoldfarb-nvidia Jul 1, 2026
8d053a0
Merge remote-tracking branch 'origin/develop' into mgoldfarb/cutedsl-…
mgoldfarb-nvidia Jul 1, 2026
ede0744
Keep Torch out of the JAX top-k kernel path
mgoldfarb-nvidia Jul 1, 2026
7b0b348
Co-locate Torch and JAX kernel APIs
mgoldfarb-nvidia Jul 1, 2026
1debe3a
Move RMSNorm launch config into its package
mgoldfarb-nvidia Jul 1, 2026
ca5ff45
Share lazy framework API selection
mgoldfarb-nvidia Jul 1, 2026
f2843bc
Require explicit JAX operation namespaces
mgoldfarb-nvidia Jul 1, 2026
8f39582
Auto-discover JAX import boundaries
mgoldfarb-nvidia Jul 1, 2026
611d4d2
Require JAX 0.9.1 or newer
mgoldfarb-nvidia Jul 1, 2026
1a95752
Simplify JAX namespace loading
mgoldfarb-nvidia Jul 1, 2026
f7ec0bf
Add JAX DSA score recompute APIs
mgoldfarb-nvidia Jul 1, 2026
e31341e
Add JAX dense GEMM APIs
mgoldfarb-nvidia Jul 2, 2026
8cc82d6
Share GEMM validation across Torch and JAX
mgoldfarb-nvidia Jul 2, 2026
115037a
Add JAX APIs for remaining FE-OSS kernels
mgoldfarb-nvidia Jul 2, 2026
cc91e3b
Add framework-neutral API base and JAX class APIs
mgoldfarb-nvidia Jul 2, 2026
5483abd
Co-locate JAX CuTe adapter with API base
mgoldfarb-nvidia Jul 2, 2026
b72d686
Simplify JAX API infrastructure
mgoldfarb-nvidia Jul 2, 2026
e6a46e4
Expand JAX kernel coverage and validation
mgoldfarb-nvidia Jul 6, 2026
4aea843
Add block-scaled JAX GEMM SwiGLU support
mgoldfarb-nvidia Jul 6, 2026
ee72523
Merge remote-tracking branch 'origin/develop' into mgoldfarb/api-base…
mgoldfarb-nvidia Jul 6, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 55 additions & 2 deletions docs/fe-oss-apis/attention/sdpa_bwd_d256.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,13 @@ Here, the `sum(dP \odot P)` term is reduced row-wise over the key/softmax dimens

### High-level wrapper

``````{tab-set}
:sync-group: frontend-framework

`````{tab-item} PyTorch
:sync: torch
:selected:

```python
import torch
from cudnn import sdpa_bwd_wrapper_sm100_d256
Expand Down Expand Up @@ -73,6 +80,50 @@ dq, dk, dv = result
# Key access: result["dq_tensor"], result["dk_tensor"], result["dv_tensor"]
```

`````

`````{tab-item} JAX
:sync: jax

```python
import jax
from cudnn.jax import sdpa_bwd_wrapper_sm100_d256

@jax.jit
def sdpa_bwd(q, k, v, o, do, lse):
return sdpa_bwd_wrapper_sm100_d256(
q,
k,
v,
o,
do,
lse,
mma_tiler_mn=(128, 128),
dkdv_mma_tiler_mn=(128, 64),
is_causal=False,
window_size=(-1, -1),
scale_softmax=None,
)

result = sdpa_bwd(q, k, v, o, do, lse)
dq, dk, dv = result["dq"]_tensor, result["dk_tensor"], result["dv_tensor"]
```
Comment on lines +108 to +110

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Fix the unpacking typo in the JAX example.

result["dq"]_tensor is invalid Python, so the snippet won't run if copied. It should be result["dq_tensor"] to match the documented JAX return keys.

🛠️ Proposed fix
-dq, dk, dv = result["dq"]_tensor, result["dk_tensor"], result["dv_tensor"]
+dq, dk, dv = result["dq_tensor"], result["dk_tensor"], result["dv_tensor"]
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
result = sdpa_bwd(q, k, v, o, do, lse)
dq, dk, dv = result["dq"]_tensor, result["dk_tensor"], result["dv_tensor"]
```
result = sdpa_bwd(q, k, v, o, do, lse)
dq, dk, dv = result["dq_tensor"], result["dk_tensor"], result["dv_tensor"]
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/fe-oss-apis/attention/sdpa_bwd_d256.md` around lines 108 - 110, The JAX
example in sdpa_bwd_d256 has an unpacking typo in the result handling:
result["dq"]_tensor is invalid Python and should be corrected to the proper
documented key access used by sdpa_bwd. Update the example to reference
result["dq_tensor"] alongside result["dk_tensor"] and result["dv_tensor"] so the
snippet is copy-paste runnable and consistent with the return structure shown in
the JAX API docs.


The JAX wrapper returns
`TupleDict(dq_tensor=..., dk_tensor=..., dv_tensor=...)`. It supports
fixed BHSD arrays only: Q, K, V, O, and dO have logical shape
`(B,H,S,256)`, all floating inputs except FP32 LSE share FP16 or BF16 dtype,
and `H_q % H_kv == 0`. Packed THD inputs, cumulative lengths, `max_s_*`,
explicit output buffers, and stream arguments remain PyTorch-only. Tensor
inputs are runtime operands; shapes, dtypes, tilers, mask/window selection,
and `scale_softmax` are static compilation state. XLA owns dQ/dK/dV and the
zero-initialized hidden workspace used by the two-kernel implementation. This
is an explicit backward operation and does not register a JAX autodiff rule.

`````

``````

### Class API

```python
Expand Down Expand Up @@ -201,14 +252,17 @@ sdpa_bwd.execute(

### Wrapper return values

Returns a `TupleDict` with keys:
The PyTorch wrapper returns a `TupleDict` with keys:

- `dq_tensor`
- `dk_tensor`
- `dv_tensor`

Tuple unpacking order is: `(dq_tensor, dk_tensor, dv_tensor)`.

The JAX wrapper returns `TupleDict` with the same field names and tuple
order.

### Class-specific parameters: `SdpabwdSm100D256`

#### `SdpabwdSm100D256` (constructor)
Expand Down Expand Up @@ -277,4 +331,3 @@ For runnable examples and reference-comparison checks, see:

- `test/python/fe_api/test_sdpa_bwd.py`
- `test/python/fe_api/test_sdpa_bwd_utils.py`

51 changes: 50 additions & 1 deletion docs/fe-oss-apis/attention/sdpa_fwd_d256.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,13 @@ This is available through a standalone API (documented below) and is also used b

### High-level wrapper

``````{tab-set}
:sync-group: frontend-framework

`````{tab-item} PyTorch
:sync: torch
:selected:

```python
import torch
from cudnn import sdpa_fwd_wrapper_sm100_d256
Expand All @@ -39,6 +46,45 @@ o_tensor, lse_tensor = result
# Key access: result["o_tensor"], result["lse_tensor"]
```

`````

`````{tab-item} JAX
:sync: jax

```python
import jax
from cudnn.jax import sdpa_fwd_wrapper_sm100_d256

@jax.jit
def sdpa_fwd(q, k, v):
return sdpa_fwd_wrapper_sm100_d256(
q,
k,
v,
mma_tiler_mn=(128, 128),
is_causal=False,
window_size=(-1, -1),
scale_softmax=None,
scale_output=1.0,
)

result = sdpa_fwd(q, k, v)
o_tensor, lse_tensor = result["o_tensor"], result["lse_tensor"]
```

The JAX wrapper returns `TupleDict(o_tensor=..., lse_tensor=...)`. It
supports fixed BHSD arrays only: Q, K, and V use logical shape
`(B,H,S,256)`, share FP16 or BF16 dtype, and satisfy `H_q % H_kv == 0`.
Packed THD inputs, cumulative lengths, `max_s_*`, explicit outputs, and stream
arguments remain PyTorch-only. Tensor data is runtime; shapes, dtypes,
accumulator types, tiling, mask/window configuration, and both scale values
are static compilation state. XLA supplies the runtime stream and owns both
results.

`````

``````

### Class API

```python
Expand Down Expand Up @@ -124,13 +170,16 @@ sdpa_fwd.execute(

## Wrapper return values

Returns a `TupleDict` with keys:
The PyTorch wrapper returns a `TupleDict` with keys:

- `o_tensor`
- `lse_tensor`

Tuple unpacking order is `(o_tensor, lse_tensor)`.

The JAX wrapper returns `TupleDict` with the same `o_tensor` and
`lse_tensor` field names and tuple order.

## Support surface and constraints

- `head_dim` must be exactly `256`
Expand Down
Loading