Modernize cuDNN wrappers around the backend graph API

[maleadt]

Adds the foundation for cuDNN v9 backend/graph-based wrappers, including typed backend descriptor helpers, a small graph frontend, and graph-backed attention/attention!. Also adds modern operation names, keeps legacy fixed-function APIs and tests cordoned under legacy, and documents the new wrapper design.

Fixes #2266
Supersedes #3174

[codecov]

Codecov Report

❌ Patch coverage is 0% with 2074 lines in your changes missing coverage. Please review.
✅ Project coverage is 14.41%. Comparing base (2816631) to head (e8c2433).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
lib/cudnn/src/graph/ops.jl	0.00%	436 Missing ⚠️
lib/cudnn/src/backend.jl	0.00%	319 Missing ⚠️
lib/cudnn/src/ops/convolution.jl	0.00%	189 Missing ⚠️
lib/cudnn/src/ops/normalization.jl	0.00%	167 Missing ⚠️
lib/cudnn/test/sdpa.jl	0.00%	140 Missing ⚠️
lib/cudnn/src/ops/attention.jl	0.00%	137 Missing ⚠️
lib/cudnn/test/graph_ops.jl	0.00%	105 Missing ⚠️
lib/cudnn/src/graph/graph.jl	0.00%	82 Missing ⚠️
lib/cudnn/test/convolution.jl	0.00%	75 Missing ⚠️
lib/cudnn/test/pooling.jl	0.00%	72 Missing ⚠️
... and 14 more

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3191      +/-   ##
==========================================
- Coverage   17.42%   14.41%   -3.01%     
==========================================
  Files         124      141      +17     
  Lines        9885    11819    +1934     
==========================================
- Hits         1722     1704      -18     
- Misses       8163    10115    +1952     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

CUDA.jl Benchmarks

Details

Benchmark suite	Current: 6b7ddd1	Previous: 2816631	Ratio
array/accumulate/Float32/1d	98632 ns	97897 ns	1.01
array/accumulate/Float32/dims=1	74827 ns	73632 ns	1.02
array/accumulate/Float32/dims=1L	1599598 ns	1598835 ns	1.00
array/accumulate/Float32/dims=2	140333 ns	139330 ns	1.01
array/accumulate/Float32/dims=2L	659665 ns	659621 ns	1.00
array/accumulate/Int64/1d	118956 ns	117503 ns	1.01
array/accumulate/Int64/dims=1	78830 ns	77994 ns	1.01
array/accumulate/Int64/dims=1L	1716868 ns	1715449 ns	1.00
array/accumulate/Int64/dims=2	153367 ns	152392 ns	1.01
array/accumulate/Int64/dims=2L	986756 ns	986467 ns	1.00
array/broadcast	18286 ns	18161 ns	1.01
array/construct	870.6666666666666 ns	895.2765957446809 ns	0.97
array/copy	15944 ns	15839 ns	1.01
array/copyto!/cpu_to_gpu	208742 ns	207371 ns	1.01
array/copyto!/gpu_to_cpu	242285 ns	240102 ns	1.01
array/copyto!/gpu_to_gpu	8627 ns	10026.333333333334 ns	0.86
array/iteration/findall/bool	132246 ns	131518 ns	1.01
array/iteration/findall/int	147053 ns	145844 ns	1.01
array/iteration/findfirst/bool	69053 ns	67401 ns	1.02
array/iteration/findfirst/int	70271 ns	68594 ns	1.02
array/iteration/findmin/1d	64689 ns	61333 ns	1.05
array/iteration/findmin/2d	99889 ns	99158 ns	1.01
array/iteration/logical	188403 ns	184693 ns	1.02
array/iteration/scalar	62161 ns	60392 ns	1.03
array/permutedims/2d	49408 ns	48886 ns	1.01
array/permutedims/3d	50619 ns	50584 ns	1.00
array/permutedims/4d	50426 ns	49953 ns	1.01
array/random/rand/Float32	11286 ns	10837 ns	1.04
array/random/rand/Int64	22717 ns	23059 ns	0.99
array/random/rand!/Float32	7759.75 ns	7669.75 ns	1.01
array/random/rand!/Int64	20710 ns	20281 ns	1.02
array/random/randn/Float32	33730 ns	32667 ns	1.03
array/random/randn!/Float32	28004 ns	25627 ns	1.09
array/reductions/mapreduce/Float32/1d	31700 ns	30981 ns	1.02
array/reductions/mapreduce/Float32/dims=1	37301 ns	36898 ns	1.01
array/reductions/mapreduce/Float32/dims=1L	50283 ns	49612 ns	1.01
array/reductions/mapreduce/Float32/dims=2	54696 ns	54387 ns	1.01
array/reductions/mapreduce/Float32/dims=2L	66570 ns	65753 ns	1.01
array/reductions/mapreduce/Int64/1d	39589 ns	38322 ns	1.03
array/reductions/mapreduce/Int64/dims=1	40593 ns	39816 ns	1.02
array/reductions/mapreduce/Int64/dims=1L	88271 ns	88025 ns	1.00
array/reductions/mapreduce/Int64/dims=2	57520 ns	56769 ns	1.01
array/reductions/mapreduce/Int64/dims=2L	83290 ns	82022 ns	1.02
array/reductions/reduce/Float32/1d	31901 ns	30927 ns	1.03
array/reductions/reduce/Float32/dims=1	37540 ns	36928 ns	1.02
array/reductions/reduce/Float32/dims=1L	50004 ns	49998 ns	1.00
array/reductions/reduce/Float32/dims=2	54906 ns	54309 ns	1.01
array/reductions/reduce/Float32/dims=2L	68022 ns	67612 ns	1.01
array/reductions/reduce/Int64/1d	40029 ns	37970 ns	1.05
array/reductions/reduce/Int64/dims=1	40246 ns	39651 ns	1.02
array/reductions/reduce/Int64/dims=1L	88282 ns	88000 ns	1.00
array/reductions/reduce/Int64/dims=2	57102 ns	56538 ns	1.01
array/reductions/reduce/Int64/dims=2L	83073 ns	82016 ns	1.01
array/reverse/1d	16217 ns	16230 ns	1.00
array/reverse/1dL	69028 ns	68819 ns	1.00
array/reverse/1dL_inplace	67052 ns	66950 ns	1.00
array/reverse/1d_inplace	9867.666666666666 ns	8078.666666666667 ns	1.22
array/reverse/2d	19309 ns	19139 ns	1.01
array/reverse/2dL	72646 ns	72557 ns	1.00
array/reverse/2dL_inplace	66770 ns	66671 ns	1.00
array/reverse/2d_inplace	9777 ns	9392 ns	1.04
array/sorting/1d	2667361 ns	2666654 ns	1.00
array/sorting/2d	1043307 ns	1037930 ns	1.01
array/sorting/by	3191646 ns	3190978 ns	1.00
cuda/synchronization/context/auto	1073.1 ns	1034 ns	1.04
cuda/synchronization/context/blocking	799.7777777777778 ns	792.8947368421053 ns	1.01
cuda/synchronization/context/nonblocking	5761.166666666667 ns	5768.5 ns	1.00
cuda/synchronization/stream/auto	887.38 ns	888.7407407407408 ns	1.00
cuda/synchronization/stream/blocking	689.8466666666667 ns	680.775641025641 ns	1.01
cuda/synchronization/stream/nonblocking	5621.142857142857 ns	5479 ns	1.03
integration/byval/reference	147317 ns	147451 ns	1.00
integration/byval/slices=1	149520 ns	149601 ns	1.00
integration/byval/slices=2	292423 ns	292115 ns	1.00
integration/byval/slices=3	435296 ns	434848 ns	1.00
integration/cudadevrt	104392 ns	104306 ns	1.00
integration/volumerhs	9306828 ns	9307376 ns	1.00
kernel/indexing	12484 ns	12392 ns	1.01
kernel/indexing_checked	13255 ns	13162 ns	1.01
kernel/launch	2048.3333333333335 ns	1990.6 ns	1.03
kernel/occupancy	655.4753086419753 ns	703.027397260274 ns	0.93
kernel/rand	14232 ns	13514 ns	1.05
latency/import	3942625287 ns	3924943583 ns	1.00
latency/precompile	4690298805 ns	4669190824 ns	1.00
latency/ttfp	4909011869 ns	4902496159 ns	1.00

This comment was automatically generated by workflow using github-action-benchmark.