[Bug]: Segmentation fault in libcudnn_engines_precompiled.so.9.4.0 when executing fused INT8 convolution with vectCount=4

[lesaff-r]

Before submitting

• I searched existing issues and did not find a duplicate.

Affected area

C++ frontend API or graph construction

Bug report

Description

Executing a fused INT8 convolution graph with vectorized INT8x4 layout causes a host segmentation fault inside libcudnn_engines_precompiled.so.

The graph validates successfully, builds successfully, passes check_support(), builds execution plans successfully, and graph->get_workspace_size() returns 0. However, graph->execute() crashes before returning.

The same graph works correctly with scalar INT8 (vectCount = 1) and with INT8x32 (vectCount = 32).

The crash occurs inside libcudnn_engines_precompiled.so, not in user code.

Steps to reproduce

1. Create a fused graph:

   Conv
   ↓
   Scale (Mul)
   ↓
   Add (Residual)
   ↓
   Add (Bias)
   ↓
   ReLU

2. Use blocked INT8 tensors.

3. Execute the graph with the following configurations:

Data type	vectCount	Result
INT8	1	Works
INT8	4	Segmentation fault
INT8x4	4	Segmentation fault
INT8x32	32	Works

All graph construction stages succeed:

• validate()
• build_operation_graph()
• create_execution_plans()
• check_support()
• build_plans()

The crash occurs during:
graph->execute(...)

Expected behavior

Either:

• the graph executes successfully, or
• one of the build stages (check_support(), build_plans(), or execute()) returns CUDNN_STATUS_NOT_SUPPORTED.

The library should not terminate with a segmentation fault.

Actual behavior

graph->execute() causes a host segmentation fault inside:

libcudnn_engines_precompiled.so

The crash occurs before graph->execute() returns.

Compute Sanitizer reports:

========= ERROR SUMMARY: 0 errors

indicating no device-side memory error was detected.

Environment (where applicable)

cuDNN Frontend Version: 1.27.0
cuDNN Backend Version: 9.4.0
CUDA Toolkit Version: 12.6
GPU: NVIDIA GeForce GTX 3090
Driver: 560.35.03
OS: Ubuntu 24.04
Compiler: GCC 13.3

Last known working version

Unknown.

The issue has been reproduced on cuDNN Frontend 1.27.0 with cuDNN Backend 9.4.0.

Relevant logs

gdb bt
#0 0x00007fff95c5626b in ?? () from /lib/x86_64-linux-gnu/libcudnn_engines_precompiled.so.9.4.0
No symbol table info available.
#1 0x00007fff95f9aa6e in ?? () from /lib/x86_64-linux-gnu/libcudnn_engines_precompiled.so.9.4.0
No symbol table info available.
#2 0x00007fff95629c34 in ?? () from /lib/x86_64-linux-gnu/libcudnn_engines_precompiled.so.9.4.0
No symbol table info available.
#3 0x00007fff9562a71c in ?? () from /lib/x86_64-linux-gnu/libcudnn_engines_precompiled.so.9.4.0
No symbol table info available.
#4 0x00007fff9563b721 in ?? () from /lib/x86_64-linux-gnu/libcudnn_engines_precompiled.so.9.4.0
No symbol table info available.
#5 0x00007fffdc724b68 in cudnn::backend::execute(cudnnContext*, cudnn::backend::ExecutionPlan const&, cudnn::backend::VariantPack&) () from /lib/x86_64-linux-gnu/libcudnn_graph.so.9
No symbol table info available.
#6 0x00007fffdc724c5b in cudnnBackendExecute () from /lib/x86_64-linux-gnu/libcudnn_graph.so.9
No symbol table info available.
#7 0x00005555555c908b in cudnn_frontend::detail::execute(cudnnContext*, cudnn_frontend::ExecutionPlan_v8*, void* const*, std::vector<long, std::allocator > const&, void*) ()
No symbol table info available.
#8 0x00005555555d5ef7 in cudnn_frontend::graph::Graph::execute_plan_at_index(cudnnContext*, void**, int, void*, long, std::vector<long, std::allocator > const&, std::vector<std::vector<long, std::allocator >, std::allocator<std::vector<long, std::allocator > > > const&, std::vector<std::vector<long, std::allocator >, std::allocator<std::vector<long, std::allocator > > > const&) const ()
No symbol table info available.
#9 0x00005555555d7b7e in cudnn_frontend::graph::Graph::execute_plan_at_index(cudnnContext*, std::unordered_map<long, void*, std::hash, std::equal_to, std::allocator<std::pair<long const, void*> > >&, void*, long, std::vector<long, std::allocator > const&, std::vector<std::vector<long, std::allocator >, std::allocator<std::vector<long, std::allocator > > > const&, std::vector<std::vector<long, std::allocator >, std::allocator<std::vector<long, std::allocator > > > const&) const ()
No symbol table info available.
#10 0x00005555555d8185 in cudnn_frontend::graph::Graph::execute(cudnnContext*, std::unordered_map<std::shared_ptr<cudnn_frontend::graph::Tensor_attributes>, void*, std::hash<std::shared_ptr<cudnn_frontend::graph::Tensor_attributes> >, std::equal_to<std::shared_ptr<cudnn_frontend::graph::Tensor_attributes> >, std::allocator<std::pair<std::shared_ptr<cudnn_frontend::graph::Tensor_attributes> const, void*> > >&, void*) const ()
No symbol table info available.
#11 0x00005555555614e5 in main ()
No symbol table info available.

cudnn.log