[BUG] to_parquet fatal cudaErrorIllegalAddress on large list row groups (write ceiling ~2.6x below construction ceiling)

[VibhuJawa]

DRAFT for upstream rapidsai/cudf. Crash bracket re-verified on 2026-07-30 on cuDF 25.10.00 and 26.08.00a990 (nightly), H100 80GB. Open before filing: the exact threshold is not identified — we report an empirical bracket.

Describe the bug

DataFrame.to_parquet() on a list<T> column dies with a fatal cudaErrorIllegalAddress — not a clean exception — once a single row group's leaf child column gets large enough. The CUDA context is poisoned, so the process cannot continue.

RuntimeError: Fatal CUDA error encountered at:
/__w/cudf/cudf/cpp/src/utilities/stream_pool.cpp:189:
700 cudaErrorIllegalAddress an illegal memory access was encountered

  File "pylibcudf/io/parquet.pyx", line 1346, in pylibcudf.io.parquet.write_parquet
  File "pylibcudf/io/parquet.pyx", line 1368, in pylibcudf.io.parquet.write_parquet

(Traceback from nightly; on 25.10.00 the same failure surfaces at stream_pool.cpp:200.)

The practical consequence is that the write ceiling for list columns is far below the construction ceiling. At dim=1024, float32:

	rows	leaf elements	result
can construct in cuDF	2,097,151	2,147,482,624	OK
can write to Parquet	797,696	816,840,704	OK
	798,720	817,889,280	fatal crash

So roughly 2.6× fewer rows can be written than can be held.

Steps/Code to reproduce bug

Run each case in a fresh process — an illegal address poisons the context.

import sys, os, tempfile
import cupy as cp, cudf, pylibcudf as plc

rows, dim = int(sys.argv[1]), int(sys.argv[2])
rg = int(sys.argv[3]) if len(sys.argv) > 3 else None

arr = cp.zeros((rows, dim), dtype=cp.float32)
df = cudf.DataFrame({"e": cudf.Series.from_pylibcudf(plc.Column.from_cuda_array_interface(arr))})
kw = {"row_group_size_rows": rg} if rg else {}
with tempfile.TemporaryDirectory() as d:
    df.to_parquet(os.path.join(d, "t.parquet"), index=False, **kw)
print(f"OK rows={rows} dim={dim} leaf={rows * dim} rg={rg}")

$ python repro.py 797696 1024
OK rows=797696 dim=1024 leaf=816840704 rg=None

$ python repro.py 798720 1024
RuntimeError: Fatal CUDA error ... 700 cudaErrorIllegalAddress

$ python repro.py 1000000 1024 100000
OK rows=1000000 dim=1024 leaf=1024000000 rg=100000     <-- workaround

Needs ~3.3 GiB of device memory for the failing case; it is not an OOM.

Characterisation

• Deterministic. Reproduced repeatedly, in separate sessions and separate processes.
• Present on both 25.10.00 and 26.08.00a990 — long-standing, not a nightly regression. Both versions pass at 797,696 rows and crash at 798,720.
• Per row group, not per column, and it tracks leaf elements. dim=512 at 2 M rows passes, because the default row_group_size_rows=1,000,000 splits it; forcing row_group_size_rows=2,000,000 crashes. Same at dim=256 (4 M rows) and dim=128 (8 M rows) — all three crash at 1,024,000,000 leaf elements in a single row group, so the trigger follows leaf element count per row group rather than dim or row count.
• Specific to LIST columns. A flat float32 column of 817,889,280 elements — the same element count as the failing list — writes fine, as does a flat float64 column of the same byte size.
• Data-independent, but codec-dependent. It crashes with zeros and with standard_normal random data alike. However, at 817,889,280 leaf elements the default codec (SNAPPY) crashes while compression=None and compression="ZSTD" both succeed. compression=None is not a general escape: at 1,536,000,000 leaf elements it crashes too. The bracket below is therefore for the default codec.
• The surfaced message varies between stream_pool.cpp:189, transform: failed inside CUB, and rmm/cuda_stream_view.cpp:45, but it is always cudaErrorIllegalAddress.

Workaround: cap row_group_size_rows (e.g. 100_000). Verified to let a 1,024,000,000-leaf-element write succeed. This is a viable mitigation but a silent trap — the default is what crashes. Switching to compression=None raises the ceiling but does not remove it.

Expected behavior

Either the writer handles large list row groups correctly, or it raises a clean, catchable exception naming the limit. A fatal illegal memory access that kills the CUDA context is the worst available outcome, and the default row_group_size_rows should not lead to it.

Environment overview

• Environment location: bare-metal, single node of a SLURM cluster
• Method of cuDF install: pip — cudf-cu12==26.8.0a990 (nightly wheel) and cudf-cu12==25.10.0

Environment details

• NVIDIA H100 80GB HBM3, CUDA 12.9, Python 3.12
• ~78 GiB device memory free — this is not an OOM

Additional context

The exact threshold is not identified. The empirical bracket at float32 is [816,840,704 , 817,889,280] leaf elements (probed in steps of 1,024 rows). Notably it is not a clean 2^31-bytes boundary: 816,840,704 × 4 B = 3.27 GB. A list<float64> column of 409,600,000 leaf elements (3.277 GB) writes fine while sitting marginally above the failing float32 case's 3.272 GB, so a pure byte threshold does not fit either. Reported as an empirical bracket only — someone with libcudf internals knowledge will place it faster than we can.

Related drafts from us (this fork, to be filed upstream alongside): #6 (list child element cap — this crash sits well inside that cap, and is arguably the more urgent of the two since it is a crash rather than a limit) · #12 (Parquet write scratch scales with leaf element count — the same "size in elements, not bytes" theme, but that issue is about memory volume and this one is about a hard crash).

Related upstream: rapidsai/cudf#17361 is a different trigger (zero-row table with invalid partition_info) but the same symptom class — a write-side illegal memory access in the Parquet writer. rapidsai/cudf#23378 (ParquetDatasetWriter max_file_size overestimates list columns).