bug: WSL2 CUDA + Open3D libdxcore conflict crashes visualize.py

[engichang1467]

bug: WSL2 CUDA + Open3D libdxcore conflict crashes visualize.py

Bug Description

Running scripts/visualize.py on WSL2 with a CUDA-capable GPU crashes with an ld.so assertion failure before any Python output is produced. The crash is triggered when the model is moved to the GPU (model.to("cuda")), which initializes the full CUDA context and loads WSL2's libdxcore.so.

Environment

• OS: WSL2 (Linux 6.6.87.2-microsoft-standard-WSL2)
• Python: 3.11
• PyTorch: (check torch.__version__)
• Open3D: 0.19.0
• CUDA driver: /usr/lib/wsl/drivers/nv_dispig.inf_amd64_f4c7a2fd13e0f763/libcuda.so.1.1

Steps to Reproduce

python scripts/visualize.py --config configs/evaluate_mini.yaml \
    --no-show --export outputs/test.ply

Error

Inconsistency detected by ld.so: dl-version.c: 204:
_dl_check_map_versions: Assertion `needed != NULL' failed!

Root Cause

Running with LD_DEBUG=versions reveals the underlying failures:

libcuda.so.1.1: error: symbol lookup error: undefined symbol: cuPvtCompilePtx (fatal)
libdxcore.so: error: symbol lookup error: undefined symbol: D3DKMTOpenSyncObjectFromNtHandle (fatal)
python: error: symbol lookup error: undefined symbol: InitializeInjectionCaskEnhanced (fatal)

When model.to("cuda") is called, the CUDA runtime fully initializes and loads libdxcore.so — a WSL2-specific DirectX/CUDA interop library. libdxcore.so expects Windows kernel D3DKMT functions that are not fully available in the current WSL2 kernel/driver version, causing the dynamic linker to crash. This is not a Python exception and cannot be caught with try/except.

Current Workaround

Pass --device cpu to force inference on CPU, which avoids full CUDA context initialization entirely. Output quality is unaffected — only inference speed is impacted.

python scripts/visualize.py --config configs/evaluate_mini.yaml \
    --no-show --export outputs/test.ply --device cpu

Potential Fixes to Investigate

• Update WSL2 kernel / NVIDIA drivers — the missing D3DKMT symbols may be resolved by a newer WSL2 kernel (wsl --update) or a newer NVIDIA Windows driver.
• Run inference separately from visualization — split the script into two steps: (1) save model outputs to disk with a pure-PyTorch script (no open3d import), then (2) load and visualize with a separate script that never touches CUDA. This would allow GPU inference without the conflict.
• Use EGL/offscreen Open3D backend — investigate whether setting OPEN3D_CPU_RENDERING=true or using Open3D's headless EGL backend avoids loading the conflicting rendering libraries while still allowing CUDA.
• LD_PRELOAD approach — preload the system libstdc++.so.6 or libgcc_s.so.1 before Python starts to see if it resolves the symbol conflict without requiring CPU fallback.

References

• WSL2 CUDA known issues: https://github.com/microsoft/WSL/issues
• Open3D headless rendering: https://www.open3d.org/docs/latest/tutorial/visualization/headless_rendering.html