Skip to content

Fix Windows heap corruption from literal-pointer symbol name collisions#3288

Open
gbaraldi wants to merge 1 commit into
mainfrom
fix-literal-ptr-symbol-collision
Open

Fix Windows heap corruption from literal-pointer symbol name collisions#3288
gbaraldi wants to merge 1 commit into
mainfrom
fix-literal-ptr-symbol-collision

Conversation

@gbaraldi

@gbaraldi gbaraldi commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

Fixes the Windows 1.10 CI crash (STATUS_HEAP_CORRUPTION, exit code 3221226356) debugged via #3228.

The bug

On Julia 1.10, ccalls reach Enzyme's module as literal inttoptr addresses. For each such call site, check_ir! recovers a symbol name via jl_lookup_code_address, creates a declaration under that name carrying enzymejl_needs_restoration=<ptr>, and restore_lookups later replaces the declaration with its recorded address.

jl_lookup_code_address reports the nearest preceding symbol, and on Windows (export-table-only symbol info) two distinct pointers can come back with the same name. The second call site then reused the first site's declaration, and restoration stamped one address onto both call sites.

In the post-restore_lookups IR from the #3228 debug run, the compiled Base.GMP.MPZ.set_si(1) (building the literal 1 of the BigFloat @easy_rule shadow) shows both GMP calls resolved to the same address:

call void inttoptr (i64 1791842560 to void (i8*, i32)*)(i8* %coercion, i32 0)  ; dbg scope: init2!
call void inttoptr (i64 1791842560 to void (i8*, i32)*)(i8* %coercion, i32 1)  ; dbg scope: set_si!

so __gmpz_init2 never runs and mpz_set_si writes a limb through the uninitialized d pointer of a non-zeroed pool allocation → heap corruption. On Julia 1.11+ the same call sites go through the GOT path and keep their identity as ejlstr$__gmpz_init2$libgmp-10.dll / ejlstr$__gmpz_set_si$libgmp-10.dll, which is why only 1.10 crashes. (Windows is also the only platform where the easy-rule literal 1 takes the BigInt/GMP path at all, since Clong == Int32 there.)

The fix

Two layers in check_ir!:

  1. Verify recovered names against the loader (resolve_symbol_name): ask which module contains the pointer (dladdr on POSIX, GetModuleHandleExW(GET_MODULE_HANDLE_EX_FLAG_FROM_ADDRESS) on Windows) and resolve the candidate name within exactly that module. If it resolves to a different address, the name is provably wrong and is dropped (the call site keeps its correct literal pointer). The lookup is module-scoped on purpose: a process-wide search could find an unrelated library's copy of a symbol (libm vs openlibm, etc.) and misreport a correct name; comparing addresses instead of names also keeps exported aliases working.
  2. Never merge call sites whose pointers differ: when a declaration with the recovered name already exists but records a different restoration pointer, key the new declaration by pointer instead of reusing it. This is the backstop for names that cannot be verified.

This is the same disease as the existing macOS memcpy/memmove collision workaround in FFI.ptr_map (curated names remain trusted as-is).

Testing

  • Julia 1.10.11 and 1.12.6 (Linux): the [DEBUG] Windows bigfloat error #3228 reproducer passes (autodiff(Forward, +, Duplicated, Duplicated(a, da), Duplicated(b, db))[1] ≈ da + db), full test/rules/internal_rules/bigfloat.jl passes, and test/passes.jl passes including the new "Literal-pointer symbol resolution" testset covering match / nearest-symbol mismatch / unresolvable-file / non-module-pointer cases.
  • Windows 1.10 is the real target; CI on this PR is the proof.

🤖 Generated with Claude Code

When check_ir! encounters a call to a literal inttoptr address (as Julia
1.10 codegen emits for already-resolved ccalls), it recovers a symbol
name via jl_lookup_code_address, creates a declaration under that name
carrying enzymejl_needs_restoration=<ptr>, and restore_lookups later
replaces the declaration with its recorded address.

jl_lookup_code_address reports the nearest preceding symbol, and on
Windows (export-table-only symbol info) two distinct pointers can come
back with the same name. The second call site then reused the first
site's declaration, and restore_lookups stamped one address onto both.
In the BigFloat easy_rule shadow path this merged __gmpz_init2 and
__gmpz_set_si onto a single address, so mpz_set_si ran on an
uninitialized BigInt and wrote a limb through a garbage d pointer,
crashing CI with STATUS_HEAP_CORRUPTION (0xC0000374).

Fix:
- Verify recovered names by asking the loader which module contains the
  pointer (dladdr on POSIX, GetModuleHandleExW(FROM_ADDRESS) on
  Windows) and resolving the name within exactly that module; drop
  names that provably belong to a different address.
- Never merge call sites whose pointers differ: when a declaration with
  the same name but a different recorded restoration pointer already
  exists, key the new declaration by pointer instead.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results

main 3664eb6... main / 3664eb6...
basics/make_zero/namedtuple 0.0567 ± 0.0023 μs 0.056 ± 0.0017 μs 1.01 ± 0.052
basics/make_zero/struct 0.263 ± 0.0063 μs 0.261 ± 0.0063 μs 1.01 ± 0.034
basics/overhead 4.34 ± 0.01 ns 5.26 ± 0.02 ns 0.825 ± 0.0037
basics/remake_zero!/namedtuple 0.227 ± 0.011 μs 0.227 ± 0.01 μs 1 ± 0.066
basics/remake_zero!/struct 0.223 ± 0.01 μs 0.223 ± 0.011 μs 1 ± 0.068
fold_broadcast/multidim_sum_bcast/1D 10.3 ± 1.6 μs 10.2 ± 0.18 μs 1.01 ± 0.16
fold_broadcast/multidim_sum_bcast/2D 12 ± 0.26 μs 12 ± 0.27 μs 0.996 ± 0.031
time_to_load 1.44 ± 0.017 s 1.45 ± 0.015 s 0.998 ± 0.015

Benchmark Plots

A plot of the benchmark results has been uploaded as an artifact at https://github.com/EnzymeAD/Enzyme.jl/actions/runs/28825930507/artifacts/8122763617.

@gbaraldi gbaraldi requested a review from vchuravy July 6, 2026 22:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant