Forward-over-reverse: no forward rule for jl_field_isdefined_checked / jl_idset_peek_bp in make_zero/make_zero! (inner reverse shadow of value with undef-able/abstract field)

[ChrisRackauckas-Claude]

Forward-over-reverse: make_zero/make_zero! shadow-init has no forward rule (jl_field_isdefined_checked / jl_idset_peek_bp) when the inner reverse shadows a value with an undef-able/abstract field

When nesting Enzyme forward over reverse (e.g. for HVPs/Hessians), the outer forward pass cannot differentiate the make_zero / make_zero! shadow-allocation the inner reverse uses — if the value the inner reverse differentiates contains an undef-able / abstract / isbits-Union field. Enzyme forward has no rule for the jl_field_isdefined_checked builtin (emitted by make_zero when it recurses into a type with possibly-undefined fields) nor for jl_idset_peek_bp (emitted by make_zero!'s IdSet aliasing check).

Reduced to Enzyme-only MWEs (no other packages). Julia 1.11.9, Enzyme v0.13.151.

MWE 1 — make_zero → jl_field_isdefined_checked

import Enzyme
RA_R = Enzyme.set_runtime_activity(Enzyme.Reverse)
RA_F = Enzyme.set_runtime_activity(Enzyme.Forward)

struct Par
    w::Vector{Float64}
    extra::Any            # undef-able pointer slot  <-- the necessary ingredient
end
loss1(p) = sum(abs2, p.w)

p  = Par([1.0, 2.0, 3.0], "ignored")
dp = Par([1.0, 0.0, 0.0], "ignored")
Enzyme.autodiff(RA_F, Enzyme.Const(x -> Enzyme.gradient(RA_R, loss1, x)[1]), Enzyme.Duplicated(p, dp))

EnzymeNoDerivativeError: No forward mode derivative found for jl_field_isdefined_checked
 at context:   %51 = call i32 @jl_field_isdefined_checked(... i64 noundef 0) ...
  [1] make_zero        @ Enzyme/src/typeutils/make_zero.jl:255
  [3] macro expansion  @ Enzyme/src/sugar.jl:334      # inner reverse's shadow init
  [4] gradient         @ Enzyme/src/sugar.jl:274

MWE 2 — make_zero! → jl_idset_peek_bp

import Enzyme
RA_R = Enzyme.set_runtime_activity(Enzyme.Reverse)
RA_F = Enzyme.set_runtime_activity(Enzyme.Forward)

lossv(x) = sum(abs2, Float64[xi for xi in x])

x0 = Any[1.0, 2.0, 3.0]; dx = Any[0.0, 0.0, 0.0]
v  = Any[1.0, 0.0, 0.0]; ddx = Any[0.0, 0.0, 0.0]
Enzyme.autodiff(RA_F, Enzyme.Const((d, y) -> (Enzyme.gradient!(RA_R, d, lossv, y); nothing)),
    Enzyme.Duplicated(dx, ddx), Enzyme.Duplicated(x0, v))

EnzymeNoDerivativeError: No forward mode derivative found for jl_idset_peek_bp
 at context:   %235 = call i64 @jl_idset_peek_bp(...) ...
  [1] haskey      @ ./idset.jl:41
  [3] make_zero!  @ Enzyme/src/typeutils/make_zero.jl:485

Control — succeeds (no undef-able field)

lossc(x) = sum(abs2, x)
x0 = [1.0, 2.0, 3.0]; v = [1.0, 0.0, 0.0]
Enzyme.autodiff(RA_F, Enzyme.Const(x -> Enzyme.gradient(RA_R, lossc, x)[1]), Enzyme.Duplicated(x0, v))
# -> ([2.0, 0.0, 0.0],)   no error

The plain Vector{Float64} case returns the correct HVP, so the undef-able/abstract field is the necessary-and-sufficient ingredient.

Where this bites in practice

This blocks Enzyme forward-over-reverse Hessians/HVPs of SciML neural-ODE losses (SciML/SciMLSensitivity.jl#1427): Lux / ComponentArray parameter structs carry abstract/Union/undef-able fields, so the inner reverse's make_zero recurses into them and the outer forward hits this gap. (For completeness, SecondOrder(AutoForwardDiff(), AutoEnzyme(Reverse)) fails differently — the inner Enzyme reverse can't return a ForwardDiff.Dual.)

---

Found while reducing the failure behind SciML/SciMLSensitivity.jl#1427. Filed by @ChrisRackauckas-Claude; please review before acting on it.

[wsmoses]

@ChrisRackauckas-Claude can you add both of these functions to the list of inactive functions and list of nofree functions in Enzyme.jl (https://github.com/EnzymeAD/Enzyme.jl/blob/main/src/llvm/attributes.jl).

Make sure to add both jl_ and ijl_ variants.

Add tests for the errors you see here

[ChrisRackauckas-Claude]

@wsmoses I implemented the suggested fix — added jl_field_isdefined_checked, ijl_field_isdefined_checked, jl_idset_peek_bp, ijl_idset_peek_bp to both nofreefns and inactivefns in src/llvm/attributes.jl — and tested it on a dev'd checkout of main (0670cc23, Julia 1.11.9, dev'd Enzyme + in-repo EnzymeCore). It removes the two named errors, but the MWEs still fail: the forward-over-reverse make_zero / make_zero! path cascades to the rest of the IdDict/IdSet seen-table bookkeeping builtins.

• MWE1 (make_zero, IdDict seen): with jl_field_isdefined_checked suppressed →
  No forward mode derivative found for jl_eqtable_get (the haskey(seen, prev) lookup, make_zero.jl:39) → and once that's added too, Enzyme: Not yet implemented forward for jl_eqtable_put.
• MWE2 (make_zero!, IdSet seen): with jl_idset_peek_bp suppressed →
  No forward mode derivative found for jl_idset_put_key → then jl_idset_put_idx (the push!(::IdSet, prev) chain, base/idset.jl).

Adding all of those to the inactive/nofree lists too is not sufficient, because jl_eqtable_get / jl_eqtable_put have explicit erroring custom forward rules in src/rules/llvmrules.jl (eqtableget_fwd, eqtableput_fwd). eqtableput_fwd only short-circuits when is_constant_value(gutils, orig) && is_constant_inst(gutils, orig), and marking the function inactive didn't make the dynamically-reallocated seen table's instruction result constant, so it still errors.

So this looks like it needs more than list membership — either forward rules for the jl_eqtable_* / jl_idset_put_* family, or making make_zero's seen bookkeeping provably inactive at the activity-analysis level. How would you like to handle it? Happy to PR whichever direction you prefer; the four-name change is ready locally.

(The CONTROL — the same forward-over-reverse nesting with a plain Vector{Float64} and no undef-able field — keeps working throughout, returning ([2.0, 0.0, 0.0],).)

[wsmoses]

Open the pr for those two functions to start we can do the eqtable stuff in a follow up

[ChrisRackauckas-Claude]

Opened #3137 with just the two functions (both jl_/ijl_ variants in nofreefns + inactivefns), as suggested.

Verified locally on main (0670cc2, Julia 1.11.9): both named errors (jl_field_isdefined_checked, jl_idset_peek_bp) are gone, and isolated forward-mode reproducers for each now return the correct derivative (added as a regression testset; full test/make_zero.jl green). As noted before, the MWEs then cascade to the seen-table mutation builtins (jl_eqtable_* / jl_idset_put_*) — that's the follow-up.

One thing to flag: src/llvm/attributes.jl isn't Runic-formatted globally (full-file Runic wants ~1100 line changes; the Set blocks are 4-space where Runic wants 8). I kept the new entries at the surrounding 4-space indent rather than adding 8-space stragglers. If you want the Runic check green, the file needs a separate full reformat — let me know if you'd like that in this PR or its own.

[ChrisRackauckas-Claude]

Retested on the released Enzyme 0.13.163 (with #3137 merged), Julia 1.11.9 — confirming the predicted jl_eqtable_* cascade is now what's left, both in the reduced MWE and in the real SciML/SciMLSensitivity.jl#1427 neural-ODE forward-over-reverse.

Reduced Enzyme-only MWE1 (make_zero, IdDict seen), unchanged from this issue:

import Enzyme
RA_R = Enzyme.set_runtime_activity(Enzyme.Reverse)
RA_F = Enzyme.set_runtime_activity(Enzyme.Forward)
struct Par; w::Vector{Float64}; extra::Any; end   # undef-able field = the ingredient
loss1(p) = sum(abs2, p.w)
p  = Par([1.0,2.0,3.0], "ignored"); dp = Par([1.0,0.0,0.0], "ignored")
Enzyme.autodiff(RA_F, Enzyme.Const(x -> Enzyme.gradient(RA_R, loss1, x)[1]), Enzyme.Duplicated(p, dp))

now gives (the jl_field_isdefined_checked error from #3137 is gone):

EnzymeRuntimeException: Enzyme: Not yet implemented forward for jl_eqtable_get
  [1] get        @ ./iddict.jl:92
  [3] haskey     @ ./abstractdict.jl:19
  [4] make_zero  @ Enzyme/src/typeutils/make_zero.jl:39
  [5] make_zero  @ Enzyme/src/typeutils/make_zero.jl:257
  ...
  [8] gradient   @ Enzyme/src/sugar.jl:274          # inner reverse shadow init

The control (plain Vector{Float64}, no undef-able field) still returns the correct HVP ([2.0, 0.0, 0.0],).

In the real workload, SecondOrder(AutoEnzyme(Forward), AutoEnzyme(Reverse)) HVP of the docs neural-ODE loss (252-param Lux + ComponentArray, SciMLSensitivity 7.112.0 / OrdinaryDiffEq 7.1.0) hits the same jl_eqtable_get site in make_zero — so this is the last thing standing between Enzyme forward-over-reverse and the SciML #1427 use case.

As noted earlier, this is the jl_eqtable_get / jl_eqtable_put / jl_idset_put_key / jl_idset_put_idx family — and just adding them to the inactive/nofree lists isn't enough, because eqtableget_fwd / eqtableput_fwd in src/rules/llvmrules.jl have explicit erroring forward rules that only short-circuit when the instruction is provably constant, which the dynamically-reallocated seen table isn't. Want me to take a crack at the follow-up (forward rules for the jl_eqtable_* / jl_idset_put_* family, or making make_zero's seen bookkeeping provably inactive at the activity-analysis level), or would you prefer to handle the rule side?

[vchuravy]

@ChrisRackauckas-Claude #3043 should add eqtable support.

[ChrisRackauckas-Claude]

Opened #3213 with the eqtable follow-up: real forward-mode rules for jl_eqtable_get / jl_eqtable_put (mirroring the reverse augmented-forward rules), replacing the erroring emit_error bodies.

Verified locally (Julia 1.11.9 + 1.10.11, dev'd main):

• forward-over-reverse through make_zero's seen IdDict now works for aliased objects (loss = 2·sum(abs2,a) ⇒ Hessian 4I, HVP matches 4·seed); fails on stock main with jl_eqtable_get, passes with the PR.
• plain forward-mode Dict/IdDict build+lookup now differentiates (the forward rule stores the tangent as the shadow value, so it handles active scalar values that the reverse eqtable_shadow_active guard rejects — I dropped that guard for the forward rule only).
• full test/make_zero.jl green on both versions.

Next blocker in the cascade (for the original struct-with-undef-able-field MWE / SciML #1427): with eqtable handled, make_zero of the Par-style struct now gets one step further and stops at forward-mode mixed-activity jl_new_struct — make_zero reconstructing a struct with one active field (w) and one constant field (extra):

EnzymeRuntimeException: Enzyme: Not yet implemented, mixed activity for jl_new_struct constants=Bool[0, 1]
  ... ijl_new_structv(... Par ...)
  make_zero @ src/typeutils/make_zero.jl:264

So the reduced MWE1 from this issue isn't fully unblocked yet — it needs forward-mode mixed-activity struct construction in make_zero. Want me to take that as the next follow-up, or would you prefer a different approach (e.g. a forward rule on make_zero itself, since its output is all-zeros and its forward derivative is structurally zero)?

[wsmoses]

@ChrisRackauckas-Claude don't add a rule for that, instead mark haskey as inactive_noinl

[wsmoses]

overarching issue resolved in #3220