Skip to content

Forward-mode rules for jl_eqtable_get/jl_eqtable_put (#3135 follow-up)#3213

Draft
ChrisRackauckas-Claude wants to merge 1 commit into
EnzymeAD:mainfrom
ChrisRackauckas-Claude:eqtable-forward-rules
Draft

Forward-mode rules for jl_eqtable_get/jl_eqtable_put (#3135 follow-up)#3213
ChrisRackauckas-Claude wants to merge 1 commit into
EnzymeAD:mainfrom
ChrisRackauckas-Claude:eqtable-forward-rules

Conversation

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Contributor

Forward-mode rules for jl_eqtable_get / jl_eqtable_put

This is the jl_eqtable_* follow-up deferred in #3137 / #3135.

Problem

Forward mode had erroring rules for the eqtable builtins (eqtableget_fwd /
eqtableput_fwd just emit_error("Not yet implemented forward for jl_eqtable_*")
unless the result was provably constant). So forward-over-reverse nesting
(HVPs / Hessians) failed whenever the inner reverse's make_zero recursed
through its seen IdDict — which it does to dedup aliased mutable objects.
After #3137 fixed jl_field_isdefined_checked / jl_idset_peek_bp, the
make_zero forward-over-reverse path advanced to exactly this jl_eqtable_get
site (reproduced on released Enzyme in #3135 and SciML/SciMLSensitivity.jl#1427).

Fix

Replace the erroring eqtableget_fwd / eqtableput_fwd bodies with real
shadow-propagating rules that mirror the existing reverse augmented-forward
rules (eqtableget_augfwd / eqtableput_augfwd): invert the table (and
default / value) pointers and re-issue the call on the inverted arguments via
batch_call_same_with_inverted_arg_if_active!, including the same mixed-activity
handling for a constant default/value.

One deliberate difference from the reverse put rule: forward mode does not
use the eqtable_shadow_active guard. In forward mode the tangent is the
shadow value, so storing an active scalar value is correct — whereas the reverse
rule rejects it because active scalars are accumulated via the tape. The
constant-result short-circuit at the top of each rule is unchanged, so this can
only enable previously-erroring forward cases, never change a constant one.

What this enables

  • Forward-over-reverse through make_zero's seen table for aliased objects.
  • Plain forward-mode differentiation through Dict / IdDict build + lookup.

Tests

Added to test/make_zero.jl (auto-discovered, runs on 1.10 and 1.11):

  • forward-over-reverse HVP through an aliased value (exercises the seen IdDict;
    loss = 2·sum(abs2,a) ⇒ Hessian 4I, checked against 4·seed),
  • plain forward-mode Dict{Int,Float64} and IdDict{Any,Float64} build+lookup
    (the active-scalar-value case the reverse rule can't take).

Verified locally on Julia 1.11.9 and 1.10.11 (dev'd checkout): full
test/make_zero.jl green, and the aliased case fails on stock main
(EnzymeRuntimeException / jl_eqtable_get) and passes with this change.

Not yet closed

The struct-with-undef-able-field make_zero path (#3135 MWE, the SciML #1427
neural ODE) advances past jl_eqtable_get with this change but then hits
forward-mode mixed-activity jl_new_struct (make_zero reconstructing a
struct with one active + one constant field). That is a separate, deeper
follow-up.


🤖 Generated with Claude Code

@wsmoses this is the eqtable follow-up you asked about in #3135 — opening as draft for your review (per my workflow, please treat as not-ready-to-merge until reviewed).

Forward mode had erroring rules for the eqtable builtins
("Not yet implemented forward for jl_eqtable_get/_put"), so
forward-over-reverse nesting (HVPs/Hessians) failed whenever the inner
reverse's make_zero recursed through its `seen` IdDict to dedup aliased
mutable objects. This is the eqtable follow-up deferred in EnzymeAD#3137 (EnzymeAD#3135).

Replace the erroring `eqtableget_fwd` / `eqtableput_fwd` bodies with real
shadow-propagating rules mirroring the existing reverse augmented-forward
rules: invert the table (and default/value) pointers and re-issue the call
on the inverted arguments via batch_call_same_with_inverted_arg_if_active!.

Unlike the reverse put rule, forward mode does not use the
`eqtable_shadow_active` guard: in forward mode the tangent is itself the
shadow value, so storing an active scalar value is correct (the reverse
rule rejects it because active scalars are accumulated via the tape).

This also enables plain forward-mode differentiation through Dict/IdDict
build + lookup. The constant-result short-circuit is unchanged, so this
can only enable previously-erroring forward cases.

The struct-with-undef-able-field make_zero path (EnzymeAD#3135
MWE, SciML/SciMLSensitivity.jl#1427) advances past jl_eqtable_get with
this change but still needs forward-mode mixed-activity jl_new_struct
support; that remains a separate follow-up.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
@ChrisRackauckas-Claude

Copy link
Copy Markdown
Contributor Author

Confirmed fixed on the released Enzyme 0.13.169 (Julia 1.11.9) — looks resolved by #3220. The reduced forward-over-reverse make_zero MWE from #3135 that was throwing jl_eqtable_get on 0.13.163 now returns the correct HVP:

import Enzyme
RA_R = Enzyme.set_runtime_activity(Enzyme.Reverse)
RA_F = Enzyme.set_runtime_activity(Enzyme.Forward)
struct Par; w::Vector{Float64}; extra::Any; end   # undef-able field = the ingredient
loss1(p) = sum(abs2, p.w)
p  = Par([1.0,2.0,3.0], "ignored"); dp = Par([1.0,0.0,0.0], "ignored")
Enzyme.autodiff(RA_F, Enzyme.Const(x -> Enzyme.gradient(RA_R, loss1, x)[1]), Enzyme.Duplicated(p, dp))
# 0.13.163: EnzymeRuntimeException: Not yet implemented forward for jl_eqtable_get
# 0.13.169: @NamedTuple{1}((Par([2.0, 0.0, 0.0], "ignored"),))   ✅

Good to close. The downstream SciML neural-ODE forward-over-reverse (SciML/SciMLSensitivity.jl#1427) now gets past this and stops one step later at Not yet implemented, mixed activity for jl_new_struct; I'll open that separately once I can reduce it to a dependency-free MWE.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants