Forward-mode rules for jl_eqtable_get/jl_eqtable_put (#3135 follow-up)#3213
Draft
ChrisRackauckas-Claude wants to merge 1 commit into
Draft
Forward-mode rules for jl_eqtable_get/jl_eqtable_put (#3135 follow-up)#3213ChrisRackauckas-Claude wants to merge 1 commit into
ChrisRackauckas-Claude wants to merge 1 commit into
Conversation
Forward mode had erroring rules for the eqtable builtins
("Not yet implemented forward for jl_eqtable_get/_put"), so
forward-over-reverse nesting (HVPs/Hessians) failed whenever the inner
reverse's make_zero recursed through its `seen` IdDict to dedup aliased
mutable objects. This is the eqtable follow-up deferred in EnzymeAD#3137 (EnzymeAD#3135).
Replace the erroring `eqtableget_fwd` / `eqtableput_fwd` bodies with real
shadow-propagating rules mirroring the existing reverse augmented-forward
rules: invert the table (and default/value) pointers and re-issue the call
on the inverted arguments via batch_call_same_with_inverted_arg_if_active!.
Unlike the reverse put rule, forward mode does not use the
`eqtable_shadow_active` guard: in forward mode the tangent is itself the
shadow value, so storing an active scalar value is correct (the reverse
rule rejects it because active scalars are accumulated via the tape).
This also enables plain forward-mode differentiation through Dict/IdDict
build + lookup. The constant-result short-circuit is unchanged, so this
can only enable previously-erroring forward cases.
The struct-with-undef-able-field make_zero path (EnzymeAD#3135
MWE, SciML/SciMLSensitivity.jl#1427) advances past jl_eqtable_get with
this change but still needs forward-mode mixed-activity jl_new_struct
support; that remains a separate follow-up.
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Contributor
Author
|
Confirmed fixed on the released Enzyme 0.13.169 (Julia 1.11.9) — looks resolved by #3220. The reduced forward-over-reverse import Enzyme
RA_R = Enzyme.set_runtime_activity(Enzyme.Reverse)
RA_F = Enzyme.set_runtime_activity(Enzyme.Forward)
struct Par; w::Vector{Float64}; extra::Any; end # undef-able field = the ingredient
loss1(p) = sum(abs2, p.w)
p = Par([1.0,2.0,3.0], "ignored"); dp = Par([1.0,0.0,0.0], "ignored")
Enzyme.autodiff(RA_F, Enzyme.Const(x -> Enzyme.gradient(RA_R, loss1, x)[1]), Enzyme.Duplicated(p, dp))
# 0.13.163: EnzymeRuntimeException: Not yet implemented forward for jl_eqtable_get
# 0.13.169: @NamedTuple{1}((Par([2.0, 0.0, 0.0], "ignored"),)) ✅Good to close. The downstream SciML neural-ODE forward-over-reverse (SciML/SciMLSensitivity.jl#1427) now gets past this and stops one step later at |
This was referenced Jun 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Forward-mode rules for
jl_eqtable_get/jl_eqtable_putThis is the
jl_eqtable_*follow-up deferred in #3137 / #3135.Problem
Forward mode had erroring rules for the eqtable builtins (
eqtableget_fwd/eqtableput_fwdjustemit_error("Not yet implemented forward for jl_eqtable_*")unless the result was provably constant). So forward-over-reverse nesting
(HVPs / Hessians) failed whenever the inner reverse's
make_zerorecursedthrough its
seenIdDict— which it does to dedup aliased mutable objects.After #3137 fixed
jl_field_isdefined_checked/jl_idset_peek_bp, themake_zero forward-over-reverse path advanced to exactly this
jl_eqtable_getsite (reproduced on released Enzyme in #3135 and SciML/SciMLSensitivity.jl#1427).
Fix
Replace the erroring
eqtableget_fwd/eqtableput_fwdbodies with realshadow-propagating rules that mirror the existing reverse augmented-forward
rules (
eqtableget_augfwd/eqtableput_augfwd): invert the table (anddefault / value) pointers and re-issue the call on the inverted arguments via
batch_call_same_with_inverted_arg_if_active!, including the same mixed-activityhandling for a constant default/value.
One deliberate difference from the reverse put rule: forward mode does not
use the
eqtable_shadow_activeguard. In forward mode the tangent is theshadow value, so storing an active scalar value is correct — whereas the reverse
rule rejects it because active scalars are accumulated via the tape. The
constant-result short-circuit at the top of each rule is unchanged, so this can
only enable previously-erroring forward cases, never change a constant one.
What this enables
make_zero'sseentable for aliased objects.Dict/IdDictbuild + lookup.Tests
Added to
test/make_zero.jl(auto-discovered, runs on 1.10 and 1.11):seenIdDict;loss = 2·sum(abs2,a)⇒ Hessian4I, checked against4·seed),Dict{Int,Float64}andIdDict{Any,Float64}build+lookup(the active-scalar-value case the reverse rule can't take).
Verified locally on Julia 1.11.9 and 1.10.11 (dev'd checkout): full
test/make_zero.jlgreen, and the aliased case fails on stockmain(
EnzymeRuntimeException/jl_eqtable_get) and passes with this change.Not yet closed
The struct-with-undef-able-field
make_zeropath (#3135 MWE, the SciML #1427neural ODE) advances past
jl_eqtable_getwith this change but then hitsforward-mode mixed-activity
jl_new_struct(make_zeroreconstructing astruct with one active + one constant field). That is a separate, deeper
follow-up.
🤖 Generated with Claude Code
@wsmoses this is the eqtable follow-up you asked about in #3135 — opening as draft for your review (per my workflow, please treat as not-ready-to-merge until reviewed).