Skip to content

Port Symbolics extension to the Symbolics v7 APIs#72

Merged
ChrisRackauckas merged 2 commits into
SciML:mainfrom
ChrisRackauckas-Claude:fix-symbolics-v7-ext
Jun 10, 2026
Merged

Port Symbolics extension to the Symbolics v7 APIs#72
ChrisRackauckas merged 2 commits into
SciML:mainfrom
ChrisRackauckas-Claude:fix-symbolics-v7-ext

Conversation

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Contributor

Important

This PR should be ignored until reviewed by @ChrisRackauckas.

Investigation of #69 revealed the Symbolics extension has been entirely non-functional on Symbolics v7 — which is the only version the compat allows — because it still uses the v6 extension APIs:

  1. Symbolics.array_term does not exist in any Symbolics v7 release. Symbolically calling an array-output (N_out >= 1) interpolation throws UndefVarError: array_term not defined.
  2. Scalar-output terms had Unknown shape. SymbolicUtils.term(interp, t...; type = Real) leaves shape = Unknown(-1), which expand_derivatives rejects with Differentiation with array expressions is not yet supported. SymbolicUtils v4 needs promote_shape (or an explicit shape) for non-builtin callables.
  3. The derivative registrations were dead code. The Symbolics.derivative(op, args, ::Val{I}) overload API was removed in Symbolics v7 in favor of @register_derivative/derivative_rule (the same migration DataInterpolations.jl did in its Symbolics ext behind a pkgversion(Symbolics) >= v"7" check). Defining methods on the old function does nothing, so derivatives of interpolation terms were left as unexpanded Differential operators.

Why CI never caught this: test/runtests.jl has an Extensions group containing test_symbolics_ext.jl, but test/test_groups.toml only declares Core and QA, so the extension tests never ran on CI. (Locally, on current main with Symbolics v7.26, test_symbolics_ext.jl errors immediately at interp(x, y).)

Changes

  • Build terms with SymbolicUtils.term with explicit type and shape, and define SymbolicUtils.promote_shape alongside promote_symtype for both NDInterpolation and DifferentiatedNDInterpolation.
  • Register derivatives with @register_derivative; partial derivatives are still represented by DifferentiatedNDInterpolation so arbitrary mixed orders compose.
  • Add the Extensions group to test_groups.toml so these tests actually run on CI.
  • Extend tests: scalar-output (N_out == 0) interpolations (the Differentiating through DataInterpolationsND functions throws error #69 use case — previously untested), full expand_derivatives chain rule through both arguments, second/mixed derivatives, and numeric evaluation of every generated expression. Also fixed an ex isa BasicSymbolic{Vector{Real}} assertion that is invalid under SymbolicUtils v4 (the type parameter is now the variant, not the symtype).

All APIs used (@register_derivative, Symbolics.SymbolicT, SymbolicUtils.ShapeVecT) exist since Symbolics v7.0.0, so no compat change is needed.

What this does and does not fix for #69

With this PR, directly embedding NDInterpolation objects in MTK equations works end-to-end: the #69 MWE rewritten with direct embedding compiles via mtkcompile (9-equation fully determined system) and solves successfully with Rodas5P (verified locally).

The original MWE uses MTK callable parameters ((itp::NDInterpolation)(..)), and that form still fails: in the symbolic expression the operation is the symbolic stand-in for the parameter (symtype FnType{Tuple, Real, NDInterpolation}), not an NDInterpolation instance, so no instance-dispatched derivative rule can ever fire and expand_derivatives leaves Differential(p(t))(itp(p(t), T(t))) behind, which validate_operator then rejects. That gap is upstream (Symbolics needs a hook to dispatch derivative rules on the FnType's callable type) and affects DataInterpolations.jl 1-D the same way whenever a callable parameter is called with an unknown rather than t (its ODEProblem construction fails at the initialization system even though mtkcompile happens to pass). I am filing that separately upstream.

Local test output

GROUP=Extensions Pkg.test():
Test Summary:       | Pass  Total   Time
Symbolics Extension |   23     23  44.3s

GROUP=Core Pkg.test():
Interpolations |  206    206
Derivatives    |  369    369
DataInterpolations |   28     28
Interface      |   21     21

Runic.jl run on the changed files.

🤖 Generated with Claude Code

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Contributor Author

CI status note: all 9 Extensions jobs (the tests validating this fix) pass on ubuntu/macos/windows × julia lts/1/pre. The remaining red is unrelated to this PR:

🤖 Generated with Claude Code

The extension was written against the Symbolics v6 extension APIs and was
never functional on Symbolics v7, which is the only version allowed by
compat:

- `Symbolics.array_term` does not exist in v7, so symbolically calling
  any array-output interpolation threw an UndefVarError.
- Scalar-output terms were built without shape information, getting
  `Unknown` shape, which `expand_derivatives` rejects with
  "Differentiation with array expressions is not yet supported".
- The `Symbolics.derivative(op, args, ::Val{I})` overload API was
  removed in v7 in favor of `@register_derivative`/`derivative_rule`,
  so the registered derivatives were dead code and derivatives of
  interpolation calls were left as unexpanded `Differential` operators.

This went unnoticed because the Extensions test group is declared in
test/runtests.jl but was missing from test/test_groups.toml, so the
extension tests never ran on CI.

Changes:
- Build terms with `SymbolicUtils.term` passing explicit `type` and
  `shape`, and define `SymbolicUtils.promote_shape` alongside
  `promote_symtype` for both interpolation callable types.
- Register derivatives via `@register_derivative`, keeping the
  `DifferentiatedNDInterpolation` representation for partial
  derivatives of arbitrary order.
- Add the Extensions group to test_groups.toml so the tests run on CI.
- Extend the tests to cover scalar-output (N_out == 0) interpolations,
  full `expand_derivatives` chain rules, and second derivatives
  (SciML#69), and fix a
  `BasicSymbolic{Vector{Real}}` assertion that is invalid under
  SymbolicUtils v4's type parameterization.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
The extension is pure Julia with no platform-specific behavior, so the
os matrix adds CI cost without coverage. Windows grouped jobs currently
do not even run their declared group (SciML/.github#81).

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
@ChrisRackauckas ChrisRackauckas marked this pull request as ready for review June 10, 2026 10:40
@ChrisRackauckas ChrisRackauckas merged commit f6e841d into SciML:main Jun 10, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants