Skip to content

Add analyze_initialization_jacobian — diagnose rank-deficient initialization#4666

Merged
AayushSabharwal merged 3 commits into
SciML:masterfrom
baggepinnen:fbc/init-nullspace-report
Jun 30, 2026
Merged

Add analyze_initialization_jacobian — diagnose rank-deficient initialization#4666
AayushSabharwal merged 3 commits into
SciML:masterfrom
baggepinnen:fbc/init-nullspace-report

Conversation

@baggepinnen

@baggepinnen baggepinnen commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Motivation

Rank-deficient initialization systems are a recurring source of intermittent ("flaky") initialization failures. When the init Jacobian doesn't have full rank, the solve can land on different solutions — or fail — depending on the solver and on the (nondeterministic) ordering of the assembled init unknowns/equations: some orderings hit a singular realization and throw SingularException, others succeed, so the same model fails maybe 1 run in 10.

MTK already warns structurally (overdetermined/underdetermined … N equations for M unknowns, plus singular_check's problematic-variable list), but it's often hard to tell which unknowns form the free direction, and — when an init has more equations than unknowns yet is still underdetermined — which equations are redundant. I hit this in a multibody model and ended up computing the init-Jacobian SVD by hand; it seems generally useful.

What this adds

analyze_initialization_jacobian(prob; rtol, atol, threshold, verbose) evaluates the Jacobian of the initialization residual at the initial guess u0 (via ForwardDiff), computes its SVD, and reports both sides of the rank deficiency:

  • Underdetermined unknowns — the right null space (directions the init is free to move along). Pinning these makes the init well-posed and deterministic.
  • Redundant equations — the left null space (combinations of init equations whose Jacobian rows are linearly dependent at u0, so they don't locally constrain any further DOF). This explains the confusing case of "more equations than unknowns, yet still underdetermined."

prob may carry initialization data (an ODEProblem/DAEProblem built from a System) or be an initialization NonlinearProblem/NonlinearLeastSquaresProblem directly.

Per-item weights are the diagonal of the respective null-space projector (squared row norm over an orthonormal basis), so they're invariant to the arbitrary basis chosen within the null space. Returns (; jacobian, singular_values, rank, nullity, redundancy, underdetermined_unknowns, redundant_equations) and prints a report by default.

Example

julia> analyze_initialization_jacobian(prob)
Initialization Jacobian rank analysis
  residual Jacobian: 58×51, rank  47, nullity  4, redundancy  11
  smallest 5 singular value(s): [0.41, 0.2, 0.07, 3e-15, 1e-15]
  Underdetermined unknowns (participation  [0, 1]):
      0.746  rotational_losses_bl₊w_rel(t)
      ...
  Redundant equations (participation  [0, 1]):
      0.62   0 ~ () - springdamper₊length(t)
      ...

which immediately points at both the unpinned degrees of freedom to constrain and the equations that aren't pulling their weight.

Notes

  • Added to src/initialization.jl, exported, with a docstring and tests in test/initialization_jacobian_analysis.jl (registered under the Initialization group): an underdetermined DAE (nonzero nullity), an overdetermined system with a redundant init equation (nonzero redundancy, equation named), a determined algebraic init (full rank), and a fully-IC-determined system (no init problem — handled gracefully).
  • The Jacobian is evaluated at a single point, so it reports local rank structure (the docstring calls this out).
  • Happy to rename or fold into existing init diagnostics if you'd prefer.
  • I couldn't run the full MTK test suite in my (offline) environment; the function's logic and all four test cases were validated against MTK on small models + a downstream multibody model. CI here will exercise the added testset.

🤖 Generated with Claude Code

baggepinnen and others added 2 commits June 25, 2026 13:46
Underdetermined or rank-deficient initialization systems are a common source of
intermittent ("flaky") initialization failures: the solve lands on different
solutions depending on the solver and on the nondeterministic ordering of the
assembled init unknowns/equations, and some orderings hit a singular realization
and throw while others succeed.

`report_initialization_nullspace(prob)` makes the offending degrees of freedom
explicit. It evaluates the Jacobian of the initialization residual at the initial
guess (via ForwardDiff), computes its SVD, and reports the unknowns that span the
null space — the directions along which the initialization is free to move.
Pinning those unknowns (or adding constraints) makes the init well-posed and
deterministic.

`prob` may carry initialization data (an `ODEProblem`/`DAEProblem` built from a
`System`) or be an initialization `NonlinearProblem`/`NonlinearLeastSquaresProblem`
directly. The per-variable weight reported is the diagonal of the null-space
projector (squared row norm over an orthonormal null-space basis), which is
invariant to the arbitrary basis chosen within the null space. A nullity of 0
means the init Jacobian has full column rank at the guess and is locally
well-posed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FSDc4tFYygs26KhqZBACLS
…on_jacobian`

Extend the initialization diagnostic to report both sides of the rank deficiency of
the initialization residual Jacobian:

  - underdetermined unknowns — its (right) null space, the directions the
    initialization is free to move along; and
  - redundant equations — its left null space, combinations of initialization
    equations whose Jacobian rows are linearly dependent at `u0` and therefore do not
    locally constrain any further degree of freedom.

The latter explains the common, confusing situation where an initialization has more
equations than unknowns yet is still underdetermined.

Renamed from `report_initialization_nullspace` to `analyze_initialization_jacobian`
now that it analyzes the full rank structure (rank, nullity, redundancy) rather than
only the null space. The return value gains `rank`, `redundancy`, and
`redundant_equations` (a vector of `equation => weight` pairs), and the per-unknown
result is renamed `underdetermined_unknowns`. Redundant equations are mapped back to
the symbolic initialization equations when available.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FSDc4tFYygs26KhqZBACLS
@baggepinnen baggepinnen changed the title Add report_initialization_nullspace — diagnose rank-deficient initialization Add analyze_initialization_jacobian — diagnose rank-deficient initialization Jun 26, 2026
…acobian`

The overdetermined/underdetermined initialization warnings now suggest calling
`analyze_initialization_jacobian(prob)` on the constructed problem to see exactly
which unknowns are underdetermined and which equations are redundant, rather than
just reporting the equation/unknown counts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FSDc4tFYygs26KhqZBACLS
@AayushSabharwal AayushSabharwal merged commit 3154f2f into SciML:master Jun 30, 2026
66 of 89 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants