Skip to content

Fix ChunkSplitters v3 API (precompile failure), re-export solve, add stdlib compat#114

Merged
ChrisRackauckas merged 5 commits into
SciML:mainfrom
ChrisRackauckas-Claude:fix-chunksplitters-v3-api
Jun 21, 2026
Merged

Fix ChunkSplitters v3 API (precompile failure), re-export solve, add stdlib compat#114
ChrisRackauckas merged 5 commits into
SciML:mainfrom
ChrisRackauckas-Claude:fix-chunksplitters-v3-api

Conversation

@ChrisRackauckas-Claude

@ChrisRackauckas-Claude ChrisRackauckas-Claude commented Jun 15, 2026

Copy link
Copy Markdown
Contributor

Problem

main CI is fully red on Julia 1.12 (the 1 channel) and on the lts/pre channels. Several stacked failures across the test, downgrade, and docs jobs.

1. Precompile crash (ChunkSplitters v3)

ChunkSplitters 3.x removed the positional chunks(x, n) form, breaking precompilation at the PrecompileTools workload (MethodError: no method matching chunks(::Vector{...}, ::Int64) at src/solve.jl:13). Switched to index_chunks(x; n = nt) and rebuilt the (range, chunk_idx) tuples the kernels unpack. Also re-exported solve (using CommonSolve: CommonSolve, solve) and added stdlib compat that the precompile crash had masked from Aqua.

2. TRBDF2 / MatrixOperator UndefVarError

OrdinaryDiffEq 7.0 dropped its blanket @reexport, so TRBDF2 (SDIRK) and MatrixOperator are no longer brought in by using OrdinaryDiffEq. Added using OrdinaryDiffEqSDIRK / using SciMLOperators to test/equations.jl and the literate files that need them, with the matching test/docs deps + compat.

3. SciMLBase 3 solution-indexing change (this commit)

After loading worked again, Core was still red at the FVMSystem testset and across the literate examples:

BoundsError: attempt to access 11-element Vector{Matrix{Float64}} at index [1, 2, 1]
  @ test/equations.jl:106 (FVMSystem)

Under SciMLBase 3, eachindex(sol) / length(sol) on an ODESolution now span the full flattened (state..., time) index space (CartesianIndices{2}/{3}, length = nodes×timepoints) instead of the time points, and scalar sol[i] returns a single flattened element rather than sol.u[i]. The tests/examples assumed eachindex(sol) iterated time points.

Fix: iterate the time-point storage explicitly via eachindex(sol.u) / length(sol.u), and use sol.u[i] for the per-timepoint state — in test/equations.jl, every literate tutorial/wyos .jl run by Core, the generated docs .md mirrors, and the README animation. The 2D matrix-style accessors the WYOS tutorials use (sol[ind, :], sol[begin:(end-1), 2:end]) are unchanged — those still behave as the documented (state, time) layout under SciMLBase 3.

4. Downgrade (lts) resolution — root [compat] floor raises

The Downgrade job pins the root Project.toml deps to their compat floors; several floors were below what the package/test stack can actually co-install:

  • DelaunayTriangulation 1.0 → 1.6.6: the package calls DelaunayTriangulation.get_edge_midpoints (src/geometry.jl), introduced in DelaunayTriangulation v1.1.0 — so 1.0.0 fails at runtime. 1.6.6 also matches the test floor so the downgraded root pin is co-installable.
  • SciMLBase "2.34, 3.1" → "3.1": OrdinaryDiffEq 7 (the pinned solver) requires SciMLBase 3, so the 2.34 lower option can never resolve.
  • PrecompileTools 1.2 → 1.2.1: SciMLBase 3 requires PrecompileTools ≥ 1.2.1.
  • CommonSolve 0.2 → 0.2.4: OrdinaryDiffEq 7 requires CommonSolve ≥ 0.2.4.
  • PreallocationTools "0.4, 1.2" → "1.2": the 0.4 floor is incompatible with the NonlinearSolveBase the modern stack pulls in.

Each is a floor raise (never lowers an upper bound; latest-version resolution is unaffected, confirmed below).

Local verification (Julia 1.12 = "1", and 1.10 = lts)

  • test/equations.jl passes: 1,267,805 / 0, including the previously-failing FVMSystem block (2/2).
  • Brusselator (system) and square-plate (scalar) literate examples run clean — no BoundsError; their @test_reference figures match.
  • QA group passes: Aqua 10/10 (incl. Compat bounds) + ExplicitImports 2/2, with the new root compat.
  • Normal test env re-resolves to latest (SciMLBase 3.21, DelaunayTriangulation 1.6.6, PrecompileTools 1.3.4) — floor raises don't disturb it.
  • The downgraded root floors (=-pinned to the new minimums) instantiate, precompile, and load FiniteVolumeMethod + TRBDF2 + MatrixOperator on Julia 1.10.
  • All changed .jl files are Runic-clean (runic --check).

Known out-of-scope blocker (not fixed here)

  • Documentation job fails at dependency resolution (before examples, which are CI-skipped): the docs env pins SimpleGraphs = "0.8.10", which requires Optim 2.xNLSolversBase 8, while NLsolve (pulled by DiffEqDevTools/NonlinearSolve) caps NLSolversBase at 7.x. NLsolve 4.5.1 (latest) and Optim 2.x have mutually-exclusive NLSolversBase bounds — an upstream version-coordination conflict requiring an upstream fix (NLsolve supporting NLSolversBase 8, or dropping the NLsolve dependency), independent of this PR.

Please ignore until reviewed by @ChrisRackauckas.


Update (commit 52dbed5): remaining example errors + Documentation fixed

The prior commits got Core/Downgrade down to 8 errors (1288209 passed) and left Documentation red at resolution. This finishes both.

Example errors (Core + Downgrade, same 8):

  • autodiff = falseAutoFiniteDiff() in the disk (FBDF) and maze (TRBDF2) tutorials — OrdinaryDiffEq no longer accepts a Bool for autodiff. AutoFiniteDiff is exported by the already-loaded using OrdinaryDiffEq.
  • More SciMLBase 3 solution-indexing leftovers the earlier commit didn't reach: lastindex(sol)lastindex(sol.u) (porous_fisher BoundsError), eachindex(fvm_sol)eachindex(fvm_sol.u) (diffusion_equations + linear_reaction_diffusion WYOS; fig[2, j] with j::CartesianIndex).
  • piecewise_linear get_errs: length(_sol)/eachindex(_sol).u forms. The old form iterated CartesianIndices under SciMLBase 3, so the has_vertex guard skipped everything and @test all(≤(0.16), _errs) passed against an all-zero array. Restored so the assertion again checks FVM-vs-exact error (passes, max ≤ 0.16).

Reference images regenerated (not threshold-loosened): conditions.png, internal_conditions.png, and the piecewise natural-neighbour figure. These are upstream Makie/DelaunayTriangulation triplot recipe rendering changes since the references were last cut (Feb 2024), not FVM regressions — verified the underlying data is correct (annulus mesh: 152 triangles, 0 in the hole; condition-node counts and the piecewise error assertion all check out) and the regenerated images pass on a fresh non-update run locally. The other ~42 reference figures were left untouched (they already pass on the CI renderer).

Documentation (now fixed, was listed below as a blocker): relax docs SimpleGraphs compat 0.8.100.8.8 - 0.8. SimpleGraphs 0.8.10 forces Optim 2.xNLSolversBase 8, mutually exclusive with NLsolve 4.5.1 (capped at NLSolversBase 7). Allowing 0.8.9 lets the resolver pick Optim 1.13NLSolversBase 7.10, which co-installs. SimpleGraphs is not imported anywhere (docs-only dep) and CI skips example execution, so the patch downgrade is inert. Verified the docs env now resolves.

Local verification (Julia 1.12 = "1" channel): maze/disk/porous_fisher/diffusion_equations/linear_reaction_diffusion examples run with no error; piecewise 5/5 pass; conditions reference tests pass; docs env resolves (SimpleGraphs 0.8.9 / Optim 1.13 / NLSolversBase 7.10); all changed .jl files Runic-clean. Note: Makie reference-image PSNR comparisons are mildly environment-sensitive, so the three regenerated images should be confirmed once on the CI runner.


Update (commit 6630294): flaky natural-neighbour reference image (Core julia 1 / pre)

After the prior commits, the Core group passed on lts but stayed red on the julia 1 (1.12) and pre channels, failing at the piecewise tutorial's *_natural_neighbour_interpolation.png @test_reference comparison.

Root cause (verified, not a tolerance issue): the tutorial built its mesh via refine!(tri; ...) and a grid triangulation via triangulate([...]), both of which default to rng = Random.default_rng() in DelaunayTriangulation. The Core group runs all tutorials sequentially in one process (safe_include), so by the time this 8th tutorial runs, the seven preceding tutorials have already advanced the global RNG by a data-dependent amount. Because the default RNG stream is not stable across Julia versions, the resulting triangulation — and thus every panel of the 16-panel figure — differed from the committed reference on 1.12/pre while happening to stay within PSNR threshold on lts. Rendered standalone in a fresh process, the figure reproduced the committed reference byte-for-byte (md5 match); only the in-suite RNG ordering tripped it.

Fix: pass rng = StableRNG(123) to both refine! and the grid triangulate, making the triangulation (and figure) deterministic regardless of preceding RNG consumption or Julia version. StableRNGs is already a dep of both the test and docs environments (used by the maze tutorial) — no new dependency. Mirrored the change into the generated .md, and regenerated the four reference images to match the seeded triangulation.

Local verification (Julia 1.12.6, the "1" channel):

  • Rendered the tutorial in a fresh process and in a process where the global RNG was first advanced by ~1,000,060 draws → byte-identical output for all four figures (md5 match), confirming RNG-independence.
  • Re-ran the tutorial in non-update mode after advancing the global RNG (the exact condition that fails in the full Core sequence): 5/5 pass against the newly committed seeded references, including the previously failing natural-neighbour reference and the real piecewise error assertion.
  • Changed .jl is Runic-clean.

Please ignore until reviewed by @ChrisRackauckas.

ChrisRackauckas and others added 4 commits June 15, 2026 10:27
ChunkSplitters 3.x removed the positional `chunks(x, n)` form (now
`chunks(x; n)` returning value views), which made the package fail to
precompile at the PrecompileTools workload (`MethodError: no method
matching chunks(::Vector{...}, ::Int64)` at src/solve.jl:13). This took
every test group (Core, QA on all Julia versions) red, plus the
Documentation and Downgrade jobs.

Switch to `index_chunks(x; n = nt)` (returns index ranges, as the
downstream consumers require via `boundary_edges[edge_idx]`) and rebuild
the `(range, chunk_idx)` tuples the equation kernels unpack, preserving
the documented contract in main_equations.jl. Tighten the ChunkSplitters
compat to `3.2` (the only major with `index_chunks` and the version that
actually resolves).

Also fixes two pre-existing latent QA findings that were masked by the
precompile crash and surface once loading works:
- `solve` was exported but never brought into the module namespace
  (only methods added to `CommonSolve.solve`), so Aqua flagged an
  undefined export. Add `using CommonSolve: CommonSolve, solve`.
- Add missing `[compat]` entries for the stdlib deps LinearAlgebra and
  SparseArrays (Aqua deps_compat).

Verified locally on Julia 1.11: package precompiles and loads with
ChunkSplitters 3.2.0, and the full QA group passes (Aqua 10/10 incl.
undefined-exports and compat-bounds, ExplicitImports 2/2).

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… 7.x split)

OrdinaryDiffEq 7.0 dropped its blanket @reexport: the meta-package now
re-exports only a curated solver set (Tsit5, Vern*, Rosenbrock23, Rodas5P,
FBDF) plus problem/callback types. TRBDF2 (an SDIRK method) and MatrixOperator
are no longer exported by `using OrdinaryDiffEq`, so the Core test group hit
`UndefVarError: TRBDF2 not defined` in test/equations.jl and ~14 literate
example files, and `MatrixOperator not defined` in two literate_wyos files.

- Add `using OrdinaryDiffEqSDIRK` (provides TRBDF2) to test/equations.jl and
  every literate tutorial/wyos file that solves with TRBDF2.
- Add `using SciMLOperators` (provides MatrixOperator) to the two literate_wyos
  files that build ODEProblem(MatrixOperator(...)).
- Declare OrdinaryDiffEqSDIRK and SciMLOperators as direct deps + compat in
  test/Project.toml and docs/Project.toml.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… floors

Core test group (all Julia channels) was red after the FVMSystem testset
and many literate examples hit errors from a SciMLBase 3 behavior change:
`eachindex(sol)` / `length(sol)` on an `ODESolution` now span the full
flattened `(state..., time)` index space (e.g. `CartesianIndices{2}` /
`{3}`, `length` = nodes*timepoints) instead of the time points, and scalar
`sol[i]` returns a single flattened element rather than `sol.u[i]`.

`test/equations.jl` (FVMSystem) errored with
`BoundsError: ... Vector{Matrix{Float64}} at index [1, 2, 1]` because
`[solsys.u[i][1, :] for i in eachindex(solsys)]` now iterates
`CartesianIndex{3}`. Iterate the time-point storage explicitly via
`eachindex(sol.u)` / `length(sol.u)`, and use `sol.u[i]` for the
per-timepoint state. The 2D matrix-style accessors the WYOS tutorials rely
on (`sol[ind, :]`, `sol[begin:(end-1), 2:end]`) are unchanged — those still
behave as the documented `(state, time)` layout under SciMLBase 3.

Applied the same fix to every literate tutorial/wyos `.jl` run by the Core
group, the generated docs `.md` mirrors, and the README animation snippet.

Also raise root [compat] floors so the Downgrade (lts) job resolves; each
is a floor raise (the pinned minimum was below what the package/test stack
can actually co-install):
- DelaunayTriangulation 1.0 -> 1.6.6: the package calls
  `DelaunayTriangulation.get_edge_midpoints` (src/geometry.jl), introduced
  in DelaunayTriangulation v1.1.0, so 1.0.0 fails at runtime; 1.6.6 also
  matches the test floor so the downgraded root pin is co-installable.
- SciMLBase "2.34, 3.1" -> "3.1": OrdinaryDiffEq 7 (the solver the tests
  pin) requires SciMLBase 3, so the 2.34 lower option can never resolve in
  the test stack.
- PrecompileTools 1.2 -> 1.2.1: SciMLBase 3 requires PrecompileTools >= 1.2.1.
- CommonSolve 0.2 -> 0.2.4: OrdinaryDiffEq 7 requires CommonSolve >= 0.2.4.
- PreallocationTools "0.4, 1.2" -> "1.2": the 0.4 floor is incompatible with
  the NonlinearSolveBase the modern stack pulls in.

Verified locally on Julia 1.12 (the "1" channel) and 1.10 (lts):
- `test/equations.jl` passes (1,267,805 / 0; FVMSystem block 2/2).
- The brusselator (system) and square-plate (scalar) literate examples run
  clean (no BoundsError; their @test_reference figures match).
- QA group passes (Aqua 10/10 incl. Compat bounds, ExplicitImports 2/2).
- Downgraded root floors (`=`-pinned) instantiate + load on Julia 1.10.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…resolution

After the prior commits on this branch, Core (all Julia channels) and the
Downgrade job were still red with 8 errors (1288209 passed, 8 errored), and
the Documentation job failed at dependency resolution. This finishes them.

Test/example errors (Core + Downgrade, identical set):

- `autodiff = false` -> `AutoFiniteDiff()` in the disk (FBDF) and maze
  (TRBDF2) tutorials. OrdinaryDiffEq no longer accepts a `Bool` for
  `autodiff`; it requires an ADTypes specifier. `AutoFiniteDiff` is exported
  by `using OrdinaryDiffEq` (already loaded in both), preserving the original
  finite-difference Jacobian intent.

- `lastindex(sol)` -> `lastindex(sol.u)` in porous_fisher (BoundsError:
  index [102] into a 101-element Vector), and `eachindex(fvm_sol)` ->
  `eachindex(fvm_sol.u)` in the diffusion_equations and
  linear_reaction_diffusion WYOS tutorials (MethodError: `fig[2, j]` with
  `j::CartesianIndex`). Same SciMLBase 3 solution-index change the earlier
  commit handled elsewhere: `eachindex(sol)`/`length(sol)`/`lastindex(sol)`
  now span the flattened `(state..., time)` index space, so iterate the
  time-point storage via `sol.u` instead.

- piecewise_linear `get_errs`: `length(_sol)`/`eachindex(_sol)` ->
  `length(_sol.u)`/`eachindex(_sol.u)`. Under SciMLBase 3 the old form
  iterated `CartesianIndices`, so the `has_vertex` guard skipped every
  iteration and `@test all(≤(0.16), _errs)` passed vacuously against an
  all-zero error array. Restored to iterate time points so the assertion
  again checks the FVM-vs-exact solution error (verified: max error ≤ 0.16).

Reference images (regenerated, not threshold-loosened): conditions.png,
internal_conditions.png, and the piecewise natural-neighbour interpolation
figure. These are upstream Makie/DelaunayTriangulation `triplot` recipe
rendering changes since the references were last cut (Feb 2024), not FVM
regressions. Verified the underlying data is correct (the annulus mesh has
152 triangles and 0 in the hole; the condition-node counts and the piecewise
error assertion all check out) and the regenerated images pass the reference
tests on a fresh non-update run locally.

Documentation job dependency resolution: relax docs `SimpleGraphs` compat
`0.8.10` -> `0.8.8 - 0.8`. `SimpleGraphs 0.8.10` forces `Optim 2.x` ->
`NLSolversBase 8`, which is mutually exclusive with `NLsolve 4.5.1` (pulled
by DiffEqDevTools) capping `NLSolversBase` at 7. Allowing 0.8.9 lets the
resolver pick `Optim 1.13` -> `NLSolversBase 7.10`, which co-installs.
SimpleGraphs is not imported anywhere (docs-only dep) and CI skips example
execution, so the patch downgrade is inert. Verified the docs environment now
resolves cleanly.

Local verification on Julia 1.12 (the "1" channel):
- maze, disk, porous_fisher, diffusion_equations, linear_reaction_diffusion
  examples all run with no error.
- piecewise example: 5/5 pass (incl. the now-real error assertion) and its
  regenerated reference passes.
- conditions.jl: BoundaryConditions + InternalConditions reference tests pass.
- docs env resolves with SimpleGraphs 0.8.9 / Optim 1.13 / NLSolversBase 7.10.
- All changed .jl files are Runic-clean.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ChrisRackauckas-Claude

Copy link
Copy Markdown
Contributor Author

Triage update (independent re-verification, no new code changes)

Re-checked the latest CI run for this branch (run 27834234855, commit 52dbed5). State of that run:

  • PASS: Documentation, Downgrade, Runic, Runic Suggestions, Spell Check, QA (julia 1 + lts), Core (julia lts / 1.10).
  • FAIL: Core (julia 1 / 1.12) and Core (julia pre / 1.13) only.

Both failing jobs ran 68 min (1.12) and 86 min (pre), well under the 120-min group timeout, so these are genuine in-suite failures, not timeouts. The per-job "Run tests" logs for that run are no longer retrievable via the GitHub API (purged for the 18h-old run), so I triaged at the source level instead.

Code-level fixes on this branch are complete and correct

Scanned every Core-run file (tutorials, wyos, test/) for the four API-break classes this PR addresses:

  • autodiff = false/true (Bool) → none remain (all AutoFiniteDiff()).
  • bare TRBDF2 without using OrdinaryDiffEqSDIRK → none.
  • MatrixOperator without using SciMLOperators → none.
  • eachindex/length/lastindex(sol) / scalar sol[i] on a solution (SciMLBase 3) without .u → none.

Independently verified the load-bearing imports on the failing channel (Julia 1.12.6, +1)

Fresh env, dev'd this checkout + OrdinaryDiffEqSDIRK + SciMLOperators:

  • FiniteVolumeMethod precompiles cleanly (ChunkSplitters v3 index_chunks fix confirmed — no MethodError: chunks(::Vector, ::Int)).
  • TRBDF2 resolves, MatrixOperator resolves, solve is defined/exported by the module.

Remaining blocker (not cleanly fixable here)

With all load/import/API patterns clean and lts (1.10) green while only 1.12/1.13 are red, the remaining Core failures are almost certainly the environment-sensitive @test_reference image PSNR comparisons (45 reference PNGs, CairoMakie rendering) — exactly the CI-runner-confirmation caveat noted in the commit above. These cannot be correctly resolved without the actual CI failure logs identifying which images mismatch by what margin; blindly regenerating images on a different local machine would not be verifiable and risks masking a real regression. Recommend a fresh CI run to capture the Core (1.12) "Run tests" log and pinpoint the specific failing reference image(s).

Please ignore until reviewed by @ChrisRackauckas.

…image (Core julia 1/pre)

The Core test group passed on lts but failed on the julia 1 (1.12) and pre
channels at the piecewise/natural-neighbour tutorial's
`*_natural_neighbour_interpolation.png` @test_reference comparison (PSNR
below threshold).

Root cause: the tutorial built its mesh with
`refine!(tri; ...)` and a grid triangulation with
`triangulate([...])`, both of which default to `rng = Random.default_rng()`
in DelaunayTriangulation. The Core group runs every tutorial sequentially in
one process via safe_include, so by the time this tutorial (8th in the list)
runs, the seven preceding tutorials have already advanced the global RNG by a
data-dependent amount. Since the default RNG stream is not stable across Julia
versions, the resulting triangulation -- and therefore every panel of the
16-panel figure -- differed from the committed reference on 1.12/pre while it
happened to stay within PSNR threshold on lts. Run standalone (fresh process)
the figure reproduced the committed reference byte-for-byte; only the in-suite
RNG ordering tripped it.

Fix: pass `rng = StableRNG(123)` to both `refine!` (line 42) and the grid
`triangulate` (line 221), making the triangulation -- and the figure --
deterministic regardless of preceding RNG consumption or Julia version.
StableRNGs is already a dep of both the test and docs environments (used by
the maze tutorial), so no new dependency is added. Mirrored the same two-line
change into the generated tutorials .md. Regenerated the four reference images
in this tutorial to match the seeded triangulation.

Local verification on Julia 1.12.6 (the "1" channel):
- Rendering the tutorial in a fresh process and in a process where the global
  RNG was first advanced by ~1,000,060 draws produced byte-identical output
  for all four figures (md5 match) -- confirming the figure is now independent
  of RNG state.
- Re-ran the tutorial in non-update mode after advancing the global RNG by the
  same amount (the exact condition that fails in the full Core sequence): 5/5
  pass against the newly committed seeded references, including the previously
  failing natural-neighbour reference and the real piecewise error assertion.
- The changed .jl is Runic-clean (`Runic.main(["--check", ...])` exit 0).

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ChrisRackauckas ChrisRackauckas marked this pull request as ready for review June 21, 2026 11:12
@ChrisRackauckas ChrisRackauckas merged commit 1973a14 into SciML:main Jun 21, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants