Skip to content

Use SciMLTesting v1.1.0 run_tests harness#1487

Draft
ChrisRackauckas-Claude wants to merge 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:sciml-testing-rollout
Draft

Use SciMLTesting v1.1.0 run_tests harness#1487
ChrisRackauckas-Claude wants to merge 1 commit into
SciML:masterfrom
ChrisRackauckas-Claude:sciml-testing-rollout

Conversation

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Contributor

Replaces the hand-written GROUP if-ladder in test/runtests.jl with a single declarative run_tests call from SciMLTesting v1.1.0.

What changed

  • The six functional groups (Modeling, Simulation, Hybrid, Misc, Spatial, Extensions) are passed as groups entries.
  • A curated all = ["Modeling", "Simulation", "Hybrid", "Misc", "Spatial", "Extensions"] reproduces the original All membership exactly: bare/All runs all six groups in the original order. This repo has no QA group.
  • The Extensions group keeps its isolated test/extensions environment via the env kwarg, which performs the same Pkg.activate / Pkg.develop(repo root) / Pkg.instantiate that the old activate_extensions_env() helper did.

Behavior equivalence

This is a behavior-equivalent refactor — no test assertions or logic changed, and the set of testsets run under each GROUP value (including bare/All) is identical before and after. Verified with a dispatch dry-run:

GROUP groups run
All Modeling, Simulation, Hybrid, Misc, Spatial, Extensions
Modeling / Simulation / Hybrid / Misc / Spatial / Extensions the named group only

Project.toml

  • Added SciMLTesting (09d9d899-5365-40a9-917a-5f67fddea283) to [extras], [compat] SciMLTesting = "1", and the test target.
  • Dropped Pkg from the test deps — only the old harness used it; run_tests bundles Pkg.

Ignore until reviewed by @ChrisRackauckas.

Replace the hand-written `GROUP` if-ladder in `test/runtests.jl` with a
single declarative `run_tests` call from SciMLTesting v1.1.0.

The six functional groups (Modeling, Simulation, Hybrid, Misc, Spatial,
Extensions) are passed as `groups` entries, and a curated `all = [...]`
reproduces the original `All` membership exactly: All runs all six groups,
in the original order. There is no QA group in this repo. The Extensions
group keeps its isolated `test/extensions` env via the `env` kwarg (which
performs the same Pkg.activate / develop(root) / instantiate that the old
`activate_extensions_env` did).

This is a behavior-equivalent refactor: the set of testsets run under each
GROUP value (including bare/All) is identical before and after, verified
with a dispatch dry-run.

Project.toml: add SciMLTesting (09d9d899-5365-40a9-917a-5f67fddea283) to
[extras], [compat] SciMLTesting = "1", and the test target; drop Pkg from
the test deps (only the old harness used it; run_tests bundles Pkg).

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ChrisRackauckas-Claude

Copy link
Copy Markdown
Contributor Author

Rebased onto current master (picks up #1497 "QA: run_qa v1.6" and #1491 codecov bump).

Rebase integration notes:

  • The QA group that QA: run_qa v1.6 form + ExplicitImports #1497 added (hand-written if GROUP == "QA"; activate_qa_env(); include("qa/qa.jl")) is now expressed via the run_tests qa kwarg: qa = (; env = joinpath(@__DIR__, "qa"), body = joinpath(@__DIR__, "qa", "qa.jl")). This preserves the exact behavior: QA runs only under GROUP=QA, never as part of All (verified against SciMLTesting 1.1.0 and 1.8.0 — curated all omits "QA", so All runs only the 6 functional groups).
  • Dropped the stale test/miscellaneous_tests/explicit_imports.jl include from the Misc group — QA: run_qa v1.6 form + ExplicitImports #1497 deleted that file (its ExplicitImports checks moved into the QA run_qa suite), so the pre-rebase include would have errored the Misc lane. Misc now matches master (only "ReactionSystem Serialisation").
  • Root Project.toml compat bumped to SciMLTesting = "1.1" (was "1"): 1.0.0 lacks the all/qa kwargs, so the Downgrade lane would have resolved a version that cannot run run_tests. Both kwargs exist since 1.1.0.

Group dispatch verified locally (dry-run, stubbed bodies) on both the compat floor (1.1.0) and latest (1.8.0): All -> Modeling, Simulation, Hybrid, Misc, Spatial, Extensions; each named group -> itself; QA -> QA; QA excluded from All. All 42 include/env paths resolve to existing files.

Ignore until reviewed by @ChrisRackauckas.

@TorkelE

TorkelE commented Jul 5, 2026

Copy link
Copy Markdown
Member

Currently trying to fix tests here: #1488. Was done, but then some new breaking things popped up, and then I went travelling and was unable toacess computer for a few days

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants