Skip to content

Canonical CI: grouped-tests.yml + root test/test_groups.toml#111

Merged
ChrisRackauckas merged 4 commits into
SciML:mainfrom
ChrisRackauckas-Claude:grouped-tests-ci
Jun 15, 2026
Merged

Canonical CI: grouped-tests.yml + root test/test_groups.toml#111
ChrisRackauckas merged 4 commits into
SciML:mainfrom
ChrisRackauckas-Claude:grouped-tests-ci

Conversation

@ChrisRackauckas-Claude

@ChrisRackauckas-Claude ChrisRackauckas-Claude commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Summary

Converts the root test workflow to the canonical SciML grouped-tests thin caller, with the version/group matrix declared in a root test/test_groups.toml.

  • .github/workflows/Tests.yml — replaced the inline version-matrix job (which called tests.yml@v1 over version: [1.11, pre, lts]) with the canonical thin caller: 
    jobs:
  tests:
    uses: "SciML/.github/.github/workflows/grouped-tests.yml@v1"
    secrets: "inherit"
    on: and concurrency: are preserved verbatim, name: "Tests" is kept, filename unchanged. All grouped-tests inputs left at defaults (runtests.jl reads the canonical GROUP env var; Linux-only, no os field).
  • test/test_groups.toml (new, root) — [Core] on [lts, 1, pre]; [QA] on [lts, 1].
  • test/runtests.jl — refactored for GROUP dispatch. Functional testsets (Geometry, Conditions, Problem, Equations, README, Tutorials, Custom Templates) run under Core (and the default All). The Aqua + ExplicitImports checks move into a QA group gated on GROUP == "QA", which activates an isolated env, develops the package, instantiates, and includes qa.jl.
  • test/qa/Project.toml (new) — isolated QA env: Aqua, ExplicitImports, Test + the package via [sources] path = "../.."; [compat] julia = "1.10".
  • test/qa.jl (new) — the Aqua + ExplicitImports checks, moved unchanged (no exclusions added).
  • Project.toml — benign metadata fixes: [compat] julia bumped "1" → "1.10" (LTS floor; also clears Aqua's <=1.1 project_extras check), and added Test = "1" compat for the existing [extras] Test entry.

Matrix match

Verified statically with compute_affected_sublibraries.jl --root-matrix@v1, emitted set:

group versions runner
Core lts, 1, pre ubuntu-latest
QA lts, 1 ubuntu-latest

Mapping vs the old matrix (version: [1.11, pre, lts], single un-grouped run of the whole suite including Aqua/ExplicitImports):

  • Functional version coverage preserved/broadened: old {1.11→1, pre, lts} → Core {lts, 1, pre}.
  • Aqua/ExplicitImports now run as the dedicated QA group on [lts, 1] (canonical pattern). Previously they ran on all three including pre; QA dropping pre is the intended canonical reduction.

Notes

  • This is a structural conversion. No tests, Aqua, or JET were run locally.
  • The QA group is newly wired; Aqua/JET (Aqua + ExplicitImports here) run in CI — any failures will be triaged in a follow-up. No checks were skipped, silenced, or excluded.

Ignore until reviewed by @ChrisRackauckas.

🤖 Generated with Claude Code

@ChrisRackauckas-Claude

ChrisRackauckas-Claude commented Jun 14, 2026

Copy link
Copy Markdown
Contributor Author

Added the SciMLTesting v1.2 folder-based run_tests harness on top of this grouped-tests conversion; merges as one PR (grouped-tests + SciMLTesting v1.2).

@ChrisRackauckas ChrisRackauckas marked this pull request as ready for review June 15, 2026 04:57
ChrisRackauckas and others added 4 commits June 15, 2026 10:49
@ChrisRackauckas @claude
Canonical CI: grouped-tests.yml + root test/test_groups.toml
c8a84bc 
Convert the root test workflow (Tests.yml) to the canonical thin caller
delegating to SciML/.github grouped-tests.yml@v1, with the version/group
matrix declared in test/test_groups.toml.

- Tests.yml: replace the version-matrix job with the grouped-tests.yml@v1
  thin caller (on:/concurrency: preserved verbatim, name: kept). All
  defaults; runtests.jl reads canonical GROUP, Linux-only.
- test/test_groups.toml: [Core] on [lts,1,pre]; [QA] on [lts,1].
- Refactor test/runtests.jl for GROUP dispatch. Functional testsets run
  under Core (and the default All). Aqua + ExplicitImports move into a QA
  group gated on GROUP=="QA".
- test/qa/ isolated environment (Aqua, ExplicitImports, Test + the package
  via [sources] path) and test/qa.jl carrying the moved checks unchanged.
- Project.toml: bump julia compat 1 -> 1.10 (LTS floor); add Test = "1"
  compat for the [extras] Test entry (benign Aqua metadata fixes).

No tests/Aqua/JET were run locally; QA is newly wired and will run in CI.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ChrisRackauckas @claude
Add Pkg to test deps for grouped-tests Core group
596ac6b 
The Core test group runs with project='.', and test/runtests.jl does
`using Pkg` to activate the QA sub-environment. Pkg was not a declared
test dependency in the root test env, so the Core job failed with
`ArgumentError: Package Pkg not found in current path`.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ChrisRackauckas @claude
Drop unused test deps with unsatisfiable LTS/1.x compat pins
3ad9f43 
The grouped-tests-ci branch added hard single-version compat pins to the
shared root test/Project.toml. Three of those deps are never imported by
any test or doc (JET, Cthulhu, SparseDiffTools) yet their pins make the
shared test env unsatisfiable on the QA julias (lts=1.10, 1=1.11):

- JET = 0.11.3 and Cthulhu = 3.0.2 require julia >= 1.12.
- SparseDiffTools = 2.26.0 forces SciMLOperators 0.3-0.4, conflicting with
  the OrdinaryDiffEq 7 / NonlinearSolve 4 stack the tests actually use
  (which needs SciMLOperators 1.x).

Removing these dead deps lets the root test env resolve on 1.10/1.11 and
unblocks every group (QA and Core) without weakening any real test.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ChrisRackauckas @claude
Adopt SciMLTesting v1.2 folder-based run_tests (on top of grouped-tes…
00d007b 
…ts conversion)

FVM's Core group is a capability/shared-module driver: the test files use
`using ..FiniteVolumeMethod` and a shared `test_functions.jl` include, and Core also
runs the docs/src/literate_tutorials and literate_wyos example files (outside test/)
with file-count assertions. That does not partition into self-contained group
folders, so this uses v1.2 explicit-args mode rather than forcing folder-discovery:

  * The whole original Core driver (the `safe_include` helper + the nested
    `@testset`s, verbatim) moves into test/core_tests.jl and is passed as the
    `core =` file body. The `@safetestset` wrapper supplies `using Test`, and
    core_tests.jl's `using FiniteVolumeMethod` makes the nested-module
    `using ..FiniteVolumeMethod` in each included test file resolve exactly as before.
  * QA: top-level test/qa.jl moves into test/qa/qa.jl (the existing qa sub-env), run
    as `qa = (; env = test/qa, body = test/qa/qa.jl)`.
  * `all = ["Core"]` keeps GROUP=All to Core only (QA stays its own lane), matching
    the original `if GROUP == "QA" ... else <Core> end` routing.

Adds SciMLTesting + SafeTestsets to the main test env and the qa sub-env; drops Pkg
from the test env (only the old hand-rolled harness used it).

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ChrisRackauckas ChrisRackauckas merged commit ff86515 into SciML:main Jun 15, 2026
4 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@ChrisRackauckas-Claude @ChrisRackauckas