Add QA (Aqua/JET) groups to sublibraries (SciML/.github#77)#1223
Merged
Conversation
…, OptimizationBBO Part of SciML/.github#77: canonical QA groups for sublibraries. Each unit gets an isolated test/qa environment (Aqua + JET), a QA dispatch in test/runtests.jl gated on OPTIMIZATION_TEST_GROUP == "QA" (mirroring the root package's test/qa pattern), and a [QA] cell in test/test_groups.toml on versions [lts, 1]. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…timizationEvolutionary, OptimizationGCMAES Part of SciML/.github#77: canonical QA groups for sublibraries, same pattern as the previous commit. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…B, OptimizationMadNLP Part of SciML/.github#77: canonical QA groups for sublibraries, same pattern as the previous commits. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…heuristics, OptimizationMOI Part of SciML/.github#77: canonical QA groups for sublibraries, same pattern as the previous commits. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…OptimizationNLopt, OptimizationNLPModels Part of SciML/.github#77: canonical QA groups for sublibraries, same pattern as the previous commits. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…OptimizationOptimisers Part of SciML/.github#77: canonical QA groups for sublibraries, same pattern as the previous commits. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…yalgorithms, OptimizationPRIMA Part of SciML/.github#77: canonical QA groups for sublibraries, same pattern as the previous commits. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…IRECT, OptimizationSciPy Part of SciML/.github#77: canonical QA groups for sublibraries, same pattern as the previous commits. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…dMapping, SimpleOptimization Part of SciML/.github#77: canonical QA groups for sublibraries, same pattern as the previous commits. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ChrisRackauckas-Claude
pushed a commit
to ChrisRackauckas-Claude/Optimization.jl
that referenced
this pull request
Jul 3, 2026
Convert the root test/qa and every lib/<Sub>/test/qa to SciMLTesting 1.6's
`run_qa(<Module>; explicit_imports = true, ...)` form and enable the six
ExplicitImports checks across the whole monorepo.
Per QA env:
* qa.jl -> `using SciMLTesting, <Module>, JET; using Test` + a single run_qa call.
* test/qa/Project.toml -> SciMLTesting compat "1.6" (+ dep); Aqua and JET kept
(both run); ExplicitImports comes transitively from SciMLTesting. [sources]
wiring preserved.
ExplicitImports: 0 hard FAIL everywhere. no_implicit_imports is the facade
`@reexport`/`using` module-name reliance and is marked ei_broken fleet-wide; the
public-API and via-owners findings are per-env `ignore`-lists of names owned by
SciMLBase/OptimizationBase/the backend/Base that are not (yet) declared public;
stale explicit imports are kept+ignored (re-surfaced API / reached as Mod.name).
PyCMA and SciPy load CodaPkg-backed Python at precompile, so their EI checks are
conservatively ei_broken (verified statically from their source).
Also clears the pre-existing Aqua reds that left the new sublibrary QA lanes red
since SciML#1223 (these fail identically under the old `Aqua.test_all` form):
* deps_compat: add the missing [compat] entries for [extras]/[deps]/[weakdeps]
across the sublibraries (stdlibs -> "1"; others to a permissive floor).
* stale_deps: drop the unused `PDMats` dep from OptimizationBase; move the
test-only `Manifolds` from [deps] to [extras]+test target in OptimizationManopt.
* piracies: each solver-wrapper intentionally extends SciMLBase's
trait/__init/__solve interface onto its backend optimizer types -> declare
those foreign types via `aqua_kwargs = (; piracies = (; treat_as_own = [...]))`.
* undefined_exports: MadNLP/Manopt (`solve!`), Metaheuristics (`summary`) and
NOMAD (`solve`) `@reexport` two packages exporting the same name, leaving it an
unresolved conflict binding -> tracked as `aqua_broken = (:undefined_exports,)`.
Verified against released SciMLTesting 1.6.0 on Julia 1.10 (QA lanes run lts+1,
never pre): root, OptimizationBase, OptimizationBBO, OptimizationMOI,
OptimizationEvolutionary, OptimizationOptimisers, OptimizationOptimJL,
OptimizationManopt, OptimizationMadNLP, OptimizationMetaheuristics and
OptimizationNOMAD all green-or-broken with 0 FAIL; the remaining Aqua sub-checks
(ambiguities/unbound_args/project_extras/persistent_tasks) verified clean across
all 24 loadable sublibraries. CI covers the rest.
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ChrisRackauckas
added a commit
that referenced
this pull request
Jul 5, 2026
* QA: run_qa v1.6 form + ExplicitImports (root + sublibs)
Convert the root test/qa and every lib/<Sub>/test/qa to SciMLTesting 1.6's
`run_qa(<Module>; explicit_imports = true, ...)` form and enable the six
ExplicitImports checks across the whole monorepo.
Per QA env:
* qa.jl -> `using SciMLTesting, <Module>, JET; using Test` + a single run_qa call.
* test/qa/Project.toml -> SciMLTesting compat "1.6" (+ dep); Aqua and JET kept
(both run); ExplicitImports comes transitively from SciMLTesting. [sources]
wiring preserved.
ExplicitImports: 0 hard FAIL everywhere. no_implicit_imports is the facade
`@reexport`/`using` module-name reliance and is marked ei_broken fleet-wide; the
public-API and via-owners findings are per-env `ignore`-lists of names owned by
SciMLBase/OptimizationBase/the backend/Base that are not (yet) declared public;
stale explicit imports are kept+ignored (re-surfaced API / reached as Mod.name).
PyCMA and SciPy load CodaPkg-backed Python at precompile, so their EI checks are
conservatively ei_broken (verified statically from their source).
Also clears the pre-existing Aqua reds that left the new sublibrary QA lanes red
since #1223 (these fail identically under the old `Aqua.test_all` form):
* deps_compat: add the missing [compat] entries for [extras]/[deps]/[weakdeps]
across the sublibraries (stdlibs -> "1"; others to a permissive floor).
* stale_deps: drop the unused `PDMats` dep from OptimizationBase; move the
test-only `Manifolds` from [deps] to [extras]+test target in OptimizationManopt.
* piracies: each solver-wrapper intentionally extends SciMLBase's
trait/__init/__solve interface onto its backend optimizer types -> declare
those foreign types via `aqua_kwargs = (; piracies = (; treat_as_own = [...]))`.
* undefined_exports: MadNLP/Manopt (`solve!`), Metaheuristics (`summary`) and
NOMAD (`solve`) `@reexport` two packages exporting the same name, leaving it an
unresolved conflict binding -> tracked as `aqua_broken = (:undefined_exports,)`.
Verified against released SciMLTesting 1.6.0 on Julia 1.10 (QA lanes run lts+1,
never pre): root, OptimizationBase, OptimizationBBO, OptimizationMOI,
OptimizationEvolutionary, OptimizationOptimisers, OptimizationOptimJL,
OptimizationManopt, OptimizationMadNLP, OptimizationMetaheuristics and
OptimizationNOMAD all green-or-broken with 0 FAIL; the remaining Aqua sub-checks
(ambiguities/unbound_args/project_extras/persistent_tasks) verified clean across
all 24 loadable sublibraries. CI covers the rest.
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* QA: trim public-API EI ignore-lists to what's still non-public on released base libs
SciMLTesting 1.7.0 now runs check_all_qualified_accesses_are_public and
check_all_explicit_imports_are_public only on Julia >= 1.11 (skipped on the LTS),
and the base-lib make-public round shipped (SciMLBase 3.24.0, OptimizationBase 5.1.3,
CommonSolve 0.2.9, Symbolics 7.28.1, SymbolicUtils 4.36.0, ModelingToolkit 11.29.0,
ArrayInterface 7.26.0). Many per-repo public-API ignore entries are therefore now
redundant.
Emptied both public-API ignore-lists in every qa env, then re-derived the minimal
surviving set by running each QA group on Julia 1.12 against the registered releases
(Pkg-resolved, no dev) and restoring only the names the checks still flag as
non-public. The flagged names are owned by SciMLBase / OptimizationBase / Base /
SciMLStructures / SymbolicUtils / the backend package and remain non-public on the
released versions (imported/accessed from a module that has not declared them public);
the proper fix is upstream `public` declarations, tracked against SciML/Optimization.jl.
Removed entries are names the make-public round turned public (e.g. Success, Failure,
MaxIters, MaxTime, build_solution, has_init, NullParameters, setup, update, ...).
Verified on Julia 1.12 (public checks run): each QA group's two public checks pass with
the trimmed lists. Verified on Julia 1.10 LTS (public checks skipped per SciMLTesting
1.7): root + OptimizationBase + OptimizationOptimisers + OptimizationNLPModels QA groups
green-or-broken (15 pass / 1 broken / 0 fail). OptimizationPyCMA and OptimizationSciPy
keep all_qualified_accesses_are_public in ei_broken (CondaPkg-gated envs, not analyzable
without conda). no_implicit_imports / no_stale_explicit_imports / *_via_owners ignores
and aqua/jet markers are untouched.
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* QA: re-trim public-API EI ignore-lists against the live make-public releases (SciMLBase 3.27.0)
The prior trim was derived against SciMLBase 3.24.0 / OptimizationBase 5.1.3. The
full make-public round is now registered (SciMLBase 3.27.0), which turned several
more accessed/imported names public — most consistently the solver-trait predicates
`allowsbounds` and `allowsconstraints`, plus `build_solution` and the abstract
algorithm type `AbstractDAEAlgorithm`. Those entries are now redundant in the
per-repo public-API ignore-lists.
Re-derived the minimal surviving set for every analyzable qa env by running
ExplicitImports' check_all_qualified_accesses_are_public /
check_all_explicit_imports_are_public with an empty ignore on Julia 1.12 against the
Pkg-resolved registered releases (no dev), and set each ignore-list to exactly the
names the checks still flag. OptimizationQuadDIRECT was resolved with its git-only
`QuadDIRECT` dependency added so it could be analyzed like CI does.
Removed (now public on the resolved SciMLBase 3.27.0): `allowsbounds`,
`allowsconstraints` (Auglag/BBO/CMAEvolutionStrategy/Evolutionary/GCMAES/Ipopt/
LBFGSB/MadNLP/Metaheuristics/MOI/MultistartOptimization/NLopt/NOMAD/ODE/OptimJL/
PRIMA/QuadDIRECT/SpeedMapping as applicable), `build_solution` (Optimisers),
`AbstractDAEAlgorithm` (ODE). OptimizationBase drops `allowsbounds`/`allowsconstraints`
from all_explicit_imports_are_public.
Survivors kept are genuinely still non-public on the resolved releases and are owned
by SciMLBase (`__init`/`__solve`/`allowscallback`/`requires*`/`MaxSense`/`MinSense`/
`ObjSense`/`NoAD`/`Abstract*`/...), OptimizationBase (`OptimizationStats`/`OptimizationState`/
`_check_and_convert_*`/`ReInitCache`/`instantiate_function`/`supports_sense`/...), Base
(`@logmsg`/`LogLevel`/`structdiff`), ForwardDiff (`gradient`), or each backend
(BlackBoxOptim/CMAEvolutionStrategy/Evolutionary/Ipopt/MadNLP/Manopt/Metaheuristics/
MathOptInterface/QuadDIRECT/...). The proper fix for the SciMLBase/OptimizationBase
survivors is upstream `public` declarations, tracked against SciML/Optimization.jl.
OptimizationManopt is unchanged: its dependency tree resolves SciMLBase 2.155.1, where
`build_solution`/`allowsbounds` remain non-public, so its list is already minimal there.
Verified on Julia 1.12 (public checks run): root + OptimizationBase +
OptimizationOptimisers + OptimizationMOI QA groups each pass both public-API EI checks
with the trimmed lists (ExplicitImports testset 5 pass / 1 broken / 0 fail — the broken
one is the intentional no_implicit_imports). The remaining `JET-test failed` in the
OptimizationBase/Optimisers/MOI QA groups is a pre-existing OptimizationBase 5.1.3 source
finding (undefined local `lagrangian`/`cons_oop` in OptimizationDISparseExt.jl) and is
independent of this EI-only change. no_implicit_imports / no_stale_explicit_imports /
*_via_owners ignores and aqua/jet markers are untouched. OptimizationPyCMA and
OptimizationSciPy keep their public checks in ei_broken (CondaPkg-gated, not analyzable
without conda).
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* QA: fix conversion-surfaced JET/EI reds in sublibrary qa lanes
The run_qa v1.6 conversion newly runs JET typo-mode and the public-API
ExplicitImports checks, surfacing issues the old hand-rolled qa.jl never
exercised. Resolve each per its nature:
Real fixes (verified JET-clean):
* OptimizationGCMAES: hoist the gradient closure `g` out of the
`!isnothing(cache.f.grad)` guard. `g` was defined only inside the guard
but referenced in the `isnothing(...) ? _loss : (_loss, g)` ternary, which
JET flagged as `g may be undefined`. The closure body never calls
`cache.f.grad` at construction, so defining it unconditionally is behavior-
preserving.
* OptimizationSciPy: `safe_get_message` called `pytypeof`, which does not
exist in PythonCall (the function is `pytype`). This is a genuine latent
UndefVarError on the non-str/non-list message fallback path; JET caught it.
EI ignore-list gap:
* SimpleOptimization: add the still-non-public `OptimizationStats`, `__solve`,
and `_check_and_convert_maxiters` to `all_qualified_accesses_are_public`.
Documented jet_broken=true (pre-existing latent issues the conversion merely
surfaces; src unchanged from master, fixes belong in separate solver PRs):
* OptimizationLBFGSB (`res`), OptimizationOptimisers (`x`),
OptimizationSophia (`x`), OptimizationMetaheuristics (`opt_bounds`):
loss/result variables assigned only inside a loop/guard and read after.
* OptimizationNOMAD (`bb`/`bbcons`): closures defined in correlated if/else
arms JET cannot prove correlated.
* OptimizationNLopt (`thetacache`/`Jthetacache`): closure-captured-and-
reassigned bindings boxed by Julia; benign closure-capture artifact.
Verified locally on Julia 1.12.6 (QA lane = julia-1): each affected sublib's
QA group passes with 0 fail / 0 error (broken markers as documented).
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
* QA: drop spurious jet_broken from LBFGSB/Sophia (JET reports nothing)
Both lib/OptimizationLBFGSB and lib/OptimizationSophia qa.jl marked JET as
`jet_broken = true`, on the premise that JET typo-mode would flag a possibly-
undefined local (`res`/`x`) in `__solve`. That premise is wrong: in both
solvers the variable is initialized unconditionally *before* the loop that
mutates it — `res = (cache.u0, nothing)` (OptimizationLBFGSB.jl:163, before the
`for i in 1:maxiters` at :180) and `x = cache.f(θ, first(data))`
(OptimizationSophia.jl:117, before the `for epoch in 1:maxiters` at :118). So
there is no latent undefined-variable path and JET produces zero reports.
With `jet_broken = true` the internal `@test_broken isempty(JET.get_reports(rep))`
therefore hits an "Unexpected Pass" and errors the QA job. These were the only
two sublibs using `jet_broken`, and they were the only two failing QA lanes.
Fix: remove `jet_broken = true` (JET now runs as a normal passing `@test`) and
drop the incorrect explanatory comment. No source change is warranted since the
described bug does not exist.
Verified GROUP=QA on Julia 1.12.6 vs released SciMLTesting 1.6.0:
LBFGSB 17 pass / 1 broken / 0 error; Sophia 17 pass / 1 broken / 0 error
(the remaining 1 broken is the pre-existing legit ei_broken=no_implicit_imports).
Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
---------
Co-authored-by: Chris Rackauckas <accounts@chrisrackauckas.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of SciML/.github#77: add canonical QA (Aqua + JET) groups to the sublibraries.
Each unit gets:
test/qa/Project.toml— isolated QA environment (Aqua 0.8, JET 0.9/0.10/0.11, the package via[sources]path), so QA tooling deps never enter the main test target's resolve.test/qa/qa.jl—Aqua.test_all+JET.test_package(...; target_defined_modules = true).test/runtests.jlgated onOPTIMIZATION_TEST_GROUP == "QA", mirroring the root package'stest/qapattern (including the Julia < 1.11Pkg.developfallback since[sources]is ignored there).Pkgwired into the unit's test target ([extras]+[targets], with a[compat]entry so Aquadeps_compatstays green).[QA]cell intest/test_groups.tomlwithversions = ["lts", "1"](nopre), picked up by the SublibraryCI matrix.Units in this batch (9):
Verification: static only (TOML parse,
Meta.parseallof the new/changed.jlfiles,[sources]path resolution, Runic check on changed files all pass locally). QA findings that CI surfaces will be triaged per the established mark-broken+issue policy.Further batches may be pushed to this branch. Ignore until reviewed by @ChrisRackauckas.
🤖 Generated with Claude Code