Sublibrary QA-lts fails: OptimizationBase test-only [sources] (LBFGSB/Manopt) transitively developed into every dependent sublibrary on Julia < 1.11

[ChrisRackauckas-Claude]

Summary

Every sublibrary's QA group fails on Julia lts (< 1.11) with Aqua Stale dependencies and deps-compat findings for OptimizationLBFGSB and OptimizationManopt, even when the sublibrary has nothing to do with those packages. QA on Julia 1 (>= 1.11) is green. This is independent of any feature PR; it reproduces on a clean master.

Example (CMAEvolutionStrategy QA-lts, but the same applies to ~26 sublibraries):

Stale dependencies: Test Failed at .../Aqua/.../stale_deps.jl:32
  Evaluated: isempty(Base.PkgId[OptimizationLBFGSB [22f7324a-...], OptimizationManopt [e57b7fff-...]])
OptimizationCMAEvolutionStrategy does not declare a compat entry for the following deps:
  OptimizationLBFGSB [22f7324a-...]
  OptimizationManopt [e57b7fff-...]

Root cause

1. lib/OptimizationBase/Project.toml declares a [sources] table for its test-only deps:

   [sources]
   OptimizationLBFGSB = {path = "../OptimizationLBFGSB"}
   OptimizationManopt = {path = "../OptimizationManopt"}

   OptimizationLBFGSB/OptimizationManopt are NOT in OptimizationBase's [deps] — they appear only in [extras]/[targets].test (OptimizationBase's own test suite dev-deps them). The [sources] is there so OptimizationBase's own tests resolve on Julia < 1.11.

2. The centralized SciML/.github tests.yml "Develop in-repo [sources] path deps" step walks the [sources] graph transitively on Julia < 1.11:

   queue = [project]; while !isempty(queue): read [sources]; develop each path-dep; push it to queue (resolve its [sources] too)
   

   So testing sublibrary X (which sources OptimizationBase) develops OptimizationBase, then reads OptimizationBase's [sources] and Pkg.develops OptimizationLBFGSB + OptimizationManopt into X's environment as phantom direct deps. Aqua (run against X) then flags them as stale / missing-compat.

On Julia >= 1.11 the whole helper block is a no-op (if VERSION < v"1.11.0-DEV.0"), which is exactly why QA / Julia 1 passes and QA / Julia lts fails.

Affected sublibraries (26)

All sublibraries with OptimizationBase = {path = ...} in [sources]: OptimizationMetaheuristics, OptimizationManopt, OptimizationCMAEvolutionStrategy, OptimizationAuglag, OptimizationBBO, OptimizationLBFGSB, OptimizationMOI, OptimizationIpopt, OptimizationQuadDIRECT, OptimizationNOMAD, OptimizationEvolutionary, OptimizationMultistartOptimization, OptimizationNLPModels, OptimizationPyCMA, OptimizationNLopt, OptimizationPolyalgorithms, OptimizationOptimJL, OptimizationGCMAES, OptimizationSpeedMapping, SimpleOptimization, OptimizationSophia, OptimizationSciPy, OptimizationODE, OptimizationMadNLP, OptimizationPRIMA, OptimizationOptimisers.

Reproduction (local, Julia 1.10)

Replicate the develop helper, then run QA:

using Pkg
proj = "lib/OptimizationCMAEvolutionStrategy"
Pkg.activate(proj)
developed = Set{String}([normpath(abspath(proj))]); specs = Pkg.PackageSpec[]; queue = String[proj]
while !isempty(queue)
    dir = popfirst!(queue); t = joinpath(dir, "Project.toml"); isfile(t) || continue
    toml = Pkg.TOML.parsefile(t); haskey(toml, "sources") || continue
    for (_, s) in toml["sources"]
        if s isa Dict && haskey(s, "path")
            p = normpath(abspath(joinpath(dir, s["path"])))
            if isdir(p) && !(p in developed); push!(developed, p); push!(specs, Pkg.PackageSpec(path = p)); push!(queue, p); end
        end
    end
end
isempty(specs) || Pkg.develop(specs)
ENV["OPTIMIZATION_TEST_GROUP"] = "QA"; Pkg.test()

Observed on clean master: Quality Assurance | 9 pass 3 fail (Stale-deps + deps-compat for LBFGSB/Manopt + a separate Test extras-compat that #1227 fixes).

Fix options

• Preferred: Stop the centralized tests.yml develop helper from walking a developed dep's test-only [sources]. It should only develop the active project's own [sources] (the runtime path-deps it needs to load), not recurse into a dep's [sources] (which exist for that dep's own tests). Fix in SciML/.github.
• Alternative: Move OptimizationBase's OptimizationLBFGSB/OptimizationManopt test-only resolution out of the package [sources] (e.g. into a test sub-environment), so they're not exposed to transitive developers.

Either fix clears QA-lts across all 26 sublibraries. Not addressed in #1227 (CMAES kwargs), which only fixes the orthogonal Test extras-compat finding for its own sublibrary.