[BUG] Fortran source files don't propagate through `library-manifest` dependency resolution outside `task: 'build'`

[batpigandme]

Description

Error is encountered when a Fortran benchmark for a Level-2+ BLAS routine whose Fortran source declares an EXTERNAL Fortran dependency on another stdlib package (e.g., dgemm calling xerbla, dger calling xerbla). The benchmark fails to link with a missing-symbol error for the dependency.

The underlying defect is in @stdlib/utils/library-manifest. When the resolver is queried with any task other than build (notably task: 'benchmark'), Fortran source files belonging to dependencies are not surfaced.

Fortran sources are currently declared only inside the build task entry of each Fortran-using package's manifest — so when library-manifest walks the dependency tree under task: 'benchmark', it merges each dependency's benchmark entry, which contains only C sources, and the dependency's .f files never appear in the merged result.

For example, in blas/base/xerbla/manifest.json:

• task: 'build', os: 'linux', blas: '', wasm: false → src: ["./src/xerbla.f", "./src/xerbla.c"]
• task: 'benchmark', os: 'linux', blas: '', wasm: false → src: ["./src/xerbla.c"] (no .f)

The native-addon build path works bc binding.gyp/include.gypi calls library-manifest with no explicit task, defaulting to task: 'build', where the .f declarations exist.

The same condition silently affects every merged Fortran benchmark for a Level-2+ BLAS routine that exercises a cross-package Fortran symbol. Level-1 BLAS Fortran benchmarks (e.g., daxpy, dscal, dasum) have no cross-package Fortran dependencies and link cleanly, which is why the gap has not surfaced more broadly.

The recurring local fix is a Makefile patch in the consuming package that hardcodes a relative path to a sibling package's .f file (e.g., ../../../xerbla/src/xerbla.f inside dger's benchmark Makefile). This couples one package to another package's file-tree layout. If xerbla is ever moved within the package tree (e.g., to @stdlib/blas/base/utils/xerbla), every consumer's hardcoded path silently breaks, and a contributor performing the migration has no way to discover the breakage short of running every affected benchmark. The manifest-based @stdlib/... reference model exists precisely so packages don't reach into each other's file trees this way. See @kgryte's comment on PR #11332.

A second, downstream contributor to the symptom is that tools/scripts/compile_fortran_benchmark only resolves include directories from the manifest, while its C counterpart tools/scripts/compile_c_benchmark resolves include, src, libraries, and libpath. Patching only this wrapper might be too narrow, since it wouldn't address future Fortran-linking contexts that go through library-manifest.

Per office hours discussion, the fix should live in @stdlib/utils/library-manifest so that Fortran source files are resolved uniformly when resolving a package's dependency tree, regardless of the task the caller queried.

Once the resolver is correct, downstream cleanup for the Fortran benchmark wrapper in tools/scripts/ should consume the resolved source list (mirroring its C counterpart), and the merged Fortran benchmark Makefiles' hardcoded SOURCE_FILES defaults should match the C-benchmark Makefile convention.

Related Issues

Related PRs #11332 and #11333.

Questions

No.

Demo

No response

Reproduction

# In a checkout of stdlib develop:
cd lib/node_modules/@stdlib/blas/base/dger/benchmark/fortran
make

Expected Results

Benchmark links successfully; xerbla resolves via the manifest dependency graph (dger's manifest already lists @stdlib/blas/base/xerbla as a dependency under task: 'benchmark'.

Actual Results

Undefined symbols for architecture arm64:
  "_xerbla", referenced from:
      _dger in ccCfvnCt.o
ld: symbol(s) not found for architecture arm64
collect2: error: ld returned 1 exit status
make: *** [benchmark.length.out] Error 1

Linker fails with an unresolved reference to xerbla. The dependency's .f source is never surfaced because library-manifest only declares Fortran sources under task: 'build'.

Version

develop

Environments

N/A

Browser Version

No response

Node.js / npm Version

No response

Platform

Reproduces on any platform with gfortran + make (linux/macOS verified).

Checklist

• Read and understood the Code of Conduct.
• Searched for existing issues and pull requests.

[officiallyanee]

Hi, I think a possible solution for this would be to include the fortran source files in the source of task source config and filter them out in C script for resolving source file (like this line of compile_c_benchmark), and similarly to update compile_fortran_benchmark & the Makefiles in benchmark to be more general (without hardcoded paths) following C's example with fortran compiler.
Since C & Fortran benchmarks use their source files directly (that is .c & .f respectively) it shouldn't cause an issue.

[batpigandme]

Thanks @officiallyanee, the script-parity observation is the right one — compile_fortran_benchmark should pull src/libraries/libpath from the manifest the way compile_c_benchmark does, and the per-package Fortran Makefiles can drop their hardcoded SOURCE_FILES ?= ../../src/<pkg>.f lines once it does.

One thing to fold in: per @kgryte's framing at office hours last week, I think the language-aware bit should ideally live in @stdlib/utils/library-manifest rather than in the wrappers.

I took a look at it to see what the lang-aware change would touch. Ideally the change would low-impact shape that doesn't require a manifest.json schema change. One thought was that we could add lang as an options filter (third arg to manifest()), applied post-resolve as an extension-based partition over src. Three files:

• lib/defaults.js: add 'lang': '' to the defaults.
• lib/validate.js: validate as a string with allowed values (e.g. 'c' | 'fortran' | '').
• lib/main.js: apply the filter at the end of the final loop, after dedupe and before path-convention conversion.

Callers then look like:

manifest( fpath, { 'task': 'benchmark', ... }, { 'lang': 'c', ... } ).src;       // .c only
manifest( fpath, { 'task': 'benchmark', ... }, { 'lang': 'fortran', ... } ).src; // .f only
manifest( fpath, { 'task': 'benchmark', ... }, opts ).src;                       // unfiltered (default)

So compile_fortran_benchmark passes { 'lang': 'fortran' }, compile_c_benchmark passes { 'lang': 'c' }, and any future Fortran-linking context that goes through library-manifest gets the right partition for free. Plus, empty-string default = no filter, so existing call sites are unaffected.

If I understand correctly, if we were to do lang as a conditions key (like task, os, blas), every manifest.json would need a lang entry in options and per-language confs.

I could be missing something in the blast radius, though—so definitely take a peek for yourself.

Worth flagging that the manifest edits will need to land on every Fortran-bearing transitive dep (xerbla and friends), not just the leaf packages whose benchmarks happen to be broken today, but I think that's already what you have in mind.

Want to take a stab at this one? No rush. I know everyone's got exams!

[batpigandme]

Reconsidering this in the context of last week's office hours: I got the scope wrong in my last comment. The lang post-filter idea is at the wrong layer, and I want to correct that before anyone starts building on it.

The Fortran missing-symbol thing is actually one symptom of a bigger structural mismatch in how library-manifest resolves dependencies.

Right now when library-manifest resolves task: 'benchmark' for a top-level package, it propagates task: 'benchmark' recursively to every dep in the tree. That works by accident — deps tend to have a task: 'benchmark' conf with a reasonable src list, so things compile. It breaks when a dep doesn't have native benchmarks at all (I think float32 base assert packages were mentioned last week), and it's why Fortran symbols go missing: the dep traversal isn't expressing "what files do I contribute when I'm pulled in as someone else's dependency?" — it's just reusing each dep's own benchmark conf as a stand-in.

The lang filter idea would have been filtering an already-incomplete src list. The list itself isn't being built correctly when crossing dep boundaries — filtering it differently doesn't help.

Revised thoughts on a structural fix:

1. Add an explicit "as-dependency" configuration to every C/Fortran package — separate from task: 'build' | 'benchmark' | 'examples'. This declares what files/includes/libraries to merge in when the package is consumed as a dep, regardless of the top-level task.
2. Update the recursive dep resolution in library-manifest (specifically the dep-merge loop in lib/main.js, ~lines 161–204) so it uses that as-dependency conf instead of propagating the top-level task straight through.

Once dep-confs explicitly list their Fortran contributions (.f files), the missing-symbol case goes away. Per @kgryte's description last week, the Fortran resolution issue is general — it applies to native add-ons, examples, source implementations, and benchmarks — so it has to be fixed at this layer, not in any one compile script.

What this means for the thread: the manifest-level change is bigger than a single-file (or few-file) patch — it touches every C/Fortran package's manifest.json (hundreds of files), the library-manifest recursion, backward-compat considerations, and schema docs. Probably needs its own issue with maintainer scoping before anyone takes a stab.

That said, the compile_fortran_benchmark parity work you identified, @officiallyanee, is still real and still actionable on its own — the wrapper is underpowered relative to compile_c_benchmark, and fixing that doesn't depend on the manifest restructuring. That's still a helpful patch if you want to focus there.

I could still be wrong about the blast radius — definitely take a peek at the dep-merge loop in lib/main.js yourself. And still no rush on any of it — exam season is real 🙂.

[officiallyanee]

Thanks for the pointers!
Some ref for as-dependency: Athan's comment on missing task config, the issues raised for the same: 1 & 2.
I will add the patch for the parity work as soon as I can! And would look into the blast radius too.

[batpigandme]

Some notes from @kgryte's observations in today's office hours on the broader library-manifest refactor direction (we can iterate and revise, etc., just didn't want to lose the info):

Two coupled fronts:

• Configuration inheritance for manifest.json. ESLint-style presets — you define named base configs, and other configs can extend/inherit from them, only specifying overrides. This addresses the repetition problem in packages with lots of permutations (especially BLAS with hardware-accelerated bindings). Named configs may live outside the confs array entirely, and multiple inheritance is on the table.
• "Dependency mode." What's currently missing from every manifest.json is a conf that says "when this package is used as a dependency, here's the minimal set of files/headers/libraries that are necessary." That's why the current dep resolution works by accident (propagating the top-level task down the tree) and breaks when a dep doesn't have a matching task conf. The dependency-mode conf would use the inheritance mechanism from the first front.

Implementation plan per Athan:

• Incubate: add inheritance support to @stdlib/utils/library-manifest
• Prototype: test on one or two packages of varying complexity to prove it out
• Automate: develop a script to convert the existing ~thousands of binding files to the new format

Mara's postulation
What this means for the Fortran issue specifically: the missing-symbol bug falls out of the dependency-mode conf — once deps explicitly declare their Fortran contributions (.f files) in a dep-mode conf, the resolver picks them up correctly.

I think the compile_fortran_benchmark wrapper parity issue remains (it only resolves include from the manifest, while compile_c_benchmark resolves include, src, libraries, and libpath), since doesn't depend on the inheritance work. So, that could probably be its own little patch.