CI: `ext-link` red on every PR — perry-ext-fetch fails to link (undefined js_file_new / js_headers_init_from_value)

[proggeramlug]

ext-link is failing on every PR based on current main, and the failure is main-side rather than any one PR's.

Symptom — cargo test --release --no-run for perry-ext-fetch fails to link:

perry_runtime...global_this_file_thunk+0x80:     undefined reference to `js_file_new'
perry_runtime...global_this_headers_thunk+0x78:  undefined reference to `js_headers_init_from_value'
perry_runtime...global_this_request_thunk+0x255: undefined reference to `js_headers_init_from_value'
perry_runtime...global_this_response_thunk+0x13c: undefined reference to `js_headers_init_from_value'
collect2: error: ld returned 1 exit status
error: could not compile `perry-ext-fetch` (lib test) due to 1 previous error

Some runs also show js_blob_new and js_fetch_notify_signal_aborted.

Evidence that it is not a single PR's fault: the identical failure appears on unrelated branches — e.g. perf/issue-8079-guarded-param-specialization (#8079's branch) and fix/moving-gc-rooting-sweep (#8131), whose diffs share no files. Of the open PRs sampled, four fail ext-link and one older one (based on an earlier main) passes.

Why it matters: ext-link exists precisely because cargo-test's scope keeps perry-ext-* out of the fan-out (#7656, after #7650). A gate that is red for every PR is one that no PR can be blocked by in practice — the same shape as #8092.

perry-runtime's global_this fetch thunks reference symbols that live outside it, so the fix is presumably to make perry-ext-fetch's link surface provide them (or to gate the thunks on the same feature that provides the definitions).

[proggeramlug]

Root cause, and one approach I tried and rejected

Reproduced locally: cargo test -p perry-ext-fetch --no-run fails with exactly the three symbols reported — js_file_new, js_headers_init_from_value, js_fetch_notify_signal_aborted.

Why it happens

crates/perry-ext-fetch/Cargo.toml's dev-dependency builds the runtime as:

perry-runtime = { workspace = true, features = ["default", "external-fetch-symbols", "stdlib"] }

Those two features are individually correct and jointly fatal for this crate's own test binary:

• stdlib compiles the fallback stubs out entirely — lib.rs:216 is #[cfg(not(feature = "stdlib"))] pub mod stdlib_stubs;. Its stated purpose (runtime: ext staticlibs bundle perry-runtime and shadow perry-stdlib's real symbols with its no-op stubs (node:http servers get no tokio reactor) #6314) is so the no-op stubs cannot shadow perry-stdlib's real symbols when the staticlib is linked before stdlib.
• external-fetch-symbols turns on the unsafe extern "C" block in object/global_fetch.rs, which perry-runtime then calls (call_global_file_new, call_global_headers_init_from_value, notify_fetch_abort).

So the test link has live references, no stubs, and the real implementations are in perry-stdlib (fetch_blob.rs:213, fetch/abort_bridge.rs) — which is not in the link.

Note js_file_new has no stub at all in stdlib_stubs.rs even when that module is compiled; the other two do. So the stub set is also incomplete, independently of this.

Contrast with perry-ext-ws, which carries the same stdlib + external-ws-symbols pairing and does not fail: perry-runtime does not call the ws externs the way it calls the fetch ones, so there is no live reference.

The approach I tried, and why I am not proposing it

Adding perry-stdlib as a dev-dependency does nothing on its own — rustc only pulls an rlib's objects when Rust code names the crate. Adding #[cfg(test)] extern crate perry_stdlib; to force it does make the link succeed.

But it then emits:

ld: duplicate symbol '_js_headers_delete' in:
      perry_ext_fetch...rcgu.o
      libperry_stdlib...rlib
ld: duplicate symbol '_perry_ffi_promise_resolve_bits' in: ...

macOS ld downgrades these to warnings, so the build "passes". That is worse than the current red: perry-ext-fetch exists precisely to replace parts of the fetch surface, so it and perry-stdlib legitimately define overlapping symbols, and which one the test binary binds is link-order-dependent. Given #8156 was exactly a "which image won the link" bug, shipping a fix that silently introduces symbol duplication in a test binary is the wrong trade. Reverted.

Options that seem sound, for whoever picks this up

1. Add the three missing symbols to perry-ext-fetch itself. It is a fetch implementation; js_file_new and friends arguably belong in its surface. Largest change, no duplication, and it makes the crate self-contained in the way ext_crates_bundle_a_full_featured_perry_runtime already expects.
2. Complete the stub set and let the test binary use it — i.e. do not enable stdlib for the dev-dependency, and add the missing js_file_new stub. Needs care: cargo feature unification means the dev-dependency and the normal dependency of the same crate share a feature set, so this may not be expressible without a separate test target.
3. Have ext-link link stdlib the way a real binary does, rather than relying on cargo test --no-run to be self-sufficient. Changes what the gate proves, so it needs a deliberate decision about what ext-link is for.

I have not picked between these because each has a real trade and this is not my subsystem. Diagnosis and the ruled-out path are above so nobody repeats the dev-dependency attempt.

Worth restating the stakes from the issue: ext-link exists because cargo-test's scope deliberately keeps perry-ext-* out of the fan-out (#7656). While it is red on every PR, that whole family has no gate.

[proggeramlug]

PR #8197 takes option 1's spirit without its cost, and closes this.

What it does: defines the four symbols in a #[cfg(test)] module, so only this crate's test binary gets them.

Why not real implementations (the literal option 1): this crate is a staticlib whose objects win the final link ahead of perry-stdlib (prefer_well_known_before_stdlib), so defining js_blob_new / js_file_new / js_headers_init_from_value / js_fetch_notify_signal_aborted for real would replace perry-stdlib's Blob/File/Headers constructors and abort bridge in every shipped binary — a behaviour change smuggled in under a CI fix, and one that would have to reimplement FETCH_ABORT_WATCHERS and the blob registry to be equivalent. cfg(test) code never reaches libperry_ext_fetch.a, and perry-stdlib is absent from the test link, so the duplicate-symbol problem that killed the dev-dependency attempt cannot arise either. A guard test (sabotage-verified) fails if any shipped file in the crate later defines one of the four.

Option 2 would not have worked as described, and it is worth recording why: dropping stdlib from the dev-dependency does not restore these stubs, because they carry a second gate — #[cfg(not(feature = "external-fetch-symbols"))] — and external-fetch-symbols stays on. The module-level #[cfg(not(feature = "stdlib"))] is only half the story.

One correction to the analysis above: the missing js_file_new stub (and js_fetch_notify_signal_aborted) is not an independent gap. Both call sites are cfg'd on external-fetch-symbols, and the non-external arms use the GLOBAL_FETCH_* function-pointer registry and a no-op respectively — nothing references them in the stub configuration, so no stub is owed. I left stdlib_stubs.rs alone.

Worth noting: with the link restored, this crate's 13 tests run for the first time — they had never executed, since the binary could not be produced. All pass.

#8187 is complementary rather than overlapping: it makes ext-link opt-in on PRs and adds nightly/tag arms, which stops the noise but would have left the nightly arm red. #8197 touches no workflow file, so the two cannot conflict.