fix(test): C host + two-level namespace for the Next App Route dylib gate (#8205)

[proggeramlug]

Fixes #8205.

What was wrong

tests/test_next_app_route_dylib.sh (the #8161 gate behind next-app-route.yml) aborted on cold start 1, first request — Node oracle PASS, app dylib compiled, providers linked, ABI check clean, PERRY_NEXT_APP_ROUTE_READY printed, then Abort trap: 6 on the readiness curl:

libsystem_malloc  ___BUG_IN_CLIENT_OF_LIBMALLOC_POINTER_BEING_FREED_WAS_NOT_ALLOCATED
libperry_stdlib.dylib  <alloc::vec::IntoIter<i64> as Drop>::drop
libperry_stdlib.dylib  js_node_http_server_process_pending
libperry_stdlib.dylib  js_stdlib_process_pending
libperry_runtime.dylib js_run_stdlib_pump
provider-host          provider_host::main

It is an allocator-shim binding problem, not a route or compiler problem (the same commit/compiler/app served 500 verifier batches under tests/release/packages/next-app-route/fixture.sh the same day). The nm facts from #8205:

• libperry_runtime.dylib defines rustc's allocator shim and it is mimalloc: __RNv…7___rustc14___rust_dealloc: b _mi_free.
• libperry_stdlib.dylib imports __rust_alloc / __rust_dealloc / __rust_alloc_zeroed (U) — correct, it must use the runtime image's allocator.
• provider-host (built from provider-host.rs, a Rust executable) also defines __RNv…7___rustc14___rust_dealloc — the System-allocator shim every Rust binary carries.
• provider-linker.sh linked the stdlib image with -Wl,-flat_namespace -Wl,-interposable. In a flat namespace an undefined symbol binds at load time to the first image defining it, and the main executable is first — so the stdlib image's __rust_dealloc bound to the host's System shim while the buffers it drops were allocated by the runtime image's mimalloc. First cross-image Vec<i64> drop (draining the pending-request list) → libsystem free() on a mimalloc pointer → abort().

The fix (test-only, tests/ only, no crates/**)

1. provider-host.rs → provider-host.c. Same load order and flags (runtime RTLD_NOW|RTLD_GLOBAL, stdlib RTLD_NOW|RTLD_GLOBAL, app RTLD_NOW|RTLD_LOCAL), same next_app_route_provider_runtime_probe() == js_gc_init "one runtime image" check, same js_gc_init → perry_module_init → {microtasks, 3 timer ticks, stdlib pump, wait} loop. The only behavioural difference is what is absent: a C executable carries no Rust allocator shim, so the only __rust_* definitions in the process are the runtime image's. This is the release-tier layout (tests/release/packages/next-app-route/provider-host.c), which has the production evidence. (On Linux, where the workflow actually runs, a Rust executable's shim is not in .dynsym unless exported, so the flat ELF lookup would most likely have skipped it — I have not run the gate on Linux and do not claim it was red there for this reason; the C host simply leaves no shim to find on either platform.)
2. Drop -flat_namespace -interposable from provider-linker.sh. With the default two-level namespace every undefined symbol in the stdlib image is bound at link time to the image that defines it, so __rust_dealloc resolves to @rpath/libperry_runtime.dylib regardless of what else the process defines. The exported-symbols list, the -Wl,-u retention of the app's required ABI, the runtime-dylib substitution on the link line and the @loader_path rpath are unchanged.
3. tests/test_next_app_route_dylib.sh builds the host with "$real_cc" -O2 … -ldl instead of rustc. Everything else the script guarantees is intact: the required_symbols/missing_symbols ABI check, the forbidden_diagnostics grep on every host log (including the routeModule.handle bypass guard), and the Provider ABI hash line.

.github/workflows/next-app-route.yml does not name the host file, so it needs no change.

Two more gate-only defects the fix exposed (both were invisible behind the abort)

Fixing the abort let the gate serve for the first time, and it immediately hit two more reasons it could never have gone green — both fixed here, both in the gate script only:

1. Host cwd. The gate ran the host from $fixture/.next/server so the webpack runtime could resolve ./chunks/*.js — a workaround that fix(next): make computed relative chunk requires resolve in a compiled App Route #8146 made obsolete (computed relative chunk requires now resolve against the route bundle). Meanwhile Next opens .next/routes-manifest.json against the working directory on every request, so from .next/server each request 500'd with ENOENT (measured: 1,043 ENOENT lines, readiness never reached). The host now runs from the fixture root — the release-tier layout with production evidence.
2. Host kill reached the subshell, not the host. Each cold start launched the host inside a ( cd …; "$host" … ) & subshell and killed $!; the host survived orphaned (observed directly: a provider-host with ppid 1 still serving after the gate exited) and held the port, so cold start 2 could never bind. The launch now execs the host so $host_pid is the host itself. (The node-oracle script has the same shape around npm start — a leaked next-server was observed on the oracle port — but that leaks once per gate run, not per cold start, and only bites a second run on the same machine; noted, not fixed here.)

What this does and does not make green

This makes the host survive its first request and serve; it does not make the 100-batch loop reliably green on today's main. #8163 ("Reproduces under DEFAULT GC") shows ~2% of warm batches lose one response after a default-mode copying minor (TypeError: value is not a function in the host log right after a [gc-copy-minor] ran line, then an empty body in verify.mjs). With 10 verifier passes per cold start this gate can hit that. That is a different defect (a stale closure held outside the GC heap), it is tracked in #8163, and this PR does not paper over it.

Validation (bench mini, macOS arm64, perry-dev @ this branch, LLVM 22.1.8, PERRY_GC_DIAG=1)

tests/test_next_app_route_dylib.sh end to end (defaults: 10 cold starts x 10 verifiers), PERRY_NEXT_PORT=3400, PERRY_NEXT_CARGO_JOBS=3, PERRY_MODULE_JOBS=2, PERRY_CODEGEN_UNIT_JOBS=2:

• Node oracle PASS; app-only dylib compiled (104 modules, external LLVM 22 clang/opt); ABI check clean (missing-symbols empty); Provider ABI hash printed.
• The [Next.js/dylib] tests/test_next_app_route_dylib.sh aborts on the first request: Rust provider-host + flat_namespace bind the stdlib image's __rust_dealloc to the System shim (mimalloc buffer freed by libsystem) #8205 abort is gone: every cold start came up, served its readiness curl, and ran full verifier passes. Before this PR the host died on request 1 of cold start 1, every time.
• 59 of 100 batches green, then the run stopped on [Next.js/dylib] Forced-evacuation App Route arm: stale closure from a holder outside the GC heap #8163, not on this fix: cold starts 1–5 passed 10/10 verifiers each, cold start 6 passed 9 and lost one response in verifier 10 — verify.mjs Unexpected end of JSON input (empty body), host log TypeError: value is not a function immediately after a [gc-copy-minor] ran / nursery_minor safepoint drain. Copying minors per cold start: 3/3/3/3/3/3; TypeError count: 0/0/0/0/0/1; bypass-guard fires: 0/0/0/0/0/0. That is precisely [Next.js/dylib] Forced-evacuation App Route arm: stale closure from a holder outside the GC heap #8163's default-GC signature ("Reproduces under DEFAULT GC", [Next.js/dylib] Forced-evacuation App Route arm: stale closure from a holder outside the GC heap #8163's comment), and the script's set -e stops at the first failed verifier, so cold starts 7–10 did not run. Since this measurement, fix(gc): root the two holders outside the GC heap behind the forced-evacuation App Route arm (#8163) #8211 landed and fixed the two forced-evacuation-arm holders, but [Next.js/dylib] Forced-evacuation App Route arm: stale closure from a holder outside the GC heap #8163 is reopened on exactly this default-GC residual — so a full 10x10 green run is still not expected on main until that residual lands.
• Zero routes-manifest.json ENOENT lines after the cwd fix (1,043 before it); zero orphaned hosts after the exec fix (one observed before it).

Mini log paths: gate run ~/perry-bench.noindex/tmp-8205/gate2.log, kept scratch with per-cold-start host logs ~/perry-bench.noindex/tmp-8205/perry-next-app-route.hp7c6V/host-{1..6}.log, first (pre-cwd-fix) run ~/perry-bench.noindex/tmp-8205/gate.log + perry-next-app-route.KVazL1/.

What I could NOT validate, and why

A clean 100/100 gate run is not reachable on main today, for two reasons that are both other people's bugs:

1. [Next.js/dylib] Forced-evacuation App Route arm: stale closure from a holder outside the GC heap #8163's default-GC residual stopped the run at batch 60 (evidence above). fix(gc): root the two holders outside the GC heap behind the forced-evacuation App Route arm (#8163) #8211 has since fixed the two forced-arm holders; the default-GC residual is now named with a fix in flight — fix(gc): root the Headers/FormData iteration frame slots (#8163, #8217) #8220, rooting js_headers_for_each's hoisted closure pointer (crates/perry-stdlib/src/fetch/headers.rs:460) plus eight sibling sites, measured 8 from-space faults → 0 and 3 failures/300 warm passes → 0. Once that lands the gate's 10x10 should be reachable; I have not re-run it on top of fix(gc): root the Headers/FormData iteration frame slots (#8163, #8217) #8220.
2. [codegen] In-process LLVM backend fails the 5 biggest Next App Route modules on current main (silent per-module failure; regression in 3c95020f8..07c8040bf) #8228 blocks the fixture compile on current main outright (filed from this work): the in-process LLVM backend — the default when PERRY_LLVM_INPROCESS is unset — fails 5 of the 104 modules, root-caused since filing to insertelement <2 x i64> from perf(gc, codegen): recover the instruction cost of the 56 B → 48 B header shrink (#8122) #8204's module-init header-image compose having no case in perry-codegen/src/dialect/mod.rs (bails bad binary op). app-route.runtime.prod.js is one of the failures, so PERRY_ALLOW_PARTIAL_CODEGEN=1 is not a workaround. This gate is unaffected in practice — tests/test_next_app_route_dylib.sh drives the external LLVM 22 clang/opt path, which compiles all 104 modules clean, and that is how the run above was produced. The release-tier fixture.sh (in-process by default) is the one blocked.

Neither is introduced or worked around by this PR.

Summary by CodeRabbit

• Bug Fixes

  • Improved Next App Route dynamic library loading on macOS by ensuring runtime and standard library components bind consistently.
  • Reduced the risk of allocator mismatches when loading application components.

• Tests

  • Updated dynamic library integration coverage to use a native host and verify loading, symbol resolution, initialization, and event processing.
  • Preserved existing compatibility checks and platform-specific launch behavior.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: f8401e6a-59d5-4f4e-8846-40d292849a11

📥 Commits

Reviewing files that changed from the base of the PR and between 537f74a and f259dbd.

📒 Files selected for processing (5)

• changelog.d/8209-next-gate-c-host-two-level.md
• tests/fixtures/next-app-route/provider-host.c
• tests/fixtures/next-app-route/provider-host.rs
• tests/fixtures/next-app-route/provider-linker.sh
• tests/test_next_app_route_dylib.sh

💤 Files with no reviewable changes (1)

• tests/fixtures/next-app-route/provider-host.rs

Included review availability: Your plan includes up to 8 reviews per rolling hour; 3 remain after this review.

---

📝 Walkthrough

Walkthrough

The Next App Route dylib gate replaces its Rust host with a C host, removes Darwin flat-namespace linking, and updates the test harness to compile and run the host with explicit process control.

Changes

Next App Route dylib gate

Layer / File(s)	Summary
Provider two-level linking tests/fixtures/next-app-route/provider-linker.sh, changelog.d/8209-next-gate-c-host-two-level.md	The Darwin provider link removes -flat_namespace and -interposable. The changelog records the C host and two-level linking changes.
C provider host tests/fixtures/next-app-route/provider-host.c	The C host loads the runtime, stdlib provider, and app dylibs. It validates symbols and runtime binding, initializes the app, and processes runtime events.
Gate compilation and process control tests/test_next_app_route_dylib.sh	The test compiles the C host, runs it from the fixture root, and uses exec for platform-specific launches.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: ⚪ Minimal · up to f259d

This test-only change replaces the host and linker setup to prevent allocator binding failures, fixes the gate’s working-directory and process-cleanup behavior, and preserves the existing ABI checks. No actionable merge-blocking risk remains beyond normal checks and review.

Possibly related issues

• PerryTS/perry issue 8205: Replaces the Rust host and flat-namespace linking associated with the allocator crash.
• PerryTS/perry issue 8040: Implements separate runtime and stdlib provider loading with a C-host gate.

Possibly related PRs

• PerryTS/perry#8081: Updates the same provider-host dylib integration flow and preserves runtime, stdlib, app, GC, and event-loop validation.
• PerryTS/perry#8082: Refines the same Next.js App Route provider-host dylib gate with a native host and provider linking changes.
• PerryTS/perry#8161: Updates the same Next App Route dylib harness and changes provider linking to two-level namespace binding.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies the test fix, the C host change, and the two-level namespace change.
Description check	✅ Passed	The description fully explains the problem, changes, related issues, scope, and detailed validation results.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/8205-next-gate-host-alloc

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[proggeramlug]

---

State at ready-for-review (2026-08-16)

• Rebased onto current main. The earlier lint red was a stale base: the branch predated refactor: split gc/layout.rs and codegen/artifacts.rs under the 2000-line cap (unblocks lint on main) #8212, which split the two files over the 2000-line cap. scripts/check_file_size.sh is clean on the rebased tree.
• The remaining gate failure in the validation run above is [Next.js/dylib] Forced-evacuation App Route arm: stale closure from a holder outside the GC heap #8163's default-GC residual, not this PR: it is fixed in flight by fix(gc): root the Headers/FormData iteration frame slots (#8163, #8217) #8220 (js_headers_for_each's hoisted closure pointer, a native Rust frame slot). Once fix(gc): root the Headers/FormData iteration frame slots (#8163, #8217) #8220 lands, this gate's 10x10 should be re-run for a clean 100/100.
• Independently, [codegen] In-process LLVM backend fails the 5 biggest Next App Route modules on current main (silent per-module failure; regression in 3c95020f8..07c8040bf) #8228 currently prevents the in-process LLVM backend from compiling 5 of the fixture's 104 modules on main, so a HEAD validation of the full gate is blocked until that is fixed. The validation above was run before that regression was attributed.