fix(gc): retag mimalloc's OS mappings before main, not from js_gc_init (#7450)

changelog.d/7456-mimalloc-os-tag-pre-main.md

@@ -0,0 +1,9 @@
+**GC/profiling: mimalloc's VM retag now runs before `main`, so the heap really does render as `Memory Tag 240`** (fixes #7450) — #6882 set mimalloc's `os_tag` option to `VM_MEMORY_APPLICATION_SPECIFIC_1` (240) from `js_gc_init`, on the assumption that only "regions mapped before this call (early Rust startup)" would keep the default tag 100 that macOS decodes as `IOAccelerator`. mimalloc does not `mmap` per allocation: it reserves an *arena* (`mi_option_arena_reserve`, 1 GiB on 64-bit) on the first allocation and afterwards commits pages *inside* that reservation, and committing does not re-tag. The one mapping that matters was therefore made during `std`'s pre-`main` startup and the retag reached nothing — on tree.ts, ~230 MB of `IOAccelerator` against ~64 KB of `Memory Tag 240`. Labeling-only (no accounting or correctness impact), but it sends anyone following the documented profiling recipe hunting for GPU memory in a GC benchmark.
+
+Measured: setting the option as the *first statement of `main`* still yields a 100-tagged heap, so no placement in Rust code can fix this. The retag now runs from a `__DATA,__mod_init_func` constructor (`crates/perry-runtime/src/mimalloc_os_tag.rs`), which dyld invokes while preparing the image — before `main`, before `std`'s runtime setup, before mimalloc's arena reservation. That path must not allocate (an allocation there would reserve the arena before the tag was set, reintroducing the bug), so the `MIMALLOC_OS_TAG` opt-out is read with `libc::getenv` rather than `std::env::var_os`, which allocates an `OsString`. `js_gc_init` still calls into the module: the call selects that archive member so the constructor survives the link, and it re-applies the option idempotently. Note that the obvious way to write the keepalive — taking the address of the `#[used]` static — does *not* link: `ld` rewrites `__DATA,__mod_init_func` into a `__TEXT,__init_offsets` table of 32-bit offsets, so the static has no address left and the reference fails as `ADRP out of range ... to 0x00000000`.
+
+Verified end-to-end, not just in unit tests: a Perry-compiled binary under `vmmap` now reports its whole 260 MB heap (and the 768 MB reserved tail) as `Memory Tag 240` with zero `IOAccelerator`, and `otool -l` confirms the `__init_offsets` entry survived Perry's own strip/dedup link path.
+
+The reason this regressed invisibly is that #6882's test asserted `mi_option_get(mi_option_os_tag) == 240` — which was true the whole time it was broken. `gc/tests/os_tag.rs` now asserts the kernel's `user_tag` for a live heap address via `mach_vm_region`, i.e. exactly what `vmmap` renders; sabotaging the constructor's `link_section` fails it with `got 100` while the option-value assertion stays green. `docs/src/internals/memory-model.md` §profiling is corrected accordingly: large `IOAccelerator` regions on a current build now mean a dropped module initializer, not an expected caveat.
+
+Coverage caveat, stated rather than left to be discovered: the new test is macOS-only and **no required CI job runs `cargo test -p perry-runtime --lib` on a macOS runner** — `cargo-test` moved to `ubuntu-latest` in v0.5.392, where this module compiles to an empty function. So the test gates local runs and any future macOS arm, but not `main` today. Wiring it into a macOS job is a deliberate separate step (a new gate has to go green once before it can be promoted), not something this PR does silently.

crates/perry-runtime/src/gc/mod.rs

@@ -771,25 +771,16 @@ pub extern "C" fn js_gc_init() {
     // a second js_gc_init on another thread is harmless.
     #[cfg(windows)]
     crate::win_console::enable_vt_output();
-    // #6882: mimalloc (the global allocator, #62) tags its OS mappings with
-    // VM tag 100, which macOS tooling — vmmap, Instruments' VM Tracker,
-    // `footprint` — decodes as `IOAccelerator`: the entire JS heap renders
-    // as GPU-driver memory (644 MB of "IOAccelerator" on an allocation-heavy
-    // benchmark). Retag to VM_MEMORY_APPLICATION_SPECIFIC_1 (240) so heap
-    // regions show up as a neutral, distinctive "Memory Tag 240" instead.
-    // An explicit `MIMALLOC_OS_TAG` env setting still wins — skip the
-    // override so profilers can keep steering the tag themselves. Regions
-    // mapped before this call (early Rust startup) keep tag 100; the bulk
-    // of the heap (arena blocks, GC metadata) maps afterwards. Idempotent,
-    // like the rest of this function.
-    #[cfg(all(
-        target_pointer_width = "64",
-        target_vendor = "apple",
-        feature = "alloc-mimalloc"
-    ))]
-    if std::env::var_os("MIMALLOC_OS_TAG").is_none() {
-        unsafe { libmimalloc_sys::mi_option_set(libmimalloc_sys::mi_option_os_tag, 240) };
-    }
+    // #6882/#7450: macOS decodes mimalloc's default VM tag (100) as
+    // `IOAccelerator`, so the whole JS heap renders as GPU-driver memory in
+    // vmmap/Instruments/`footprint`. The retag to tag 240 that fixes this
+    // cannot live here — mimalloc reserves its 1 GiB arena during Rust's
+    // pre-`main` startup and every later allocation just commits pages inside
+    // that already-tagged mapping, so a retag at `js_gc_init` time reaches
+    // nothing (#7450). It now runs from a `__DATA,__mod_init_func`
+    // constructor; this call keeps that constructor in the link and re-applies
+    // the option idempotently. See `crate::mimalloc_os_tag`.
+    crate::mimalloc_os_tag::ensure_mimalloc_os_tag_applied();
     crate::node_submodules::diagnostics_channel_init_main_thread();
     crate::node_submodules::init_trace_events_runtime();
     // #5093: force every class-field access back through the full guard call —

crates/perry-runtime/src/gc/tests/os_tag.rs

@@ -1,24 +1,134 @@
-//! #6882: `js_gc_init` must move mimalloc's OS-mapping VM tag off 100 —
-//! macOS tooling decodes tag 100 as `IOAccelerator`, so the whole JS heap
-//! shows up as GPU-driver memory in vmmap/Instruments/footprint.
+//! #6882/#7450: mimalloc's OS mappings must carry VM tag 240, not the default
+//! 100 — macOS tooling decodes tag 100 as `IOAccelerator`, so the whole JS
+//! heap shows up as GPU-driver memory in vmmap/Instruments/footprint.
+//!
+//! The primary assertion here is on the *mapping*, not on
+//! `mi_option_get(mi_option_os_tag)`. #6882 shipped with only the
+//! option-value assertion, and it passed for the entire time the feature was
+//! broken: the option really was 240, it had just been set after mimalloc
+//! already reserved — and thereby tagged — the arena backing the whole heap.
+//! Reading the option back tests that a store landed in mimalloc's static
+//! table. Reading the kernel's `user_tag` for a live heap address tests what
+//! anyone following the profiling recipe actually sees.
 
-#[cfg(all(
+#![cfg(all(
     target_pointer_width = "64",
     target_vendor = "apple",
     feature = "alloc-mimalloc"
 ))]
+
+use crate::mimalloc_os_tag::PERRY_HEAP_VM_TAG;
+
+const VM_REGION_EXTENDED_INFO: libc::c_int = 13;
+
+/// `struct vm_region_extended_info` from `<mach/vm_region.h>`. `user_tag` is
+/// the field `vmmap` renders as the region type.
+#[repr(C)]
+#[derive(Clone, Copy, Default)]
+struct VmRegionExtendedInfo {
+    protection: libc::c_int,
+    user_tag: libc::c_uint,
+    pages_resident: libc::c_uint,
+    pages_shared_now_private: libc::c_uint,
+    pages_swapped_out: libc::c_uint,
+    pages_dirtied: libc::c_uint,
+    ref_count: libc::c_uint,
+    shadow_depth: libc::c_ushort,
+    external_pager: libc::c_uchar,
+    share_mode: libc::c_uchar,
+    pages_reusable: libc::c_uint,
+}
+
+extern "C" {
+    /// The calling task's port. `libc::mach_task_self()` wraps this but is
+    /// deprecated in favour of the `mach2` crate, which the runtime does not
+    /// depend on; the underlying global is stable ABI.
+    static mach_task_self_: libc::mach_port_t;
+
+    fn mach_vm_region(
+        target_task: libc::mach_port_t,
+        address: *mut u64,
+        size: *mut u64,
+        flavor: libc::c_int,
+        info: *mut libc::c_int,
+        info_count: *mut libc::c_uint,
+        object_name: *mut libc::mach_port_t,
+    ) -> libc::kern_return_t;
+}
+
+/// The kernel's VM tag for the mapping containing `addr`, or `None` if the
+/// lookup failed or walked past `addr` into a later region.
+fn mapping_vm_tag(addr: usize) -> Option<u32> {
+    let mut region_start = addr as u64;
+    let mut region_size: u64 = 0;
+    let mut info = VmRegionExtendedInfo::default();
+    let mut info_count =
+        (std::mem::size_of::<VmRegionExtendedInfo>() / std::mem::size_of::<libc::c_uint>()) as u32;
+    let mut object_name: libc::mach_port_t = 0;
+    // SAFETY: self-inspection of the calling task. `mach_vm_region` writes at
+    // most `info_count` words into `info`, and `info_count` is derived from
+    // `size_of::<VmRegionExtendedInfo>()`.
+    let kr = unsafe {
+        mach_vm_region(
+            mach_task_self_,
+            &mut region_start,
+            &mut region_size,
+            VM_REGION_EXTENDED_INFO,
+            (&raw mut info).cast::<libc::c_int>(),
+            &mut info_count,
+            &mut object_name,
+        )
+    };
+    if kr != 0 {
+        return None;
+    }
+    // `mach_vm_region` returns the first region at or above the address it is
+    // handed; if it skipped forward, `addr` itself is unmapped.
+    let contains = region_start <= addr as u64 && (addr as u64) < region_start + region_size;
+    contains.then_some(info.user_tag)
+}
+
 #[test]
-fn js_gc_init_retags_mimalloc_os_mappings() {
-    // Only asserts when the profiler override isn't steering the tag —
-    // js_gc_init deliberately defers to an explicit MIMALLOC_OS_TAG.
+fn mimalloc_heap_mappings_carry_the_perry_vm_tag() {
+    // The runtime defers to an operator steering the tag by hand, so the
+    // assertion only holds when nothing is.
     if std::env::var_os("MIMALLOC_OS_TAG").is_some() {
         return;
     }
     crate::gc::js_gc_init();
-    let tag = unsafe { libmimalloc_sys::mi_option_get(libmimalloc_sys::mi_option_os_tag) };
+
+    // A multi-megabyte allocation through the global allocator — i.e. through
+    // mimalloc, i.e. backed by one of the OS mappings under test. Touched at
+    // both ends so the pages are real rather than a lazy reservation.
+    let mut heap_block = vec![0u8; 8 << 20];
+    let last = heap_block.len() - 1;
+    heap_block[0] = 1;
+    heap_block[last] = 1;
+
+    let tag = mapping_vm_tag(heap_block.as_ptr() as usize)
+        .expect("a live mimalloc allocation must sit inside a mapped VM region");
     assert_eq!(
-        tag, 240,
-        "js_gc_init must retag mimalloc mappings to VM_MEMORY_APPLICATION_SPECIFIC_1 \
-         (240); tag 100 renders the heap as IOAccelerator in macOS tools (#6882)"
+        libc::c_long::from(tag),
+        PERRY_HEAP_VM_TAG,
+        "mimalloc's heap mappings must carry VM tag {PERRY_HEAP_VM_TAG} \
+         (VM_MEMORY_APPLICATION_SPECIFIC_1); got {tag}. Tag 100 renders the \
+         heap as IOAccelerator in vmmap/Instruments (#6882), and is what you \
+         get whenever the retag runs after mimalloc has already reserved its \
+         arena — i.e. from anywhere other than a pre-`main` module initializer \
+         (#7450). Check that the `__DATA,__mod_init_func` entry in \
+         `mimalloc_os_tag` survived the link."
     );
 }
+
+#[test]
+fn js_gc_init_leaves_the_os_tag_option_set() {
+    // Strictly weaker than the mapping assertion above, and kept only as a
+    // localizer: if both fail, the option store itself is broken; if only the
+    // mapping test fails, the constructor did not run early enough (or at all).
+    if std::env::var_os("MIMALLOC_OS_TAG").is_some() {
+        return;
+    }
+    crate::gc::js_gc_init();
+    let tag = unsafe { libmimalloc_sys::mi_option_get(libmimalloc_sys::mi_option_os_tag) };
+    assert_eq!(tag, PERRY_HEAP_VM_TAG);
+}

crates/perry-runtime/src/lib.rs

@@ -87,6 +87,7 @@ pub mod macos_bundle;
 pub mod map;
 pub mod math;
 pub mod messaging;
+pub mod mimalloc_os_tag;
 pub mod module_require;
 pub mod native_abi;
 pub mod native_arena;

crates/perry-runtime/src/mimalloc_os_tag.rs

@@ -0,0 +1,111 @@
+//! macOS VM tag for mimalloc's OS mappings (#6882, repaired by #7450).
+//!
+//! mimalloc tags every `mmap` it makes with `mi_option_os_tag`, default 100.
+//! macOS decodes tag 100 as `IOAccelerator`, so `vmmap`, Instruments' VM
+//! Tracker and `footprint` render Perry's entire JS heap as GPU-driver
+//! memory. #6882 retagged it to `VM_MEMORY_APPLICATION_SPECIFIC_1` (240),
+//! which those tools show as the neutral `Memory Tag 240`.
+//!
+//! **The retag has to happen before mimalloc's first OS mapping, which means
+//! before `main`.** #6882 set the option from `js_gc_init` and reasoned that
+//! only "regions mapped before this call (early Rust startup)" would keep tag
+//! 100. That is wrong on a per-mapping basis and catastrophically wrong in
+//! aggregate: mimalloc does not `mmap` per allocation, it reserves an *arena*
+//! — `mi_option_arena_reserve`, 1 GiB on 64-bit — on the very first
+//! allocation, and then satisfies everything afterwards by committing pages
+//! *inside* that existing reservation. Committing does not re-tag: the tag is
+//! fixed by the `mmap` that created the region. So the one mapping that
+//! matters is made during Rust's own startup, long before any Perry code
+//! runs, and every later retag lands on an option nobody reads again. On
+//! tree.ts that showed up as ~230 MB of `IOAccelerator` against ~64 KB of
+//! `Memory Tag 240` — the retag covering literally the rounding error.
+//!
+//! Moving the call earlier inside Rust does not help; measured, setting the
+//! option as the first statement of `main` still yields a 100-tagged heap,
+//! because `std`'s pre-`main` runtime setup has already allocated. The only
+//! placement that beats mimalloc's first mapping is a Mach-O module
+//! initializer (`__DATA,__mod_init_func`), which dyld runs while preparing the
+//! image — i.e. before `main`, and before any Rust allocation.
+//!
+//! Consequently this file runs on the pre-`main` path and **must not
+//! allocate**: an allocation here would initialize mimalloc and reserve the
+//! arena before we had set the tag, reintroducing the bug it exists to fix.
+//! That is why the `MIMALLOC_OS_TAG` opt-out is read with `libc::getenv`
+//! rather than `std::env::var_os`, which allocates an `OsString`.
+
+/// `VM_MEMORY_APPLICATION_SPECIFIC_1`. Chosen because macOS reserves 240-255
+/// for applications and renders them as a plain `Memory Tag <n>`, so the heap
+/// is both distinctive and obviously not a system subsystem.
+#[cfg(all(
+    target_pointer_width = "64",
+    target_vendor = "apple",
+    feature = "alloc-mimalloc"
+))]
+pub const PERRY_HEAP_VM_TAG: libc::c_long = 240;
+
+/// Point mimalloc's `os_tag` at [`PERRY_HEAP_VM_TAG`], unless an operator is
+/// already steering it with `MIMALLOC_OS_TAG` (profilers use that to isolate
+/// a run; the runtime defers rather than fighting them).
+///
+/// Runs before `main` from [`MOD_INIT_FUNC_ENTRY`]. Allocation-free — see the
+/// module docs. Idempotent: `mi_option_set` just writes mimalloc's static
+/// option table, so the second call from `js_gc_init` is a no-op write.
+#[cfg(all(
+    target_pointer_width = "64",
+    target_vendor = "apple",
+    feature = "alloc-mimalloc"
+))]
+extern "C" fn apply_mimalloc_os_tag() {
+    // SAFETY: `getenv` is well-defined pre-`main` on Apple platforms (dyld has
+    // published `environ` before it runs module initializers), and takes a NUL-
+    // terminated string, which the C literal is. `mi_option_set` only stores
+    // into mimalloc's static option table — no allocation, no lock, no
+    // dependency on mimalloc having been initialized.
+    unsafe {
+        if libc::getenv(c"MIMALLOC_OS_TAG".as_ptr()).is_null() {
+            libmimalloc_sys::mi_option_set(libmimalloc_sys::mi_option_os_tag, PERRY_HEAP_VM_TAG);
+        }
+    }
+}
+
+/// The Mach-O equivalent of `__attribute__((constructor))`: dyld walks
+/// `__DATA,__mod_init_func` when the image is prepared and calls every
+/// function pointer in it, before `main` and before `std`'s runtime setup —
+/// the only window that precedes mimalloc's arena reservation.
+#[cfg(all(
+    target_pointer_width = "64",
+    target_vendor = "apple",
+    feature = "alloc-mimalloc"
+))]
+#[used]
+#[link_section = "__DATA,__mod_init_func"]
+static MOD_INIT_FUNC_ENTRY: extern "C" fn() = apply_mimalloc_os_tag;
+
+/// Called from `js_gc_init`. Does two things, neither of them the retag that
+/// actually matters — that already happened pre-`main`:
+///
+/// 1. Gives the linker a reason to pull this module's object file out of
+///    `libperry_runtime.a`. `#[used]` stops `-dead_strip` from dropping
+///    [`MOD_INIT_FUNC_ENTRY`], but it cannot pull an otherwise unreferenced
+///    archive member into the link in the first place, and at
+///    `codegen-units > 1` this module is its own object. Being `#[inline(never)]`
+///    and defined here, this function *is* that reason: the call from
+///    `js_gc_init` selects the member, and the constructor rides along.
+/// 2. Re-applies the option, covering the (currently hypothetical) case of a
+///    host that ran Perry's runtime without honouring module initializers. Any
+///    arena mimalloc reserves *after* this point is then tagged correctly.
+///
+/// It deliberately does *not* reference `MOD_INIT_FUNC_ENTRY` from code, which
+/// is the obvious way to write step 1 and does not link: `ld` rewrites
+/// `__DATA,__mod_init_func` into a `__TEXT,__init_offsets` table of 32-bit
+/// offsets, so the static has no address left to take and the reference fails
+/// as `ADRP out of range ... to 0x00000000`.
+#[inline(never)]
+pub fn ensure_mimalloc_os_tag_applied() {
+    #[cfg(all(
+        target_pointer_width = "64",
+        target_vendor = "apple",
+        feature = "alloc-mimalloc"
+    ))]
+    apply_mimalloc_os_tag();
+}

docs/src/internals/memory-model.md

@@ -178,13 +178,28 @@ Two things to know before reading `vmmap`, Instruments' VM Tracker, or
   zones.
 - **Those regions used to render as `IOAccelerator`** — i.e. GPU driver
   memory — because mimalloc tags its mappings with VM tag 100, which macOS
-  tooling decodes as `IOAccelerator`. Since #6882 the runtime retags them to
-  `VM_MEMORY_APPLICATION_SPECIFIC_1` (240) during `js_gc_init`, so the JS
-  heap shows up as **`Memory Tag 240`**. If you see large `IOAccelerator`
-  regions in a Perry process, you are either on a pre-#6882 build or looking
-  at the few pages mapped before `js_gc_init` ran; there is no GPU memory
-  involved. Set `MIMALLOC_OS_TAG=<n>` to steer the tag yourself — the
-  runtime's retag defers to an explicit env setting.
+  tooling decodes as `IOAccelerator`. The runtime retags them to
+  `VM_MEMORY_APPLICATION_SPECIFIC_1` (240), so the JS heap shows up as
+  **`Memory Tag 240`** and there is no GPU memory involved. Set
+  `MIMALLOC_OS_TAG=<n>` to steer the tag yourself — the runtime defers to an
+  explicit env setting.
+
+  The retag runs from a `__DATA,__mod_init_func` constructor
+  (`perry-runtime/src/mimalloc_os_tag.rs`), *not* from `js_gc_init`, and that
+  placement is load-bearing: mimalloc reserves a 1 GiB arena on its first
+  allocation — during `std`'s pre-`main` startup — and every later allocation
+  just commits pages inside that already-tagged region, so setting the option
+  from any Rust code, `main` included, retags nothing that matters. #6882 set
+  it from `js_gc_init` and was therefore inert for the whole heap until #7450
+  moved it pre-`main`. On a build between those two, expect ~all of the heap
+  as `IOAccelerator` and a token `Memory Tag 240` region.
+
+  So: large `IOAccelerator` regions on a current build are a *bug*, not a
+  documentation caveat — most likely the module initializer was dropped from
+  the link. `crates/perry-runtime/src/gc/tests/os_tag.rs` asserts the real
+  thing (the kernel's `user_tag` for a live heap address, via
+  `mach_vm_region`); note that `mi_option_get(mi_option_os_tag)` reports 240
+  in the broken case too, so it is not a diagnosis.
 
 Also note that mimalloc purges freed memory with `MADV_FREE`-style advice:
 macOS keeps such pages counted in RSS and `phys_footprint` until memory