Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 7 additions & 6 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1145,12 +1145,13 @@ jobs:

# GATING, and deliberately so. CLAUDE.md's GC knob kill-policy requires
# every GC knob to have an arm that exercises it; the #7154 instruments
# (PERRY_GC_PROTECT_FROMSPACE, PERRY_GC_ZEAL) would otherwise be dark
# knobs on the subsystem with this repo's worst history of configuration
# rot. This asserts BOTH defaults — inert with the knobs unset, live with
# them set — and refuses to pass unless the zeal arm forced strictly more
# collections than the pressure-only arm, so it cannot go green having
# run zero copying minors (the #6942 / #7024 / #7025 failure mode).
# (PERRY_GC_PROTECT_FROMSPACE, PERRY_GC_SCHEDULE_SEED) would otherwise be
# dark knobs on the subsystem with this repo's worst history of
# configuration rot. This asserts BOTH defaults — inert with the knobs
# unset, live with them set — and refuses to pass unless the
# PERRY_GC_SCHEDULE_RATE=1 arm forced strictly more collections than the
# pressure-only arm, so it cannot go green having run zero copying minors
# (the #6942 / #7024 / #7025 failure mode).
Comment thread
coderabbitai[bot] marked this conversation as resolved.
# The detection property itself is a required-gate unit test:
# gc/tests/fromspace_protect.rs::quarantine_catches_a_planted_stale_from_space_deref.
# ~20s: the fixture is sized for ~1200 back-edge polls, not #7154's 240k.
Expand Down
9 changes: 5 additions & 4 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,16 +134,17 @@ Generational mark-sweep GC in `crates/perry-runtime/src/gc.rs` (default since v0

### Rooting-bug instruments (#7154 family) — what each knob ACTUALLY gates

A "GC value live but not rooted across a collection point" bug is invisible at collection time: there is nothing for the collector to find. It surfaces one or more cycles later, in a different function, as `TypeError: value is not a function`. These three knobs exist to collapse that latency. **All default-off; every boolean knob's OFF state is asserted in `gc/tests/fromspace_protect.rs`** (`…_DEPTH` is a magnitude, not a mode, so its floor and default are asserted instead). The instruments are **sabotage-tested**, not merely exercised: `quarantine_catches_a_planted_stale_from_space_deref` plants a #7184/#7192-shaped stale from-space pointer and asserts the instrument distinguishes it from the live object that would otherwise be recycled into those bytes — so a green protected run means the detector works, not that nothing was tried.
A "GC value live but not rooted across a collection point" bug is invisible at collection time: there is nothing for the collector to find. It surfaces one or more cycles later, in a different function, as `TypeError: value is not a function`. These knobs exist to collapse that latency. **All default-off; every knob's OFF state is asserted — the quarantine's in `gc/tests/fromspace_protect.rs`, the schedule's in `gc/tests/schedule.rs`** (`…_DEPTH` and `…_RATE` are magnitudes, not modes, so their floors, defaults and clamping are asserted instead). The instruments are **sabotage-tested**, not merely exercised: `quarantine_catches_a_planted_stale_from_space_deref` plants a #7184/#7192-shaped stale from-space pointer and asserts the instrument distinguishes it from the live object that would otherwise be recycled into those bytes — so a green protected run means the detector works, not that nothing was tried.

| knob | gates EXACTLY | does NOT |
|---|---|---|
| `PERRY_GC_PROTECT_FROMSPACE=1` (or `poison`) | the from-space reset performed by the **copying minor** (`arena::copying_reset_from_spaces_and_flip`). Retired Eden + active-survivor blocks are detached into a bounded quarantine, poison-filled (`0xDEADBEEFBAADF0DE`, `obj_type = 0xDE`) and, at `=1`, `mprotect(PROT_NONE)`d. A stale deref then SIGSEGVs at the faulting instruction; the installed reporter names the address, the retiring minor, and the last-known object's `obj_type`/size, then restores `SIG_DFL` and re-faults so a core/debugger still sees the real site. `poison` skips `mprotect`. | change the non-moving minor's `arena_reset_empty_blocks`, the full mark-sweep's reclaim, old-gen defrag, or the malloc sweep. **A run with zero copying minors protects nothing** — check that `PERRY_GC_DIAG=1` prints a `[gc-fromspace-protect] retired_set=#N` line. |
| `PERRY_GC_PROTECT_FROMSPACE_DEPTH=N` (default 4) | how many retired page-sets stay quarantined. Evicted sets are restored to RW and **recycled back into Eden**, never `dealloc`'d, so footprint is bounded at `N × from-space bytes`. `0` is clamped to 1 — a depth of 0 would read as ON and protect nothing. **Raise this when a suspected bug does not fault**: a value can cross hundreds of collections between its last valid observation and its stale use (one per back-edge poll under zeal). #7154's `new C(…)` reproducer needs `800` — its constructor crosses 600 polls, so the default 4 misses it silently. | — |
| `PERRY_GC_ZEAL=1` | forces an evacuating minor at every **GC safepoint**: `js_gc_loop_safepoint` (loop back-edge) and the outermost microtask-pump safepoint. It bypasses exactly two things — the `GC_SAFEPOINT_PENDING` requirement in `js_gc_loop_safepoint`, and the `gc_budgeted_due_trigger()` "is anything due?" test in `gc_safepoint_moving_minor`. Also makes `gc_force_evacuate_enabled()` true, so survivors actually MOVE. | bypass `gc_safepoint_moving_minor`'s **entry guards**: a safepoint reached mid-allocation (`GC_FLAG_IN_ALLOC`), suppressed (`GC_FLAG_SUPPRESSED`), inside an unsafe FFI zone, under a non-zero `GC_ROOT_LOCK_DEPTH`, or during a budgeted cycle still returns without collecting. Nor does it override an explicit `PERRY_GEN_GC_EVACUATE=0` — that wins, and with it set zeal moves nothing and surfaces nothing. Nor does it emit loop polls — those need the **compile-time** `PERRY_GC_MOVING_LOOP_POLLS=1` (default off since #7161). Zeal on a binary compiled without polls only fires at event-loop boundaries; a compute-only loop never collects. Check `crate::gc::zeal_forced_collections()` is nonzero. There is deliberately **no level 2**: the alloc-point arm forces a conservative stack scan, which makes the copying minor ineligible, so an "every allocation" zeal would run non-moving minors and move nothing. |
| `PERRY_GC_PROTECT_FROMSPACE_DEPTH=N` (default 4) | how many retired page-sets stay quarantined. Evicted sets are restored to RW and **recycled back into Eden**, never `dealloc`'d, so footprint is bounded at `N × from-space bytes`. `0` is clamped to 1 — a depth of 0 would read as ON and protect nothing. **Raise this when a suspected bug does not fault**: a value can cross hundreds of collections between its last valid observation and its stale use (one per back-edge poll at `PERRY_GC_SCHEDULE_RATE=1`). #7154's `new C(…)` reproducer needs `800` — its constructor crosses 600 polls, so the default 4 misses it silently. | — |
| `PERRY_GC_FROMSPACE_SCAN_ABORT=1` | now **implies** `PERRY_GC_FROMSPACE_SCAN=1`. It used to be inert alone (the scan never ran, so nothing aborted, and the run reported success). | — |
| `PERRY_GC_SCHEDULE_SEED=<u64>` | seeded GC-schedule fuzzing — the collection schedule as a knob, from normal pacing up to a collection at every handled safepoint (`PERRY_GC_SCHEDULE_RATE=1`). Three things, exactly: (1) `js_gc_loop_safepoint` stops requiring `GC_SAFEPOINT_PENDING` before descending into `gc_safepoint_moving_minor`; (2) inside `gc_safepoint_moving_minor`, **past the entry guards**, a per-thread safepoint counter advances once per handled safepoint and, when `gc_budgeted_due_trigger()` reports nothing due, a minor runs anyway iff `splitmix64(splitmix64(seed) ^ counter) < threshold`; (3) `gc_force_evacuate_enabled()` becomes true, so survivors MOVE. **A value that does not parse as `u64` reads as OFF, not as seed 0.** The seed is printed at startup, from the process-exit teardown funnel every exit path routes through (`report_exit_summary`, on the collection-side-allocation release — perry's `_exit` paths never reach `atexit`, which is only a libc-return backstop), and on panic/SIGSEGV/SIGBUS/SIGABRT/SIGILL/SIGTRAP — the signal reporter chains to (and is re-layered on top of) the from-space quarantine's, so the two compose. Live-subject counters: `gc::gc_schedule_safepoints()` / `gc::gc_schedule_forced_collections()`. | bypass `gc_safepoint_moving_minor`'s entry guards — and a blocked safepoint deliberately does **not** tick the counter, so the ordinal sequence tracks the program's safepoints rather than its allocation state. Nor override `PERRY_GEN_GC_EVACUATE=0`. Nor emit loop polls (those need the compile-time `PERRY_GC_MOVING_LOOP_POLLS=1`, default off since #7161; without them a seeded run only fires at event-loop boundaries and a compute-only loop never collects). Nor *suppress* pressure-driven collections — the rate is additional density, never less. Determinism is **per-thread**: the counter is thread-local, so a single-threaded program replays exactly, while a `perry/thread` program is only as reproducible as its OS scheduling. Say which you measured. |
| `PERRY_GC_SCHEDULE_RATE=<0..1>` (default `0.05`) | **only** the threshold `PERRY_GC_SCHEDULE_SEED`'s hash is compared against — the expected fraction of handled safepoints that collect. Out-of-range values clamp (a `2` reads as 1.0); unparseable and NaN fall back to the default. | do anything at all without a seed. It is inert alone. `=0` is an on-but-selects-nothing control (banner and reporters still install), `=1` collects at every handled safepoint — the maximum-density endpoint, where the seed stops mattering because every ordinal is selected whatever it hashes to. There is deliberately **no allocation-point level**: the alloc-point arm forces a conservative stack scan, which makes the copying minor ineligible, so an "every allocation" density would run non-moving minors and move nothing. |

`PERRY_GC_ZEAL=1 PERRY_GC_PROTECT_FROMSPACE=1` together is the pairing that turns a #7154 bug into an immediate precise fault. Compile *and* run with `PERRY_GC_MOVING_LOOP_POLLS=1` for in-loop coverage.
`PERRY_GC_SCHEDULE_SEED=<u64> PERRY_GC_PROTECT_FROMSPACE=1` together is the pairing that turns a #7154 bug into an immediate precise fault. Compile *and* run with `PERRY_GC_MOVING_LOOP_POLLS=1` for in-loop coverage. Reach for `PERRY_GC_SCHEDULE_RATE=1` when you want maximum pressure and can afford the slowdown, and for a lower rate when rate 1 distorts the workload's timing enough that it dies somewhere uninteresting first — either way the seed hands back a reproducer. `scripts/gc_schedule_fuzz.sh <binary> [seed-count]` sweeps it (see `changelog.d/7317-seeded-gc-schedule-fuzzing.md`).

### GC knob kill-policy (binding)

Expand Down
6 changes: 3 additions & 3 deletions changelog.d/7196-gc-rooting-bug-instruments.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,12 @@ A GC value that is live but not rooted across a collection point leaves nothing

**1. From-space quarantine / protection — `PERRY_GC_PROTECT_FROMSPACE`.** After an evacuating minor, do not recycle from-space. `arena/quarantine.rs` detaches the retired Eden + active-survivor blocks (leaving `data = null` tombstones so block-index semantics are unchanged), fills them with a poison word whose first byte reads as an invalid `obj_type` (`0xDE`), and `mprotect(PROT_NONE)`s their page-aligned interior. A stale dereference now SIGSEGVs **at the faulting instruction**, with the holder still live on the stack. The installed SIGSEGV/SIGBUS reporter prints the faulting address, which minor retired it, the last-known object that occupied that offset (`obj_type` + size, from a census taken before poisoning) and a native backtrace, then restores `SIG_DFL` and returns so the instruction re-faults — a core file, debugger or crash reporter still sees the real site. `=poison` selects poison without `mprotect`, which is also what the sub-page block edges `mprotect` cannot cover always get; protected and poisoned byte counts are reported separately so a run can never claim page protection it did not get. `PERRY_GC_PROTECT_FROMSPACE_DEPTH` (default 4, minimum 1) bounds memory: the quarantine is a ring, and expired sets are restored to read/write and recycled **back into Eden** rather than freed, so nothing that was ever `mprotect`ed is handed to the system allocator and steady-state footprint is `depth × from-space bytes`.

**2. GC zeal — `PERRY_GC_ZEAL=1`.** Force an evacuating minor at every GC safepoint (loop back-edge polls and the outermost microtask-pump boundary) instead of only when nursery pressure is due, so an unrooted value moves on its FIRST exposure rather than whenever an unrelated allocation burst happens to line up. Implies `gc_force_evacuate_enabled()` — a zealous minor that left survivors in place would move nothing and could not surface the bug — while still losing to an explicit `PERRY_GEN_GC_EVACUATE=0`, so the two knobs cannot silently disagree.
**2. Forced collections — `PERRY_GC_SCHEDULE_SEED=<u64> PERRY_GC_SCHEDULE_RATE=1`.** Force an evacuating minor at every GC safepoint (loop back-edge polls and the outermost microtask-pump boundary) instead of only when nursery pressure is due, so an unrooted value moves on its FIRST exposure rather than whenever an unrelated allocation burst happens to line up. Implies `gc_force_evacuate_enabled()` — a forced minor that left survivors in place would move nothing and could not surface the bug — while still losing to an explicit `PERRY_GEN_GC_EVACUATE=0`, so the two knobs cannot silently disagree.

**3. Verify-roots gap closures.** `PERRY_GC_FROMSPACE_SCAN_ABORT=1` now **implies** `PERRY_GC_FROMSPACE_SCAN=1`; on its own it used to be completely inert (`run_fromspace_scan` returned at the enablement gate, so there was nothing to abort and the run reported success — an investigator reaching for the abort switch mid-hunt got a green run and no scan). The abort path now also prints a collector backtrace, and every offender sample reports the **target's** `obj_type` alongside the owner's, which is the field that distinguishes a dead closure (`4`) from a dead object (`2`) when triaging `value is not a function`.

**Documented against the knob kill-policy.** CLAUDE.md and `docs/src/internals/memory-model.md` gain a table stating what each knob gates *exactly* and, as importantly, what it does not — prior rounds were misled by knobs whose real effect differed from their name. Two caveats are called out explicitly because both can produce a vacuously green run: `PERRY_GC_PROTECT_FROMSPACE` gates **only** the copying minor's from-space reset (a run with zero copying minors protects nothing — check for the `[gc-fromspace-protect] retired_set=#N` line under `PERRY_GC_DIAG=1`), and `PERRY_GC_ZEAL` cannot emit loop back-edge polls that codegen never produced (those need the compile-time `PERRY_GC_MOVING_LOOP_POLLS=1`, default off since #7161; `crate::gc::zeal_forced_collections()` is the live-subject counter). There is deliberately no `PERRY_GC_ZEAL=2` "every allocation" level: the allocation-point arm forces a conservative stack scan, which makes the copying minor ineligible, so that level would run non-moving minors and move nothing — the exact shape of the `PERRY_GC_FORCE_EVACUATE` inertness defect (#6942 / #6946).
**Documented against the knob kill-policy.** CLAUDE.md and `docs/src/internals/memory-model.md` gain a table stating what each knob gates *exactly* and, as importantly, what it does not — prior rounds were misled by knobs whose real effect differed from their name. Two caveats are called out explicitly because both can produce a vacuously green run: `PERRY_GC_PROTECT_FROMSPACE` gates **only** the copying minor's from-space reset (a run with zero copying minors protects nothing — check for the `[gc-fromspace-protect] retired_set=#N` line under `PERRY_GC_DIAG=1`), and the seeded schedule cannot select loop back-edge polls that codegen never produced (those need the compile-time `PERRY_GC_MOVING_LOOP_POLLS=1`, default off since #7161; `crate::gc::gc_schedule_forced_collections()` is the live-subject counter). There is deliberately no "every allocation" level: the allocation-point arm forces a conservative stack scan, which makes the copying minor ineligible, so that level would run non-moving minors and move nothing — the exact shape of the `PERRY_GC_FORCE_EVACUATE` inertness defect (#6942 / #6946).

**Exercised, not dark.** The kill-policy's requirement is an arm that exercises the knob, and these get two. `gc/tests/fromspace_protect.rs` (12 tests, in the required `cargo-test` gate) asserts **both** states of every boolean knob — the OFF arm proves the collector is byte-for-byte unchanged when the instrument is off, the ON arm asserts its subject was live (an object actually moved) before believing the result — and `quarantine_catches_a_planted_stale_from_space_deref` **sabotage-tests** the detector: it plants the #7184 / #7192 shape (a mutator keeping a pre-collection address across an evacuating minor), then asserts the instrument reports poison where the un-instrumented control reads a valid recycled object. That control is what makes the verdict meaningful — it demonstrates the bug is genuinely invisible without the instrument. On top of that, `scripts/gc_instrument_smoke.sh` runs in `gc-stress` as a gating step covering the integrated path a unit test cannot reach (codegen emitting back-edge polls → zeal firing on them → the copying minor → quarantine retirement in a real compiled program); it fails unless the zeal arm forces strictly more collections than the pressure-only arm, so it cannot go green having run zero copying minors.
**Exercised, not dark.** The kill-policy's requirement is an arm that exercises the knob, and these get two. `gc/tests/fromspace_protect.rs` (12 tests, in the required `cargo-test` gate) asserts **both** states of every boolean knob — the OFF arm proves the collector is byte-for-byte unchanged when the instrument is off, the ON arm asserts its subject was live (an object actually moved) before believing the result — and `quarantine_catches_a_planted_stale_from_space_deref` **sabotage-tests** the detector: it plants the #7184 / #7192 shape (a mutator keeping a pre-collection address across an evacuating minor), then asserts the instrument reports poison where the un-instrumented control reads a valid recycled object. That control is what makes the verdict meaningful — it demonstrates the bug is genuinely invisible without the instrument. On top of that, `scripts/gc_instrument_smoke.sh` runs in `gc-stress` as a gating step covering the integrated path a unit test cannot reach (codegen emitting back-edge polls → the schedule firing on them → the copying minor → quarantine retirement in a real compiled program); it fails unless the rate-1 arm forces strictly more collections than the pressure-only arm, so it cannot go green having run zero copying minors.

Platform note: page protection is Unix-only. `mprotect` / `sigaction` / `sysconf` are not exposed by the `libc` crate on `x86_64-pc-windows-msvc`, a target `perry-runtime` is genuinely built for (`test.yml`'s `windows-build`, and `release-packages.yml` via `perry-ui-windows`). The syscall helpers are `cfg(unix)`-gated per the existing `pty::native` precedent; off Unix `=1` degrades to `poison`, visibly rather than silently, because `bytes_protected` stays 0 while `bytes_poisoned` counts the whole retired range.
Loading