fix(gc): refuse the knob combination that emits a rootless binary (#7326)

changelog.d/7331-rootless-combination-refused.md

@@ -0,0 +1,21 @@
+`PERRY_SHADOW_STACK=0` combined with `PERRY_STATEPOINTS=1` (or `PERRY_RS4GC=1`)
+produced a binary with **no precise frame roots at all**, silently: no
+`__perry_gcmap` section, the same size as a plain shadow-off build, and it ran
+and printed the correct answer. Nothing distinguished it from a correct build
+until a collection moved something live.
+
+The cause is structural rather than a missing check. The statepoint backends are
+an alternative *lowering* of the shadow stack's root-set analysis, not an
+independent mechanism: `reserve_shadow_slot()` is the single entry point that,
+under `native_stack_roots_enabled()`, allocates a stack-map slot instead — and
+the caller of that analysis returns empty maps outright when the shadow stack is
+off. Switching one off switches the other off with it.
+
+The combination is now a hard error. Each knob alone is unchanged, so the
+bisection knob keeps its meaning; it simply cannot be combined with a backend
+that depends on the analysis it disables.
+
+Worth recording for the adoption plan: because the two share this analysis,
+"delete the shadow stack and keep statepoints" is not currently expressible, and
+any plan treating them as interchangeable mechanisms needs that premise
+corrected first.

crates/perry-codegen/src/codegen/helpers.rs

@@ -67,10 +67,37 @@ pub(super) fn shadow_stack_enabled() -> bool {
     use std::sync::OnceLock;
     static CACHED: OnceLock<bool> = OnceLock::new();
     *CACHED.get_or_init(|| {
-        !matches!(
+        let on = !matches!(
             std::env::var("PERRY_SHADOW_STACK").as_deref(),
             Ok("0") | Ok("off") | Ok("false")
-        )
+        );
+        // #7326: the statepoint backends are an alternative *lowering* of this
+        // analysis, not an independent mechanism. `reserve_shadow_slot()` is
+        // the single entry point that, under `native_stack_roots_enabled()`,
+        // allocates a stack-map slot instead of a shadow-stack slot — and the
+        // caller of that analysis returns empty maps outright when this is off.
+        //
+        // So switching the shadow stack off switches the statepoint roots off
+        // with it, and the result is a binary with NO precise frame roots that
+        // still runs and prints the right answer: measured, no `__perry_gcmap`
+        // section at all, same size as a plain shadow-off build. Nothing about
+        // the run distinguishes it from a correct one until a collection frees
+        // a live object.
+        //
+        // Refuse, rather than emit it. The bisection knob keeps its meaning on
+        // its own; it simply cannot be combined with a backend that depends on
+        // the analysis it disables.
+        if !on && native_stack_roots_enabled() {
+            panic!(
+                "perry: PERRY_SHADOW_STACK=0 cannot be combined with \
+                 PERRY_STATEPOINTS/PERRY_RS4GC. The statepoint backends reuse the \
+                 shadow stack's root-set analysis to decide what to root, so \
+                 disabling it produces a binary with no precise frame roots at all \
+                 — silently, since such a binary still runs correctly until a \
+                 collection moves something live (#7326). Drop one of the two."
+            );
+        }
+        on
     })
 }