perf(class): stop disarming every dispatch guard when a class prototype is materialized

changelog.d/7800-class-dispatch-prototype-latch.md

@@ -0,0 +1,87 @@
+### Fixed
+
+**Materializing `Class.prototype` no longer disarms class-dispatch speculation and every element-shape proof for the rest of the process.**
+
+`class_decl_prototype_value()` — the lazy materializer that creates a declared
+class's prototype object on first demand — called
+`invalidate_class_prototype_fast_guards()`. That is not a hint; it trips a
+process-global, **monotonic** latch that:
+
+* makes `js_method_direct_shape_guard` and
+  `js_typed_feedback_method_direct_call_guard` return "miss" for every receiver,
+  for the rest of the run, so every `recv.m()` on a declared class falls into
+  the `js_native_call_method` dispatch tower;
+* calls `crate::array::invalidate_all_element_shapes()`, retiring every
+  outstanding element-shape record (#7480), so `arr[i]` reads fall back to the
+  generic `js_require_object_coercible` + `js_is_symbol` +
+  `js_object_get_index_polymorphic` path;
+* bumps `VTABLE_GEN`, retiring the `vtable_ic` / `obj_dispatch_ic` dispatch
+  caches (#7769).
+
+The latch exists for the one event that can change which member `recv.m()`
+resolves to: a **write** to a prototype (`Class.prototype.m = fn`). Those are
+the two call sites in `class_registry/prototype_methods.rs`, and they keep it.
+Reaching the materializer changes none of it — the object is fresh and
+unobserved, and the writes immediately below install `constructor` plus exactly
+the methods the class already declares, which are the same answers the vtable
+already gives.
+
+What reaches the materializer, measured with a name-printing probe on the
+materializer itself: `new` on a class that `extends` anything, which
+materializes the instance's whole prototype ancestor chain (`class B extends A`
++ `new B()` = 2 materializations; a three-level chain = 3). NOT `instanceof`
+and NOT `Object.getPrototypeOf` — both trip zero. So an ordinary
+class-hierarchy program disarmed its own dispatch speculation the first time it
+constructed a subclass.
+
+Measured on `gc-handoff/apps/shapes.ts` with a counter on each precondition of
+the guard: **384,000 of 384,000 probes failed on this latch and on nothing
+else** (`descriptors_in_use`, the GC-header checks, and the object-type check
+all rejected zero). `gc-handoff/bench/shapes_dispatch.ts` shows the same for a
+program containing no `instanceof` at all.
+
+### Added
+
+`js_method_direct_shape_class` — the class-id half of
+`js_method_direct_shape_guard`, factored out so a call site can test more than
+one `(class id, keys token)` pair per probe. `js_method_direct_shape_guard` is
+now defined in terms of it, so the single-pair semantics are unchanged by
+construction.
+
+Codegen uses it to widen the shape-guarded direct call at a method callsite
+from ONE arm (the declared receiver class) to the declared class plus its
+subclass closure, each paired with the body the method resolves to when walked
+from that class. The declared-class guard is a bet that the receiver's dynamic
+class equals its static class; for a base-typed collection — `nodes: Node2D[]`
+holding `Rect` / `Circle` / `Square` / `Marker` / `Group` — that bet loses on
+every element. Arms are capped at `MAX_SUBCLASS_DISPATCH_ARMS` (8) so a wide
+hierarchy keeps today's single-arm form rather than growing a long compare
+chain, and only the shape-only guard is widened (the typed-feedback guard
+records a single-contract observation per site and keeps its one arm).
+
+### Notes for the next reader
+
+`gc-handoff/bench/shapes_{build,describe,dispatch,dispatch_static}.ts` are the
+committed decomposition of `apps/shapes.ts`, each annotated with its measured
+seconds. They record, among other things, that **class dispatch is not where
+`shapes.ts` loses**: on the quiet mini the whole `.area()` term is 0.013 s of a
+0.224 s program, and widening the dispatch guard alone moved it 0.2237 →
+0.2241 s. The two cost centres that decomposition does find are `build()`
+(0.1035 s, 46%) and `describe()`'s string concatenation (0.074 s, 33%, ~620 ns
+per `"lit" + this.stringField`).
+
+
+### Blast radius
+
+The latch is monotonic in production (the only `store(false)` is `#[cfg(test)]`),
+and nearly every class-hierarchy program trips it, so the obvious worry is that it
+silently disarms the element-shape repsel work (#7770/#7771/#7766/#7702). Measured:
+it does not. `invalidate_all_element_shapes()` bumps a GENERATION; each record
+carries the generation it was installed under and `ensure_element_shape`
+re-establishes it on the next query, so one bump costs at most one
+re-establishment per array. Adding a single `instanceof` or `Object.getPrototypeOf`
+before an otherwise identical hot loop moves nothing: 0.0222 s for a `churn_read`
+-shaped element-read loop over object literals AND over class instances, 2.46 s
+for a method-call-per-element loop. Only the dispatch-guard half is permanent, and
+on its own it is worth 1.0% on `shapes.ts`; it reaches 16.6% only combined with the
+multi-arm widening.

crates/perry-codegen/src/lower_call/method_override.rs

@@ -155,6 +155,28 @@ pub(super) fn emit_own_method_override_check(
     )
 }
 
+/// One additional `(class id, keys token) -> concrete method` arm for the
+/// shape-guarded direct call, describing a class in the DECLARED receiver
+/// class's subclass closure.
+///
+/// The declared-class guard speculates that the receiver's dynamic class is
+/// exactly its static class. For a receiver typed as the base of a hierarchy —
+/// `nodes: Node2D[]`, every element a `Rect` / `Circle` / `Square` / `Marker` /
+/// `Group` — that speculation is wrong for EVERY element, so the guard misses
+/// 100% of the time and each call pays a wasted guard plus the full
+/// `js_native_call_method` dispatch tower. Each arm here is the same proof the
+/// declared-class guard performs (exact class id + exact keys token), applied
+/// to one more class whose implementation of the method codegen already
+/// resolved statically.
+pub(super) struct SubclassDispatchArm {
+    /// `class_id` of the concrete subclass this arm matches.
+    pub class_id: u32,
+    /// Name of the module global holding that subclass's canonical keys array.
+    pub keys_global: String,
+    /// The method body `property` resolves to when walked from that subclass.
+    pub target_fn: String,
+}
+
 /// Emit a typed-feedback runtime guard before a known class-method direct call.
 ///
 /// The guard validates that the receiver still has the expected class shape,
@@ -175,9 +197,16 @@ pub(super) fn emit_guarded_direct_method_call(
     typed_i1_direct_fn: Option<(&str, Vec<crate::codegen::TypedParamRep>)>,
     typed_string_direct_fn: Option<(&str, Vec<crate::codegen::TypedParamRep>)>,
     shape_only_guard: bool,
+    subclass_arms: &[SubclassDispatchArm],
 ) -> Option<String> {
     let expected_class_id = *ctx.class_ids.get(receiver_class_name)?;
     let keys_global_name = ctx.class_keys_globals.get(receiver_class_name)?.clone();
+    // Only the shape-only guard is widened. The typed-feedback guard records an
+    // observation keyed to ONE (class, method, func ptr) contract per site; a
+    // multi-class site would feed it a stream of "different class" observations
+    // and it would (correctly) mark the site polymorphic. That form keeps its
+    // single-arm shape.
+    let subclass_arms: &[SubclassDispatchArm] = if shape_only_guard { subclass_arms } else { &[] };
 
     // Representation-selection Phase 5a: the proven-`this` clone for this
     // (class, method), when the emission loop produced one.
@@ -228,18 +257,82 @@ pub(super) fn emit_guarded_direct_method_call(
         ))
     };
 
+    // Per-arm keys tokens, loaded through the same entry-block init the
+    // declared class's token uses (module-init populates `@perry_class_keys_*`
+    // after the prelude, so the load may not be hoisted above it).
+    let subclass_keys: Vec<String> = subclass_arms
+        .iter()
+        .map(|arm| {
+            let slot = ctx.func.entry_init_load_global(&arm.keys_global, I64);
+            ctx.block().load(I64, &slot)
+        })
+        .collect();
+
     let guard_idx = ctx.new_block("method_direct.guard");
     let fast_idx = ctx.new_block("method_direct.fast");
+    // One test block and one case block per subclass arm. The declared class's
+    // own test lives in the guard block, so arm 0's test block is the guard's
+    // false edge.
+    let sub_test_idxs: Vec<usize> = (0..subclass_arms.len())
+        .map(|i| ctx.new_block(&format!("method_direct.subtest{i}")))
+        .collect();
+    let sub_case_idxs: Vec<usize> = (0..subclass_arms.len())
+        .map(|i| ctx.new_block(&format!("method_direct.sub{i}")))
+        .collect();
     let fallback_idx = ctx.new_block("method_direct.fallback");
     let merge_idx = ctx.new_block("method_direct.merge");
     let guard_label = ctx.block_label(guard_idx);
     let fast_label = ctx.block_label(fast_idx);
     let fallback_label = ctx.block_label(fallback_idx);
     let merge_label = ctx.block_label(merge_idx);
+    let sub_test_labels: Vec<String> = sub_test_idxs.iter().map(|&i| ctx.block_label(i)).collect();
+    let sub_case_labels: Vec<String> = sub_case_idxs.iter().map(|&i| ctx.block_label(i)).collect();
     ctx.block().br(&guard_label);
 
     ctx.current_block = guard_idx;
-    let guard_ok = if shape_only_guard {
+    // Multi-arm form: ONE probe resolves the receiver's class id and keys
+    // token (every precondition `js_method_direct_shape_guard` checks except
+    // the comparison itself), then an inline compare chain picks the arm. The
+    // single-arm form keeps its original single call.
+    let multi_arm = !subclass_arms.is_empty();
+    if multi_arm {
+        let keys_slot = ctx.func.alloca_entry(I64);
+        let cid = ctx.block().call(
+            I32,
+            "js_method_direct_shape_class",
+            &[(DOUBLE, recv_box), (crate::types::PTR, &keys_slot)],
+        );
+        let keys = ctx.block().load(I64, &keys_slot);
+        {
+            let next = sub_test_labels[0].clone();
+            let blk = ctx.block();
+            let cid_ok = blk.icmp_eq(I32, &cid, &expected_class_id_str);
+            let keys_ok = blk.icmp_eq(I64, &keys, &expected_keys);
+            let pass = blk.and(I1, &cid_ok, &keys_ok);
+            blk.cond_br(&pass, &fast_label, &next);
+        }
+        for (i, arm) in subclass_arms.iter().enumerate() {
+            ctx.current_block = sub_test_idxs[i];
+            let next = sub_test_labels
+                .get(i + 1)
+                .cloned()
+                .unwrap_or_else(|| fallback_label.clone());
+            let case_label = sub_case_labels[i].clone();
+            let class_id_str = arm.class_id.to_string();
+            let arm_keys = subclass_keys[i].clone();
+            let blk = ctx.block();
+            let cid_ok = blk.icmp_eq(I32, &cid, &class_id_str);
+            let keys_ok = blk.icmp_eq(I64, &keys, &arm_keys);
+            let pass = blk.and(I1, &cid_ok, &keys_ok);
+            blk.cond_br(&pass, &case_label, &next);
+        }
+        ctx.current_block = guard_idx;
+    }
+    let guard_ok = if multi_arm {
+        // The chain above already terminated the guard block and every test
+        // block; `fast_idx` / `fallback_idx` are entered from it unchanged.
+        String::new()
+    } else if shape_only_guard {
         ctx.block().call(
             I32,
             "js_method_direct_shape_guard",
@@ -267,9 +360,11 @@ pub(super) fn emit_guarded_direct_method_call(
             ],
         )
     };
-    let guard_pass = ctx.block().icmp_ne(I32, &guard_ok, "0");
-    ctx.block()
-        .cond_br(&guard_pass, &fast_label, &fallback_label);
+    if !multi_arm {
+        let guard_pass = ctx.block().icmp_ne(I32, &guard_ok, "0");
+        ctx.block()
+            .cond_br(&guard_pass, &fast_label, &fallback_label);
+    }
 
     ctx.current_block = fast_idx;
     let fast_value = {
@@ -795,6 +890,21 @@ pub(super) fn emit_guarded_direct_method_call(
         ctx.block().br(&merge_label);
     }
 
+    // One direct call per subclass arm. Reached only from that arm's test,
+    // which proved the receiver's class id AND keys token exactly — the same
+    // proof the declared-class arm rests on, so the statically resolved body
+    // is the one the dispatch tower would have found.
+    let mut sub_values: Vec<(String, String)> = Vec::with_capacity(subclass_arms.len());
+    for (i, arm) in subclass_arms.iter().enumerate() {
+        ctx.current_block = sub_case_idxs[i];
+        let value = ctx.block().call(DOUBLE, &arm.target_fn, direct_arg_slices);
+        let after = ctx.block().label.clone();
+        if !ctx.block().is_terminated() {
+            ctx.block().br(&merge_label);
+        }
+        sub_values.push((value, after));
+    }
+
     ctx.current_block = fallback_idx;
     let (args_ptr, args_len) = if fallback_user_args.is_empty() {
         ("null".to_string(), "0".to_string())
@@ -838,11 +948,11 @@ pub(super) fn emit_guarded_direct_method_call(
     }
 
     ctx.current_block = merge_idx;
-    Some(ctx.block().phi(
-        DOUBLE,
-        &[
-            (fast_value.as_str(), after_fast.as_str()),
-            (fallback_value.as_str(), after_fallback.as_str()),
-        ],
-    ))
+    let mut phi_inputs: Vec<(&str, &str)> = Vec::with_capacity(sub_values.len() + 2);
+    phi_inputs.push((fast_value.as_str(), after_fast.as_str()));
+    for (value, label) in &sub_values {
+        phi_inputs.push((value.as_str(), label.as_str()));
+    }
+    phi_inputs.push((fallback_value.as_str(), after_fallback.as_str()));
+    Some(ctx.block().phi(DOUBLE, &phi_inputs))
 }

crates/perry-codegen/src/lower_call/property_get/dynamic_dispatch.rs

@@ -15,9 +15,15 @@ use crate::types::{DOUBLE, I32, I64};
 // Reach the override-emit helpers (`pub(super)` of `lower_call`) by their
 // canonical crate-relative path.
 use crate::lower_call::method_override::{
-    emit_guarded_direct_method_call, emit_own_method_override_check,
+    emit_guarded_direct_method_call, emit_own_method_override_check, SubclassDispatchArm,
 };
 
+/// Cap on the number of extra `(class id, keys token)` arms a shape-guarded
+/// direct call may carry. A wide hierarchy would turn every callsite into a
+/// long inline compare chain — more instruction cache than the single tower
+/// call it replaces — so past this width the site keeps the single-arm guard.
+const MAX_SUBCLASS_DISPATCH_ARMS: usize = 8;
+
 /// #7142: the proven-`this` clone a class-id dispatch-tower case may route to,
 /// plus the keys token the routed path must re-check inline.
 struct TowerPshapeRoute {
@@ -892,6 +898,95 @@ pub(crate) fn try_lower_instance_method_call(
             let arg_slices: Vec<(crate::types::LlvmType, &str)> =
                 lowered_args.iter().map(|s| (DOUBLE, s.as_str())).collect();
 
+            // Arms for the shape-guarded direct call: every class in
+            // `class_name`'s subclass closure, paired with the body `property`
+            // resolves to from THAT class. Includes subclasses that do NOT
+            // override — a `Marker` receiver fails a `Node2D` class-id guard
+            // just as hard as a `Rect` one does, and the tower it falls into
+            // costs the same either way.
+            //
+            // The declared-class guard alone is a bet that the receiver's
+            // dynamic class equals its static class. Where a base-typed
+            // collection is the whole point of the hierarchy that bet loses
+            // every single time, and the miss is not free: it pays a guard
+            // call AND the full `js_native_call_method` tower.
+            let mut subclass_arms: Vec<SubclassDispatchArm> = Vec::new();
+            {
+                let mut seen_ids: Vec<u32> = vec![*ctx.class_ids.get(&class_name).unwrap_or(&0)];
+                let mut roots: Vec<(&String, u32)> =
+                    ctx.class_ids.iter().map(|(k, &v)| (k, v)).collect();
+                roots.sort_unstable_by(|a, b| a.1.cmp(&b.1).then_with(|| a.0.cmp(b.0)));
+                for (sub_name, sub_id) in roots {
+                    if *sub_name == class_name || sub_id == 0 || seen_ids.contains(&sub_id) {
+                        continue;
+                    }
+                    let mut parent = ctx
+                        .classes
+                        .get(sub_name)
+                        .and_then(|c| c.extends_name.clone());
+                    let mut is_subclass = false;
+                    while let Some(p) = parent {
+                        if p == class_name {
+                            is_subclass = true;
+                            break;
+                        }
+                        parent = ctx.classes.get(&p).and_then(|c| c.extends_name.clone());
+                    }
+                    if !is_subclass {
+                        continue;
+                    }
+                    let Some(keys_global) = ctx.class_keys_globals.get(sub_name).cloned() else {
+                        continue;
+                    };
+                    // Resolve through the SUBCLASS's own chain, and remember
+                    // where it landed: the rest-param shape is a property of
+                    // the declaring class, and a rest-bearing target cannot be
+                    // called with this site's flat, base-arity argument list.
+                    let mut cur = Some(sub_name.clone());
+                    let mut resolved: Option<(String, String)> = None;
+                    while let Some(c) = cur {
+                        let key = (c.clone(), property.to_string());
+                        if let Some(fname) = ctx.methods.get(&key).cloned() {
+                            resolved = Some((c, fname));
+                            break;
+                        }
+                        cur = ctx.classes.get(&c).and_then(|c| c.extends_name.clone());
+                    }
+                    let Some((decl_class, target_fn)) = resolved else {
+                        continue;
+                    };
+                    if target_fn.starts_with("perry_static_") {
+                        continue;
+                    }
+                    if matches!(
+                        ctx.method_has_rest
+                            .get(&(decl_class.clone(), property.to_string())),
+                        Some(&true)
+                    ) {
+                        continue;
+                    }
+                    if ctx
+                        .method_param_counts
+                        .get(&(decl_class, property.to_string()))
+                        .is_some_and(|&n| n > max_explicit_arity)
+                    {
+                        continue;
+                    }
+                    seen_ids.push(sub_id);
+                    subclass_arms.push(SubclassDispatchArm {
+                        class_id: sub_id,
+                        keys_global,
+                        target_fn,
+                    });
+                }
+            }
+            // A wide hierarchy would turn every callsite into a long compare
+            // chain — more instruction cache than the tower call it replaces.
+            // Beyond the cap the site keeps today's single-arm guard.
+            if subclass_arms.len() > MAX_SUBCLASS_DISPATCH_ARMS {
+                subclass_arms.clear();
+            }
+
             if !method_has_rest {
                 let typed_method_key = (class_name.clone(), property.to_string());
                 let typed_formal_count = ctx
@@ -1145,6 +1240,7 @@ pub(crate) fn try_lower_instance_method_call(
                     typed_i1_direct,
                     typed_string_direct,
                     shape_only_guard,
+                    &subclass_arms,
                 ) {
                     return Ok(Some(guarded));
                 }