Skip to content

repsel: the coverage work converted exactly as predicted, and almost none of it is faster (one −4.1% win, one +14.9% regression) #7128

Closed
#7133
Closed
repsel: the coverage work converted exactly as predicted, and almost none of it is faster (one −4.1% win, one +14.9% regression)#7128
#7133

Description

@proggeramlug
proggeramlug
opened on Jul 31, 2026

Follow-up measurement for #7106 / #7034, on main at 95988fb4d.

The coverage work converted exactly as #7122 predicted — per local, with zero
residue and no further mechanism. Almost none of it is faster. Corpus-wide the
campaign is a null on speed with one real win and one real regression, and
the regression is larger than the win.

The reason is one sentence: selection is not emission. canonical-i32 now
promotes in 17 of 18 real workloads, and changes the emitted object on 2.

Method, and what had to be true before any number was believed

Four compiler arms per host, all built from one target dir in one session, so
the rustc/LLVM toolchain is identical across arms by construction:

arm commit what it is
base 3f1a0853a census-only commit, before the campaign
pre7122 e7bc73bd6 before the loop-induction rule
at7122 4d3ddc9a3 #7122 in, module-init gate still on
main 95988fb4d after #7107 #7113 #7117 #7119 #7121 #7122

cargo build --release -p perry -p perry-runtime-static -p perry-stdlib-static
throughout — building the runtime crates alone leaves a stale .a and both arms
behave identically, which for this experiment would have been a catastrophic
false null. Archive mtimes verified to move per arm; perry and
libperry_runtime.a md5s verified distinct on both hosts.

Hosts, with load stated per measurement:

  • Raspberry Pi 5, aarch64 Linux, 4 cores. Load 0.16 when timing started,
    1.01 after. perf stat -e instructions:u, 11 repeats, warmed. Primary.
  • Mac mini M1, 8 cores. Load ~4–8 (it runs the long-lived perry-ship).
    Wall time only — macOS has no unprivileged instruction counter, so
    instructions retired on Apple silicon is something I could not measure.

Preconditions, all checked:

  1. The compiler is deterministic — on macOS. All 26 census workloads
    compiled twice with identical flags gave byte-identical objects (0
    nondeterministic, 0 failures). On Linux it is not (see finding E), so
    every object-level claim below is from the mini.
  2. The knobs are not spuriously keyed into codegen. PERRY_..._LOCALS=1 and
    an unrelated PERRY_TOTALLY_UNRELATED=0 both reproduce the default object
    bit-for-bit; only =0 moves it.
  3. Objects, not executables, decide "did emission change." On macOS the
    linked executable hash moves even when the object is byte-identical (the
    output path lands in the debug map) — suite_13_factorial "changed" at the
    executable level while its object was identical. Emission claims are at the
    --no-link object level.
  4. Correctness. All 32 measured binaries on each host match the pinned Node
    26.5.1 oracle on their deterministic output lines; harness exit 0 on
    both. Two earlier versions of that check were vacuous and I am recording
    why: the first filter stripped primes:78498 along with prime_sieve:123
    and compared empty to empty; the second masked lines that differ across two
    runs, which misses a timing line whose value happens to repeat. The shipped
    check derives the timing key from the source and prints how many lines were
    actually compared. 13_factorial prints sum:49950000000 correctly in every
    arm, so perf(repsel): admit constant-bounded loop induction variables to canonical i32 (#7110) #7122's deliberate refusal to admit accumulators is intact.

1. The coverage delta

Measured by running the census on my own builds of both arms, not read out of
baseline.json. Both reproduce the committed baseline exactly.

Corpus-wide totals:

census key base 3f1a085 main 95988fb
canonical-i32 13 64
canonical-u32 1 2
canonical-str 1 3
ptr-shape 4 7
ptr-shape-consumed (not instrumented) 3
ptr-numarray 5 6
int-valued-ta 1 1
spec-abi-entry 6 8
spec-abi-taptr-slot 2 2

Per workload (c-i32 / c-u32 / c-str, ptr-shape, shape-consumed, numarr;
- = 0 in both arms, x = workload did not exist at base):

workload c-i32 c-u32 c-str ptr-shape shape-cons numarr
batch 3→5 - - 0→2 0→1 -
suite_02_loop_overhead 0→2 - - - - -
suite_03_array_write 0→2 - - - - 1→1
suite_04_array_read 0→2 - - - - 1→1
suite_05_fibonacci 0→1 - - - - -
suite_06_math_intensive 0→2 - - - - -
suite_07_object_create 0→2 - - 1→1 - -
suite_08_string_concat 0→2 - 0→1 - - -
suite_09_method_calls 0→2 - - 1→1 - -
suite_10_nested_loops 0→3 - - - - 1→1
suite_11_prime_sieve 0→3 - - - - -
suite_12_binary_trees 0→2 - - 1→1 - -
suite_13_factorial 0→2 - - - - -
suite_14_closure 0→2 - - - - -
suite_15_mandelbrot 0→6 - - - - -
suite_16_matrix_multiply 3→5 - - - - -
suite_17_loop_data_dependent 0→4 - - - - 1→1
suite_01_startup - - - - - -
fixture_canonical_slots 2→3 1 1 - 0 -
fixture_int_valued_ta 1→3 - - - 0 -
fixture_ptr_numarray 2 - - - 0 1
fixture_ptr_shape - - - 1 0→1 -
fixture_spec_abi_taptr 2→3 - - - 0 -
fixture_loop_bounded_i32 (new) x→3 x x x x x
fixture_module_init_canonical (new) x→3 x→1 x→1 x x x→1
fixture_ptr_shape_sites (new) x x x x→1 x→1 x

Coverage on the 18 non-fixture workloads:

representation before after
canonical-i32 2/18 17/18
canonical-str 0/18 1/18
canonical-u32 0/18 0/18
ptr-shape (selected) 3/18 4/18
ptr-shape (consumed) — 1/18 (batch only)
ptr-numarray 4/18 4/18
int-valued-ta / spec-abi-entry / spec-abi-taptr 0/18 0/18

ptr-shape at main: 7 selected, 3 consumed — 4 proven and thrown away, by
module_init_context (2) and scalar_replaced (2), plus 8 proven-this
receiver consumptions counted separately.

2. #7122's predicted conversions: all of them, per local, zero residue

Matched by local identity across three compilers over one source tree
(the 3f1a0853a checkout, so file scope is identical by construction and only
the compiler varies), 483 files: test-files/test_gap_*.ts + app-pattern
kernels + benchmarks/suite + census fixtures. 483/483 reported, 0 failures.

canonical-slot verdicts:

verdict pre7122 at7122 main
selected I32 145 178 387
selected Str 67 67 244
selected U32 3 3 6
denied module_init_context 331 389 0
denied not_index_used_or_bounded 227 124 124
denied closure_referenced 14 14 14
denied declared_bigint 5 5 5
total selected 215 248 637

The predicted set — a local that pre7122 denied
not_index_used_or_bounded, that at7122 proves and denies only
module_init_context:

scope predicted converted residue
app-pattern kernels only 11 11 0
kernels + 200 test_gap files 38 38 0
all 483 files 66 66 0

100% conversion at every scope, all to I32. #7122's own figure was 18 on
its 201-file scope; that exact file set is not recorded, so I could not
reproduce the integer 18 — but the property it asserts is verified per local.

The same holds for the whole population: all 389 module_init_context
denials at at7122 are selected at main — zero residue, no fourth
mechanism. They split 209 I32 / 177 Str / 3 U32, exactly the +389 in the
selected total, and at main the region breakdown reads 209/177/3 in
module-init regions. The 248 non-entry selections are preserved 1:1.

One thing that looks like a regression and is not: 37 keys selected at at7122
have no match at main. All 37 are in generated closure_N regions whose
numbering shifts when module-init lowering changes; the per-file verdict
multisets are identical across arms in every affected file. Renumbering, not
loss.

3. Speed

Which workloads can move (mini, object level)

--no-link object hash, main compiler, default vs each knob. Only rows that
differ can possibly time differently:

workload i32 off str off shape off numarr off base vs main
batch DIFF same DIFF same DIFF
suite_09_method_calls DIFF DIFF DIFF same DIFF
suite_11_prime_sieve DIFF DIFF same same DIFF
suite_15_mandelbrot DIFF DIFF same same DIFF
suite_08_string_concat same DIFF same same DIFF
the other 13 real workloads same DIFF same same mostly same

canonical-i32 promotes in 17 of 18 real workloads and changes the object in
4 — and two of those four (batch, suite_09) are confounded, because the
i32 knob also disables all Ptr<Shape> consumption (finding A). Clean
canonical-i32 emission changes: 11_prime_sieve and 15_mandelbrot, 2 of
18.

Verified in IR, not from a counter — --trace llvm, pre-optimization:

workload arm alloca i32 alloca double add i32 fadd double
11_prime_sieve default 8 4 7 1
11_prime_sieve i32 off 8 9 7 4
15_mandelbrot default 8 8 3 4
15_mandelbrot i32 off 7 14 2 7
02_loop_overhead default 3 3 1 1
02_loop_overhead i32 off 3 5 1 2

02_loop_overhead is the point: the storage genuinely moves in the
pre-optimization IR, and the object is byte-identical — under the
parallel-shadow model every LocalGet already read the i32 slot, so -O3 was
already deleting the double slot. That is the shape of most of the 64.

Instructions retired — Raspberry Pi 5, load 0.16, 11 repeats

Controls first. Four rows whose two arms are the same binary (hash
verified equal), so any reading other than zero would mean the rig is
contaminated:

control instructions Δ
suite_01_startup (nothing promotes) +0.02%
suite_02_loop_overhead, i32 off +0.00%
suite_13_factorial, i32 off +0.00%
suite_16_matrix_multiply, i32 off +0.00%

Noise floor ≈ 0.02%. Every number below is far outside it.

workload arm instructions Δ vs default wall Δ
08_string_concat (committed, 100k) default 30,281,732 — —
canonical Str off 31,581,927 +4.29% +0.00%
at7122 31,581,897 +4.29% +0.00%
base compiler 31,581,755 +4.29% +0.00%
strconcat3M (scaled, #7125's program) default 887,474,088 — —
at7122 926,474,641 +4.39% +3.39%
canonical Str off 926,473,763 +4.39% +5.08%
11_prime_sieve default 2,597,179,419 — —
canonical i32 off 2,624,763,984 +1.06% +0.00%
at7122 2,624,766,517 +1.06% +0.25%
15_mandelbrot default 120,738,726 — —
canonical i32 off 105,110,004 −12.94% +2.08%
at7122 105,110,054 −12.94% +2.08%
pre7122 105,110,217 −12.94% +2.08%
batch canonical i32 off 3,370,404,098 −0.04% −1.16%
Ptr<Shape> off 3,371,040,568 −0.02% −1.16%
09_method_calls i32 off / shape off / base — −0.00% +0.00%
03_array_write base compiler — −0.00% +0.00%
02_loop_overhead i32 off / base — +0.00% +0.00%

Read as "what the campaign bought": canonical Str −4.1%, canonical i32
−1.05% on 11_prime_sieve and +14.87% on 15_mandelbrot, everything
else zero.

The 15_mandelbrot regression, bisected

at7122 and pre7122 produce the identical binary (f32490a4…), which is
also byte-identical to main-with-PERRY_CANONICAL_I32_LOCALS=0. So the
+15,628,722 instructions are #7121 alone, and the mechanism is exactly
module-init canonical-i32 selection on its top-level loops (0 → 6
promotions). Spread across 11 runs is 4 instructions. Output is correct
(total_iter:8011148 matches Node).

Wall time does not follow: 48 ms default vs 49 ms with i32 off, on both
hosts. Mandelbrot is FP-latency-bound and the extra integer work hides in the
shadow. Both facts are real and I am not picking the flattering one. The same
#7121 mechanism produced the −1.05% win on 11_prime_sieve, where wall time is
also flat (404 ms both).

Mac mini, wall time only (load ~4–8, 11 repeats)

Same sign wherever the Pi shows an effect, so nothing here is
microarchitecture-specific: strconcat3M 75 ms vs 79 ms for at7122 and 79 ms
for Str-off (−5.1%, intra-arm spread 0.0–1.3%, distributions disjoint);
15_mandelbrot 43 vs 44; 11_prime_sieve 166 vs 166, base 169; batch
229/229/229; 09_method_calls 106/106/106; controls 0.00%.

Reconciliation with #7125

measurement program metric result
#7125 3M, in-program timer (startup excluded), load ≈26 wall −6.8% median
here, Pi, load 0.16 3M, whole process instructions −4.21%
here, Pi, load 0.16 3M, whole process wall −3.4%
here, mini, load ~4–8 3M, whole process wall −5.1%
here, Pi, load 0.16 committed 100k, whole process instructions −4.12%

These agree. The spread is almost entirely whether process startup is in
the denominator. The committed 100k program runs in ~24 ms wall on the mini and
is essentially all startup, so the mini's 3M numbers net of startup are ~51 ms
vs ~55 ms — −7.3%, bracketing #7125's −6.8%. #7125's figure is right for
loop-only wall; −4.2% is the instruction cost actually removed. Time falls
further than instruction count because the change removes a call
(js_get_string_pointer_unified) whose cost exceeds its instruction count.

Two things #7125 could not show and this does:

  • The effect survives at the committed 100,000-iteration size — −4.12%
    instructions, on the unmodified committed file. Wall clock reads 21 ms in both
    arms there, which is why the 30× scaling was needed for a timer; instructions
    resolve it without changing the program, its working set, or its cache
    behaviour.
  • It is canonical Str, and nothing else: the at7122 arm and the Str-off
    arm agree to within 900 instructions out of 926 million.

Per the note in the brief: this is a different representation and a different
idiom from the −19% Ptr<Shape> per-site figure, and I am not treating it as
bearing on that number either way.

4. Binary size (separate from speed)

Mach-O __text, Mac mini, main compiler, knob-off minus default:

workload canonical i32 canonical Str Ptr<Shape>
suite_08_string_concat 0 −2,808 0
batch +1,532 0 +1,532
suite_11_prime_sieve +52 +4 0
suite_15_mandelbrot −12 +4 0
suite_09_method_calls 0 +4 0
suite_01_startup / 02_loop_overhead 0 0 / +4 0
  • canonical Str costs 2,808 bytes where it fires — size bought for speed.
  • Ptr<Shape> saves 1,532 bytes on batch, and that is the entire
    measurable effect of the representation on real code. It is also exactly the
    figure perf(repsel): return-shape facts — Ptr<Shape> survives the return escape (#7034 §4) #7107 quoted, reproduced independently.
  • canonical i32 is ±0 essentially everywhere, consistent with most of its 64
    promotions producing a byte-identical object.

A confound worth stating loudly: every main binary is 13,236 bytes
larger than the base binary, including suite_01_startup, which promotes
nothing at all. That is unrelated perry-runtime growth in the window
(node:test mock/property work, #7098–#7100), not repsel. Any base→main
size comparison must subtract it — e.g. batch reads −11,700 at base, which is
−13,236 runtime plus the +1,532 Ptr<Shape> saving.

Linux sizes are not reported: ELF .text moved in ≥4 KB quanta on the same
changes that shift Mach-O __text by 4 bytes, and Linux objects are
nondeterministic (finding E).

5. Five things the instrumentation gets wrong

A. PERRY_CANONICAL_I32_LOCALS=0 is not representation-isolating — it also
turns off all Ptr<Shape> consumption. codegen/function.rs:786,
method.rs:530 and :1590, closure.rs:953 all set
repsel_context_allows_ptr_shape: repsel_allows, and repsel_allows includes
canonical_i32_locals_enabled(). expr/mod.rs:1532 then returns None from
ptr_shape_receiver_fact at every access site. Census under that knob:
ptr-shape 7 selected, 0 consumed, with all six consumption sites printing
NEVER FIRES. #7121 split this flag for entry contexts; ordinary bodies still
key Ptr<Shape> on the canonical-i32 env gate. Any A/B on that knob measures
two representations, and batch/suite_09 above are exactly that case.

B. PERRY_CANONICAL_STR_LOCALS=0 is not scoped to Str locals either. It
also gates nanbox_string_ref_boxed (native_value/materialize.rs:444), which
fires on every string materialization regardless of any local being selected —
so 24 of 26 workloads emit differently under it, including workloads whose
canonical-str count is 0. The isolation that is valid for #7121's Str win is
the at7122-vs-main compiler arm, which is what I used.

C. Every __pshape clone is dead code. suite_09_method_calls emits two
(Counter__get__pshape, Counter__increment__pshape) and batch one
(Row__rescore__pshape); in each module the symbol appears on exactly the
define line and nowhere else — zero call sites. nm on the emitted object
finds zero pshape symbols in batch, 07_object_create,
09_method_calls, 12_binary_trees and fixture_ptr_shape: LLVM dead-strips
them before the object. The census's 8 consumed_receiver entries are therefore
counting consumption inside functions that never ship. suite_09 measures
−0.00% instructions on the shape knob, which is what dead code is worth.

D. Ptr<NumArray> emits nothing on real code. 6 selected corpus-wide, and
PERRY_PTR_NUMARRAY_LOCALS=0 leaves every real workload byte-identical —
only fixture_ptr_numarray differs. The knob works; the representation reaches
no shipped byte outside its own fixture. This is the Ptr<Shape>-on-batch
situation of #7034, one representation over.

E. Object emission is nondeterministic on Linux. Compiling the same file
twice with identical flags produced different objects for all 26 workloads
on aarch64 Linux (0/26 on macOS). Cause: the LLVM module name
perry_llvm_<pid>_<nanotime>_0.ll is embedded in the object — exactly 10 bytes
differ on suite_01_startup. This defeats object-hash A/B on Linux and is a
reproducible-builds problem independent of repsel.

6. My read

Did the coverage work compose into a real win? No — the aggregate is a null,
and it contains a regression.

What did compose is conversion: #7122's rule proves more, #7121 removes the
gate, and 100% of the blocked locals convert with no residue. That part of the
thesis is confirmed cleanly and per local. The census now measures something
real, and the selected/consumed split added in #7117/#7119 is what made this
measurable at all — without it the honest answer would have been "64 promotions,
looks great."

What did not compose is the step from promotion to instruction. canonical-i32
went from 2/18 to 17/18 real workloads and produced one −1.05%, one +14.87%, and
fifteen exact zeros. The zeros are not mysterious: the parallel-shadow model
already fed the i32 slot to every LocalGet, so -O3 was already deleting the
double slot, and canonical-i32 mostly deletes IR that was going to be deleted
anyway. That is a real simplification of the emitted IR and a prerequisite for
later phases; it is not, on this corpus, a speedup.

The one genuine speed win in the campaign is canonical Str on a top-level +=
self-append: −4.1% instructions, stable across a 30× change in iteration count,
costing 2.8 KB of __text. It fires on 1 of 18 workloads.

What I would do next, in order:

  1. Investigate 15_mandelbrot (+14.87%). Bisected to perf(codegen): module-init / program-entry bodies select canonical i32/u32/Str (#7109) #7121, mechanism
    named, arms byte-identical to the knob-off build. It is invisible in wall
    time on both hosts, so nothing will catch it except an instruction gate.
  2. Split the Ptr<Shape> context flag in ordinary bodies (finding A), so
    the i32 bisection knob stops moving two representations.
  3. Decide about the __pshape clone path (finding C). It has never had a
    call site anywhere in this corpus and is dead-stripped before the object.
  4. Add instructions retired to the census-adjacent gates. Every effect in
    this report except one is invisible in wall time on a quiet box, and the one
    regression is negatively correlated with it.

What I could not measure

  • Instructions retired on Apple silicon — no unprivileged counter on macOS.
    Mini numbers are wall time only, at load ~4–8; used as a sign check against
    the Pi, not as primary evidence.
  • Linux binary size deltas — nondeterministic objects (finding E) plus ELF
    .text quantisation.
  • perf(repsel): admit constant-bounded loop induction variables to canonical i32 (#7110) #7122's exact "18" — the 201-file scope is not recorded, so I verified the
    property (100% conversion, zero residue) at three scopes rather than
    reproducing the integer.
  • Whether the base→main arm is purely repsel — perry-runtime also changed
    in the window (node:test, timer.rs), worth +13,236 bytes of __text on a
    workload that promotes nothing. Knob A/B and the at7122 arm are clean;
    base→main is not, and I flagged each place it is used.

Refs: #7106, #7034, #7109, #7110, #7121, #7122, #7123, #7125.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions