Skip to content

gc-ratchet: 7 gating rows red on main since 0.5.1451 — and 04_dead_after_deep_stack / 11_collect_at_depth are NOT among them #7843

Description

@proggeramlug

gc-ratchet: 7 gating rows red on main — and 04_dead_after_deep_stack / 11_collect_at_depth are NOT among them

Two separate findings from measuring the pinned gc-ratchet on current main
(ab1bd464b, perry 0.5.1464) on a darwin-arm64 host.

1. The two probes everyone believes are red are green

04_dead_after_deep_stack and 11_collect_at_depth have been carried as
"pre-existing red on main, unowned" since #7799. They are not red on current
main.
Measured, shared_ci profile, 7 repeats:

  • 04_dead_after_deep_stack — all nine gating cells +0.00%, bit-identical
    to the pin.
  • 11_collect_at_depth — all gating cells ok; heap_used_bytes is an
    improvement (−11.99%), copied_objects 6,139 → 6,141 (+0.03%, allowance
    307), promoted_objects 0 → 0, minor_cycles 1 → 1.

Why they looked red

The six rows that were reported (04.copied_objects, and 11's
heap_used_bytes / copied_objects / copied_bytes / promoted_objects /
promoted_bytes) are exactly the six cells that #7783's re-pin already
absorbed
, and every observation of them comes from a checkout that predates
that re-pin.

  • The re-pin landed as db44b31b7 at 2026-08-10 18:11 UTC.
  • perf(gc): predict the futile full mark-sweep instead of pricing it afterwards (retain 3.99x -> 2.54x node, retain_wide 7.01x -> 2.89x) #7799's PR body names its reference build as origin/main @ 0a2bf15bd.
    git merge-base --is-ancestor 0a2bf15bd db44b31b7yes, it predates the
    re-pin. Its baseline artifact therefore still carried the OLD values — and it
    had 13 probes, which is why the PR body says "all 13 probes". Main's
    baseline has 14 (14_grow_then_churn came with the re-pin).
  • The only two gc-ratchet CI runs that have executed and failed in the last
    200 (31415780870, 31408538105) both started before 18:11 UTC on 08-10,
    and their Baseline column is the pre-re-pin artifact verbatim:
    04.copied_objects 565, 11.heap_used_bytes 5,097,776,
    11.promoted_objects 6,150. Their Current column is exactly what main now
    pins (605 / 5,307,584 / 0).

So both arms of #7799's A/B compared against a stale artifact and went red
identically. "Not my PR" was correct; "pre-existing on main" did not follow.

No code fix is needed for these two probes. The stale premise is the bug.

2. Seven different gating rows ARE red on main

Same run, check exit code 1:

probe metric baseline current Δ
01_nursery_churn heap_used_bytes 5,229,288 5,640,688 +7.87%
06_string_retention heap_used_bytes 4,960,920 5,373,128 +8.31%
09_try_catch_roots heap_used_bytes 4,777,680 5,190,960 +8.65%
14_grow_then_churn heap_used_bytes 4,651,912 5,069,000 +8.97%
14_grow_then_churn step_cycles 26 24 −7.69%
14_grow_then_churn copied_objects 594 561 −5.56%
14_grow_then_churn copied_bytes 2,442,528 2,306,832 −5.56%

The four heap_used_bytes deltas are near-constant in absolute terms —
+411,400 / +412,208 / +413,280 / +417,088 bytes. That is not a fixed addition
to resident heap, though: across all 14 probes the shift is bimodal, with
five probes moving down by a similarly tight −552k…−636k and five unchanged
to the byte (full table below). Four rows fire only because the gate's
[increase] direction is the one that fails; the equally large moves the other
way pass as improvements. A retention boundary moved, not a constant.

This is code, not the host, and not noise

The baseline was pinned on the bench mini (perry-macos.fritz.box, Apple M1,
8 cores, 8 GB). I measured on a different machine (M1 Max, 10 cores, 68 GB), so
"the pin is not portable off its host" had to be excluded before any of this
could be called a regression. A binary A/B settles it — same host, same
harness, same probes, same baseline artifact, only the artifact triple
(perry + libperry_runtime.a + libperry_stdlib.a) swapped:

arm built at check --profile shared_ci
pin a1fd5a7e8 (perry 0.5.1451) — the commit the baseline was measured at gc-ratchet: OK, exit 0
main ab1bd464b (perry 0.5.1464) FAILED, exit 1, the 7 rows above

So the pinned artifact reproduces green on all 14 probes on a machine that is
not the one it was pinned on
— the shared_ci profile's claim that its gating
metrics "transfer across machine classes" holds, and the baseline is a valid
reference here. The 7 red rows are therefore code drift between 0.5.1451 and
0.5.1464
.

Repeatability control: arm-main was measured twice, in separate processes,
and produced an identical red-row set both times. Within each run the
harness reports heap_used_bytes spread 0% across 7 repeats on every probe,
and correctness=pass on all 14 in both arms.

Per-probe heap_used_bytes, both arms

The shift is bimodal, not one constant — which is why only four rows fire
(the gate's [increase] direction) while five equally-large moves in the other
direction pass as improvements:

probe pin 0.5.1451 main 0.5.1464 Δ
01_nursery_churn 5,229,288 5,640,688 +411,400
06_string_retention 4,960,920 5,373,128 +412,208
09_try_catch_roots 4,777,680 5,190,960 +413,280
14_grow_then_churn 4,651,912 5,069,000 +417,088
03_cross_gen_writes 1,394,736 1,394,736 0
04_dead_after_deep_stack 4,890,016 4,890,016 0
05_closure_capture 5,329,880 5,329,880 0
08_map_set_sidetables 1,512,024 1,512,024 0
12_large_live_set 50,435,552 50,435,552 0
02_survivor_promotion 9,414,464 8,862,256 −552,208
07_array_grow_evacuate 15,631,696 15,002,592 −629,104
10_store_receiver_across_alloc 4,664,720 4,029,640 −635,080
11_collect_at_depth 5,307,584 4,671,224 −636,360
13_large_eden_survivors 7,686,560 7,051,232 −635,328

Both arms genuinely exercised the collector — 64 vs 63 total minor_cycles,
~684k copied_objects, no probe with zero minors — so the green arm is not a
vacuous pass.

Reproducing

The build is cheap (5m32s incremental for the artifact triple), so bisecting
this is a short job:

git checkout --detach <commit>
CARGO_TARGET_DIR=<tgt> cargo build --release -p perry -p perry-runtime-static -p perry-stdlib-static
cp <tgt>/release/{perry,libperry_runtime.a,libperry_stdlib.a} /tmp/arm/
git checkout main            # harness, probes and baseline must be MAIN's
PERRY_NO_AUTO_OPTIMIZE=1 PERRY_RUNTIME_DIR=/tmp/arm \
  python3 benchmarks/gc_ratchet/gc_ratchet.py measure \
    --perry /tmp/arm/perry --repeats 7 --node "$(command -v node)" --output /tmp/m.json
python3 benchmarks/gc_ratchet/gc_ratchet.py check --current /tmp/m.json --profile shared_ci

Swap only the artifact triple; keep the checked-out harness on main so the
probe set and baseline are constant across arms.

Prime suspects in the window, all collector- or layout-adjacent:
1d22273a1 (#7809, per-object layout tables, +296 lines in
gc/layout_tables.rs), 99705c937 (#7812, single-slot pointer masks),
d04cc3f29 (#7834, pointer-free typed layout stamped into the allocation
header). #7799's author measured its own branch as gating-identical to its
reference, so it is unlikely to be ad8240b5b.

3. The gate has not run on main since 2026-08-09

Independent of either finding, gc-ratchet is producing no signal:

A gate that is not required, and whose runs never reach a runner, is dark in
both of the ways CLAUDE.md's "four ways a gate can be unable to fail" list
warns about. Both findings above went unnoticed for two days because of it.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions