Skip to content

GC: old-gen fragmentation — scattered survivors pin 105 MB of blocks for a ~1 MB live set #7437

Description

@proggeramlug

Found while building the 12_large_live_set gc-ratchet probe for #7432 (adaptive tenuring). This is a pre-existing old-gen characteristic that mass promotion now makes routine, not a #7432 regression.

Symptom

A program that promotes a ~100 MB linked structure, then drops all but every 64th node (~1 MB truly live), retains 105.6 MB of arena in-use after an explicit gc(). The full collection itself works — PERRY_GC_DIAG=1 shows freed_bytes=87,679,624 on the final manual collect — but the in-use accounting barely moves:

[gc] blocks: general=50 (49 live), longlived=46 (40 live), freed_bytes=87679624
#gcmetric heap_used_bytes=105588616

49 of 50 blocks still count as live because keeping every 64th node scatters ~218 live objects into every 1 MB block. Block reclaim (arena_reset_empty_blocks / old block reclaim) can only reclaim fully-dead blocks, and the swept holes don't lower block offsets, so heapUsed — and RSS, since the blocks stay mapped — tracks the scattered-survivor high-water forever. V8 compacts old space in this situation; Node reports a proportionally small heap for the same program.

The old-page defrag policy (C4b) did not engage: the traced run reports old_page_candidate_pages: 0 throughout.

Repro

benchmarks/gc_ratchet/probes/12_large_live_set.ts (in #7432), compiled with PERRY_NO_AUTO_OPTIMIZE=1; run with PERRY_GC_DIAG=1. The probe pins today's 105.6 MB deterministically (0% spread), so any improvement must deliberately re-pin.

Why it matters more after #7432

The adaptive tenuring threshold promotes long-lived cohorts on first copy (that is its job — re-copying them was 4.39 GB of waste on tree.ts), so old-gen now receives the scattered-survivor pattern wholesale. The tree.ts residual RSS gap vs PERRY_GC_SCAVENGE=0 (221 MB vs 103 MB) is this plus old-reclaim pacing (#SIBLING), not survivor re-copying.

Directions

  • Old-page defrag policy: why zero candidate pages here — thresholds, or the pattern is outside its selection criteria?
  • Sparse-block evacuation: blocks whose live fraction is < ~25% are exactly the C4b evacuation case; the survivors are unpinned class instances.
  • At minimum, heapUsed could report offsets minus free-listed holes so the metric distinguishes "mapped high-water" from "live".

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions