From b2d82b52a2eccddc1dd1eac5daebc282a7a8fc83 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ralph=20K=C3=BCpper?= Date: Sun, 2 Aug 2026 21:29:27 +0200 Subject: [PATCH] test(gc): gate matrix arm liveness, and put a relocating arm back in the PR subset (#7255) --- .github/workflows/gc-moving-witnesses.yml | 23 +- .github/workflows/test.yml | 14 + changelog.d/7275-gc-matrix-liveness-gate.md | 48 ++ docs/representation-selection-rfc.md | 30 +- scripts/gc_matrix_liveness_check.py | 468 ++++++++++++++++++++ scripts/gc_repsel_matrix.sh | 259 +++++++---- test-parity/gc_matrix_inert_arms.txt | 42 ++ 7 files changed, 774 insertions(+), 110 deletions(-) create mode 100644 changelog.d/7275-gc-matrix-liveness-gate.md create mode 100755 scripts/gc_matrix_liveness_check.py create mode 100644 test-parity/gc_matrix_inert_arms.txt diff --git a/.github/workflows/gc-moving-witnesses.yml b/.github/workflows/gc-moving-witnesses.yml index 70b4050c49..81098cf2eb 100644 --- a/.github/workflows/gc-moving-witnesses.yml +++ b/.github/workflows/gc-moving-witnesses.yml @@ -24,12 +24,23 @@ name: GC Moving Witnesses # reproducers. # # The witnesses themselves reach CI only through `gc-stress`, which runs the -# matrix's PR arm subset — `default,evac_minor,verify_evac,force_verify, -# cons_scan_off,shipped_default` — and not one of those arms compiles with the -# flag. So on every pull request these files were compiled into IR in which -# their bug is not expressible, then run to completion, green, proving nothing. -# They were reproductions, not gates — the fourth of CLAUDE.md's "four ways a -# gate can be unable to fail": the job runs, but its subject never does. +# matrix's PR arm subset, and when this job was written not one arm in that +# subset compiled with the flag. So on every pull request these files were +# compiled into IR in which their bug is not expressible, then run to +# completion, green, proving nothing. They were reproductions, not gates — the +# fourth of CLAUDE.md's "four ways a gate can be unable to fail": the job runs, +# but its subject never does. +# +# #7255 CHANGED HALF OF THAT AND DOES NOT REPLACE THIS JOB. The PR subset now +# carries `safepoint_minor`, which does compile and run with +# `PERRY_GC_MOVING_LOOP_POLLS=1`, so a witness is at least *expressible* on +# every PR. That arm is deliberately the UNFORCED route — polls only, no +# `PERRY_GC_INCREMENTAL=0`, no `PERRY_CONSERVATIVE_STACK_SCAN=off`, no +# `PERRY_GC_FORCE_EVACUATE=1` — because it is what the shipped default becomes +# the day #7161's stopgap lifts. `loop_polls` here stacks the evacuating base +# and forced evacuation on top of the same polls, and rejects `UNVER` per CELL +# rather than per arm. Strictly stronger, and still the only place the +# witnesses run that way. # # #7161 makes this sharper rather than softer. It flipped the evacuating minor # default-OFF as a stopgap for #7154, so `PERRY_GC_MOVING_LOOP_POLLS=1` is now diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 43182032d9..afe5630e64 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -205,6 +205,20 @@ jobs: python3 scripts/gc_gate_wiring_check.py --self-test python3 scripts/gc_gate_wiring_check.py + # The other half of the same hazard (#7255). `gc_gate_wiring_check.py` + # asserts the matrix JOB can run; this asserts the matrix ARMS can fail. + # Four of the six PR-gating arms sat at copy-minor 0/50 for five weeks + # while the script's header advertised 12/22, because an all-UNVER table + # exits 0. Both checks are build-free, so they belong in `lint`: + # `--self-test` covers the red/green rule itself, `--check-registry` + # covers the known-inert list (names resolve to real arms, every entry + # cites an issue, and the matrix still calls the checker at all — a gate + # nobody invokes is the same hazard one level up). + - name: GC matrix liveness gate + run: | + python3 scripts/gc_matrix_liveness_check.py --self-test + python3 scripts/gc_matrix_liveness_check.py --check-registry + # --------------------------------------------------------------------------- # Clippy — enforces the deny-level lints in [workspace.lints] (root # Cargo.toml). `cargo clippy` exits nonzero only on `deny` lints, so diff --git a/changelog.d/7275-gc-matrix-liveness-gate.md b/changelog.d/7275-gc-matrix-liveness-gate.md new file mode 100644 index 0000000000..d9b0f2368d --- /dev/null +++ b/changelog.d/7275-gc-matrix-liveness-gate.md @@ -0,0 +1,48 @@ +### Fixed + +**The GC stress matrix can now fail when an arm stops exercising the collector (#7255).** + +`scripts/gc_repsel_matrix.sh` classifies a cell as `UNVER` when the output matched +the Node oracle but the arm was measurably inert, and that classification is +correct. What was missing is that `UNVER` could not turn a run red: the exit +status counted only `FAIL`. An arm that went inert across the *whole* corpus +therefore produced a yellow table and exit 0, which is CLAUDE.md's fourth way a +gate can be unable to fail — the job runs, and its subject never does. + +That is not a hypothetical. #7024 measured the `default` arm at copy-minor +`0/22`, fixed it to `12/22`, and recorded the fix in the script header in its +strongest formatting. #7161 then flipped `PERRY_GC_MOVING_LOOP_POLLS` +default-OFF as a stopgap for #7154 — the gate for *both* halves of that route +(perry-codegen's `moving_safepoint_polls_enabled`, which decides whether +`js_gc_loop_safepoint` back-edge polls are emitted at all, and perry-runtime's +`gc_moving_loop_polls_enabled`, which decides whether the alloc-point nursery +trigger defers to them) — taking the same arm back to `0/50`. The header still +read `12/22`, because it was a hand-maintained number in a comment. Four of the +six PR-gating arms were inert, and #6982 and #7018 could not be judged in either +direction because both crash *inside* a copying minor that no longer ran. + +Three changes, in the order the issue asked for them. + +* **A relocating arm is back in the PR subset.** New arm `safepoint_minor` + compiles *and* runs with `PERRY_GC_MOVING_LOOP_POLLS=1` and nothing else — no + `PERRY_GC_INCREMENTAL=0`, no `PERRY_CONSERVATIVE_STACK_SCAN=off`, no forced + evacuation. That is the sound route: the copying minor runs at a loop + back-edge where roots are precise and rewritable, which is exactly what + `default` was between #7024 and #7161 and what it becomes again when the + stopgap lifts. It is in `PR_ARMS`. +* **The declarations are honest again, and self-healing.** `default`, + `verify_evac`, `cons_scan_off` and `cons_scan_off_force` keep + `requires=scavenge` — that is still what they are *for* — and are listed in the + new `test-parity/gc_matrix_inert_arms.txt` with the issue that blocks them. + Registering an arm is the only way to be inert without failing. +* **`scripts/gc_matrix_liveness_check.py` runs at the end of every matrix + invocation** and fails it when an unregistered arm satisfied its `requires=` on + zero cells — *and* when a registered arm starts biting again, so the registry + cannot rot the way the header did. It has `--self-test` and `--check-registry` + modes wired into `lint`, so the rule that decides red-versus-green is a tested + program rather than untested bash, and it is verified to be *wired* (the + registry check fails if the matrix stops calling it). + +No liveness number is written down anywhere any more. The per-arm table is +derived from the collector's own `PERRY_GC_TRACE` / `PERRY_GC_DIAG` output and +printed on every run. diff --git a/docs/representation-selection-rfc.md b/docs/representation-selection-rfc.md index 6dfddd513a..cece8c6607 100644 --- a/docs/representation-selection-rfc.md +++ b/docs/representation-selection-rfc.md @@ -221,8 +221,8 @@ one-time arguments: GC arm — `PERRY_GC_FORCE_EVACUATE`, `PERRY_GC_VERIFY_EVACUATION`, `PERRY_GEN_GC=0`, `PERRY_WRITE_BARRIERS=0`, `PERRY_CONSERVATIVE_STACK_SCAN=off`, `PERRY_GC_MOVING_LOOP_POLLS=1`, their combinations, and *each representation flag OFF x evacuation* — byte-exact against the - pinned Node oracle. Wired into the `gc-stress` CI job: a fast 4-arm subset gates every PR, the - full arm list runs on push. + pinned Node oracle. Wired into the `gc-stress` CI job: the `PR_ARMS` subset gates every PR, the + full arm list runs on push and on the nightly schedule. - **A NEW REPRESENTATION MUST REGISTER ITS GAP FILE** in `test-parity/gc_repsel_corpus.txt`. The script fails when a `test_gap_repsel_*` / `test_gap_specabi_*` file exists that is not registered. This is the GC-side counterpart of the single-decoder refactor #6910 established for mark/rewrite: @@ -235,13 +235,25 @@ one-time arguments: `test-files/test_gap_repsel_gc_stress.ts` is the corpus member built to be live: it holds each representation's local across escaping allocation churn heavy enough to reach the collector. A new representation should extend *that* file as well as adding its own, or its GC arms stay inert. -- **What the matrix cannot verify today.** No reachable configuration in an AOT-compiled program - performs an *evacuating minor*: every automatic collection is a full mark-sweep taken under - `ManualGcScanGuard::force_full_scan()`, which additionally pins raw locals conservatively (#6950, - extending #6946 from the `gc()` path). The rebase-after-safepoint contract in the bullets above — - the core GC claim of every pointer representation — is therefore still argued, not tested. #6942 - tracks making it testable; when that lands, the matrix's evacuating arms flip from UNVERIFIED to - green with no change to the harness. + **Since #7255 liveness is also a GATE, not only a label**: an arm that satisfied its declared + `requires=` on zero cells fails the run (`scripts/gc_matrix_liveness_check.py`), because an + all-UNVERIFIED table used to exit 0 — four of the six PR-gating arms were inert for five weeks + and nothing said so. Arms that are known-inert for a named reason live in + `test-parity/gc_matrix_inert_arms.txt`, and the same checker fails when one of them starts biting + again, so that list cannot outlive its cause either. +- **What the matrix cannot verify today.** This bullet used to say that no reachable configuration + performs an *evacuating minor*, so the rebase-after-safepoint contract — the core GC claim of + every pointer representation — was argued rather than tested. Two of its three clauses have since + moved. #6977 made the allocation-point collection reachable as a real copying minor under + `PERRY_GC_INCREMENTAL=0 PERRY_CONSERVATIVE_STACK_SCAN=off` (the matrix's `evac_minor`), and #7019 + / #7024 made the precise loop-back-edge safepoint reach it too (`safepoint_minor`), so the + contract IS tested on those arms. What remains untested is the **shipped default**: #7161 flipped + `PERRY_GC_MOVING_LOOP_POLLS` off pending #7154, and with it off a default binary emits no + back-edge polls and its alloc-point minor runs under `ManualGcScanGuard::force_full_scan()`, + which pins raw locals conservatively and makes the copying minor ineligible. That is why + `default`, `verify_evac`, `cons_scan_off` and `cons_scan_off_force` are listed in + `test-parity/gc_matrix_inert_arms.txt` rather than deleted — the claim is still theirs, it is + just blocked, and the liveness gate demands the list shrink when the stopgap lifts. ### 5.7 Typed heap (Phase 4) Unboxed storage extends to heap slots where the *container's* shape is proven and stable: diff --git a/scripts/gc_matrix_liveness_check.py b/scripts/gc_matrix_liveness_check.py new file mode 100755 index 0000000000..caadb67d51 --- /dev/null +++ b/scripts/gc_matrix_liveness_check.py @@ -0,0 +1,468 @@ +#!/usr/bin/env python3 +"""Assert every GC-matrix arm actually exercised the collector path it claims. + +WHY THIS EXISTS (#7255) +----------------------- +`scripts/gc_repsel_matrix.sh` classifies a cell as UNVER when the output matched +the oracle but the arm was measurably inert. UNVER is deliberately "not green". +It is also, deliberately, **not red** — the matrix's exit status counts only +FAIL. So an arm that stops biting corpus-wide produces a table of yellow cells +and exit 0, and nothing in CI says a word. + +That is not hypothetical. #7024 measured the `default` arm at copy-minor 0/22, +fixed it to 12/22, and the script's header recorded the fix in its strongest +formatting. #7161 then flipped `PERRY_GC_MOVING_LOOP_POLLS` default-OFF as a +stopgap for #7154, which took the same arm back to **0/49** — and the header +still read `12/22`, because it was a hand-maintained number. Four of the six +PR-gating arms were inert for weeks. Two open crash reports (#6982, #7018) could +not be judged in either direction, because both crash *inside* a copying minor +and the arms that were supposed to run one ran none. + +This is CLAUDE.md's fourth way a gate can be unable to fail, in its purest form: +the gate ran, and its subject never did. The rule it mechanises is CLAUDE.md's +own — *"a gate must assert its subject was live"*. + +WHAT IT ASSERTS +--------------- +For every arm in a matrix JSON report whose `requires` is not `none`: + + * **inert + unregistered → FAIL.** The arm satisfied its own liveness + requirement on ZERO cells and nothing in the tree says that is expected. + * **live + registered → FAIL.** `test-parity/gc_matrix_inert_arms.txt` lists + an arm as known-inert, and it just bit. The blocker named in the entry has + lifted; delete the entry (and, usually, promote the arm back into + `PR_ARMS`). A registry that can only be added to rots exactly the way the + header did. + +The registry is the *only* way to be inert without failing, every entry must +cite an issue number, and each entry's own text says what must become true for +it to be deleted. There is no hand-maintained count anywhere: the numbers are +derived from the run and printed on every invocation. + +USAGE +----- + python3 scripts/gc_matrix_liveness_check.py + python3 scripts/gc_matrix_liveness_check.py --check-registry # from `lint` + python3 scripts/gc_matrix_liveness_check.py --self-test # from `lint` +""" + +from __future__ import annotations + +import argparse +import json +import re +import sys +from pathlib import Path + +REPO_ROOT = Path(__file__).resolve().parent.parent +REGISTRY = REPO_ROOT / "test-parity" / "gc_matrix_inert_arms.txt" +MATRIX = REPO_ROOT / "scripts" / "gc_repsel_matrix.sh" + +# requires= value -> (human name, cell counter key(s) summed to decide "it bit") +REQUIREMENTS = { + "scavenge": ("copying young-gen minor", ("scavenged",)), + "move": ("any relocation", ("evacuated", "scavenged")), + "collect": ("any GC cycle", ("cycles",)), + "none": ("nothing (explicit control)", ()), +} + + +class Violation(Exception): + pass + + +def parse_registry(text: str) -> dict: + """`arm | #issue | reason` per line; `#` in column 1 starts a comment.""" + entries = {} + for lineno, raw in enumerate(text.splitlines(), 1): + line = raw.strip() + if not line or line.startswith("#"): + continue + parts = [p.strip() for p in line.split("|")] + if len(parts) != 3: + raise Violation( + "%s:%d: expected `arm | #issue | reason`, got %r" % (REGISTRY.name, lineno, raw) + ) + arm, issue, reason = parts + if arm in entries: + raise Violation("%s:%d: duplicate entry for arm %r" % (REGISTRY.name, lineno, arm)) + if not re.fullmatch(r"#\d+", issue): + raise Violation( + "%s:%d: the second field must be the blocking issue as `#NNNN`, got %r. " + "An entry with no issue is an arm nobody is going to turn back on." + % (REGISTRY.name, lineno, issue) + ) + if not reason: + raise Violation("%s:%d: empty reason" % (REGISTRY.name, lineno)) + entries[arm] = (issue, reason) + return entries + + +def matrix_arms(text: str) -> dict: + """{arm id: requires} as declared in `scripts/gc_repsel_matrix.sh`'s ARMS array.""" + body = re.search(r"^ARMS=\((.*?)^\)$", text, re.S | re.M) + if not body: + raise Violation("could not find the ARMS=( ... ) array in %s" % MATRIX.name) + arms = {} + for line in re.finditer(r'^"([^"]*)"$', body.group(1), re.M): + fields = line.group(1).split("|") + if len(fields) < 4: + raise Violation("malformed arm record: %r" % line.group(1)[:60]) + arms[fields[0]] = fields[3] + if not arms: + raise Violation("the ARMS=( ... ) array in %s parsed to nothing" % MATRIX.name) + return arms + + +def matrix_pr_arms(text: str) -> list: + """The `PR_ARMS=` subset — what `gc-stress` actually runs on a pull request.""" + match = re.search(r'^PR_ARMS="([^"]*)"', text, re.M) + if not match: + raise Violation("could not find PR_ARMS= in %s" % MATRIX.name) + return [a for a in match.group(1).split(",") if a] + + +def arm_liveness(cells: list, arm_id: str, requires: str) -> tuple: + """(live_cell_count, total_cell_count, summed counter) for one arm.""" + keys = REQUIREMENTS[requires][1] + live = total = summed = 0 + for cell in cells: + if cell.get("arm") != arm_id: + continue + total += 1 + value = sum(int(cell.get(k, 0) or 0) for k in keys) + summed += value + if value > 0: + live += 1 + return live, total, summed + + +def check_report(report: dict, registry: dict, out=sys.stdout) -> list: + """Returns a list of violation strings; prints the derived liveness table.""" + arms = report.get("arms") or [] + cells = report.get("cells") or [] + if not arms: + return ["the report declares no arms — the matrix never ran an arm"] + + violations = [] + out.write("liveness gate (#7255): every arm must have exercised what it claims\n") + for arm in arms: + arm_id, requires = arm["id"], arm["requires"] + if requires not in REQUIREMENTS: + violations.append( + "arm %s declares requires=%s, which is not one of %s" + % (arm_id, requires, "/".join(sorted(REQUIREMENTS))) + ) + continue + live, total, summed = arm_liveness(cells, arm_id, requires) + registered = registry.get(arm_id) + if requires == "none": + verdict = "n/a" + elif live == 0 and registered is None: + verdict = "INERT" + violations.append( + "arm %s declares requires=%s (%s) and satisfied it on 0/%d cells. " + "The arm is inert: every green cell under it is green for the wrong " + "reason. Either fix the arm, re-declare its requires=, or register it " + "in %s with the issue that blocks it." + % (arm_id, requires, REQUIREMENTS[requires][0], total, REGISTRY.name) + ) + elif live == 0: + verdict = "inert (known %s)" % registered[0] + elif registered is not None: + verdict = "STALE-REGISTRY" + violations.append( + "arm %s is listed as known-inert in %s (%s) but satisfied requires=%s " + "on %d/%d cells. The blocker has lifted — delete the entry, and put the " + "arm back where it belongs (usually PR_ARMS)." + % (arm_id, REGISTRY.name, registered[0], requires, live, total) + ) + else: + verdict = "live" + out.write( + " %-24s requires=%-9s live %2d/%-3d counter=%-12d %s\n" + % (arm_id, requires, live, total, summed, verdict) + ) + + exercised = {a["id"] for a in arms} + for arm_id, (issue, _) in sorted(registry.items()): + if arm_id not in exercised: + out.write(" %-24s (registered inert %s; not exercised by this run)\n" % (arm_id, issue)) + return violations + + +def check_registry(out=sys.stdout) -> list: + """Static checks that need no run: names resolve, and the gate is wired in.""" + violations = [] + try: + registry = parse_registry(REGISTRY.read_text()) if REGISTRY.exists() else {} + except Violation as exc: + return [str(exc)] + + matrix_text = MATRIX.read_text() + try: + known = matrix_arms(matrix_text) + pr_arms = matrix_pr_arms(matrix_text) + except Violation as exc: + return [str(exc)] + + for arm_id, (issue, _) in sorted(registry.items()): + if arm_id not in known: + violations.append( + "%s lists arm %r (%s), which no longer exists in %s. A registry entry " + "for a deleted arm hides nothing and outlives its reason." + % (REGISTRY.name, arm_id, issue, MATRIX.name) + ) + + for arm_id in pr_arms: + if arm_id not in known: + violations.append("PR_ARMS names %r, which is not in the ARMS table" % arm_id) + + # THE STRUCTURAL HALF OF #7255, and the reason this check is worth having in + # `lint` at all: the run-time gate can be satisfied by registering every + # inert arm, which would leave a PR subset that cannot relocate anything and + # still exits 0. So the subset must retain at least one arm that claims to + # relocate AND is not on the known-inert list. This is the property the + # matrix header used to assert in prose ("THIS SUBSET CAN NOW REPRODUCE THE + # RELOCATING-MINOR DEFECT CLASS") with a number that went stale. + relocating = [ + arm + for arm in pr_arms + if known.get(arm) in ("scavenge", "move") and arm not in registry + ] + if not relocating: + violations.append( + "PR_ARMS (%s) contains no arm that both claims to relocate " + "(requires=scavenge/move) and is absent from %s. On a pull request the " + "matrix would then be unable to observe a raw reference held across a " + "relocating collection at all — the #6993 class (#6951, #6972, #6982, " + "#6991, #6992), which is the hole #7255 was filed about. Registering " + "arms as known-inert is not a way to buy silence: put a live relocating " + "arm back in the subset." % (",".join(pr_arms), REGISTRY.name) + ) + + out.write( + "registry: %d known-inert arm(s), %d arms declared in %s\n" + % (len(registry), len(known), MATRIX.name) + ) + out.write( + "PR subset: %s\n relocating and not known-inert: %s\n" + % (",".join(pr_arms), ",".join(relocating) or "NONE") + ) + for arm_id, (issue, reason) in sorted(registry.items()): + out.write(" %-24s %-8s %s\n" % (arm_id, issue, reason[:100])) + return violations + + +def _report(arms, cells): + return {"arms": arms, "cells": cells} + + +def _cell(arm, **kw): + base = {"test": kw.pop("test", "t"), "arm": arm, "result": kw.pop("result", "PASS")} + base.update({"cycles": 0, "evacuated": 0, "scavenged": 0}) + base.update(kw) + return base + + +def self_test() -> int: + import io + + failures = [] + + def expect(name, violations, want): + got = len(violations) + if (got > 0) != want: + failures.append("%s: wanted %s, got %r" % (name, "a violation" if want else "clean", violations)) + + sink = io.StringIO() + + # An arm that never scavenged and is not registered is the #7255 defect. + expect( + "inert scavenge arm is rejected", + check_report( + _report([{"id": "default", "requires": "scavenge"}], [_cell("default", cycles=3)]), + {}, + sink, + ), + True, + ) + # ...and is accepted once registered, because the blocker is named. + expect( + "registered inert arm is accepted", + check_report( + _report([{"id": "default", "requires": "scavenge"}], [_cell("default", cycles=3)]), + {"default": ("#7161", "stopgap")}, + sink, + ), + False, + ) + # A registry entry that has outlived its cause must be loud, not silent. + expect( + "stale registry entry is rejected", + check_report( + _report([{"id": "default", "requires": "scavenge"}], [_cell("default", scavenged=17)]), + {"default": ("#7161", "stopgap")}, + sink, + ), + True, + ) + expect( + "live arm passes", + check_report( + _report([{"id": "evac", "requires": "scavenge"}], [_cell("evac", scavenged=17)]), + {}, + sink, + ), + False, + ) + # #7025: `move` may be satisfied by the C4b mark-sweep evacuation alone, but + # `scavenge` may NOT — that distinction is the whole point of two counters. + expect( + "move is satisfied by evacuated-only", + check_report( + _report([{"id": "m", "requires": "move"}], [_cell("m", evacuated=9)]), + {}, + sink, + ), + False, + ) + expect( + "scavenge is NOT satisfied by evacuated-only", + check_report( + _report([{"id": "s", "requires": "scavenge"}], [_cell("s", evacuated=9)]), + {}, + sink, + ), + True, + ) + expect( + "requires=none is never a violation", + check_report( + _report([{"id": "ctl", "requires": "none"}], [_cell("ctl")]), + {}, + sink, + ), + False, + ) + expect( + "collect arm needs a cycle", + check_report( + _report([{"id": "c", "requires": "collect"}], [_cell("c")]), + {}, + sink, + ), + True, + ) + expect( + "an unknown requires= value is rejected", + check_report( + _report([{"id": "x", "requires": "sometimes"}], [_cell("x")]), + {}, + sink, + ), + True, + ) + expect( + "an empty report is rejected", + check_report(_report([], []), {}, sink), + True, + ) + # A liveness verdict must be per-arm, not pooled: one live arm must not + # vouch for an inert one (the shape #7025 describes for counters). + expect( + "one live arm does not vouch for an inert one", + check_report( + _report( + [{"id": "live", "requires": "scavenge"}, {"id": "dead", "requires": "scavenge"}], + [_cell("live", scavenged=5), _cell("dead", cycles=5)], + ), + {}, + sink, + ), + True, + ) + + for text, want_ok in [ + ("arm | #123 | reason", True), + ("arm | 123 | reason", False), # issue number must be an issue number + ("arm | #123", False), # three fields + ("arm | #123 | a\narm | #124 | b", False), # duplicate + ("arm | #123 |", False), # empty reason + ("# just a comment\n\n", True), + ]: + try: + parse_registry(text) + ok = True + except Violation: + ok = False + if ok != want_ok: + failures.append("registry parse of %r: wanted ok=%s" % (text, want_ok)) + + # The parsers must survive the real file, or the registry check is vacuous — + # a regex that silently matches nothing is its own way for a gate to pass. + try: + text = MATRIX.read_text() + arms = matrix_arms(text) + pr_arms = matrix_pr_arms(text) + if len(arms) < 5 or arms.get("default") != "scavenge": + failures.append("matrix_arms parsed %r from the real matrix script" % (arms,)) + if "default" not in pr_arms: + failures.append("matrix_pr_arms parsed %r from the real matrix script" % (pr_arms,)) + if set(pr_arms) - set(arms): + failures.append("PR_ARMS names arms that are not in ARMS: %r" % (set(pr_arms) - set(arms),)) + except Violation as exc: + failures.append("parsing the real matrix script failed: %s" % exc) + + for failure in failures: + print("SELF-TEST FAIL: %s" % failure, file=sys.stderr) + print("self-test: %d checks, %d failures" % (12 + 6 + 3, len(failures))) + return 1 if failures else 0 + + +def main() -> int: + parser = argparse.ArgumentParser(description=__doc__.splitlines()[0]) + parser.add_argument("report", nargs="?", help="matrix JSON written by --json") + parser.add_argument("--check-registry", action="store_true") + parser.add_argument("--self-test", action="store_true") + parser.add_argument( + "--report-only", + action="store_true", + help="print the verdict but always exit 0 (local exploration; never in CI)", + ) + args = parser.parse_args() + + if args.self_test: + return self_test() + + if args.check_registry: + violations = check_registry() + elif args.report: + try: + registry = parse_registry(REGISTRY.read_text()) if REGISTRY.exists() else {} + except Violation as exc: + print("ERROR: %s" % exc, file=sys.stderr) + return 1 + with open(args.report) as handle: + report = json.load(handle) + violations = check_report(report, registry) + else: + parser.error("pass a report path, --check-registry, or --self-test") + + for violation in violations: + print("LIVENESS GATE: %s" % violation, file=sys.stderr) + if violations and not args.report_only: + print( + "\n%d arm(s) did not exercise the collector path they claim. See #7255: an " + "arm that never bit makes every green cell under it green for the wrong " + "reason." % len(violations), + file=sys.stderr, + ) + return 1 + if violations: + print("(--report-only: %d violation(s) not enforced)" % len(violations), file=sys.stderr) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/gc_repsel_matrix.sh b/scripts/gc_repsel_matrix.sh index c84d478e78..c92d89b46a 100755 --- a/scripts/gc_repsel_matrix.sh +++ b/scripts/gc_repsel_matrix.sh @@ -15,10 +15,11 @@ # # ***LIVENESS IS PART OF THE RESULT.*** # Setting a GC env var does not prove the GC did anything (#6942, #6946, #6950). -# Measured on main: the whole representation corpus performs ZERO collections -- -# the first-GC trigger needs ~1M escaping allocations and a gap test allocates a -# few KB. So every GC arm is INERT against it, and "passes under -# PERRY_GC_FORCE_EVACUATE=1" asserts nothing. +# Without the pressure knob a gap test allocates a few KB against a first-GC +# trigger that needs ~1M escaping allocations, so every GC arm is INERT against +# it and "passes under PERRY_GC_FORCE_EVACUATE=1" asserts nothing. That is what +# `--pressure` is for, and it is not sufficient on its own -- see the liveness +# table every run prints, and the known-inert registry it is checked against. # # Therefore every run is executed with `PERRY_GC_TRACE=1` (one `[gc] cycle` # marker per completed collection -- a sound cycle counter) and `PERRY_GC_DIAG=1` @@ -37,22 +38,39 @@ # confidence this gate exists to remove. # # `test_gap_repsel_gc_stress` is the corpus member deliberately built to be -# LIVE (measured: 2 collections at default settings, 11 under `--pressure 8`). -# If a collector change stops it collecting, its cells go UNVER and the arm -# liveness summary shows 0/N -- that is the signal to re-tune its churn budget. +# LIVE: it collects even in the shipped configuration with no pressure knob at +# all, which almost nothing else in the corpus does. If a collector change stops +# it collecting, its cells go UNVER, the arm liveness summary drops, and the +# liveness gate fails -- that is the signal to re-tune its churn budget. (Its +# per-run cycle counts belong in the run's own output, not here.) # # ADDING A REPRESENTATION: register its gap file in # test-parity/gc_repsel_corpus.txt. This script FAILS if a `test_gap_repsel_*` # or `test_gap_specabi_*` file exists that is not registered (see # docs/representation-selection-rfc.md 5.6). # +# ***AND LIVENESS IS NOW GATED, NOT MERELY REPORTED (#7255).*** +# UNVER was "not green" but it was never red either: the exit status counts only +# FAIL, so an arm that went inert across the WHOLE corpus produced a yellow table +# and exit 0. Every run therefore ends in +# `scripts/gc_matrix_liveness_check.py`, which fails when an arm satisfied its +# own `requires=` on zero cells — and equally when an arm listed in +# `test-parity/gc_matrix_inert_arms.txt` starts biting again, so the registry +# cannot rot in the other direction either. +# +# ***DO NOT PUT LIVENESS NUMBERS IN THIS HEADER.*** The last hand-maintained +# pair (`default: 0/22 -> 12/22` after #7024) read as settled fact for five weeks +# after #7161 took that same arm back to 0/49, and it is the reason nobody +# re-derived it (#7255). The per-arm table printed by every run is the only place +# those numbers are true, and it is derived from the collector's own output. +# # Portable to bash 3.2 (macOS system bash): no associative arrays, no mapfile. # # Usage: # scripts/gc_repsel_matrix.sh [--arms pr|all|] [--filter ] # [--pressure ] [--jobs N] [--no-build] # [--profile ] [--json ] -# [--list-arms] +# [--list-arms] [--liveness-report-only] set -uo pipefail SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" @@ -68,6 +86,7 @@ JOBS="$(sysctl -n hw.ncpu 2>/dev/null || nproc 2>/dev/null || echo 4)" DO_BUILD=1 JSON_OUT="" PROFILE="release" +LIVENESS_REPORT_ONLY=0 while [ $# -gt 0 ]; do case "$1" in @@ -79,7 +98,11 @@ while [ $# -gt 0 ]; do --no-build) DO_BUILD=0; shift ;; --json) JSON_OUT="$2"; shift 2 ;; --list-arms) ARMS_SEL="__list__"; shift ;; - -h|--help) sed -n '1,56p' "$0"; exit 0 ;; + # Local exploration only (e.g. a `--filter` narrow enough that an arm + # legitimately has nothing to bite). CI never passes this: the whole + # point of #7255 is that an inert arm must be able to turn a run red. + --liveness-report-only) LIVENESS_REPORT_ONLY=1; shift ;; + -h|--help) sed -n '1,70p' "$0"; exit 0 ;; *) echo "unknown flag: $1" >&2; exit 2 ;; esac done @@ -129,53 +152,53 @@ RED=$'\033[0;31m'; GREEN=$'\033[0;32m'; YELLOW=$'\033[0;33m'; NC=$'\033[0m' # what turns the automatic collection into a precise-rooted copying minor # that actually relocates survivors. # -# Measured on this pair (`--pressure 8`, arm `evac_minor`): -# test_gap_repsel_canonical_i32 0 cycles -> 1 cycle, 4 579 objects copied -# test_gap_repsel_ptr_shape_locals 0 cycles -> 1 cycle, 4 640 objects copied -# test_gap_repsel_gc_stress 24 cycles -> 31 cycles, 1 556 543 copied -# every one with `[gc-copy-minor] eligible=true fallback=none`. +# Every %E% arm reports `[gc-copy-minor] eligible=true fallback=none`; the +# per-run liveness table at the bottom of the output says how many cells that +# was, on the tree you are actually running. (There used to be a hard-coded +# before/after table here. It was true when written and false a month later -- +# see the DO NOT PUT LIVENESS NUMBERS IN THIS HEADER note above, and #7255.) # # NOTE this is a MEASUREMENT configuration, not the shipped one. It says the # collector's evacuating path is exercised; it does not say the shipped default # reaches that path. # -# ***AS OF #7024 THE SHIPPED DEFAULT DOES REACH IT*** -- by the sound route, -# which is not this one. `default` (pressure knob only, no GC env) now defers -# the alloc-point trigger to a precise-root safepoint and runs the copying -# minor there. Measured on this corpus at `--pressure 8`: -# -# arm copy-minor before #7024 after -# default 0/22 12/22 -# verify_evac 0/22 12/22 +# THE SHIPPED DEFAULT DOES NOT REACH IT TODAY. #7019/#7024 made it reach it by +# the sound route -- defer the alloc-point trigger to a precise-root safepoint +# and run the copying minor there -- and #7161 then turned that route off by +# default, pending #7154. So the WHERE distinction still stands and still +# matters (a safepoint has an unwound JS stack and roots precise by +# construction; %E% forces relocation at the register-imprecise allocation +# point, which is the only place an unrooted runtime-side local is exposed), +# but the arm that carries the safepoint route is `safepoint_minor`, which opts +# the polls back in at compile AND run time. `default` is registered known-inert +# in test-parity/gc_matrix_inert_arms.txt until the stopgap lifts. # -# The difference between `default` and %E% is now WHERE the relocation happens: -# `default` relocates at a real safepoint (the JS stack has unwound, roots are -# precise by construction), %E% forces it at the register-imprecise allocation -# point. Both belong in the matrix; only the first is a configuration anyone -# ships. -# -# ***AND IT IS RED.*** The first `--arms all` run in which anything actually -# moved failed 14 of the 20 corpus files: 5 crashes and 9 output mismatches -# (#6981), plus one intermittent SIGSEGV that does not even need precise roots -# (#6982). The discriminator is NOT relocation -- with the conservative stack -# scan still on, the same evacuating cycles pass 19/20 while copying thousands -# of objects. It is precise roots: the values only the conservative scan was -# keeping alive. That is the finding this gate was built to produce, and the -# arms stay configured to keep producing it. Do not quiet them down. +# ***AND WHEN THESE ARMS FIRST MOVED, THEY WERE RED.*** The first `--arms all` +# run in which anything actually moved failed 14 of the 20 corpus files then in +# the corpus: 5 crashes and 9 output mismatches (#6981), plus one intermittent +# SIGSEGV that does not even need precise roots (#6982). The discriminator is +# NOT relocation -- with the conservative stack scan still on, the same +# evacuating cycles passed 19/20 while copying thousands of objects. It is +# precise roots: the values only the conservative scan was keeping alive. That +# is the finding this gate was built to produce, and the arms stay configured to +# keep producing it. Do not quiet them down -- and note that "quiet" now has a +# second, cheaper failure mode than lowering a requires=: letting an arm go +# inert. The liveness gate exists because that one is invisible on screen. # --------------------------------------------------------------------------- ARMS=( -"default||%P%|scavenge|as-shipped GC configuration under allocation pressure. Since #7024 this is a RELOCATING arm with no env override at all beyond the pressure knob: the alloc-point trigger defers to js_gc_loop_safepoint -> gc_safepoint_moving_minor, which runs the copying minor on precise, rewritable roots. requires=scavenge, not collect: before #7024 it collected on 13/22 rows while running ZERO copying minors, so a collect requirement certified the pre-#7019 non-moving path under a name that says otherwise." -"evac_minor||%P% %E%|move|THE evacuating arm: the automatic alloc-point collection as a precise-rooted COPYING minor that relocates survivors. No stress knob -- this is the collector's own moving path." +"default||%P%|scavenge|as-shipped GC configuration under allocation pressure. ***INERT AT THE MOMENT, AND REGISTERED AS SUCH*** in test-parity/gc_matrix_inert_arms.txt. #7024 made this a relocating arm (the alloc-point trigger defers to js_gc_loop_safepoint -> gc_safepoint_moving_minor, which runs the copying minor on precise rewritable roots); #7161 then flipped PERRY_GC_MOVING_LOOP_POLLS default-OFF pending #7154, and that one env gates BOTH halves of the route -- perry-codegen's moving_safepoint_polls_enabled decides whether the back-edge polls are emitted at all, and perry-runtime's gc_moving_loop_polls_enabled decides whether the trigger defers to them. A default binary has neither, so the minor runs behind ManualGcScanGuard::force_full_scan and the copying minor is ineligible by construction. requires=scavenge STAYS: it is what the shipped default is FOR, the registry entry names what blocks it, and the liveness gate fails the day it scavenges again so the entry cannot outlive its cause. safepoint_minor carries the relocating claim meanwhile." +"safepoint_minor|PERRY_GC_MOVING_LOOP_POLLS=1|%P% PERRY_GC_MOVING_LOOP_POLLS=1|scavenge|THE SOUND RELOCATING ARM, and what keeps the #6993 defect class reachable per-PR while #7161's stopgap holds. Sets the poll flag at BOTH compile and run time (same env on both sides -- keyed into the object cache as env_gc_moving_loop_polls, so a warm cache cannot serve poll-free objects). The copying minor then runs at js_gc_loop_safepoint -> gc_safepoint_moving_minor, where the loop body has completed and every live heap value is a named local on the shadow stack: precise, rewritable roots. No %E%, no force -- this is exactly what default was between #7024 and #7161, and what default becomes again when the stopgap lifts. NOT a replacement for the %E% arms: a back-edge poll only fires while user JS runs, so it cannot expose an unrooted local inside runtime code that never re-enters user JS (#7249). evac_minor and force_verify remain in the PR subset for that." +"evac_minor||%P% %E%|move|THE evacuating arm, and the STRONGER acceptance route (#7249): the automatic alloc-point collection as a COPYING minor that relocates survivors at a register-imprecise point, which is where an unrooted runtime-side local is exposed. No stress knob -- this is the collector's own moving path." "force_evac||%P% %E% PERRY_GC_FORCE_EVACUATE=1|move|stress-copy every marked non-pinned nursery object" -"verify_evac||%P% PERRY_GC_VERIFY_EVACUATION=1|scavenge|panic if a live slot still points at a forwarded object. requires=scavenge: a verifier that runs over zero relocations verifies nothing, which is what it did on all 22 rows before #7024." +"verify_evac||%P% PERRY_GC_VERIFY_EVACUATION=1|scavenge|panic if a live slot still points at a forwarded object. requires=scavenge: a verifier that runs over zero relocations verifies nothing. REGISTERED KNOWN-INERT (#7161) -- same route as default, same blocker, and the same reason the declaration is not being weakened to hide it." "force_verify||%P% %E% PERRY_GC_FORCE_EVACUATE=1 PERRY_GC_VERIFY_EVACUATION=1|move|force + verify" "gen_gc_off||%P% PERRY_GEN_GC=0|collect|full mark-sweep only; no nursery => no evacuation by construction" "wb_off|PERRY_WRITE_BARRIERS=0|%P% PERRY_WRITE_BARRIERS=0|collect|no codegen write barriers => copying nursery ineligible by construction" "gen_off_verify||%P% PERRY_GEN_GC=0 PERRY_GC_VERIFY_EVACUATION=1|collect|full mark-sweep + evacuation verifier" "wb_off_force|PERRY_WRITE_BARRIERS=0|%P% PERRY_WRITE_BARRIERS=0 PERRY_GC_FORCE_EVACUATE=1|collect|force-evacuate is a documented no-op without barriers (barriers_inactive)" "all_four|PERRY_WRITE_BARRIERS=0|%P% PERRY_GEN_GC=0 PERRY_WRITE_BARRIERS=0 PERRY_GC_FORCE_EVACUATE=1 PERRY_GC_VERIFY_EVACUATION=1|collect|every escape hatch at once" -"cons_scan_off||%P% PERRY_CONSERVATIVE_STACK_SCAN=off|scavenge|PRECISE ROOTS ONLY -- removes the conservative-stack pinning that the alloc-point fallback otherwise forces (ManualGcScanGuard::force_full_scan). An arm that can observe a missing shadow-slot binding." -"cons_scan_off_force||%P% PERRY_CONSERVATIVE_STACK_SCAN=off PERRY_GC_FORCE_EVACUATE=1 PERRY_GC_VERIFY_EVACUATION=1|scavenge|precise roots + force/verify evacuation" +"cons_scan_off||%P% PERRY_CONSERVATIVE_STACK_SCAN=off|scavenge|PRECISE ROOTS ONLY -- removes the conservative-stack pinning that the alloc-point fallback otherwise forces (ManualGcScanGuard::force_full_scan). An arm that can observe a missing shadow-slot binding. REGISTERED KNOWN-INERT (#7161): precise roots beat that guard, but with the incremental stepper at its default the nursery trigger never reaches the direct arm in the first place -- registered_root_scanners_block_budgeted_gc() reduces to 'any copy-only scanner' under gc_incremental_enabled(), a compiled program has none, so the trigger goes to the budgeted stepper, which is non-moving by construction. Adding PERRY_GC_INCREMENTAL=0 is what turns it live, and that arm is evac_minor." +"cons_scan_off_force||%P% PERRY_CONSERVATIVE_STACK_SCAN=off PERRY_GC_FORCE_EVACUATE=1 PERRY_GC_VERIFY_EVACUATION=1|scavenge|precise roots + force/verify evacuation. REGISTERED KNOWN-INERT (#7161), same reason as cons_scan_off: PERRY_GC_FORCE_EVACUATE is read on a minor path this arm never reaches, which is the #6942/#6946 shape exactly." "loop_polls|PERRY_GC_MOVING_LOOP_POLLS=1|%P% %E% PERRY_GC_MOVING_LOOP_POLLS=1 PERRY_GC_FORCE_EVACUATE=1|move|defer the alloc-point collection to a loop back-edge precise-root safepoint, where the copying minor may MOVE survivors" "rep_i32_off|PERRY_CANONICAL_I32_LOCALS=0|%P% %E% PERRY_GC_FORCE_EVACUATE=1|move|repsel Phase 1 OFF x evacuation" "rep_str_off|PERRY_CANONICAL_STR_LOCALS=0|%P% %E% PERRY_GC_FORCE_EVACUATE=1|move|repsel Phase 3a OFF x evacuation" @@ -187,41 +210,51 @@ ARMS=( "shipped_default||-|none|control: exactly the as-shipped configuration -- no pressure knob, no GC env at all" ) -# PR-gating subset: the arms with the most detection power per second -- -# as-shipped under pressure, the evacuation verifier, precise-roots-only, and -# the untouched shipped configuration as a control. +# PR-gating subset: the arms with the most detection power per second -- the +# shipped configuration under pressure, the two routes that actually relocate, +# the evacuation verifier, precise-roots-only, and the untouched shipped +# configuration as a control. +# +# WHAT THIS SUBSET MUST BE ABLE TO DO, AND HOW THAT IS ENFORCED +# ------------------------------------------------------------ +# It must be able to reproduce the relocating-minor defect class (#6993: #6951, +# #6972, #6982, #6991, #6992 -- "a raw reference held across a relocating +# collection"). Before #7024 it could not, and the previous revision of this +# comment recorded the fix in bold, with the measurement that justified it: +# `default` had gone from copy-minor 0/22 to 12/22. # -# ***THIS SUBSET CAN NOW REPRODUCE THE RELOCATING-MINOR DEFECT CLASS (#6993).*** -# Until #7024 it could not, and that was the hole: `default` and `verify_evac` -# ran the NON-moving alloc-point minor (the deferral to the precise-root -# safepoint was unreachable whenever the pressure knob was set, because the -# deferral cap and the trigger ceiling shared a formula), and `cons_scan_off` -# relocated only with incremental mode still on. So the whole "raw reference -# held across a relocating collection" class -- #6951, #6972, #6982, #6991, -# #6992 -- was invisible per PR and could only go red after merge, on push. +# ***THAT SENTENCE OUTLIVED ITS MEASUREMENT BY FIVE WEEKS (#7255).*** #7161 +# flipped PERRY_GC_MOVING_LOOP_POLLS default-OFF pending #7154, which took +# `default` -- and `verify_evac`, `cons_scan_off`, `cons_scan_off_force` -- back +# to copy-minor 0 across the whole corpus. The comment still said 12/22, and +# because an all-UNVER table exits 0, nothing else said anything. Two open crash +# reports (#6982, #7018) sat un-judgeable for the duration because both fail +# INSIDE a copying minor and the arms meant to run one ran none. # -# The proof that the hole is closed is a cell that changed colour, not an -# argument: `default x test_gap_repsel_p4a3_numarray_barriers` was PASS -# (`cycles=1 scavenged=0` -- it collected, and relocated nothing) and is now -# FAIL (`exit=139 scavenged=3594`) -- the same SIGSEGV that only `cons_scan_off` -# and the %E% arms could produce before. #6981's redness now reaches the arm -# named after the shipped configuration. +# So the property is no longer asserted in prose here. It is enforced, twice: # -# `evac_minor` AND `force_verify` ARE BACK IN, as the previous revision of this -# comment instructed. They were held out only while #6981 was red. It is fixed: -# the memoized `Array.prototype` address is a raw pointer to a MOVABLE object, -# so a relocation (`js_array_grow` or the copying minor) left the hole-read -# fallback's `proto != receiver` self-recursion guard comparing a from-space -# address against a forwarding-resolved one; the guard stopped firing and the -# mutator recursed until the stack guard page. Measured on the fix, -# `--arms all --pressure 8`: PASS=339 UNVER=100 XFAIL=1 FAIL=0 over 440 cells, -# with both arms at copy-minor 21/22 (was PASS=325 … FAIL=14). So "a -# representation regressed GC correctness under relocation" is now a per-PR -# signal on the arms that actually relocate, which is the whole reason this -# matrix exists. The single XFAIL is the pre-existing -# `repsel_ptr_shape_locals x rep_ptr_shape_off` entry (#6976), not in this -# subset. -PR_ARMS="default,evac_minor,verify_evac,force_verify,cons_scan_off,shipped_default" +# * scripts/gc_matrix_liveness_check.py --check-registry (from `lint`, no +# build) fails unless PR_ARMS contains at least one arm that claims to +# relocate AND is not on the known-inert list. Registering every inert arm +# is therefore not a way to buy a green subset. +# * the same checker, run against every matrix invocation, fails when any arm +# satisfied its own requires= on zero cells. +# +# WHY THE RELOCATING ARMS ARE THE ONES THEY ARE +# --------------------------------------------- +# `safepoint_minor` is the SOUND route (precise roots at a loop back-edge) and +# is what `default` becomes again when #7161's stopgap lifts. `evac_minor` and +# `force_verify` are the STRONGER acceptance route (#7249): a back-edge poll +# only fires while user JS runs, so it cannot expose an unrooted local inside +# runtime code that never re-enters user JS -- the register-imprecise +# allocation point can. Both kinds are in, deliberately; neither substitutes for +# the other. +# +# `default`, `verify_evac` and `cons_scan_off` STAY in the subset while they are +# registered known-inert. They still verify byte-exactness against the oracle +# under a collecting GC, they cost one run each, and leaving them in is what +# makes the registry entry visible on every PR instead of quietly true. +PR_ARMS="default,safepoint_minor,evac_minor,verify_evac,force_verify,cons_scan_off,shipped_default" arm_field() { # $1 = arm record, $2 = 1..5 printf '%s' "$1" | cut -d'|' -f"$2" @@ -381,7 +414,7 @@ triage_reason() { # $1 test, $2 arm END { exit(found ? 0 : 1) }' } -CELLS=(); EVID=() +CELLS=(); EVID=(); CYC=(); EVA=(); SCA=() n_pass=0; n_unver=0; n_fail=0; n_xfail=0 ai=0 while [ "$ai" -lt "$NARMS" ]; do @@ -392,6 +425,7 @@ while [ "$ai" -lt "$NARMS" ]; do ti=0 while [ "$ti" -lt "${#CORPUS[@]}" ]; do b="${CORPUS[$ti]}"; bin="$WORK/bin/$slug/$b"; idx=$((ti*NARMS+ai)) + cycles=0; evacuated=0; scavenged=0 if [ ! -x "$bin" ]; then result="FAIL"; ev="compile-failed" else @@ -450,6 +484,9 @@ while [ "$ai" -lt "$NARMS" ]; do fi fi CELLS[$idx]="$result"; EVID[$idx]="$ev" + # The liveness gate reads these as NUMBERS, not by re-parsing `$ev`: + # a triage reason is free text and has already contained `=`. + CYC[$idx]="$cycles"; EVA[$idx]="$evacuated"; SCA[$idx]="$scavenged" case "$result" in PASS) n_pass=$((n_pass+1)) ;; UNVER) n_unver=$((n_unver+1)) ;; @@ -527,32 +564,64 @@ echo "summary: PASS=$n_pass UNVER=$n_unver XFAIL=$n_xfail FAIL=$n_fail" echo " (pressure=${PRESSURE_MB}MB, node $NODE_V, $PERRY_BIN)" echo " UNVER = output matched but the arm was inert here; see #6942 / #6946 / #6950." -if [ -n "$JSON_OUT" ]; then - { - printf '{"node":"%s","pressure_mb":"%s","arms":[' "$NODE_V" "$PRESSURE_MB" +# The report is written UNCONDITIONALLY -- the liveness gate below consumes it, +# so `--json` only decides whether a copy is kept where the caller asked for it. +# A gate that runs only when someone remembered a flag is not a gate. +JSON_REPORT="${JSON_OUT:-$WORK/matrix.json}" +{ + printf '{"node":"%s","pressure_mb":"%s","arms":[' "$NODE_V" "$PRESSURE_MB" + ai=0 + while [ "$ai" -lt "$NARMS" ]; do + [ "$ai" = 0 ] || printf ',' + printf '{"id":"%s","requires":"%s"}' "${ARM_IDS[$ai]}" "${ARM_LIVES[$ai]}" + ai=$((ai+1)) + done + printf '],"cells":[' + first=1; ti=0 + while [ "$ti" -lt "${#CORPUS[@]}" ]; do ai=0 while [ "$ai" -lt "$NARMS" ]; do - [ "$ai" = 0 ] || printf ',' - printf '{"id":"%s","requires":"%s"}' "${ARM_IDS[$ai]}" "${ARM_LIVES[$ai]}" + [ "$first" = 1 ] || printf ','; first=0 + idx=$((ti*NARMS+ai)) + # Evidence is free text (triage reasons quote code); strip the two + # characters that would make this invalid JSON, so a malformed + # report can never be the reason the gate fails. + ev_json="$(printf '%s' "${EVID[$idx]:-}" | tr '"\\' "''")" + printf '{"test":"%s","arm":"%s","result":"%s","cycles":%d,"evacuated":%d,"scavenged":%d,"evidence":"%s"}' \ + "${CORPUS[$ti]}" "${ARM_IDS[$ai]}" "${CELLS[$idx]:-?}" \ + "${CYC[$idx]:-0}" "${EVA[$idx]:-0}" "${SCA[$idx]:-0}" "$ev_json" ai=$((ai+1)) done - printf '],"cells":[' - first=1; ti=0 - while [ "$ti" -lt "${#CORPUS[@]}" ]; do - ai=0 - while [ "$ai" -lt "$NARMS" ]; do - [ "$first" = 1 ] || printf ','; first=0 - printf '{"test":"%s","arm":"%s","result":"%s","evidence":"%s"}' \ - "${CORPUS[$ti]}" "${ARM_IDS[$ai]}" "${CELLS[$((ti*NARMS+ai))]:-?}" "${EVID[$((ti*NARMS+ai))]:-}" - ai=$((ai+1)) - done - ti=$((ti+1)) - done - printf '],"summary":{"pass":%d,"unverified":%d,"xfail":%d,"fail":%d}}\n' \ - "$n_pass" "$n_unver" "$n_xfail" "$n_fail" - } > "$JSON_OUT" - echo "json: $JSON_OUT" + ti=$((ti+1)) + done + printf '],"summary":{"pass":%d,"unverified":%d,"xfail":%d,"fail":%d}}\n' \ + "$n_pass" "$n_unver" "$n_xfail" "$n_fail" +} > "$JSON_REPORT" +[ -n "$JSON_OUT" ] && echo "json: $JSON_OUT" + +# --------------------------------------------------------------------------- +# LIVENESS GATE (#7255). Half of this script's exit status. +# +# `n_fail` alone cannot express "the arm never bit": an all-UNVER table is +# yellow on screen and exit 0 in CI, which is how four of the six PR-gating arms +# sat at copy-minor 0/49 for five weeks while this file's header advertised +# 12/22. The checker is a separate, self-testable program (it runs `--self-test` +# and `--check-registry` from `lint`, where no build is needed) so the rule that +# decides red-vs-green is not 30 lines of untested bash. +# --------------------------------------------------------------------------- +echo +liveness_rc=0 +if command -v python3 > /dev/null 2>&1; then + liveness_args="" + [ "$LIVENESS_REPORT_ONLY" = 1 ] && liveness_args="--report-only" + # shellcheck disable=SC2086 + python3 "$SCRIPT_DIR/gc_matrix_liveness_check.py" $liveness_args "$JSON_REPORT" || liveness_rc=1 +else + echo "${RED}LIVENESS GATE SKIPPED${NC}: python3 not on PATH." >&2 + echo " This is a gate, not a report. Refusing to claim a green run without it." >&2 + liveness_rc=1 fi [ "$n_fail" = 0 ] || exit 1 +[ "$liveness_rc" = 0 ] || exit 1 exit 0 diff --git a/test-parity/gc_matrix_inert_arms.txt b/test-parity/gc_matrix_inert_arms.txt new file mode 100644 index 0000000000..bee95645fd --- /dev/null +++ b/test-parity/gc_matrix_inert_arms.txt @@ -0,0 +1,42 @@ +# Known-inert arms of scripts/gc_repsel_matrix.sh (#7255). +# +# Format: | <#issue> | +# +# An arm listed here is allowed to satisfy its `requires=` on ZERO cells without +# failing the run. Anything NOT listed here that goes inert corpus-wide fails it +# (scripts/gc_matrix_liveness_check.py). So does an arm listed here that starts +# biting again: the entry is then stale, and the checker says so. Both directions +# matter — this file exists because the one-directional version of it (a prose +# paragraph in the matrix header quoting `default: 0/22 -> 12/22`) stayed +# authoritative for five weeks after the measurement it quoted became false. +# +# ***AN ENTRY HERE IS A LOSS OF COVERAGE, NOT A WAIVER.*** The arms below are +# the ones that certify the shipped configuration relocates. While they are +# listed, that property is NOT tested per-PR by them, and only `safepoint_minor` +# (the sound route) and the %E% arms carry the relocating-minor defect class +# (#6993: #6951, #6972, #6982, #6991, #6992). Every entry must name an open +# issue and say what must become true for it to be deleted. +# +# DO NOT add an entry to make a table green. An arm that went quiet without +# anyone deciding it should is the defect this file was created to surface. + +# --- #7161's stopgap: the shipped default's minor is NON-MOVING ------------- +# +# #7019 made the evacuating young-gen minor the default and #7024 made it +# reachable under the pressure knob; `PERRY_GC_MOVING_LOOP_POLLS` is the gate for +# BOTH halves of that route — perry-codegen's `moving_safepoint_polls_enabled` +# decides whether `js_gc_loop_safepoint` back-edge polls are emitted at all, and +# perry-runtime's `gc_moving_loop_polls_enabled` decides whether the alloc-point +# nursery trigger defers to them. #7161 flipped it default-OFF pending #7154 +# (a use-after-free that corrupts the heap in the DEFAULT config). #7154 itself +# is closed, but its last measurement says the stopgap stays until the +# from-space protector is clean, so a default binary has no back-edge polls and +# its alloc-point minor runs behind a forced conservative scan, which makes the +# copying minor ineligible by construction. +# +# DELETE ALL FOUR when `PERRY_GC_MOVING_LOOP_POLLS` goes default-ON again. The +# liveness gate will demand it: the entries go stale the moment the arms bite. +default | #7161 | Shipped config + pressure. No back-edge polls are emitted into a default binary and the runtime does not defer to them, so the #7024 route (alloc-point trigger -> js_gc_loop_safepoint -> gc_safepoint_moving_minor) does not exist. It still COLLECTS; it cannot scavenge. `safepoint_minor` carries the relocating claim while this is listed. +verify_evac | #7161 | Same route as `default`, plus PERRY_GC_VERIFY_EVACUATION=1. A verifier that runs over zero relocations verifies nothing — which is exactly what it does today. +cons_scan_off | #7161 | Precise-roots-only (PERRY_CONSERVATIVE_STACK_SCAN=off) beats the alloc-point `force_full_scan`, but with the incremental stepper at its default the nursery trigger never reaches the direct arm: `registered_root_scanners_block_budgeted_gc()` reduces to "any copy-only scanner" under `gc_incremental_enabled()`, and a compiled program has none, so the trigger is handed to the budgeted stepper, which is non-moving by construction (`low_pause_non_moving = is_budgeted()`). With polls off there is no deferral to a safepoint either. Adding PERRY_GC_INCREMENTAL=0 is what turns it live — that arm is `evac_minor`. +cons_scan_off_force | #7161 | Same as `cons_scan_off`, plus force/verify. PERRY_GC_FORCE_EVACUATE is read on the minor path the arm never reaches (#6942/#6946), so the force knob does not rescue it.