public-baseline freshness gate: 17 trigger paths for a 2-hour regeneration is not sustainable — narrow the fingerprint to what can change a measured number

[proggeramlug]

The public-baseline freshness gate costs ~2 hours on a specific, quiet, correctly-pinned host and is triggered by 17 paths, several of which change for reasons unrelated to what the benchmark measures. That ratio is not sustainable, and the consequences are already on the record.

What it actually fingerprints today

SOURCE_PATHS  = ("Cargo.toml",
                 "benchmarks/suite/*.ts",
                 "benchmarks/json_polyglot/*.ts",
                 "benchmarks/app-patterns/kernels/*.ts")
HARNESS_PATHS = (13 entries: every harness script, plus the honest_bench
                 workloads/ scripts/ harness/ directories and expected.json)

Why this is unsustainable

Cargo.toml is the worst offender. It changes on every new perry-ext-* crate, every dependency bump, every workspace restructure — none of which alter the measured kernels. #6758/#6761's restructuring tripped this gate with the .ts kernels untouched, and the gate then sat red on every PR and nightly from ~2026-07-25.

Harness scripts are the second. #7265 fixed error handling in benchmarks/json_polyglot/run.sh — a script that printed a success line and exited 0 while writing nothing. That fix cannot change a single measured number, yet it invalidates the artifact because run.sh is in HARNESS_PATHS.

The observed cost: the artifact went 40+ commits stale. Because lint is a required status check and the freshness step runs at position 9, every subsequent step (file-size, GC store-site inventory, addr-class audit, and #7253's new gate-wiring check) never executed in CI at all — they had to be reproduced locally to even be seen. And every merge required --admin bypass, which makes branch protection theatre. A permanently-red required gate does not protect anything; it trains everyone to route around it.

What the gate is actually for

Published performance claims must describe the code they claim to describe. So the fingerprint should cover what could change a measured number:

• the benchmark kernels themselves ✅ (already covered, correctly)
• the build profile — opt-level, lto, codegen-units, panic (a profile change absolutely invalidates comparability)
• the pinned toolchain versions used in the comparison (node/bun/zig)
• harness content that sets run parameters — run counts, warmup, workload sizes

It should not cover: workspace membership, dependency additions unrelated to benchmarked code, harness error handling, logging, or output formatting.

Proposal

1. Replace whole-file Cargo.toml with a targeted extract. Fingerprint the [profile.*] tables and the workspace toolchain/version pins, not the file. This is the single biggest win and removes the most common false trigger.
2. Split HARNESS_PATHS into measurement-affecting vs plumbing. Run counts, warmup, workload sizing and expected.json stay fingerprinted; error handling, logging and output formatting do not. If a clean split is impractical per-file, extract the run parameters into one small fingerprinted config and let the scripts read it.
3. Keep the gate required, and keep it hard-failing. The problem is trigger breadth, not strictness. A narrower fingerprint that fires rarely and blocks absolutely is strictly better than a broad one everyone bypasses.

Do not do

• Do not make it warn-only, and do not add a manual "measurements unaffected" attestation. Both re-create the failure mode this issue exists to fix.
• Note the deliberate circularity: ci_public_baseline_check.py reuses public_baseline.py's functions rather than reimplementing them, precisely so the checker's own file is not in HARNESS_PATHS. Any fix must preserve that — otherwise changing the checker requires a 2-hour regeneration to satisfy the checker.

Context: the current regeneration is being run now, blocked until today by #7264 (a JSON.stringify data-loss bug that made the json_polyglot leg impossible to publish).