Add quantile accuracy sweep to differential-testing harness

[synesenom]

Summary

Adds a third sweep dimension to the differential-testing harness (scripts/difftest-special.py/difftest-dist.py, ADR-0052): quantile accuracy. A new scripts/difftest-quantile.py + scripts/eval-quantile.js pair implements both metrics #1269 requires — a round-trip sweep (|cdf(q(p)) - p|) across every one of ranjs's ~146 distributions with log-uniform tail-focused p sampling, and an absolute ULP accuracy sweep against an independent mpmath inverse-CDF reference for the #1265 pilot family (Gamma, Beta, Chi2, F, StudentT, InverseGamma) — plus closed-form-vs-numerical quantile source distinction and separate non-convergence/out-of-support/non-monotonicity hard-failure reporting.

Closes #1269

Design decisions

No new ADR — this extends the harness pattern already governed by ADR-0052 (live mpmath, out-of-band from npm test, standalone no-cross-script-imports), the same way difftest-dist.py rode on it without a fresh ADR. The architecture (new sibling script + bridge pair, catalog-mode population sourced from test/dist-cases-*.js rather than a hand-maintained list) was settled via a design-propose/design-critique review during planning; see thoughts/plans/2026-08-11-1900-quantile-accuracy-sweep.md in this branch's history for the full reasoning.

Non-trivial changes

• scripts/eval-quantile.js (new): Node/@babel-register bridge with two modes — catalog enumerates every distribution's canonical valid parameter tuple, type, closed-form-vs-numerical quantile status (typeof instance._q === 'function'), and support bounds straight from test/dist-cases-continuous.js/-discrete.js; eval computes q(p)/cdf(q(p)) at supplied points. Code Health 10.0.
• scripts/difftest-quantile.py (new): round-trip sweep (generate_roundtrip_ps, sweep_roundtrip, _hard_failures) and pilot-family absolute-ULP sweep (mpmath_quantile, sweep_pilot_absolute). Two genuinely new pieces of numerical logic worth a reviewer's attention:
  • Tail-depth bound (P_TAIL_LO = 1e-6, not 1e-12): probing at 1e-12 hangs the harness indefinitely on several discrete distributions (e.g. BetaNegativeBinomial) whose _cdf is an O(k) recurrence sum that saturates in double precision before reaching that close to 1 — confirmed empirically and documented in solutions/tooling/2026-08-11-2026-quantile-sweep-tail-depth-hang-beta-negative-binomial.md.
  • mpmath_quantile's log-space/logit-space bracket search: a first linear/additive bracket-and-bisect attempt silently returned wrong (even negative) reference values for extreme shape parameters, because the true root can sit hundreds of orders of magnitude from the seed and a fixed-precision linear bisection loses the far endpoint entirely. Fixed by bisecting in log-space (domains bounded at 0) / logit-space ((0,1) domains) instead. Documented in solutions/tooling/2026-08-11-2026-mpmath-quantile-reference-linear-bracket-magnitude-collapse.md. A self-check (_formula_self_check_quantile) forces genuine bracket expansion through all three reparameterization branches before any sweep result is trusted.
• Real quantile defects surfaced (left unfixed per Add quantile accuracy sweep to the differential-testing harness #1269's explicit "Out of Scope"): LogCauchy's closed-form _q underflows/overflows at extreme tails, StudentT's closed-form _q is non-monotonic and wrong-signed for small nu near p=1, Beta's numerically-inverted _q underflows to its boundary for extreme shape parameters, and Gamma/InverseGamma's closed-form _q returns NaN for extreme shape parameters. Each filed as its own follow-up issue: Fix Beta quantile underflow to 0 for extreme shape parameters #1394 (Beta), Fix StudentT quantile sign flip and non-monotonicity for small nu #1395 (StudentT), Fix LogCauchy quantile overflow/underflow in exp() step #1396 (LogCauchy), Fix Gamma/InverseGamma quantile NaN for extreme alpha #1397 (Gamma/InverseGamma).

⚠ Production-code diff over the ~400-line guideline: this PR's production code (scripts/eval-quantile.js + scripts/difftest-quantile.py + the package.json/CHANGELOG.md additions) totals 591 lines, over the issue's ~400-line cap. This isn't padding — real numerical bugs were discovered mid-implementation (the two documented above) that needed genuine fixes with WHY-comments, not shortcuts. A design-critique review during planning weighed splitting the round-trip and absolute-accuracy metrics into two PRs against shipping together, and recommended shipping together since both are mandatory, separately-checked acceptance criteria on this one filed issue and the combined estimate had (what turned out to be optimistic) headroom under the cap. Flagging this transparently for review rather than silently declaring the criterion met.

Comprehension checklist

• I can explain what this code does without reading line-by-line
• I understand why this approach was chosen over alternatives
• WHY comments are present where the logic is non-obvious
• Tests verify observable behavior, not internal state (inline self-checks — see below)
• A developer encountering this code in 6 months could understand it

Checklist

• npm run standard passes (no linter errors)
• npm test passes (all tests green — 10176 passing; harness is out-of-band per ADR-0052, coverage thresholds unaffected)
• npm run typecheck passes
• Tests added or updated for changed behavior — this harness has no mocha tests by design (ADR-0052); correctness is established by inline, unconditional self-checks (_self_check() for the ULP metric, _formula_self_check_quantile() for the mpmath reference formulas and the new bracket-search algorithm, both run at the start of main())
• If a new distribution was added: entry added to test/dist-cases.js — N/A, no new distribution
• If a new distribution was added: class added to dist/ranjs.d.ts — N/A, no new distribution
• CHANGELOG.md updated under ## [Unreleased]
• Production-code diff is under ~400 lines (tests excluded) — 591 lines, see the flagged note above

Trivial changes

• package.json: two new npm scripts, difftest:quantile and accuracy:quantile, mirroring the existing difftest:dist/accuracy:dist pair. Deliberately not wired into the accuracy aggregate chain, accuracy:docs, or difftest:ci-gate — those are Generate documented accuracy bounds table from differential-testing harness reports #1266/Add scheduled CI workflow running the differential-testing harness #1267 scope.
• CHANGELOG.md: one ### Added bullet under [Unreleased].
• solutions/tooling/*.md (2 new files): documentation of the two numerical bugs described above, for future-session searchability.

---

Generated with Claude Code

---

Generated by Claude Code