Problem
A handful of statistical MCMC comparison tests in test/mc/ are individually very expensive and, together, are one of the largest contributors to npm test's ~6-10 minute runtime. Measured directly via a fresh full-suite run (10,176 tests, 627s wall clock under mocha --parallel, 3 workers on a 4-core container):
test/mc/adaptive-metropolis.js — "5D correlated Normal ESS comparison" › "should achieve higher effective sample size than RWM for equal iteration counts": 82stest/mc/mala.js — same test name/pattern ("5D correlated Normal ESS comparison" › "should achieve higher effective sample size than RWM for equal iteration counts"): 34stest/mc/nuts.js — "should achieve a higher effective sample size per iteration than HMC on a strongly correlated target": ~13stest/mc/hmc.js/nuts.js — "...per-dimension ESS on a target with very different scales, and leave it unbalanced without warm-up": ~42stest/mc/hmc.js — "should achieve higher effective sample size than the diagonal metric on a strongly correlated target": ~13stest/mc/nuts.js — "should not let a NaN acceptance statistic permanently freeze the dual-averaging step size": ~12s
Together these ~6 tests account for roughly 3-4 minutes of the suite's total wall-clock time.
Why this is expensive by design (not obviously a bug)
Per this repo's CLAUDE.md testing conventions ("Fixed seed sweeps for every statistical assertion" in the MCMC test rigor section), these tests deliberately run multiple seeds (typically [0, 42, 12345] or similar, sometimes 5 seeds) × several thousand samples in 5 dimensions, to get a statistically robust ESS (effective sample size) comparison between samplers rather than relying on a single seed that could pass or fail by chance. That rigor is intentional and should not be casually dropped — a naive reduction risks reintroducing flakiness the fixed-seed-sweep convention exists to prevent.
Suggested investigation
- For each of the tests listed above, determine whether the current sample size / warm-up batch count / seed count is meaningfully larger than what's needed to reliably distinguish the compared samplers' ESS at the assertion's tolerance — i.e. is there headroom to shrink without the assertion becoming seed-dependent/flaky.
- Consider whether warm-up length (not just sampling length) can be reduced — some of these tests use
warmUpBatches = 20 with sampleSize = 2000; the warm-up phase may be over-provisioned for reliable adaptation in 5D.
- Any change must be validated by re-running the affected test(s) several times (per this repo's own guidance on not trusting a single timing/stability measurement — see
solutions/testing/2026-07-21-1055-guess-default-pool-latent-fit-cliff.md's "Prevention Strategy" section) to confirm the trimmed test still passes reliably and isn't just faster by luck.
- This is explicitly not asking to remove the statistical rigor (multi-seed sweeps) — only to check whether the per-seed cost (sample count, dimensionality, warm-up length) is larger than necessary for the assertion being made.
Non-goals
- Do not reduce below what's needed for the fixed-seed-sweep convention (still sweep
SEEDS/comparable fixed seeds — do not collapse to a single seed).
- Do not touch the
guess() pool cost issue (filed separately) — this issue is scoped only to test/mc/ sampler-comparison test cost.
Problem
A handful of statistical MCMC comparison tests in
test/mc/are individually very expensive and, together, are one of the largest contributors tonpm test's ~6-10 minute runtime. Measured directly via a fresh full-suite run (10,176 tests, 627s wall clock undermocha --parallel, 3 workers on a 4-core container):test/mc/adaptive-metropolis.js— "5D correlated Normal ESS comparison" › "should achieve higher effective sample size than RWM for equal iteration counts": 82stest/mc/mala.js— same test name/pattern ("5D correlated Normal ESS comparison" › "should achieve higher effective sample size than RWM for equal iteration counts"): 34stest/mc/nuts.js— "should achieve a higher effective sample size per iteration than HMC on a strongly correlated target": ~13stest/mc/hmc.js/nuts.js— "...per-dimension ESS on a target with very different scales, and leave it unbalanced without warm-up": ~42stest/mc/hmc.js— "should achieve higher effective sample size than the diagonal metric on a strongly correlated target": ~13stest/mc/nuts.js— "should not let a NaN acceptance statistic permanently freeze the dual-averaging step size": ~12sTogether these ~6 tests account for roughly 3-4 minutes of the suite's total wall-clock time.
Why this is expensive by design (not obviously a bug)
Per this repo's
CLAUDE.mdtesting conventions ("Fixed seed sweeps for every statistical assertion" in the MCMC test rigor section), these tests deliberately run multiple seeds (typically[0, 42, 12345]or similar, sometimes 5 seeds) × several thousand samples in 5 dimensions, to get a statistically robust ESS (effective sample size) comparison between samplers rather than relying on a single seed that could pass or fail by chance. That rigor is intentional and should not be casually dropped — a naive reduction risks reintroducing flakiness the fixed-seed-sweep convention exists to prevent.Suggested investigation
warmUpBatches = 20withsampleSize = 2000; the warm-up phase may be over-provisioned for reliable adaptation in 5D.solutions/testing/2026-07-21-1055-guess-default-pool-latent-fit-cliff.md's "Prevention Strategy" section) to confirm the trimmed test still passes reliably and isn't just faster by luck.Non-goals
SEEDS/comparable fixed seeds — do not collapse to a single seed).guess()pool cost issue (filed separately) — this issue is scoped only totest/mc/sampler-comparison test cost.