Skip to content

Add a bias-controlled large-N benchmark runner - #158

Open
michael-denyer wants to merge 2 commits into
masterfrom
experiment/large-n-stage-benchmark
Open

Add a bias-controlled large-N benchmark runner#158
michael-denyer wants to merge 2 commits into
masterfrom
experiment/large-n-stage-benchmark

Conversation

@michael-denyer

@michael-denyer michael-denyer commented Aug 1, 2026

Copy link
Copy Markdown
Owner

What changed

  • adds an interleaved A/B stage benchmark for kinship, eigendecomposition, rotation, and Mode 4
  • verifies bitwise-identical numerical outputs before reporting timings
  • rebuilds workers for each ABBA/BAAB block and reverses creation order to remove allocation and process-layout bias
  • documents use and adds regression coverage for the ordering protocol

Evidence

  • focused regression tests: 2 passed
  • full suite: 2,295 passed, 6 skipped, 3 xfailed
  • full repository quality gate passed

Measurement note

A first 3,000-sample eigensolver comparison showed a stable 3-4% apparent gap between source-identical workers. Alternating worker creation order removed that false signal. The corrected result crosses zero and is reported as inconclusive. The runner is intended for comparisons on the Databricks ILP64/MKL large-scale machines, where the expected change appears near 100k samples.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant