fix(graphical): seed inherently-stochastic test_full_hierachical (#1352)#1355
Merged
Conversation
test_full_hierachical fits a marginal hierarchical EP model whose Laplace refinement (n_refine=3) draws samples from the global np.random state via NormalMessage.sample, so its recovered mu_logt hyperparameter converges to a good value or a sigma-collapsed one depending on the ambient RNG state left by whatever ran before it. The test seeded np.random only before data generation, not before the fit. When #1351 added test_ep_statistics_fixes.py, its Monte-Carlo KL-direction tests consumed np.random and shifted the ambient state into a failing region, turning the Tests workflow red on main (green at #1347 -> red at #1351/#1354). The fit's math is unchanged; this is a test-determinism fix. Seed np.random immediately before the fit so the test is reproducible. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DgCFQoGUbVkspbCjRuKnHJ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the red Tests workflow on PyAutoFit
main.test_autofit/graphical/hierarchical/test_hierarchical.py::test_full_hierachicalis an inherently stochastic test that was failing deterministically at HEAD.The test fits a marginal hierarchical EP model. The Laplace optimiser refines each factor with
n_refine=3stochastic samples drawn from the globalnp.randomstate (NormalMessage.sample→np.random.randn), so the recoveredmu_logthyperparameter converges to a good value or a sigma-collapsed one depending on the ambient RNG state left by whatever ran before the test. The test seedednp.randomonly before data generation, never before the fit.When #1351 added
test_autofit/graphical/functionality/test_ep_statistics_fixes.py(Monte-Carlo KL-direction tests that consumenp.random), the ambient state going into the hierarchical fit shifted into a failing region — flipping the full-suite result from green (#1347,2e33b175) to red (#1351/#1354). #1351's F1/F2/F4/F8 math is not implicated.Fix
Seed
np.randomimmediately before the fit so the test is reproducible. One line (+ explanatory comment); no library or API change.API Changes
None — test-only change.
Validation
test_autofit/graphical/— 216 passed, 0 failedtest_autofit/— 1475 passed, 14 skipped, 0 failed (was 1474 passed / 1 failed at HEAD)Notes
The underlying observation — EP/Laplace fits draw from global
np.random, so results aren't fully reproducible across arbitrary process states (the fit also depends on variable-id ordering) — is documented on #1352. A controlled-RNG design was prototyped and reverted as out of proportion for a marginal stochastic test; seeding is the pragmatic fix per maintainer steer. Flagged there if reproducibility becomes a first-class requirement.Closes #1352.
🤖 Generated with Claude Code
https://claude.ai/code/session_01DgCFQoGUbVkspbCjRuKnHJ