research: EP review phase 0 — Fable reassessment of priors/messages findings

[Jammy2211]

Overview

Phase 0 of the EP framework review (umbrella prompt: PyAutoMind/research/graphical_ep/ep_framework_review.md). An earlier Opus-era audit of priors and messages produced 14 parked findings (PyAutoMind/bug/priors/01–14) plus a confirmed-bug census (PyAutoMind/research/autofit/priors_and_messages_math_audit.md). Those findings predate fixes such as e95295b83 (log_prior density-form sign convention) and #1329 (LogUniform -inf for value<=0), so before anything is fixed, every finding is re-verified with Fable against clean main and given a verdict. No PyAutoFit source is edited in this phase — confirmed bugs graduate to their own fix tasks.

Baseline for reassessment: main @ 0f26ff2d8 ("fix: LogUniform NumPy log-prior returns -inf for value<=0 (emcee NaN crash) (#1329)").

Plan

• Re-verify all 14 bug/priors findings and the audit census's confirmed-bug list against main @ 0f26ff2d8: reproduce crash-class findings, re-derive math-class findings analytically and cross-check numerically against scipy.stats.
• Issue a verdict per finding — fix now / park / close stale — with re-ranked severity and a minimal reproduction where confirmed.
• Write verdicts back into each bug/priors/*.md prompt; update or retire the parked audit census.
• Post the verdict table here and recommend which bugs graduate to immediate fix tasks and what Phase 1 (the wider EP statistics review of autofit/graphical) should scrutinise first.

Detailed implementation plan

Affected Repositories

• PyAutoFit (primary — read-only: reproductions against clean main)
• PyAutoMind (verdict writes into bug/priors/*.md + audit census)

Branch Survey

Repository	Current Branch	Dirty?
./PyAutoFit	main	clean (claimed by aggregator-output-contracts worktree — not a blocker for read-only work)
./PyAutoMind	main	clean

Suggested branch: none — research task, no source edits; fixes become separate feature/<bug> tasks.

Implementation Steps

1. Crash-class findings — scratchpad reproduction scripts against the installed stack (numpy/scipy only, no JAX):
   • 01 LogGaussianPrior.with_limits / _new_for_base_message ctor crash (autofit/mapper/prior/log_gaussian.py)
   • 02 UniformPrior logpdf array handling
   • 05 inv-beta suffstats clamp no-op
   • 06 NormalMessage negative sigma unchecked
   • 10 fixed-message cache growth (small-loop memory profile)
2. Math-class findings — re-derive analytically, numerically cross-check vs scipy.stats:
   • 03 GammaPrior.from_mode formula
   • 04 truncated-normal log-partition completeness
   • 07 log_prior normalisation convention (check interaction with e95295b83 density-form change)
   • 08 relative-width modifier safety
3. Design/meta prompts — reassess relevance/priority, no reproduction:
   • 09 prior property tests; 11 transformed-message semantics doc; 12 single-source density refactor; 13 collapse prior+message; 14 bijector transform stack
   • several may be absorbed by Phases 1–2 of the umbrella review (formal-equations documentation).
4. Write a ## Fable verdict (2026-07-08) block into each PyAutoMind/bug/priors/*.md with verdict, severity, and reproduction status.
5. Update or retire PyAutoMind/research/autofit/priors_and_messages_math_audit.md.
6. Post the verdict table to this issue with the Phase 1 hand-off recommendation.

Key Files

• autofit/mapper/prior/ — prior classes under reassessment
• autofit/messages/ — message algebra under reassessment
• PyAutoMind/bug/priors/01–14.md — the findings receiving verdicts
• PyAutoMind/research/graphical_ep/ep_framework_review.md — umbrella prompt (Phases 0–7)

Original Prompt

Click to expand starting prompt

So, we used Opus to do a review of messages and priors in autofit, which led to all the issues in bug/priors. I want us reassess this using Fable, and for us to go on to do a wider review of the Expectation propagation framework (source code PyAutoFit/autofit/graphical, with examples giving a run through of the graphical / EP modeling in HowToFit/scripts/chapter_3_graphical_models, and the cancer use case in /mnt/c/Users/Jammy/Science/ic50_workspace, and cosmology in /mnt/c/Users/Jammy/Science/concr/scripts/cosmology . Tasks include: (i) assessing if priors, messages and the underlying statistics of EP in autofit has bugs we should fix; (ii) documentation of the graphical package (including formal bayesian equations which ensure an AI agent knows exactly what bayes statistics are being used) and more thorough integration tests in autofit_workspace_test (see autofit_workspace_test/scripts/graphical) and more thorough end-to-end example scripts, written in the style of eamples in autofit_workspace/scripts, which explain the EP framework, its stats and allow for step-by-step run trhoughs of th eEP framework, e.g. something written in the style of something like autolens_workspace/scripts/imaging/likelihood_function.py which allows a user to run EP at the lower level API step-by-step and follow what is happening; (iii) currently there are not many built in tools for analysing the results of an EP fit or monitoring how the fit is progress, can we add more of these (could be funciton calls at end of examples, visuals output during EP and text file metadata in the output folder; (iv) deterministic variables in the cancer use case use inbuilt aspects of the EP composition (find an old PR by rhayes777 on this) but do not use the determinitic_variable API in the low level EP code, should we do anything about that?; (v) scope out analytic likelihood updates in the EP source code; (vi) What other features can we plan? This is obviously a huge prompt, so use your fable magic to plan everything out thoroughly

(Umbrella prompt covers Phases 0–7; this issue is Phase 0 only. Later phases are issued separately as each predecessor nears shipping.)

[Jammy2211]

Phase 0 verdict table — Fable reassessment on main @ 0f26ff2d8

Every finding was re-verified: crash-class by direct reproduction, math-class by analytic re-derivation + numerical cross-check against scipy.stats. All nine reproducible findings CONFIRM on clean main — none aged out. Full verdict blocks are written into each PyAutoMind/bug/priors/*.md.

#	Finding	Verdict	Notes
01	LogGaussianPrior.with_limits crash	CONFIRMED — fix now	TypeError as predicted; _new_for_base_message fails via a different path (AttributeError: 'TransformedMessage' object has no attribute 'instance' after __getattr__ forwarding) — same conclusion
02	UniformPrior.logpdf array handling	CONFIRMED — fix now	scalar OK, array raises ValueError; TODO still in source
03	GammaMessage.from_mode formula	CONFIRMED, sharpened	correction: mean matches exactly (β=α/m); variance is inversely wrong — from_mode(2, V=0.25) → var 2.0, from_mode(2, V=4) → var 0.235. V=1 test points cannot discriminate the candidate fixes; use V≠1. Needs invariant decision (lean B: α=m²/V, β=m/V)
04	TruncatedNormalMessage.log_partition incomplete	CONFIRMED, numerically exact — top-priority math fix	∫pdf via generic interface = 2.2663 = predicted σ·exp(μ²/2σ²) to 4 d.p.; the generic path is what autofit/graphical (EP) consumes
05	inv_beta_suffstats clamp no-op	CONFIRMED — fix now	warning fires, returns a=b=−498.8; lean raise over clamp (no silent guards)
06	NormalMessage negative sigma unchecked	CONFIRMED — fix now	constructs silently with deceptive variance=1.0; GaussianPrior.with_limits(5,5) → σ=0; lean strict sigma <= 0 reject; prerequisite for 08
07	log_prior_from_value convention	CONFIRMED — design decision open	integrals 2.000 / 4.605 / 2.507 / 2.507 / 1.000; settle Option A/B inside the EP review's formal-equations documentation (Phase 2)
08	RelativeWidthModifier safety	CONFIRMED — after 06	σ=−0.5 flows through silently, value_for(0.84) = −0.497 (flipped scale)
09	Prior property tests	still wanted	coverage gap re-verified; schedule after 01–08
10	FixedMessage.logpdf_cache growth	CONFIRMED — fix now	0→500 entries; aliasing corrupts shared array; one-line fix
11	TransformedMessage semantics doc	valid — fold into EP review Phase 2	anchors unchanged
12	Single-source density refactor	valid — design, sequence behind EP review Phases 1–2	bundle with 13
13	Collapse Prior/Message	valid — bundle with 12	__getattr__ delegation unchanged
14	Bijector migration	valid — go/no-go after 12/13	JAX-native priors landing since weakens the numpyro dependency objection

Recommendation

1. Graduate a single mechanical fix batch: 01, 02, 04, 05, 06, 10 — each has an unambiguous, verified fix (with two small decisions to take at plan approval: 05 raise-vs-clamp, 06 strictness). Note PyAutoFit is currently claimed by the aggregator-output-contracts worktree, so this batch queues behind it or coordinates.
2. 03 and 08 need their invariant/scope decisions first; 08 additionally waits on 06.
3. 07, 11 fold into Phase 2 (formal Bayesian documentation) of the EP framework review; 12, 13, 14 stay parked behind Phases 1–2 output.
4. Phase 1 hand-off: the EP statistics review should scrutinise first the generic exponential-family interface (log_partition / logpdf / projection), since 04 shows that path can be silently wrong while the direct paths stay correct — the exact failure shape EP is most exposed to.

Phase 0 of the umbrella review (PyAutoMind/research/graphical_ep/ep_framework_review.md) is complete with this comment.

[Jammy2211]

Decision hub for the confirmed findings: #1331 (concise, awaiting maintainer guidance + contributor input). This issue remains the Phase 0 record; Phase 1 (EP statistics audit of autofit/graphical) proceeds in parallel.

[Jammy2211]

EP framework review — execution complete (all 8 phases, 2026-07-08)

This closes the execution of the review that started here. One-line-per-phase record:

Phase	Outcome	Where
0 — Fable reassessment of the priors/messages audit	all 9 findings confirmed on clean main, none stale; verdicts written into bug/priors/01–14	#1330, decisions #1331
1 — EP statistics audit	core EP algebra verified correct; findings F1–F10 (2 new confirmed bugs w/ repro, reversed KL directions, 3-legged evidence breakage, sigma-collapse pathology); truncated-KL error quantified 1.5%→140%	#1332
2 — Formal documentation	autofit/graphical/README.md (16 code-anchored equations) + statistical docstrings; shipped	PR #1334 (awaiting merge)
3 — Examples + integration tests	step-by-step low-level EP feature example (validated: centre 49.96±0.12 from offset priors) + ep_parity/ep_deterministic/ep_exact scripts	autofit_workspace#81 (parked at ship sign-off)
4 — Diagnostics & monitoring	EPDiagnostics (live CSVs, mean-field evolution + per-factor plots, post-fit summary, F10 sigma-collapse guard) + F3 fix; full suite 1429 pass	#1335 (parked at ship sign-off)
5 — Deterministic variables	rhayes777 PR located (#1153, test disabled); analysis: compound/shared patterns are statistically tighter, factor_out trades exactness for modularity; recommendation A (keep both, document, resurrect test)	#1336 (decision pending)
6 — Analytic updates scoping	machinery half-exists incl. undocumented projection contract; declarative path never uses it (PriorFactor strips the hooks — one optimiser run per parameter per cycle wasted); 4-WP implementation plan filed	#1337, plan #1338
7 — Feature ideation	11 provenance-tagged ideas harvested to PyAutoMind/ideas.md (adaptive damping, block-Gaussian mean field, power EP, EP health report, warm-started searches, resumable EP, JAX-native Laplace, …)	PyAutoMind

Private statistical write-up: PyAutoMemory/methods_wiki/concepts/expectation-propagation.md (shipped); a public generalisation is planned for the future autofit_assistant wiki.

Human queue to unblock the next wave: decisions on #1331 (priors + EP fix batches) and #1336 (A/B/C); merge/amend PR #1334; ship sign-offs on autofit_workspace#81 and #1335; prioritisation of #1338's WP1. Keeping this issue open as the review's index until the queue clears.