docs(notes): record the item-difficulty exchangeability review and likelihood sensitivities

METHODS.md

@@ -58,6 +58,8 @@ For the post-hoc P/N floor-transition estimands, a power-scaling conflict for `t
 
 **Evaluate.** Check convergence _before_ reading any estimate: R-hat ≤ 1.01, adequate effective sample size (ESS ≥ 400), and zero divergent transitions. A non-converged or divergent fit's posterior is not interpretable — fix the model, do not report it. The pipeline writes a `diagnostics_summary.json` whose pass/fail gate requires **0 divergences, BFMI ≥ 0.3, R-hat ≤ 1.01, ESS ≥ 400** (evaluated unrounded, over the model's free RVs plus the headline scalars) and the report renders it as a **pass/fail convergence banner first**, before any τ. This is only the sampling gate: `pareto_k.csv` separately flags importance-sampled LOO points above ArviZ's reliability threshold and makes that LOO score unreliable. In single-period ITT/joint models, where each point is one child, `scripts/influence_sensitivity.py` performs the required direct treatment-effect refit without all flagged children before a robustness claim. It standardises the full and leave-out posteriors over the same retained children, so the headline refit shift is not contaminated by changing the covariate-averaging population; the smaller composition shift and the total full-sample-to-leave-out shift are reported separately. This checks whether the treatment estimate is sensitive to those children; it does not repair or replace the unreliable LOO score. In repeated-measures random-intercept families, each point is a child-by-period row conditional on that child's fitted intercept; use an observation-level sensitivity or exact/moment-matched LOO for that same predictive target, not the whole-child ITT/joint runner. Prior- and posterior-predictive checks assess whether the working likelihood reproduces the outcome by arm, baseline band and distribution shape; a material shape flag remains a qualification even when convergence passes. The 1,000 prior draws are persisted onto `trace.nc` (the `prior` / `prior_predictive` / `log_prior` groups) so the report can show the prior-predictive check, the prior-vs-posterior overlay, the estimand-scale prior pushforward, and power-scaling sensitivity without refitting.
 
+**Reading diagnostics — always via `sampling_quality`.** Never re-derive R-hat, ESS, BFMI or divergences from a trace by hand, in pipeline code or in a one-off script: call `statistical_models.sampling_quality.sampling_quality(trace, var_names=…)`, which returns the four signals unrounded and correctly coerced. Hand-rolled extraction has gone wrong twice for the same reason. `az.summary()` rounds to `rcParams["stats.round_to"]` (`"2g"`, two significant figures) unless passed `round_to="none"` — the **string**; `round_to=None` and `"auto"` both fall through to the rounded default, and omitting the argument returns a _string_-dtype frame that raises on float formatting. Rounding erases exactly the digits the gate turns on: across the whole gate-relevant range **every R-hat from 1.011 to 1.049 rounds to `1.0`**, so an `R-hat ≤ 1.01` test silently becomes `R-hat < 1.05` (dseinternational/research#65; found again in #440 in the exact-LOO-refit gate, which had been gating refits on rounded values since #438, and in a prototype script that reported four fits as `1.0000` when they were 1.0011–1.0022). ESS must be the minimum of `ess_bulk` and `ess_tail`, as the gate is defined, not `ess_bulk` alone. The helper deliberately does not decide pass/fail — call sites differ in which variables they gate over and in how they treat a missing BFMI — so those policies stay at the call site while the extraction stays in one place.
+
 **Compare.** PSIS-LOO via ArviZ: prefer the higher-`elpd` model only when the difference clears its standard error (`elpd_diff` against `dse`). The interaction models are tested against their own no-interaction baselines as clean nested comparisons; `scripts/compare_statistical_models.py` collects the cross-model views. For the families with a per-child random intercept (mechanism, gain-/level-factors, DiD, dose-response, LCSM, growth) the pointwise unit is a child × phase/period row and the LOO is **conditional** (within-child, one-row-out) — elpd scores predicting a held-out row given that child's own fitted intercept, not new-child prediction — so it is comparable only across models sharing the same grouping on the same rows (which the nested comparators do).
 
 Two separate questions govern whether a nested contrast may be read, and the comparison CSVs record them in separate columns (#438). **Reliability** — `comparison_valid` — asks whether the number can be trusted at all: a comparison is refused whenever the models do not share ordered rows, or when importance sampling is unreliable (`pareto_k` above ArviZ's `good_k`). The HSGP-curve mechanism models each carry one or two such influential rows out of ~150, because the basis coefficients plus a child random intercept at this sample size make a single child-phase row pivotal for the curve near its own exposure value; the linear-mechanism pairs carry none. Those points are repaired by **exact refit** (`reloo`, at most `_RELOO_MAX_REFITS` per model), and `loo_method` / `n_exact_refits` record that the repair happened rather than leaving a reader to infer it from a silently-valid row. **Discrimination** — `elpd_verdict` — is the different question of whether a trustworthy number separates the models, and follows the standing `|elpd_diff| < 4` rule. A nested pair differing by one regularised coefficient sits near the floor of what predictive comparison can resolve at n ≈ 54, so **an inconclusive contrast is the expected outcome and is a pass, not a failed comparison**: it is recorded and reported as "LOO does not distinguish these models at this sample size", and the scientific claim rests on the coefficient's own posterior (median, 50 %/89 % intervals, direction probability, ROPE) in the usual way. Do not read the `|elpd_diff| / dse` ratio in that regime — when two models are near-identical pointwise the standard error is itself unreliable, and a ratio above 5 on a difference of ~1 elpd says the models agree, not that one wins.

config/spellcheck/allow-en.txt

@@ -84,6 +84,7 @@ Gottman
 Graphviz
 Grasman
 Gunn
+Guttman
 HPDI
 HSGP
 Haldane
@@ -106,6 +107,7 @@ Isager
 JCPP
 Jakulin
 Jarrold
+Kadane
 Kapalková
 Kass
 Kostewicz
@@ -131,7 +133,6 @@ LRPLF
 LRPMM
 LRPSURV
 LSAM
-MAXJOBS
 Laan
 Lakens
 Lawson
@@ -146,6 +147,7 @@ Loveall
 Lucas
 Lyster
 Lüdecke
+MAXJOBS
 MCMC
 MCSE
 MIMIC
@@ -219,6 +221,7 @@ RLOG
 RMSE
 RMSEA
 Raftery
+Rasch
 Ratz
 Reichow
 Rhemtulla
@@ -284,6 +287,7 @@ agespeak
 anchoring
 aptgram
 aptinfo
+argmax
 asis
 assoc
 autoregressive
@@ -308,6 +312,7 @@ brms
 bytree
 caveated
 celf
+chokepoint
 cloglog
 coef
 coeffs
@@ -410,8 +415,8 @@ importorskip
 incl
 invlogit
 isfinite
-isna
 isinstance
+isna
 issubset
 iterdir
 iterrows
@@ -461,30 +466,30 @@ miscalibration
 monofont
 monofontoptions
 morphosyntax
-mypy
 multicollinear
 multicollinearity
 multimethod
 multitrait
 mumedu
 mumedupost
 mumocc
+mypy
 ncol
 ndarray
 networkx
 noconstruct
 nogp
 nohup
-noqa
 noncentred
 nonlinear
 nonparametric
 nonword
 nonwords
 nonzero
-nunique
+noqa
 numchil
 numpy
+nunique
 nutpie
 offfloor
 operationalisation
@@ -494,6 +499,8 @@ operatorname
 optuna
 orcid
 othertimeread
+overcover
+overcoverage
 overdispersion
 overfit
 overfits
@@ -524,11 +531,11 @@ recomputation
 redescribing
 reframing
 reloo
+removesuffix
 reparameterisation
 reparameterise
 reparameterised
 reparameterising
-removesuffix
 replite
 repr
 resid
@@ -560,6 +567,7 @@ scikit
 scipy
 scrartcl
 scratchpad
+secondaries
 sens
 shutil
 sigmas
@@ -588,10 +596,13 @@ trivariate
 trog
 unadjustable
 unblockable
-chokepoint
+uncentred
 unclustered
 unconfounded
 unconverged
+undercoverage
+underdispersed
+underdispersion
 unfittable
 unidimensionality
 univariates
@@ -615,10 +626,3 @@ yarcsi
 ypre
 zproc
 ΔELPD
-secondaries
-uncentred
-argmax
-cloglog
-disattenuated
-disattenuation
-surv