Promote the word-reading → nonword-decoding model into the registered suite (off-floor primary, log exposure) — blocked on #428

[frankbuckley]

Note

Drafted by a LLM-based AI tool (Claude Code/Opus 4.8).

What this promotes

notes/202607241900-findings-word-reading-threshold.md (pending commit) asked whether children need a minimum word-reading level before they can progress in letter sounds, blending or nonword reading. The answer is no minimum for any of the three, but the nonword result is strong enough to deserve a registered model: dropping the period-start word-reading term costs 18 ± 4.4 elpd, the largest effect of any specification choice in the probe. This issue promotes that one fit out of notes/assets/202607241900-wr-threshold-probe.py and into the suite. The letter-sound and blending results are negative and are explicitly not proposed for registration.

The finding, and the shape that matters

Period-start word reading predicts end-of-period nonword reading with diminishing returns, not a threshold. PSIS-LOO over four specifications that differ only in how word reading enters (all sharing own baseline, age, arm, on-intervention status, hearing, speech, and a child random intercept):

Spec	elpd difference	dse	weight	Same, restricted to ≤ 25 words
log (diminishing returns)	0.0	—	1.00	1.00
linear	−4.0	2.6	0.00	−7.0 ± 1.7
hinge (free breakpoint — the prerequisite hypothesis)	−6.0	2.9	0.00	−10.0 ± 2.5
null	−18.0	4.4	0.00	−11.0 ± 3.3

Fitted expected nonword score by period-start words read, covariates at their means: 0.29 (0 words), 0.48 (1), 0.63 (2), 0.98 (5), 1.41 (10), 2.21 (25). Roughly half the climb across 0–40 words happens over the first five. The slope survives adjusting for period-start letter sounds (+0.71 [+0.50, +0.94] → +0.57 [+0.33, +0.81]), so it is not letter-sound knowledge in disguise. The probe fits were clean (0 divergences, max R-hat 1.01, min bulk ESS 581) but bypass the production gate, publish no config.json / diagnostics_summary.json / report, and should not be cited outside the note until this lands.

Blocked on #428 — read that first

WR_t → NW_{t+1} is not an edge in dag/dag-language-reading-lagged.dagitty. #428 is the open decision on whether it should be. That decision determines what this model can claim, and there is no point building it twice:

• If the edge is adopted — derive the minimal sufficient adjustment set against the amended graph before specifying the module, and confirm it is fittable at n ≈ 52. On the current graph the implied set is 8 nodes on the randomised transition and 12 post-crossover, which is not fittable; an amended graph may or may not improve that. If it does, the model ships as an adjusted association in the usual idiom.
• If the edge is rejected — the model still has descriptive value (it is the cleanest statement of a pattern the cohort clearly shows), but it ships as causal_status="none", estimand_type="descriptive", with report prose stating that the quantity is a marginal lagged association whose content includes latent general ability, shared intervention dose, and whatever runs through letter sounds and blending.

Do not start the module until #428 closes. Everything below is the specification to build once it has.

Proposed design

Family. kind="mechanism", mechanism_symbol="W", outcome_symbol="N", using the mechanism_at_pre=True capability salvaged in #423 — this is exactly the lagged readout it was kept for (mechanism_pre → outcome_post ∣ outcome_pre). Id to be confirmed next-free against definitions.MODEL_REGISTRY and docs/models/registry-counts.json; note the mech-19N docstring-alias trap recorded in #430.

Two estimands, and the floored one should lead. Nonword reading is 6 items and floored for 72 / 64 / 52 / 40 % of children at t1–t4, so most of what "progress" means here is floor exit:

1. Primary — off-floor risk. A Bernoulli on the off-the-floor-at-post indicator, reusing the bernoulli_offfloor likelihood the ITT and gain-factor families already carry for P and N. Reported as an off-floor risk difference across the word-reading range. This is the estimand the descriptive result is in (4/29 at zero words, 10/37 at 1–4, 11/14 at 5–9), and it is the better-powered one.
2. Secondary — graded count. The Beta-Binomial on the count, flagged as the weaker readout, mirroring the suite's floor-rule convention of an exploratory binary headline plus a flagged graded secondary.

Functional form. The winning form is a straight line in log1p(words) — a fixed concave transform. That is not currently an option on the mechanism factory, which offers an HSGP curve or linear_mechanism=True. Options, in preference order:

• add a mechanism_transform option ("identity" | "log1p") to the mechanism factory — small, generic, and the honest representation of what the data prefer;
• or fit linear_mechanism=True on a pre-transformed exposure and record the transform in spec_extra.

An HSGP curve is not recommended as primary: mech-072 already established that a nonparametric dose-response is not identifiable on a floored 6-item outcome, #421 applies the same reasoning to its proposed mech-102 / mech-103, and #430 is currently repairing a GP knee test that failed for exactly this reason. If a curve is wanted at all it belongs as a sensitivity, and its divergence behaviour should be checked before it is promised in a report.

Comparators. A null (no word-reading term) companion for the PSIS-LOO contrast, following the mech-072 / mech-172 convention. The linear and hinge specifications do not need registering — their role was to establish the shape, and that belongs in the note, not the suite.

Covariates. Own period-start nonword score, age, arm, on-intervention status, hearing, speech, child random intercept — the probe's set — unless #428's re-derivation says otherwise, in which case the DAG wins.

Acceptance criteria

• Decide whether WR_t → NW_{t+1} belongs in the lagged DAG (and refresh the stale d-separation asset) #428 closed, and this issue's estimand framing (causal_status, estimand_type) set to match its outcome.
• Module(s) in src/language_reading_predictors/statistical_models/, each a SPEC = ModelSpec(...) + fit(), id confirmed next-free; registered via module auto-discovery; python scripts/check_statistical_documentation.py --write run and registry-counts.json regenerated.
• mechanism_transform (or the recorded pre-transform equivalent) implemented with tests.
• Off-floor primary and graded secondary both reported, with the floor-rule caveat stated in the report prose.
• Thin report template under docs/models/{model_id}/ reusing _results_mechanism; docs/models/README.md mechanism table updated; PSIS-LOO contrast against the no-word-reading companion wired into scripts/compare_statistical_models.py.
• --config reporting fit passes the convergence gate (R-hat ≤ 1.01, ESS ≥ 400, BFMI ≥ 0.3, 0 divergences); estimates reported as median + 50 % / 89 % CrI + P(> 0).
• Report prose states the three things the note cannot resolve: direction (a child beginning to decode also reads more words), the 6-item floor (a concave curve on a logit link with a floored bounded outcome is partly manufacturable by the instrument), and latent general ability.
• notes/202607241900-findings-word-reading-threshold.md updated with the registered model's numbers.
• ruff check src/, npm run format:check, npm run spellcheck pass.

Related

Source note notes/202607241900-findings-word-reading-threshold.md and probe notes/assets/202607241900-wr-threshold-probe.py. Blocked on #428 (the WR_t → NW_{t+1} DAG decision). Uses the mechanism_at_pre capability from #423. Sibling follow-ups from the same pair of reviews: #421 (letter-sound → word-reading models, including RW → NW and SP → NW mechanisms this would sit beside), #404, #429, #430. Descriptive antecedent: notes/202607241600-findings-word-reading-bands.md.

[frankbuckley]

Note

Drafted by an LLM-based AI tool (Claude Code/Opus 5).

The nested-LOO acceptance criterion here has been restated (#438)

This issue stays blocked on #428; the flag is so the criterion in the acceptance box is not taken at face value when it unblocks. It specifies a null companion "following the mech-072 / mech-172 convention", and that convention changed in #438 (merged in #439).

What changed. The nested contrast only produces an interpretable delta when the mechanism enters linearly. Every HSGP-curve pair in the suite exceeded ArviZ's Pareto-k reliability threshold and degraded to per-model elpd_loo; mech-072 / mech-172 validated only because nonword decoding is floored and forced a linear form. #439 adds an exact-refit (reloo) repair that restores validity at 1–2 refits per model, plus loo_method, n_exact_refits and elpd_verdict columns recording that the repair happened. The restated criterion (now in METHODS.md) separates reliability — comparison_valid=True, after repair where needed — from discrimination, where |elpd_diff| < 4 is inconclusive and counts as a pass.

This issue is better placed than most. It already argues against an HSGP primary, so its planned specifications are exactly the linear/parametric class that has never tripped the threshold. The probe's 18 ± 4.4 elpd separation is also an order of magnitude clear of the |elpd_diff| < 4 band, unlike the near-identical nested pairs elsewhere in the suite — so if that survives registration, this would be the first nested contrast in the suite that actually discriminates. Worth stating that expectation explicitly in the report prose rather than letting a reader assume the suite's usual inconclusive result.

One concrete gap to plan for. The repair path is mechanism-family only and assumes the Beta-Binomial observed variable name: statistical_models/loo_refit.py reads idata.log_likelihood["y_post"] for its alignment guards, while the off-floor likelihood registers its observed node as y_offfloor (factories.py). The primary estimand proposed here is the off-floor Bernoulli, so if it does exceed the Pareto-k threshold the repair will not run as written — it needs the observed-variable name taken from the built model rather than hard-coded. That is a small change, but it should be scoped into the build rather than discovered at comparison time.

[frankbuckley]

Note

Drafted by an LLM-based AI tool (Claude Code/Opus 5).

Correction and update: the refit-path gap is fixed, but it was the smaller of two prerequisites

Two amendments to the comment above.

The y_post hard-coding is fixed — #441 derives the observed-variable name from the built model and verifies it against the stored trace, so the exact-refit repair no longer assumes the Beta-Binomial node name. It also now refuses a multi-outcome model explicitly rather than silently splicing against one of its outcomes. Verified as a no-op for the models that exist today: both live mechanism traces still resolve to y_post and pass every alignment guard unchanged.

But my comment implied that was the blocker, and it is the second of two. build_mechanism_model has no likelihood parameter at all — the mechanism family is Beta-Binomial-only, and the bernoulli_offfloor path exists in the ITT and gain-factor factories but has never been plumbed into this one. So the off-floor primary proposed here needs the factory extended first; the refit path only becomes reachable after that. The issue body already anticipates the plumbing ("reusing the bernoulli_offfloor likelihood the ITT and gain-factor families already carry"), so this is a scope note rather than news, but the earlier comment made the sequencing sound the wrong way round.

Nothing else in that comment changes. The restated nested-LOO criterion still applies — a reliable contrast is required (comparison_valid=True, repaired if needed) and its discrimination is reported separately, with |elpd_diff| < 4 counting as an inconclusive pass. And the observation that this issue is better placed than most still stands: the specifications proposed here are the linear/parametric class that has never tripped the Pareto-k threshold, and the probe's 18 ± 4.4 elpd separation is an order of magnitude clear of the inconclusive band.