refactor(level-factors): typed run plan + recorded estimand/population metadata (#389 finding 6)

[ethanbuckley]

Note

Drafted by an LLM-based AI tool (Claude Code/Opus 4.8).

Towards #389 (the level-factor critical-review follow-up), mirroring the gain-factor #418 run-plan for the level family. Deliberately not an auto-closing Closes — this addresses findings 4 and 6; the rest are follow-ups (below).

What this does (findings 6 & 4)

A new level_factors.py gives the family the same typed boundary the ITT and gain-factor families already have:

• LevelFactorsModelSettings — immutable, slotted, validated; from_legacy_extra rejects unknown spec.extra keys so a misspelling (group_by_tyme) fails before data loading instead of silently taking a default (finding 6).
• LevelFactorsRunPlan — resolved and validated before the context resets an output directory or the loader reads data. One object drives load_and_prepare, the factory call, the teaching model_recipe.md, and config.json, and records the design, estimand, causal status, analysis population and missing-data assumption (finding 4 — the level family previously persisted null metadata while its report published an unqualified cause-and-effect statement).

fit_level_factors now consumes plan.prepare_kwargs() / plan.factory_kwargs(effective_adjustment=…), and reporting._resolved_run_plan gains a level_factors branch so write_model_recipe and config.json's resolved_run_plan pick it up.

Byte-identical

The resolved kwargs reproduce the former inline fit_level_factors logic exactly (same phase_mode="levels", outcome-only load, wave-split covariate timing, factory flags), so the build of every one of the 11 registered level-factor models is unchanged. Verified:

• model-definition / model-id drift guards, the level-factor factory tests, key-findings, and the prior-fallback-defaults lock all pass;
• lrp-rli-lf-001 dev fit is clean (210 obs = 53×4, 0 divergences) and now emits model_recipe.md + a populated resolved_run_plan in config.json;
• 15 new run-plan tests (test_level_factors.py): fail-fast on unknown key, typed-vs-legacy source, mixed-declaration rejection, off-floor node/estimand, wave-split timing, effective-adjustment override, and every registered model resolves with recorded metadata.

ruff, npm run format:check, npm run spellcheck all clean.

Estimand honesty

The run-plan states the t2 quantity as currently implemented and explicitly flags that its precise definition — population-standardised average vs conditional-at-a-profile, and the treatment of the currently time-invariant group × ability term — is the open methodological decision in finding 1. This PR does not pre-empt that decision.

Follow-ups (separate PRs / gated)

• Finding 1 — the dated methodological decision defining the t2 estimand + group × ability handling (@frankbuckley's call).
• Finding 2 — the intercept prior recentring to substantive locations (needs refit validation).
• Finding 3 — emit the level-family prior_pushforward.csv + surface the t2 psense warning beside key findings.
• Finding 5 — name the SP/RW coefficients as baseline-predictor associations.
• The 11-model reporting-tier refit comparing revised t2 effects with ITT / gain-factor / DiD (VM).

Assigning to a human for review and merge.

[Copilot]

Pull request overview

This PR extends the statistical-model “typed run plan” pattern to the level_factors family, mirroring the earlier ITT and gain-factors refactors, so level-factor fits resolve/validate settings before loading data and persist a concrete, audited run plan (including estimand/population metadata) into config.json and model_recipe.md.

Changes:

• Added LevelFactorsModelSettings + LevelFactorsRunPlan and resolve_level_factors_run_plan() to strictly validate/resolve level-factor configuration (including fail-fast rejection of unknown legacy spec.extra keys).
• Updated fit_level_factors to drive data preparation and factory construction via the resolved plan, and updated reporting to persist/restore the resolved plan for the level-factors family.
• Added comprehensive unit tests covering settings validation, run-plan resolution, and “all registered level-factor models resolve with required metadata” acceptance criteria.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
src/language_reading_predictors/statistical_models/level_factors.py	New typed settings + resolved run-plan implementation for the level-factors family, including recorded estimand/population metadata and loader/factory kwargs helpers.
src/language_reading_predictors/statistical_models/pipeline.py	Refactors fit_level_factors to resolve/attach the plan before any data loading/output reset, and to use plan.prepare_kwargs() / plan.factory_kwargs().
src/language_reading_predictors/statistical_models/reporting.py	Adds a level-factors branch so resolved_run_plan and model_recipe.md generation can consume/reconstruct the typed plan.
tests/statistical_models/test_level_factors.py	New tests validating strict settings translation, plan resolution, and full registered-suite metadata coverage.