Longitudinal IPCW sandwich: per-period censoring γ block for AIPW + IPW (and IPW propensity-weighting fix)#16
Merged
Conversation
… AIPW IPCW sandwich Under IPCW the longitudinal AIPW (ICE-AIPW) stacked-EE sandwich now includes the per-period censoring model in its parameter vector: theta = (alpha, beta, gamma, mu). Each period's logistic censoring score pins gamma, and the stabilized IPCW weight is threaded into every per-step outcome score as `external x ipcw(gamma)` at the step's fit rows, so the numerical bread captures the censoring-model estimation cross-terms (gamma -> outcome beta -> marginal mean) mechanically -- no hand-derived cross-derivative, consistent with the existing stacked-EE construction. A shared helper `make_ipcw_weight_fn_longitudinal()` (the per-period generalisation of the point `make_ipcw_weight_fn()`) supplies the censoring scores and the gamma->weight closure; it reproduces `details$ipcw_weights` exactly at the fitted gamma (~1e-16). The augmented (doubly-robust) estimator is first-order insensitive to the censoring nuisance, so the cross-term is near-zero in practice (it moves the marginal-mean SE by ~0.03%): the bread's (beta, gamma) block is non-zero (the block is wired) but (mu, gamma) is zero. The block is carried for an exactly correct sandwich, not to fix a sizeable bias -- the cross-package audit's ~1.03 SE ratio for longitudinal AIPW was Monte-Carlo noise, confirmed here. Testing: a new test-aipw-longitudinal-ipcw.R adds the first coverage for the longitudinal AIPW + IPCW sandwich (previously untested): the shared helper reproduces the IPCW weights and its scores vanish; the stacked EE is faithful with the gamma block; the (beta, gamma) cross-block is wired; a non-IPCW fit carries no gamma block; the with-gamma vs known-gamma SE agree to 0.5% (orthogonality); the per-arm sandwich SE tracks the Monte-Carlo empirical SD; and it matches the bootstrap. Non-IPCW longitudinal AIPW tests are unchanged (the gamma block is gated on `ipcw`). Chunk 1 of the longitudinal-IPCW-sandwich plan; the delicatessen cross-language oracle is reserved for the longitudinal IPW chunk, where the censoring cross-term is large (~15%) and load-bearing.
…opensity-weighting fix Two changes to longitudinal IPW under built-in IPCW (estimator = "ipw", type = "longitudinal", ipcw = TRUE). Estimator fix -- the IPCW weight no longer enters the propensity. causat() previously folded the censoring weights into the master weight vector that fed the per-period treatment-density models, so the propensity was fit IPCW-weighted on the uncensored rows. The standard IPW+IPCW construction estimates the treatment and censoring models separately and multiplies their weights, applying the censoring weight to the marginal structural model only (Hernan & Robins, Causal Inference: What If, 2020, Ch. 12.6 & 17). The per-period treatment-density and stabilization-numerator models are now ordinary regressions on all observed rows; the IPCW factor is carried by a new `propensity_weights` argument threaded causat() -> fit_ipw() -> fit_longitudinal_ipw() (and through refit_ipw() for the bootstrap), so only the folded `weights` reach the MSM. This shifts the IPCW longitudinal IPW point estimate onto the textbook estimator; non-IPCW fits are byte-identical, and point IPW is unchanged (a separate pre-existing path). Sandwich -- the censoring cross-term is now propagated. The id-level analytic sandwich gains compute_ipw_ipcw_correction_longitudinal() (R/variance_if_ipw_longitudinal_ipcw.R): it propagates the per-period censoring model's estimation uncertainty through the MSM (the gamma -> beta cross-term), reusing the shared make_ipcw_weight_fn_longitudinal() gamma block and the same apply_model_correction() per-period projection the propensity correction uses. The correction is subtracted, recovering the Robins-Rotnitzky-Zhao (1994) censoring-estimation efficiency gain. Unlike the doubly-robust AIPW case (where the cross-term is ~0.03% by orthogonality), the IPW cross-term is large and load-bearing -- ~5% of the treated-arm SE on an informatively-censored DGP. With the propensity now IPCW-unweighted, gamma reaches mu only through the MSM, so there is no gamma -> alpha term. Testing: new test-longitudinal-ipw-ipcw.R validates against a delicatessen M-estimation oracle (longitudinal_ipw_ipcw_delicatessen.py) that stacks the per-period propensity scores, the censoring gamma score, and the IPCW-weighted Hajek marginal-mean equations -- point to ~1e-13, per-arm/ATE SE to ~1e-4. The oracle's companion known-weights (gamma-fixed) sandwich pins the efficiency-gain magnitude two-sided (full < known, ~5% on the treated arm). Also: Monte-Carlo SE-vs-empirical-SD calibration (ratio ~1.0), bootstrap parity, and a propensity-is-unweighted regression anchor (prior weights == 1, coefs == unweighted GLM to ~1e-8). Existing longitudinal IPW / IPCW / lmtp-oracle / multivariate / AIPW-IPCW / NHEFS tests unchanged. Full suite (Tier-1) clean; R CMD check Status: OK. Chunk 2 of the longitudinal-IPCW-sandwich plan.
…orrection compute_ipw_ipcw_correction_longitudinal() builds the censoring cross-derivative from per-row products of the MSM score ingredients (X_msm, r_msm, mu_eta_msm, length nrow(model.matrix)) and the final-period fit-row vectors (final_global_rows, other_w, length n_final). These coincide for the supported `Y ~ 1` (and baseline-EM) MSM, where model.frame() drops nothing, but a future MSM that dropped a final-period row (e.g. an NA effect modifier) would silently misalign the two and corrupt the correction. Add the explicit nrow(msm_prep$X_fit) == n_final guard (classed causatr_variance_row_mismatch), mirroring the invariant the point-IPW primitive compute_ipw_if_self_contained_one() already enforces, so the condition fails loudly instead of silently. Defensive only -- no active bug (the supported path satisfies the invariant; the 24 test-longitudinal-ipw-ipcw.R assertions are unchanged). Surfaced by the critical review of the longitudinal IPW IPCW chunk. 1st-round critical review (longitudinal IPW IPCW)
…red point IPW fix While building the longitudinal IPCW sandwich (this PR) an audit found a single root cause: causat() folds the IPCW weights into the master weight vector, which feeds every nuisance fit including the treatment-density (propensity) models. The textbook construction estimates the treatment and censoring models separately and multiplies their weights (Hernan & Robins 2025 Ch 12.6 & 17), so the censoring weight must reweight the outcome/MSM side only. Sampling/transport weights are not folded, so they are clean. Findings (delicatessen-verified): longitudinal IPW + IPCW was both propensity-IPCW-weighted and missing the gamma->beta cross-term -- FIXED in this PR. Longitudinal AIPW (orthogonal) and ICE (no propensity; orthogonal outcome regression) are valid. Point IPW + IPCW has a CONFIRMED 3-7% per-arm sandwich error (the IPCW-weighted propensity makes the sandwich omit the gamma->alpha cross-term; the point estimate stays consistent). Point AIPW + IPCW is non-standard but DR-orthogonal (SE acceptable). SNM + IPCW (18g) carries a censoring block but was not oracle-validated here. The point IPW fix is deferred to a focused implement-feature effort: standardize the propensity to unweighted-on-all-rows (deeper than the IF -- make_weight_fn() and the point contrast are row-coupled to the uncensored-fit propensity) via a gated stacked-EE sandwich, then tighten the loose NHEFS IPCW test. Recorded in PHASE_14_IPCW.md (Post-shipment audit), FEATURE_COVERAGE_MATRIX.md (known limitation), CLAUDE.md, .claude/hard-rules.md (do-not-rediscover), and the variance-theory vignette (point IPW + IPCW callout: use bootstrap meanwhile). No code change -- documentation only.
The 2026-06-25 IPCW audit's deferred work (standardize the point IPW + AIPW propensity to unweighted-on-all-rows, gated stacked-EE sandwich for point IPW + IPCW, validate SNM + IPCW, tighten the loose NHEFS test) is now a first-class design doc, PHASE_27_POINT_IPCW_PROPENSITY.md, rather than a subsection buried in the completed Phase 14 -- so it stays findable and is a direct target for the implement-feature workflow. PHASE_14's post-shipment audit now points to Phase 27 for the fix detail; the CLAUDE.md phase-status Pending list adds 27 and clarifies that the earlier "point IPW/AIPW IPCW need no extra term" note held only for the multinomial direct dmu/dgamma term -- the gamma->alpha propensity-weighting cross-term was Monte-Carlo-masked and is a real 3-7% point IPW SE bug. No code change -- documentation only.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Completes the longitudinal IPCW sandwich so the per-period censoring model's estimation uncertainty is propagated analytically for both longitudinal AIPW and longitudinal IPW.
What's here (3 commits)
bb7fa24— Longitudinal AIPW IPCW γ block (chunk 1). The ICE-AIPW stacked-EE sandwich gains a per-period censoring γ block inθ = (α, β, γ, μ); the IPCW weight is threaded into each outcome score so the numerical bread captures γ→β→μ. By double-robust orthogonality this cross-term is near-zero (~0.03% of the SE) — wired and carried for an exactly-correct sandwich.e84c103— Longitudinal IPW IPCW γ→β correction + propensity-weighting fix (chunk 2).propensity_weightsargument threadedcausat()→fit_ipw()→fit_longitudinal_ipw()(and throughrefit_ipw()for the bootstrap). Non-IPCW fits are byte-identical.compute_ipw_ipcw_correction_longitudinal()(R/variance_if_ipw_longitudinal_ipcw.R) propagates the censoring model's estimation uncertainty through the MSM (γ→β), reusing the sharedmake_ipcw_weight_fn_longitudinal()γ block and the per-periodapply_model_correction()projection. Load-bearing for IPW (~5% of the treated-arm SE on an informatively-censored DGP), unlike the orthogonal AIPW case.b2aafd4— Review hardening. Explicitnrow(X_fit) == n_finalrow-alignment guard (classedcausatr_variance_row_mismatch), mirroring the point-IPW primitive's invariant.Validation
tests/testthat/fixtures/python/longitudinal_ipw_ipcw_delicatessen.py): point to ~1e-13, per-arm/ATE SE to ~1e-4. Its companion known-weights (γ-fixed) sandwich pins the efficiency-gain magnitude two-sided.test-longitudinal-ipw-ipcw.R(24 assertions); existing longitudinal IPW / IPCW / lmtp-oracle / multivariate / AIPW-IPCW / NHEFS tests unchanged.Docs
NEWS.md, FEATURE_COVERAGE_MATRIX.md, CLAUDE.md,
.claude/hard-rules.md, and variance-theory vignette §6.5 (differentiates the IPW load-bearing / AIPW orthogonal / ICE mildly-conservative cases) updated.Follow-up (separate PR)
A package-wide IPCW/nuisance-weight audit: point IPW IPCW has a confirmed 3–7% sandwich bug (same IPCW-weighted-propensity root cause; delicatessen-verified) — it needs a point-IF re-architecture to standardize. Also point AIPW (orthogonal), ICE a0 conservatism, and SNM+IPCW (silently accepted) to validate/fix.