Skip to content

feat(leakfree): auto-resolve CPCV/horizon embargo to the label horizon (L4488a)#230

Merged
cipher813 merged 1 commit into
mainfrom
feat/l4488a-cpcv-embargo-fix
Jun 2, 2026
Merged

feat(leakfree): auto-resolve CPCV/horizon embargo to the label horizon (L4488a)#230
cipher813 merged 1 commit into
mainfrom
feat/l4488a-cpcv-embargo-fix

Conversation

@cipher813
Copy link
Copy Markdown
Owner

L4488a — first item of the model-rotation scaffolding arc (L4488); scoring-honesty prerequisite.

The bug

WF_EMBARGO_DAYS defaulted to 0, leaving the CPCV after-test boundary open. The excluded zone around a test group [a,b] is [a − forward_days, b + embargo_days] (leakfree_meta_ic.py:53), so with embargo=0 train rows at b+1 … b+forward_days leak via overlapping labels with the test's late observations. That inflated every leak-free read — and more at longer horizons (60d needs a ~60-row embargo vs ~21 for 21d), making it a real confound for the 60d-vs-21d horizon call. The planned raise was bundled into W1.4, which challenger-first superseded (CF4), so it got orphaned.

The fix

WF_EMBARGO_DAYS default → None = auto = the label horizon (the correct overlapping-label embargo; makes the CPCV exclusion symmetric). Resolved inside leakfree_meta_oos_ic + cpcv_meta_oos_ic + expanding_wf_folds (None → forward_days); the horizon curve inherits per-horizon h automatically since it passes forward_days=h through. An explicit int (incl. 0) still overrides.

  • Prod predictor.yaml doesn't set embargo_days → auto applies in production.
  • predictor.sample.yaml updated to document the auto behavior (was a misleading explicit 0).
  • Note: the proper overlapping-label embargo is ≈forward_days, not the plan's old ~0.01·T (that's only the residual serial-correlation buffer, undersized here).

Effect

The leaderboard's leak-free CPCV/horizon reads become honest — the prerequisite for L4488b (net-of-cost scoring) and L4488d (the horizon call). The current ~0.18 CPCV is expected to drop somewhat (the honest number), and the 60d>21d gap will shrink once both are properly embargoed.

Tests

+1 (None auto-resolves to horizon; explicit 0 honored; cpcv too). Suite 1362→1363.

…n (L4488a)

WF_EMBARGO_DAYS defaulted to 0, which left the CPCV after-test boundary open:
the excluded zone around a test group [a,b] is [a-forward_days, b+embargo_days],
so with embargo=0 train rows at b+1..b+forward_days leak via OVERLAPPING LABELS
with the test's late obs. That inflated every leak-free read, and MORE at longer
horizons (60d needs a ~60-row embargo vs ~21 for 21d) -- a real confound for the
60d-vs-21d horizon call. The raise was bundled into W1.4, which challenger-first
superseded (CF4), so it got orphaned.

Fix: WF_EMBARGO_DAYS default None = auto = the label horizon (the correct
overlapping-label embargo that makes the CPCV exclusion symmetric). Resolved
inside leakfree_meta_oos_ic + cpcv_meta_oos_ic + expanding_wf_folds (None ->
forward_days); the horizon curve inherits per-horizon h since it passes
forward_days=h through. Explicit int (incl. 0) still overrides. Prod predictor
.yaml does not set embargo_days -> auto applies; sample yaml updated to document
the auto behavior (was a misleading explicit 0).

Decouples the embargo fix from the dead W1.4. First item of the model-rotation
scaffolding arc (L4488) -- scoring-honesty prerequisite for the leaderboard +
the horizon decision.

Tests: +1 (None auto-resolves to horizon; explicit 0 honored; cpcv too).
Suite 1362 -> 1363.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@cipher813 cipher813 merged commit 9087fd8 into main Jun 2, 2026
1 check passed
@cipher813 cipher813 deleted the feat/l4488a-cpcv-embargo-fix branch June 2, 2026 20:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant