Skip to content

Add probabilistic multi-horizon forecasting with conformal calibration#7

Open
xungirl wants to merge 1 commit into
mainfrom
codex/upgrade-to-interval-forecasting-with-quantiles
Open

Add probabilistic multi-horizon forecasting with conformal calibration#7
xungirl wants to merge 1 commit into
mainfrom
codex/upgrade-to-interval-forecasting-with-quantiles

Conversation

@xungirl

@xungirl xungirl commented Apr 6, 2026

Copy link
Copy Markdown
Collaborator

Motivation

  • Provide backward-compatible probabilistic, multi-horizon forecasts (default pred_len=96) on top of existing point-forecast backbones.
  • Enable calibrated interval outputs via split/joint conformal methods and optional online adaptation to meet coverage guarantees.
  • Make the implementation modular and minimally invasive so existing point-only experiments continue to work when probabilistic features are disabled.

Description

  • Added a model-agnostic forecasting package app/forecasting/ implementing ForecastConfig and ForecastPipeline to wrap existing backbones and expose flags for quantiles, conformal mode, and regularizers.
  • Implemented a monotonic quantile head MonotonicQuantileHead with median-centered softplus increments to avoid quantile crossing and produce outputs shaped [B, H, D, Q].
  • Implemented pinball loss, optional crossing penalty, horizon smoothness regularizer, and optional trajectory-level consistency loss under app/forecasting/losses.py.
  • Added split conformal utilities SplitConformalCalibrator supporting horizon-wise and joint trajectory calibration, optional adaptive updates (rolling-window + EMA), and a coverage-drop reset hook in app/forecasting/conformal.py.
  • Added probabilistic metrics (horizon/average coverage, mean interval width, Winkler, WIS approximation, joint trajectory coverage) in app/forecasting/metrics.py.
  • Added an end-to-end demo scripts/forecasting_example.py, unit tests tests/test_forecasting_probabilistic.py, and README examples; and updated requirements (requirements.txt, requirements-backend.txt) to include numpy.
  • Modified/added files: app/forecasting/__init__.py, app/forecasting/quantile.py, app/forecasting/losses.py, app/forecasting/conformal.py, app/forecasting/metrics.py, app/forecasting/pipeline.py, scripts/forecasting_example.py, tests/test_forecasting_probabilistic.py, README.md, requirements.txt, and requirements-backend.txt.

Testing

  • Compiled the new modules with python -m compileall app/forecasting scripts/forecasting_example.py tests/test_forecasting_probabilistic.py which completed successfully.
  • Ran pytest -q tests/test_forecasting_probabilistic.py initially, which failed during collection with ModuleNotFoundError: No module named 'numpy' in the environment before dependency update.
  • Added numpy>=1.24.0 to requirements.txt and requirements-backend.txt and recompiled; unit tests cover monotonic quantile ordering, conformal horizon widening, and point-only backward compatibility (intended to pass once numpy is installed in the test environment).

Codex Task

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant