Finding: 30 systematic experiments across 6 rounds took a daily SPY direction model from a walk-forward Sharpe of -0.86 to +0.91, and the gain survived a leakage stress test (stricter purge and embargo only reduced it to +0.88).
This repo contains the final winning model (train.py + config.py), the full experiment log including everything that failed, and the literature synthesis that guided the search. It is backtest research, not a trading system. Read the disclaimer before drawing any conclusions.
| Round | Change | Walk-Forward Sharpe |
|---|---|---|
| 0 | Baseline: XGBoost + triple barrier + 3-class labels | -0.86 |
| 1 | LightGBM + rolling z-score features | -0.05 |
| 2 | Expanding window + class balancing | +0.33 |
| 3 | LightGBM + XGBoost ensemble | +0.47 |
| 4 | 3-year minimum train window | +0.74 |
| 5 | Cross-asset features (IWM, TLT) | +0.91 |
| 6 | Validation: stricter purge/embargo | +0.88 |
| Change | Sharpe Impact |
|---|---|
| LightGBM + rolling z-score normalization | -0.86 -> +0.21 |
| Expanding window + class balance (with binary labels) | +0.33 |
| 3-year minimum train window | +0.47 -> +0.74 |
| Cross-asset features (IWM, TLT) | +0.74 -> +0.91 |
| LGB + XGB equal-weight ensemble | +0.33 -> +0.47 |
Most ideas failed. That is the point of logging all 30 experiments.
| Idea | Result |
|---|---|
| 3-class labels (stop/time/profit) | Much worse; model defaults to majority class |
| Removing calendar features | Always worse; they encode real seasonality |
| Adding many extra features | Worse; dilutes strong signals with noise |
| RandomForest as third ensemble member | Worse (+0.22) |
| Shorter (3-day) prediction horizon | Worse; more noise at short horizons |
| Heavier regularization | Much worse (+0.27) |
| 60/40 weighted ensemble | Slightly worse than equal weight |
| Probability-based position sizing | Slightly worse than binary positions |
| Same features on BTC-USD | Failed (-0.19); equity features do not transfer |
The full per-experiment table is in docs/EXPERIMENT_LOG.md.
- Validation: purged walk-forward with an expanding window (3-year minimum train, 63-bar test folds), a purge gap between train and test, and an embargo after the purge.
- Labels: binary up/down over a 10-day forward return.
- Features: 35 rolling z-score normalized features (returns, volatility, RSI/MACD/Bollinger, calendar sin/cos, VIX, plus IWM and TLT cross-asset signals).
- Leakage check: increasing the purge gap from 10 to 15 bars and the embargo from 5 to 10 bars only moved Sharpe from +0.91 to +0.88, which argues the edge is not lookahead leakage.
The +0.91 Sharpe and +57.08% total out-of-sample return come from 819 test bars (2022-06-29 to 2025-10-02) with 5 bps per-trade transaction costs assumed. Annualized return was 14.91% with a 17.65% max drawdown and a 1.18 profit factor. That is one instrument, one period, one cost assumption.
pip install -r requirements.txt
python train.pyPrice data (SPY, VIX, IWM, TLT) is downloaded at runtime via yfinance; no data files ship with the repo. Edit config.py to change the ticker, windows, barriers, or model parameters, then re-run. Results are written to results/ as timestamped JSON.
This is walk-forward backtest research. Transaction costs are assumed at 5 bps per trade; real costs, slippage, and capacity are not modeled. This strategy has never been traded live. Backtest performance does not predict future results. Nothing in this repository is financial advice.
- docs/EXPERIMENT_LOG.md: all 30 experiments, round by round, winners and failures.
- docs/RESEARCH_SYNTHESIS.md: literature survey on ML price prediction that informed the experiment design.