Master-LightGBM — kitchen-sink 4h vol forecast

2026-05-20 · status: promoted · 39.1s

Hypothesis: A LightGBM regressor on a unified causal feature panel (lagged returns, multi-window RV, funding, BOCPD p_short, HMM p_state1, VRP, stablecoin Δ7d, ETF flow, DXY z-score, time) beats HAR-RV on OOS log-vol R² at 4h by ≥ 3 pp.

Verdict: PROMOTE — LightGBM lifts 4h vol-forecast R² by +10.96 pp vs HAR-RV baseline (0.5541 → 0.6636). Top features by importance: log_rv_7d_ann, iv_ann, hour_cos. Replace HAR-RV with LGBM in ml/forecast/ after a clean production run.

Key metrics

Approach

Build a unified 1h causal panel with lagged returns (1/4/24/168h), multi-window realized vol (1h/4h/1d/7d, annualised), funding (rate, z-score, cum-1d), BOCPD p_short (from 15m, causal forward filter), HMM p_state1 (from 1h, re-fit per walk-forward split), VRP (DVOL annualised IV − trailing 4h RV), stablecoin Δ7d (1d shift), ETF flow (1d shift), and DXY 4h z-score.

Walk-forward 12mo train / 3mo test, embargo = 1440 min, starting 2022-01. Target = log of forward 4h annualised RV. Models: Persistence, HAR-RV (baseline), LightGBM-all (kitchen sink, native NaN handling), LightGBM-new-only (new features only).

Pooled OOS metrics

Lift over HAR-RV baseline

• LightGBM-all R² lift: +10.96 pp (0.5541 → 0.6636)

• LightGBM-new-only R²: +0.5009

Per-window R² (13 windows)

Feature importance (LightGBM-all, mean gain across folds)