What Demonstrably Works in Botty — Evidence Ranking Across All Sweeps, Tests & ML Experiments

The Bar

"Demonstrably reliable" here means: passed walk-forward across multiple windows, a shuffle test, or PBO — not "looked good in the backtest". This bar is deliberately brutal: of 22 megasweeps only 4 have a trustworthy PBO (<0.20), of 45 ML experiments ~8 survived as promoted, and the Indicator Lab found exactly one significant raw signal among 13 indicators × 6 timeframes. What follows below survived these filters.

Tier 1 — Confirmed by multiple walk-forwards: the volatility axis

In plain terms: BTC volatility comes in blocks — turbulent stays turbulent, calm stays calm. How violent tomorrow will be, we can predict well. Which direction it goes, we cannot. That is why everything that turns vol knowledge into money (position size, stop width, not trading through a structural break) is our most reliable track.

Tier 2 — Structural rules that hold consistently across many sweeps

1. The edge lives in the regime gate, not the entry trigger. Every robust sweep winner carries a trend gate (min_adx≥25, ema200, Hurst) or is the regime. Raw entries have been broadly exposed as overfit stand-alones (PBO 0.5–0.9) — the trigger is interchangeable, the gate does the work. (Full derivation: Megasweep PBO synthesis: the edge sits in the regime gate, not the entry trigger, usage consequence: Detecting & predicting market regimes: ADX/DMI is only one lens among many.)
2. Timeframe law: 30m–1d. Anything ≤15m overfits reliably. The 5m hurst+macd trap remains the cautionary tale: in-sample score 112, Monte Carlo prob_profit 0.0015.
3. Exits: adaptive beats tight-fixed (for breakouts). The stop-robustness sweep isolated the exit question cleanly (entry+filter pinned, 48 exit variants, 18,410 Phase-2 runs): fixed stops dominated the raw leaderboard (486 of the top 500!), but only because they rode ONE window — all died on the two-window criterion. What survived: ATR trailing (initial 1×, trail 3×) + partial TP: PBO 0.164, prob_profit 0.93, profitable even in the 2022 bear. Signal exits (ema/macd cross) consistently do harm everywhere.
4. Validated filters — but conditional. vrp_filter (shuffle-z +2.53 on Donchian, +37.7%) and bocpd_filter (z>1.5 on bb_extreme, +7–17%) are genuine signals. But: BOCPD hurts breakouts (−15%, it blocks exactly the changepoint bars on which they fire), and with an adaptive ATR exit even vrp loses its OOS contribution. Filter value depends on entry type AND exit — it is never absolute.

Tier 3 — The three validated directional candidates

Case study Donchian: why "good" always needs a timeframe qualifier

Donchian is the best example that evidence is conditional — the same entry idea, two verdicts:

• On 4h, gated: one of our best-validated building blocks (numbers above; predecessor sweep PBO 0.33 with vrp as the driver).
• On 1d, in the direct three-entry comparison (daily-trend sweep, identical methodology): dead. 16/56 OOS windows positive (avg −0.17%), best finalists MC prob_profit 0.127–0.499, median MC return ≤0. ema_crossover won 1d by a mile (33/56, pp 0.974).
• Ungated, in the old broad sweeps: PBO 0.77 — overfit as a stand-alone.

Mechanistically plausible: on 1d there are simply too few channel breakouts per year, and the 20-day channel only fires once the daily trend has already run far — the 4h grid sees the same trend earlier and often enough to earn back the losses from false breakouts. Rule of thumb: "Does X work?" for us is always "does X work on which TF, with which gate, with which exit?"

Downgrade of the ema@1d family (2026-06-11) — the methodology catches the next ghost

DOWNGRADE 2026-06-11: The follow-up sweep MS_20260610_050521 (ema_crossover_edge_v1: filters pinned, full 48-exit matrix, 1d+4h, identical training 2024+2025, 45,494 P2 runs) does NOT reproduce the finding — PBO 0.77, best finalist wf_avg only 0.53 (previously 8.69), prob_profit 0.77 (previously 0.974), MC p5 negative; the 4h variants did not even reach the final round. The star finding was presumably a selection artifact of the smaller search space. The ema@1d family is therefore NO LONGER a deploy candidate — the same one-window/selection pattern that already debunked archetypes A/B. This is not a contradiction of the methodology but its confirmation: an enlarged search space + PBO exposes what a smaller search space made look robust. Consequence for the live wallets: only donchian@4h (gated, adaptive exit) is currently deploy-grade; outside_inside_day@4h is the next candidate but first needs its own validation sweep.

Downgrade of the OID strategy (2026-06-11) — significant is not the same as large

DOWNGRADED AS A STRATEGY 2026-06-11 (sweep MS_20260611_051553, oid_validation_v1): as a full strategy OID@4h already fails in Phase 1 — 96 variants (full 48-exit matrix × {none, cooldown}), median -1.18%% across 2024+2025, best variant +0.10%% (pf 1.03), zero Phase-2 qualifiers, done_no_winners. The signal edge itself remains statistically valid but is economically too small: +10.85 bps net × ~28 signals/year = approx. +0.3%%/year gross ceiling. LESSON: the Lab "robust" verdict answers significance, not size — before every deploy, compute bps-per-trade × frequency. That leaves exactly ONE directional candidate: donchian@4h.

Tier 4 — The methodology itself is the most important validated building block

The stack (PBO/CSCV + multi-window walk-forward + min_trades≥30 + bear/crash in OOS + shuffle tests + residual checks) has repeatedly exposed convincing fakes before money was riding on them: archetypes A/B (one-window wonders, cross-check PBO 0.77), regime_switch (in-sample +10.9%, zero OOS survivors), 5/6 ML-overview findings, the ETF-flow signal (−70% momentum artifact in the residual test), the 3-lucky-trade winners of the min_trades-8 sweeps. Each of these catches would otherwise have become a live loss.

Honest footnotes

• Directional forecasting is dead — in every form: best directional IC +0.02 (47 features), divergences debunked 3× across 2,900+ cells, RSI failure swings dead, regime direction noise. Reliable negative knowledge: resources spent there are wasted.
• FOMC (+50 bps, t=2.28) is plausible, but CI lower bound +8 bps → watchlist, not "reliable".
• The live bb_extreme config belongs to archetype B, which the clean cross-check classified as a one-window wonder (best MC pp 0.61) — it runs on weaker evidence than anything in Tier 1–3.

The whole picture in one sentence

In Botty, reliable performance comes almost exclusively from the risk side (predict vol → sizing, stops, not trading) plus structural discipline (gates, TF ≥30m, adaptive exits) — on the directional side there exist exactly three narrow, validated candidates, and a methodology that keeps us from imagining more.