Module 5 — Strategy Design

Why Most Backtests Lie

Trend, reversion, and the diagnostic discipline that separates real edge from a curve fit.

Module lessons

Moving Averages and SmoothingMomentum vs Mean ReversionWhy Most Backtests Lie

Learning objectives

  • ▸Identify look-ahead, survivorship, and selection biases.
  • ▸Account for transaction costs, borrow costs, and capacity.
  • ▸Use out-of-sample testing and deflated Sharpe.

TEXT

The hierarchy of backtest sins

1. Look-ahead bias: using information at time t that wasn't available at time t. 2. Survivorship bias: testing on today's index constituents. 3. Data-snooping: trying 100 strategies and reporting the best one. 4. Ignoring costs: 1 bp on a 250%-turnover strategy is 2.5% per year of return gone. 5. Capacity: a strategy that works at $1M may move the market at $100M.

FORMULA

Deflated Sharpe (Bailey & López de Prado)

Adjusts the observed Sharpe downward to account for the number of trials N you ran.
Intuition: if you tried 100 strategies, the best one will look great even if none have real edge.

TEXT

Costs you actually pay

• Commission: ~0.1–1 bp for institutional flow on liquid equities. • Spread: half the bid-ask, larger for small caps and crypto. • Slippage: your own market impact, grows roughly as sqrt(participation rate). • Borrow: short rebate for shorts. Hard-to-borrow names can cost 5–50% annualised. • Financing: leverage costs SOFR + spread.

EXAMPLE

Out-of-sample discipline

Lock the final 20% of your data and don't touch it until you've fully committed to a strategy. If OOS Sharpe is materially worse than in-sample, your in-sample number was a curve fit. Resist the temptation to 'fix' the strategy after peeking — once you look, your test set is contaminated.

Checkpoint quiz

A momentum backtest on the current S&P 500 over the last 30 years would suffer from:

  • A.Look-ahead bias
  • B.Survivorship bias
  • C.Heteroskedasticity

The current S&P 500 only contains firms that survived 30 years — by construction you're trading a winning subset.

A strategy turns over 400% per year and reports a 0.8 Sharpe before costs. At 2 bps round-trip, the after-cost edge is roughly:

  • A.Unchanged
  • B.Cut to roughly 0.5
  • C.Negative

400% turnover × 2 bps = 0.8% per year of return given up. On a strategy earning ~5%/yr vol ~6%, that knocks Sharpe from ~0.8 to ~0.5.

← PreviousNext →