Quantitative Methods & RiskBacktesting Hygiene: Why Most Backtests Lie

Costs, Slippage & the Bootstrap

13 min read

The frictions that eat the edge

A backtest with zero transaction costs is a fantasy, and the higher the turnover, the more dangerous the fantasy. Real trading pays the spread, commissions, financing/swap, and — most insidiously — slippage and market impact, the gap between the price you assumed and the price you actually got. Many backtested edges are entirely consumed by costs once modeled honestly. The faster a strategy trades, the more its survival depends on getting the cost assumptions right.

Cost	What it is	Why backtests understate it
Spread	Bid-ask gap paid on entry and exit	Backtests often fill at the mid price, ignoring it
Commission	Broker fee per trade	Easy to forget on high-frequency strategies
Slippage	Difference between expected and executed price	Assumed zero; worst exactly when you most want to trade
Market impact	Your own order moving the price	Invisible in a backtest that assumes infinite liquidity
Swap / financing	Overnight cost of leverage	Omitted, yet compounds on every held position

Heads up

Slippage is worst when you need it least

Slippage and impact are not constant — they balloon in fast, illiquid, or gapping markets, which is exactly when many strategies generate signals. A stop assumed to fill at the trigger price may fill far worse in a gap. Model costs conservatively, and stress them: re-run the backtest with double the assumed slippage and see if the edge survives. If it doesn't, it wasn't an edge.

Formula

Edge net of costs

E_net = E_gross − (spread + commission + E[slippage] + financing)

The bootstrap: how confident can you be?

Even a clean, cost-aware, out-of-sample backtest gives you a single equity curve — one realization. The bootstrap is a resampling technique that turns that one history into a distribution of plausible histories, so you can put error bars on your results. Resample the trade returns (with replacement) thousands of times, rebuild an equity curve each time, and look at the spread of outcomes. A strategy whose bootstrapped Sharpe distribution comfortably straddles zero has not proven anything.

Bootstrap

Term

A resampling method: repeatedly draw samples (with replacement) from your observed data to approximate the sampling distribution of a statistic. In backtesting, bootstrapping trade returns gives a distribution of possible Sharpe ratios, drawdowns, and final equities — error bars on a result that otherwise looks like a single point.

bootstrap_sharpe.py

python

1import numpy as np
2 
3def bootstrap_sharpe(trade_returns, n_boot=10_000, seed=0):
4    rng = np.random.default_rng(seed)
5    r = np.asarray(trade_returns, dtype=float)
6    n = len(r)
7    sharpes = np.empty(n_boot)
8    for i in range(n_boot):
9        sample = rng.choice(r, size=n, replace=True)   # resample trades
10        sharpes[i] = sample.mean() / sample.std(ddof=1)
11    lo, hi = np.percentile(sharpes, [2.5, 97.5])
12    return lo, hi      # 95% confidence interval for the Sharpe
13 
14lo, hi = bootstrap_sharpe(my_trade_returns)
15print(f"95% CI for per-trade Sharpe: [{lo:.3f}, {hi:.3f}]")
16# If this interval includes 0, the 'edge' is not statistically distinguishable
17# from noise -- no matter how good the single backtested curve looked.

The bootstrap converts one equity curve into a confidence interval. If zero sits inside the interval, you cannot reject 'no edge' — the honest verdict the single backtest hid.

Tip

Block-bootstrap for autocorrelated returns

Plain bootstrap assumes the trade returns are independent. Because of volatility clustering (Module 1), real returns are not — so resample in contiguous BLOCKS to preserve the autocorrelation structure. A block bootstrap gives wider, more honest confidence intervals than the naive version, which understates uncertainty for serially dependent data.

Knowledge check

After cleaning your backtest, the bootstrapped 95% confidence interval for the strategy's Sharpe is [−0.2, 1.1]. The correct interpretation is: