Overfitting, out-of-sample and walk-forward testing

What it is

Overfitting (curve fitting) means tuning a strategy so closely to past data that it captures the noise of that period rather than a real edge. The more rules and settings you tweak, the better the backtest looks and the worse it usually does afterwards.

The defences: split the data, develop on one part (in-sample) and check on a part you haven't touched (out-of-sample). Walk-forward testing, set out by Robert Pardo, repeats that step through time: optimise on one window, test on the next, slide forward, and judge the strategy on the combined out-of-sample results. Forward testing (on a demo or small size) is the final check.

How to spot it

• Warning signs: many rules, very specific settings (a 37-period average), and results that collapse if a setting moves slightly.

How to use it

• Keep rules few and settings round, and check that nearby settings work nearly as well.
• Hold back the most recent months of data as an out-of-sample test.

Common mistakes

• Testing many ideas on the same data and keeping the best, without an out-of-sample check.
• Re-optimising after every losing month.

Related