Walk-Forward Validation Under Non-Stationarity

The assumption that breaks

Standard $k$-fold cross-validation assumes that observations are exchangeable. Shuffle the data, split it into folds, and each held-out fold is a fair test of a model trained on the rest. For time-ordered data, this assumption is false in two ways. Observations are serially dependent, and the process generating them changes over time. A model that looks excellent under shuffled folds may simply have memorized local structure that will not persist.

Walk-forward validation respects the arrow of time. The model is trained on a window of the past and evaluated on the period immediately after it, then the window moves forward.

Leakage through overlapping labels

Time ordering alone is not sufficient. Many targets are defined over a horizon: an observation at time $t$ has a label that depends on the path between $t$ and $t + h$. If a training sample's label window overlaps the test window, information from the test period leaks into training, even though no test row was used directly.

Let the test period be $\mathcal{T} = \{t_0, \dots, t_1\}$ and let the label horizon be $h$. A training index $t$ is safe to use only if its label window $[t, t+h]$ lies entirely before the test period. Equivalently, the purged training set is

$$\mathcal{I}_{\text{train}} = \{\, t \;:\; t + h < t_0 \,\} \;\cup\; \{\, t \;:\; t > t_1 + e \,\},$$

where the first set removes all samples whose labels overlap the test period, and the second, with embargo length $e$, drops observations that immediately follow the test window. The embargo guards against leakage through serial dependence in features, which purging alone does not remove.

Walk-forward versus combinatorial splits

Walk-forward splits are simple and match how a model would be deployed. Their weakness is that they produce a single historical path, so one unlucky regime can dominate the conclusion. Combinatorial purged splits build many train and test paths from the same data, which yields a distribution of out-of-sample scores rather than a point estimate. That distribution is more informative, and it makes the sensitivity of conclusions to the sample explicit.

What to report

Validation output should include:

• the purge horizon $h$ and the embargo length $e$, stated explicitly;
• the number of independent test periods, not only the number of folds;
• a dispersion measure for the out-of-sample metric, such as an interval from a block bootstrap;
• a comparison with a naive baseline evaluated under the same protocol.

A single average score over folds, reported without these details, does not support a conclusion about out-of-sample behavior.

A note on interpretation

Non-stationarity means that the past is an imperfect guide to the future by construction. Good validation does not remove that uncertainty. It makes the uncertainty visible, so that a reader can judge whether a method's behavior is stable across regimes. Methods that only succeed in one regime should be presented as such.

Nothing in this note is a statement about the future performance of any strategy or investment.

@misc{research2026walkforward,
  author       = {{Valence Quantitative Research}},
  title        = {Walk-Forward Validation Under Non-Stationarity},
  howpublished = {Valence Quantitative Research LP},
  year         = {2026},
  url          = {https://readscripts.online/blog/walk-forward-validation/}
}