walk-forward-validation

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Walk-Forward Validation

向前验证（Walk-forward Validation）

Walk-forward validation framework for trading strategies and ML models. Standard cross-validation (k-fold, random splits) fails catastrophically for financial time series because it introduces lookahead bias and ignores autocorrelation. This skill covers proper time-series validation techniques including rolling and expanding windows, purged cross-validation, combinatorial purged cross-validation (CPCV), and overfit detection metrics.

适用于交易策略和ML模型的向前验证框架。标准交叉验证（k折、随机分割）在金融时间序列场景下完全失效，因为它会引入前瞻偏差并忽略自相关性。本框架涵盖了合适的时间序列验证技术，包括滚动窗口、扩展窗口、清洗交叉验证（purged cross-validation）、组合清洗交叉验证（CPCV）以及过拟合检测指标。

Why Standard Cross-Validation Fails

为何标准交叉验证失效

Standard k-fold CV assumes data points are independent and identically distributed (IID). Financial time series violate both assumptions:

Lookahead bias — Random splits let the model train on future data and predict past data, artificially inflating performance.
Autocorrelation — Adjacent observations are correlated. A random split that puts Monday in test and Tuesday in train leaks information.
Regime dependence — Markets shift between regimes. A model trained on a bull market and tested on a bull market tells you nothing about bear market performance.
Label overlap — If labels are computed over windows (e.g., 24h forward return), adjacent train/test samples share label computation periods, leaking information.

标准k折交叉验证假设数据点是独立同分布（IID）的。金融时间序列违反了这两个假设：

前瞻偏差——随机分割会让模型在未来数据上训练，对过去数据进行预测，人为夸大性能表现。
自相关性——相邻观测值存在相关性。将周一数据放入测试集、周二数据放入训练集的随机分割会泄露信息。
状态依赖性——市场会在不同状态间切换。在牛市中训练并测试的模型，无法体现其在熊市中的性能。
标签重叠——如果标签是基于时间窗口计算的（例如24小时远期收益），相邻的训练/测试样本会共享标签计算周期，从而泄露信息。

Walk-Forward Framework

向前验证框架

Rolling Window (Fixed Train Size)

滚动窗口（固定训练集大小）

The train window has a fixed size and slides forward in time. This is preferred when you believe older data is less relevant (common in crypto).

Window 1: [===TRAIN===][=TEST=]
Window 2:    [===TRAIN===][=TEST=]
Window 3:       [===TRAIN===][=TEST=]

Parameters:

```
train_size
```
: Number of bars/days in the training window
```
test_size
```
: Number of bars/days in the test window
```
step_size
```
: How far to advance between folds (often equals
```
test_size
```
)

训练窗口大小固定，随时间向前滑动。当你认为旧数据相关性较低时（加密货币场景中常见），这种方法更适用。

Window 1: [===TRAIN===][=TEST=]
Window 2:    [===TRAIN===][=TEST=]
Window 3:       [===TRAIN===][=TEST=]

参数：

```
train_size
```
: 训练窗口包含的K线/天数
```
test_size
```
: 测试窗口包含的K线/天数
```
step_size
```
: 每次折叠前进的步长（通常等于
```
test_size
```
）

Expanding Window (Growing Train)

扩展窗口（训练集递增）

The train window starts at the beginning and expands forward. This uses all available historical data, which helps when data is scarce.

Window 1: [==TRAIN==][=TEST=]
Window 2: [====TRAIN====][=TEST=]
Window 3: [======TRAIN======][=TEST=]

Parameters:

```
min_train_size
```
: Minimum training samples before first fold
```
test_size
```
: Fixed test window size
```
step_size
```
: How far to advance between folds

训练窗口从起始位置开始，随时间向前扩展。这种方法会使用所有可用历史数据，在数据稀缺时更有帮助。

Window 1: [==TRAIN==][=TEST=]
Window 2: [====TRAIN====][=TEST=]
Window 3: [======TRAIN======][=TEST=]

参数：

```
min_train_size
```
: 首次折叠前所需的最小训练样本量
```
test_size
```
: 固定的测试窗口大小
```
step_size
```
: 每次折叠前进的步长

Choosing Between Them

方法选择

Factor	Rolling	Expanding
Data recency	Prioritizes recent data	Uses all history
Regime changes	Better adapts to new regimes	May dilute recent regime
Sample size	Fixed, may be small	Grows over time
Crypto preference	Preferred for < 6mo horizons	Better for regime-stable models

因素	滚动窗口	扩展窗口
数据时效性	优先使用近期数据	使用全部历史数据
状态变化	更好地适应新市场状态	可能稀释近期市场状态的影响
样本大小	固定，可能较小	随时间递增
加密货币场景偏好	适用于周期<6个月的策略	更适合状态稳定的模型

Purging and Embargo

清洗（Purging）与禁运（Embargo）

Purging

清洗

Remove training samples whose labels overlap with the test set's time range. If a label is computed as the 24h forward return starting at time

, any training sample where

t + 24h

extends into the test period must be purged.

python

def purge_train_indices(
    train_idx: list[int],
    test_start: int,
    label_horizon: int,
    timestamps: list[int],
) -> list[int]:
    """Remove train samples whose label windows overlap test period."""
    test_start_time = timestamps[test_start]
    return [
        i for i in train_idx
        if timestamps[i] + label_horizon < test_start_time
    ]

移除标签时间范围与测试集重叠的训练样本。如果标签是基于时间

开始的24小时远期收益计算的，那么任何满足

t + 24h

延伸至测试期的训练样本都必须被清洗。

python

def purge_train_indices(
    train_idx: list[int],
    test_start: int,
    label_horizon: int,
    timestamps: list[int],
) -> list[int]:
    """Remove train samples whose label windows overlap test period."""
    test_start_time = timestamps[test_start]
    return [
        i for i in train_idx
        if timestamps[i] + label_horizon < test_start_time
    ]

Embargo

禁运

Add a buffer gap between the end of training and start of testing to account for serial correlation that purging alone does not eliminate.

[===TRAIN===][--EMBARGO--][=TEST=]

Typical embargo sizes:

1-minute bars: 60–240 bars (1–4 hours)
5-minute bars: 12–48 bars (1–4 hours)
Hourly bars: 6–24 bars (6–24 hours)
Daily bars: 2–5 bars (2–5 days)
Crypto rule of thumb: Embargo >= 2x the label computation horizon

在训练集结束与测试集开始之间添加一个缓冲区间，以消除仅靠清洗无法解决的序列相关性问题。

[===TRAIN===][--EMBARGO--][=TEST=]

典型的禁运区间大小：

1分钟K线: 60–240根（1–4小时）
5分钟K线: 12–48根（1–4小时）
小时级K线: 6–24根（6–24小时）
日线K线: 2–5根（2–5天）
加密货币经验法则: 禁运区间 >= 2倍标签计算周期

Combinatorial Purged Cross-Validation (CPCV)

组合清洗交叉验证（CPCV）

CPCV (Lopez de Prado, 2018) generates all possible train/test combinations from

groups while maintaining temporal ordering. This produces far more test paths than standard walk-forward, enabling statistical tests for overfitting.

Key properties:

Splits data into
```
N
```
contiguous groups
For each combination of
```
k
```
test groups, the remaining
```
N-k
```
groups form the training set
Applies purging and embargo at each train/test boundary
Produces
```
C(N, k)
```
backtest paths (e.g., N=6, k=2 gives 15 paths)

See

references/methodology.md

for the full CPCV algorithm and formulas.

CPCV（Lopez de Prado, 2018）从

个分组中生成所有可能的训练/测试组合，同时保持时间顺序。与标准向前验证相比，它能生成更多测试路径，从而支持过拟合的统计检验。

核心特性：

将数据分割为
```
N
```
个连续分组
对于每
```
k
```
个测试分组的组合，剩余的
```
N-k
```
个分组构成训练集
在每个训练/测试边界应用清洗与禁运规则
生成
```
C(N, k)
```
条回测路径（例如N=6，k=2时可生成15条路径）

完整的CPCV算法与公式请参考

references/methodology.md

。

Overfit Detection

过拟合检测

Deflated Sharpe Ratio (DSR)

调整夏普比率（DSR）

The observed Sharpe ratio must be adjusted for:

Number of strategies tested (multiple testing)
Non-normality of returns (skewness, kurtosis)
Length of the backtest

python

import numpy as np
from scipy.stats import norm

def deflated_sharpe_ratio(
    observed_sr: float,
    num_trials: int,
    backtest_length: int,
    skewness: float = 0.0,
    kurtosis: float = 3.0,
) -> float:
    """Compute the probability that observed SR > 0 after deflation.

    Args:
        observed_sr: Annualized Sharpe ratio of the selected strategy.
        num_trials: Number of strategies tested (including discarded ones).
        backtest_length: Number of return observations.
        skewness: Skewness of returns.
        kurtosis: Excess kurtosis of returns.

    Returns:
        p-value (probability SR is genuinely > 0).
    """
    sr_std = np.sqrt(
        (1 - skewness * observed_sr + (kurtosis - 1) / 4 * observed_sr**2)
        / (backtest_length - 1)
    )
    # Expected max SR under null (Euler-Mascheroni approximation)
    euler_mascheroni = 0.5772156649
    expected_max_sr = norm.ppf(1 - 1 / num_trials) * (
        1 - euler_mascheroni
    ) + euler_mascheroni * norm.ppf(1 - 1 / (num_trials * np.e))
    dsr = norm.cdf((observed_sr - expected_max_sr) / sr_std)
    return dsr

A DSR below 0.95 suggests the observed performance is likely due to overfitting across the trials tested.

观测到的夏普比率（Sharpe Ratio）需要针对以下因素进行调整：

测试的策略数量（多重测试）
收益的非正态性（偏度、峰度）
回测时长

python

import numpy as np
from scipy.stats import norm

def deflated_sharpe_ratio(
    observed_sr: float,
    num_trials: int,
    backtest_length: int,
    skewness: float = 0.0,
    kurtosis: float = 3.0,
) -> float:
    """Compute the probability that observed SR > 0 after deflation.

    Args:
        observed_sr: Annualized Sharpe ratio of the selected strategy.
        num_trials: Number of strategies tested (including discarded ones).
        backtest_length: Number of return observations.
        skewness: Skewness of returns.
        kurtosis: Excess kurtosis of returns.

    Returns:
        p-value (probability SR is genuinely > 0).
    """
    sr_std = np.sqrt(
        (1 - skewness * observed_sr + (kurtosis - 1) / 4 * observed_sr**2)
        / (backtest_length - 1)
    )
    # Expected max SR under null (Euler-Mascheroni approximation)
    euler_mascheroni = 0.5772156649
    expected_max_sr = norm.ppf(1 - 1 / num_trials) * (
        1 - euler_mascheroni
    ) + euler_mascheroni * norm.ppf(1 - 1 / (num_trials * np.e))
    dsr = norm.cdf((observed_sr - expected_max_sr) / sr_std)
    return dsr

若DSR低于0.95，表明观测到的性能很可能是由测试过程中的过拟合导致的。

Probability of Backtest Overfitting (PBO)

回测过拟合概率（PBO）

PBO uses CPCV to measure the fraction of backtest paths where the in-sample optimal strategy underperforms the median out-of-sample. A PBO above 0.50 indicates more-likely-than-not overfitting.

See

references/overfit_detection.md

for complete derivations and implementation details.

PBO利用CPCV来衡量：在样本内表现最优的策略，在样本外表现低于中位数的回测路径占比。若PBO高于0.50，说明过拟合的可能性大于50%。

完整的推导与实现细节请参考

references/overfit_detection.md

。

Crypto-Specific Considerations

加密货币场景特有的注意事项

Shorter windows: Crypto regimes change faster than equities. A 90-day rolling window may be more appropriate than 252 days.
24/7 markets: No weekends or holidays to account for, but funding rate resets (every 8h on perps) create microstructure effects.
Survivorship bias: Many tokens delist. Validation must include delisted tokens or at minimum acknowledge this limitation.
Liquidity regime shifts: A token's liquidity profile can change dramatically (new CEX listing, liquidity mining end). Train/test splits should ideally not straddle major liquidity events.
Data availability: Many tokens have < 1 year of data. Expanding windows with small
```
min_train_size
```
may be necessary.

更短的窗口: 加密货币市场状态变化比股票更快。90天的滚动窗口可能比252天更合适。
7×24小时市场: 无需考虑周末或节假日，但永续合约的资金费率重置（每8小时一次）会产生微观结构效应。
生存偏差: 许多代币会被下架。验证过程必须包含已下架代币，或至少承认这一局限性。
流动性状态切换: 代币的流动性状况可能发生巨大变化（如上线新中心化交易所、流动性挖矿结束）。训练/测试分割应尽量避免跨越重大流动性事件。
数据可用性: 许多代币的历史数据不足1年。此时可能需要使用带有较小
```
min_train_size
```
的扩展窗口。

Practical Window Sizes for Crypto

加密货币场景的实用窗口大小

Strategy Timeframe	Train Window	Test Window	Embargo
Scalping (1-5min)	3-7 days	1 day	2-4 hours
Intraday (15min-1h)	14-30 days	3-7 days	12-24 hours
Swing (4h-daily)	30-90 days	7-14 days	2-5 days
Position (daily-weekly)	90-180 days	30 days	5-10 days

策略时间周期	训练窗口	测试窗口	禁运区间
高频 scalp（1-5分钟）	3-7天	1天	2-4小时
日内（15分钟-1小时）	14-30天	3-7天	12-24小时
波段（4小时-日线）	30-90天	7-14天	2-5天
持仓（日线-周线）	90-180天	30天	5-10天

Quick Start

快速开始

python

from walk_forward import WalkForwardValidator, WalkForwardConfig

config = WalkForwardConfig(
    train_size=90,
    test_size=14,
    step_size=14,
    window_type="rolling",
    embargo_size=3,
    purge_horizon=1,
)

validator = WalkForwardValidator(config)
for fold in validator.split(price_data):
    model.fit(fold.train_X, fold.train_y)
    predictions = model.predict(fold.test_X)
    fold.record_performance(predictions, fold.test_y)

results = validator.aggregate_results()
print(f"OOS Sharpe: {results.oos_sharpe:.3f}")
print(f"Train/Test Sharpe ratio: {results.sharpe_ratio_ratio:.2f}")

python

from walk_forward import WalkForwardValidator, WalkForwardConfig

config = WalkForwardConfig(
    train_size=90,
    test_size=14,
    step_size=14,
    window_type="rolling",
    embargo_size=3,
    purge_horizon=1,
)

validator = WalkForwardValidator(config)
for fold in validator.split(price_data):
    model.fit(fold.train_X, fold.train_y)
    predictions = model.predict(fold.test_X)
    fold.record_performance(predictions, fold.test_y)

results = validator.aggregate_results()
print(f"OOS Sharpe: {results.oos_sharpe:.3f}")
print(f"Train/Test Sharpe ratio: {results.sharpe_ratio_ratio:.2f}")

Files

文件说明

References

参考文档

```
references/methodology.md
```
— Walk-forward theory, window types, purging, embargo, CPCV algorithm with formulas
```
references/overfit_detection.md
```
— Deflated Sharpe ratio, probability of backtest overfitting, multiple testing corrections
```
references/practical_guide.md
```
— Window size selection for crypto, regime considerations, common validation mistakes

```
references/methodology.md
```
— 向前验证理论、窗口类型、清洗、禁运、CPCV算法及公式
```
references/overfit_detection.md
```
— 调整夏普比率、回测过拟合概率、多重测试校正
```
references/practical_guide.md
```
— 加密货币场景的窗口大小选择、状态考量、常见验证错误

Scripts

脚本文件

```
scripts/walk_forward.py
```
— Walk-forward validation engine with rolling and expanding windows;
```
--demo
```
mode with synthetic data
```
scripts/overfit_detector.py
```
— Deflated Sharpe ratio and PBO computation;
```
--demo
```
mode with synthetic backtest results

```
scripts/walk_forward.py
```
— 支持滚动与扩展窗口的向前验证引擎；
```
--demo
```
模式可使用合成数据
```
scripts/overfit_detector.py
```
— 调整夏普比率与PBO计算脚本；
```
--demo
```
模式可使用合成回测结果