---
title: Backtest overfitting in the machine learning era: A comparison of out-of-sample testing methods in a synthetic controlled environment
url: https://www.ml-quant.com/papers/ssrn/4686376/
site: ML-Quant (https://www.ml-quant.com)
updated: 2026-09-26
license: Summaries CC BY 4.0; links go to the original sources
index: https://www.ml-quant.com/llms.txt
identifier: SSRN 4686376
source_url: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4686376
featured: 2024-01-09
citations: 18
topic: Trading, Microstructure & Execution
---


# Backtest overfitting in the machine learning era: A comparison of out-of-sample testing methods in a synthetic controlled environment

The Combinatorial Purged Cross-Validation (CPCV) method is superior in financial analytics for reducing overfitting risks, outperforming traditional methods like K-Fold and Walk-Forward.

- Source: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4686376
- Identifier: SSRN 4686376
- Released: 2024-01-06
- First featured: Quant Letter No. 32 (2024-01-09): https://www.ml-quant.com/issues/2024-01-09/
- Citations (Semantic Scholar): 18
- Published in: Knowl. Based Syst.
- Topic: Trading, Microstructure & Execution

## Related

- [LIGHT Benchmark - Comprehensive Backtesting Framework for Market Risk Models Comparison](https://www.ml-quant.com/papers/ssrn/4586897/): Market Risk Backtesting: The article presents LIGHT Benchmark, a tool for comparing market risk models, including a scoring system for evaluating Value at Risk and Expected Shortfall models.
- [E-backtesting](https://www.ml-quant.com/papers/arxiv/2209.00991/): A new backtesting procedure for Expected Shortfall forecasts is proposed using e-values and e-processes.
- [Optimizing Portfolio Performance through Clustering and Sharpe Ratio-Based Optimization: A Comparative Backtesting Approach](https://www.ml-quant.com/papers/arxiv/2501.12074/): The article presents a new method for improving portfolio performance using clustering-based segmentation and Sharpe ratio-based optimization, tested with historical data from various asset classes.
- [The bias of IID resampled backtests for rolling window mean-variance portfolios](https://www.ml-quant.com/papers/arxiv/2505.06383/): A study finds that resampling techniques in backtests can cause bias in Sharpe Ratio estimates, suggesting a need for structure-preserving resampling methods.
- [Forecasting and backtesting gradient allocations of expected shortfall](https://www.ml-quant.com/papers/arxiv/2401.11701/): The study introduces a new semiparametric model for predicting dynamic Expected Shortfall contributions, which has shown excellent results in analyzing stock returns.
- [Backtestability and the Ridge Backtest](https://www.ml-quant.com/papers/repec/wsi-wschap-9789811280306-0003/): The article proposes a formal definition of backtestability for a statistical functional of a distribution and explores its connection with elicitability and identifiability.