Researchers Miquel Noguer i Alonso from New York’s Artificial Intelligence in Finance Institute and Sonam Srivastava from Wright Research examine a critical question in finance: the use of large nonlinear over-parametrized models or simpler models to forecast financial time series and the balance between underfitting and overfitting, the bias-variance trade-off, and the absolute performance in the test set.
The traditional shape of the performance curve was U-shaped due to a bias-variance trade-off. Still, some recent research has pointed out that the performance curve may have a double descent shape in some specific domains. They discuss some of the recent discoveries in the mathematical theory of machine learning that reduce the gap between theory and practice and conduct experiments in the financial time series domain using deep neural networks and tree ensembles: random forests and XGBoost from under to over-parametrized.
The performance function doesn’t show a U-shape or a double descent shape but a flat profile, meaning that larger models have the same performance in the test set than smaller models. However, the training error function shows a descent profile consistent with the idea that while training error can be very low when we increase the models’ dimensionality, the test error is more stable in the equity financial time series domain. This is consistent with the finance practitioner’s theory that backtesting frequently overestimates the test or real-life performance in financial time series prediction. The irreducible error limits the prediction performance.
In the conclusion, researchers note that: “…the training error function shows a descent profile consistent with the
idea that while training error can be deficient when we increase dimensionality, test error is essentially more stable. This experimental result tells us that overfitting might not be a consequence of the dimensionality of the algorithm but comes from the fundamental or irreducible error and the potential non-stationarity nature of financial time series.”