A Comprehensive Study on K ‐Fold Cross‐Validation for Hyperparameter Tuning of Deep Neural Network
Authors:
- Siwaporn KunnapapdeelertSKSiwaporn Kunnapapdeelert
- Tossawat ChantanomTCTossawat Chantanom
- Nicharee ManakitrungruengNMNicharee Manakitrungrueng
- Nguyen Trong Tri DucNDNguyen Trong Tri Duc
- Jirachai BuddhakulsomsiriJBJirachai Buddhakulsomsiri
- Pornprom AteetananPAPornprom Ateetanan
- Pham Duc TaiPTPham Duc Tai
Open Access
- 1 January 2026
- journal article
- Published by Wiley in Journal of Engineering
- Vol. 2026 (1)
Abstract
Nested cross‐validation is widely used for evaluating hyperparameter‐tuning strategies, but its effectiveness depends strongly on the choice of fold numbers, particularly the outer and inner. This issue is especially important in deep neural network tuning, where performance estimates may be sensitive to both data partitioning and hyperparameter search strategy. This paper investigates the effect of different outer‐ and inner‐fold values in nested k ‐fold cross‐validation for neural network models in both regression and classification tasks. Furthermore, a design of experiments approach, specifically the Plackett–Burman design, is employed as a screening baseline, and Bayesian optimization is used as an adaptive search strategy to generate candidate hyperparameter configurations for comparison under nested cross‐validation. By comparing these methodologies for hyperparameter tuning and model optimization, the study is aimed at providing practical guidance for improving the performance and reliability of hyperparameter tuning under nested cross‐validation. The results show that within the context of this study, predictive performance tends to improve as the inner‐fold value increases, although the gains become smaller at higher values and require substantially greater computational cost. Statistical analysis based on paired outer‐fold results demonstrates that Bayesian optimization significantly outperforms the Plackett–Burman design for white wine regression and red wine classification across all experimented inner‐fold values. For the obesity dataset, the relative advantage depends on the value of the inner fold. In particular, Bayesian optimization performs better at low inner‐fold values; the two methods are statistically not different at intermediate values; and the Plackett–Burman design performs better at larger inner‐fold values.Keywords
This publication has 35 references indexed in Scilit:
- Integration of nested cross-validation, automated hyperparameter optimization, high-performance computing to reduce and quantify the variance of test performance estimation of deep learning modelsComputer Methods and Programs in Biomedicine, 2025
- Reducing cross-validation variance through seed blocking in hyperparameter tuningMachine Learning, 2025
- The usage of 10-fold cross-validation and grid search to enhance ML methods performance in solar farm power generation predictionCleaner Engineering and Technology, 2023
- Modeling dissolved oxygen concentration using machine learning techniques with dimensionality reduction approachEnvironmental Monitoring and Assessment, 2023
- Overview and cross-validation of COVID-19 forecasting univariate modelsAlexandria Engineering Journal, 2021
- What is an optimal value of k in k-fold cross-validation in discrete Bayesian network analysis?Computational Statistics, 2020
- Evaluating explorative prediction power of machine learning algorithms for materials discovery using-fold forward cross-validationComputational Materials Science, 2020
- Prediction of compressive strength and portland cement composition using cross-validation and feature ranking techniquesConstruction and Building Materials, 2019
- Combination of Support Vector Machine and K-Fold cross validation to predict compressive strength of concrete in marine environmentConstruction and Building Materials, 2019
- Estimates of residential building energy consumption using a multi-verse optimizer-based support vector machine with k-fold cross-validationEvolving Systems, 2019