An Experimental Study in Adaptive Kernel Selection for Bayesian Optimization
Open Access
- 18 December 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 7, 184294-184302
Abstract
Bayesian Optimization has been widely used along with Gaussian Processes for solving expensive-to-evaluate black-box optimization problems. Overall, this approach has shown good results, and particularly for parameter tuning of machine learning algorithms. Nonetheless, Bayesian Optimization has to be also configured to achieve the best possible performance, being the selection of the kernel function a crucial choice. This paper investigates the convenience of adaptively changing the kernel function during the optimization process, instead of fixing it a priori. Six adaptive kernel selection strategies are introduced and tested in well-known synthetic and real-world optimization problems. In order to provide a more complete evaluation of the proposed kernel selection variants, two major kernel parameter setting approaches have been tested. According to our results, apart from having the advantage of removing the selection of the kernel out of the equation, adaptive kernel selection criteria show a better performance than fixed-kernel approaches.Keywords
This publication has 16 references indexed in Scilit:
- Analysis of multi-objective Kriging-based methods for constrained global optimizationComputational Optimization and Applications, 2015
- Efficient global optimization algorithm assisted by multiple surrogate techniquesJournal of Global Optimization, 2012
- DiceKriging,DiceOptim: TwoRPackages for the Analysis of Computer Experiments by Kriging-Based Metamodeling and OptimizationJournal of Statistical Software, 2012
- Sequential Bayesian Prediction in the Presence of Changepoints and FaultsThe Computer Journal, 2010
- Discrete mixtures of kernels for Kriging‐based optimizationQuality and Reliability Engineering International, 2008
- A surrogate-model-based method for constrained optimization2000
- The Bayesian Approach to Local Optimization1989
- Algorithm 659ACM Transactions on Mathematical Software, 1988
- A New Method of Locating the Maximum Point of an Arbitrary Multipeak Curve in the Presence of NoiseJournal of Basic Engineering, 1964
- Use of Ranks in One-Criterion Variance AnalysisJournal of the American Statistical Association, 1952