Joint Optimization and Variable Selection of High-dimensional Gaussian Processes
Preprint
Abstract
Maximizing high-dimensional, non-convex functions through noisy observations is a notoriously hard problem, but one that arises in many applications. In this paper, we tackle this challenge by modeling the unknown function as a sample from a high-dimensional Gaussian process (GP) distribution. Assuming that the unknown function only depends on few relevant variables, we show that it is possible to perform joint variable selection and GP optimization. We provide strong performance guarantees for our algorithm, bounding the sample complexity of variable selection, and as well as providing cumulative regret bounds. We further provide empirical evidence on the effectiveness of our algorithm on several benchmark optimization problems.Keywords
All Related Versions
This publication has 5 references indexed in Scilit:
- Distilled Sensing: Adaptive Sampling for Sparse Detection and EstimationIEEE Transactions on Information Theory, 2011
- Rodeo: Sparse, greedy nonparametric regressionThe Annals of Statistics, 2008
- For most large underdetermined systems of linear equations the minimal 𝓁1‐norm solution is also the sparsest solutionCommunications on Pure and Applied Mathematics, 2006
- Regression Shrinkage and Selection Via the LassoJournal of the Royal Statistical Society Series B: Statistical Methodology, 1996
- Bayesian InterpolationNeural Computation, 1992