Bayesian Smoothing and Feature Selection Using variational Automatic Relevance Determination
Preprint
Abstract
This study introduces Variational Automatic Relevance Determination (VARD), a novel approach tailored for fitting sparse additive regression models in high-dimensional settings. VARD distinguishes itself by its ability to independently assess the smoothness of each feature while enabling precise determination of whether a feature's contribution to the response is zero, linear, or nonlinear. Further, an efficient coordinate descent algorithm is introduced to implement VARD. Empirical evaluations on simulated and real-world data underscore VARD's superiority over alternative variable selection methods for additive models.Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- Bayesian generalized additive model selection including a fast variational optionAStA Advances in Statistical Analysis, 2023
- Variational Bayes for High-Dimensional Linear Regression With Sparse PriorsJournal of the American Statistical Association, 2021
- Practical variable selection for generalized additive modelsComputational Statistics & Data Analysis, 2011
- Fast Stable Restricted Maximum Likelihood and Marginal Likelihood Estimation of Semiparametric Generalized Linear ModelsJournal of the Royal Statistical Society Series B: Statistical Methodology, 2010
- Sparse Additive ModelsJournal of the Royal Statistical Society Series B: Statistical Methodology, 2009
- Fast Stable Direct Fitting and Smoothness Selection for Generalized Additive ModelsJournal of the Royal Statistical Society Series B: Statistical Methodology, 2008
- Component selection and smoothing in multivariate nonparametric regressionThe Annals of Statistics, 2006
- Model Selection and Estimation in Regression with Grouped VariablesJournal of the Royal Statistical Society Series B: Statistical Methodology, 2005
- Stable and Efficient Multiple Smoothing Parameter Estimation for Generalized Additive ModelsJournal of the American Statistical Association, 2004
- Modelling and Smoothing Parameter Estimation With Multiple Quadratic PenaltiesJournal of the Royal Statistical Society Series B: Statistical Methodology, 2000