Rethinking Bayesian Learning for Data Analysis: The art of prior and inference in sparsity-aware modeling
Top Cited Papers
- 27 October 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Signal Processing Magazine
- Vol. 39 (6) , 18-52
Abstract
Sparse modeling for signal processing and machine learning, in general, has been at the focus of scientific research for over two decades. Among others, supervised sparsity-aware learning (SAL) consists of two major paths paved by 1) discriminative methods that establish direct input–output mapping based on a regularized cost function optimization and 2) generative methods that learn the underlying distributions. The latter, more widely known as Bayesian methods , enable uncertainty evaluation with respect to the performed predictions. Furthermore, they can better exploit related prior information and also, in principle, can naturally introduce robustness into the model, due to their unique capacity to marginalize out uncertainties related to the parameter estimates. Moreover, hyperparameters (tuning parameters) associated with the adopted priors, which correspond to cost function regularizers, can be learned via the training data and not via costly cross-validation techniques, which is, in general, the case with the discriminative methods.Keywords
All Related Versions
Funding Information
- National Natural Science Foundation of China (62001309)
- Science and Technology on Sonar Laboratory (6142109KF212204)
- National Natural Science Foundation of China (62271433)
- Shenzhen Science and Technology Program (JCYJ20190813171003723)
- National Natural Science Foundation of China (62071409, 61731018)
- European Union’s Horizon 2020 Research and Innovation Program (872139)
This publication has 54 references indexed in Scilit:
- A Unified Convergence Analysis of Block Successive Minimization Methods for Nonsmooth OptimizationSIAM Journal on Optimization, 2013
- From $K$-Means to Higher-Way Co-Clustering: Multilinear Decomposition With Sparse Latent FactorsIEEE Transactions on Signal Processing, 2012
- Distributed Optimization and Statistical Learning via the Alternating Direction Method of MultipliersFoundations and Trends in Machine Learning, 2011
- The Quest for a Dictionary2010
- Sparse Bayesian Learning for Basis SelectionIEEE Transactions on Signal Processing, 2004
- The bootstrap and its application in signal processingIEEE Signal Processing Magazine, 1998
- Long Short-Term MemoryNeural Computation, 1997
- Bayesian Inductive Inference and Maximum Entropy1988
- A generalized probability density function for double-bounded random processesJournal of Hydrology, 1980
- Estimating the Dimension of a ModelThe Annals of Statistics, 1978