Merging two cultures: Deep and statistical learning
Open Access
- 13 March 2024
- journal article
- review article
- Published by Wiley in WIREs Computational Statistics
- Vol. 16 (2)
Abstract
Our goal is to provide a review of deep learning methods which provide insight into structured high‐dimensional data. Merging the two cultures of algorithmic and statistical learning sheds light on model construction and improved prediction and inference, leveraging the duality and trade‐off between the two. Prediction, interpolation, and uncertainty quantification can be achieved using probabilistic methods at the output layer of the model. Rather than using shallow additive architectures common to most statistical models, deep learning uses layers of semi‐affine input transformations to provide a predictive rule. Applying these layers of transformations leads to a set of attributes (or, features) to which probabilistic statistical methods can be applied. Thus, the best of both worlds can be achieved: scalable prediction rules fortified with uncertainty quantification where sparse regularization finds the features. We review the duality between shallow and wide models such as principal components regression, and partial least squares and deep but skinny architectures such as autoencoders, multilayer perceptrons, convolutional neural net, and recurrent neural net. The connection with data transformations is of practical importance for finding good network architectures. By incorporating probabilistic components at the output level, the predictive uncertainty is allowed. We illustrate this idea by comparing plain Gaussian processes (GP) with partial least squares + Gaussian process (PLS + GP) and deep learning + Gaussian process (DL + GP).This article is categorized under: Statistical Learning and Exploratory Methods of the Data Sciences > Deep LearningKeywords
All Related Versions
This publication has 53 references indexed in Scilit:
- Active Learning Methods with Deep Gaussian Processes2018
- Deep learning for spatio‐temporal modeling: Dynamic traffic flows and high frequency tradingApplied Stochastic Models in Business and Industry, 2018
- Big data and partial least‐squares predictionCanadian Journal of Statistics, 2017
- Why and when can deep-but not shallow-networks avoid the curse of dimensionality: A reviewInternational Journal of Automation and Computing, 2017
- Deep learning for finance: deep portfoliosApplied Stochastic Models in Business and Industry, 2016
- U-Net: Convolutional Networks for Biomedical Image Segmentation2015
- A Generalized Linear Model With “Gaussian” Regressor Variables2011
- The Elements of Statistical Learning2009
- Global sensitivity analysis for a numerical model of radionuclide migration from the RRC “Kurchatov Institute” radwaste disposal siteStochastic Environmental Research and Risk Assessment, 2006
- Model-Oriented Design of Experiments1997