Sanvesh Srivastava
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Sanvesh Srivastava has a total of 37 co-authors. Here are the top co-authors by number of shared publications:
- David Brian Dunson
Department of Statistical Science, Duke University
5 shared publications - Cheng Li
Department of Statistics and Data Science National University of Singapore
2 shared publications - Chuanhai Liu
Purdue University
2 shared publications - Glen DePalma
Department of Statistics, Purdue University, West Lafayette, IN
2 shared publications - Lizhen Lin
Department of Mathematics University of Maryland College Park
2 shared publications - Steven Winter
Steven Winter is Ph.D. Student, Department of Statistical Science, Duke University, Durham, North Carolina 27710, USA
2 shared publications
Latest Publications
- Open Access
Coordinated Brain-Network Dynamics of Maternal Adaptation
/Summary: The maternal brain exhibits impressive plasticity, undergoing many anatomical and functional alterations over the transition to parenting 1–3 . Such alterations can reflect maternal adaptation to offspring needs 4–6 , aligning with the onset and maintenance of maternal care behaviors 7–9 . These brain-wide neural changes occur across stages, starting during pregnancy and continuing through the peripartal transition 10,11 . We therefore wanted to understand brain changes with pregnancy, parenting, and care behaviors over time. Many brain regions have been implicated in the coordination of maternal adaptation and corresponding care behaviors 10,12,13 . We used computational modeling of multi-region neural oscillation data to identify distinct electrical networks during naturally-occurring care behaviors and across multiple maternal stages in outbred, highly maternal mice 14 ,. This work demonstrates that multi-region oscillatory electrical dynamics at high spatiotemporal resolution reflect these functions. Distinct, discriminative networks predict maternal behavior and stage, showcasing network adaptation with parental transition. Maternal behavior and stage also had substantial effects on other distinct, multi-region oscillatory electrical networks relevant to stress susceptibility 15 , further linking maternal adaptation to the neurophysiology of stress. While these stress networks were perturbed by maternal experience of early life stress, networks trained on maternal behavior and stage remained robust against such perturbation. This suggests a uniquely persistent strength of maternal electrical dynamics. Together, this work supports the role of oscillatory electrical dynamics as networks underlying fundamentally conserved behaviors and transitions, demonstrating wide relevance and significance to brain–behavior discovery.
Open AccessCoordinated Brain-Network Dynamics of Maternal Adaptation
/Summary: The maternal brain exhibits impressive plasticity, undergoing many anatomical and functional alterations over the transition to parenting 1–3 . Such alterations can reflect maternal adaptation to offspring needs 4–6 , aligning with the onset and maintenance of maternal care behaviors 7–9 . These brain-wide neural changes occur across stages, starting during pregnancy and continuing through the peripartal transition 10,11 . We therefore wanted to understand brain changes with pregnancy, parenting, and care behaviors over time. Many brain regions have been implicated in the coordination of maternal adaptation and corresponding care behaviors 10,12,13 . We used computational modeling of multi-region neural oscillation data to identify distinct electrical networks during naturally-occurring care behaviors and across multiple maternal stages in outbred, highly maternal mice 14 ,. This work demonstrates that multi-region oscillatory electrical dynamics at high spatiotemporal resolution reflect these functions. Distinct, discriminative networks predict maternal behavior and stage, showcasing network adaptation with parental transition. Maternal behavior and stage also had substantial effects on other distinct, multi-region oscillatory electrical networks relevant to stress susceptibility 15 , further linking maternal adaptation to the neurophysiology of stress. While these stress networks were perturbed by maternal experience of early life stress, networks trained on maternal behavior and stage remained robust against such perturbation. This suggests a uniquely persistent strength of maternal electrical dynamics. Together, this work supports the role of oscillatory electrical dynamics as networks underlying fundamentally conserved behaviors and transitions, demonstrating wide relevance and significance to brain–behavior discovery.
- Open Access
- 1 February 2024
- Published by Institute of Mathematical Statistics in Statistical Science
- Vol. 39 (1) , 62-89
Emerging Directions in Bayesian Computation
Bayesian models are powerful tools for studying complex data, allowing the analyst to encode rich hierarchical dependencies and leverage prior information. Most importantly, they facilitate a complete characterization of uncertainty through the posterior distribution. Practical posterior computation is commonly performed via MCMC, which can be computationally infeasible for high-dimensional models with many observations. In this article, we discuss the potential to improve posterior computation using ideas from machine learning. Concrete directions are explored in vignettes on normalizing flows, statistical properties of variational approximations, Bayesian coresets and distributed Bayesian inference.
- 1 February 2024
- Published by Institute of Mathematical Statistics in Statistical Science
- Vol. 39 (1) , 62-89
Open AccessEmerging Directions in Bayesian Computation
Bayesian models are powerful tools for studying complex data, allowing the analyst to encode rich hierarchical dependencies and leverage prior information. Most importantly, they facilitate a complete characterization of uncertainty through the posterior distribution. Practical posterior computation is commonly performed via MCMC, which can be computationally infeasible for high-dimensional models with many observations. In this article, we discuss the potential to improve posterior computation using ideas from machine learning. Concrete directions are explored in vignettes on normalizing flows, statistical properties of variational approximations, Bayesian coresets and distributed Bayesian inference.
- Open Access
Pseudo Bayesian Estimation of One-way ANOVA Model in Complex Surveys
We devise survey-weighted pseudo posterior distribution estimators under two-stage informative sampling of both primary clusters and secondary nested units for a one-way analysis of variance (ANOVA) population generating model as a simple canonical case where population model random effects are defined to be coincident with the primary clusters, for example student performance based on a survey of schools and students such as the 2000 OECD Programme for International Student Assessment (PISA). We consider estimation on an observed informative sample under both an augmented pseudo likelihood that co-samples the random effects, as well as an integrated likelihood that marginalizes out the random effects from the survey-weighted augmented pseudo likelihood. This paper includes a theoretical exposition that enumerates easily verified conditions for which estimation under the augmented pseudo posterior is guaranteed to be consistent at the true generating parameters. We reveal in simulation that both approaches produce asymptotically unbiased estimation of the generating hyperparameters for the random effects when a key condition on the sum of within cluster weighted residuals is met. We present a comparison with two frequentist alternatives, an expectation-maximization approach and a composite likelihood method that requires pairwise sampling weights.
Open AccessPseudo Bayesian Estimation of One-way ANOVA Model in Complex Surveys
We devise survey-weighted pseudo posterior distribution estimators under two-stage informative sampling of both primary clusters and secondary nested units for a one-way analysis of variance (ANOVA) population generating model as a simple canonical case where population model random effects are defined to be coincident with the primary clusters, for example student performance based on a survey of schools and students such as the 2000 OECD Programme for International Student Assessment (PISA). We consider estimation on an observed informative sample under both an augmented pseudo likelihood that co-samples the random effects, as well as an integrated likelihood that marginalizes out the random effects from the survey-weighted augmented pseudo likelihood. This paper includes a theoretical exposition that enumerates easily verified conditions for which estimation under the augmented pseudo posterior is guaranteed to be consistent at the true generating parameters. We reveal in simulation that both approaches produce asymptotically unbiased estimation of the generating hyperparameters for the random effects when a key condition on the sum of within cluster weighted residuals is met. We present a comparison with two frequentist alternatives, an expectation-maximization approach and a composite likelihood method that requires pairwise sampling weights.
- Open Access
Machine Learning and the Future of Bayesian Computation
Bayesian models are a powerful tool for studying complex data, allowing the analyst to encode rich hierarchical dependencies and leverage prior information. Most importantly, they facilitate a complete characterization of uncertainty through the posterior distribution. Practical posterior computation is commonly performed via MCMC, which can be computationally infeasible for high dimensional models with many observations. In this article we discuss the potential to improve posterior computation using ideas from machine learning. Concrete future directions are explored in vignettes on normalizing flows, Bayesian coresets, distributed Bayesian inference, and variational inference.
Open AccessMachine Learning and the Future of Bayesian Computation
Bayesian models are a powerful tool for studying complex data, allowing the analyst to encode rich hierarchical dependencies and leverage prior information. Most importantly, they facilitate a complete characterization of uncertainty through the posterior distribution. Practical posterior computation is commonly performed via MCMC, which can be computationally infeasible for high dimensional models with many observations. In this article we discuss the potential to improve posterior computation using ideas from machine learning. Concrete future directions are explored in vignettes on normalizing flows, Bayesian coresets, distributed Bayesian inference, and variational inference.
- Open Access
Gaussian process regression and classification using International Classification of Disease codes as covariates
In electronic health records (EHRs) data analysis, nonparametric regression and classification using International Classification of Disease (ICD) codes as covariates remain understudied. Automated methods have been developed over the years for predicting biomedical responses using EHRs, but relatively less attention has been paid to developing patient similarity measures that use ICD codes and chronic conditions, where a chronic condition is defined as a set of ICD codes. We address this problem by first developing a string kernel function for measuring the similarity between a pair of primary chronic conditions, represented as subsets of ICD codes. Second, we extend this similarity measure to a family of covariance functions on subsets of chronic conditions. This family is used in developing Gaussian process (GP) priors for Bayesian nonparametric regression and classification using diagnoses and other demographic information as covariates. Markov chain Monte Carlo (MCMC) algorithms are used for posterior inference and predictions. The proposed methods are tuning free, so they are ideal for automated prediction of biomedical responses depending on chronic conditions. We evaluate the practical performance of our method on EHR data collected from 1660 patients at the University of Iowa Hospitals and Clinics (UIHC) with six different primary cancer sites. Our method provides better sensitivity and specificity than its competitors in classifying different primary cancer sites and estimates the marginal associations between chronic conditions and primary cancer sites.
Open AccessGaussian process regression and classification using International Classification of Disease codes as covariates
In electronic health records (EHRs) data analysis, nonparametric regression and classification using International Classification of Disease (ICD) codes as covariates remain understudied. Automated methods have been developed over the years for predicting biomedical responses using EHRs, but relatively less attention has been paid to developing patient similarity measures that use ICD codes and chronic conditions, where a chronic condition is defined as a set of ICD codes. We address this problem by first developing a string kernel function for measuring the similarity between a pair of primary chronic conditions, represented as subsets of ICD codes. Second, we extend this similarity measure to a family of covariance functions on subsets of chronic conditions. This family is used in developing Gaussian process (GP) priors for Bayesian nonparametric regression and classification using diagnoses and other demographic information as covariates. Markov chain Monte Carlo (MCMC) algorithms are used for posterior inference and predictions. The proposed methods are tuning free, so they are ideal for automated prediction of biomedical responses depending on chronic conditions. We evaluate the practical performance of our method on EHR data collected from 1660 patients at the University of Iowa Hospitals and Clinics (UIHC) with six different primary cancer sites. Our method provides better sensitivity and specificity than its competitors in classifying different primary cancer sites and estimates the marginal associations between chronic conditions and primary cancer sites.
- 8 November 2022
- Published by Taylor & Francis in Journal of Computational and Graphical Statistics
- Vol. 32 (3) , 895-907
Asynchronous and Distributed Data Augmentation for Massive Data Settings
Data augmentation (DA) algorithms are slow in massive data settings due to multiple passes through the entire data. We address this problem by developing a DA extension that exploits asynchronous and distributed computing. The extended DA algorithm is called Asynchronous and Distributed (AD) DA with the original DA as its parent. Any ADDA is indexed by a parameter and starts by dividing the entire data into k disjoint subsets and storing them on k processes. Every iteration of ADDA augments only an r-fraction of the k data subsets with some positive probability and leaves the remaining -fraction of the augmented data unchanged. The parameter draws are obtained using the r-fraction of new and -fraction of old augmented data. We show that the ADDA Markov chain is Harris ergodic with the desired stationary distribution under mild conditions on the parent DA algorithm. We demonstrate that ADDA is significantly faster than its parent for many (k, r) choices in three representative models. We also establish the geometric ergodicity of the ADDA Markov chain for all the three models, which yields asymptotically valid standard errors for estimates of desired posterior quantities.- 8 November 2022
- Published by Taylor & Francis in Journal of Computational and Graphical Statistics
- Vol. 32 (3) , 895-907
Asynchronous and Distributed Data Augmentation for Massive Data Settings
Data augmentation (DA) algorithms are slow in massive data settings due to multiple passes through the entire data. We address this problem by developing a DA extension that exploits asynchronous and distributed computing. The extended DA algorithm is called Asynchronous and Distributed (AD) DA with the original DA as its parent. Any ADDA is indexed by a parameter and starts by dividing the entire data into k disjoint subsets and storing them on k processes. Every iteration of ADDA augments only an r-fraction of the k data subsets with some positive probability and leaves the remaining -fraction of the augmented data unchanged. The parameter draws are obtained using the r-fraction of new and -fraction of old augmented data. We show that the ADDA Markov chain is Harris ergodic with the desired stationary distribution under mild conditions on the parent DA algorithm. We demonstrate that ADDA is significantly faster than its parent for many (k, r) choices in three representative models. We also establish the geometric ergodicity of the ADDA Markov chain for all the three models, which yields asymptotically valid standard errors for estimates of desired posterior quantities.An algorithm for distributed Bayesian inference
Monte Carlo algorithms, such as Markov chain Monte Carlo (MCMC) and Hamiltonian Monte Carlo (HMC), are routinely used for Bayesian inference; however, these algorithms are prohibitively slow in massive data settings because they require multiple passes through the full data in every iteration. Addressing this problem, we develop a scalable extension of these algorithms using the divide‐and‐conquer (D&C) technique that divides the data into a sufficiently large number of subsets, draws parameters in parallel on the subsets using a powered likelihood and produces Monte Carlo draws of the parameter by combining parameter draws obtained from each subset. The combined parameter draws play the role of draws from the original sampling algorithm. Our main contributions are twofold. First, we demonstrate through diverse simulated and real data analyses focusing on generalized linear models (GLMs) that our distributed algorithm delivers comparable results as the current state‐of‐the‐art D&C algorithms in terms of statistical accuracy and computational efficiency. Second, providing theoretical support for our empirical observations, we identify regularity assumptions under which the proposed algorithm leads to asymptotically optimal inference. We also provide illustrative examples focusing on normal linear and logistic regressions where parts of our D&C algorithm are analytically tractable.
An algorithm for distributed Bayesian inference
Monte Carlo algorithms, such as Markov chain Monte Carlo (MCMC) and Hamiltonian Monte Carlo (HMC), are routinely used for Bayesian inference; however, these algorithms are prohibitively slow in massive data settings because they require multiple passes through the full data in every iteration. Addressing this problem, we develop a scalable extension of these algorithms using the divide‐and‐conquer (D&C) technique that divides the data into a sufficiently large number of subsets, draws parameters in parallel on the subsets using a powered likelihood and produces Monte Carlo draws of the parameter by combining parameter draws obtained from each subset. The combined parameter draws play the role of draws from the original sampling algorithm. Our main contributions are twofold. First, we demonstrate through diverse simulated and real data analyses focusing on generalized linear models (GLMs) that our distributed algorithm delivers comparable results as the current state‐of‐the‐art D&C algorithms in terms of statistical accuracy and computational efficiency. Second, providing theoretical support for our empirical observations, we identify regularity assumptions under which the proposed algorithm leads to asymptotically optimal inference. We also provide illustrative examples focusing on normal linear and logistic regressions where parts of our D&C algorithm are analytically tractable.
- 8 March 2021
- Published by Taylor & Francis in Journal of Computational and Graphical Statistics
- Vol. 30 (3) , 594-611
Distributed Bayesian Inference in Linear Mixed-Effects Models
Linear mixed-effects models play a fundamental role in statistical methodology. A variety of Markov chain Monte Carlo (MCMC) algorithms exist for fitting these models, but they are inefficient in massive data settings because every iteration of any such MCMC algorithm passes through the full data. Many divide-and-conquer methods have been proposed to solve this problem, but they lack theoretical guarantees, impose restrictive assumptions, or have complex computational algorithms. Our focus is one such method called the Wasserstein Posterior (WASP), which has become popular due to its optimal theoretical properties under general assumptions. Unfortunately, practical implementation of the WASP either requires solving a complex linear program or is limited to one-dimensional parameters. The former method is inefficient and the latter method fails to capture the joint posterior dependence structure of multivariate parameters. We develop a new algorithm for computing the WASP of multivariate parameters that is easy to implement and is useful for computing the WASP in any model where the posterior distribution of parameter belongs to a location-scatter family of probability measures. The algorithm is introduced for linear mixed-effects models with both implementation details and theoretical properties. Our algorithm outperforms the current state-of-the-art method in inference on the functions of the covariance matrix of the random effects across diverse numerical comparisons. Supplemental materials for this article are available online.
- 8 March 2021
- Published by Taylor & Francis in Journal of Computational and Graphical Statistics
- Vol. 30 (3) , 594-611
Distributed Bayesian Inference in Linear Mixed-Effects Models
Linear mixed-effects models play a fundamental role in statistical methodology. A variety of Markov chain Monte Carlo (MCMC) algorithms exist for fitting these models, but they are inefficient in massive data settings because every iteration of any such MCMC algorithm passes through the full data. Many divide-and-conquer methods have been proposed to solve this problem, but they lack theoretical guarantees, impose restrictive assumptions, or have complex computational algorithms. Our focus is one such method called the Wasserstein Posterior (WASP), which has become popular due to its optimal theoretical properties under general assumptions. Unfortunately, practical implementation of the WASP either requires solving a complex linear program or is limited to one-dimensional parameters. The former method is inefficient and the latter method fails to capture the joint posterior dependence structure of multivariate parameters. We develop a new algorithm for computing the WASP of multivariate parameters that is easy to implement and is useful for computing the WASP in any model where the posterior distribution of parameter belongs to a location-scatter family of probability measures. The algorithm is introduced for linear mixed-effects models with both implementation details and theoretical properties. Our algorithm outperforms the current state-of-the-art method in inference on the functions of the covariance matrix of the random effects across diverse numerical comparisons. Supplemental materials for this article are available online.
- Open Access
A Divide-and-Conquer Bayesian Approach to Large-Scale Kriging
We propose a three-step divide-and-conquer strategy within the Bayesian paradigm that delivers massive scalability for any spatial process model. We partition the data into a large number of subsets, apply a readily available Bayesian spatial process model on every subset, in parallel, and optimally combine the posterior distributions estimated across all the subsets into a pseudo-posterior distribution that conditions on the entire data. The combined pseudo posterior distribution replaces the full data posterior distribution for predicting the responses at arbitrary locations and for inference on the model parameters and spatial surface. Based on distributed Bayesian inference, our approach is called "Distributed Kriging" (DISK) and offers significant advantages in massive data applications where the full data are stored across multiple machines. We show theoretically that the Bayes $L_2$-risk of the DISK posterior distribution achieves the near optimal convergence rate in estimating the true spatial surface with various types of covariance functions, and provide upper bounds for the number of subsets as a function of the full sample size. The model-free feature of DISK is demonstrated by scaling posterior computations in spatial process models with a stationary full-rank and a nonstationary low-rank Gaussian process (GP) prior. A variety of simulations and a geostatistical analysis of the Pacific Ocean sea surface temperature data validate our theoretical results.
Open AccessA Divide-and-Conquer Bayesian Approach to Large-Scale Kriging
We propose a three-step divide-and-conquer strategy within the Bayesian paradigm that delivers massive scalability for any spatial process model. We partition the data into a large number of subsets, apply a readily available Bayesian spatial process model on every subset, in parallel, and optimally combine the posterior distributions estimated across all the subsets into a pseudo-posterior distribution that conditions on the entire data. The combined pseudo posterior distribution replaces the full data posterior distribution for predicting the responses at arbitrary locations and for inference on the model parameters and spatial surface. Based on distributed Bayesian inference, our approach is called "Distributed Kriging" (DISK) and offers significant advantages in massive data applications where the full data are stored across multiple machines. We show theoretically that the Bayes $L_2$-risk of the DISK posterior distribution achieves the near optimal convergence rate in estimating the true spatial surface with various types of covariance functions, and provide upper bounds for the number of subsets as a function of the full sample size. The model-free feature of DISK is demonstrated by scaling posterior computations in spatial process models with a stationary full-rank and a nonstationary low-rank Gaussian process (GP) prior. A variety of simulations and a geostatistical analysis of the Pacific Ocean sea surface temperature data validate our theoretical results.
- 14 November 2018
- Published by Taylor & Francis in Journal of Computational and Graphical Statistics
- Vol. 28 (2) , 233-243
An Asynchronous Distributed Expectation Maximization Algorithm for Massive Data: The DEM Algorithm
The family of Expectation-Maximization (EM) algorithms provides a general approach to fitting flexible models for large and complex data. The expectation (E) step of EM-type algorithms is time consuming in massive data applications because it requires multiple passes through the full data. We address this problem by proposing an asynchronous and distributed generalization of the EM called the Distributed EM (DEM). Using DEM, existing EM-type algorithms are easily extended to massive data settings by exploiting the divide-and-conquer technique and widely available computing power, such as grid computing. The DEM algorithm reserves two groups of computing processes called workers and managers for performing the E step and the maximization step (M step), respectively. The samples are randomly partitioned into a large number of disjoint subsets and are stored on the worker processes. The E step of DEM algorithm is performed in parallel on all the workers, and every worker communicates its results to the managers at the end of local E step. The managers perform the M step after they have received results from a γ-fraction of the workers, where γ is a fixed constant in (0, 1]. The sequence of parameter estimates generated by the DEM algorithm retains the attractive properties of EM: convergence of the sequence of parameter estimates to a local mode and linear global rate of convergence. Across diverse simulations focused on linear mixed-effects models, the DEM algorithm is significantly faster than competing EM-type algorithms while having a similar accuracy. The DEM algorithm maintains its superior empirical performance on a movie ratings database consisting of 10 million ratings.
- 14 November 2018
- Published by Taylor & Francis in Journal of Computational and Graphical Statistics
- Vol. 28 (2) , 233-243
An Asynchronous Distributed Expectation Maximization Algorithm for Massive Data: The DEM Algorithm
The family of Expectation-Maximization (EM) algorithms provides a general approach to fitting flexible models for large and complex data. The expectation (E) step of EM-type algorithms is time consuming in massive data applications because it requires multiple passes through the full data. We address this problem by proposing an asynchronous and distributed generalization of the EM called the Distributed EM (DEM). Using DEM, existing EM-type algorithms are easily extended to massive data settings by exploiting the divide-and-conquer technique and widely available computing power, such as grid computing. The DEM algorithm reserves two groups of computing processes called workers and managers for performing the E step and the maximization step (M step), respectively. The samples are randomly partitioned into a large number of disjoint subsets and are stored on the worker processes. The E step of DEM algorithm is performed in parallel on all the workers, and every worker communicates its results to the managers at the end of local E step. The managers perform the M step after they have received results from a γ-fraction of the workers, where γ is a fixed constant in (0, 1]. The sequence of parameter estimates generated by the DEM algorithm retains the attractive properties of EM: convergence of the sequence of parameter estimates to a local mode and linear global rate of convergence. Across diverse simulations focused on linear mixed-effects models, the DEM algorithm is significantly faster than competing EM-type algorithms while having a similar accuracy. The DEM algorithm maintains its superior empirical performance on a movie ratings database consisting of 10 million ratings.