Sohom Bhattacharya
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Sohom Bhattacharya has a total of 15 co-authors. Here are the top co-authors by number of shared publications:
- Rajarshi Mukherjee
Department of Biostatistics Harvard University
2 shared publications - Sourav Chatterjee
Department of Statistics, Stanford University, Stanford, USA
2 shared publications - Subhabrata Sen
Department of Statistics, Harvard University, Cambridge, USA
2 shared publications Amir Dembo
Department of Mathematics Stanford University Stanford California USA
1 shared publication- Amir Dembo
Department of Statistics and Department of Mathematics, Stanford University, Stanford, USA
1 shared publication - Bhaswar B. Bhattacharya
Department of Statistics and Data Science, University of Pennsylvania, Philadelphia, USA
1 shared publication
Latest Publications
- 18 May 2026
- Published by Taylor & Francis in Journal of the American Statistical Association
- p. 1-29
Inferences on mixing probabilities and ranking in mixed-membership models
1 IntroductionIn various fields of study, such as citation networks, protein interactions, health, finance, trade, and social networks, we often come across large amounts of data that describe the ...
- 18 May 2026
- Published by Taylor & Francis in Journal of the American Statistical Association
- p. 1-29
Inferences on mixing probabilities and ranking in mixed-membership models
1 IntroductionIn various fields of study, such as citation networks, protein interactions, health, finance, trade, and social networks, we often come across large amounts of data that describe the ...
- Open Access
Causal inference under interference: computational barriers and algorithmic solutions
We study causal effect estimation under interference from network data. We work under the chain-graph formulation pioneered in Tchetgen Tchetgen et. al (2021). Our first result shows that polynomial time evaluation of treatment effects is computationally hard in this framework without additional assumptions on the underlying chain graph. Subsequently, we assume that the interactions among the study units are governed either by (i) a dense graph or (ii) an i.i.d. Gaussian matrix. In each case, we show that the treatment effects have well-defined limits as the population size diverges to infinity. Additionally, we develop polynomial time algorithms to consistently evaluate the treatment effects in each case. Finally, we estimate the unknown parameters from the observed data using maximum pseudo-likelihood estimates, and establish the stability of our causal effect estimators under this perturbation. Our algorithms provably approximate the causal effects in polynomial time even in low-temperature regimes where the canonical MCMC samplers are slow mixing. For dense graphs, our results use the notion of regularity partitions; for Gaussian interactions, our approach uses ideas from spin glass theory and Approximate Message Passing.
Open AccessCausal inference under interference: computational barriers and algorithmic solutions
We study causal effect estimation under interference from network data. We work under the chain-graph formulation pioneered in Tchetgen Tchetgen et. al (2021). Our first result shows that polynomial time evaluation of treatment effects is computationally hard in this framework without additional assumptions on the underlying chain graph. Subsequently, we assume that the interactions among the study units are governed either by (i) a dense graph or (ii) an i.i.d. Gaussian matrix. In each case, we show that the treatment effects have well-defined limits as the population size diverges to infinity. Additionally, we develop polynomial time algorithms to consistently evaluate the treatment effects in each case. Finally, we estimate the unknown parameters from the observed data using maximum pseudo-likelihood estimates, and establish the stability of our causal effect estimators under this perturbation. Our algorithms provably approximate the causal effects in polynomial time even in low-temperature regimes where the canonical MCMC samplers are slow mixing. For dense graphs, our results use the notion of regularity partitions; for Gaussian interactions, our approach uses ideas from spin glass theory and Approximate Message Passing.
- Open Access
Late Fusion Multi-task Learning for Semiparametric Inference with Nuisance Parameters
In the age of large and heterogeneous datasets, the integration of information from diverse sources is essential to improve parameter estimation. Multi-task learning offers a powerful approach by enabling simultaneous learning across related tasks. In this work, we introduce a late fusion framework for multi-task learning with semiparametric models that involve infinite-dimensional nuisance parameters, focusing on applications such as heterogeneous treatment effect estimation across multiple data sources, including electronic health records from different hospitals or clinical trial data. Our framework is two-step: first, initial double machine-learning estimators are obtained through individual task learning; second, these estimators are adaptively aggregated to exploit task similarities while remaining robust to task-specific differences. In particular, the framework avoids individual level data sharing, preserving privacy. Additionally, we propose a novel multi-task learning method for nuisance parameter estimation, which further enhances parameter estimation when nuisance parameters exhibit similarity across tasks. We establish theoretical guarantees for the method, demonstrating faster convergence rates compared to individual task learning when tasks share similar parametric components. Extensive simulations and real data applications complement the theoretical findings of our work while highlight the effectiveness of our framework even in moderate sample sizes.
Open AccessLate Fusion Multi-task Learning for Semiparametric Inference with Nuisance Parameters
In the age of large and heterogeneous datasets, the integration of information from diverse sources is essential to improve parameter estimation. Multi-task learning offers a powerful approach by enabling simultaneous learning across related tasks. In this work, we introduce a late fusion framework for multi-task learning with semiparametric models that involve infinite-dimensional nuisance parameters, focusing on applications such as heterogeneous treatment effect estimation across multiple data sources, including electronic health records from different hospitals or clinical trial data. Our framework is two-step: first, initial double machine-learning estimators are obtained through individual task learning; second, these estimators are adaptively aggregated to exploit task similarities while remaining robust to task-specific differences. In particular, the framework avoids individual level data sharing, preserving privacy. Additionally, we propose a novel multi-task learning method for nuisance parameter estimation, which further enhances parameter estimation when nuisance parameters exhibit similarity across tasks. We establish theoretical guarantees for the method, demonstrating faster convergence rates compared to individual task learning when tasks share similar parametric components. Extensive simulations and real data applications complement the theoretical findings of our work while highlight the effectiveness of our framework even in moderate sample sizes.
- 4 June 2025
- Published by Oxford University Press (OUP) in Biometrika
- Vol. 112 (4)
Nonsense associations in Markov random fields with pairwise dependence
Summary: identified the issue of ′nonsense correlations’ in time series data, where dependence within each of two random vectors causes overdispersion, i.e., variance inflation, for measures of dependence between the two. Since then much has been written about nonsense correlations, but nearly all of it confined to the time series literature. In this paper we provide the first, to our knowledge, rigorous study of this phenomenon for other forms of (positive) dependence, specifically for Markov random fields on lattices and graphs. We consider binary and continuous random vectors and three different measures of association: correlation, covariance and the ordinary least-squares coefficient from projecting one random vector onto the other. In some settings we find variance inflation consistent with Yule’s nonsense correlation. Surprisingly, we also find variance deflation in some settings, and in others the variance is unchanged under dependence. Perhaps most notably, we find general conditions under which ordinary least-squares inference that ignores dependence is valid despite positive dependence in the regression errors, contradicting the presentation of ordinary least squares in countless textbooks and courses.
- 4 June 2025
- Published by Oxford University Press (OUP) in Biometrika
- Vol. 112 (4)
Nonsense associations in Markov random fields with pairwise dependence
Summary: identified the issue of ′nonsense correlations’ in time series data, where dependence within each of two random vectors causes overdispersion, i.e., variance inflation, for measures of dependence between the two. Since then much has been written about nonsense correlations, but nearly all of it confined to the time series literature. In this paper we provide the first, to our knowledge, rigorous study of this phenomenon for other forms of (positive) dependence, specifically for Markov random fields on lattices and graphs. We consider binary and continuous random vectors and three different measures of association: correlation, covariance and the ordinary least-squares coefficient from projecting one random vector onto the other. In some settings we find variance inflation consistent with Yule’s nonsense correlation. Surprisingly, we also find variance deflation in some settings, and in others the variance is unchanged under dependence. Perhaps most notably, we find general conditions under which ordinary least-squares inference that ignores dependence is valid despite positive dependence in the regression errors, contradicting the presentation of ordinary least squares in countless textbooks and courses.
- 3 March 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Theory
- Vol. 71 (5) , 3925-3949
Sharp Signal Detection Under Ferromagnetic Ising Models
In this paper, we study a structured signal detection problem in Ferromagnetic Ising models with examples encompassing Ising Models on lattices, and Mean-Field type Ising Models such as dense Erdős-Rényi, and dense random regular graphs. We provide sharp constants of detection in each of these cases and thereby pinpoint an asymptotically precise relationship between the detection problem with the underlying dependence. To obtain this sharp characterization of the detection boundary at the level of sharp multiplicative constants, we derive necessary moderate deviation bounds for partial summands of magnetizations which might be of independent interest. Finally, we demonstrate how our tests can be designed to be adaptive over the strength of dependence present in the respective models.
- 3 March 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Theory
- Vol. 71 (5) , 3925-3949
Sharp Signal Detection Under Ferromagnetic Ising Models
In this paper, we study a structured signal detection problem in Ferromagnetic Ising models with examples encompassing Ising Models on lattices, and Mean-Field type Ising Models such as dense Erdős-Rényi, and dense random regular graphs. We provide sharp constants of detection in each of these cases and thereby pinpoint an asymptotically precise relationship between the detection problem with the underlying dependence. To obtain this sharp characterization of the detection boundary at the level of sharp multiplicative constants, we derive necessary moderate deviation bounds for partial summands of magnetizations which might be of independent interest. Finally, we demonstrate how our tests can be designed to be adaptive over the strength of dependence present in the respective models.
- Open Access
Causal effect estimation under network interference with mean-field methods
We study causal effect estimation from observational data under interference. The interference pattern is captured by an observed network. We adopt the chain graph framework of Tchetgen Tchetgen et. al. (2021), which allows (i) interaction among the outcomes of distinct study units connected along the graph and (ii) long range interference, whereby the outcome of an unit may depend on the treatments assigned to distant units connected along the interference network. For ``mean-field" interaction networks, we develop a new scalable iterative algorithm to estimate the causal effects. For gaussian weighted networks, we introduce a novel causal effect estimation algorithm based on Approximate Message Passing (AMP). Our algorithms are provably consistent under a ``high-temperature" condition on the underlying model. We estimate the (unknown) parameters of the model from data using maximum pseudo-likelihood and establish $\sqrt{n}$-consistency of this estimator in all parameter regimes. Finally, we prove that the downstream estimators obtained by plugging in estimated parameters into the aforementioned algorithms are consistent at high-temperature. Our methods can accommodate dense interactions among the study units -- a setting beyond reach using existing techniques. Our algorithms originate from the study of variational inference approaches in high-dimensional statistics; overall, we demonstrate the usefulness of these ideas in the context of causal effect estimation under interference.
Open AccessCausal effect estimation under network interference with mean-field methods
We study causal effect estimation from observational data under interference. The interference pattern is captured by an observed network. We adopt the chain graph framework of Tchetgen Tchetgen et. al. (2021), which allows (i) interaction among the outcomes of distinct study units connected along the graph and (ii) long range interference, whereby the outcome of an unit may depend on the treatments assigned to distant units connected along the interference network. For ``mean-field" interaction networks, we develop a new scalable iterative algorithm to estimate the causal effects. For gaussian weighted networks, we introduce a novel causal effect estimation algorithm based on Approximate Message Passing (AMP). Our algorithms are provably consistent under a ``high-temperature" condition on the underlying model. We estimate the (unknown) parameters of the model from data using maximum pseudo-likelihood and establish $\sqrt{n}$-consistency of this estimator in all parameter regimes. Finally, we prove that the downstream estimators obtained by plugging in estimated parameters into the aforementioned algorithms are consistent at high-temperature. Our methods can accommodate dense interactions among the study units -- a setting beyond reach using existing techniques. Our algorithms originate from the study of variational inference approaches in high-dimensional statistics; overall, we demonstrate the usefulness of these ideas in the context of causal effect estimation under interference.
- Open Access
Nonsense associations in Markov random fields with pairwise dependence
Yule (1926) identified the issue of "nonsense correlations" in time series data, where dependence within each of two random vectors causes overdispersion -- i.e. variance inflation -- for measures of dependence between the two. During the near century since then, much has been written about nonsense correlations -- but nearly all of it confined to the time series literature. In this paper we provide the first, to our knowledge, rigorous study of this phenomenon for more general forms of (positive) dependence, specifically for Markov random fields on lattices and graphs. We consider both binary and continuous random vectors and three different measures of association: correlation, covariance, and the ordinary least squares coefficient that results from projecting one random vector onto the other. In some settings we find variance inflation consistent with Yule's nonsense correlation. However, surprisingly, we also find variance deflation in some settings, and in others the variance is unchanged under dependence. Perhaps most notably, we find general conditions under which OLS inference that ignores dependence is valid despite positive dependence in the regression errors, contradicting the presentation of OLS in countless textbooks and courses.
Open AccessNonsense associations in Markov random fields with pairwise dependence
Yule (1926) identified the issue of "nonsense correlations" in time series data, where dependence within each of two random vectors causes overdispersion -- i.e. variance inflation -- for measures of dependence between the two. During the near century since then, much has been written about nonsense correlations -- but nearly all of it confined to the time series literature. In this paper we provide the first, to our knowledge, rigorous study of this phenomenon for more general forms of (positive) dependence, specifically for Markov random fields on lattices and graphs. We consider both binary and continuous random vectors and three different measures of association: correlation, covariance, and the ordinary least squares coefficient that results from projecting one random vector onto the other. In some settings we find variance inflation consistent with Yule's nonsense correlation. However, surprisingly, we also find variance deflation in some settings, and in others the variance is unchanged under dependence. Perhaps most notably, we find general conditions under which OLS inference that ignores dependence is valid despite positive dependence in the regression errors, contradicting the presentation of OLS in countless textbooks and courses.
- 25 April 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Theory
- Vol. 68 (10) , 6762-6773
Matrix Completion With Data-Dependent Missingness Probabilities
The problem of completing a large matrix with lots of missing entries has received widespread attention in the last couple of decades. Two popular approaches to the matrix completion problem are based on singular value thresholding and nuclear norm minimization. Most of the past works on this subject assume that there is a single number $p$ such that each entry of the matrix is available independently with probability $p$ and missing otherwise. This assumption may not be realistic for many applications. In this work, we replace it with the assumption that the probability that an entry is available is an unknown function $f$ of the entry itself. For example, if the entry is the rating given to a movie by a viewer, then it seems plausible that high value entries have greater probability of being available than low value entries. We propose two new estimators, based on singular value thresholding and nuclear norm minimization, to recover the matrix under this assumption. The estimators involve no tuning parameters, and are shown to be consistent under a low rank assumption. We also provide a consistent estimator of the unknown function $f$ .
- 25 April 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Theory
- Vol. 68 (10) , 6762-6773
Matrix Completion With Data-Dependent Missingness Probabilities
The problem of completing a large matrix with lots of missing entries has received widespread attention in the last couple of decades. Two popular approaches to the matrix completion problem are based on singular value thresholding and nuclear norm minimization. Most of the past works on this subject assume that there is a single number $p$ such that each entry of the matrix is available independently with probability $p$ and missing otherwise. This assumption may not be realistic for many applications. In this work, we replace it with the assumption that the probability that an entry is available is an unknown function $f$ of the entry itself. For example, if the entry is the rating given to a movie by a viewer, then it seems plausible that high value entries have greater probability of being available than low value entries. We propose two new estimators, based on singular value thresholding and nuclear norm minimization, to recover the matrix under this assumption. The estimators involve no tuning parameters, and are shown to be consistent under a low rank assumption. We also provide a consistent estimator of the unknown function $f$ .
- Open Access
Matrix completion with data-dependent missingness probabilities
The problem of completing a large matrix with lots of missing entries has received widespread attention in the last couple of decades. Two popular approaches to the matrix completion problem are based on singular value thresholding and nuclear norm minimization. Most of the past works on this subject assume that there is a single number $p$ such that each entry of the matrix is available independently with probability $p$ and missing otherwise. This assumption may not be realistic for many applications. In this work, we replace it with the assumption that the probability that an entry is available is an unknown function $f$ of the entry itself. For example, if the entry is the rating given to a movie by a viewer, then it seems plausible that high value entries have greater probability of being available than low value entries. We propose two new estimators, based on singular value thresholding and nuclear norm minimization, to recover the matrix under this assumption. The estimators involve no tuning parameters, and are shown to be consistent under a low rank assumption. We also provide a consistent estimator of the unknown function $f$.Open AccessMatrix completion with data-dependent missingness probabilities
The problem of completing a large matrix with lots of missing entries has received widespread attention in the last couple of decades. Two popular approaches to the matrix completion problem are based on singular value thresholding and nuclear norm minimization. Most of the past works on this subject assume that there is a single number $p$ such that each entry of the matrix is available independently with probability $p$ and missing otherwise. This assumption may not be realistic for many applications. In this work, we replace it with the assumption that the probability that an entry is available is an unknown function $f$ of the entry itself. For example, if the entry is the rating given to a movie by a viewer, then it seems plausible that high value entries have greater probability of being available than low value entries. We propose two new estimators, based on singular value thresholding and nuclear norm minimization, to recover the matrix under this assumption. The estimators involve no tuning parameters, and are shown to be consistent under a low rank assumption. We also provide a consistent estimator of the unknown function $f$. - 6 April 2021
- Published by Wiley in Random Structures & Algorithms
- Vol. 59 (3) , 315-338
Upper tail for homomorphism counts in constrained sparse random graphs
- 6 April 2021
- Published by Wiley in Random Structures & Algorithms
- Vol. 59 (3) , 315-338
Upper tail for homomorphism counts in constrained sparse random graphs