SBSohom Bhattacharya

Sohom Bhattacharya

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Sohom Bhattacharya has a total of 15 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Inferences on mixing probabilities and ranking in mixed-membership models

    by Sohom Bhattacharya, Jianqing Fan, Jikai Hou

    1 IntroductionIn various fields of study, such as citation networks, protein interactions, health, finance, trade, and social networks, we often come across large amounts of data that describe the ...


  • Open Access

    Causal inference under interference: computational barriers and algorithmic solutions

    by Sohom Bhattacharya, Subhabrata Sen

    We study causal effect estimation under interference from network data. We work under the chain-graph formulation pioneered in Tchetgen Tchetgen et. al (2021). Our first result shows that polynomial time evaluation of treatment effects is computationally hard in this framework without additional assumptions on the underlying chain graph. Subsequently, we assume that the interactions among the study units are governed either by (i) a dense graph or (ii) an i.i.d. Gaussian matrix. In each case, we show that the treatment effects have well-defined limits as the population size diverges to infinity. Additionally, we develop polynomial time algorithms to consistently evaluate the treatment effects in each case. Finally, we estimate the unknown parameters from the observed data using maximum pseudo-likelihood estimates, and establish the stability of our causal effect estimators under this perturbation. Our algorithms provably approximate the causal effects in polynomial time even in low-temperature regimes where the canonical MCMC samplers are slow mixing. For dense graphs, our results use the notion of regularity partitions; for Gaussian interactions, our approach uses ideas from spin glass theory and Approximate Message Passing.

  • Open Access

    Late Fusion Multi-task Learning for Semiparametric Inference with Nuisance Parameters

    by Sohom Bhattacharya, Yongzhuo Chen, Muxuan Liang

    In the age of large and heterogeneous datasets, the integration of information from diverse sources is essential to improve parameter estimation. Multi-task learning offers a powerful approach by enabling simultaneous learning across related tasks. In this work, we introduce a late fusion framework for multi-task learning with semiparametric models that involve infinite-dimensional nuisance parameters, focusing on applications such as heterogeneous treatment effect estimation across multiple data sources, including electronic health records from different hospitals or clinical trial data. Our framework is two-step: first, initial double machine-learning estimators are obtained through individual task learning; second, these estimators are adaptively aggregated to exploit task similarities while remaining robust to task-specific differences. In particular, the framework avoids individual level data sharing, preserving privacy. Additionally, we propose a novel multi-task learning method for nuisance parameter estimation, which further enhances parameter estimation when nuisance parameters exhibit similarity across tasks. We establish theoretical guarantees for the method, demonstrating faster convergence rates compared to individual task learning when tasks share similar parametric components. Extensive simulations and real data applications complement the theoretical findings of our work while highlight the effectiveness of our framework even in moderate sample sizes.

  • Nonsense associations in Markov random fields with pairwise dependence

    by Sohom Bhattacharya, Rajarshi Mukherjee, Elizabeth L Ogburn

    Summary: identified the issue of ′nonsense correlations’ in time series data, where dependence within each of two random vectors causes overdispersion, i.e., variance inflation, for measures of dependence between the two. Since then much has been written about nonsense correlations, but nearly all of it confined to the time series literature. In this paper we provide the first, to our knowledge, rigorous study of this phenomenon for other forms of (positive) dependence, specifically for Markov random fields on lattices and graphs. We consider binary and continuous random vectors and three different measures of association: correlation, covariance and the ordinary least-squares coefficient from projecting one random vector onto the other. In some settings we find variance inflation consistent with Yule’s nonsense correlation. Surprisingly, we also find variance deflation in some settings, and in others the variance is unchanged under dependence. Perhaps most notably, we find general conditions under which ordinary least-squares inference that ignores dependence is valid despite positive dependence in the regression errors, contradicting the presentation of ordinary least squares in countless textbooks and courses.


  • Sharp Signal Detection Under Ferromagnetic Ising Models

    by Sohom Bhattacharya, Rajarshi Mukherjee, Gourab Ray

    In this paper, we study a structured signal detection problem in Ferromagnetic Ising models with examples encompassing Ising Models on lattices, and Mean-Field type Ising Models such as dense Erdős-Rényi, and dense random regular graphs. We provide sharp constants of detection in each of these cases and thereby pinpoint an asymptotically precise relationship between the detection problem with the underlying dependence. To obtain this sharp characterization of the detection boundary at the level of sharp multiplicative constants, we derive necessary moderate deviation bounds for partial summands of magnetizations which might be of independent interest. Finally, we demonstrate how our tests can be designed to be adaptive over the strength of dependence present in the respective models.


  • Open Access

    Causal effect estimation under network interference with mean-field methods

    by Sohom Bhattacharya, Subhabrata Sen

    We study causal effect estimation from observational data under interference. The interference pattern is captured by an observed network. We adopt the chain graph framework of Tchetgen Tchetgen et. al. (2021), which allows (i) interaction among the outcomes of distinct study units connected along the graph and (ii) long range interference, whereby the outcome of an unit may depend on the treatments assigned to distant units connected along the interference network. For ``mean-field" interaction networks, we develop a new scalable iterative algorithm to estimate the causal effects. For gaussian weighted networks, we introduce a novel causal effect estimation algorithm based on Approximate Message Passing (AMP). Our algorithms are provably consistent under a ``high-temperature" condition on the underlying model. We estimate the (unknown) parameters of the model from data using maximum pseudo-likelihood and establish $\sqrt{n}$-consistency of this estimator in all parameter regimes. Finally, we prove that the downstream estimators obtained by plugging in estimated parameters into the aforementioned algorithms are consistent at high-temperature. Our methods can accommodate dense interactions among the study units -- a setting beyond reach using existing techniques. Our algorithms originate from the study of variational inference approaches in high-dimensional statistics; overall, we demonstrate the usefulness of these ideas in the context of causal effect estimation under interference.

  • Open Access

    Nonsense associations in Markov random fields with pairwise dependence

    by Sohom Bhattacharya, Rajarshi Mukherjee, Elizabeth Ogburn

    Yule (1926) identified the issue of "nonsense correlations" in time series data, where dependence within each of two random vectors causes overdispersion -- i.e. variance inflation -- for measures of dependence between the two. During the near century since then, much has been written about nonsense correlations -- but nearly all of it confined to the time series literature. In this paper we provide the first, to our knowledge, rigorous study of this phenomenon for more general forms of (positive) dependence, specifically for Markov random fields on lattices and graphs. We consider both binary and continuous random vectors and three different measures of association: correlation, covariance, and the ordinary least squares coefficient that results from projecting one random vector onto the other. In some settings we find variance inflation consistent with Yule's nonsense correlation. However, surprisingly, we also find variance deflation in some settings, and in others the variance is unchanged under dependence. Perhaps most notably, we find general conditions under which OLS inference that ignores dependence is valid despite positive dependence in the regression errors, contradicting the presentation of OLS in countless textbooks and courses.

  • Matrix Completion With Data-Dependent Missingness Probabilities

    by Sohom Bhattacharya, Sourav Chatterjee

    The problem of completing a large matrix with lots of missing entries has received widespread attention in the last couple of decades. Two popular approaches to the matrix completion problem are based on singular value thresholding and nuclear norm minimization. Most of the past works on this subject assume that there is a single number $p$ such that each entry of the matrix is available independently with probability $p$ and missing otherwise. This assumption may not be realistic for many applications. In this work, we replace it with the assumption that the probability that an entry is available is an unknown function $f$ of the entry itself. For example, if the entry is the rating given to a movie by a viewer, then it seems plausible that high value entries have greater probability of being available than low value entries. We propose two new estimators, based on singular value thresholding and nuclear norm minimization, to recover the matrix under this assumption. The estimators involve no tuning parameters, and are shown to be consistent under a low rank assumption. We also provide a consistent estimator of the unknown function $f$ .


  • Open Access

    Matrix completion with data-dependent missingness probabilities

    by Sohom Bhattacharya, Sourav Chatterjee

    The problem of completing a large matrix with lots of missing entries has received widespread attention in the last couple of decades. Two popular approaches to the matrix completion problem are based on singular value thresholding and nuclear norm minimization. Most of the past works on this subject assume that there is a single number $p$ such that each entry of the matrix is available independently with probability $p$ and missing otherwise. This assumption may not be realistic for many applications. In this work, we replace it with the assumption that the probability that an entry is available is an unknown function $f$ of the entry itself. For example, if the entry is the rating given to a movie by a viewer, then it seems plausible that high value entries have greater probability of being available than low value entries. We propose two new estimators, based on singular value thresholding and nuclear norm minimization, to recover the matrix under this assumption. The estimators involve no tuning parameters, and are shown to be consistent under a low rank assumption. We also provide a consistent estimator of the unknown function $f$.