RMRahul Mazumder

Rahul Mazumder

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Rahul Mazumder has a total of 95 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Improved heritability partitioning and enrichment analyses using summary statistics with graphREML

    by Hui Li, Tushar Kamath, Rahul Mazumder, Xihong Lin, Luke Jen O’COnnor

    Heritability enrichment analysis using data from genome-wide association studies is often used to understand the functional basis of genetic architecture. Stratified linkage disequilibrium score regression (S-LDSC) is a widely used method-of-moments estimator for heritability enrichment, but S-LDSC has low statistical power compared with likelihood-based approaches. We introduce graphREML, a precise and powerful likelihood-based heritability partitioning and enrichment analysis method. It utilizes summary statistics from genome-wide association studies and sparse linkage disequilibrium graphical models, which make likelihood calculations tractable. We validate our method using extensive simulations and in analyses of a wide range of real traits. On average across traits, graphREML produces enrichment estimates that are concordant with S-LDSC, indicating that both methods are unbiased; however, graphREML identifies 2.5 times more significant trait-annotation enrichments, demonstrating greater power compared with the moment-based S-LDSC approach. Furthermore, graphREML flexibly models the relationship between the annotations of an SNP and its heritability, producing well-calibrated estimates of per-SNP heritability.


  • Open Access

    Robust Batch-Level Query Routing for Large Language Models under Cost and Capacity Constraints

    by Jelena Markovic-Voronov, Kayhan Behdin, Yuanda Xu, Zhengze Zhou, Zhipeng Wang, Rahul Mazumder

  • Sparse Gaussian Graphical Models with Discrete Optimization: Computational and Statistical Perspectives

    by Kayhan Behdin, Wenyu Chen, Rahul Mazumder

    GraphL0: Sparse Gaussian Graphical Models with Discrete Optimization Recovering sparse dependency graphs in undirected Gaussian graphical models is a well-known problem in statistical machine learning. Given samples from a [ ]-dimensional Gaussian distribution, the task amounts to estimating the [ ] precision (inverse covariance) matrix under the assumption that only a small fraction of its entries are nonzero. In “Sparse Gaussian Graphical Models with Discrete Optimization: Computational and Statistical Perspectives,” the authors introduce GraphL0. GraphL0 is an estimator based on an [ ]-penalized pseudo-likelihood, departing from the more common [ ] relaxation. The resulting formulation is a convex mixed-integer program, which becomes challenging for standard commercial solvers at moderate-to-large [ ]. To make the approach practical, the authors develop a custom nonlinear branch-and-bound algorithm, alongside scalable approximate solvers. The paper also provides new statistical guarantees for estimation accuracy and support recovery, and experiments on synthetic and real data sets show substantial computational gains over off-the-shelf solvers and competitive runtime and accuracy versus leading alternatives.

  • Open Access

    A GPU-accelerated Nonlinear Branch-and-Bound Framework for Sparse Linear Models

    by Xiang Meng, Ryan Lucas, Rahul Mazumder

    We study exact sparse linear regression with an $\ell_0-\ell_2$ penalty and develop a branch-and-bound (BnB) algorithm explicitly designed for GPU execution. Starting from a perspective reformulation, we derive an interval relaxation that can be solved by ADMM with closed-form, coordinate-wise updates. We structure these updates so that the main work at each BnB node reduces to batched matrix-vector operations with a shared data matrix, enabling fine-grained parallelism across coordinates and coarse-grained parallelism across many BnB nodes on a single GPU. Feasible solutions (upper bounds) are generated by a projected gradient method on the active support, implemented in a batched fashion so that many candidate supports are updated in parallel on the GPU. We discuss practical design choices such as memory layout, batching strategies, and load balancing across nodes that are crucial for obtaining good utilization on modern GPUs. On synthetic and real high-dimensional datasets, our GPU-based approach achieves clear runtime improvements over a CPU implementation of our method, an existing specialized BnB method, and commercial MIP solvers.

  • Open Access

    Theoretical Compression Bounds for Wide Multilayer Perceptrons

    by Houssam El Cheairi, David Gamarnik, Rahul Mazumder

    Pruning and quantization techniques have been broadly successful in reducing the number of parameters needed for large neural networks, yet theoretical justification for their empirical success falls short. We consider a randomized greedy compression algorithm for pruning and quantization post-training and use it to rigorously show the existence of pruned/quantized subnetworks of multilayer perceptrons (MLPs) with competitive performance. We further extend our results to structured pruning of MLPs and convolutional neural networks (CNNs), thus providing a unified analysis of pruning in wide networks. Our results are free of data assumptions, and showcase a tradeoff between compressibility and network width. The algorithm we consider bears some similarities with Optimal Brain Damage (OBD) and can be viewed as a post-training randomized version of it. The theoretical results we derive bridge the gap between theory and application for pruning/quantization, and provide a justification for the empirical success of compression in wide multilayer perceptrons.

  • Open Access

    Differentially Private High-dimensional Variable Selection via Integer Programming

    by Petros Prastakos, Kayhan Behdin, Rahul Mazumder

    Sparse variable selection improves interpretability and generalization in high-dimensional learning by selecting a small subset of informative features. Recent advances in Mixed Integer Programming (MIP) have enabled solving large-scale non-private sparse regression - known as Best Subset Selection (BSS) - with millions of variables in minutes. However, extending these algorithmic advances to the setting of Differential Privacy (DP) has remained largely unexplored. In this paper, we introduce two new pure differentially private estimators for sparse variable selection, levering modern MIP techniques. Our framework is general and applies broadly to problems like sparse regression or classification, and we provide theoretical support recovery guarantees in the case of BSS. Inspired by the exponential mechanism, we develop structured sampling procedures that efficiently explore the non-convex objective landscape, avoiding the exhaustive combinatorial search in the exponential mechanism. We complement our theoretical findings with extensive numerical experiments, using both least squares and hinge loss for our objective function, and demonstrate that our methods achieve state-of-the-art empirical support recovery, outperforming competing algorithms in settings with up to $p=10^4$.

  • Open Access

    SPARTA: An Optimization Framework for Differentially Private Sparse Fine-Tuning

    by Mehdi Makni, Kayhan Behdin, Gabriel Afriat, Zheng Xu, Sergei Vassilvitskii, Natalia Ponomareva, Rahul Mazumder, Hussein Hazimeh


  • Open Access

    TSENOR: Highly-Efficient Algorithm for Finding Transposable N:M Sparse Masks

    by Xiang Meng, Mehdi Makni, Rahul Mazumder

    Network pruning reduces the computational requirements of large neural networks, with N:M sparsity -- retaining only N out of every M consecutive weights -- offering a compelling balance between compressed model quality and hardware acceleration. However, N:M sparsity only accelerates forward-pass computations, as N:M patterns are not preserved during matrix transposition, limiting efficiency during training where both passes are computationally intensive. While transposable N:M sparsity has been proposed to address this limitation, existing methods for finding transposable N:M sparse masks either fail to scale to large models or are restricted to M=4 which results in suboptimal compression-accuracy trade-off. We introduce an efficient solver for transposable N:M masks that scales to billion-parameter models. We formulate mask generation as optimal transport problems and solve through entropy regularization and Dykstra's algorithm, followed by a rounding procedure. Our tensor-based implementation exploits GPU parallelism, achieving up to 100x speedup with only 1-10% error compared to existing methods. Our approach can be integrated with layer-wise N:M pruning frameworks including Wanda, SparseGPT and ALPS to produce transposable N:M sparse models with arbitrary N:M values. Experiments show that LLaMA3.2-8B with transposable 16:32 sparsity maintains performance close to its standard N:M counterpart and outperforms standard 2:4 sparse model, showing the practical value of our approach.