Muhammad Shahbaz
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Muhammad Shahbaz has a total of 44 co-authors. Here are the top co-authors by number of shared publications:
- Annus Zulfiqar
University of Michigan, Ann Arbor, USA
5 shared publications - Ali Imran
Department of Computer Science and Engineering, University of Michigan, Ann Arbor, MI, USA
3 shared publications - Josep Torrellas
University of Illinois at Urbana-Champaign, Urbana, Illinois, USA
3 shared publications Balaji Prabhakar
Stanford University
2 shared publications- Ben Pfaff
Feldern
2 shared publications - Chunao Liu
Purdue University, West Lafayette, USA
2 shared publications
Latest Publications
- Open Access
- 11 August 2026
- Published by Association for Computing Machinery (ACM)
- p. 2033-2039
Towards Network-Efficient Cross-Regional Inference via Learned Activation Compression
Large transformer models are increasingly deployed across geographically distributed GPU clusters due to capacity, cost, and locality constraints. When inference is partitioned across sites, intermediate activations must be transmitted over wide area network (WAN) links at each partition boundary, introducing significant communication overhead. We present Feather, a system that reduces this overhead by compressing intermediate activations before transmission and reconstructing them before downstream layers resume execution. Feather learns a compact representation of activation tensors using a lightweight neural codec trained with a reconstruction objective while keeping the original model frozen. Across encoder and decoder transformer models, Feather achieves up to 48× activation compression while maintaining accuracy close to the baseline model. Under representative WAN conditions (e.g., 10 Gbps bandwidth and a 10 ms RTT), this reduction yields up to 4.96× improvement in end-to-end latency, consistently outperforming existing compression schemes, including linear autoencoder, PCA, and SVD.
- 11 August 2026
- Published by Association for Computing Machinery (ACM)
- p. 2033-2039
Open AccessTowards Network-Efficient Cross-Regional Inference via Learned Activation Compression
Large transformer models are increasingly deployed across geographically distributed GPU clusters due to capacity, cost, and locality constraints. When inference is partitioned across sites, intermediate activations must be transmitted over wide area network (WAN) links at each partition boundary, introducing significant communication overhead. We present Feather, a system that reduces this overhead by compressing intermediate activations before transmission and reconstructing them before downstream layers resume execution. Feather learns a compact representation of activation tensors using a lightweight neural codec trained with a reconstruction objective while keeping the original model frozen. Across encoder and decoder transformer models, Feather achieves up to 48× activation compression while maintaining accuracy close to the baseline model. Under representative WAN conditions (e.g., 10 Gbps bandwidth and a 10 ms RTT), this reduction yields up to 4.96× improvement in end-to-end latency, consistently outperforming existing compression schemes, including linear autoencoder, PCA, and SVD.
- 22 October 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Architecture Letters
- Vol. 24 (2) , 393-396
Reimagining RDMA Through the Lens of ML
As distributed machine learning (ML) workloads scale to thousands of GPUs connected by ultra-high-speed interconnects, tail latency in collective communication has emerged as a primary bottleneck. Prior RDMA designs, like RoCE, IRN, and SRNIC, enforce strict reliability and in-order delivery, relying on retransmissions and packet sequencing to ensure correctness. While effective for general-purpose workloads, these mechanisms introduce complexity and latency that scale poorly, where even rare packet losses or delays can consistently degrade system performance. We introduce Celeris, a domain-specific RDMA transport that revisits traditional reliability guarantees based on ML's tolerance for lost or partial data. Celeris removes retransmissions and in-order delivery from the RDMA NIC, enabling best-effort transport that exploits the robustness of ML workloads. It retains congestion control (e.g., DCQCN) and manages communication with software-level mechanisms such as adaptive timeouts and data prioritization, while shifting loss recovery to the ML pipeline (e.g., using the Hadamard Transform). Early results show that Celeris reduces 99th-percentile latency by up to 2.3×, cuts BRAM usage by 67%, and nearly doubles NIC resilience to faults—delivering a resilient, scalable transport tailored for ML at cluster scale.
- 22 October 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Computer Architecture Letters
- Vol. 24 (2) , 393-396
Reimagining RDMA Through the Lens of ML
As distributed machine learning (ML) workloads scale to thousands of GPUs connected by ultra-high-speed interconnects, tail latency in collective communication has emerged as a primary bottleneck. Prior RDMA designs, like RoCE, IRN, and SRNIC, enforce strict reliability and in-order delivery, relying on retransmissions and packet sequencing to ensure correctness. While effective for general-purpose workloads, these mechanisms introduce complexity and latency that scale poorly, where even rare packet losses or delays can consistently degrade system performance. We introduce Celeris, a domain-specific RDMA transport that revisits traditional reliability guarantees based on ML's tolerance for lost or partial data. Celeris removes retransmissions and in-order delivery from the RDMA NIC, enabling best-effort transport that exploits the robustness of ML workloads. It retains congestion control (e.g., DCQCN) and manages communication with software-level mechanisms such as adaptive timeouts and data prioritization, while shifting loss recovery to the ML pipeline (e.g., using the Hadamard Transform). Early results show that Celeris reduces 99th-percentile latency by up to 2.3×, cuts BRAM usage by 67%, and nearly doubles NIC resilience to faults—delivering a resilient, scalable transport tailored for ML at cluster scale.
- Open Access
- 17 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 958-974
NetSparse: In-Network Acceleration of Distributed Sparse Kernels
- 17 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 958-974
Open AccessNetSparse: In-Network Acceleration of Distributed Sparse Kernels
- Open Access
- 8 September 2025
- Published by Association for Computing Machinery (ACM)
- p. 28-30
Kairo - Incremental View Maintenance for Scalable Virtual Switch Caching
- 8 September 2025
- Published by Association for Computing Machinery (ACM)
- p. 28-30
Open AccessKairo - Incremental View Maintenance for Scalable Virtual Switch Caching
- Open Access
- 27 August 2025
- Published by Association for Computing Machinery (ACM)
- p. 1238-1240
SpliDT: Partitioned Decision Trees for Scalable Stateful Inference at Line Rate
- 27 August 2025
- Published by Association for Computing Machinery (ACM)
- p. 1238-1240
Open AccessSpliDT: Partitioned Decision Trees for Scalable Stateful Inference at Line Rate
- Open Access
- 20 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 708-722
HardHarvest: Hardware-Supported Core Harvesting for Microservices
- 20 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 708-722
Open AccessHardHarvest: Hardware-Supported Core Harvesting for Microservices
- Open Access
- 30 March 2025
- Published by Association for Computing Machinery (ACM)
- p. 467-481
G igaflow : Pipeline-Aware Sub-Traversal Caching for Modern SmartNICs
- 30 March 2025
- Published by Association for Computing Machinery (ACM)
- p. 467-481
Open AccessG igaflow : Pipeline-Aware Sub-Traversal Caching for Modern SmartNICs
- Open Access
- 4 November 2024
- Published by Association for Computing Machinery (ACM)
- p. 17-20
Caravan: Practical Online Learning of In-Network ML Models with Labeling Agents
Recent work on in-network machine learning (ML) anticipates offline models to operate well in modern networking environments. However, upon deployment, these models struggle to cope with fluctuating traffic patterns and network conditions and, therefore, must be validated and updated frequently in an online fashion. This abstract presents Caravan, a practical online learning system for in-network ML models. We tackle two primary challenges in facilitating online learning for networking: (a) the automatic labeling of evolving traffic and (b) the efficient monitoring and detection of model performance degradation to trigger retraining. Caravan repurposes existing systems (e.g., heuristics, access control lists, and foundation models)---not directly suitable for such dynamic environments---into high-quality labeling sources for generating labeled data for online learning. Caravan also introduces a new metric, accuracy proxy, to track model degradation and potential drift to efficiently trigger retraining. Our evaluations show that Caravan's labeling strategy enables in-network ML models to closely follow the changes in the traffic dynamics with a 30.3% improvement in F1 score (on average), compared to offline models. Moreover, Caravan sustains comparable inference accuracy to that of a continuous-learning system while consuming 61.3% less GPU compute time (on average) via accuracy proxy and retraining triggers. This abstract summarizes a previously published work [53] at the 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24).
- 4 November 2024
- Published by Association for Computing Machinery (ACM)
- p. 17-20
Open AccessCaravan: Practical Online Learning of In-Network ML Models with Labeling Agents
Recent work on in-network machine learning (ML) anticipates offline models to operate well in modern networking environments. However, upon deployment, these models struggle to cope with fluctuating traffic patterns and network conditions and, therefore, must be validated and updated frequently in an online fashion. This abstract presents Caravan, a practical online learning system for in-network ML models. We tackle two primary challenges in facilitating online learning for networking: (a) the automatic labeling of evolving traffic and (b) the efficient monitoring and detection of model performance degradation to trigger retraining. Caravan repurposes existing systems (e.g., heuristics, access control lists, and foundation models)---not directly suitable for such dynamic environments---into high-quality labeling sources for generating labeled data for online learning. Caravan also introduces a new metric, accuracy proxy, to track model degradation and potential drift to efficiently trigger retraining. Our evaluations show that Caravan's labeling strategy enables in-network ML models to closely follow the changes in the traffic dynamics with a 30.3% improvement in F1 score (on average), compared to offline models. Moreover, Caravan sustains comparable inference accuracy to that of a continuous-learning system while consuming 61.3% less GPU compute time (on average) via accuracy proxy and retraining triggers. This abstract summarizes a previously published work [53] at the 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24).
- Open Access
- 24 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 660-677
Modeling and Generating Control-Plane Traffic for Cellular Networks
With 5G deployment gaining momentum, the control-plane traffic volume of cellular networks is escalating. Such rapid traffic growth motivates the need to study the mobile core network (MCN) control-plane design and performance optimization. Doing so requires realistic, large control-plane traffic traces in order to profile and debug the mobile network performance under real workload. However, large-scale control-plane traffic traces are not made available to the public by mobile operators due to business and privacy concerns. As such, it is critically important to develop accurate, scalable, versatile, and open-to-innovation control traffic generators, which in turn critically rely on an accurate traffic model for the control plane. Developing an accurate model of control-plane traffic faces several challenges: (1) how to capture the dependence among the control events generated by each User Equipment (UE), (2) how to model the inter-arrival time and sojourn time of control events of individual UEs, and (3) how to capture the diversity of control-plane traffic across UEs. We present a novel two-level hierarchical state-machine-based control-plane traffic model. We further show how our model can be easily adjusted from LTE to NextG networks (e.g., 5G) to support modeling future control-plane traffic. We experimentally validate that the proposed model can generate large realistic control-plane traffic traces. We have open-sourced our traffic generator to the public to foster MCN research.
- 24 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 660-677
Open AccessModeling and Generating Control-Plane Traffic for Cellular Networks
With 5G deployment gaining momentum, the control-plane traffic volume of cellular networks is escalating. Such rapid traffic growth motivates the need to study the mobile core network (MCN) control-plane design and performance optimization. Doing so requires realistic, large control-plane traffic traces in order to profile and debug the mobile network performance under real workload. However, large-scale control-plane traffic traces are not made available to the public by mobile operators due to business and privacy concerns. As such, it is critically important to develop accurate, scalable, versatile, and open-to-innovation control traffic generators, which in turn critically rely on an accurate traffic model for the control plane. Developing an accurate model of control-plane traffic faces several challenges: (1) how to capture the dependence among the control events generated by each User Equipment (UE), (2) how to model the inter-arrival time and sojourn time of control events of individual UEs, and (3) how to capture the diversity of control-plane traffic across UEs. We present a novel two-level hierarchical state-machine-based control-plane traffic model. We further show how our model can be easily adjusted from LTE to NextG networks (e.g., 5G) to support modeling future control-plane traffic. We experimentally validate that the proposed model can generate large realistic control-plane traffic traces. We have open-sourced our traffic generator to the public to foster MCN research.
- Open Access
- 17 June 2023
- Published by Association for Computing Machinery (ACM)
- p. 1-15
μManycore: A Cloud-Native CPU for Tail at Scale
Microservices are emerging as a popular cloud-computing paradigm. Microservice environments execute typically-short service requests that interact with one another via remote procedure calls (often across machines), and are subject to stringent tail-latency constraints. In contrast, current processors are designed for traditional monolithic applications. They support global hardware cache coherence, provide large caches, incorporate microarchitecture for long-running, predictable applications (such as advanced prefetching), and are optimized to minimize average latency rather than tail latency. To address this imbalance, this paper proposes μManycore, an architecture optimized for cloud-native microservice environments. Based on a characterization of microservice applications, μManycore is designed to minimize unnecessary microarchitecture and mitigate overheads to reduce tail latency. Indeed, rather than supporting manycore-wide hardware cache coherence, μManycore has multiple small hardware cache-coherent domains, called Villages. Clusters of villages are interconnected with an on-package leaf-spine network, which has many redundant, low-hop-count paths between clusters. To minimize latency overheads, μManycore schedules and queues service requests in hardware, and includes hardware support to save and restore process state when doing a context-switch. Our simulation-based results show that μManycore delivers high performance. A cluster of 10 servers with a 1024-core μManycore in each server delivers 3.7× lower average latency, 15.5× higher throughput, and, importantly, 10.4× lower tail latency than a cluster with iso-power conventional server-class multicores. Similar good results are attained compared to a cluster with power-hungry iso-area conventional server-class multicores.
- 17 June 2023
- Published by Association for Computing Machinery (ACM)
- p. 1-15
Open AccessμManycore: A Cloud-Native CPU for Tail at Scale
Microservices are emerging as a popular cloud-computing paradigm. Microservice environments execute typically-short service requests that interact with one another via remote procedure calls (often across machines), and are subject to stringent tail-latency constraints. In contrast, current processors are designed for traditional monolithic applications. They support global hardware cache coherence, provide large caches, incorporate microarchitecture for long-running, predictable applications (such as advanced prefetching), and are optimized to minimize average latency rather than tail latency. To address this imbalance, this paper proposes μManycore, an architecture optimized for cloud-native microservice environments. Based on a characterization of microservice applications, μManycore is designed to minimize unnecessary microarchitecture and mitigate overheads to reduce tail latency. Indeed, rather than supporting manycore-wide hardware cache coherence, μManycore has multiple small hardware cache-coherent domains, called Villages. Clusters of villages are interconnected with an on-package leaf-spine network, which has many redundant, low-hop-count paths between clusters. To minimize latency overheads, μManycore schedules and queues service requests in hardware, and includes hardware support to save and restore process state when doing a context-switch. Our simulation-based results show that μManycore delivers high performance. A cluster of 10 servers with a 1024-core μManycore in each server delivers 3.7× lower average latency, 15.5× higher throughput, and, importantly, 10.4× lower tail latency than a cluster with iso-power conventional server-class multicores. Similar good results are attained compared to a cluster with power-hungry iso-area conventional server-class multicores.