Characterizing Communication Patterns in Distributed Large Language Model Inference
- 19 March 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Micro
- Vol. PP (02721732) , 1-8
Abstract
Large Language Models (LLMs) built on transformer architectures have transformed natural language processing, achieving remarkable performance across diverse applications. While distributed inference frameworks enable practical deployment of these models, inter-GPU communication creates significant performance constraints that limit service quality in real-world systems. This paper investigates communication dynamics in distributed LLM serving—analyzing how various parallelization approaches coordinate data exchange between GPU workers during inference. We study transformer-based models as representative examples of contemporary architectures widely used in operational deployments. Our work combines detailed profiling measurements with predictive analytical models to characterize communication behavior across different parallelization configurations. These insights offer practical recommendations for selecting appropriate parallelization schemes in production LLM services and identify key opportunities for optimizing inference frameworks and communication infrastructure.Keywords
Funding Information
- National Science Foundation (2311830)
- XRAC (NCR-130002)
This publication has 6 references indexed in Scilit:
- Demystifying the Communication Characteristics for Distributed Transformer Models2024
- Thorough Characterization and Analysis of Large Transformer Model Training At-ScaleProceedings of the ACM on Measurement and Analysis of Computing Systems, 2024
- Efficient Memory Management for Large Language Model Serving with PagedAttention2023
- DeepSpeed- Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale2022
- Performance Characterization of DNN Training using TensorFlow and PyTorch on Modern Clusters2019
- Scalable Distributed DNN Training using TensorFlow and CUDA-Aware MPI: Characterization, Designs, and Performance Evaluation2019