A Systematic Characterization of LLM Inference on GPUs
Preprint
Abstract
This work presents a systematic characterization of Large Language Model (LLM) inference to address fragmented understanding. Through comprehensive experiments, we establish a four-dimensional analytical framework: (1) Two-Phase Heterogeneity Observation; (2) Microarchitectural Root Cause Analysis; (3) System Scaling Principles; and (4) Emerging Paradigm Boundaries. Our investigation progresses systematically from observation to foresight: identifying performance phenomena, revealing hardware causes, validating system behavior, and exploring new paradigms. This study not only consolidates a reliable empirical foundation for existing research but also provides new discoveries and practical optimization guidance for LLM inference.Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- A Survey on Inference Engines for Large Language Models: Perspectives on Optimization and EfficiencyACM Transactions on Intelligent Systems and Technology, 2026
- Characterizing Communication Patterns in Distributed Large Language Model InferenceIEEE Micro, 2026
- Mooncake: A KVCache-centric Disaggregated Architecture for LLM ServingACM Transactions on Storage, 2025
- ShuffleInfer: Disaggregate LLM Inference for Mixed Downstream WorkloadsACM Transactions on Architecture and Code Optimization, 2025
- A Survey on Mixture of Experts in Large Language ModelsIEEE Transactions on Knowledge and Data Engineering, 2025
- A Review on Edge Large Language Models: Design, Execution, and ApplicationsACM Computing Surveys, 2025
- LightRAG: Simple and Fast Retrieval-Augmented Generation2025
- Energy Considerations of Large Language Model Inference and Efficiency Optimizations2025
- MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attention2024
- Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline2023