A Comprehensive Survey on Large Language Model Compression for Artificial Intelligence Applications in Edge Systems
Top Cited Papers
- 19 March 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 13 (10) , 20583-20599
Abstract
Large language models (LLMs) have achieved remarkable performance across various artificial intelligence (AI) applications. However, current LLMs cannot be deployed directly on edge nodes due to their large number of parameters. Fortunately, model compression technology has been proposed to reduce the computational workload and memory usage of LLMs, enabling further edge-based LLM services. However, existing research typically concentrates on isolated compression algorithms and lacks a comprehensive perspective on how to leverage these techniques for practical, end-to-end LLM deployment in edge environments. In this survey, we review edge-oriented LLM compression techniques and software–hardware codesign strategies to enable efficient LLM deployment on resource-constrained edge systems and guide future research in this area. First, we analyze techniques for LLM compression from the perspective of cloud–edge collaborative intelligence, including model quantization, parameter pruning, and knowledge distillation (KD). Second, we present several hybrid model frameworks tailored to dynamic, heterogeneous edge environments, based on model architecture, application scenarios, and combination selection. Third, we further refine a four-layer software–hardware codesign and an overhead-aware LLM deployment optimization. Finally, we discuss the challenges of current model compression approaches and offer insights into future research directions, with a focus on edge-based LLM services.Keywords
Funding Information
- National Natural Science Foundation of China (U25A20436)
- National Natural Science Foundation of China (62372047)
- Guangxi Key Research and Development Program (FN2504240036)
- Guangxi Key Research and Development Program (2025FN96441087)
- Guangdong S&T Program (2025B0101120006)
- Natural Science Foundation of Guangdong Province (2024A1515011323)
- Supplemental Funds for Major Scientific Research Projects of Beijing Normal University, Zhuhai (ZHPT2023002)
- Fundamental Research Funds for the Central Universities (2025JKZG069)
- Higher Education Research Topics of Guangdong Association of Higher Education in the 14th Five-Year Plan (24GYB207)
This publication has 85 references indexed in Scilit:
- Conversion rate prediction in online advertising: modeling techniques, performance evaluation and future directionsInformation Processing & Management, 2025
- Profiling and Optimization of Multicard GPU Machine Learning JobsConcurrency and Computation: Practice and Experience, 2025
- PT-BitNet: Scaling up the 1-Bit large language model with post-training quantizationNeural Networks, 2025
- Reducing the Teacher-Student Gap via Elastic Student2023
- Compressing medical deep neural network models for edge devices using knowledge distillationJournal of King Saud University Computer and Information Sciences, 2023
- Pruning-and-distillation: One-stage joint compression framework for CNNs via clusteringImage and Vision Computing, 2023
- Federated learning by employing knowledge distillation on edge devices with limited hardware resourcesNeurocomputing, 2023
- Falcon: lightweight and accurate convolution based on depthwise separable convolutionKnowledge and Information Systems, 2023
- PTQ4ViT: Post-training Quantization for Vision Transformers with Twin Uniform Quantization2022
- Knowledge Distillation: A SurveyInternational Journal of Computer Vision, 2021