Abstract
Large language models (LLMs) have achieved remarkable performance across various artificial intelligence (AI) applications. However, current LLMs cannot be deployed directly on edge nodes due to their large number of parameters. Fortunately, model compression technology has been proposed to reduce the computational workload and memory usage of LLMs, enabling further edge-based LLM services. However, existing research typically concentrates on isolated compression algorithms and lacks a comprehensive perspective on how to leverage these techniques for practical, end-to-end LLM deployment in edge environments. In this survey, we review edge-oriented LLM compression techniques and software–hardware codesign strategies to enable efficient LLM deployment on resource-constrained edge systems and guide future research in this area. First, we analyze techniques for LLM compression from the perspective of cloud–edge collaborative intelligence, including model quantization, parameter pruning, and knowledge distillation (KD). Second, we present several hybrid model frameworks tailored to dynamic, heterogeneous edge environments, based on model architecture, application scenarios, and combination selection. Third, we further refine a four-layer software–hardware codesign and an overhead-aware LLM deployment optimization. Finally, we discuss the challenges of current model compression approaches and offer insights into future research directions, with a focus on edge-based LLM services.
Funding Information
  • National Natural Science Foundation of China (U25A20436)
  • National Natural Science Foundation of China (62372047)
  • Guangxi Key Research and Development Program (FN2504240036)
  • Guangxi Key Research and Development Program (2025FN96441087)
  • Guangdong S&T Program (2025B0101120006)
  • Natural Science Foundation of Guangdong Province (2024A1515011323)
  • Supplemental Funds for Major Scientific Research Projects of Beijing Normal University, Zhuhai (ZHPT2023002)
  • Fundamental Research Funds for the Central Universities (2025JKZG069)
  • Higher Education Research Topics of Guangdong Association of Higher Education in the 14th Five-Year Plan (24GYB207)

This publication has 85 references indexed in Scilit: