Large Language Model Compression for Autoregressive Text Generation: Methods, Evaluation, and Deployment Trade-Offs
Open Access
- 19 October 2026
- journal article
- review article
- Published by MDPI AG in Electronics
- Vol. 15 (20) , 4826
Abstract
Compression changes both the information retained by a large language model and the resources required to complete a request. This narrative review examines quantization, pruning and sparsity, knowledge distillation, low-rank factorization, and token, context, and key–value cache compression for autoregressive text generation, principally with decoder-only LLMs. Its central contribution is a synthesis of changes relative to each study’s baseline, connecting compression mechanisms to capability and deployment choices. Four numerical tables distinguish quality changes at stated compression budgets, distillation gains over same-size students, and paired or independent deployment measurements. Reporting is appraised by source identity, comparison conditions, measurement scope, artefacts, and uncertainty. The reviewed cases illustrate how capacity constraints change the feasible precision choices, how allocating a shared budget changes different task scores, and how compact-model construction and request compression address complementary resource demands. A deployment-validation sequence turns these findings into candidate checks for a target workload. Remaining questions concern transfer across capabilities, delayed use of compressed information, interactions between compression stages, and complete-task cost, including agent interactions.Keywords
This publication has 73 references indexed in Scilit:
- On-device large language models: a survey of model compression and system optimizationArtificial Intelligence Review, 2026
- Resource-Efficient Pruning for Transformer via Low-Rank Importance EstimationIEEE Transactions on Emerging Topics in Computational Intelligence, 2026
- A Survey of Quantization in LLM: Unlocking Potential Hardware EfficiencyJournal of Computer Science and Technology, 2026
- DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learningNature, 2025
- Context Embeddings for Efficient Answer Generation in Retrieval-Augmented Generation2025
- A Survey on Model Compression for Large Language ModelsTransactions of the Association for Computational Linguistics, 2024
- PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU2024
- Efficient Memory Management for Large Language Model Serving with PagedAttention2023
- Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured SparsityProceedings of the VLDB Endowment, 2023
- The PRISMA 2020 statement: an updated guideline for reporting systematic reviewsBMJ, 2021