ImPart: Importance-Aware Delta-Sparsification for Improved Model Compression and Merging in LLMs
Open Access
- 1 January 2025
- conference paper
- Published by Association for Computational Linguistics (ACL)
- p. 18817-18829
Abstract
Yan Yang, Yixia Li, Hongru Wang, Xuetao Wei, James Jianqiao Yu, Yun Chen, Guanhua Chen. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- DeltaZip: Efficient Serving of Multiple Full-Model-Tuned LLMs2025
- Agile-Quant: Activation-Guided Quantization for Faster Inference of LLMs on the EdgeProceedings of the AAAI Conference on Artificial Intelligence, 2024
- BitDelta: Your Fine-Tune May Only Be Worth One Bit2024
- Change Is the Only Constant: Dynamic LLM Slicing based on Layer Redundancy2024
- FollowBench: A Multi-level Fine-grained Constraints Following Benchmark for Large Language Models2024
- Adaptive Rank Selections for Low-Rank Approximation of Language Models2024
- Merging Models with Fisher-Weighted Averaging2022
- Are NLP Models really able to Solve Simple Math Word Problems?2021
- Optimization based Layer-wise Magnitude-based Pruning for DNN Compression2018