Large Multimodal Model Compression via Iterative Efficient Pruning and Distillation
Open Access
- 13 May 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 235-244
Abstract
The deployment of Large Multimodal Models (LMMs) within Ant Group has significantly advanced multimodal tasks in payment, security, and advertising, notably enhancing advertisement audition tasks in Alipay. However, the deployment of such sizable models introduces challenges, particularly in increased latency and carbon emissions, which are antithetical to the ideals of Green AI. This paper introduces a novel multi-stage compression strategy for our proprietary LLM, AntGMM. Our methodology pivots on three main aspects: employing small training sample sizes, addressing multi-level redundancy through multi-stage pruning, and introducing an advanced distillation loss design. In our research, we constructed a dataset, the Multimodal Advertisement Audition Dataset (MAAD), from real-world scenarios within Alipay, and conducted experiments to validate the reliability of our proposed strategy. Furthermore, the effectiveness of our strategy is evident in its operational success in Alipay's real-world multimodal advertisement audition for three months from September 2023. Notably, our approach achieved a substantial reduction in latency, decreasing it from 700ms to 90ms, while maintaining online performance with only a slight performance decrease. Moreover, our compressed model is estimated to reduce electricity consumption by approximately 75 million kWh annually compared to the direct deployment of AntGMM, demonstrating our commitment to green AI initiatives.Keywords
All Related Versions
Funding Information
- City University of Hong Kong
- Ant Group
This publication has 17 references indexed in Scilit:
- A Survey on Model Compression and Acceleration for Pretrained Language ModelsProceedings of the AAAI Conference on Artificial Intelligence, 2023
- CLIP2: Contrastive Language-Image-Point Pretraining from Real-World Point Cloud Data2023
- Distilling Reasoning Capabilities into Smaller Language Models2023
- EBERT: Efficient BERT Inference with Dynamic Structured Pruning2021
- Knowledge distillation and data augmentation for NLP light pre-trained modelsJournal of Physics: Conference Series, 2020
- Structured Compression by Weight Encryption for Unstructured Pruning and Quantization2020
- Q-BERT: Hessian Based Ultra Low Precision Quantization of BERTProceedings of the AAAI Conference on Artificial Intelligence, 2020
- Distilling Knowledge from Well-Informed Soft Labels for Neural Relation ExtractionProceedings of the AAAI Conference on Artificial Intelligence, 2020
- Importance Estimation for Neural Network Pruning2019
- Structured Pruning of Deep Convolutional Neural NetworksACM Journal on Emerging Technologies in Computing Systems, 2017