Application Specific Compression of Deep Learning Models
Preprint
Abstract
Large Deep Learning models are compressed and deployed for specific applications. However, current Deep Learning model compression methods do not utilize the information about the target application. As a result, the compressed models are application agnostic. Our goal is to customize the model compression process to create a compressed model that will perform better for the target application. Our method, Application Specific Compression (ASC), identifies and prunes components of the large Deep Learning model that are redundant specifically for the given target application. The intuition of our work is to prune the parts of the network that do not contribute significantly to updating the data representation for the given application. We have experimented with the BERT family of models for three applications: Extractive QA, Natural Language Inference, and Paraphrase Identification. We observe that customized compressed models created using ASC method perform better than existing model compression methods and off-the-shelf compressed models.Keywords
All Related Versions
This publication has 10 references indexed in Scilit:
- ShortGPT: Layers in Large Language Models are More Redundant Than You Expect2025
- LaCo: Large Language Model Pruning via Layer Collapse2024
- Head-wise Shareable Attention for Large Language Models2024
- Greedy-layer pruning: Speeding up transformer models for natural language processingPattern Recognition Letters, 2022
- Compression of Deep Learning Models for Text: A SurveyACM Transactions on Knowledge Discovery From Data, 2022
- BinaryBERT: Pushing the Limit of BERT Quantization2021
- TinyBERT: Distilling BERT for Natural Language Understanding2020
- DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference2020
- Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned2019
- SQuAD: 100,000+ Questions for Machine Comprehension of Text2016