CHEX: CHannel EXploration for CNN Model Compression
Authors:
- 1 June 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 12277-12288
Abstract
Channel pruning has been broadly recognized as an effective technique to reduce the computation and memory cost of deep convolutional neural networks. However, conventional pruning methods have limitations in that: they are restricted to pruning process only, and they require a fully pre-trained large model. Such limitations may lead to sub-optimal model quality as well as excessive memory and training cost. In this paper, we propose a novel Channel Exploration methodology, dubbed as CHEX, to rectify these problems. As opposed to pruning-only strategy, we propose to repeatedly prune and regrow the channels throughout the training process, which reduces the risk of pruning important channels prematurely. More exactly: From intra-Layer's aspect, we tackle the channel pruning problem via a well-known column subset selection (CSS) formulation. From inter-Layer's aspect, our regrowing stages open a path for dynamically re-allocating the number of channels across all the layers under a global channel sparsity constraint. In addition, all the exploration process is done in a single training from scratch without the need of a pre-trained large model. Experimental results demonstrate that CHEX can effectively reduce the FLOPs of diverse CNN architectures on a variety of computer vision tasks, including image classification, object detection, instance segmentation, and 3D vision. For example, our compressed ResNet-50 model on ImageNet dataset achieves 76% top-l accuracy with only 25% FLOPs of the original ResNet-50 model, outperforming previous state-of-the-art channel pruning methods. The checkpoints and code are available at here.Keywords
All Related Versions
This publication has 40 references indexed in Scilit:
- ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression2017
- Learning Efficient Convolutional Networks through Network Slimming2017
- Channel Pruning for Accelerating Very Deep Neural Networks2017
- Mask R-CNN2017
- A scalable active framework for region annotation in 3D shape collectionsACM Transactions on Graphics, 2016
- Deep Residual Learning for Image Recognition2016
- Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification2015
- Provable deterministic leverage score sampling2014
- Matrix Analysis2012
- Efficient Algorithms for Computing a Strong Rank-Revealing QR FactorizationSIAM Journal on Scientific Computing, 1996