Accelerating convolutional neural networks by group-wise 2D-filter pruning
Authors:
- 1 May 2017
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 2502-2509
Abstract
Network pruning is an effective way to accelerate Convolutional Neural Networks (CNNs). In recent years, structured pruning methods are proposed in favor of unstructured methods as they have shown greater speedup in practical use. Existing structured methods does pruning along two main dimensions: 3D-filter wise, i.e., remove a 3D-fllter as a whole, and filter-shape wise, i.e., remove a same position from all 3D-filters. In this work, we propose a new group-wise 2D-fllter pruning approach that is orthogonal and complementary to the existing methods. The proposed approach removes a portion of 2D-fllters from each 3D-filter according to the pruning patterns learned from the data, and leads to compressed models that do not require sophisticated implementation of convolution operations. A fine-tuning process is followed to recover the accuracy. The knowledge distillation (KD) framework is explored in the fine-tuning process to improve the performance. We present our method for learning the pruning pattens as well as the fine-tuning strategy based on knowledge distillation. The proposed approach is validated on two representative CNN models - ZF and VGG16, pre-trained on ILSVRC12. Experimental results demonstrate the effectiveness of our approach. In VGG16, we get even higher accuracy after speeding-up the network by 4 times.Keywords
This publication has 8 references indexed in Scilit:
- Conditional Random Fields as Recurrent Neural Networks2015
- Channel-level acceleration of deep face representationsIEEE Access, 2015
- Going deeper with convolutions2015
- Fully convolutional networks for semantic segmentation2015
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Fixed point optimization of deep convolutional neural networks for object recognition2015
- Fixed-point feedforward deep neural network design using weights +1, 0, and −12014
- Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation2014