Channel Pruning Base on Joint Reconstruction Error for Neural Network
Open Access
Abstract
In this paper, we propose a neural network channel pruning method based on the joint reconstruction error (JRE). To preserve the global discrimination ability of a pruned neural network, we propose the global reconstruction error. To ensure the integrity of information in the forward propagation process of a neural network, we propose the local reconstruction error. Finally, through normalization, the two magnitude mismatched losses are combined to obtain the joint error. The baseline network and pruned network are symmetrical structures. The importance of each channel in the pruned network is determined by the joint error between the channel and the corresponding channel in the baseline network. The proposed method prunes the channels in the pruned network according to the importance score and then restores its accuracy. The proposed method reduces the scale of the neural network and speeds up the model inferring speed without losing the accuracy of the neural network. Experimental results show the effectiveness of the method. For example, on the CIFAR-10 dataset, the proposed method prunes 50% of the channels of the VGG16 model, and the accuracy of the pruned model is 0.46% higher than that of the original model.Keywords
Funding Information
- Jilin Provincial Department of Science and Technology (20200201165JC)
This publication has 13 references indexed in Scilit:
- Pipeline FPGA-Based Implementations of ANNs for the Prediction of up to 600-Steps-Ahead of Chaotic Time SeriesJournal of Circuits, Systems and Computers, 2020
- HRank: Filter Pruning Using High-Rank Feature Map2020
- Centripetal SGD for Pruning Very Deep Convolutional Networks With Complicated Structure2019
- NISP: Pruning Networks Using Neuron Importance Score Propagation2018
- ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression2017
- Learning Efficient Convolutional Networks through Network Slimming2017
- Channel Pruning for Accelerating Very Deep Neural Networks2017
- A Gift from Knowledge Distillation: Fast Optimization, Network Minimization and Transfer Learning2017
- CP-decomposition with Tensor Power Method for Convolutional Neural Networks compression2017
- Deep Residual Learning for Image Recognition2016