Structured Pruning of Convolutional Neural Networks via L1 Regularization
Open Access
- 5 August 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 7, 106385-106394
Abstract
Deep learning architecture has achieved amazing success in many areas with the recent advancements in convolutional neural networks (CNNs). However, real-time applications of CNNs are seriously hindered by the significant storage and computational costs. Structured pruning is a promising method to compress and accelerate CNNs and does not need special hardware or software for an auxiliary calculation. Here a simple strategy of structured pruning approach is proposed to crop unimportant filters or neurons automatically during the training stage. The proposed method introduces a mask for all filters or neurons to evaluate their importance. Thus the filters or neurons with zero mask are removed. To achieve this, the proposed method adopted L1 regularization to zero filters or neurons of CNNs. Experiments were conducted to assess the validity of this technique. The experiments showed that the proposed approach could crop 90.4%, 95.6% and 34.04% parameters on LeNet-5, VGG-16, and ResNet-32respectively, with a negligible loss of accuracy.Keywords
Funding Information
- Project of Scientific Operating Expenses from Ministry of Education of China (2017PT19)
This publication has 15 references indexed in Scilit:
- Pattern-Growth Based Mining Mouse-Interaction Behavior for an Active User Authentication SystemIEEE Transactions on Dependable and Secure Computing, 2017
- Learning Efficient Convolutional Networks through Network Slimming2017
- Channel Pruning for Accelerating Very Deep Neural Networks2017
- Performance Analysis of Multi-Motion Sensor Behavior for Active Smartphone AuthenticationIEEE Transactions on Information Forensics and Security, 2017
- Structured Pruning of Deep Convolutional Neural NetworksACM Journal on Emerging Technologies in Computing Systems, 2017
- Deep Residual Learning for Image Recognition2016
- Deep learningNature, 2015
- Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation2014
- Model Selection and Estimation in Regression with Grouped VariablesJournal of the Royal Statistical Society Series B: Statistical Methodology, 2005
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998