Boosting Lightweight CNNs Through Network Pruning and Knowledge Distillation for SAR Target Recognition
Authors:
Open Access
- 11 August 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
- Vol. 14 (19391404) , 8386-8397
Abstract
Deep convolutional neural networks (CNNs) have yielded unusually brilliant results in synthetic aperture radar (SAR) target recognition. However, overparameterization is a widely-recognized property of deep CNNs, and most previous works excessively pursued high accuracy but neglected the requirement of model deployment in radar systems, i.e., small computations and low memory cost. Therefore, further research on lightweight CNNs for SAR target recognition is necessary. In this article, we devise an effective CNN with channel-wise attention mechanism for SAR target recognition and then compress the network structure and recover lightweight network performance through network pruning and knowledge distillation, respectively. The attention values produced by the network are utilized to evaluate the importance of convolution kernels, and unimportant kernels are pruned. In addition, a novel bridge connection based knowledge distillation method is proposed. Instead of directly mimicking the hidden layer output or artificially designing a function to extract the knowledge in hidden layers, bridge connections are introduced to distill internal knowledge via teacher network. Experiments are conducted on the moving and stationary target acquisition and recognition benchmark dataset. The proposed network has excellent generalization performance and reaches an accuracy of 99.46% on the classification of ten-class targets without any data augmentation. Furthermore, through the network pruning and knowledge distillation algorithm, we cut down 90% parameters of the proposed CNN while maintaining model performance.Keywords
Funding Information
- National Natural Science Foundation of China (61771362)
- NSFC for Excellent Young Scholars
This publication has 28 references indexed in Scilit:
- Model Compression and Acceleration for Deep Neural Networks: The Principles, Progress, and ChallengesIEEE Signal Processing Magazine, 2018
- Learning Efficient Convolutional Networks through Network Slimming2017
- A Gift from Knowledge Distillation: Fast Optimization, Network Minimization and Transfer Learning2017
- Deep Convolutional Highway Unit Network for SAR Target Classification With Limited Labeled Training DataIEEE Geoscience and Remote Sensing Letters, 2017
- DeepSAR-Net: Deep convolutional neural networks for SAR target recognition2017
- Deep Residual Learning for Image Recognition2016
- Target Classification Using the Deep Convolutional Networks for SAR ImagesIEEE Transactions on Geoscience and Remote Sensing, 2016
- Convolutional Neural Network With Data Augmentation for SAR Target RecognitionIEEE Geoscience and Remote Sensing Letters, 2016
- SAR target recognition based on deep learning2014
- DeepFace: Closing the Gap to Human-Level Performance in Face Verification2014