Enhance the Hidden Structure of Deep Neural Networks by Double Laplacian Regularization
- 24 March 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Circuits and Systems II: Express Briefs
- Vol. 70 (8) , 3114-3118
Abstract
The Laplacian regularization has been widely used in neural networks due to its ability to improve generalization performance, which enforces adjacent samples with the same labels to share similar features. However, most existing methods only consider the global structure of the data with the same labels, but neglect samples in boundary areas with different labels. To address this limitation and improve performance, this paper proposes a novel regularization method that enhances the hidden structure of deep neural networks. Our proposed method imposes a double Laplacian regularization on the objective function and leverages full data information to capture its hidden structure in the manifold space. The double Laplacian regularization applies both attraction and repulsion effects on the hidden layer, which encourages the hidden features of instances with the same label to be closer, and forces those of different categories to be further away.Extensive experiments demonstrate the proposed method leads to significant improvements in accuracy on different types of deep neural networks.Keywords
This publication has 19 references indexed in Scilit:
- Graph Learning for Multiview ClusteringIEEE Transactions on Cybernetics, 2017
- Look Closer to See Better: Recurrent Attention Convolutional Neural Network for Fine-Grained Image Recognition2017
- Group sparse regularization for deep neural networksNeurocomputing, 2017
- Online gradient method with smoothing ℓ 0 regularization for feedforward neural networksNeurocomputing, 2017
- Deep learningNature, 2015
- Discriminative graph regularized extreme learning machine and its application to face recognitionNeurocomputing, 2015
- Deep learning in neural networks: An overviewNeural Networks, 2015
- A pruning algorithm with L 1/2 regularizer for extreme learning machineJournal of Zhejiang University SCIENCE C, 2014
- LIBSVMACM Transactions on Intelligent Systems and Technology, 2011
- A Biomedical System Based on Artificial Neural Network and Principal Component Analysis for Diagnosis of the Heart Valve DiseasesJournal of Medical Systems, 2010