Compact Broad Learning System Based on Fused Lasso and Smooth Lasso
- 1 May 2023
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Cybernetics
- Vol. 54 (1) , 435-448
Abstract
Aiming at simplifying the network structure of broad learning system (BLS), this article proposes a novel simplification method called compact BLS (CBLS). Groups of nodes play an important role in the modeling process of BLS, and it means that there may be a correlation between nodes. The proposed CBLS not only focuses on the compactness of network structure but also pays closer attention to the correlation between nodes. Learning from the idea of Fused Lasso and Smooth Lasso, it uses the $L_{1}$ -regularization term and the fusion term to penalize each output weight and the difference between adjacent output weights, respectively. The $L_{1}$ -regularization term determines the correlation between the nodes and the outputs, whereas the fusion term captures the correlation between nodes. By optimizing the output weights iteratively, the correlation between the nodes and the outputs and the correlation between nodes are attempted to be considered in the simplification process simultaneously. Without reducing the prediction accuracy, finally, the network structure is simplified more reasonably and a sparse and smooth output weights solution is provided, which can reflect the characteristic of group learning of BLS. Furthermore, according to the fusion terms used in Fused Lasso and Smooth Lasso, two different simplification strategies are developed and compared. Multiple experiments based on public datasets are used to demonstrate the feasibility and effectiveness of the proposed methods.Keywords
Funding Information
- National Natural Science Foundation of China (61973304, 61873049, 62073060, 61751202, 61751205)
- Selection and Training Project of High-Level Talents in the Sixteenth/Six Talent Peak of Jiangsu Province (DZXX-045)
- Fundamental Research Funds for the Central Universities (2022ZZCX01K01)
This publication has 32 references indexed in Scilit:
- Model Compression and Acceleration for Deep Neural Networks: The Principles, Progress, and ChallengesIEEE Signal Processing Magazine, 2018
- Insights into randomized algorithms for neural networks: Practical issues and common pitfallsInformation Sciences, 2017
- Low-Rank Discriminant Embedding for Multiview LearningIEEE Transactions on Cybernetics, 2016
- A Multiobjective Sparse Feature Learning Model for Deep Neural NetworksIEEE Transactions on Neural Networks and Learning Systems, 2015
- Coordinate descent algorithmsMathematical Programming, 2015
- Dimensionality reduction for large-scale neural recordingsNature Neuroscience, 2014
- Regularization‐based statistical batch process modeling for final product quality predictionAIChE Journal, 2014
- Fast decorrelated neural network ensembles with random weightsInformation Sciences, 2014
- Combining sparseness and smoothness improves classification accuracy and interpretabilityNeuroImage, 2012
- The Smooth-Lasso and other ℓ1+ℓ2-penalized methodsElectronic Journal of Statistics, 2011