Divide-and-Merge Parallel Hierarchical Ensemble DNNs with Local Knowledge Augmentation
Open Access
Abstract
Traditional deep neural networks (DNNs) often suffer from a time-consuming training process, which is restricted by accumulation of excessive network layers and a large amount of parameters. More neural units are required to be stacked to achieve desirable performance. Specifically, when dealing with large-scale datasets, a single DNN can hardly obtain the best performance on the available limited computing resources. To address the issues above, in this paper, a novel Parallel Hierarchical Ensemble Deep Neural Network (PH-E-DNN) is proposed to improve accuracy and efficiency of the deep network. Firstly, the fuzzy C-means algorithm (FCM) is adopted so that the large-scale dataset is separated into several small data partitions. As a benefit of the fuzzy partitioning of the FCM, several sub-models can be obtained through learning their respective data partitions and isolating them from the others. Secondly, the prediction results of each sub-model in the current level are used as the discriminative knowledge appended to original regional subsets, and predictions from each level symmetrically augment inputs for the next level. In the PH-E-DNN architecture, predictions from each level symmetrically augment inputs for the next level, creating a symmetrical flow of discriminative knowledge across the hierarchical structure. Finally, multiple regional subsets are merged to form a global augmented dataset, while multi-level parallel sub-models are stacked to organize a large-scale deep ensemble network. More importantly, only the multiple DNNs in the last level are ensembled to generate the decision result of the proposed PH-E-DNN. Extensive experiments demonstrate that the PH-E-DNN is superior to some traditional and deep learning models, only requiring a few parameters to be set, which demonstrates its efficiency and flexibility.Keywords
Funding Information
- National Natural Science Foundation of China (62376094, 62206177, 62106145, LY23F020007, LQ22F020024, Y202248951)
- Natural Science Foundation of Zhejiang Province (62376094, 62206177, 62106145, LY23F020007, LQ22F020024, Y202248951)
- General Scientific Research Project of Zhejiang Education Department (62376094, 62206177, 62106145, LY23F020007, LQ22F020024, Y202248951)
This publication has 39 references indexed in Scilit:
- Deep Learning for Image Reconstruction in Electrical Tomography: A ReviewIEEE Sensors Journal, 2025
- A stacked autoencoder‐based convolutional and recurrent deep neural network for detecting cyberattacks in interconnected power control systemsInternational Journal of Intelligent Systems, 2021
- Fuzzy decision trees embedded with evolutionary fuzzy clustering for locating users using wireless signal strength in an indoor environmentInternational Journal of Intelligent Systems, 2021
- Gated Stacked Target-Related Autoencoder: A Novel Deep Feature Extraction and Layerwise Ensemble Method for Industrial Soft Sensor ApplicationIEEE Transactions on Cybernetics, 2020
- A Novel Ensemble Deep Learning Approach for Sleep-Wake Detection Using Heart Rate Variability and AccelerationIEEE Transactions on Emerging Topics in Computational Intelligence, 2020
- Coupled Deep Autoencoder for Single Image Super-ResolutionIEEE Transactions on Cybernetics, 2015
- A Novel Evolutionary Kernel Intuitionistic Fuzzy $C$ -means Clustering AlgorithmIEEE Transactions on Fuzzy Systems, 2014
- Sparse Extreme Learning Machine for ClassificationIEEE Transactions on Cybernetics, 2014
- Support Vector Machine-Based Classification Scheme for Myoelectric Control Applied to Upper LimbIEEE Transactions on Biomedical Engineering, 2008
- Stacked generalizationNeural Networks, 1992