Online Knowledge Distillation for Machine Health Prognosis Considering Edge Deployment
- 22 May 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Internet of Things Journal
- Vol. 11 (16) , 27828-27839
Abstract
Complex neural networks with deep structures are beneficial for solving problems, such as fault classification and health prediction of industrial equipment, due to their powerful feature extraction capabilities. Unfortunately, corresponding complex models designed based on deep learning algorithms require huge computational and memory resources, making them difficult to achieve effective edge deployment. In order to solve this difficulty with practical industrial significance, this article proposes an online knowledge distillation framework for machine health prognosis. Within this framework, the learned knowledge of complex networks can be distilled to simple networks that can be deployed on edge devices in sites. Specifically, the response-based knowledge distillation module, feature-based knowledge distillation module, and relation-based knowledge distillation module are, respectively, designed to achieve effective information transmission from different levels. Furthermore, the inherent differences between simple and complex networks have been fully considered for their impact on the efficiency of knowledge distillation, and an adaptive mutual learning strategy has been contrapuntally proposed to address this limitation. Multiple online knowledge distillation experiments were conducted on two different sets of run-to-failure data sets of mechanical key components with different pairs of complex and simple networks to verify the effectiveness of the proposed framework. The experimental results show that the simple student-networks can effectively improve prediction performance after receiving knowledge distillation from the complex teacher-networks, providing a new solution for machine health prognosis under the premise of edge deployment.Keywords
Funding Information
- National Natural Science Foundation of China (52075095)
- Postgraduate Research and Practice Innovation Program of Jiangsu Province (SJCX23_0055)
This publication has 27 references indexed in Scilit:
- Meta-learning with elastic prototypical network for fault transfer diagnosis of bearings under unstable speedsReliability Engineering & System Safety, 2024
- Uncertainty-aware deep learning for reliable health monitoring in safety-critical energy systemsEnergy, 2024
- Bayesian variational transformer: A generalizable model for rotating machinery fault diagnosisMechanical Systems and Signal Processing, 2023
- Challenges and opportunities for battery health estimation: Bridging laboratory research and real-world applicationsJournal of Energy Chemistry, 2023
- Semi-supervised adversarial discriminative learning approach for intelligent fault diagnosis of wind turbineInformation Sciences, 2023
- Complex domain extension network with multi-channels information fusion for remaining useful life prediction of rotating machineryMechanical Systems and Signal Processing, 2023
- Hybrid robust convolutional autoencoder for unsupervised anomaly detection of machine tools under noisesRobotics and Computer-Integrated Manufacturing, 2022
- Efficient temporal flow Transformer accompanied with multi-head probsparse self-attention mechanism for remaining useful life prognosticsReliability Engineering & System Safety, 2022
- A novel time–frequency Transformer based on self–attention mechanism and its application in fault diagnosis of rolling bearingsMechanical Systems and Signal Processing, 2022
- Hierarchical attention graph convolutional network to fuse multi-sensor signals for remaining useful life predictionReliability Engineering & System Safety, 2021