Evaluating deep learning models for fault diagnosis of a rotating machinery with epistemic and aleatoric uncertainty
Preprint
Abstract
Uncertainty-aware deep learning (DL) models recently gained attention in fault diagnosis as a way to promote the reliable detection of faults when out-of-distribution (OOD) data arise from unseen faults (epistemic uncertainty) or the presence of noise (aleatoric uncertainty). In this paper, we present the first comprehensive comparative study of state-of-the-art uncertainty-aware DL architectures for fault diagnosis in rotating machinery, where different scenarios affected by epistemic uncertainty and different types of aleatoric uncertainty are investigated. The selected architectures include sampling by dropout, Bayesian neural networks, and deep ensembles. Moreover, to distinguish between in-distribution and OOD data in the different scenarios two uncertainty thresholds, one of which is introduced in this paper, are alternatively applied. Our empirical findings offer guidance to practitioners and researchers who have to deploy real-world uncertainty-aware fault diagnosis systems. In particular, they reveal that, in the presence of epistemic uncertainty, all DL models are capable of effectively detecting, on average, a substantial portion of OOD data across all the scenarios. However, deep ensemble models show superior performance, independently of the uncertainty threshold used for discrimination. In the presence of aleatoric uncertainty, the noise level plays an important role. Specifically, low noise levels hinder the models' ability to effectively detect OOD data. Even in this case, however, deep ensemble models exhibit a milder degradation in performance, dominating the others. These achievements, combined with their shorter inference time, make deep ensemble architectures the preferred choice.Keywords
All Related Versions
This publication has 44 references indexed in Scilit:
- Uncertainty-aware deep learning for reliable health monitoring in safety-critical energy systemsEnergy, 2024
- Towards trustworthy rotating machinery fault diagnosis via attention uncertainty in transformerJournal of Manufacturing Systems, 2023
- University of Ottawa constant load and speed rolling-element bearing vibration and acoustic fault signature datasetsData in Brief, 2023
- Intelligent manufacturing execution systems: A systematic reviewJournal of Manufacturing Systems, 2022
- A review of uncertainty quantification in deep learning: Techniques, applications and challengesInformation Fusion, 2021
- A probabilistic Bayesian recurrent neural network for remaining useful life prognostics considering epistemic and aleatory uncertaintiesStructural Control and Health Monitoring, 2021
- Fault detection and diagnosis for rotating machinery: A model based on convolutional LSTM, Fast Fourier and continuous wavelet transformsComputers in Industry, 2021
- Fault detection and identification using Bayesian recurrent neural networksComputers & Chemical Engineering, 2020
- An enhanced modulation signal bispectrum analysis for bearing fault detection based on non-Gaussian noise suppressionMeasurement, 2020
- The MBPEP: a deep ensemble pruning algorithm providing high quality uncertainty predictionApplied Intelligence, 2019