Exploring void search for fault detection on extreme scale systems
- 1 September 2014
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 1-9
Abstract
Mean Time Between Failures (MTBF), now calculated in days or hours, is expected to drop to minutes on exascale machines. The advancement of resilience technologies greatly depends on a deeper understanding of faults arising from hardware and software components. This understanding has the potential to help us build better fault tolerance technologies. For instance, it has been proved that combining checkpointing and failure prediction leads to longer checkpoint intervals, which in turn leads to fewer total checkpoints. In this paper we present a new approach for fault detection based on the Void Search (VS) algorithm. VS is used primarily in astrophysics for finding areas of space that have a very low density of galaxies. We evaluate our algorithm using real environmental logs from Mira Blue Gene/Q supercomputer at Argonne National Laboratory. Our experiments show that our approach can detect almost all faults (i.e., sensitivity close to 1) with a low false positive rate (i.e., specificity values above 0.7). We also compare our algorithm with a number of existing detection algorithms, and find that ours outperforms all of them.Keywords
This publication has 28 references indexed in Scilit:
- A study of dynamic meta-learning for failure prediction in large-scale systemsJournal of Parallel and Distributed Computing, 2010
- The WEKA data mining softwareACM SIGKDD Explorations Newsletter, 2009
- Anomaly detectionACM Computing Surveys, 2009
- A cosmic watershed: the WVF void detection techniqueMonthly Notices of the Royal Astronomical Society, 2007
- Understanding failures in petascale computersJournal of Physics: Conference Series, 2007
- Basic concepts and taxonomy of dependable and secure computingIEEE Transactions on Dependable and Secure Computing, 2004
- Voids in the Two‐Degree Field Galaxy Redshift SurveyThe Astrophysical Journal, 2004
- A Simple Void‐searching AlgorithmThe Astrophysical Journal, 1998
- Algorithm AS 136: A K-Means Clustering AlgorithmJournal of the Royal Statistical Society Series C: Applied Statistics, 1979
- Analysis of a complex of statistical variables into principal components.Journal of Educational Psychology, 1933