Predicting Node Failure in High Performance Computing Systems from Failure and Usage Logs
Authors:
- Nithin NakkaNNNithin Nakka
- Ankit AgrawalAAAnkit Agrawal
- Alok ChoudharyACAlok Choudhary
- 1 May 2011
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 15302075,p. 1557-1566
Abstract
In this paper, we apply data mining classification schemes to predict failures in a high performance computer system. Failure and Usage data logs collected on supercomputing clusters at Los Alamos National Laboratory (LANL) were used to extract instances of failure information. For each failure instance, past and future failure information is accumulated -- time of usage, system idle time, time of unavailability, time since last failure, time to next failure. We performed two separate analyses, with and without classifying the failures based on their root cause. Based on this data, we applied some popular decision tree classifiers to predict if a failure would occur within 1 hour. Our experiments show that our prediction system predicts failures with a high-degree of precision up to 73% and recall of about 80%. We also observed that employing the usage data along with the failure data has improved the accuracy of prediction.Keywords
This publication has 10 references indexed in Scilit:
- Adaptive Fault Management of Parallel Applications for High-Performance ComputingIEEE Transactions on Computers, 2008
- A large-scale study of failures in high-performance computing systems2006
- Failure data analysis of a large-scale heterogeneous server environment2004
- Experimental assessment of workstation failures and their impact on checkpointing systems2002
- A longitudinal survey of Internet host reliability2002
- Improving cluster availability using workstation validation2002
- Random ForestsMachine Learning, 2001
- A Bayesian method for the induction of probabilistic networks from dataMachine Learning, 1992
- Ridge Estimators in Logistic RegressionJournal of the Royal Statistical Society Series C: Applied Statistics, 1992
- Measurement and modeling of computer reliability as affected by system activityACM Transactions on Computer Systems, 1986