Ensemble of Bayesian Predictors and Decision Trees for Proactive Failure Management in Cloud Computing Systems
Authors:
- Qiang GuanQGQiang Guan
- Ziming ZhangZZZiming Zhang
- Song FuSFSong Fu
- 1 January 2012
- journal article
- Published by Engineering and Technology Publishing in Journal of Communications
- Vol. 7 (1)
Abstract
In modern cloud computing systems, hundreds and even thousands of cloud servers are interconnected by multi-layer networks. In such large-scale and complex systems, failures are common. Proactive failure management is a crucial technology to characterize system behaviors and forecast failure dynamics in the cloud. To make failure predictions, we need to monitor the system execution and collect health-related runtime performance data. However, in newly deployed or managed cloud systems, these data are usually unlabeled. Supervised learning based approaches are not suitable in this case. In this paper, we present an unsupervised failure detection method using an ensemble of Bayesian models. It characterizes normal execution states of the system and detects anomalous behaviors. After the anomalies are verified by system administrators, labeled data are available. Then, we apply supervised learning based on decision tree classifiers to predict future failure occurrences in the cloud. Experimental results in an institute-wide cloud computing system show that our methods can achieve high true positive rate and low false positive rate for proactive failure managementKeywords
This publication has 23 references indexed in Scilit:
- Quantifying event correlations for proactive failure management in networked computing systemsJournal of Parallel and Distributed Computing, 2010
- Failure-aware resource management for high-availability computing clusters with distributed virtual machinesJournal of Parallel and Distributed Computing, 2010
- Proactive Resource Management for Failure Resilient High Performance Computing Clusters2009
- Model-based performance evaluation of distributed checkpointing protocolsPerformance Evaluation, 2008
- Exploring event correlation for failure prediction in coalitions of clusters2007
- Quantifying Temporal and Spatial Correlation of Failure Events for Proactive Management2007
- Analytical Models for Architecture-Based Software Reliability Prediction: A Unification FrameworkIEEE Transactions on Reliability, 2006
- Proactive Fault Tolerance in MPI Applications Via Task Migration2006
- A survey of rollback-recovery protocols in message-passing systemsACM Computing Surveys, 2002
- Elements of Information Theory2001