A Hybrid Intelligent Method for Performance Modeling and Prediction of Workflow Activities in Grids
Authors:
- Rubing DuanRDRubing Duan
- Farrukh NadeemFNFarrukh Nadeem
- Jie WangJWJie Wang
- Yun ZhangYZYun Zhang
- Radu ProdanRPRadu Prodan
- Thomas FahringerTFThomas Fahringer
- 1 January 2009
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 339-347
Abstract
Grid schedulers require individual activity performance predictions to map workflow activities on different Grid sites. The effectiveness of the scheduling systems is hampered by inaccurate predictions due to the inability of existing predictors to effectively model the dynamic and heterogeneous nature of Grid resources, or the wide range of problem sizes and runtime arguments. To address this deficiency, we propose a hybrid Bayesian-neural network approach to dynamically model and predict the execution time of activities in real workflow applications. Bayesian network is used for a high-level representation of activities performance probability distribution against different factors affecting the performance. The important attributes are dynamically selected by the Bayesian network and fed into a radial basis function neural network to make further predictions. Our approach is generic to any type of scientific applications, and flexible to import expert knowledge to further improve accuracies. Experimental results for activities from three realworld workflow applications are presented to show effectiveness of our approach.Keywords
This publication has 18 references indexed in Scilit:
- Performance prediction with skeletonsCluster Computing, 2007
- An Investigation into the Application of Different Performance Prediction Methods to Distributed Enterprise ApplicationsThe Journal of Supercomputing, 2005
- Metal enrichment processes in the intra-cluster mediumAstronomy & Astrophysics, 2005
- Grid Resource Selection by Application Benchmarking for Computational Haemodynamics ApplicationsLecture Notes in Computer Science, 2005
- Metal enrichment and energetics of galactic winds in galaxy clustersAdvances In Space Research, 2005
- Predicting application run times with historical informationJournal of Parallel and Distributed Computing, 2004
- Pace—A Toolset for the Performance Prediction of Parallel and Distributed SystemsThe International Journal of High Performance Computing Applications, 2000
- A historical application profiler for use by parallel schedulers1997
- An Experimental Comparison of Numerical and Qualitative Probabilistic Reasoning1994
- A Bayesian method for the induction of probabilistic networks from dataMachine Learning, 1992