Evaluation of Self-Healing Systems: An Analysis of the State-of-the-Art and Required Improvements
Open Access
Abstract
Evaluating the performance of self-adaptive systems is challenging due to their interactions with often highly dynamic environments. In the specific case of self-healing systems, the performance evaluations of self-healing approaches and their parameter tuning rely on the considered characteristics of failure occurrences and the resulting interactions with the self-healing actions. In this paper, we first study the state-of-the-art for evaluating the performances of self-healing systems by means of a systematic literature review. We provide a classification of different input types for such systems and analyse the limitations of each input type. A main finding is that the employed inputs are often not sophisticated regarding the considered characteristics for failure occurrences. To further study the impact of the identified limitations, we present experiments demonstrating that wrong assumptions regarding the characteristics of the failure occurrences can result in large performance prediction errors, disadvantageous design-time decisions concerning the selection of alternative self-healing approaches, and disadvantageous deployment-time decisions concerning parameter tuning. Furthermore, the experiments indicate that employing multiple alternative input characteristics can help with reducing the risk of premature disadvantageous design-time decisions.Keywords
This publication has 67 references indexed in Scilit:
- Intermodal freight transport planning – A receding horizon control approachTransportation Research Part C: Emerging Technologies, 2015
- Robustness-Driven Resilience Evaluation of Self-Adaptive Software SystemsIEEE Transactions on Dependable and Secure Computing, 2015
- Optimal planning for architecture-based self-adaptation via model checking of stochastic games2015
- SHõ WAACM Transactions on Autonomous and Adaptive Systems, 2015
- A survey on self-healing systems: approaches and systemsComputing, 2010
- Evaluating the adaptivity of computing systemsPerformance Evaluation, 2010
- Performance Implications of Failures in Large-Scale Cluster Scheduling2005
- Basic concepts and taxonomy of dependable and secure computingIEEE Transactions on Dependable and Secure Computing, 2004
- Improving cluster availability using workstation validationACM SIGMETRICS Performance Evaluation Review, 2002
- Dependability measurement and modeling of a multicomputer systemIEEE Transactions on Computers, 1993