What Can We Learn from Four Years of Data Center Hardware Failures?
Authors:
- Guosai WangGWGuosai Wang
- Lifei ZhangLZLifei Zhang
- Wei XuWXWei Xu
- 1 June 2017
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 25-36
Abstract
Hardware failures have a big impact on the dependability of large-scale data centers. We present studies on over 290,000 hardware failure reports collected over the past four years from dozens of data centers with hundreds of thousands of servers. We examine the dataset statistically to discover failure characteristics along the temporal, spatial, product line and component dimensions. We specifically focus on the correlations among different failures, including batch and repeating failures, as well as the human operators' response to the failures. We reconfirm or extend findings from previous studies. We also find many new failure and recovery patterns that are the undesirable by-product of the state-of-the-art data center hardware and software design.Keywords
This publication has 22 references indexed in Scilit:
- Understanding and Exploiting Spatial Properties of System Failures on Extreme-Scale HPC Systems2015
- Understanding GPU errors on large-scale HPC systems and the implications for system design and operation2015
- Failure Analysis of Virtual and Physical Machines: Patterns, Causes and Characteristics2014
- Lessons Learned from the Analysis of System Failures at Petascale: The Case of Blue Waters2014
- Lazy Checkpointing: Exploiting Temporal Locality in Failures to Mitigate Checkpointing Overheads on Extreme-Scale Systems2014
- Failure Analysis and Modeling in Large Multi-site Infrastructures2013
- DRAM errors in the wild2009
- Understanding failures in petascale computersJournal of Physics: Conference Series, 2007
- What Supercomputers Say: A Study of Five System Logs2007
- BlueGene/L Failure Analysis and Prediction Models2006