Design and modeling of a non-blocking checkpointing system
Authors:
- 1 November 2012
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 21674329,p. 1-10
Abstract
As the capability and component count of systems increase, the MTBF decreases. Typically, applications tolerate failures with checkpoint/restart to a parallel file system (PFS). While simple, this approach can suffer from contention for PFS resources. Multi-level checkpointing is a promising solution. However, while multi-level checkpointing is successful on today's machines, it is not expected to be sufficient for exascale class machines, which are predicted to have orders of magnitude larger memory sizes and failure rates. Our solution combines the benefits of non-blocking and multi-level checkpointing. In this paper, we present the design of our system and model its performance. Our experiments show that our system can improve efficiency by 1.1 to 2.0x on future machines. Additionally, applications using our checkpointing system can achieve high efficiency even when using a PFS with lower bandwidth.Keywords
This publication has 13 references indexed in Scilit:
- On the role of burst buffers in leadership-class storage systems2012
- DataStager2009
- Comparative evaluation of overlap strategies with study of I/O overlap in MPI-IOACM SIGOPS Operating Systems Review, 2008
- Investigation of leading HPC I/O performance using a scientific-application derived benchmark2007
- Understanding failures in petascale computersJournal of Physics: Conference Series, 2007
- Improving the Performance of Remote I/O Using Asynchronous Primitives2006
- Providing Efficient I/O Redundancy in MPI Environments2004
- A case for two-level distributed recovery schemesACM SIGMETRICS Performance Evaluation Review, 1995
- A case for redundant arrays of inexpensive disks (RAID)1988
- A first order approximation to the optimum checkpoint intervalCommunications of the ACM, 1974