Leveraging burst buffer coordination to prevent I/O interference
Authors:
- 1 October 2016
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 371-380
Abstract
Concurrent accesses to the shared storage resources in current HPC machines lead to severe performance degradation caused by I/O contention. In this study, we identify some key challenges to efficiently handling interleaved data accesses, and we propose a system-wide solution to optimize global performance. We implemented and tested several I/O scheduling policies, including prioritizing specific applications by leveraging burst buffers to defer the conflicting accesses from another application and/or directing the requests to different storage servers inside the parallel file system infrastructure. The results show that we mitigate the negative effects of interference and optimize the performance up to 2x depending on the selected I/O policy.Keywords
This publication has 20 references indexed in Scilit:
- TRIO: Burst Buffer Based I/O Orchestration2015
- Scheduling the I/O of HPC Applications Under Congestion2015
- CALCioM: Mitigating I/O Interference in HPC Systems through Cross-Application Coordination2014
- Insights for exascale IO APIs from building a petascale IO API2013
- Poster: Reservation-Based I/O Performance Guarantee for MPI-IO Applications Using Shared Storage Systems2012
- Characterizing output bottlenecks in a supercomputer2012
- Server-side I/O coordination for parallel file systems2011
- CA-NFSACM Transactions on Storage, 2009
- Efficient management of idleness in storage systemsACM Transactions on Storage, 2009
- A Novel network request scheduler for a large scale storage systemComputer Science - Research and Development, 2009