A Checkpoint of Research on Parallel I/O for High-Performance Computing
Open Access
- 12 March 2018
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Computing Surveys
- Vol. 51 (2) , 1-35
Abstract
We present a comprehensive survey on parallel I/O in the high-performance computing (HPC) context. This is an important field for HPC because of the historic gap between processing power and storage latency, which causes application performance to be impaired when accessing or generating large amounts of data. As the available processing power and amount of data increase, I/O remains a central issue for the scientific community. In this survey article, we focus on a traditional I/O stack, with a POSIX parallel file system. We present background concepts everyone could benefit from. Moreover, through the comprehensive study of publications from the most important conferences and journals in a 5-year time window, we discuss the state of the art in I/O optimization approaches, access pattern extraction techniques, and performance modeling, in addition to general aspects of parallel I/O research. With this approach, we aim at identifying the general characteristics of the field and the main current and future research topics.Keywords
Funding Information
- EU H2020 Programme
- MCTI/RNP-Brazil under the HPC4E project (689772)
This publication has 91 references indexed in Scilit:
- Hybrid hierarchy storage system in MilkyWay-2 supercomputerFrontiers of Computer Science, 2014
- CALCioM: Mitigating I/O Interference in HPC Systems through Cross-Application Coordination2014
- EDM: An Endurance-Aware Data Migration Scheme for Load Balancing in SSD Storage Clusters2014
- A survey on techniques for improving the energy efficiency of large-scale distributed systemsACM Computing Surveys, 2014
- Design and Implementation of an Asymmetric Block-Based Parallel File SystemIEEE Transactions on Computers, 2013
- A dynamic and adaptive load balancing strategy for parallel file system with large-scale I/O serversJournal of Parallel and Distributed Computing, 2012
- Understanding I/O Performance Using I/O Skeletal ApplicationsLecture Notes in Computer Science, 2012
- Understanding and Improving Computational Science Storage Access through Continuous CharacterizationACM Transactions on Storage, 2011
- Implementation and evaluation of active storage in modern parallel file systemsParallel Computing, 2010
- The Google file systemACM SIGOPS Operating Systems Review, 2003