On the Scalability of Data Reduction Techniques in Current and Upcoming HPC Systems from an Application Perspective
Preprint
Abstract
We implement and benchmark parallel I/O methods for the fully-manycore driven particle-in-cell code PIConGPU. Identifying throughput and overall I/O size as a major challenge for applications on today's and future HPC systems, we present a scaling law characterizing performance bottlenecks in state-of-the-art approaches for data reduction. Consequently, we propose, implement and verify multi-threaded data-transformations for the I/O library ADIOS as a feasible way to trade underutilized host-side compute potential on heterogeneous systems for reduced I/O latency.Keywords
All Related Versions
This publication has 15 references indexed in Scilit:
- On the Scalability of Data Reduction Techniques in Current and Upcoming HPC Systems from an Application Perspective2017
- Real-Time Synthesis of Compression Algorithms for Scientific Data2016
- Performance-Portable Many-Core Plasma Simulations: Porting PIConGPU to OpenPower and Beyond2016
- Fixed-Rate Compressed Floating-Point ArraysIEEE Transactions on Visualization and Computer Graphics, 2014
- A User-Friendly Approach for Tuning Parallel File Operations2014
- Hello ADIOS: the challenges and lessons of developing leadership class I/O frameworksConcurrency and Computation: Practice and Experience, 2013
- dc-Magnetic-Field Generation in Unmagnetized Shear FlowsPhysical Review Letters, 2013
- DataSpaces: an interaction and coordination framework for coupled simulation workflowsCluster Computing, 2011
- PIConGPU: A Fully Relativistic Particle-in-Cell Code for a GPU ClusterIEEE Transactions on Plasma Science, 2010
- DataStager: scalable data staging services for petascale applicationsCluster Computing, 2010