Processing MPI Derived Datatypes on Noncontiguous GPU-Resident Data
Authors:
- 16 September 2013
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Parallel and Distributed Systems
- Vol. 25 (10) , 2627-2637
Abstract
Driven by the goals of efficient and generic communication of noncontiguous data layouts in GPU memory, for which solutions do not currently exist, we present a parallel, noncontiguous data-processing methodology through the MPI datatypes specification. Our processing algorithm utilizes a kernel on the GPU to pack arbitrary noncontiguous GPU data by enriching the datatypes encoding to expose a fine-grained, data-point level of parallelism. Additionally, the typically tree-based datatype encoding is preprocessed to enable efficient, cached access across GPU threads. Using CUDA, we show that the computational method outperforms DMA-based alternatives for several common data layouts as well as more complex data layouts for which reasonable DMA-based processing does not exist. Our method incurs low overhead for data layouts that closely match best-case DMA usage or that can be processed by layout-specific implementations. We additionally investigate usage scenarios for data packing that incur resource contention, identifying potential pitfalls for various packing strategies. We also demonstrate the efficacy of kernel-based packing in various communication scenarios, showing multifold improvement in point-to-point communication and evaluating packing within the context of the SHOC stencil benchmark and HACC mesh analysis.Keywords
This publication has 11 references indexed in Scilit:
- Meshing the Universe: Integrating Analysis in Cosmological Simulations2012
- Optimized Non-contiguous MPI Datatype Communication for GPU Clusters: Design, Implementation and Evaluation with MVAPICH22011
- MVAPICH2-GPU: optimized GPU to GPU communication for InfiniBand clustersComputer Science - Research and Development, 2011
- High Performance Stencil Code Algorithms for GPGPUsProcedia Computer Science, 2011
- 3.5-D Blocking Optimization for Stencil Computations on Modern CPUs and GPUs2010
- The Scalable Heterogeneous Computing (SHOC) benchmark suite2010
- Message passing on data-parallel architectures2009
- Message passing for GPGPU clusters: CudaMPI2009
- Zippy: A Framework for Computation and Visualization on a GPU ClusterComputer Graphics Forum, 2008
- Implementing Fast and Reusable Datatype Processing2003