GPU Peer-to-Peer Techniques Applied to a Cluster Interconnect
Authors:
- Roberto AmmendolaRARoberto Ammendola
- Massimo BernaschiMBMassimo Bernaschi
- Andrea BiagioniABAndrea Biagioni
- Mauro BissonMBMauro Bisson
- Massimiliano FaticaMFMassimiliano Fatica
- Ottorino FrezzaOFOttorino Frezza
- Francesca Lo CiceroFCFrancesca Lo Cicero
- Alessandro LonardoALAlessandro Lonardo
- Enrico MastrostefanoEMEnrico Mastrostefano
- Pier Stanislao PaolucciPPPier Stanislao Paolucci
- Davide RossettiDRDavide Rossetti
- Francesco SimulaFSFrancesco Simula
- 1 May 2013
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 7490, 806-815
Abstract
Modern GPUs support special protocols to exchange data directly across the PCI Express bus. While these protocols could be used to reduce GPU data transmission times, basically by avoiding staging to host memory, they require specific hardware features which are not available on current generation network adapters. In this paper we describe the architectural modifications required to implement peer-to-peer access to NVIDIA Fermi- and Kepler-class GPUs on an FPGA-based cluster interconnect. Besides, the current software implementation, which integrates this feature by minimally extending the RDMA programming model, is discussed, as well as some issues raised while employing it in a higher level API like MPI. Finally, the current limits of the technique are studied by analyzing the performance improvements on low-level benchmarks and on two GPU-accelerated applications, showing when and how they seem to benefit from the GPU peer-to-peer method.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- Benchmarking of communication techniques for GPUsJournal of Parallel and Distributed Computing, 2013
- Breadth First Search on APEnet+2012
- An introduction to multi-GPU programming for physicistsThe European Physical Journal Special Topics, 2012
- MPI-ACC: An Integrated and Extensible Approach to Data Movement in Accelerator-based Systems2012
- OMB-GPU: A Micro-Benchmark Suite for Evaluating MPI Libraries on GPU Clusters2012
- Efficient Parallel Graph Exploration on Multi-Core CPU and GPU2011
- Benchmarking GPU and CPU codes for Heisenberg spin glass over-relaxationComputer Physics Communications, 2011
- APENet: LQCD clusters à la APENuclear Physics B - Proceedings Supplements, 2005
- LogP: towards a realistic model of parallel computation1993