Optimization of All-to-All Communication on the Blue Gene/L Supercomputer
Authors:
- Sameer KumarSKSameer Kumar
- Yogish SabharwalYSYogish Sabharwal
- Rahul GargRGRahul Garg
- Philip HeidelbergerPHPhilip Heidelberger
- 1 September 2008
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 01903918,p. 320-329
Abstract
All-to-all communication is a well known performance bottleneck for many applications, such as the ones that use the Fast-Fourier-transform (FFT) algorithm. We analyze the performance of all-to-all communication on the BlueGene/L torus interconnect that has link contention even for all-to-all operations with short messages. We observed that the performance of all-to-all depends on the shape of the processor partition. We present a performance analysis of all-to-all on partitions of various shapes. We then present optimization schemes that substantially improve the performance of all-to-all with short and large messages.In particular, throughput improved from 64% to over 99% of peak on the 65,536 (64 times 32 times 32) node Blue Gene/L machine at the Lawrence Livermore National Lab. We show the impact of the all-to-all performance optimizations in 1-D and 3-D FFT benchmarks. We achieved a performance of over 2.8 TF for the HPC Challenge 1D FFT benchmark with our optimized all-to-all.Keywords
This publication has 12 references indexed in Scilit:
- Balanced Multicasting: High-throughput Communication for Grid Applications2005
- Efficient All-to-All Communication Patterns in Hypercube and Mesh Topologies2005
- Optimization of MPI collective communication on BlueGene/L systems2005
- Overview of the Blue Gene/L system architectureIBM Journal of Research and Development, 2005
- Blue Gene/L torus interconnection networkIBM Journal of Research and Development, 2005
- A framework for collective personalized communication2004
- Near-optimal all-to-all broadcast in multidimensional all-port meshes and toriIEEE Transactions on Parallel and Distributed Systems, 2002
- High-performance parallel FFT algorithms for the HITACHI SR80002000
- Optimal Algorithms for All-to-All Personalized Communication on Rings and Two Dimensional ToriJournal of Parallel and Distributed Computing, 1997
- Hybrid algorithms for complete exchange in 2D meshes1996