An evaluative study on the effect of contention on message latencies in large supercomputers
Authors:
- 1 May 2009
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 52 (15302075) , 1-8
Abstract
Significant theoretical research was done on interconnect topologies and topology aware mapping for parallel computers in the 80s. With the deployment of virtual cut-through, wormhole routing and faster interconnects, message latencies reduced and research in the area died down. This paper presents a study showing that with the emergence of very large supercomputers, typically connected as a 3D torus or mesh, topology effects have become important again. It presents an evaluative study on the effect of contention on message latencies on torus and mesh networks. The paper uses three MPI benchmarks to evaluate the effect of hops (links) traversed by messages, on their latencies. The benchmarks demonstrate that when multiple messages compete for network resources, link occupancy or contention can increase message latencies by up to a factor of 8 times. In other words, contention leads to increased message latencies and reduces effective available bandwidth for each message. This suggests that application developers should consider interconnect topologies when mapping tasks to processors in order to obtain the best performance. Results are shown for two parallel machines - ANL's Blue Gene/P and PSC's XT3.Keywords
This publication has 10 references indexed in Scilit:
- Benefits of Topology Aware Mapping for Mesh InterconnectsParallel Processing Letters, 2008
- Application-specific topology-aware mapping for three dimensional topologies2008 IEEE International Symposium on Parallel and Distributed Processing, 2008
- Architecture---A performance comparison through benchmarking and modeling of three leading supercomputers2006
- Blue Gene/L torus interconnection networkIBM Journal of Research and Development, 2005
- Impact of PE Mapping on Cray T3E Message-Passing PerformanceLecture Notes in Computer Science, 2000
- Contention in the Cray T3D communication network1996
- A survey of wormhole routing techniques in direct networksComputer, 1993
- Performance analysis of virtual cut-through switching in HARTS: a hexagonal mesh multicomputerIEEE Transactions on Computers, 1991
- Nearest-Neighbor Mapping of Finite Element Graphs onto Processor MeshesIEEE Transactions on Computers, 1987
- On the Mapping ProblemIEEE Transactions on Computers, 1981