Group Operation Assembly Language - A Flexible Way to Express Collective Communication
Authors:
- Torsten HoeflerTHTorsten Hoefler
- Christian SiebertCSChristian Siebert
- Andrew LumsdaineALAndrew Lumsdaine
- 1 September 2009
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 01903918,p. 574-581
Abstract
The implementation and optimization of collective communication operations is an important field of active research. Such operations directly influence application performance and need to map the communication requirements in an optimal way to steadily changing network architectures. In this work, we define an abstract domain-specific language to express arbitrary group communication operations. We show the universality of this language and how all existing collective operations can be implemented with it. By design, it readily lends itself to blocking and nonblocking execution, as well as to off-loaded execution of complex group communication operations. We also define several offline and online optimizations (compiler transformations and scheduling decisions, respectively) to improve the overall performance of the operation. Performance results show that the overhead to express current collective operations is negligible in comparison to the potential gains in a highly optimized implementation.Keywords
This publication has 9 references indexed in Scilit:
- Sparse collective operations for MPI2009
- Message progression in parallel computing - to thread or not to thread?2008
- Architecture of the Component Collective Messaging InterfaceLecture Notes in Computer Science, 2008
- Performance Analysis of MPI Collective Operations2005
- The implementation of the Cilk-5 multithreaded language1998
- Thread scheduling for multiprogrammed multiprocessors1998
- Efficient sparse LU factorization with partial pivoting on distributed memory architecturesIEEE Transactions on Parallel and Distributed Systems, 1998
- Two algorithms for barrier synchronizationInternational Journal of Parallel Programming, 1988
- Data-Driven and Demand-Driven Computer ArchitectureACM Computing Surveys, 1982