Optimizing MPI Collectives on Shared Memory Multi-Cores
Open Access
- 11 November 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1-15
Abstract
Message Passing Interface (MPI) programs often experience performance slowdowns due to collective communication operations, like broadcasting and reductions. As modern CPUs integrate more processor cores, running multiple MPI processes on shared-memory machines to take advantage of hardware parallelism is becoming increasingly common. In this context, it is crucial to optimize MPI collective communications for shared-memory execution. However, existing MPI collective implementations on shared-memory systems have two primary drawbacks. The first is extensive redundant data movements when performing reduction collectives, and the second is the ineffective use of non-temporal instructions to optimize streamed data processing. To address these limitations, this paper proposes two optimization techniques that minimize data movements and enhance the use of non-temporal instructions. We evaluated our techniques by integrating them into the OpenMPI library and tested their performance using micro-benchmarks and real-world applications running on two multi-core clusters. Experimental results show that our approach significantly outperforms existing techniques, yielding a 1.2--6.4x performance improvement.Keywords
Funding Information
- National Key Research and Development Program of China (2021YFB0300101)
- National Natural Science Foundation of China (61972408)
This publication has 35 references indexed in Scilit:
- Quantifying Performance Bottlenecks of Stencil Computations Using the Execution-Cache-Memory Model2015
- Comparison of different propagation steps for lattice Boltzmann methodsComputers & Mathematics with Applications, 2013
- Ownership passingACM SIGPLAN Notices, 2013
- KNEM: A generic and scalable kernel-assisted intra-node MPI communication frameworkJournal of Parallel and Distributed Computing, 2013
- Bandwidth optimal all-reduce algorithms for clusters of workstationsJournal of Parallel and Distributed Computing, 2009
- MPI Support for Multi-core Architectures: Optimized Shared Memory Collectives2008
- Efficient Shared Memory and RDMA Based Design for MPI_Allgather over InfiniBand2006
- Optimization of Collective Communication Operations in MPICHThe International Journal of High Performance Computing Applications, 2005
- MPICH2: A New Start for MPI Implementations2002
- Cache write policies and performanceACM SIGARCH Computer Architecture News, 1993