SHMEMPMI -- Shared Memory Based PMI for Improved Performance and Scalability
Authors:
- Sourav ChakrabortySCSourav Chakraborty
- Hari SubramoniHSHari Subramoni
- Jonathan PerkinsJPJonathan Perkins
- Dhabaleswar K. PandaDPDhabaleswar K. Panda
- 1 May 2016
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 60-69
Abstract
Dense systems with large number of cores per node are becoming increasingly popular. Existing designs of the Process Management Interface (PMI) show poor scalability in terms of performance and memory consumption on such systems with large number of processes concurrently accessing the PMI interface. Our analysis shows the local socket-based communication scheme used by PMI to be a major bottleneck. While using a shared memory based channel can avoid this bottleneck and thus reduce memory consumption and improve performance, there are several challenges associated with such a design. We investigate several such alternatives and propose a novel design that is based on a hybrid socket+shared memory based communication protocol and uses multiple shared memory regions. This design can reduce the memory usage per node by a factor of Processes per Node. Our evaluations show that memory consumption per node can be reduced by an estimated 1GB with 1 million MPI processes and 16 processes per node. Additionally, performance of PMI Get is improved by 1,000 times compared to the existing design. The proposed design is backward compatible, secure, and imposes negligible overhead.Keywords
This publication has 7 references indexed in Scilit:
- Non-Blocking PMI Extensions for Fast MPI Startup2015
- PMI Extensions for Scalable MPI Startup2014
- PMI: A Scalable Parallel Process-Management Interface for Extreme-Scale Systems2010
- MPI on a Million ProcessorsLecture Notes in Computer Science, 2009
- ScELA: Scalable and Extensible Launching Architecture for ClustersLecture Notes in Computer Science, 2008
- Dynamic hash tablesCommunications of the ACM, 1988
- Hash Table MethodsACM Computing Surveys, 1975