Scalable critical-path analysis and optimization guidance for hybrid MPI-CUDA applications
- 1 August 2016
- journal article
- research article
- Published by SAGE Publications in The International Journal of High Performance Computing Applications
- Vol. 31 (6) , 485-498
Abstract
The use of accelerators in heterogeneous systems is an established approach in designing petascale applications. Today, Compute Unified Device Architecture (CUDA) offers a rich programming interface for GPU accelerators but requires developers to incorporate several layers of parallelism on both the CPU and the GPU. From this increasing program complexity emerges the need for sophisticated performance tools. This work contributes by analyzing hybrid MPI-CUDA programs for properties based on wait states, such as the critical path, a metric proven to identify application bottlenecks effectively. We developed a tool to construct a dependency graph based on an execution trace and the inherent dependencies of the programming models CUDA and Message Passing Interface (MPI). Thereafter, it detects wait states and attributes blame to responsible activities. Together with the property of being on the critical path, we can identify activities that are most viable for optimization. To evaluate the global impact of optimizations to critical activities, we predict the program execution using a graph-based performance projection. The developed approach has been demonstrated with suitable examples to be both scalable and correct. Furthermore, we establish a new categorization of CUDA inefficiency patterns ensuing from the dependencies between CUDA activities.Keywords
This publication has 14 references indexed in Scilit:
- Characterizing Load and Communication Imbalance in Large-Scale Parallel Applications2012
- Analyzing lock contention in multithreaded applicationsACM SIGPLAN Notices, 2010
- The Vampir Performance Analysis Tool-Set2008
- Challenges and Opportunities in Many-Core ComputingProceedings of the IEEE, 2008
- Performance data collection using a hybrid approachACM SIGSOFT Software Engineering Notes, 2005
- Automatic performance analysis of hybrid MPI/OpenMP applicationsJournal of Systems Architecture, 2003
- Critical path analysis for the execution of parallel and distributed programs2003
- Waiting time analysis and performance visualization in Carnival1996
- IPS-2: the second generation of a parallel program measurement systemIEEE Transactions on Parallel and Distributed Systems, 1990
- Time, clocks, and the ordering of events in a distributed systemCommunications of the ACM, 1978