LB4OMP: A Dynamic Load Balancing Library for Multithreaded Applications
Open Access
- 27 August 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Parallel and Distributed Systems
- Vol. 33 (4) , 830-841
Abstract
Exascale computing systems will exhibit high degrees of hierarchical parallelism, with thousands of computing nodes and hundreds of cores per node. Efficiently exploiting hierarchical parallelism is challenging due to load imbalance that arises at multiple levels. OpenMP is the most widely-used standard for expressing and exploiting the ever-increasing node-level parallelism. The scheduling options in OpenMP are insufficient to address the load imbalance that arises during the execution of multithreaded applications. The limited scheduling options in OpenMP hinder research on novel scheduling techniques which require comparison with others from the literature. This work introduces LB4OMP, an open-source dynamic load balancing library that implements successful scheduling algorithms from the literature. LB4OMP is a research infrastructure designed to spur and support present and future scheduling research, for the benefit of multithreaded applications performance. Through an extensive performance analysis campaign, we assess the effectiveness and demystify the performance of all loop scheduling techniques in the library. We show that, for numerous applications-systems pairs, the scheduling techniques in LB4OMP outperform the scheduling options in OpenMP. Node-level load balancing using LB4OMP leads to reduced cross-node load imbalance and to improved MPI+OpenMP applications performance, which is critical for Exascale computing.Keywords
All Related Versions
Funding Information
- European Unions Horizon 2020 Research and Innovation Programme (957407)
- Schweizerischer Nationalfonds zur Frderung der Wissenschaftlichen Forschung (169123)
- Swiss Platform for Advanced Scientific Computing
This publication has 28 references indexed in Scilit:
- Knowledge-Based Adaptive Self-Scheduling2012
- The International Exascale Software Project roadmapThe International Journal of High Performance Computing Applications, 2011
- Evaluation of OpenMP Task Scheduling StrategiesLecture Notes in Computer Science, 2008
- Detecting Application Load Imbalance on High End Massively Parallel Systems2007
- GROMACS: Fast, flexible, and freeJournal of Computational Chemistry, 2005
- Is the Schedule Clause Really Necessary in OpenMP?2003
- Allocating Independent Tasks to Parallel Processors: An Experimental StudyJournal of Parallel and Distributed Computing, 1997
- Trapezoid self-scheduling: a practical scheduling scheme for parallel compilersIEEE Transactions on Parallel and Distributed Systems, 1993
- Factoring: a method for scheduling parallel loopsCommunications of the ACM, 1992
- The NP-completeness column: an ongoing guideJournal of Algorithms, 1985