MIMD Programs Execution Support on SIMD Machines: A Holistic Survey
Open Access
- 4 March 2024
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 12, 34354-34377
Abstract
The Single Instruction Multiple Data (SIMD) architecture, supported by various high-performance computing platforms, efficiently utilizes data-level parallelism. The SIMD model is used in traditional CPUs, dedicated vector systems, and accelerators such as GPUs, vector extensions, and Xeon Phi. It provides performance throughput in computation-intensive and data-parallel applications. Despite the similarity of data-processing principles between these architectures, porting various programming models between the reviewed platforms is challenging. Furthermore, enhancing the programmability of these architectures is an important feature for utilizing their emerging computing power and simplifying programming complexity. This paper reviews the basic principles of optimization techniques to run asynchronous Multiple Instruction Multiple Data (MIMD) on SIMD accelerators. It also surveys several GPU programming paradigms and application programming interfaces (APIs) and classifies these frameworks into different groups based on their criteria. In addition, a review of studies that performed a comparison of the collaborative execution of GPUs with CPUs and Xeon Phi is presented in this paper. This study will be beneficial for developers and researchers in the field of computer architecture and parallel computing of intensive scientific applications, specifically for early-stage high-performance computing researchers, to obtain a brief overview of performance optimization opportunities as well as the challenges of existing SIMD platforms.Keywords
This publication has 241 references indexed in Scilit:
- Preparing Ginkgo for AMD GPUs – A Testimonial on Porting CUDA Code to HIP2021
- A Survey of Accelerator Architectures for Deep Neural NetworksEngineering, 2020
- Efficiency analysis of modern vector architectures: vector ALU sizes, core counts and clock frequenciesThe Journal of Supercomputing, 2019
- Potential of a modern vector supercomputer for practical applications: performance evaluation of SX-ACEThe Journal of Supercomputing, 2017
- Accelerating Detailed Tissue-Scale 3D Cardiac Simulations Using Heterogeneous CPU-Xeon Phi ComputingInternational Journal of Parallel Programming, 2016
- Experiences of Using the OpenMP Accelerator Model to Port DOE Stencil ApplicationsLecture Notes in Computer Science, 2015
- How Many Threads will be too Many? On the Scalability of OpenMP Implementations2015
- A Performance Study for Iterative Stencil Loops on GPUs with Ghost Zone OptimizationsInternational Journal of Parallel Programming, 2010
- Supporting OpenMP on CellInternational Journal of Parallel Programming, 2008
- Automatic performance analysis tools for the GridConcurrency and Computation: Practice and Experience, 2005