Communication and Optimization Aspects of Parallel Programming Models on Hybrid Architectures
Authors:
- 1 February 2003
- journal article
- other
- Published by SAGE Publications in The International Journal of High Performance Computing Applications
- Vol. 17 (1) , 49-62
Abstract
Most HPC systems are clusters of shared memory nodes. Parallel programming must combine the distributed memory parallelization on the node interconnect with the shared memory parallelization inside each node. The hybrid MPI+OpenMP programming model is compared with pure MPI, compiler based parallelization, and other parallel programming models on hybrid architectures. The paper focuses on bandwidth and latency aspects, and also on whether programming paradigms can separate the optimization of communication and computation. Benchmark results are presented for hybrid and pure MPI communication. This paper analyzes the strengths and weaknesses of several parallel programming models on clusters of SMP nodes.Keywords
This publication has 16 references indexed in Scilit:
- Experiences Using OpenMP Based on Compiler Directed Software DSM on a PC Cluster2003
- Pseudo-Vectorization and RISC Optimization Techniques for the Hitachi SR8000 Architecture2003
- An MPI Benchmark Program Library and Its Application to the Earth SimulatorLecture Notes in Computer Science, 2002
- Compiler-Generated Vector-based Prefetching on Architectures with Distributed Memory2002
- High-Level Data Mapping for Clusters of SMPsLecture Notes in Computer Science, 2001
- UPC benchmarking issues2001
- Performance of Hybrid Message-Passing and Shared-Memory Parallelism for Discrete Element Modeling2000
- MPI versus MPI+OpenMP on the IBM SP for the NAS Benchmarks2000
- Co-array Fortran for parallel programmingACM SIGPLAN Fortran Forum, 1998
- A high-performance, portable implementation of the MPI message passing interface standardParallel Computing, 1996