Communication Optimizations Used in the Paradigm Compiler for Distributed-Memory Multicomputers
Authors:
- Daniel PalermoDPDaniel Palermo
- Ernesto SuESErnesto Su
- John ChandyJCJohn Chandy
- Prithviraj BanerjeePBPrithviraj Banerjee
- 1 August 1994
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 2, 1-10
Abstract
The PARADIGM (PARAllelizing compiler for DIstributed-memory General-purpose Multicomputers) project at the University of Illinois provides a fully automated means to parallelize programs, written in a serial programming model, for execution on distributed-memory multicomputers. To provide efficient execution, PARADIGM automatically performs various optimizations to reduce the overhead and idle time caused by interprocessor communication. Optimizations studied in this paper include message coalescing, message vectorization, message aggregation, and coarse gram pipelining. To separate the optimization algorithms from machine-specific details, parameterized models are used to estimate communication and computation costs for a given machine. The models are also used in coarse gram pipelining to automatically select a task granularity that balances the available parallelism with the costs of communication. To determine the applicability of the optimizations on different machines, we analyzed their performance on an Intel iPSC/860, an Intel iPSC/2, and a Thinking Machines CM-5.Keywords
This publication has 13 references indexed in Scilit:
- A Convex Programming Approach for Exploiting Data and Functional Parallelism on Distributed Memory Multicomputers1994
- Techniques to overlap computation and communication in irregular iterative applications1994
- Automating Parallelism of Regular Computations for Distributed-Memory Multicomputers in the Paradigm Compiler1993
- Communication optimization and code generation for distributed memory machines1993
- Fortran 90D/HPF compiler for distributed memory MIMD computers1993
- Compiling Fortran D for MIMD distributed-memory machinesCommunications of the ACM, 1992
- Visualizing the performance of parallel programsIEEE Software, 1991
- Compiling communication-efficient programs for massively parallel machinesIEEE Transactions on Parallel and Distributed Systems, 1991
- Updating distributed variables in local computationsConcurrency: Practice and Experience, 1990
- Why systolic architectures?Computer, 1982