A performance study of grid workflow engines
Authors:
- 1 September 2008
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 21521085,p. 25-32
Abstract
To benefit from grids, scientists require grid workflow engines that automatically manage the execution of inter-related jobs on the grid infrastructure. So far, the workflows community has focused on scheduling algorithms and on interface tools. Thus, while several grid workflow engines have been deployed, little is known about their performance-related characteristics, and there are no commonly-used testing practices. This situation limits the adoption of the grid workflow engines, and hampers their tuning and their further development. In this work we propose a testing methodology for grid workflow engines that focuses on five characteristics: overhead, raw performance, stability, scalability, and reliability. Using this methodology, we evaluate in a real test environment several middleware stacks that include grid workflow engines, including two based on DAGMan/Condor and on Karajan/Globus.Keywords
This publication has 23 references indexed in Scilit:
- The Grid Workloads ArchiveFuture Generation Computer Systems, 2008
- The performance of bags-of-tasks in large-scale distributed systems2008
- Overhead Analysis of Scientific Workflows in Grid EnvironmentsIEEE Transactions on Parallel and Distributed Systems, 2008
- On the dynamic resource availability in grids2007
- Build-and-Test Workloads for Grid Middleware: Problem, Analysis, and Applications2007
- Measuring the Performance and Reliability of Production Computational Grids2006
- Scientific workflow management and the Kepler systemConcurrency and Computation: Practice and Experience, 2005
- Taverna: lessons in creating a workflow environment for the life sciencesConcurrency and Computation: Practice and Experience, 2005
- A taxonomy of scientific workflow systems for grid computingACM SIGMOD Record, 2005
- DSC: scheduling parallel tasks on an unbounded number of processorsIEEE Transactions on Parallel and Distributed Systems, 1994