Enabling ScientificWorkflow Reuse through Structured Composition of Dataflow and Control-Flow
Authors:
- B. LudascherBLB. Ludascher
- A.H.H. NguANA.H.H. Ngu
- T. CritchlowTCT. Critchlow
- 1 January 2006
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 70
Abstract
Data-centric scientific workflows are often modeled as dataflow process networks. The simplicity of the dataflow framework facilitates workflow design, analysis, and optimization. However, modeling "control-flow intensive" tasks using dataflow constructs often leads to overly complicated workflows that are hard to comprehend, reuse, and maintain. We describe a generic framework, based on scientific workflow templates and frames, for embedding control-flow intensive subtasks within dataflow process networks. This approach can seamlessly handle complex control-flow without sacrificing the benefits of dataflow. We illustrate our approach with a real-world scientific workflow from the astrophysics domain, requiring remote execution and file transfer in a semi-reliable environment. For such workflows, we also describe a 3-layered architecture based on frames and templates where the top-layer consists of an overall dataflow process network, the second layer consists of a tranducer template for modeling the desired control-flow behavior, and the bottom layer consists of frames inside the template that are specialized by embedding the desired component implementation. Our approach can enable scientific workflows that are more robust (faulttolerance strategies can be defined by control-flow driven transducer templates) and at the same time more reusable, since the embedding of frames and templates yields more structured and modular workflow designs.Keywords
This publication has 19 references indexed in Scilit:
- Tools for composite web servicesACM SIGMOD Record, 2005
- Actor-Oriented Design of Scientific Workflows2005
- Seven Bottlenecks to Workflow Reuse and Repurposing2005
- Actor-Oriented Models for Codesign2004
- Workflow PatternsDistributed and Parallel Databases, 2003
- Managing process and service fusion in virtual enterprisesInformation Systems, 1999
- Hierarchical finite state machines with multiple concurrency modelsIEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 1999
- Dataflow process networksProceedings of the IEEE, 1995
- Managing heterogeneous multi-system tasks to support enterprise-wide operationsDistributed and Parallel Databases, 1995
- Static Scheduling of Synchronous Data Flow Programs for Digital Signal ProcessingIEEE Transactions on Computers, 1987