Integrating databases and workflow systems
Authors:
Open Access
- 1 September 2005
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM SIGMOD Record
- Vol. 34 (3) , 5-11
Abstract
There has been an information explosion in fields of science such as high energy physics, astronomy, environmental sciences and biology. There is a critical need for automated systems to manage scientific applications and data. Database technology is well-suited to handle several aspects of workflow management. Contemporary workflow systems are built from multiple, separately developed components and do not exploit the full power of DBMSs in handling data of large magnitudes. We advocate a holistic view of a WFMS that includes not only workflow modeling but planning, scheduling, data management and cluster management. Thus, it is worthwhile to explore the ways in which databases can be augmented to manage workflows in addition to data. We present a language for modeling workflows that is tightly integrated with SQL. Each scientific program in a workflow is associated with an active table or view. The definition of data products is in relational format, and invocation of programs and querying is done in SQL. The tight coupling between workflow management and data-manipulation is an advantage for data-intensive scientific programs.Keywords
This publication has 12 references indexed in Scilit:
- Lessons Learned from Managing a Petabyte2005
- Pegasus: Mapping Scientific Workflows onto the Grid2004
- Chimera: a virtual data system for representing, querying, and automating data derivation2002
- Optimization of queries with user-defined predicatesACM Transactions on Database Systems, 1999
- Workflow, transactions and datalog1999
- Active database systemsACM Computing Surveys, 1999
- Relational transducers for electronic commerce1998
- Optimization of dynamic query evaluation plans1994
- The Gamma database machine projectIEEE Transactions on Knowledge and Data Engineering, 1990
- Query Processing in R*1985