Logical provenance in data-oriented workflows?
Authors:
- 1 April 2013
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- No. 10636382,p. 877-888
Abstract
We consider the problem of defining, generating, and tracing provenance in data-oriented workflows, in which input data sets are processed by a graph of transformations to produce output results. We first give a new general definition of provenance for general transformations, introducing the notions of correctness, precision, and minimality. We then determine when properties such as correctness and minimality carry over from the individual transformations' provenance to the workflow provenance. We describe a simple logical-provenance specification language consisting of attribute mappings and filters. We provide an algorithm for provenance tracing in workflows where logical provenance for each transformation is specified using our language. We consider logical provenance in the relational setting, observing that for a class of Select-Project-Join (SPJ) transformations, logical provenance specifications encode minimal provenance. We have built a prototype system supporting the features and algorithms presented in the paper, and we report a few preliminary experimental results.Keywords
This publication has 20 references indexed in Scilit:
- Logical provenance in data-oriented workflows?2013
- Provenance-Based Debugging and Drill-Down in Data-Oriented Workflows2012
- Provenance-based refresh in data-oriented workflows2011
- Provenance in Databases: Why, How, and WhereFoundations and Trends® in Databases, 2009
- Perm: Processing Provenance and Data on the Same Data Model through Query RewritingInternational Conference on Data Engineering, 2009
- Efficient lineage tracking for scientific workflows2008
- Efficient provenance storage2008
- Provenance semirings2007
- Lineage tracing for general data warehouse transformationsThe VLDB Journal, 2003
- Tracing the lineage of view data in a warehousing environmentACM Transactions on Database Systems, 2000