S-ProvFlow. Storing and Exploring Lineage Data as a Service
Open Access
- 1 April 2022
- journal article
- research article
- Published by China Science Publishing & Media Ltd. in Data Intelligence
- Vol. 4 (2) , 226-242
Abstract
We present a set of configurable Web service and interactive tools, s-ProvFlow, for managing and exploiting records tracking data lineage during workflow runs. It facilitates detailed analysis of single executions. It helps users manage complex tasks by exposing the relationships between data, people, equipment and workflow runs intended to combine productively. Its logical model extends the PROV standard to precisely record parallel data-streaming applications. Its metadata handling encourages users to capture the application context by specifying how application attributes, often using standard vocabularies, should be added. These metadata records immediately help productivity as the interactive tools support their use in selection and bulk operations. Users rapidly appreciate the power of the encoded semantics as they reap the benefits. This improves the quality of provenance for users and management. Which in turn facilitates analysis of collections of runs, enabling users to manage results and validate procedures. It fosters reuse of data and methods and facilitates diagnostic investigations and optimisations. We present S-ProvFlow's use by scientists, research engineers and managers as part of the DARE hyper-platform as they create, validate and use their data-driven scientific workflows.Keywords
This publication has 21 references indexed in Scilit:
- Global adjoint tomography: first-generation modelGeophysical Journal International, 2016
- Uncertainty estimations for moment tensor inversions: the issue of the 2012 May 20 Emilia earthquakeGeophysical Journal International, 2016
- PROV-O-Viz - Understanding the Role of Activities in Provenance2015
- The PBase Scientific Workflow Provenance RepositoryInternational Journal of Digital Curation, 2014
- Evaluation of Filesystem Provenance Visualization ToolsIEEE Transactions on Visualization and Computer Graphics, 2013
- MTCProv: a practical provenance query framework for many-task scientific computingDistributed and Parallel Databases, 2012
- Variations of crustal elastic properties during the 2009 L'Aquila earthquake inferred from cross-correlations of ambient seismic noiseGeophysical Research Letters, 2011
- A Survey of Radial Methods for Information VisualizationIEEE Transactions on Visualization and Computer Graphics, 2009
- Provenance Explorer-a graphical interface for constructing scientific publication packages from provenance trailsInternational Journal on Digital Libraries, 2007
- Hierarchical Edge Bundles: Visualization of Adjacency Relations in Hierarchical DataIEEE Transactions on Visualization and Computer Graphics, 2006