Guided data repair
Authors:
Top Cited Papers
Open Access
- 1 February 2011
- journal article
- Published by Association for Computing Machinery (ACM) in Proceedings of the VLDB Endowment
- Vol. 4 (5) , 279-289
Abstract
In this paper we present GDR, a Guided Data Repair framework that incorporates user feedback in the cleaning process to enhance and accelerate existing automatic repair techniques while minimizing user involvement. GDR consults the user on the updates that are most likely to be beneficial in improving data quality. GDR also uses machine learning methods to identify and apply the correct updates directly to the database without the actual involvement of the user on these specific updates. To rank potential updates for consultation by the user, we first group these repairs and quantify the utility of each group using the decision-theory concept of value of information (VOI). We then apply active learning to order updates within a group based on their ability to improve the learned model. User feedback is used to repair the database and to adaptively refine the training set for the model. We empirically evaluate GDR on a real-world dataset and show significant improvement in data quality using our user guided repairing process. We also, assess the trade-off between the user efforts and the resulting data quality.Keywords
This publication has 17 references indexed in Scilit:
- Towards certain fixes with editing rules and master dataProceedings of the VLDB Endowment, 2010
- GDR2010
- Reasoning about record matching rulesProceedings of the VLDB Endowment, 2009
- On generating near-optimal tableaux for conditional functional dependenciesProceedings of the VLDB Endowment, 2008
- Dependencies revisited for improving data quality2008
- Minimal-change integrity maintenance using tuple deletionsInformation and Computation, 2005
- Interactive deduplication using active learning2002
- Random ForestsMachine Learning, 2001
- Learning and making decisions when costs and probabilities are both unknown2001
- 10.1162/153244302760185243Applied Physics Letters, 2000