Incremental Knowledge Base Construction Using DeepDive
Preprint
Abstract
Populating a database with unstructured information is a long-standing problem in industry and research that encompasses problems of extraction, cleaning, and integration. Recent names used for this problem include dealing with dark data and knowledge base construction (KBC). In this work, we describe DeepDive, a system that combines database and machine learning ideas to help develop KBC systems, and we present techniques to make the KBC process more efficient. We observe that the KBC process is iterative, and we develop techniques to incrementally produce inference results for KBC systems. We propose two methods for incremental inference, based respectively on sampling and variational techniques. We also study the tradeoff space of these methods and develop a simple rule-based optimizer. DeepDive includes all of these contributions, and we evaluate DeepDive on five KBC systems, showing that it can speed up KBC inference tasks by up to two orders of magnitude with negligible impact on quality.Keywords
All Related Versions
This publication has 29 references indexed in Scilit:
- Shallow Information Extraction for the knowledge Web2013
- Learning to Refine an Automatically Extracted Knowledge Base Using Markov Logic2012
- Optimizing Statistical Information Extraction Programs over Evolving Text2012
- Large-Scale Learning of Relation-Extraction Rules with Distant Supervision from the Web2012
- PrDB: managing and exploiting rich correlations in probabilistic databasesThe VLDB Journal, 2009
- Efficient Information Extraction over Evolving Text Data2008
- An adaptive personalized news dissemination systemJournal of Intelligent Information Systems, 2008
- Log-determinant relaxation for approximate inference in discrete Markov random fieldsIEEE Transactions on Signal Processing, 2006
- Probability theory: the logic of scienceThe Mathematical Intelligencer, 2005
- Extracting Patterns and Relations from the World Wide Web1999