Large-Scale Collective Entity Matching
Preprint
Abstract
There have been several recent advancements in Machine Learning community on the Entity Matching (EM) problem. However, their lack of scalability has prevented them from being applied in practical settings on large real-life datasets. Towards this end, we propose a principled framework to scale any generic EM algorithm. Our technique consists of running multiple instances of the EM algorithm on small neighborhoods of the data and passing messages across neighborhoods to construct a global solution. We prove formal properties of our framework and experimentally demonstrate the effectiveness of our approach in scaling EM algorithms.Keywords
All Related Versions
This publication has 12 references indexed in Scilit:
- Large-Scale Deduplication with Constraints Using Dedupalog2009
- Unsupervised deduplication using cross-field dependencies2008
- Collective entity resolution in relational dataACM Transactions on Knowledge Discovery From Data, 2007
- Entity Resolution with Markov LogicIEEE International Conference on Data Mining (ICDM), 2006
- A Latent Dirichlet Model for Unsupervised Entity Resolution2006
- Reference reconciliation in complex information spaces2005
- Exploiting relationships for domain-independent data cleaning2005
- What Energy Functions Can Be Minimized via Graph Cuts?2002
- Eliminating Fuzzy Duplicates in Data Warehouses2002
- Automatic Linkage of Vital RecordsScience, 1959