A Generalized Fellegi-Sunter Framework for Multiple Record Linkage With Application to Homicide Record Systems
Preprint
Abstract
We present a probabilistic method for linking multiple datafiles. This task is not trivial in the absence of unique identifiers for the individuals recorded. This is a common scenario when linking census data to coverage measurement surveys for census coverage evaluation, and in general when multiple record-systems need to be integrated for posterior analysis. Our method generalizes the Fellegi-Sunter theory for linking records from two datafiles and its modern implementations. The multiple record linkage goal is to classify the record K-tuples coming from K datafiles according to the different matching patterns. Our method incorporates the transitivity of agreement in the computation of the data used to model matching probabilities. We use a mixture model to fit matching probabilities via maximum likelihood using the EM algorithm. We present a method to decide the record K-tuples membership to the subsets of matching patterns and we prove its optimality. We apply our method to the integration of three Colombian homicide record systems and we perform a simulation study in order to explore the performance of the method under measurement error and different scenarios. The proposed method works well and opens some directions for future research.Keywords
All Related Versions
This publication has 11 references indexed in Scilit:
- A Survey of Indexing Techniques for Scalable Record Linkage and DeduplicationIEEE Transactions on Knowledge and Data Engineering, 2011
- A hierarchical Bayesian approach to record linkage and population size problemsThe Annals of Applied Statistics, 2011
- Adaptive Weighted Learning for Unbalanced Multicategory ClassificationBiometrics, 2009
- Integrated methodology for multiple systems estimation and record linkage using a missing data formulationAStA Advances in Statistical Analysis, 2008
- Probabilistic record linkage is a valid and transparent tool to combine databases without a patient identification numberJournal of Clinical Epidemiology, 2007
- Adaptive name matching in information integrationIEEE Intelligent Systems, 2003
- Iterative Automated Record Linkage Using Mixture ModelsJournal of the American Statistical Association, 2001
- Record Linkage: Statistical Models for Matching Computer RecordsJournal of the Royal Statistical Society Series A: Statistics in Society, 1990
- Maximum Likelihood from Incomplete Data Via the EM AlgorithmJournal of the Royal Statistical Society Series B: Statistical Methodology, 1977
- Automatic Linkage of Vital RecordsScience, 1959