DistillER: Knowledge Distillation in Entity Resolution with Large Language Models
Preprint
Abstract
Recent advances in Entity Resolution (ER) have leveraged Large Language Models (LLMs), achieving strong performance but at the cost of substantial computational resources or high financial overhead. Existing LLM-based ER approaches operate either in unsupervised settings and rely on very large and costly models, or in supervised settings and require ground-truth annotations, leaving a critical gap between time efficiency and effectiveness. To make LLM-powered ER more practical, we investigate Knowledge Distillation (KD) as a means to transfer knowledge from large, effective models (Teachers) to smaller, more efficient models (Students) without requiring gold labels. We introduce DistillER, the first framework that systematically bridges this gap across three dimensions: (i) Data Selection, where we study strategies for identifying informative subsets of data; (ii) Knowledge Elicitation, where we compare single- and multi-teacher settings across LLMs and smaller language models (SLMs); and (iii) Distillation Algorithms, where we evaluate supervised fine-tuning and reinforcement learning approaches. Our experiments reveal that supervised fine-tuning of Students on noisy labels generated by LLM Teachers consistently outperforms alternative KD strategies, while also enabling high-quality explanation generation. Finally, we benchmark DistillER against established supervised and unsupervised ER methods based on LLMs and SLMs, demonstrating significant improvements in both effectiveness and efficiency.Keywords
All Related Versions
This publication has 13 references indexed in Scilit:
- Fine-Tuning Large Language Models for Entity Matching2025
- An in-depth analysis of pre-trained embeddings for entity resolutionThe VLDB Journal, 2024
- Learning from Natural Language Explanations for Generalizable Entity Matching2024
- Unicorn: A Unified Multi-tasking Model for Supporting Matching Tasks in Data IntegrationProceedings of the ACM on Management of Data, 2023
- Sudowoodo: Contrastive Self-supervised Learning for Multi-purpose Data Integration and Preparation2023
- Towards Better Entity Linking with Multi-View Enhanced Distillation2023
- Entity Resolution with Hierarchical Graph Attention Networks2022
- CollaborEM: A Self-Supervised Entity Matching Framework Using Multi-Features CollaborationIEEE Transactions on Knowledge and Data Engineering, 2021
- An Overview of End-to-End Entity Resolution for Big DataACM Computing Surveys, 2020
- ZeroER: Entity Resolution using Zero Labeled Examples2020