Evaluating multilingual encoder models for few-shot named entity recognition tasks
Open Access
- 1 November 2025
- journal article
- Published by Institute of Advanced Engineering and Science in Indonesian Journal of Electrical Engineering and Computer Science
- Vol. 40 (2) , 745-757
Abstract
This work provides a thorough analysis of few-shot learning approaches in the realm of multilingual named entity recognition (NER). Our research is driven by the need to enhance linguistic inclusivity and performance efficiency across diverse languages. We focus on benchmarking a selection of prominent encoder models including XLM-RoBERTa (XLM-R), multilingual BERT (mBERT), DistilBERT, character architecture for eNcoders IN embeddings (CANINE), and multilingual text-to-text transfer transformer (mT5), to illuminate their capabilities and limitations within few-shot learning paradigms, particularly for underrepresented languages. Results indicate that models like XLM-R and mT5 demonstrate superior adaptability and accuracy, outperforming others in complex linguistic settings, which suggests their potential in supporting more inclusive artificial intelligence (AI) technologies. The impact of this study extends beyond academic interest, offering pivotal insights for the development of more inclusive, adaptable and efficient NER systems. By advancing our understanding of few-shot learning in multilingual contexts, this work contributes to the broader goal of creating AI applications that are linguistically diverse and more reflective of global communication patterns. These results provide crucial insights for advancing entity recognition capabilities across diverse artificial intelligence systems, facilitating development of more precise, equitable, and sophisticated linguistic processing frameworks.Keywords
This publication has 27 references indexed in Scilit:
- SSHR: Leveraging Self-supervised Hierarchical Representations for Multilingual Automatic Speech Recognition2024
- Domain-adapted Large Language Models for Classifying Nuclear Medicine ReportsRadiology: Artificial Intelligence, 2023
- Performance of Multiple Pretrained BERT Models to Automate and Accelerate Data Annotation for Large DatasetsRadiology: Artificial Intelligence, 2022
- Canine: Pre-training an Efficient Tokenization-Free Encoder for Language RepresentationTransactions of the Association for Computational Linguistics, 2022
- Universal Representation Learning from Multiple Domains for Few-shot Classification2021
- Generalizing from a Few ExamplesACM Computing Surveys, 2020
- Focal Loss for Dense Object DetectionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2018
- Cross-Lingual Entity Linking for Web TablesProceedings of the AAAI Conference on Artificial Intelligence, 2018
- BioCreative V CDR task corpus: a resource for chemical disease relation extractionDatabase: The Journal of Biological Databases and Curation, 2016
- A survey of named entity recognition and classification2007