A Multi-Task Semantic Decomposition Framework with Task-specific Pre-training for Few-Shot NER
Open Access
- 21 October 2023
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 430-440
Abstract
The objective of few-shot named entity recognition is to identify named entities with limited labeled instances. Previous works have primarily focused on optimizing the traditional token-wise classification framework, while neglecting the exploration of information based on NER data characteristics. To address this issue, we propose a Multi-Task Semantic Decomposition Framework via Joint Task-specific Pre-training (MSDP) for few-shot NER. Drawing inspiration from demonstration-based and contrastive learning, we introduce two novel pre-training tasks: Demonstration-based Masked Language Modeling (MLM) and Class Contrastive Discrimination. These tasks effectively incorporate entity boundary information and enhance entity representation in Pre-trained Language Models (PLMs). In the downstream main task, we introduce a multi-task joint optimization framework with the semantic decomposing method, which facilitates the model to integrate two different semantic information for entity classification. Experimental results of two few-shot NER benchmarks demonstrate that MSDP consistently outperforms strong baselines by a large margin. Extensive analyses validate the effectiveness and generalization of MSDP.Keywords
All Related Versions
This publication has 24 references indexed in Scilit:
- VideoBERT: A Joint Model for Video and Language Representation Learning2019
- Few-shot classification in named entity recognition task2019
- Cross-Domain NER using Cross-Domain Language Modeling2019
- Named Entity Recognition with Bidirectional LSTM-CNNsTransactions of the Association for Computational Linguistics, 2016
- The GUM corpus: creating multilayer resources in the classroomLanguage Resources and Evaluation, 2016
- Stance Detection with Bidirectional Conditional Encoding2016
- Noise reduction and targeted exploration in imitation learning for Abstract Meaning Representation parsing2016
- Named entity recognition in query2009
- A survey of named entity recognition and classification2007
- Scalable training ofL1-regularized log-linear models2007