Packed Levitated Marker for Entity and Relation Extraction
Preprint
Abstract
Recent entity and relation extraction works focus on investigating how to obtain a better span representation from the pre-trained encoder. However, a major limitation of existing works is that they ignore the interrelation between spans (pairs). In this work, we propose a novel span representation approach, named Packed Levitated Markers (PL-Marker), to consider the interrelation between the spans (pairs) by strategically packing the markers in the encoder. In particular, we propose a neighborhood-oriented packing strategy, which considers the neighbor spans integrally to better model the entity boundary information. Furthermore, for those more complicated span pair classification tasks, we design a subject-oriented packing strategy, which packs each subject and all its objects to model the interrelation between the same-subject span pairs. The experimental results show that, with the enhanced marker feature, our model advances baselines on six NER benchmarks, and obtains a 4.1%-4.3% strict relation F1 improvement with higher speed over previous state-of-the-art models on ACE04 and ACE05.Keywords
All Related Versions
This publication has 39 references indexed in Scilit:
- PTR: Prompt Tuning with Rules for Text ClassificationAI Open, 2022
- A Trigger-Sense Memory Flow Framework for Joint Entity and Relation Extraction2021
- KEPLER: A Unified Model for Knowledge Embedding and Pre-trained Language RepresentationTransactions of the Association for Computational Linguistics, 2021
- SpanBERT: Improving Pre-training by Representing and Predicting SpansTransactions of the Association for Computational Linguistics, 2020
- Instance-Based Learning of Span Representations: A Case Study through Named Entity Recognition2020
- BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension2020
- Named Entity Recognition as Dependency Parsing2020
- Generalizing Natural Language Analysis through Span-relation Representations2020
- A Unified MRC Framework for Named Entity Recognition2020
- CorefQA: Coreference Resolution as Query-based Span Prediction2020