Pack Together: Entity and Relation Extraction with Levitated Marker
Authors:
Preprint
Abstract
Named Entity Recognition (NER) and Relation Extraction (RE) are the core sub-tasks for information extraction. Many recent works formulate these two tasks as the span (pair) classification problem, and thus focus on investigating how to obtain a better span representation from the pre-trained encoder. However, a major limitation of existing works is that they ignore the dependencies between spans (pairs). In this work, we propose a novel span representation approach, named Packed Levitated Markers, to consider the dependencies between the spans (pairs) by strategically packing the markers in the encoder. In particular, we propose a group packing strategy to enable our model to process massive spans together to consider their dependencies with limited resources. Furthermore, for those more complicated span pair classification tasks, we design a subject-oriented packing strategy, which packs each subject and all its objects into an instance to model the dependencies between the same-subject span pairs. Our experiments show that our model with packed levitated markers outperforms the sequence labeling model by 0.4%-1.9% F1 on three flat NER tasks, beats the token concat model on six NER benchmarks, and obtains a 3.5%-3.6% strict relation F1 improvement with higher speed over previous SOTA models on ACE04 and ACE05. Code and models are publicly available at https://github.com/thunlp/PL-Marker.Keywords
All Related Versions
This publication has 36 references indexed in Scilit:
- PTR: Prompt Tuning with Rules for Text ClassificationAI Open, 2022
- A Trigger-Sense Memory Flow Framework for Joint Entity and Relation Extraction2021
- A Primer in BERTology: What We Know About How BERT WorksTransactions of the Association for Computational Linguistics, 2020
- SpanBERT: Improving Pre-training by Representing and Predicting SpansTransactions of the Association for Computational Linguistics, 2020
- Hierarchical Contextualized Representation for Named Entity RecognitionProceedings of the AAAI Conference on Artificial Intelligence, 2020
- Instance-Based Learning of Span Representations: A Case Study through Named Entity Recognition2020
- Named Entity Recognition as Dependency Parsing2020
- A Joint Neural Model for Information Extraction with Global Features2020
- Generalizing Natural Language Analysis through Span-relation Representations2020
- CorefQA: Coreference Resolution as Query-based Span Prediction2020