MABERT: Mask-Attention-Based BERT for Chinese Event Extraction
Open Access
- 20 July 2023
- journal article
- research article
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Asian and Low-Resource Language Information Processing
- Vol. 22 (7) , 1-21
Abstract
Event extraction is an essential but challenging task in information extraction. This task has considerably benefited from pre-trained language models, such as BERT. However, when it comes to the trigger-word mismatch problem in languages without natural delimiters, existing methods ignore the complement of lexical information to BERT. In addition, the inherent multi-role noise problem could limit the performance of methods when one sentence contains multiple events. In this article, we propose a Mask-Attention-based BERT (MABERT) framework for Chinese event extraction to address the above problems. Firstly, in order to avoid trigger-word mismatch and integrate lexical features into BERT layers directly, a mask-attention-based transformer augmented with two mask matrices is devised to replace the original one in BERT. By the mask-attention-based transformer, the character sequence interacts with external lexical semantics sufficiently and keeps its structure information at the same time. Moreover, against the multi-role noise problem, we make use of event type information from representation and classification, two aspects to enrich entity features, where type markers and event-schema-based mask matrix are proposed. Experimental results on the widely used ACE2005 dataset show the effectiveness of our proposed MABERT on Chinese event extraction task compared with other state-of-the-art methods.Keywords
Funding Information
- National Natural Science Foundation of China (72071145)
This publication has 28 references indexed in Scilit:
- A language-independent neural network for event detectionScience China Information Sciences, 2018
- Chinese NER Using Lattice LSTM2018
- Jointly Multiple Events Extraction via Attention-based Graph Information Aggregation2018
- Directional Skip-Gram: Explicitly Distinguishing Left and Right Context for Word Embeddings2018
- A Convolution BiLSTM Neural Network Model for Chinese Event Extraction2016
- Joint Event Extraction via Recurrent Neural Networks2016
- Joint Extraction of Events and Entities within a Document Context2016
- Named Entity Recognition for Chinese Social Media with Jointly Trained Embeddings2015
- Event Extraction via Dynamic Multi-Pooling Convolutional Neural Networks2015
- Event graphs for information retrieval and multi-document summarizationExpert Systems with Applications, 2014