A multi-head adjacent attention-based pyramid layered model for nested named entity recognition
Open Access
- 1 September 2022
- journal article
- research article
- Published by Springer Nature in Neural Computing and Applications
- Vol. 35 (3) , 2561-2574
Abstract
Named entity recognition (NER) is one of the widely studied natural language processing tasks in recent years. Conventional solutions treat the NER as a sequence labeling problem, but these approaches cannot handle nested NER. This is due to the fact that nested NER refers to the case where one entity contains another entity and it is not feasible to tag each token with a single tag. The pyramid model stacksLflat NER layers for prediction, which subtly enumerates all spans with length less than or equal toL. However, the original model introduces a block consisting of a convolutional layer and a bidirectional long short-term memory (Bi-LSTM) layer as the decoder, which does not consider the dependency between adjacent inputs and the Bi-LSTM cannot perform parallel computation on sequential inputs. For the purpose of improving performance and reducing the forward computation, we propose a Multi-Head Adjacent Attention-based Pyramid Layered model. In addition, when constructing a pyramid structure for span representation, the information of the intermediate words has more proportion than words on the two sides. To address this imbalance in the span representation, we fuse the output of the attention layer with the features of head and tail words when doing classification. We conducted experiments on nested NER datasets such as GENIA, SciERC, and ADE to validate the effectiveness of our proposed model.Keywords
Funding Information
- Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea Government (2020-0-00107)
This publication has 54 references indexed in Scilit:
- Joint Mention Extraction and Classification with Mention Hypergraphs2015
- Glove: Global Vectors for Word Representation2014
- Development of a benchmark corpus to support the automatic extraction of drug-related adverse effects from medical case reportsJournal of Biomedical Informatics, 2012
- A survey on question answering technology from an information retrieval perspectiveInformation Sciences, 2011
- Framewise phoneme classification with bidirectional LSTM and other neural network architecturesNeural Networks, 2005
- A novel method for prokaryotic promoter prediction based on DNA stabilityBMC Bioinformatics, 2005
- Enhancing HMM-based biomedical named entity recognition by studying special phenomenaJournal of Biomedical Informatics, 2004
- Recognizing names in biomedical texts: a machine learning approachBioinformatics, 2004
- Named entity recognition using an HMM-based chunk tagger2001
- Representing text chunks1999