TOE: A Grid-Tagging Discontinuous NER Model Enhanced by Embedding Tag/Word Relations and More Fine-Grained Tags
- 10 November 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Vol. 31 (23299290) , 177-187
Abstract
So far, discontinuous named entity recognition (NER) has received increasing research attention and many related methods have surged such as hypergraph-based methods, span-based methods, and sequence-to-sequence (Seq2Seq) methods, etc. However, these methods more or less suffer from some problems such as decoding ambiguity and efficiency, which limit their performance. Recently, grid-tagging methods, which benefit from the flexible design of tagging systems and model architectures, have shown superiority to adapt for various information extraction tasks. In this paper, we follow the line of such methods and propose a competitive grid-tagging model for discontinuous NER. We call our model TOE because we incorporate two kinds of Tag-Oriented Enhancement mechanisms into a state-of-the-art (SOTA) grid-tagging model that casts the NER problem into word-word relationship prediction. First, we design a Tag Representation Embedding Module (TREM) to force our model to consider not only word-word relationships but also word-tag and tag-tag relationships. Concretely, we construct tag representations and embed them into TREM, so that TREM can treat tag and word representations as queries/keys/values and utilize self-attention to model their relationships. On the other hand, motivated by the Next-Neighboring-Word (NNW) and Tail-Head-Word (THW) tags in the SOTA model, we add two new symmetric tags, namely Previous-Neighboring-Word (PNW) and Head-Tail-Word (HTW), to model more fine-grained word-word relationships and alleviate error propagation from tag prediction. In the experiments of three benchmark datasets, namely CADEC, ShARe13 and ShARe14, our TOE model pushes the SOTA results by about 0.83%, 0.05% and 0.66% in F1, demonstrating its effectiveness.Keywords
All Related Versions
Funding Information
- Center for Artificial Intelligence (C4AI-USP)
- São Paulo Research Foundation (2019/07665-4)
- International Business Machines Corporation
- China Branch of BRICS Institute of Future Networks
- National Natural Science Foundation of China (62176187)
- National Key Research and Development Program of China (2017YFC1200500)
- Research Foundation of Ministry of Education of China (18JZD015)
- Youth Fund for Humanities and Social Science Research of Ministry of Education of China (22YJCZH064)
- General Project of Natural Science Foundation of Hubei Province (2021CFB385)
- Fundamental Research Funds for the Central Universities
This publication has 43 references indexed in Scilit:
- Neural Architectures for Named Entity Recognition2016
- Multilingual Language Processing From Bytes2016
- Cadec: A corpus of adverse drug event annotationsJournal of Biomedical Informatics, 2015
- Joint Mention Extraction and Classification with Mention Hypergraphs2015
- A Neural Probabilistic Structured-Prediction Model for Transition-Based Dependency Parsing2015
- Evaluating the state of the art in disorder recognition and normalization of the clinical narrativeJournal of the American Medical Informatics Association, 2014
- A Fast and Accurate Dependency Parser using Neural Networks2014
- The Impact of Named Entity Normalization on Information Retrieval for Question Answering2008
- Incorporating non-local information into information extraction systems by Gibbs sampling2005
- GENIA corpus—a semantically annotated corpus for bio-textminingBioinformatics, 2003