TENER: Adapting Transformer Encoder for Named Entity Recognition
Preprint
Abstract
The Bidirectional long short-term memory networks (BiLSTM) have been widely used as an encoder in models solving the named entity recognition (NER) task. Recently, the Transformer is broadly adopted in various Natural Language Processing (NLP) tasks owing to its parallelism and advantageous performance. Nevertheless, the performance of the Transformer in NER is not as good as it is in other NLP tasks. In this paper, we propose TENER, a NER architecture adopting adapted Transformer Encoder to model the character-level features and word-level features. By incorporating the direction and relative distance aware attention and the un-scaled attention, we prove the Transformer-like encoder is just as effective for NER as other NLP tasks.Keywords
All Related Versions
This publication has 22 references indexed in Scilit:
- GRN: Gated Relation Network to Enhance Convolutional Neural Network for Named Entity RecognitionProceedings of the AAAI Conference on Artificial Intelligence, 2019
- Contextualized Non-Local Neural Networks for Sequence LearningProceedings of the AAAI Conference on Artificial Intelligence, 2019
- Five-Stroke Based CNN-BiRNN-CRF Network for Chinese Named Entity Recognition2018
- Empower Sequence Labeling with Task-Aware Neural Language ModelProceedings of the AAAI Conference on Artificial Intelligence, 2018
- Neural Question Generation from Text: A Preliminary Study2018
- Learning Better Internal Structure of Words for Sequence Labeling2018
- Applying named entity recognition and co-reference resolution for segmenting English textsProgress in Artificial Intelligence, 2017
- Fast and Accurate Entity Recognition with Iterated Dilated Convolutions2017
- Character-Based LSTM-CRF with Radical-Level Features for Chinese Named Entity Recognition2016
- Named Entity Recognition for Chinese Social Media with Jointly Trained Embeddings2015