Deep Active Learning for Named Entity Recognition
Preprint
Abstract
Deep learning has yielded state-of-the-art performance on many natural language processing tasks including named entity recognition (NER). However, this typically requires large amounts of labeled data. In this work, we demonstrate that the amount of labeled training data can be drastically reduced when deep learning is combined with active learning. While active learning is sample-efficient, it can be computationally expensive since it requires iterative retraining. To speed this up, we introduce a lightweight architecture for NER, viz., the CNN-CNN-LSTM model consisting of convolutional character and word encoders and a long short term memory (LSTM) tag decoder. The model achieves nearly state-of-the-art performance on standard datasets for the task while being computationally much more efficient than best performing models. We carry out incremental active learning, during the training process, and are able to nearly match state-of-the-art performance with just 25\% of the original training data.Keywords
All Related Versions
This publication has 8 references indexed in Scilit:
- Active Discriminative Text Representation LearningProceedings of the AAAI Conference on Artificial Intelligence, 2017
- Neural Models for Sequence ChunkingProceedings of the AAAI Conference on Artificial Intelligence, 2017
- Named Entity Recognition with Bidirectional LSTM-CNNsTransactions of the Association for Computational Linguistics, 2016
- Globally Normalized Transition-Based Neural Networks2016
- Investigation of recurrent-neural-network architectures and learning methods for spoken language understanding2013
- Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research GroupsIEEE Signal Processing Magazine, 2012
- Agnostic active learningJournal of Computer and System Sciences, 2009
- Reducing Labeling Effort for Structured Prediction Tasks2005