A Multichannel Biomedical Named Entity Recognition Model Based on Multitask Learning and Contextualized Word Representations
Open Access
- 10 August 2020
- journal article
- research article
- Published by Wiley in Wireless Communications and Mobile Computing
- Vol. 2020, 1-13
Abstract
As the biomedical literature increases exponentially, biomedical named entity recognition (BNER) has become an important task in biomedical information extraction. In the previous studies based on deep learning, pretrained word embedding becomes an indispensable part of the neural network models, effectively improving their performance. However, the biomedical literature typically contains numerous polysemous and ambiguous words. Using fixed pretrained word representations is not appropriate. Therefore, this paper adopts the pretrained embeddings from language models (ELMo) to generate dynamic word embeddings according to context. In addition, in order to avoid the problem of insufficient training data in specific fields and introduce richer input representations, we propose a multitask learning multichannel bidirectional gated recurrent unit (BiGRU) model. Multiple feature representations (e.g., word-level, contextualized word-level, character-level) are, respectively, or collectively fed into the different channels. Manual participation and feature engineering can be avoided through automatic capturing features in BiGRU. In merge layer, multiple methods are designed to integrate the outputs of multichannel BiGRU. We combine BiGRU with the conditional random field (CRF) to address labels’ dependence in sequence labeling. Moreover, we introduce the auxiliary corpora with same entity types for the main corpora to be evaluated in multitask learning framework, then train our model on these separate corpora and share parameters with each other. Our model obtains promising results on the JNLPBA and NCBI-disease corpora, with F1-scores of 76.0% and 88.7%, respectively. The latter achieves the best performance among reported existing feature-based models.Keywords
Funding Information
- National Natural Science Foundation of China (61976124)
This publication has 33 references indexed in Scilit:
- LSTM: A Search Space OdysseyIEEE Transactions on Neural Networks and Learning Systems, 2016
- TaggerOne: joint named entity recognition and normalization with semi-Markov ModelsBioinformatics, 2016
- Hadoop Recognition of Biomedical Named Entity Using Conditional Random FieldsIEEE Transactions on Parallel and Distributed Systems, 2014
- NCBI disease corpus: A resource for disease name recognition and concept normalizationJournal of Biomedical Informatics, 2014
- Evaluating Word Representation Features in Biomedical Named Entity Recognition TasksBioMed Research International, 2014
- DNorm: disease name normalization with pairwise learning to rankBioinformatics, 2013
- Overview of BioCreative II gene mention recognitionGenome Biology, 2008
- NERBio: using selected word conjunctions, term normalization, and global patterns to improve biomedical named entity recognitionBMC Bioinformatics, 2006
- ABNER: an open source tool for automatically tagging genes, proteins and other entity names in textBioinformatics, 2005
- Long Short-Term MemoryNeural Computation, 1997