Towards reliable named entity recognition in the biomedical domain
Open Access
- 20 June 2019
- journal article
- research article
- Published by Oxford University Press (OUP) in Bioinformatics
- Vol. 36 (1) , 280-286
Abstract
Automatic biomedical named entity recognition (BioNER) is a key task in biomedical information extraction. For some time, state-of-the-art BioNER has been dominated by machine learning methods, particularly conditional random fields (CRFs), with a recent focus on deep learning. However, recent work has suggested that the high performance of CRFs for BioNER may not generalize to corpora other than the one it was trained on. In our analysis, we find that a popular deep learning-based approach to BioNER, known as bidirectional long short-term memory network-conditional random field (BiLSTM-CRF), is correspondingly poor at generalizing. To address this, we evaluate three modifications of BiLSTM-CRF for BioNER to improve generalization: improved regularization via variational dropout, transfer learning and multi-task learning. We measure the effect that each strategy has when training/testing on the same corpus (‘in-corpus’ performance) and when training on one corpus and evaluating on another (‘out-of-corpus’ performance), our measure of the model’s ability to generalize. We found that variational dropout improves out-of-corpus performance by an average of 4.62%, transfer learning by 6.48% and multi-task learning by 8.42%. The maximal increase we identified combines multi-task learning and variational dropout, which boosts out-of-corpus performance by 10.75%. Furthermore, we make available a new open-source tool, called Saber that implements our best BioNER models. Source code for our biomedical IE tool is available at https://github.com/BaderLab/saber. Corpora and other resources used in this study are available at https://github.com/BaderLab/Towards-reliable-BioNER. Supplementary data are available at Bioinformatics online.Keywords
All Related Versions
Funding Information
- National Institutes of Health (5U41 HG006623-02)
This publication has 15 references indexed in Scilit:
- A neural network multi-task learning approach to biomedical named entity recognitionBMC Bioinformatics, 2017
- Deep learning with word embeddings improves biomedical named entity recognitionBioinformatics, 2017
- A survey of transfer learningJournal of Big Data, 2016
- A modular framework for biomedical concept recognitionBMC Bioinformatics, 2013
- Gimli: open source and high-performance biomedical name recognitionBMC Bioinformatics, 2013
- CALBC SILVER STANDARD CORPUSJournal of Bioinformatics and Computational Biology, 2010
- A Survey on Transfer LearningIEEE Transactions on Knowledge and Data Engineering, 2009
- Getting Started in Text Mining: Part TwoPLOS Computational Biology, 2009
- Getting Started in Text MiningPLOS Computational Biology, 2008
- Recognizing names in biomedical texts: a machine learning approachBioinformatics, 2004