A Keyword-Enhanced Approach to Handle Class Imbalance in Clinical Text Classification
Open Access
- 3 June 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Biomedical and Health Informatics
- Vol. 26 (6) , 2796-2803
Abstract
Recent applications ofdeep learning have shown promising results for classifying unstructured text in the healthcare domain. However, the reliability of models in production settings has been hindered by imbalanced data sets in which a small subset of the classes dominate. In the absence of adequate training data, rare classes necessitate additional model constraints for robust performance. Here, we present a strategy for incorporating short sequences of text (i.e. keywords) into training to boost model accuracy on rare classes. In our approach, we assemble a set of keywords, including short phrases, associated with each class. The keywords are then used as additional data during each batch of model training, resulting in a training loss that has contributions from both raw data and keywords. We evaluate our approach on classification of cancer pathology reports, which shows a substantial increase in model performance for rare classes. Furthermore, we analyze the impact of keywords on model output probabilities for bigrams, providing a straightforward method to identify model difficulties for limited training data.Keywords
Funding Information
- Joint Design of Advanced Computing Solutions for Cancer
- DOE
- NCI
- National Institutes of Health
- DOE
- Argonne National Laboratory (DE-AC02-06-CH11357)
- Lawrence Livermore National Laboratory (DE-AC52-07NA27344)
- Los Alamos National Laboratory (DE-AC5206NA25396)
- Oak Ridge National Laboratory (DE-AC05-00OR22725)
- NCI SEER (HHSN261201800013I)
- CDC National Program of Cancer Registries (U58DP00003907)
- Commonwealth of Kentucky
- NCI
- SEER Program (HHSN261201800007I)
- NPCR (NU58DP006332-02-00)
- State of Louisiana
- NCI
- SEER Program (HHSN261201300021I)
- NPCR (NU58DP006279-02-00)
- State of New Jersey Department of Education
- Rutgers Cancer Institute of New Jersey
- NCI's SEER Program (HHSN261201800016I)
- NPCR (NU58DP0063200)
- University of Utah
- Huntsman Cancer Foundation
- National Cancer Institute's Surveillance (HHSN261201800014I, HHSN26100001)
- Epidemiology and End Results (SEER) Program
- National Cancer Institute's SEER Program (HHSN261291800004I)
- Fred Hutchinson Cancer Center
This publication has 24 references indexed in Scilit:
- Deep Learning for Automated Extraction of Primary Sites From Cancer Pathology ReportsIEEE Journal of Biomedical and Health Informatics, 2017
- Using machine learning to parse breast pathology reportsBreast Cancer Research and Treatment, 2016
- Convolutional Neural Networks for Sentence Classification2014
- A Review on Ensembles for the Class Imbalance Problem: Bagging-, Boosting-, and Hybrid-Based ApproachesIEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews), 2012
- Analysis of preprocessing vs. cost-sensitive learning for imbalanced classification. Open problems on intrinsic data characteristicsExpert Systems with Applications, 2012
- HANDLING THE IMPACT OF LOW FREQUENCY EVENTS ON CO-OCCURRENCE BASED MEASURES OF WORD SIMILARITY - A Case Study of Pointwise Mutual Information2011
- Exploiting probabilistic topic models to improve text categorization under class imbalanceInformation Processing & Management, 2010
- A unified architecture for natural language processing2008
- NCI Thesaurus: A semantic model integrating cancer-related clinical and molecular informationJournal of Biomedical Informatics, 2007
- The Unified Medical Language System (UMLS): integrating biomedical terminologyNucleic Acids Research, 2004