Context-Aware Contrastive Representation Learning for Zero-Shot Biomedical Text Classification
Authors:
- 3 December 2024
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE International Conference on Bioinformatics and Biomedicine (BIBM)
- Vol. 2024, 3611-3614
Abstract
Biomedical text classification refers to the task of annotating a biomedical text with its relevant labels from a candidate label set. Most of the existing approach operate in a fully supervised setting and thus heavily rely on human-annotated training data which is both labor-intensive and monetarily expensive. To address this, we propose to formulate biomedical text classification under the zero-shot learning (ZSL) paradigm that does not require any labeled training data and only relies on label surface names for training and inference. Specifically, we propose a new context-aware contrastive learning technique for ZSL that fully exploits the context information present in the biomedical text to generate semantically enriched feature representations needed for accurate zero-shot biomedical text classification. Unlike existing contrastive learning approaches that typically employ random text segmentation strategies to generate contrastive pairs, our approach utilizes the context information inherently present in biomedical text to generate semantically meaningful contrastive pairs. Extensive experiments on the largest available biomedical corpus validates the effectiveness of the proposed approach.Keywords
Funding Information
- Health
This publication has 19 references indexed in Scilit:
- BERTMeSH: deep contextual representation learning for large-scale high-performance MeSH indexing with full textBioinformatics, 2020
- MeSHProbeNet: a self-attentive probe net for MeSH indexingBioinformatics, 2019
- SemaTyP: a knowledge graph based literature mining method for drug discoveryBMC Bioinformatics, 2018
- Generating Medical Hypotheses Based on Evolutionary Medical Concepts2017
- DeepMeSH: deep semantic representation for improving large-scale MeSH indexingBioinformatics, 2016
- Predicting Unseen Labels Using Label Hierarchies in Large-Scale Multi-label Learning2015
- PubMed and beyond: a survey of web tools for searching biomedical literatureDatabase: The Journal of Biological Databases and Curation, 2011
- ConText: An algorithm for determining negation, experiencer, and temporal status from clinical reportsJournal of Biomedical Informatics, 2009
- Medical Subject Headings (MeSH).2000
- Dietary linoleic acid and salt-induced hypertensionCanadian Journal of Physiology and Pharmacology, 1981