Contrastive Learning With Transformer to Predict the Chronicity of Children With Immune Thrombocytopenia
- 14 March 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Journal of Biomedical and Health Informatics
- Vol. 29 (7) , 5229-5242
Abstract
Immune thrombocytopenia (ITP) is a typically self-limiting and immune-mediated bleeding disorder in children. Approximately 20% of children with ITP experience chronicity, leading to reduced quality of life and increased treatment burden. The accurate prediction of chronicity would enable clinicians to make personalized treatment plans at an early stage. However, due to the self-limiting nature of ITP and the scarcity of available children patients, the data presents two prominent issues: small data and imbalanced class, which are unfavorable for effectively training a deep learning model. To handle these issues concurrently, we proposed a novel method that integrates contrastive learning with the Transformer. First, we adopt the FT-Transformer as our backbone, which allows our model to flexibly process heterogeneous tabular data. Second, we amplify and balance the original data via random masking and oversampling, respectively. Lastly, we build contrastive pairs according to the latent representations generated by the FT-Transformer encoder, such that the amplified and oversampled synthetic data can be utilized thoroughly. The experimental results on real-world ITP children data show that our proposal outperforms the state-of-the-art methods, and demonstrate the significant advantages of dealing with insufficient and imbalanced problems.Keywords
Funding Information
- National Key Research and Development Program of China (2022YFB2703300, 2023YFC2706100)
- National Natural Science Foundation of China (62106266, 62203437, 81970111)
- Beijing Municipal Administration of Hospitals Incubating Program (PX2023044)
- Beijing Natural Science Foundation of China (7232051)
This publication has 40 references indexed in Scilit:
- Deep Pneumonia: Attention-Based Contrastive Learning for Class-Imbalanced Pneumonia Lesion Recognition in Chest X-rays2022
- Increased proportion of Th17/Treg cells at the new diagnosed stage of chronic immune thrombocytopenia in pediatrics: the pilot study from a multi-centerEuropean Journal of Pediatrics, 2021
- WCE polyp detection with triplet based embeddingsComputerized Medical Imaging and Graphics, 2020
- Retrospective evaluation of children with immune thrombocytopenic purpura and factors contributing to chronicityPediatrics & Neonatology, 2019
- Unsupervised Feature Learning via Non-parametric Instance Discrimination2018
- The Elements of Statistical Learning2009
- Wiley Series in Probability and Statistics2008
- Borderline-SMOTE: A New Over-Sampling Method in Imbalanced Data Sets Learning2005
- Learning Bayesian network parameters from small data sets: application of Noisy-OR gatesInternational Journal of Approximate Reasoning, 2001
- Support-vector networksMachine Learning, 1995