Progressive Training for Learning From Label Proportions
- 23 July 2025
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks and Learning Systems
- Vol. 36 (11) , 19951-19963
Abstract
Learning from label proportions (LLPs), which aims to learn an instance-level classifier using proportion-based grouped training data, has garnered increasing attention in the field of machine learning. Existing deep learning-based LLP methods employ end-to-end pipelines to derive proportional loss functions via the Kullback–Leibler (KL) divergence between bag-level prior and posterior class distributions. However, the optimal solutions of these methods often struggle to conform to the given proportions, inevitably leading to degradation in the final classification performance. In this article, we address this issue by proposing a novel progressive training method for LLP, termed PT-LLP, which strives to meet the proportion constraints from the bag level to the instance level. Specifically, we first train a model by using the existing KL-divergence-based LLP methods that are consistent with bag-level proportion information. Then, we impose additional constraints on strict proportion consistency to the classifier to further move toward a more ideal solution by reformulating it as a constrained optimization problem, which can be efficiently solved using optimal transport (OT) algorithms. In particular, the knowledge distillation is employed as a transition stage to transfer the bag-level information to the instance level using a teacher–student framework. Finally, our framework is model-agnostic and demonstrates significant performance improvements through extensive experiments on different datasets when incorporated into other deep LLP methods as the first training stage.Keywords
Funding Information
- National Natural Science Foundation of China (62306036, 72495125)
- science and technology innovation Program of Hunan Province (2024RC4008, AC2024040911247631FF26)
- Major Program of Sichuan Provincial Social Science Foundation (SCJJ24ZD20)
- Xin Shen was supported by the General Program of Chinese Postdoctoral Science Foundation (2024M753431)
This publication has 20 references indexed in Scilit:
- MixBag: Bag-Level Data Augmentation for Learning from Label Proportions2023
- Self-supervised knowledge distillation for complementary label learningNeural Networks, 2022
- SELF-LLP: Self-supervised learning from label proportions with self-ensemblePattern Recognition, 2022
- GAN-CL: Generative Adversarial Networks for Learning From Complementary LabelsIEEE Transactions on Cybernetics, 2021
- Symmetric Cross Entropy for Robust Learning With Noisy Labels2019
- A Probabilistic Approach for Learning with Label Proportions Applied to the US Presidential Election2017
- Co-Training for Demographic Classification Using Deep Learning from Label Proportions2017
- A brief introduction to weakly supervised learningNational Science Review, 2017
- Making Deep Neural Networks Robust to Label Noise: A Loss Correction Approach2017
- Video Event Detection by Inferring Temporal Instance Labels2014