Loss Decomposition and Centroid Estimation for Positive and Unlabeled Learning
- 16 September 2019
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 43 (3) , 918-932
Abstract
This paper studies Positive and Unlabeled learning (PU learning), of which the target is to build a binary classifier where only positive data and unlabeled data are available for classifier training. To deal with the absence of negative training data, we first regard all unlabeled data as negative examples with false negative labels, and then convert PU learning into the risk minimization problem in the presence of such one-side label noise. Specifically, we propose a novel PU learning algorithm dubbed "Loss Decomposition and Centroid Estimation" (LDCE). By decomposing the loss function of corrupted negative examples into two parts, we show that only the second part is affected by the noisy labels. Thereby, we may estimate the centroid of corrupted negative set via an unbiased way to reduce the adverse impact of such label noise. Furthermore, we propose the "Kernelized LDCE" (KLDCE) by introducing the kernel trick, and show that KLDCE can be easily solved by combining Alternative Convex Search (ACS) and Sequential Minimal Optimization (SMO). Theoretically, we derive the generalization error bound which suggests that the generalization risk of our model converges to the empirical risk with the order of $\mathcal{O}(1/ \sqrt k + 1 /{\sqrt {n-k}} + 1/ \sqrt n)$ ( $n$ and $k$ are the amounts of training data and positive data correspondingly). Experimentally, we conduct intensive experiments on synthetic dataset, UCI benchmark datasets and real-world datasets, and the results demonstrate that our approaches (LDCE and KLDCE) achieve the top-level performance when compared with both classic and state-of-the-art PU learning methods.
Keywords
Funding Information
- National Natural Science Foundation of China (61602246, 61973162, U1713208)
- NSF of Jiangsu Province (BK20171430)
- Fundamental Research Funds for the Central Universities (30918011319)
- Xidian University (ISN19-03)
- CAST (2018QNRC001)
This publication has 17 references indexed in Scilit:
- Learning from Weak and Noisy Labels for Semantic SegmentationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2016
- Co-Labeling for Multi-View Weakly Labeled LearningIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- Classification with Noisy Labels by Importance ReweightingIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- Deformed Graph Laplacian for Semisupervised LearningIEEE Transactions on Neural Networks and Learning Systems, 2015
- Positive-unlabeled learning for disease gene identificationBioinformatics, 2012
- A Positive and Unlabeled Learning Algorithm for One-Class Classification of Remote-Sensing DataIEEE Transactions on Geoscience and Remote Sensing, 2010
- Presence‐Only Data and the EM AlgorithmBiometrics, 2009
- Learning classifiers from only positive and unlabeled data2008
- Building text classifiers using positive and unlabeled examples2004
- PAC Learning from Positive Statistical Queries1998