Instance-Dependent Positive and Unlabeled Learning With Labeling Bias Estimation
- 23 February 2021
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Pattern Analysis and Machine Intelligence
- Vol. 44 (8) , 4163-4177
Abstract
This paper studies instance-dependent P ositive and U nlabeled (PU) classification, where whether a positive example will be labeled (indicated by $s$ ) is not only related to the class label $y$ , but also depends on the observation $\mathbf {x}$ . Therefore, the labeling probability on positive examples is not uniform as previous works assumed, but is biased to some simple or critical data points. To depict the above dependency relationship, a graphical model is built in this paper which further leads to a maximization problem on the induced likelihood function regarding $P(s,y|\mathbf {x})$ . By utilizing the well-known EM and Adam optimization techniques, the labeling probability of any positive example $P(s=1|y=1,\mathbf {x})$ as well as the classifier induced by $P(y|\mathbf {x})$ can be acquired. Theoretically, we prove that the critical solution always exists, and is locally unique for linear model if some sufficient conditions are met. Moreover, we upper bound the generalization error for both linear logistic and non-linear network instantiations of our algorithm, with the convergence rate of expected risk to empirical risk as $\mathcal {O}(1/\sqrt{k}+1/\sqrt{n-k}+1/\sqrt{n})$ ( $k$ and $n$ are the sizes of positive set and the entire training set, respectively). Empirically, we compare our method with state-of-the-art instance-independent and instance-dependent PU algorithms on a wide range of synthetic, benchmark and real-world datasets, and the experimental results firmly demonstrate the advantage of the proposed method over the existing PU approaches.
Keywords
Funding Information
- National Natural Science Foundation of China (61973162, U1713208, 62006202)
- Fundamental Research Funds for the Central Universities (30920032202)
- CCF-Tencent Open Fund (RAGR20200101)
- CAST (2018QNRC001)
- Hong Kong Scholars Program (XJ2019036)
- 111 Program (B13022)
- RGC Early Career Scheme (22200720)
- HKBU CSD Departmental Incentive
- Hong Kong Polytechnic University (YZ3K, UAJP/UAGK, ZVRH)
- ARC (DE190101473)
This publication has 32 references indexed in Scilit:
- Deformed Graph Laplacian for Semisupervised LearningIEEE Transactions on Neural Networks and Learning Systems, 2015
- Positive-unlabeled learning for disease gene identificationBioinformatics, 2012
- A Positive and Unlabeled Learning Algorithm for One-Class Classification of Remote-Sensing DataIEEE Transactions on Geoscience and Remote Sensing, 2010
- Presence‐Only Data and the EM AlgorithmBiometrics, 2009
- Learning classifiers from only positive and unlabeled data2008
- Relevance Feedback for Association Rules by Leveraging Concepts from Information Retrieval2007
- Learning from Positive and Unlabeled Examples with Different Data Distributions2005
- Building text classifiers using positive and unlabeled examples2004
- On Existence of Maximum Likelihood Estimators in Exponential FamiliesStatistics, 2000
- On the Existence and Uniqueness of the Maximum Likelihood Estimate of a Vector-Valued Parameter in Fixed-Size SamplesThe Annals of Statistics, 1981