Semi-supervised Learning based on Distributionally Robust Optimization
Preprint
Abstract
We propose a novel method for semi-supervised learning (SSL) based on data-driven distributionally robust optimization (DRO) using optimal transport metrics. Our proposed method enhances generalization error by using the unlabeled data to restrict the support of the worst case distribution in our DRO formulation. We enable the implementation of our DRO formulation by proposing a stochastic gradient descent algorithm which allows to easily implement the training procedure. We demonstrate that our Semi-supervised DRO method is able to improve the generalization error over natural supervised procedures and state-of-the-art SSL estimators. Finally, we include a discussion on the large sample behavior of the optimal uncertainty region in the DRO formulation. Our discussion exposes important aspects such as the role of dimension reduction in SSL.Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- Sample Out-of-Sample Inference Based on Wasserstein DistanceOperations Research, 2021
- Robust Wasserstein profile inference and applications to machine learningJournal of Applied Probability, 2019
- Unbiased Estimation with Square Root Convergence for SDE ModelsOperations Research, 2015
- Contrastive Pessimistic Likelihood Estimation for Semi-Supervised ClassificationIEEE Transactions on Pattern Analysis and Machine Intelligence, 2015
- A general method for debiasing a Monte Carlo estimatorMonte Carlo Methods and Applications, 2011
- Distributed Stochastic Subgradient Projection Algorithms for Convex OptimizationJournal of Optimization Theory and Applications, 2010
- A self-training semi-supervised SVM algorithm and its application in an EEG-based brain computer interface speller systemPattern Recognition Letters, 2008
- Multilevel Monte Carlo Path SimulationOperations Research, 2008
- The Earth Mover's Distance as a Metric for Image RetrievalInternational Journal of Computer Vision, 2000