Keypoint-Guided Optimal Transport
Preprint
Abstract
Existing Optimal Transport (OT) methods mainly derive the optimal transport plan/matching under the criterion of transport cost/distance minimization, which may cause incorrect matching in some cases. In many applications, annotating a few matched keypoints across domains is reasonable or even effortless in annotation burden. It is valuable to investigate how to leverage the annotated keypoints to guide the correct matching in OT. In this paper, we propose a novel KeyPoint-Guided model by ReLation preservation (KPG-RL) that searches for the optimal matching (i.e., transport plan) guided by the keypoints in OT. To impose the keypoints in OT, first, we propose a mask-based constraint of the transport plan that preserves the matching of keypoint pairs. Second, we propose to preserve the relation of each data point to the keypoints to guide the matching. The proposed KPG-RL model can be solved by Sinkhorn's algorithm and is applicable even when distributions are supported in different spaces. We further utilize the relation preservation constraint in the Kantorovich Problem and Gromov-Wasserstein model to impose the guidance of keypoints in them. Meanwhile, the proposed KPG-RL model is extended to the partial OT setting. Moreover, we deduce the dual formulation of the KPG-RL model, which is solved using deep learning techniques. Based on the learned transport plan from dual KPG-RL, we propose a novel manifold barycentric projection to transport source data to the target domain. As applications, we apply the proposed KPG-RL model to the heterogeneous domain adaptation and image-to-image translation. Experiments verified the effectiveness of the proposed approach.Keywords
All Related Versions
This publication has 34 references indexed in Scilit:
- Fixing Bias in Reconstruction-based Anomaly Detection with Lipschitz DiscriminatorsJournal of Signal Processing Systems, 2021
- On the Effectiveness of Image Rotation for Open Set Domain Adaptation2020
- Learning from positive and unlabeled data: a surveyMachine Learning, 2020
- Discriminative Region Proposal Adversarial Network for High-Quality Image-to-Image TranslationInternational Journal of Computer Vision, 2019
- Optimal Entropy-Transport problems and a new Hellinger–Kantorovich distance between positive measuresInventiones Mathematicae, 2017
- An Interpolating Distance Between Optimal Transport and Fisher–Rao MetricsFoundations of Computational Mathematics, 2016
- Gromov–Wasserstein Distances and the Metric Approach to Object MatchingFoundations of Computational Mathematics, 2011
- The Optimal Partial Transport ProblemArchive for Rational Mechanics and Analysis, 2009
- On the Translocation of MassesJournal of Mathematical Sciences, 2006
- On the geometry of metric measure spacesActa Mathematica, 2006