Online Meta-Learning for Multi-Source and Semi-Supervised Domain Adaptation
Preprint
Abstract
Domain adaptation (DA) is the topical problem of adapting models from labelled source datasets so that they perform well on target datasets where only unlabelled or partially labelled data is available. Many methods have been proposed to address this problem through different ways to minimise the domain shift between source and target datasets. In this paper we take an orthogonal perspective and propose a framework to further enhance performance by meta-learning the initial conditions of existing DA algorithms. This is challenging compared to the more widely considered setting of few-shot meta-learning, due to the length of the computation graph involved. Therefore we propose an online shortest-path meta-learning framework that is both computationally tractable and practically effective for improving DA performance. We present variants for both multi-source unsupervised domain adaptation (MSDA), and semi-supervised domain adaptation (SSDA). Importantly, our approach is agnostic to the base adaptation algorithm, and can be applied to improve many techniques. Experimentally, we demonstrate improvements on classic (DANN) and recent (MCD and MME) techniques for MSDA and SSDA, and ultimately achieve state of the art results on several DA benchmarks including the largest scale DomainNet.Keywords
All Related Versions
This publication has 18 references indexed in Scilit:
- Taking a Closer Look at Domain Shift: Category-Level Adversaries for Semantics Consistent Domain Adaptation2019
- Unsupervised Visual Domain Adaptation: A Deep Max-Margin Gaussian Process Approach2019
- Deep Cocktail Network: Multi-source Unsupervised Domain Adaptation with Category Shift2018
- Boosting Domain Adaptation by Discovering Latent Domains2018
- Maximum Classifier Discrepancy for Unsupervised Domain Adaptation2018
- Adversarial Discriminative Domain Adaptation2017
- Semi-supervised Domain Adaptation with Subspace Learning for visual recognition2015
- Semi-supervised Domain Adaptation with Instance Constraints2013
- A theory of learning from different domainsMachine Learning, 2009
- Shifting Inductive Bias with Success-Story Algorithm, Adaptive Levin Search, and Incremental Self-ImprovementMachine Learning, 1997