Towards Principled Disentanglement for Domain Generalization
Authors:
- 1 June 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 8014-8024
Abstract
A fundamental challenge for machine learning models is generalizing to out-of-distribution (OOD) data, in part due to spurious correlations. To tackle this challenge, we first formalize the OOD generalization problem as constrained optimization, called Disentanglement-constrained Domain Generalization (DDG). We relax this non-trivial constrained optimization problem to a tractable form with finite-dimensional parameterization and empirical approxi-mation. Then a theoretical analysis of the extent to which the above transformations deviates from the original problem is provided. Based on the transformation, we propose a primal-dual algorithm for joint representation disentanglement and domain generalization. In contrast to traditional approaches based on domain adversarial training and domain labels, DDG jointly learns semantic and variation encoders for disentanglement, enabling flexible manipulation and augmentation on training data. DDG aims to learn intrinsic representations of semantic concepts that are invariant to nuisance factors and generalizable across domains. Comprehensive experiments on popular benchmarks show that DDG can achieve competitive OOD performance and uncover interpretable salient structures within data.Keywords
All Related Versions
This publication has 28 references indexed in Scilit:
- Causal Inference by using Invariant Prediction: Identification and Confidence IntervalsJournal of the Royal Statistical Society Series B: Statistical Methodology, 2016
- Domain Generalization for Object Recognition with Multi-task Autoencoders2015
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Representation Learning: A Review and New PerspectivesIEEE Transactions on Pattern Analysis and Machine Intelligence, 2013
- Fairness through awareness2012
- Unbiased look at dataset bias2011
- A theory of learning from different domainsMachine Learning, 2009
- Convex Optimization2004
- Improving predictive inference under covariate shift by weighting the log-likelihood functionJournal of Statistical Planning and Inference, 2000
- Multilayer feedforward networks are universal approximatorsNeural Networks, 1989