Front-door Adjustment via Style Transfer for Out-of-distribution Generalisation
Preprint
Abstract
Out-of-distribution (OOD) generalisation aims to build a model that can well generalise its learnt knowledge from source domains to an unseen target domain. However, current image classification models often perform poorly in the OOD setting due to statistically spurious correlations learning from model training. From causality-based perspective, we formulate the data generation process in OOD image classification using a causal graph. On this graph, we show that prediction P(Y|X) of a label Y given an image X in statistical learning is formed by both causal effect P(Y|do(X)) and spurious effects caused by confounding features (e.g., background). Since the spurious features are domain-variant, the prediction P(Y|X) becomes unstable on unseen domains. In this paper, we propose to mitigate the spurious effect of confounders using front-door adjustment. In our method, the mediator variable is hypothesized as semantic features that are essential to determine a label for an image. Inspired by capability of style transfer in image generation, we interpret the combination of the mediator variable with different generated images in the front-door formula and propose novel algorithms to estimate it. Extensive experimental results on widely used benchmark datasets verify the effectiveness of our method.Keywords
All Related Versions
This publication has 30 references indexed in Scilit:
- Self-challenging Improves Cross-Domain Generalization2020
- Learning to Generate Novel Domains for Domain Generalization2020
- Visual Commonsense R-CNN2020
- Sequential Learning for Domain Generalization2020
- Sliced Wasserstein Discrepancy for Unsupervised Domain Adaptation2019
- Deep Domain Generalization via Conditional Invariant Adversarial Networks2018
- Domain Generalization with Adversarial Feature Learning2018
- Deep Residual Learning for Image Recognition2016
- Unbiased look at dataset bias2011
- Gradient-based learning applied to document recognitionProceedings of the IEEE, 1998