Instrumental Variable-Driven Domain Generalization with Unobserved Confounders

Authors:
Open Access
Publisher website
Google Scholar
Abstract
Domain generalization (DG) aims to learn from multiple source domains a model that can generalize well on unseen target domains. Existing DG methods mainly learn the representations with invariant marginal distribution of the input features, however, the invariance of the conditional distribution of the labels given the input features is more essential for unknown domain prediction. Meanwhile, the existing of unobserved confounders which affect the input features and labels simultaneously cause spurious correlation and hinder the learning of the invariant relationship contained in the conditional distribution. Interestingly, with a causal view on the data generating process, we find that the input features of one domain are valid instrumental variables for other domains. Inspired by this finding, we propose an instrumental variable-driven DG method (IV-DG) by removing the bias of the unobserved confounders with two-stage learning. In the first stage, it learns the conditional distribution of the input features of one domain given input features of another domain. In the second stage, it estimates the relationship by predicting labels with the learned conditional distribution. Theoretical analyses and simulation experiments show that it accurately captures the invariant relationship. Extensive experiments on real-world datasets demonstrate that IV-DG method yields state-of-the-art results.
Funding Information
  • National Key Research and Development (2022YFC2504605)
  • National Natural Science Foundation of China (62006207, U20A20387, 62037001)
  • Young Elite Scientists Sponsorship Program by CAST (2021QNRC001)
  • Zhejiang Provincial Natural Science Foundation of China (LZ22F020012)
  • Major Technological Innovation Project of Hangzhou (2022AIZD0147)
  • Zhejiang Province Natural Science Foundation (LQ21F020020)
  • Project by Shanghai AI Laboratory (P22KS00111)
  • Program of Zhejiang Province Science and Technology (2022C01044)
  • StarryNight Science Fund of Zhejiang University Shanghai Institute for Advanced Study (SN-ZJU-SIAS-0010)
  • Fundamental Research Funds for the Central Universities (226-2022-00142, 226-2022-00051)