Specify Robust Causal Representation from Mixed Observations
Authors:
- Mengyue YangMYMengyue Yang
- Xinyu CaiXCXinyu Cai
- Furui LiuFLFurui Liu
- Weinan ZhangWZWeinan Zhang
- Jun WangJWJun Wang
Preprint
Abstract
Learning representations purely from observations concerns the problem of learning a low-dimensional, compact representation which is beneficial to prediction models. Under the hypothesis that the intrinsic latent factors follow some casual generative models, we argue that by learning a causal representation, which is the minimal sufficient causes of the whole system, we can improve the robustness and generalization performance of machine learning models. In this paper, we develop a learning method to learn such representation from observational data by regularizing the learning procedure with mutual information measures, according to the hypothetical factored causal graph. We theoretically and empirically show that the models trained with the learned causal representations are more robust under adversarial attacks and distribution shifts compared with baselines. The supplementary materials are available at https://github.com/ymy $4323460 / \mathrm{CaRI} /$.
Keywords
All Related Versions
This publication has 9 references indexed in Scilit:
- Focused Context Balancing for Robust Offline Policy Evaluation2019
- Wild patterns: Ten years after the rise of adversarial machine learningPattern Recognition, 2018
- CAUSALITY FROM A DISTRIBUTIONAL ROBUSTNESS POINT OF VIEW2018
- Heterogeneous Information Network Embedding for RecommendationIEEE Transactions on Knowledge and Data Engineering, 2018
- Causal Inference by using Invariant Prediction: Identification and Confidence IntervalsJournal of the Royal Statistical Society Series B: Statistical Methodology, 2016
- Causal Inference Using the Algorithmic Markov ConditionIEEE Transactions on Information Theory, 2010
- Learning and generalization with the information bottleneckTheoretical Computer Science, 2010
- INFORMATION FLOWS IN CAUSAL NETWORKSAdvances in Complex Systems, 2008
- On the mathematical foundations of theoretical statisticsPhilosophical Transactions of the Royal Society A, 1922