On the Role of Entropy-based Loss for Learning Causal Structures with Continuous Optimization
Preprint
Abstract
Causal discovery from observational data is an important but challenging task in many scientific fields. Recently, a method with non-combinatorial directed acyclic constraint, called NOTEARS, formulates the causal structure learning problem as a continuous optimization problem using least-square loss. Though the least-square loss function is well justified under the standard Gaussian noise assumption, it is limited if the assumption does not hold. In this work, we theoretically show that the violation of the Gaussian noise assumption will hinder the causal direction identification, making the causal orientation fully determined by the causal strength as well as the variances of noises in the linear case and by the strong non-Gaussian noises in the nonlinear case. Consequently, we propose a more general entropy-based loss that is theoretically consistent with the likelihood score under any noise distribution. We run extensive empirical evaluations on both synthetic data and real-world data to validate the effectiveness of the proposed method and show that our method achieves the best in Structure Hamming Distance, False Discovery Rate, and True Positive Rate matrices.Keywords
All Related Versions
This publication has 15 references indexed in Scilit:
- Gradient-Based Local Causal Structure LearningIEEE Transactions on Cybernetics, 2023
- Causal Discovery on Discrete Data via Weighted Normalized Wasserstein DistanceIEEE Transactions on Neural Networks and Learning Systems, 2022
- Multilabel Feature Selection: A Local Causal Structure Learning ApproachIEEE Transactions on Neural Networks and Learning Systems, 2021
- Inferring Effective Connectivity Networks From fMRI Time Series With a Temporal Entropy-ScoreIEEE Transactions on Neural Networks and Learning Systems, 2021
- Causal Discovery in Linear Non-Gaussian Acyclic Model With Multiple Latent ConfoundersIEEE Transactions on Neural Networks and Learning Systems, 2021
- Mining Markov Blankets Without Causal SufficiencyIEEE Transactions on Neural Networks and Learning Systems, 2018
- Testing for Granger causality with mixed frequency dataJournal of Econometrics, 2016
- Identification of causal relations in neuroimaging data with latent confounders: An instrumental variable approachNeuroImage, 2016
- Identifiability of Gaussian structural equation models with equal error variancesBiometrika, 2013
- Judea Pearl: Causality: Models, reasoning, and inferencePolitische Vierteljahresschrift, 2001