Joseph Ramsey

Learning graphical conditional independence structures is an important machine learning problem and a cornerstone of causal discovery. However, the accuracy and execution time of learning algorithms generally struggle to scale to problems with hundreds of highly connected variables -- for instance, recovering brain networks from fMRI data. We introduce the best order score search (BOSS) and grow-shrink trees (GSTs) for learning directed acyclic graphs (DAGs) in this paradigm. BOSS greedily searches over permutations of variables, using GSTs to construct and score DAGs from permutations. GSTs efficiently cache scores to eliminate redundant calculations. BOSS achieves state-of-the-art performance in accuracy and execution time, comparing favorably to a variety of combinatorial and gradient-based learning algorithms under a broad range of conditions. To demonstrate its practicality, we apply BOSS to two sets of resting-state fMRI data: simulated data with pseudo-empirical noise distributions derived from randomized empirical fMRI cortical signals and clinical data from 3T fMRI scans processed into cortical parcels. BOSS is available for use within the TETRAD project which includes Python and R wrappers.

We give novel Python and R interfaces for the (Java) Tetrad project for causal modeling, search, and estimation. The Tetrad project is a mainstay in the literature, having been under consistent development for over 30 years. Some of its algorithms are now classics, like PC and FCI; others are recent developments. It is increasingly the case, however, that researchers need to access the underlying Java code from Python or R. Existing methods for doing this are inadequate. We provide new, up-to-date methods using the JPype Python-Java interface and the Reticulate Python-R interface, directly solving these issues. With the addition of some simple tools and the provision of working examples for both Python and R, using JPype and Reticulate to interface Python and R with Tetrad is straightforward and intuitive.

The Sparsest Permutation (SP) algorithm is accurate but limited to about 9 variables in practice; the Greedy Sparest Permutation (GSP) algorithm is faster but less weak theoretically. A compromise can be given, the Best Order Score Search, which gives results as accurate as SP but for much larger and denser graphs. BOSS (Best Order Score Search) is more accurate for two reason: (a) It assumes the "brute faithfuness" assumption, which is weaker than faithfulness, and (b) it uses a different traversal of permutations than the depth first traversal used by GSP, obtained by taking each variable in turn and moving it to the position in the permutation that optimizes the model score. Results are given comparing BOSS to several related papers in the literature in terms of performance, for linear, Gaussian data. In all cases, with the proper parameter settings, accuracy of BOSS is lifted considerably with respect to competing approaches. In configurations tested, models with 60 variables are feasible with large samples out to about an average degree of 12 in reasonable time, with near-perfect accuracy, and sparse models with an average degree of 4 are feasible out to about 300 variables on a laptop, again with near-perfect accuracy. Mixed continuous discrete and all-discrete datasets were also tested. The mixed data analysis showed advantage for BOSS over GES more apparent at higher depths with the same score; the discrete data analysis showed a very small advantage for BOSS over GES with the same score, perhaps not enough to prefer it.

In this report we describe a tool for comparing the performance of graphical causal structure learning algorithms implemented in the TETRAD freeware suite of causal analysis methods. Currently the tool is available as package in the TETRAD source code (written in Java). Simulations can be done varying the number of runs, sample sizes, and data modalities. Performance on this simulated data can then be compared for a number of algorithms, with parameters varied and with performance statistics as selected, producing a publishable report. The package presented here may also be used to compare structure learning methods across platforms and programming languages, i.e., to compare algorithms implemented in TETRAD with those implemented in MATLAB, Python, or R.

It is commonplace to encounter heterogeneous or nonstationary data, of which the underlying generating process changes across domains or over time. Such a distribution shift feature presents both challenges and opportunities for causal discovery. In this paper, we develop a framework for causal discovery from such data, called Constraint-based causal Discovery from heterogeneous/NOnstationary Data (CD-NOD), to find causal skeleton and directions and estimate the properties of mechanism changes. First, we propose an enhanced constraint-based procedure to detect variables whose local mechanisms change and recover the skeleton of the causal structure over observed variables. Second, we present a method to determine causal orientations by making use of independent changes in the data distribution implied by the underlying causal model, benefiting from information carried by changing distributions. After learning the causal structure, next, we investigate how to efficiently estimate the "driving force" of the nonstationarity of a causal mechanism. That is, we aim to extract from data a low-dimensional representation of changes. The proposed methods are nonparametric, with no hard restrictions on data distributions and causal mechanisms, and do not rely on window segmentation. Furthermore, we find that data heterogeneity benefits causal structure identification even with particular types of confounders. Finally, we show the connection between heterogeneity/nonstationarity and soft intervention in causal discovery. Experimental results on various synthetic and real-world data sets (task-fMRI and stock market data) are presented to demonstrate the efficacy of the proposed methods.

Autism spectrum disorder (ASD) is one of the major developmental disorders affecting children. Recently, it has been hypothesized that ASD is associated with atypical brain connectivities. A substantial body of researches use Pearson's correlation coefficients, mutual information, or partial correlation to investigate the differences in brain connectivities between ASD and typical controls from functional Magnetic Resonance Imaging (fMRI). However, correlation or partial correlation does not directly reveal causal influences - the information flow - between brain regions. Comparing to correlation, causality pinpoints the key connectivity characteristics and removes redundant features for diagnosis. In this paper, we propose a two-step method for large-scale and cyclic causal discovery from fMRI. It can identify brain causal structures without doing interventional experiments. The learned causal structure, as well as the causal influence strength, provides us the path and effectiveness of information flow. With the recovered causal influence strength as candidate features, we then perform ASD diagnosis by further doing feature selection and classification. We apply our methods to three datasets from Autism Brain Imaging Data Exchange (ABIDE). From experimental results, it shows that with causal connectivities, the diagnostic accuracy largely improves. A closer examination shows that information flows starting from the superior front gyrus to default mode network and posterior areas are largely reduced. Moreover, all enhanced information flows are from posterior to anterior or in local areas. Overall, it shows that long-range influences have a larger proportion of reductions than local ones, while local influences have a larger proportion of increases than long-range ones. By examining the graph properties of brain causal structure, the group of ASD shows reduced small-worldness.

We report a procedure that, in one step from continuous data with minimal preparation, recovers the graph found by Sachs et al. \cite{sachs2005causal}, with only a few edges different. The algorithm, Fast Adjacency Skewness (FASK), relies on a mixture of linear reasoning and reasoning from the skewness of variables; the Sachs data is a good candidate for this procedure since the skewness of the variables is quite pronounced. We review the ground truth model from Sachs et al. as well as some of the fluctuations seen in the protein abundances in the system, give the Sachs model and the FASK model, and perform a detailed comparison. Some variation in hyper-parameters is explored, though the main result uses values at or near the defaults learned from work modeling fMRI data.

In this paper we outline two novel scoring methods for learning Bayesian networks in the presence of both continuous and discrete variables, that is, mixed variables. While much work has been done in the domain of automated Bayesian network learning, few studies have investigated this task in the presence of both continuous and discrete variables while focusing on scalability. Our goal is to provide two novel and scalable scoring functions capable of handling mixed variables. The first method, the Conditional Gaussian (CG) score, provides a highly efficient option. The second method, the Mixed Variable Polynomial (MVP) score, allows for a wider range of modeled relationships, including nonlinearity, but it is slower than CG. Both methods calculate log likelihood and degrees of freedom terms, which are incorporated into a Bayesian Information Criterion (BIC) score. Additionally, we introduce a structure prior for efficient learning of large networks and a simplification in scoring the discrete case which performs well empirically. While the core of this work focuses on applications in the search and score paradigm, we also show how the introduced scoring functions may be readily adapted as conditional independence tests for constraint-based Bayesian network learning algorithms. Lastly, we describe ways to simulate networks of mixed variable types and evaluate our proposed methods on such simulations.

We test the adequacies of several proposed and two new statistical methods for recovering the causal structure of systems with feedback that generate noisy time series closely matching real BOLD time series. We compare: an adaptation for time series of the first correct method for recovering the structure of cyclic linear systems; multivariate Granger causal regression; the GIMME algorithm; the Ramsey et al. non-Gaussian methods; two non-Gaussian methods proposed by Hyv¨arinen and Smith; a method due to Patel, et al.; and the GlobalMIT algorithm. We introduce and also compare two new methods, the Fast Adjacency Skewness (FASK) and Two-Step, which exploit non-Gaussian features of the BOLD signal in different ways. We give theoretical justifications for the latter two algorithms. Our test models include feedback structures with and without direct feedback (2-cycles), excitatory and inhibitory feedback, models using experimentally determined structural connectivities of macaques, and empirical resting state and task data. We find that averaged over all of our simulations, including those with 2-cycles, several of these methods have a better than 80% orientation precision (i.e., the probability a directed edge is in the true generating structure given that a procedure estimates it to be so) and the two new methods also have better than 80% recall (probability of recovering an orientation in the data generating model). Recovering inhibitory direct feedback loops between two regions is especially challenging.