Distributed Statistical Analyses: A Scoping Review and Examples of Operational Frameworks Adapted to Healthcare
Preprint
Abstract
Data from multiple organizations are crucial for advancing learning health systems. However, ethical, legal, and social concerns may restrict the use of standard statistical methods that rely on pooling data. Although distributed algorithms offer alternatives, they may not always be suitable for healthcare research frameworks. This paper aims to support researchers and data custodians in three ways: (1) providing a concise overview of the literature on statistical inference methods for horizontally partitioned data; (2) describing the methods applicable to generalized linear models (GLM) and assessing their underlying distributional assumptions; (3) adapting existing methods to make them fully usable in healthcare research. A scoping review methodology was employed for the literature mapping, from which methods presenting a methodological framework for GLM analyses with horizontally partitioned data were identified and assessed from the perspective of applicability in healthcare research. From the review, 41 articles were selected, and six approaches were extracted for conducting standard GLM-based statistical analysis. However, these approaches assumed evenly and identically distributed data across nodes. Consequently, statistical procedures were derived to accommodate uneven node sample sizes and heterogeneous data distributions across nodes. Workflows and detailed algorithms were developed to highlight information-sharing requirements and operational complexity.Keywords
This publication has 53 references indexed in Scilit:
- Unbalanced distributed estimation and inference for the precision matrix in Gaussian graphical modelsStatistics and Computing, 2023
- Online two‐way estimation and inference via linear mixed‐effects modelsStatistics in Medicine, 2022
- Communication‐efficient estimation and inference for high‐dimensional quantile regression based on smoothed decorrelated scoreStatistics in Medicine, 2022
- Distributed adaptive Huber regressionComputational Statistics & Data Analysis, 2022
- Computing confidence intervals from massive data via penalized quantile smoothing splinesComputational Statistics & Data Analysis, 2020
- A partitioned quasi-likelihood for distributed statistical inferenceComputational Statistics, 2020
- Distributed inference for degenerate U‐statisticsStat, 2019
- Aggregated inferenceWIREs Computational Statistics, 2018
- Fast surrogates of U-statisticsComputational Statistics & Data Analysis, 2009
- Weak Convergence and Empirical Processes1996