Identifying correlations driven by influential observations in large datasets
- 2 December 2021
- journal article
- research article
- Published by Oxford University Press (OUP) in Briefings in Bioinformatics
- Vol. 23 (1)
Abstract
Although high-throughput data allow researchers to interrogate thousands of variables simultaneously, it can also introduce a significant number of spurious results. Here we demonstrate that correlation analysis of large datasets can yield numerous false positives due to the presence of outliers that canonical methods fail to identify. We present Correlations Under The InfluencE (CUTIE), an open-source jackknifing-based method to detect such cases with both parametric and non-parametric correlation measures, and which can also uniquely rescue correlations not originally deemed significant or with incorrect sign. Our approach can additionally be used to identify variables or samples that induce these false correlations in high proportion. A meta-analysis of various omics datasets using CUTIE reveals that this issue is pervasive across different domains, although microbiome data are particularly susceptible to it. Although the significance of a correlation eventually depends on the thresholds used, our approach provides an efficient way to automatically identify those that warrant closer examination in very large datasets.Keywords
Funding Information
- National Institutes of Health (S10OD018522)
This publication has 39 references indexed in Scilit:
- Archaea and Fungi of the Human Gut Microbiome: Correlations with Diet and Bacterial ResidentsPLOS ONE, 2013
- Inferring Correlation Networks from Genomic Survey DataPLOS Computational Biology, 2012
- Microbial Co-occurrence Relationships in the Human MicrobiomePLOS Computational Biology, 2012
- Detecting Novel Associations in Large Data SetsScience, 2011
- QIIME allows analysis of high-throughput community sequencing dataNature Methods, 2010
- Genetic Control of Human Brain Transcript Expression in Alzheimer DiseaseThe American Journal of Human Genetics, 2009
- WGCNA: an R package for weighted correlation network analysisBMC Bioinformatics, 2008
- Stability of gene contributions and identification of outliers in multivariate analysis of microarray dataBMC Bioinformatics, 2008
- The properties of high-dimensional data spaces: implications for exploring gene and protein expression dataNature Reviews Cancer, 2008
- A power primer.Psychological Bulletin, 1992