Evaluation of Excess Statistical Significance in Meta-analyses of 98 Biomarker Associations with Cancer Risk
Open Access
- 22 October 2012
- journal article
- review article
- Published by Oxford University Press (OUP) in JNCI: Journal of the National Cancer Institute
- Vol. 104 (24) , 1867-1878
Abstract
Numerous biomarkers have been associated with cancer risk. We assessed whether there is evidence for excess statistical significance in results of cancer biomarker studies, suggesting biases. We systematically searched PubMed for meta-analyses of nongenetic biomarkers and cancer risk. The number of observed studies with statistically significant results was compared with the expected number, based on the statistical power of each study under different assumptions for the plausible effect size. We also evaluated small-study effects using asymmetry tests. All statistical tests were two-sided. We included 98 meta-analyses with 847 studies. Forty-three meta-analyses (44%) found nominally statistically significant summary effects (random effects). The proportion of meta-analyses with statistically significant effects was highest for infectious agents (86%), inflammatory (67%), and insulin-like growth factor (IGF)/insulin system (52%) biomarkers. Overall, 269 (32%) individual studies observed nominally statistically significant results. A statistically significant excess of the observed over the expected number of studies with statistically significant results was seen in 20 meta-analyses. An excess of observed vs expected was observed in studies of IGF/insulin (P ≤ .04) and inflammation systems (P ≤ .02). Only 12 meta-analyses (12%) had a statistically significant summary effect size, more than 1000 case patients, and no hints of small-study effects or excess statistical significance; only four of them had large effect sizes, three of which pertained to infectious agents (Helicobacter pylori, hepatitis and human papilloma viruses). Most well-documented biomarkers of cancer risk without evidence of bias pertain to infectious agents. Conversely, an excess of statistically significant findings was observed in studies of IGF/insulin and inflammation systems, suggesting reporting biases.Keywords
This publication has 77 references indexed in Scilit:
- Primary study authors of significant studies are more likely to believe that a strong association exists in a heterogeneous meta-analysis compared with methodologistsJournal of Clinical Epidemiology, 2012
- Insulin-like growth factor 1 (IGF1), IGF binding protein 3 (IGFBP3), and breast cancer risk: pooled individual data analysis of 17 prospective studiesThe Lancet Oncology, 2010
- The evolving discipline of molecular epidemiology of cancerCarcinogenesis, 2009
- Circulating insulin‐like growth factor peptides and prostate cancer risk: A systematic review and meta‐analysisInternational Journal of Cancer, 2009
- Celecoxib for the Prevention of Sporadic Colorectal AdenomasNew England Journal of Medicine, 2006
- Insulin-like growth factors and cancer: no role in screening. Evidence from the BUPA study and meta-analysis of prospective epidemiological studiesBritish Journal of Cancer, 2006
- A modified test for small‐study effects in meta‐analyses of controlled trials with binary endpointsStatistics in Medicine, 2005
- Empirical Evidence for Selective Reporting of Outcomes in Randomized TrialsJAMA, 2004
- Quantifying heterogeneity in a meta‐analysisStatistics in Medicine, 2002
- Meta-analysis in clinical trialsControlled Clinical Trials, 1986