Reliability of meta-analysis of an association between ambient air quality and development of asthma later in life
Authors:
- S. Stanley YoungSYS. Stanley Young
- Jin Hua ChenJCJin Hua Chen
- Warren B. KindzierskiWarren B. Kindzierski
Preprint
Abstract
Claims from observational studies often fail to replicate. A study was undertaken to assess the reliability of cohort studies used in a highly cited meta-analysis of the association between ambient nitrogen dioxide, NO2, and fine particulate matter, PM2.5, concentrations early in life and development of asthma later in life. The numbers of statistical tests possible were estimated for 19 base papers considered for the meta-analysis. A p-value plot for NO2 and PM2.5 was constructed to evaluate effect heterogeneity of p-values used from the base papers. The numbers of statistical tests possible in the base papers were large - median 13,824, interquartile range 1,536-221,184; range 96-42M, in comparison to statistical test results presented. Statistical test results drawn from the base papers are unlikely to provide unbiased measures for meta-analysis. The p-value plot indicated that heterogeneity of the NO2 results across the base papers is consistent with a two-component mixture. First, it makes no sense to average across a mixture in meta-analysis. Second, the shape of the p-value plot for NO2 appears consistent with the possibility of analysis manipulation to obtain small p-values in several of the cohort studies. As for PM2.5, all corresponding p-values fall on a 45-degree line indicating complete randomness rather than a true association. Our interpretation of the meta-analysis is that the random p-values indicating no cause-effect associations are more plausible and that their meta-analysis will not likely replicate in the absence of bias. We conclude that claims made in the base papers used for meta-analysis are unreliable due to bias induced by multiple testing and multiple modelling, MTMM. We also show there is evidence that the heterogeneity across the base papers used for meta-analysis is more complex than simple sampling from a normal process.Keywords
All Related Versions
This publication has 80 references indexed in Scilit:
- Association of Long-term Exercise Training With Risk of Falls, Fractures, Hospitalizations, and Mortality in Older AdultsJAMA Internal Medicine, 2019
- Heterogeneity and the origins of asthmaAnnals of Allergy, Asthma & Immunology, 2018
- Evolution of ReportingPValues in the Biomedical Literature, 1990-2015JAMA, 2016
- Erratum to: Long-term exposure to air pollution and the incidence of asthma: meta-analysis of cohort studiesAir Quality, Atmosphere & Health, 2012
- Long-term exposure to air pollution and the incidence of asthma: meta-analysis of cohort studiesAir Quality, Atmosphere & Health, 2011
- Childhood cat exposure–related tolerance is associated with IL1A and IL10 polymorphismsJournal of Allergy and Clinical Immunology, 2005
- The role of air pollution in asthma and other pediatric morbiditiesJournal of Allergy and Clinical Immunology, 2005
- Model uncertainty and health effect studies for particulate matterEnvironmetrics, 2000
- Long-Term Ambient Ozone Concentration and the Incidence of Asthma in Nonsmoking Adults: The Ahsmog StudyEnvironmental Research, 1999
- Meta-analysis in clinical trialsControlled Clinical Trials, 1986