Warren B. Kindzierski
Research Subjects
Warren B. Kindzierski has a total of 20 co-authors. Here are the top co-authors by number of shared publications:
S. Stanley Young
7 shared publications- Padma Kaul
Canadian VIGOUR Centre, University of Alberta, Edmonton, Alberta, Canada
2 shared publications - Xiaoming Wang
School of Public Health, University of Alberta, Edmonton, Alberta, Canada
2 shared publications - Chwen-Jyh Jeng
Professor, Toxcon Health Sciences Research Centre Inc., 9607 41 Ave., Edmonton AB, Canada T6E 5X7
1 shared publication - Daniel W. Smith
Professor, Toxcon Health Sciences Research Centre Inc., 9607 41 Ave., Edmonton AB, Canada T6E 5X7
1 shared publication Jin Hua Chen
1 shared publication
Latest Publications
- Open Access
- 28 November 2024
- Published by Canadian Center of Science and Education in International Journal of Statistics and Probability
- Vol. 13  (4) , p1
The Reliability of the gender Implicit Association Test (gIAT) for Explaining Female−Male Differences in High-Ability Careers
Females are outnumbered by males in many high-ability careers in the fields of academic medicine and science, technology, engineering, and mathematics (STEM). These differences are often attributed to implicit bias as measured by the gender Implicit Association Test (gIAT). Statistical p-value plots were used to independently test the ability to reproduce research claims made in relation to female−male differences in these careers. The p-value plots were developed using data sets from two published meta-analyses. One examined predictive power of the gIAT, and the other examined predictive power of vocational interests (personal interests and behaviors) for explaining female−male differences in these careers. The gIAT (implicit bias) p-value plot showed that it is unreliable for predicting female−male differences. Whereas the p-value plot for vocational interests supported these differences. Researchers of implicit bias should expand their modeling to include vocational interests and additional relevant explanatory variables. In short, these meta-analyses and the p-value plots provided no support for the gender Implicit Association Test influencing choice and female−male differences of high-ability careers.
- 28 November 2024
- Published by Canadian Center of Science and Education in International Journal of Statistics and Probability
- Vol. 13  (4) , p1
Open AccessThe Reliability of the gender Implicit Association Test (gIAT) for Explaining Female−Male Differences in High-Ability Careers
Females are outnumbered by males in many high-ability careers in the fields of academic medicine and science, technology, engineering, and mathematics (STEM). These differences are often attributed to implicit bias as measured by the gender Implicit Association Test (gIAT). Statistical p-value plots were used to independently test the ability to reproduce research claims made in relation to female−male differences in these careers. The p-value plots were developed using data sets from two published meta-analyses. One examined predictive power of the gIAT, and the other examined predictive power of vocational interests (personal interests and behaviors) for explaining female−male differences in these careers. The gIAT (implicit bias) p-value plot showed that it is unreliable for predicting female−male differences. Whereas the p-value plot for vocational interests supported these differences. Researchers of implicit bias should expand their modeling to include vocational interests and additional relevant explanatory variables. In short, these meta-analyses and the p-value plots provided no support for the gender Implicit Association Test influencing choice and female−male differences of high-ability careers.
- Open Access
Reproducibility of Implicit Association Test (IAT) -- Case study of meta-analysis of racial bias research claims
The Implicit Association Test, IAT, is widely used to measure hidden (subconscious) human biases, implicit bias, of many topics: race, gender, age, ethnicity, religion stereotypes. There is a need to understand the reliability of these measures as they are being used in many decisions in society today. A case study was undertaken to independently test the reliability of (ability to reproduce) racial bias research claims of Black White relations based on IAT (implicit bias) and explicit bias measurements using statistical p value plots. These claims were for IAT, real world behavior correlations and explicit bias, real world behavior correlations of Black White relations. The p value plots were constructed using data sets from published literature and the plots exhibited considerable randomness for all correlations examined. This randomness supports a lack of correlation between IAT, implicit bias, and explicit bias measurements with real world behaviors of Whites towards Blacks. These findings were for microbehaviors (measures of nonverbal and subtle verbal behavior) and person perception judgments (explicit judgments about others). Findings of the p value plots were consistent with the case study research claim that the IAT provides little insight into who will discriminate against whom. It was also observed that the amount of real world variance explained by the IAT and explicit bias measurements was small, less than 5 percent. Others have noted that the poor performance of both the IAT and explicit bias measurements are mostly consistent with a (flawed instruments explanation) problems in theories that motivated development and use of these instruments.
Open AccessReproducibility of Implicit Association Test (IAT) -- Case study of meta-analysis of racial bias research claims
The Implicit Association Test, IAT, is widely used to measure hidden (subconscious) human biases, implicit bias, of many topics: race, gender, age, ethnicity, religion stereotypes. There is a need to understand the reliability of these measures as they are being used in many decisions in society today. A case study was undertaken to independently test the reliability of (ability to reproduce) racial bias research claims of Black White relations based on IAT (implicit bias) and explicit bias measurements using statistical p value plots. These claims were for IAT, real world behavior correlations and explicit bias, real world behavior correlations of Black White relations. The p value plots were constructed using data sets from published literature and the plots exhibited considerable randomness for all correlations examined. This randomness supports a lack of correlation between IAT, implicit bias, and explicit bias measurements with real world behaviors of Whites towards Blacks. These findings were for microbehaviors (measures of nonverbal and subtle verbal behavior) and person perception judgments (explicit judgments about others). Findings of the p value plots were consistent with the case study research claim that the IAT provides little insight into who will discriminate against whom. It was also observed that the amount of real world variance explained by the IAT and explicit bias measurements was small, less than 5 percent. Others have noted that the poor performance of both the IAT and explicit bias measurements are mostly consistent with a (flawed instruments explanation) problems in theories that motivated development and use of these instruments.
- Open Access
Statistical reproducibility of meta-analysis research claims for medical mask use in community settings to prevent COVID infection
The coronavirus pandemic (COVID) has been an exceptional test of current scientific evidence that inform and shape policy. Many US states, cities, and counties implemented public orders for mask use on the notion that this intervention would delay and flatten the epidemic peak and largely benefit public health outcomes. P-value plotting was used to evaluate statistical reproducibility of meta-analysis research claims of a benefit for medical (surgical) mask use in community settings to prevent COVID infection. Eight studies (seven meta-analyses, one systematic review) published between 1 January 2020 and 7 December 2022 were evaluated. Base studies were randomized control trials with outcomes of medical diagnosis or laboratory-confirmed diagnosis of viral (Influenza or COVID) illness. Self-reported viral illness outcomes were excluded because of awareness bias. No evidence was observed for a medical mask use benefit to prevent viral infections in six p-value plots (five meta-analyses and one systematic review). Research claims of no benefit in three meta-analyses and the systematic review were reproduced in p-value plots. Research claims of a benefit in two meta-analyses were not reproduced in p-value plots. Insufficient data were available to construct p-value plots for two meta-analyses because of overreliance on self-reported outcomes. These findings suggest a benefit for medical mask use in community settings to prevent viral, including COVID infection, is unproven.
Open AccessStatistical reproducibility of meta-analysis research claims for medical mask use in community settings to prevent COVID infection
The coronavirus pandemic (COVID) has been an exceptional test of current scientific evidence that inform and shape policy. Many US states, cities, and counties implemented public orders for mask use on the notion that this intervention would delay and flatten the epidemic peak and largely benefit public health outcomes. P-value plotting was used to evaluate statistical reproducibility of meta-analysis research claims of a benefit for medical (surgical) mask use in community settings to prevent COVID infection. Eight studies (seven meta-analyses, one systematic review) published between 1 January 2020 and 7 December 2022 were evaluated. Base studies were randomized control trials with outcomes of medical diagnosis or laboratory-confirmed diagnosis of viral (Influenza or COVID) illness. Self-reported viral illness outcomes were excluded because of awareness bias. No evidence was observed for a medical mask use benefit to prevent viral infections in six p-value plots (five meta-analyses and one systematic review). Research claims of no benefit in three meta-analyses and the systematic review were reproduced in p-value plots. Research claims of a benefit in two meta-analyses were not reproduced in p-value plots. Insufficient data were available to construct p-value plots for two meta-analyses because of overreliance on self-reported outcomes. These findings suggest a benefit for medical mask use in community settings to prevent viral, including COVID infection, is unproven.
- Open Access
- 30 December 2022
- Published by Canadian Center of Science and Education in International Journal of Statistics and Probability
- Vol. 12  (1) , p54
Reproducibility of Health Claims in Meta-Analysis Studies of COVID Quarantine (Stay-at-Home) Orders
The coronavirus pandemic (COVID) has been an extraordinary test of modern government scientific procedures that inform and shape policy. Many governments implemented COVID quarantine (stay-at-home) orders on the notion that this nonpharmaceutical intervention would delay and flatten the epidemic peak and largely benefit public health outcomes. The overall research capacity response to COVID since late 2019 has been massive. Given a lack of research transparency, only a small fraction of published research has been judged by others to be reproducible before COVID. Independent evaluation of published meta-analysis on a common research question can be used to assess the reproducibility of a claim coming from that field of research. We used a p-value plotting statistical method to independently evaluate reproducibility of specific research claims made in four meta-analysis studies related to benefits/harms of COVID quarantine orders. Research claims that these meta-analyses covered included: mortality, mental health symptoms, incidence of domestic violence, and suicidal ideation (thoughts of killing yourself). Three of the four meta-analysis studies that we evaluated (mortality, mental health symptoms, incidence of domestic violence) raise further questions about benefits/harms of this form of intervention. The fourth meta-analysis study (suicidal ideation) is judged to be unreliable. Given lack of research transparency and irreproducibility of published research, independent evaluation of meta-analysis studies using p-value plotting is offered as a way to strengthen or refute (falsify) claims made in COVID research.
- 30 December 2022
- Published by Canadian Center of Science and Education in International Journal of Statistics and Probability
- Vol. 12  (1) , p54
Open AccessReproducibility of Health Claims in Meta-Analysis Studies of COVID Quarantine (Stay-at-Home) Orders
The coronavirus pandemic (COVID) has been an extraordinary test of modern government scientific procedures that inform and shape policy. Many governments implemented COVID quarantine (stay-at-home) orders on the notion that this nonpharmaceutical intervention would delay and flatten the epidemic peak and largely benefit public health outcomes. The overall research capacity response to COVID since late 2019 has been massive. Given a lack of research transparency, only a small fraction of published research has been judged by others to be reproducible before COVID. Independent evaluation of published meta-analysis on a common research question can be used to assess the reproducibility of a claim coming from that field of research. We used a p-value plotting statistical method to independently evaluate reproducibility of specific research claims made in four meta-analysis studies related to benefits/harms of COVID quarantine orders. Research claims that these meta-analyses covered included: mortality, mental health symptoms, incidence of domestic violence, and suicidal ideation (thoughts of killing yourself). Three of the four meta-analysis studies that we evaluated (mortality, mental health symptoms, incidence of domestic violence) raise further questions about benefits/harms of this form of intervention. The fourth meta-analysis study (suicidal ideation) is judged to be unreliable. Given lack of research transparency and irreproducibility of published research, independent evaluation of meta-analysis studies using p-value plotting is offered as a way to strengthen or refute (falsify) claims made in COVID research.
- Open Access
Reproducibility of health claims in meta-analysis studies of COVID quarantine (stay-at-home) orders
The coronavirus pandemic (COVID) has been an extraordinary test of modern government scientific procedures that inform and shape policy. Many governments implemented COVID quarantine (stay-at-home) orders on the notion that this nonpharmaceutical intervention would delay and flatten the epidemic peak and largely benefit public health outcomes. The overall research capacity response to COVID since late 2019 has been massive. Given lack of research transparency, only a small fraction of published research has been judged by others to be reproducible before COVID. Independent evaluation of published meta-analysis on a common research question can be used to assess the reproducibility of a claim coming from that field of research. We used a p-value plotting statistical method to independently evaluate reproducibility of specific research claims made in four meta-analysis studies related to benefits/risks of COVID quarantine orders. Outcomes we investigated included: mortality, mental health symptoms, incidence of domestic violence, and suicidal ideation (thoughts of killing yourself). Three of the four meta-analyses that we evaluated (mortality, mental health symptoms, incidence of domestic violence) raise further questions about benefits/risks of this form of intervention. The fourth meta-analysis study (suicidal ideation) is unreliable. Given lack of research transparency and irreproducibility of published research, independent evaluation of meta-analysis studies using p-value plotting is offered as a way to strengthen or refute (falsify) claims made in COVID research.
Open AccessReproducibility of health claims in meta-analysis studies of COVID quarantine (stay-at-home) orders
The coronavirus pandemic (COVID) has been an extraordinary test of modern government scientific procedures that inform and shape policy. Many governments implemented COVID quarantine (stay-at-home) orders on the notion that this nonpharmaceutical intervention would delay and flatten the epidemic peak and largely benefit public health outcomes. The overall research capacity response to COVID since late 2019 has been massive. Given lack of research transparency, only a small fraction of published research has been judged by others to be reproducible before COVID. Independent evaluation of published meta-analysis on a common research question can be used to assess the reproducibility of a claim coming from that field of research. We used a p-value plotting statistical method to independently evaluate reproducibility of specific research claims made in four meta-analysis studies related to benefits/risks of COVID quarantine orders. Outcomes we investigated included: mortality, mental health symptoms, incidence of domestic violence, and suicidal ideation (thoughts of killing yourself). Three of the four meta-analyses that we evaluated (mortality, mental health symptoms, incidence of domestic violence) raise further questions about benefits/risks of this form of intervention. The fourth meta-analysis study (suicidal ideation) is unreliable. Given lack of research transparency and irreproducibility of published research, independent evaluation of meta-analysis studies using p-value plotting is offered as a way to strengthen or refute (falsify) claims made in COVID research.
- Open Access
- 25 June 2021
- Published by MDPI AG in Journal of Respiration
- Vol. 1  (3) , 173-196
Evaluation of a Meta-Analysis of Ambient Air Quality as a Risk Factor for Asthma Exacerbation
Background: An irreproducibility crisis currently afflicts a wide range of scientific disciplines, including public health and biomedical science. A study was undertaken to assess the reliability of a meta-analysis examining whether air quality components (carbon monoxide, particulate matter 10 µm and 2.5 µm (PM10 and PM2.5), sulfur dioxide, nitrogen dioxide and ozone) are risk factors for asthma exacerbation. Methods: The number of statistical tests and models were counted in 17 randomly selected base papers from 87 used in the meta-analysis. Confidence intervals from all 87 base papers were converted to p-values. p-value plots for each air component were constructed to evaluate the effect heterogeneity of the p-values. Results: The number of statistical tests possible in the 17 selected base papers was large, median = 15,360 (interquartile range = 1536–40,960), in comparison to results presented. Each p-value plot showed a two-component mixture with small p-values < 0.001 while other p-values appeared random (p-values > 0.05). Given potentially large numbers of statistical tests conducted in the 17 selected base papers, p-hacking cannot be ruled out as explanations for small p-values. Conclusions: Our interpretation of the meta-analysis is that random p-values indicating null associations are more plausible and the meta-analysis is unlikely to replicate in the absence of bias.
- 25 June 2021
- Published by MDPI AG in Journal of Respiration
- Vol. 1  (3) , 173-196
Open AccessEvaluation of a Meta-Analysis of Ambient Air Quality as a Risk Factor for Asthma Exacerbation
Background: An irreproducibility crisis currently afflicts a wide range of scientific disciplines, including public health and biomedical science. A study was undertaken to assess the reliability of a meta-analysis examining whether air quality components (carbon monoxide, particulate matter 10 µm and 2.5 µm (PM10 and PM2.5), sulfur dioxide, nitrogen dioxide and ozone) are risk factors for asthma exacerbation. Methods: The number of statistical tests and models were counted in 17 randomly selected base papers from 87 used in the meta-analysis. Confidence intervals from all 87 base papers were converted to p-values. p-value plots for each air component were constructed to evaluate the effect heterogeneity of the p-values. Results: The number of statistical tests possible in the 17 selected base papers was large, median = 15,360 (interquartile range = 1536–40,960), in comparison to results presented. Each p-value plot showed a two-component mixture with small p-values < 0.001 while other p-values appeared random (p-values > 0.05). Given potentially large numbers of statistical tests conducted in the 17 selected base papers, p-hacking cannot be ruled out as explanations for small p-values. Conclusions: Our interpretation of the meta-analysis is that random p-values indicating null associations are more plausible and the meta-analysis is unlikely to replicate in the absence of bias.
- Open Access
Reliability of meta-analysis of an association between ambient air quality and development of asthma later in life
Claims from observational studies often fail to replicate. A study was undertaken to assess the reliability of cohort studies used in a highly cited meta-analysis of the association between ambient nitrogen dioxide, NO2, and fine particulate matter, PM2.5, concentrations early in life and development of asthma later in life. The numbers of statistical tests possible were estimated for 19 base papers considered for the meta-analysis. A p-value plot for NO2 and PM2.5 was constructed to evaluate effect heterogeneity of p-values used from the base papers. The numbers of statistical tests possible in the base papers were large - median 13,824, interquartile range 1,536-221,184; range 96-42M, in comparison to statistical test results presented. Statistical test results drawn from the base papers are unlikely to provide unbiased measures for meta-analysis. The p-value plot indicated that heterogeneity of the NO2 results across the base papers is consistent with a two-component mixture. First, it makes no sense to average across a mixture in meta-analysis. Second, the shape of the p-value plot for NO2 appears consistent with the possibility of analysis manipulation to obtain small p-values in several of the cohort studies. As for PM2.5, all corresponding p-values fall on a 45-degree line indicating complete randomness rather than a true association. Our interpretation of the meta-analysis is that the random p-values indicating no cause-effect associations are more plausible and that their meta-analysis will not likely replicate in the absence of bias. We conclude that claims made in the base papers used for meta-analysis are unreliable due to bias induced by multiple testing and multiple modelling, MTMM. We also show there is evidence that the heterogeneity across the base papers used for meta-analysis is more complex than simple sampling from a normal process.
Open AccessReliability of meta-analysis of an association between ambient air quality and development of asthma later in life
Claims from observational studies often fail to replicate. A study was undertaken to assess the reliability of cohort studies used in a highly cited meta-analysis of the association between ambient nitrogen dioxide, NO2, and fine particulate matter, PM2.5, concentrations early in life and development of asthma later in life. The numbers of statistical tests possible were estimated for 19 base papers considered for the meta-analysis. A p-value plot for NO2 and PM2.5 was constructed to evaluate effect heterogeneity of p-values used from the base papers. The numbers of statistical tests possible in the base papers were large - median 13,824, interquartile range 1,536-221,184; range 96-42M, in comparison to statistical test results presented. Statistical test results drawn from the base papers are unlikely to provide unbiased measures for meta-analysis. The p-value plot indicated that heterogeneity of the NO2 results across the base papers is consistent with a two-component mixture. First, it makes no sense to average across a mixture in meta-analysis. Second, the shape of the p-value plot for NO2 appears consistent with the possibility of analysis manipulation to obtain small p-values in several of the cohort studies. As for PM2.5, all corresponding p-values fall on a 45-degree line indicating complete randomness rather than a true association. Our interpretation of the meta-analysis is that the random p-values indicating no cause-effect associations are more plausible and that their meta-analysis will not likely replicate in the absence of bias. We conclude that claims made in the base papers used for meta-analysis are unreliable due to bias induced by multiple testing and multiple modelling, MTMM. We also show there is evidence that the heterogeneity across the base papers used for meta-analysis is more complex than simple sampling from a normal process.
- Open Access
Evaluation of a meta-analysis of ambient air quality as a risk factor for asthma exacerbation
False-positive results and bias may be common features of the biomedical literature today, including risk factor-chronic disease research. A study was undertaken to assess the reliability of base studies used in a meta-analysis examining whether carbon monoxide, particulate matter 10 and 2.5 micro molar, sulfur dioxide, nitrogen dioxide and ozone are risk factors for asthma exacerbation (hospital admission and emergency room visits for asthma attack). The number of statistical tests and models were counted in 17 randomly selected base papers from 87 used in the meta-analysis. P-value plots for each air component were constructed to evaluate the effect heterogeneity of p-values used from all 87 base papers The number of statistical tests possible in the 17 selected base papers was large, median=15,360 (interquartile range=1,536 to 40,960), in comparison to results presented. Each p-value plot showed a two-component mixture with small p-values less than .001 while other p-values appeared random (p-values greater than .05). Given potentially large numbers of statistical tests conducted in the 17 selected base papers, p-hacking cannot be ruled out as explanations for small p-values. Our interpretation of the meta-analysis is that the random p-values indicating null associations are more plausible and that the meta-analysis will not likely replicate in the absence of bias. We conclude the meta-analysis and base papers used are unreliable and do not offer evidence of value to inform public health practitioners about air quality as a risk factor for asthma exacerbation. The following areas are crucial for enabling improvements in risk factor chronic disease observational studies at the funding agency and journal level: preregistration, changes in funding agency and journal editor (and reviewer) practices, open sharing of data and facilitation of reproducibility research.
Open AccessEvaluation of a meta-analysis of ambient air quality as a risk factor for asthma exacerbation
False-positive results and bias may be common features of the biomedical literature today, including risk factor-chronic disease research. A study was undertaken to assess the reliability of base studies used in a meta-analysis examining whether carbon monoxide, particulate matter 10 and 2.5 micro molar, sulfur dioxide, nitrogen dioxide and ozone are risk factors for asthma exacerbation (hospital admission and emergency room visits for asthma attack). The number of statistical tests and models were counted in 17 randomly selected base papers from 87 used in the meta-analysis. P-value plots for each air component were constructed to evaluate the effect heterogeneity of p-values used from all 87 base papers The number of statistical tests possible in the 17 selected base papers was large, median=15,360 (interquartile range=1,536 to 40,960), in comparison to results presented. Each p-value plot showed a two-component mixture with small p-values less than .001 while other p-values appeared random (p-values greater than .05). Given potentially large numbers of statistical tests conducted in the 17 selected base papers, p-hacking cannot be ruled out as explanations for small p-values. Our interpretation of the meta-analysis is that the random p-values indicating null associations are more plausible and that the meta-analysis will not likely replicate in the absence of bias. We conclude the meta-analysis and base papers used are unreliable and do not offer evidence of value to inform public health practitioners about air quality as a risk factor for asthma exacerbation. The following areas are crucial for enabling improvements in risk factor chronic disease observational studies at the funding agency and journal level: preregistration, changes in funding agency and journal editor (and reviewer) practices, open sharing of data and facilitation of reproducibility research.
- 2 January 2019
- Published by Taylor & Francis in Critical Reviews in Toxicology
- Vol. 49  (1) , 85-94
Evaluation of a meta-analysis of air quality and heart attacks, a case study
It is generally acknowledged that claims from observational studies often fail to replicate. An exploratory study was undertaken to assess the reliability of base studies used in meta-analysis of short-term air quality-myocardial infarction risk and to judge the reliability of statistical evidence from meta-analysis that uses data from observational studies. A highly cited meta-analysis paper examining whether short-term air quality exposure triggers myocardial infarction was evaluated as a case study. The paper considered six air quality components - carbon monoxide, nitrogen dioxide, sulphur dioxide, particulate matter 10 μm and 2.5 μm in diameter (PM10 and PM2.5), and ozone. The number of possible questions and statistical models at issue in each of 34 base papers used were estimated and p-value plots for each of the air components were constructed to evaluate the effect heterogeneity of p-values used from the base papers. Analysis search spaces (number of statistical tests possible) in the base papers were large, median = 12,288 (interquartile range = 2496 − 58,368), in comparison to actual statistical test results presented. Statistical test results taken from the base papers may not provide unbiased measures of effect for meta-analysis. Shapes of p-value plots for the six air components were consistent with the possibility of analysis manipulation to obtain small p-values in several base papers. Results suggest the appearance of heterogeneous, researcher-generated p-values used in the meta-analysis rather than unbiased evidence of real effects for air quality. We conclude that this meta-analysis does not provide reliable evidence for an association of air quality components with myocardial risk.
- 2 January 2019
- Published by Taylor & Francis in Critical Reviews in Toxicology
- Vol. 49  (1) , 85-94
Evaluation of a meta-analysis of air quality and heart attacks, a case study
It is generally acknowledged that claims from observational studies often fail to replicate. An exploratory study was undertaken to assess the reliability of base studies used in meta-analysis of short-term air quality-myocardial infarction risk and to judge the reliability of statistical evidence from meta-analysis that uses data from observational studies. A highly cited meta-analysis paper examining whether short-term air quality exposure triggers myocardial infarction was evaluated as a case study. The paper considered six air quality components - carbon monoxide, nitrogen dioxide, sulphur dioxide, particulate matter 10 μm and 2.5 μm in diameter (PM10 and PM2.5), and ozone. The number of possible questions and statistical models at issue in each of 34 base papers used were estimated and p-value plots for each of the air components were constructed to evaluate the effect heterogeneity of p-values used from the base papers. Analysis search spaces (number of statistical tests possible) in the base papers were large, median = 12,288 (interquartile range = 2496 − 58,368), in comparison to actual statistical test results presented. Statistical test results taken from the base papers may not provide unbiased measures of effect for meta-analysis. Shapes of p-value plots for the six air components were consistent with the possibility of analysis manipulation to obtain small p-values in several base papers. Results suggest the appearance of heterogeneous, researcher-generated p-values used in the meta-analysis rather than unbiased evidence of real effects for air quality. We conclude that this meta-analysis does not provide reliable evidence for an association of air quality components with myocardial risk.
- 7 September 2018
- Published by SAGE Publications in Statistical Methods in Medical Research
- Vol. 28  (10-11) , 3100-3111
Eliminating systematic bias from case-crossover designs
Case-crossover designs have been widely applied to epidemiological and medical investigations of associations between short-term exposures and risk of acute adverse health events. Much effort has been made in literature on understanding source of confounding and reducing systematic bias by reference-select strategies. In this paper, we explored the nature of bias in the ambi-directional and time-stratified case-crossover designs via simulation using actual air pollution data from urban Edmonton, Alberta, Canada. We further proposed a calibration approach for eliminating systematic bias in estimates (coefficient estimate, 95% confident interval, and p-value). Bias check for coefficient estimation, size check and power check for significance test were done via simulation experiments to show advantages of the calibrated case-crossover studies over the ones without calibration. An application was done to investigate associations between air pollutants and acute myocardial infarction hospitalizations in urban Edmonton. In conclusion, systematic bias in a case-crossover design is often unavoidable, leading to an obvious bias in the estimated effect and an unreliable p value in the significance test. The proposed calibration technique provides an efficient approach to eliminating systematic bias in a case-crossover study.
- 7 September 2018
- Published by SAGE Publications in Statistical Methods in Medical Research
- Vol. 28  (10-11) , 3100-3111
Eliminating systematic bias from case-crossover designs
Case-crossover designs have been widely applied to epidemiological and medical investigations of associations between short-term exposures and risk of acute adverse health events. Much effort has been made in literature on understanding source of confounding and reducing systematic bias by reference-select strategies. In this paper, we explored the nature of bias in the ambi-directional and time-stratified case-crossover designs via simulation using actual air pollution data from urban Edmonton, Alberta, Canada. We further proposed a calibration approach for eliminating systematic bias in estimates (coefficient estimate, 95% confident interval, and p-value). Bias check for coefficient estimation, size check and power check for significance test were done via simulation experiments to show advantages of the calibrated case-crossover studies over the ones without calibration. An application was done to investigate associations between air pollutants and acute myocardial infarction hospitalizations in urban Edmonton. In conclusion, systematic bias in a case-crossover design is often unavoidable, leading to an obvious bias in the estimated effect and an unreliable p value in the significance test. The proposed calibration technique provides an efficient approach to eliminating systematic bias in a case-crossover study.