Joel M. Levin
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Joel M. Levin has a total of 846 co-authors. Here are the top co-authors by number of shared publications:
- Christian Oswald
University of the Bundeswehr Munich
3 shared publications - Christian S. Czymara
Netherlands Interdisciplinary Demographic Institute, KNAW/University of Groningen, The Hague, The Netherlands
3 shared publications - Diego Marino Fages
University Business Centre, Durham University, Durham, UK
3 shared publications - Jin Kim
Chinese University of Hong Kong, Hong Kong, China
3 shared publications - Jonathan D. Hall
University of Alabama, Tuscaloosa, USA
3 shared publications - Leigh A. Bukowski
Department of Critical Care Medicine, University of Pittsburgh School of Medicine, United States
3 shared publications
Latest Publications
- Open Access
- 23 April 2026
- Published by Springer Nature in Nature
- Vol. 653 (8114) , E7
Publisher Correction: Reproducibility and robustness of economics and political science research
- 23 April 2026
- Published by Springer Nature in Nature
- Vol. 653 (8114) , E7
Open AccessPublisher Correction: Reproducibility and robustness of economics and political science research
- Top Cited Papers
- 1 April 2026
- Published by Springer Nature in Nature
- Vol. 652 (8108) , 151-156
Reproducibility and robustness of economics and political science research
Science aspires to be cumulative. Reproducibility efforts strengthen science by testing the reliability of published findings, promoting self-correction, and informing policy-making1. Computational reproductions, whereby independent researchers reproduce the results of published studies, are an essential diagnostic tool2,3,4,5,6,7,8,9,10. Such efforts should have greater visibility11,12,13,14,15,16. However, little social science reproduction and robustness has been conducted at scale10,13,17,18,19,20,21,22,23. Here we reproduced original analyses and conducted robustness checks of 110 articles that were published in leading economics and political science journals with mandatory data and code sharing policies17,18. We found that more than 85% of published claims were computationally reproducible. In robustness checks, our reanalyses showed that 72% of statistically significant estimates remain significant and in the same direction, and the median reproduced effect size is nearly the same as the originally published effect size (that is, 99% of the published effect size). Additionally, 6 independent research teams examined 12 pre-specified hypotheses about determinants of robustness. Research teams with more experience found lower levels of robustness, and robustness did not correlate with author characteristics or data availability.
- 1 April 2026
- Published by Springer Nature in Nature
- Vol. 652 (8108) , 151-156
Top Cited PapersReproducibility and robustness of economics and political science research
Science aspires to be cumulative. Reproducibility efforts strengthen science by testing the reliability of published findings, promoting self-correction, and informing policy-making1. Computational reproductions, whereby independent researchers reproduce the results of published studies, are an essential diagnostic tool2,3,4,5,6,7,8,9,10. Such efforts should have greater visibility11,12,13,14,15,16. However, little social science reproduction and robustness has been conducted at scale10,13,17,18,19,20,21,22,23. Here we reproduced original analyses and conducted robustness checks of 110 articles that were published in leading economics and political science journals with mandatory data and code sharing policies17,18. We found that more than 85% of published claims were computationally reproducible. In robustness checks, our reanalyses showed that 72% of statistically significant estimates remain significant and in the same direction, and the median reproduced effect size is nearly the same as the originally published effect size (that is, 99% of the published effect size). Additionally, 6 independent research teams examined 12 pre-specified hypotheses about determinants of robustness. Research teams with more experience found lower levels of robustness, and robustness did not correlate with author characteristics or data availability.
- Open Access
- 31 March 2026
- Published by Center for Open Science in MetaArXiv
Investigating the analytical robustness of the social and behavioural sciences
The same dataset can be analysed in different justifiable ways to answer the same research question, potentially challenging the robustness of empirical science1–3. In this crowd initiative, we investigated the degree to which research findings in the social and behavioural sciences are contingent on analysts’ choices. We examined a stratified random sample of 100 studies published between 2009 and 2018, where for one claim per study, at least five re-analysts independently re-analysed the original data. The statistical appropriateness of the re-analyses was assessed in peer evaluations, and the robustness indicators were inspected along a range of research characteristics and study designs. We found that 34% of the independent re-analyses yielded the same result (within a tolerance region of +/- 0.05 Cohen’s d) as the original report; with a four times broader tolerance region, this indicator rose to 57%. Regarding the conclusions drawn, 74% of analyses were reported to arrive at the same conclusion as in the original investigation; 24% to no effects/inconclusive result, and 2% to the opposite effect as in the original investigation. This exploratory study suggests that the common single-path analyses in social and behavioural research should not simply be assumed to be robust to alternative analyses4. Therefore, we recommend the development and use of practices to explore and communicate this neglected source of uncertainty.
- 31 March 2026
- Published by Center for Open Science in MetaArXiv
Open AccessInvestigating the analytical robustness of the social and behavioural sciences
The same dataset can be analysed in different justifiable ways to answer the same research question, potentially challenging the robustness of empirical science1–3. In this crowd initiative, we investigated the degree to which research findings in the social and behavioural sciences are contingent on analysts’ choices. We examined a stratified random sample of 100 studies published between 2009 and 2018, where for one claim per study, at least five re-analysts independently re-analysed the original data. The statistical appropriateness of the re-analyses was assessed in peer evaluations, and the robustness indicators were inspected along a range of research characteristics and study designs. We found that 34% of the independent re-analyses yielded the same result (within a tolerance region of +/- 0.05 Cohen’s d) as the original report; with a four times broader tolerance region, this indicator rose to 57%. Regarding the conclusions drawn, 74% of analyses were reported to arrive at the same conclusion as in the original investigation; 24% to no effects/inconclusive result, and 2% to the opposite effect as in the original investigation. This exploratory study suggests that the common single-path analyses in social and behavioural research should not simply be assumed to be robust to alternative analyses4. Therefore, we recommend the development and use of practices to explore and communicate this neglected source of uncertainty.
- Open Access
- 21 January 2025
- Published by Elsevier in CHEST Critical Care
- Vol. 3 (2)
Understanding Provider-Level Variation in Clinical Practice
- 21 January 2025
- Published by Elsevier in CHEST Critical Care
- Vol. 3 (2)
Open AccessUnderstanding Provider-Level Variation in Clinical Practice
- Top Cited PapersOpen Access
- 19 April 2023
- Published by Association for Computing Machinery (ACM)
- p. 1-18
Ignore, Trust, or Negotiate: Understanding Clinician Acceptance of AI-Based Treatment Recommendations in Health Care
Artificial intelligence (AI) in healthcare has the potential to improve patient outcomes, but clinician acceptance remains a critical barrier. We developed a novel decision support interface that provides interpretable treatment recommendations for sepsis, a life-threatening condition in which decisional uncertainty is common, treatment practices vary widely, and poor outcomes can occur even with optimal decisions. This system formed the basis of a mixed-methods study in which 24 intensive care clinicians made AI-assisted decisions on real patient cases. We found that explanations generally increased confidence in the AI, but concordance with specific recommendations varied beyond the binary acceptance or rejection described in prior work. Although clinicians sometimes ignored or trusted the AI, they also often prioritized aspects of the recommendations to follow, reject, or delay in a process we term “negotiation.” These results reveal novel barriers to adoption of treatment-focused AI tools and suggest ways to better support differing clinician perspectives.
- 19 April 2023
- Published by Association for Computing Machinery (ACM)
- p. 1-18
Top Cited PapersOpen AccessIgnore, Trust, or Negotiate: Understanding Clinician Acceptance of AI-Based Treatment Recommendations in Health Care
Artificial intelligence (AI) in healthcare has the potential to improve patient outcomes, but clinician acceptance remains a critical barrier. We developed a novel decision support interface that provides interpretable treatment recommendations for sepsis, a life-threatening condition in which decisional uncertainty is common, treatment practices vary widely, and poor outcomes can occur even with optimal decisions. This system formed the basis of a mixed-methods study in which 24 intensive care clinicians made AI-assisted decisions on real patient cases. We found that explanations generally increased confidence in the AI, but concordance with specific recommendations varied beyond the binary acceptance or rejection described in prior work. Although clinicians sometimes ignored or trusted the AI, they also often prioritized aspects of the recommendations to follow, reject, or delay in a process we term “negotiation.” These results reveal novel barriers to adoption of treatment-focused AI tools and suggest ways to better support differing clinician perspectives.
- Top Cited PapersOpen Access
- 8 February 2023
- Published by National Academy of Sciences in Proceedings of the National Academy of Sciences
- Vol. 120 (7)
The political polarization of COVID-19 treatments among physicians and laypeople in the United States
In the United States, liberals and conservatives disagree about facts. To what extent does expertise attenuate these disagreements? To study this question, we compare the polarization of beliefs about COVID-19 treatments among laypeople and critical care physicians. We find that political ideology predicts both groups’ beliefs about a range of COVID-19 treatments. These associations persist after controlling for a rich set of covariates, including local politics. We study two potential explanations: a) that partisans are exposed to different information and b) that they interpret the same information in different ways, finding evidence for both. Polarization is driven by preferences for partisan cable news but not by exposure to scientific research. Using a set of embedded experiments, we demonstrate that partisans perceive scientific evidence differently when it pertains to a politicized treatment (ivermectin), relative to when the treatment is not identified. These results highlight the extent to which political ideology is increasingly relevant for understanding beliefs, even among expert decision makers such as physicians.
- 8 February 2023
- Published by National Academy of Sciences in Proceedings of the National Academy of Sciences
- Vol. 120 (7)
Top Cited PapersOpen AccessThe political polarization of COVID-19 treatments among physicians and laypeople in the United States
In the United States, liberals and conservatives disagree about facts. To what extent does expertise attenuate these disagreements? To study this question, we compare the polarization of beliefs about COVID-19 treatments among laypeople and critical care physicians. We find that political ideology predicts both groups’ beliefs about a range of COVID-19 treatments. These associations persist after controlling for a rich set of covariates, including local politics. We study two potential explanations: a) that partisans are exposed to different information and b) that they interpret the same information in different ways, finding evidence for both. Polarization is driven by preferences for partisan cable news but not by exposure to scientific research. Using a set of embedded experiments, we demonstrate that partisans perceive scientific evidence differently when it pertains to a politicized treatment (ivermectin), relative to when the treatment is not identified. These results highlight the extent to which political ideology is increasingly relevant for understanding beliefs, even among expert decision makers such as physicians.
- Open Access
- 1 September 2022
- Published by Center for Open Science in OSF Preprints
Nonconcurrent Multiple-Baseline and Multiple-Probe Designs in Special Education Research: A Systematic Review of Current Practice and Future Directions
Although quality guidelines for single-case intervention research emphasize the importance of concurrent baselines in multiple-baseline and multiple-probe designs, nonconcurrent variations on these designs persist in the research literature. This study describes a systematic review of special education intervention studies (k = 406) between 1988-2020 that report using nonconcurrent multiple-baseline or multiple-probe designs to test interventions for individuals with disabilities ages 21 years and younger. We coded and synthesized study characteristics pertaining to participants, settings, interventions, data reporting, and contextual factors. Findings indicate the prevalence of nonconcurrent designs for intervention studies in a variety of settings addressing social and communication needs of individuals with disabilities. We discuss implications for research and practice and offer suggestions for improving the validity of nonconcurrent designs. A PRISMA-compliant abstract is available at https://osf.io/sdnj5/?view_only=f386b1fe5f14430a8d63fceed293718d.
- 1 September 2022
- Published by Center for Open Science in OSF Preprints
Open AccessNonconcurrent Multiple-Baseline and Multiple-Probe Designs in Special Education Research: A Systematic Review of Current Practice and Future Directions
Although quality guidelines for single-case intervention research emphasize the importance of concurrent baselines in multiple-baseline and multiple-probe designs, nonconcurrent variations on these designs persist in the research literature. This study describes a systematic review of special education intervention studies (k = 406) between 1988-2020 that report using nonconcurrent multiple-baseline or multiple-probe designs to test interventions for individuals with disabilities ages 21 years and younger. We coded and synthesized study characteristics pertaining to participants, settings, interventions, data reporting, and contextual factors. Findings indicate the prevalence of nonconcurrent designs for intervention studies in a variety of settings addressing social and communication needs of individuals with disabilities. We discuss implications for research and practice and offer suggestions for improving the validity of nonconcurrent designs. A PRISMA-compliant abstract is available at https://osf.io/sdnj5/?view_only=f386b1fe5f14430a8d63fceed293718d.