JLJoel M. Levin

Joel M. Levin

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Joel M. Levin has a total of 846 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Publisher Correction: Reproducibility and robustness of economics and political science research

    by Abel Brodeur, Derek Mikola, Nikolai Cook, Lenka Fiala, Thomas Brailey, Ryan Briggs, Alexandra de Gendre, Yannick Dupraz, Jacopo Gabani, Romain Gauriot, et al.

  • Top Cited Papers

    Reproducibility and robustness of economics and political science research

    by Abel Brodeur, Derek Mikola, Nikolai Cook, Lenka Fiala, Thomas Brailey, Ryan Briggs, Alexandra de Gendre, Yannick Dupraz, Jacopo Gabani, Romain Gauriot, et al.

    Science aspires to be cumulative. Reproducibility efforts strengthen science by testing the reliability of published findings, promoting self-correction, and informing policy-making1. Computational reproductions, whereby independent researchers reproduce the results of published studies, are an essential diagnostic tool2,3,4,5,6,7,8,9,10. Such efforts should have greater visibility11,12,13,14,15,16. However, little social science reproduction and robustness has been conducted at scale10,13,17,18,19,20,21,22,23. Here we reproduced original analyses and conducted robustness checks of 110 articles that were published in leading economics and political science journals with mandatory data and code sharing policies17,18. We found that more than 85% of published claims were computationally reproducible. In robustness checks, our reanalyses showed that 72% of statistically significant estimates remain significant and in the same direction, and the median reproduced effect size is nearly the same as the originally published effect size (that is, 99% of the published effect size). Additionally, 6 independent research teams examined 12 pre-specified hypotheses about determinants of robustness. Research teams with more experience found lower levels of robustness, and robustness did not correlate with author characteristics or data availability.


  • Open Access

    Investigating the analytical robustness of the social and behavioural sciences

    by Balazs Aczel, Barnabas Szaszi, Harry T Clelland, Marton Kovacs, Felix Holzmeister, Don van Ravenzwaaij, Hannah Schulz-Kümpel, Sabine Hoffmann, Gustav Nilsonne, Livia Kósa, et al.

    The same dataset can be analysed in different justifiable ways to answer the same research question, potentially challenging the robustness of empirical science1–3. In this crowd initiative, we investigated the degree to which research findings in the social and behavioural sciences are contingent on analysts’ choices. We examined a stratified random sample of 100 studies published between 2009 and 2018, where for one claim per study, at least five re-analysts independently re-analysed the original data. The statistical appropriateness of the re-analyses was assessed in peer evaluations, and the robustness indicators were inspected along a range of research characteristics and study designs. We found that 34% of the independent re-analyses yielded the same result (within a tolerance region of +/- 0.05 Cohen’s d) as the original report; with a four times broader tolerance region, this indicator rose to 57%. Regarding the conclusions drawn, 74% of analyses were reported to arrive at the same conclusion as in the original investigation; 24% to no effects/inconclusive result, and 2% to the opposite effect as in the original investigation. This exploratory study suggests that the common single-path analyses in social and behavioural research should not simply be assumed to be robust to alternative analyses4. Therefore, we recommend the development and use of practices to explore and communicate this neglected source of uncertainty.

  • Open Access

    Understanding Provider-Level Variation in Clinical Practice

    by Kanupriya Soni, Leigh A. Bukowski, Billie S. Davis, Joel M. Levin, Jeremy M. Kahn

  • Top Cited Papers
    Open Access

    Ignore, Trust, or Negotiate: Understanding Clinician Acceptance of AI-Based Treatment Recommendations in Health Care

    by Venkatesh Sivaraman, Leigh A Bukowski, Joel Levin, Jeremy M. Kahn, Adam Perer

    Artificial intelligence (AI) in healthcare has the potential to improve patient outcomes, but clinician acceptance remains a critical barrier. We developed a novel decision support interface that provides interpretable treatment recommendations for sepsis, a life-threatening condition in which decisional uncertainty is common, treatment practices vary widely, and poor outcomes can occur even with optimal decisions. This system formed the basis of a mixed-methods study in which 24 intensive care clinicians made AI-assisted decisions on real patient cases. We found that explanations generally increased confidence in the AI, but concordance with specific recommendations varied beyond the binary acceptance or rejection described in prior work. Although clinicians sometimes ignored or trusted the AI, they also often prioritized aspects of the recommendations to follow, reject, or delay in a process we term “negotiation.” These results reveal novel barriers to adoption of treatment-focused AI tools and suggest ways to better support differing clinician perspectives.


  • Top Cited Papers
    Open Access

    The political polarization of COVID-19 treatments among physicians and laypeople in the United States

    by Joel M. Levin, Leigh A. Bukowski, Julia A. Minson, Jeremy M. Kahn

    In the United States, liberals and conservatives disagree about facts. To what extent does expertise attenuate these disagreements? To study this question, we compare the polarization of beliefs about COVID-19 treatments among laypeople and critical care physicians. We find that political ideology predicts both groups’ beliefs about a range of COVID-19 treatments. These associations persist after controlling for a rich set of covariates, including local politics. We study two potential explanations: a) that partisans are exposed to different information and b) that they interpret the same information in different ways, finding evidence for both. Polarization is driven by preferences for partisan cable news but not by exposure to scientific research. Using a set of embedded experiments, we demonstrate that partisans perceive scientific evidence differently when it pertains to a politicized treatment (ivermectin), relative to when the treatment is not identified. These results highlight the extent to which political ideology is increasingly relevant for understanding beliefs, even among expert decision makers such as physicians.


  • Open Access

    Nonconcurrent Multiple-Baseline and Multiple-Probe Designs in Special Education Research: A Systematic Review of Current Practice and Future Directions

    by Kristi L. Morin, Esther R. Lindström, Tom Kratochwill, Joel M Levin, Alyssa Blasko, Amanda Weir, Christiana Nielsen, Samantha Kelly, Davit Janunts, Ee Rea Hong

    Although quality guidelines for single-case intervention research emphasize the importance of concurrent baselines in multiple-baseline and multiple-probe designs, nonconcurrent variations on these designs persist in the research literature. This study describes a systematic review of special education intervention studies (k = 406) between 1988-2020 that report using nonconcurrent multiple-baseline or multiple-probe designs to test interventions for individuals with disabilities ages 21 years and younger. We coded and synthesized study characteristics pertaining to participants, settings, interventions, data reporting, and contextual factors. Findings indicate the prevalence of nonconcurrent designs for intervention studies in a variety of settings addressing social and communication needs of individuals with disabilities. We discuss implications for research and practice and offer suggestions for improving the validity of nonconcurrent designs. A PRISMA-compliant abstract is available at https://osf.io/sdnj5/?view_only=f386b1fe5f14430a8d63fceed293718d.