Berna Devezer
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Berna Devezer has a total of 88 co-authors. Here are the top co-authors by number of shared publications:
Dora Matzke
Department of Psychology, University of Amsterdam, Amsterdam, The Netherlands
4 shared publicationsErkan O. Buzbas
4 shared publications- Erkan Ozge Buzbas
Department of Mathematics and Statistical Science, University of Idaho, Moscow, ID 83844, USA
4 shared publications Bert Baumgaertner
Department of Politics and Philosophy, University of Idaho, Moscow, ID 83844, USA
3 shared publications- Joachim Vandekerckhove
University of California
3 shared publications Alexander Etz
2 shared publications
Latest Publications
- 2 July 2026
- Published by Springer Nature
- p. 47-68
Toward a More Accurate Notion of Exploratory Research (And Why It Matters)
- 2 July 2026
- Published by Springer Nature
- p. 47-68
Toward a More Accurate Notion of Exploratory Research (And Why It Matters)
- 8 April 2026
- Published by Springer Nature in Computational Brain & Behavior
- Vol. 9 (3) , 446-449
Against the Pathologization of Scientific Understanding
- 8 April 2026
- Published by Springer Nature in Computational Brain & Behavior
- Vol. 9 (3) , 446-449
Against the Pathologization of Scientific Understanding
- Open Access
- 21 March 2026
- Published by Center for Open Science in PsyArXiv
Against the Pathologization of Scientific Understanding
Illusions of Understanding in the Sciences raises a concern: Scientists frequently overestimate what they understand about the systems they study. This overestimation is said to potentially distort inference, encourage premature theoretical closure, and undermine public trust in science. We argue that the article fails to distinguish between remediable problems, structural challenges, and genuine cognitive limitations; mistakes individual limitations for system-wide epistemic failures; and treats idealization and pluralism as epistemic concessions rather than constitutive features of scientific endeavor. We believe that the unified `illusion' pathology does not successfully accommodate the organized, socially distributed nature of scientific endeavor.
- 21 March 2026
- Published by Center for Open Science in PsyArXiv
Open AccessAgainst the Pathologization of Scientific Understanding
Illusions of Understanding in the Sciences raises a concern: Scientists frequently overestimate what they understand about the systems they study. This overestimation is said to potentially distort inference, encourage premature theoretical closure, and undermine public trust in science. We argue that the article fails to distinguish between remediable problems, structural challenges, and genuine cognitive limitations; mistakes individual limitations for system-wide epistemic failures; and treats idealization and pluralism as epistemic concessions rather than constitutive features of scientific endeavor. We believe that the unified `illusion' pathology does not successfully accommodate the organized, socially distributed nature of scientific endeavor.
- Open Access
Statistics in Service of Metascience: Measuring Replication Distance with Reproducibility Rate
Motivated by the recent putative reproducibility crisis, we discuss the relationship between the replicability of scientific studies, the reproducibility of results obtained in these replications, and the philosophy of statistics. Our approach focuses on challenges in specifying scientific studies for scientific inference via statistical inference and is complementary to classical discussions in the philosophy of statistics. We particularly consider the challenges in replicating studies exactly, using the notion of the idealized experiment. We argue against treating reproducibility as an inherently desirable property of scientific results, and in favor of viewing it as a tool to measure the distance between an original study and its replications. To sensibly study the implications of replicability and results reproducibility on inference, such a measure of replication distance is needed. We present an effort to delineate such a framework here, addressing some challenges in capturing the components of scientific studies while identifying others as ongoing issues. We illustrate our measure of replication distance by simulations using a toy example. Rather than replications, we present purposefully planned modifications as an appropriate tool to inform scientific inquiry. Our ability to measure replication distance serves scientists in their search for replication-ready studies. We believe that likelihood-based and evidential approaches may play a critical role towards building statistics that effectively serve the practical needs of science.
Open AccessStatistics in Service of Metascience: Measuring Replication Distance with Reproducibility Rate
Motivated by the recent putative reproducibility crisis, we discuss the relationship between the replicability of scientific studies, the reproducibility of results obtained in these replications, and the philosophy of statistics. Our approach focuses on challenges in specifying scientific studies for scientific inference via statistical inference and is complementary to classical discussions in the philosophy of statistics. We particularly consider the challenges in replicating studies exactly, using the notion of the idealized experiment. We argue against treating reproducibility as an inherently desirable property of scientific results, and in favor of viewing it as a tool to measure the distance between an original study and its replications. To sensibly study the implications of replicability and results reproducibility on inference, such a measure of replication distance is needed. We present an effort to delineate such a framework here, addressing some challenges in capturing the components of scientific studies while identifying others as ongoing issues. We illustrate our measure of replication distance by simulations using a toy example. Rather than replications, we present purposefully planned modifications as an appropriate tool to inform scientific inquiry. Our ability to measure replication distance serves scientists in their search for replication-ready studies. We believe that likelihood-based and evidential approaches may play a critical role towards building statistics that effectively serve the practical needs of science.
- 24 September 2024
- Published by Springer Nature in Nature Human Behaviour
- Vol. 8 (10) , 1890-1891
Claims about scientific rigour require rigour
- 24 September 2024
- Published by Springer Nature in Nature Human Behaviour
- Vol. 8 (10) , 1890-1891
Claims about scientific rigour require rigour
- Open Access
Statistics in service of metascience: Measuring replication distance with reproducibility rate
Motivated by the recent putative reproducibility crisis, we discuss the relationship between replicability of scientific studies, reproducibility of results obtained in these replications, and the philosophy of statistics. Our approach focuses on challenges in specifying scientific studies for scientific inference via statistical inference, and is complementary to classical discussions in philosophy of statistics. We particularly consider the challenges in replicating studies exactly, using the notion of the idealized experiment. We argue against treating reproducibility as an inherently desirable property of scientific results, and in favor of viewing it as a tool to measure distance between an original study and its replications. To sensibly study the implications of replicability and results reproducibility on inference, such a measure of replication distance is needed. We present an effort to delineate such a framework here, addressing some challenges in capturing the components of scientific studies while identifying others as ongoing issues. We illustrate our measure of replication distance by simulations using a toy example. Rather than replications, we present purposefully planned modifications as an appropriate tool to inform scientific inquiry. Our ability to measure replication distance serves scientists in their search for replication-ready studies. We believe that likelihood-based and evidential approaches may play a critical role towards building a statistics that effectively serves the practical needs of science.
Open AccessStatistics in service of metascience: Measuring replication distance with reproducibility rate
Motivated by the recent putative reproducibility crisis, we discuss the relationship between replicability of scientific studies, reproducibility of results obtained in these replications, and the philosophy of statistics. Our approach focuses on challenges in specifying scientific studies for scientific inference via statistical inference, and is complementary to classical discussions in philosophy of statistics. We particularly consider the challenges in replicating studies exactly, using the notion of the idealized experiment. We argue against treating reproducibility as an inherently desirable property of scientific results, and in favor of viewing it as a tool to measure distance between an original study and its replications. To sensibly study the implications of replicability and results reproducibility on inference, such a measure of replication distance is needed. We present an effort to delineate such a framework here, addressing some challenges in capturing the components of scientific studies while identifying others as ongoing issues. We illustrate our measure of replication distance by simulations using a toy example. Rather than replications, we present purposefully planned modifications as an appropriate tool to inform scientific inquiry. Our ability to measure replication distance serves scientists in their search for replication-ready studies. We believe that likelihood-based and evidential approaches may play a critical role towards building a statistics that effectively serves the practical needs of science.
- Open Access
- 24 July 2024
- Published by The Royal Society in Royal Society Open Science
- Vol. 11 (7) , 240125
Subjective evidence evaluation survey for many-analysts studies
Many-analysts studies explore how well an empirical claim withstands plausible alternative analyses of the same dataset by multiple, independent analysis teams. Conclusions from these studies typically rely on a single outcome metric (e.g. effect size) provided by each analysis team. Although informative about the range of plausible effects in a dataset, a single effect size from each team does not provide a complete, nuanced understanding of how analysis choices are related to the outcome. We used the Delphi consensus technique with input from 37 experts to develop an 18-item subjective evidence evaluation survey (SEES) to evaluate how each analysis team views the methodological appropriateness of the research design and the strength of evidence for the hypothesis. We illustrate the usefulness of the SEES in providing richer evidence assessment with pilot data from a previous many-analysts study.
- 24 July 2024
- Published by The Royal Society in Royal Society Open Science
- Vol. 11 (7) , 240125
Open AccessSubjective evidence evaluation survey for many-analysts studies
Many-analysts studies explore how well an empirical claim withstands plausible alternative analyses of the same dataset by multiple, independent analysis teams. Conclusions from these studies typically rely on a single outcome metric (e.g. effect size) provided by each analysis team. Although informative about the range of plausible effects in a dataset, a single effect size from each team does not provide a complete, nuanced understanding of how analysis choices are related to the outcome. We used the Delphi consensus technique with input from 37 experts to develop an 18-item subjective evidence evaluation survey (SEES) to evaluate how each analysis team views the methodological appropriateness of the research design and the strength of evidence for the hypothesis. We illustrate the usefulness of the SEES in providing richer evidence assessment with pilot data from a previous many-analysts study.
- 5 February 2024
- Published by Cambridge University Press (CUP) in Behavioral and Brain Sciences
- Vol. 47, e38
There are no shortcuts to theory
Almaatouq et al. claim that the integrative experiment design can help “develop a reliable, cohesive, and cumulative theoretical understanding.” I will contest this claim by challenging three underlying assumptions about the nature of scientific theories. I propose that the integrative experiment design should be viewed as an exploratory framework rather than a means to build or evaluate theories.
- 5 February 2024
- Published by Cambridge University Press (CUP) in Behavioral and Brain Sciences
- Vol. 47, e38
There are no shortcuts to theory
Almaatouq et al. claim that the integrative experiment design can help “develop a reliable, cohesive, and cumulative theoretical understanding.” I will contest this claim by challenging three underlying assumptions about the nature of scientific theories. I propose that the integrative experiment design should be viewed as an exploratory framework rather than a means to build or evaluate theories.
- Open Access
- 22 January 2024
- Published by Center for Open Science in PsyArXiv
Subjective Evidence Evaluation Survey For Many-Analysts Studies
Many-analysts studies explore how well an empirical claim withstands plausible alternative analyses of the same data set by multiple, independent analysis teams. Conclusions from these studies typically rely on a single outcome metric (e.g., effect size) provided by each analysis team. Although informative about the range of plausible effects in a data set, a single effect size from each team does not provide a complete, nuanced understanding of how analysis choices are related to the outcome. We used the Delphi consensus technique with input from 37 experts to develop an 18-item Subjective Evidence Evaluation Survey (SEES) to evaluate how each analysis team views the methodological appropriateness of the research design and the strength of evidence for the hypothesis. We illustrate the usefulness of the SEES in providing richer evidence assessment with pilot data from a previous many-analysts study.
- 22 January 2024
- Published by Center for Open Science in PsyArXiv
Open AccessSubjective Evidence Evaluation Survey For Many-Analysts Studies
Many-analysts studies explore how well an empirical claim withstands plausible alternative analyses of the same data set by multiple, independent analysis teams. Conclusions from these studies typically rely on a single outcome metric (e.g., effect size) provided by each analysis team. Although informative about the range of plausible effects in a data set, a single effect size from each team does not provide a complete, nuanced understanding of how analysis choices are related to the outcome. We used the Delphi consensus technique with input from 37 experts to develop an 18-item Subjective Evidence Evaluation Survey (SEES) to evaluate how each analysis team views the methodological appropriateness of the research design and the strength of evidence for the hypothesis. We illustrate the usefulness of the SEES in providing richer evidence assessment with pilot data from a previous many-analysts study.
- Open Access
- 20 November 2023
- Published by Center for Open Science in PsyArXiv
Causal claims about scientific rigor require rigorous causal evidence.
Protzko et al. (2023) claim that “High replicability of newly discovered social-behavioral findings is achievable.” They argue that the 86% rate of replication observed in their replication studies is due to “rigor-enhancing practices” such as confirmatory tests, large sample sizes, preregistration and methodological transparency. These findings promise hope as concerns over low rates of replication have plagued the social sciences for more than a decade. Unfortunately, the observational design of the study does not support its key causal claim. Instead, inference relies on a post hoc comparison of a tenuous metric of replicability to past research that relied on incommensurable metrics and sampling frames.
- 20 November 2023
- Published by Center for Open Science in PsyArXiv
Open AccessCausal claims about scientific rigor require rigorous causal evidence.
Protzko et al. (2023) claim that “High replicability of newly discovered social-behavioral findings is achievable.” They argue that the 86% rate of replication observed in their replication studies is due to “rigor-enhancing practices” such as confirmatory tests, large sample sizes, preregistration and methodological transparency. These findings promise hope as concerns over low rates of replication have plagued the social sciences for more than a decade. Unfortunately, the observational design of the study does not support its key causal claim. Instead, inference relies on a post hoc comparison of a tenuous metric of replicability to past research that relied on incommensurable metrics and sampling frames.