Erkan Ozge Buzbas
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Erkan Ozge Buzbas has a total of 8 co-authors. Here are the top co-authors by number of shared publications:
- Berna Devezer
University of Idaho
4 shared publications - Noah A. Rosenberg
Department of Biology, Stanford University, Stanford, USA
2 shared publications Bert Baumgaertner
Department of Politics and Philosophy, University of Idaho, Moscow, ID 83844, USA
1 shared publication- Bert O. Baumgaertner
University of Idaho
1 shared publication - Danielle Navarro
School of Psychology, University of New South Wales
1 shared publication Jake M. Ferguson
Center for Modeling Complex Interactions, University of Idaho, United States
1 shared publication
Latest Publications
- Open Access
Statistics in Service of Metascience: Measuring Replication Distance with Reproducibility Rate
Motivated by the recent putative reproducibility crisis, we discuss the relationship between the replicability of scientific studies, the reproducibility of results obtained in these replications, and the philosophy of statistics. Our approach focuses on challenges in specifying scientific studies for scientific inference via statistical inference and is complementary to classical discussions in the philosophy of statistics. We particularly consider the challenges in replicating studies exactly, using the notion of the idealized experiment. We argue against treating reproducibility as an inherently desirable property of scientific results, and in favor of viewing it as a tool to measure the distance between an original study and its replications. To sensibly study the implications of replicability and results reproducibility on inference, such a measure of replication distance is needed. We present an effort to delineate such a framework here, addressing some challenges in capturing the components of scientific studies while identifying others as ongoing issues. We illustrate our measure of replication distance by simulations using a toy example. Rather than replications, we present purposefully planned modifications as an appropriate tool to inform scientific inquiry. Our ability to measure replication distance serves scientists in their search for replication-ready studies. We believe that likelihood-based and evidential approaches may play a critical role towards building statistics that effectively serve the practical needs of science.
Open AccessStatistics in Service of Metascience: Measuring Replication Distance with Reproducibility Rate
Motivated by the recent putative reproducibility crisis, we discuss the relationship between the replicability of scientific studies, the reproducibility of results obtained in these replications, and the philosophy of statistics. Our approach focuses on challenges in specifying scientific studies for scientific inference via statistical inference and is complementary to classical discussions in the philosophy of statistics. We particularly consider the challenges in replicating studies exactly, using the notion of the idealized experiment. We argue against treating reproducibility as an inherently desirable property of scientific results, and in favor of viewing it as a tool to measure the distance between an original study and its replications. To sensibly study the implications of replicability and results reproducibility on inference, such a measure of replication distance is needed. We present an effort to delineate such a framework here, addressing some challenges in capturing the components of scientific studies while identifying others as ongoing issues. We illustrate our measure of replication distance by simulations using a toy example. Rather than replications, we present purposefully planned modifications as an appropriate tool to inform scientific inquiry. Our ability to measure replication distance serves scientists in their search for replication-ready studies. We believe that likelihood-based and evidential approaches may play a critical role towards building statistics that effectively serve the practical needs of science.
- 15 March 2023
- Published by The Royal Society in Royal Society Open Science
- Vol. 10 (3) , 221042
The logical structure of experiments lays the foundation for a theory of reproducibility
The scientific reform movement has proposed openness as a potential remedy to the putative reproducibility or replication crisis. However, the conceptual relationship among openness, replication experiments and results reproducibility has been obscure. We analyse the logical structure of experiments, define the mathematical notion of idealized experiment and use this notion to advance a theory of reproducibility. Idealized experiments clearly delineate the concepts of replication and results reproducibility, and capture key differences with precision, allowing us to study the relationship among them. We show how results reproducibility varies as a function of the elements of an idealized experiment, the true data-generating mechanism, and the closeness of the replication experiment to an original experiment. We clarify how openness of experiments is related to designing informative replication experiments and to obtaining reproducible results. With formal backing and evidence, we argue that the current ‘crisis’ reflects inadequate attention to a theoretical understanding of results reproducibility.
- 15 March 2023
- Published by The Royal Society in Royal Society Open Science
- Vol. 10 (3) , 221042
The logical structure of experiments lays the foundation for a theory of reproducibility
The scientific reform movement has proposed openness as a potential remedy to the putative reproducibility or replication crisis. However, the conceptual relationship among openness, replication experiments and results reproducibility has been obscure. We analyse the logical structure of experiments, define the mathematical notion of idealized experiment and use this notion to advance a theory of reproducibility. Idealized experiments clearly delineate the concepts of replication and results reproducibility, and capture key differences with precision, allowing us to study the relationship among them. We show how results reproducibility varies as a function of the elements of an idealized experiment, the true data-generating mechanism, and the closeness of the replication experiment to an original experiment. We clarify how openness of experiments is related to designing informative replication experiments and to obtaining reproducible results. With formal backing and evidence, we argue that the current ‘crisis’ reflects inadequate attention to a theoretical understanding of results reproducibility.
- Open Access
The logical structure of experiments lays the foundation for a theory of reproducibility
The scientific reform movement has proposed openness as a potential remedy to the putative reproducibility or replication crisis. However, the conceptual relationship between openness, replication experiments, and results reproducibility has been obscure. We analyze the logical structure of experiments, define the mathematical notion of idealized experiment, and use this notion to advance a theory of reproducibility. Idealized experiments clearly delineate the concepts of replication and results reproducibility, and capture key differences with precision, allowing us to study the relationship among them. We show how results reproducibility varies as a function of: the elements of an idealized experiment, the true data generating mechanism, and the closeness of the replication experiment to an original experiment. We clarify how openness of experiments is related to designing informative replication experiments and to obtaining reproducible results. With formal backing and evidence, we argue that the current “crisis” reflects inadequate attention to a theoretical understanding of results reproducibility.
Open AccessThe logical structure of experiments lays the foundation for a theory of reproducibility
The scientific reform movement has proposed openness as a potential remedy to the putative reproducibility or replication crisis. However, the conceptual relationship between openness, replication experiments, and results reproducibility has been obscure. We analyze the logical structure of experiments, define the mathematical notion of idealized experiment, and use this notion to advance a theory of reproducibility. Idealized experiments clearly delineate the concepts of replication and results reproducibility, and capture key differences with precision, allowing us to study the relationship among them. We show how results reproducibility varies as a function of: the elements of an idealized experiment, the true data generating mechanism, and the closeness of the replication experiment to an original experiment. We clarify how openness of experiments is related to designing informative replication experiments and to obtaining reproducible results. With formal backing and evidence, we argue that the current “crisis” reflects inadequate attention to a theoretical understanding of results reproducibility.
- Open Access
The case for formal methodology in scientific reform
Current attempts at methodological reform in sciences come in response to an overall lack of rigor in methodological and scientific practices in experimental sciences. However, most methodological reform attempts suffer from similar mistakes and over-generalizations to the ones they aim to address. We argue that this can be attributed in part to lack of formalism and first principles. Considering the costs of allowing false claims to become canonized, we argue for formal statistical rigor and scientific nuance in methodological reform. To attain this rigor and nuance, we propose a five-step formal approach for solving methodological problems. To illustrate the use and benefits of such formalism, we present a formal statistical analysis of three popular claims in the metascientific literature: (a) that reproducibility is the cornerstone of science; (b) that data must not be used twice in any analysis; and (c) that exploratory projects imply poor statistical practice. We show how our formal approach can inform and shape debates about such methodological claims.
Open AccessThe case for formal methodology in scientific reform
Current attempts at methodological reform in sciences come in response to an overall lack of rigor in methodological and scientific practices in experimental sciences. However, most methodological reform attempts suffer from similar mistakes and over-generalizations to the ones they aim to address. We argue that this can be attributed in part to lack of formalism and first principles. Considering the costs of allowing false claims to become canonized, we argue for formal statistical rigor and scientific nuance in methodological reform. To attain this rigor and nuance, we propose a five-step formal approach for solving methodological problems. To illustrate the use and benefits of such formalism, we present a formal statistical analysis of three popular claims in the metascientific literature: (a) that reproducibility is the cornerstone of science; (b) that data must not be used twice in any analysis; and (c) that exploratory projects imply poor statistical practice. We show how our formal approach can inform and shape debates about such methodological claims.
- 16 October 2019
- Published by Springer Nature in Computational Brain & Behavior
- Vol. 2 (3-4) , 197-199
Need of Mathematical Formalism in Proposals for Robust Modeling
- 16 October 2019
- Published by Springer Nature in Computational Brain & Behavior
- Vol. 2 (3-4) , 197-199
Need of Mathematical Formalism in Proposals for Robust Modeling
- 1 July 2018
- Published by Elsevier in Theoretical Population Biology
- Vol. 122, 149-157
Inference on admixture fractions in a mechanistic model of recurrent admixture
- 1 July 2018
- Published by Elsevier in Theoretical Population Biology
- Vol. 122, 149-157
Inference on admixture fractions in a mechanistic model of recurrent admixture
- 1 February 2015
- Published by Elsevier in Theoretical Population Biology
- Vol. 99, 31-42
AABC: Approximate approximate Bayesian computation for inference in population-genetic models
- 1 February 2015
- Published by Elsevier in Theoretical Population Biology
- Vol. 99, 31-42
AABC: Approximate approximate Bayesian computation for inference in population-genetic models
- Open Access
AABC: approximate approximate Bayesian computation when simulating a large number of data sets is computationally infeasible
Approximate Bayesian computation (ABC) methods perform inference on model-specific parameters of mechanistically motivated parametric statistical models when evaluating likelihoods is difficult. Central to the success of ABC methods is computationally inexpensive simulation of data sets from the parametric model of interest. However, when simulating data sets from a model is so computationally expensive that the posterior distribution of parameters cannot be adequately sampled by ABC, inference is not straightforward. We present approximate approximate Bayesian computation" (AABC), a class of methods that extends simulation-based inference by ABC to models in which simulating data is expensive. In AABC, we first simulate a limited number of data sets that is computationally feasible to simulate from the parametric model. We use these data sets as fixed background information to inform a non-mechanistic statistical model that approximates the correct parametric model and enables efficient simulation of a large number of data sets by Bayesian resampling methods. We show that under mild assumptions, the posterior distribution obtained by AABC converges to the posterior distribution obtained by ABC, as the number of data sets simulated from the parametric model and the sample size of the observed data set increase simultaneously. We illustrate the performance of AABC on a population-genetic model of natural selection, as well as on a model of the admixture history of hybrid populations.
Open AccessAABC: approximate approximate Bayesian computation when simulating a large number of data sets is computationally infeasible
Approximate Bayesian computation (ABC) methods perform inference on model-specific parameters of mechanistically motivated parametric statistical models when evaluating likelihoods is difficult. Central to the success of ABC methods is computationally inexpensive simulation of data sets from the parametric model of interest. However, when simulating data sets from a model is so computationally expensive that the posterior distribution of parameters cannot be adequately sampled by ABC, inference is not straightforward. We present approximate approximate Bayesian computation" (AABC), a class of methods that extends simulation-based inference by ABC to models in which simulating data is expensive. In AABC, we first simulate a limited number of data sets that is computationally feasible to simulate from the parametric model. We use these data sets as fixed background information to inform a non-mechanistic statistical model that approximates the correct parametric model and enables efficient simulation of a large number of data sets by Bayesian resampling methods. We show that under mild assumptions, the posterior distribution obtained by AABC converges to the posterior distribution obtained by ABC, as the number of data sets simulated from the parametric model and the sample size of the observed data set increase simultaneously. We illustrate the performance of AABC on a population-genetic model of natural selection, as well as on a model of the admixture history of hybrid populations.