S. Natasha Beretvas
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
S. Natasha Beretvas has a total of 18 co-authors. Here are the top co-authors by number of shared publications:
- Wim Van Den Noortgate
Katholieke Universiteit Leuven
3 shared publications - Belén Fernández-Castilla
Universidad Nacional de Educación a Distancia
2 shared publications - Brian Tinnell Keller
Department of Educational, School, and Counseling Psychology, University of Missouri
2 shared publications - Jeanne H. Freeland-Graves
Department of Nutritional Sciences, University of Texas at Austin, Austin, TX 78712, USA
2 shared publications - Jihyun Lee
University of North Texas
2 shared publications - John M. Ferron
Department of Educational and Psychological Studies, University of South Florida, Tampa, USA
2 shared publications
Latest Publications
- 26 May 2026
- Published by Taylor & Francis in Multivariate Behavioral Research
- Vol. 61 (5) , 1-25
Exploring the Use of Multiple Imputation for Handling Missing Covariates in Meta-Regression with Dependent Effect Sizes
Meta-analysis synthesizes empirical evidence from multiple studies that examine comparable research topics to provide a summary and explore heterogeneity in the findings of the studies. Meta-analyt...
- 26 May 2026
- Published by Taylor & Francis in Multivariate Behavioral Research
- Vol. 61 (5) , 1-25
Exploring the Use of Multiple Imputation for Handling Missing Covariates in Meta-Regression with Dependent Effect Sizes
Meta-analysis synthesizes empirical evidence from multiple studies that examine comparable research topics to provide a summary and explore heterogeneity in the findings of the studies. Meta-analyt...
- Open Access
- 22 May 2026
- Published by Center for Open Science in PsyArXiv
Exploring the Use of Multiple Imputation for Handling Missing Covariates in Meta-regression with Dependent Effect Sizes
Meta-analysts frequently encounter missing covariate values, which can complicate valid estimation of meta-regression models. In practice, missing data are managed often through ad hoc deletion approaches, which can reduce the validity of statistical inferences. More advanced missing data handling approaches such as multiple imputation (MI) remain underutilized, particularly in meta-analyses with dependent effect sizes within studies. This study expands the use of MI techniques for handling missing covariates in such contexts. Specifically, this study introduces adapted multilevel MI approaches that are expected to better accommodate the structure of meta-analytic data with dependent effect sizes and associated model. The study presents Monte Carlo computer simulations that compare the performance of different MI techniques including single-level and multilevel agnostic MI methods as well as multilevel substantive model-based MI and ad hoc deletion approaches. The results generally supported the use of the MI approach over deletion approaches when the dependent structure is well specified in the imputation procedure. This study demonstrates the feasibility of the use of MI techniques and underscores the importance of incorporating the hierarchical structure into the meta-regression model when analyzing dependent effect sizes with missing covariate values. Implications and directions for future research are discussed.
- 22 May 2026
- Published by Center for Open Science in PsyArXiv
Open AccessExploring the Use of Multiple Imputation for Handling Missing Covariates in Meta-regression with Dependent Effect Sizes
Meta-analysts frequently encounter missing covariate values, which can complicate valid estimation of meta-regression models. In practice, missing data are managed often through ad hoc deletion approaches, which can reduce the validity of statistical inferences. More advanced missing data handling approaches such as multiple imputation (MI) remain underutilized, particularly in meta-analyses with dependent effect sizes within studies. This study expands the use of MI techniques for handling missing covariates in such contexts. Specifically, this study introduces adapted multilevel MI approaches that are expected to better accommodate the structure of meta-analytic data with dependent effect sizes and associated model. The study presents Monte Carlo computer simulations that compare the performance of different MI techniques including single-level and multilevel agnostic MI methods as well as multilevel substantive model-based MI and ad hoc deletion approaches. The results generally supported the use of the MI approach over deletion approaches when the dependent structure is well specified in the imputation procedure. This study demonstrates the feasibility of the use of MI techniques and underscores the importance of incorporating the hierarchical structure into the meta-regression model when analyzing dependent effect sizes with missing covariate values. Implications and directions for future research are discussed.
- Open Access
- 24 September 2021
- Published by Center for Open Science in MetaArXiv
Cluster Wild Bootstrapping to Handle Dependent Effect Sizes in Meta-Analysis with a Small Number of Studies
The most common and well-known meta-regression models work under the assumption that there is only one effect size estimate per study and that the estimates are independent. However, meta-analytic reviews of social science research often include multiple effect size estimates per primary study, leading to dependence in the estimates. Some meta-analyses also include multiple studies conducted by the same lab or investigator, creating another potential source of dependence. An increasingly popular method to handle dependence is robust variance estimation (RVE), but this method can result in inflated Type I error rates when the number of studies is small. Small-sample correction methods for RVE have been shown to control Type I error rates adequately but may be overly conservative, especially for tests of multiple-contrast hypotheses. We evaluated an alternative method for handling dependence, cluster wild bootstrapping, which has been examined in the econometrics literature but not in the context of meta-analysis. Results from two simulation studies indicate that cluster wild bootstrapping maintains adequate Type I error rates and provides more power than extant small sample correction methods, particularly for multiple-contrast hypothesis tests. We recommend using cluster wild bootstrapping to conduct hypothesis tests for meta-analyses with a small number of studies. We have also created an R package that implements such tests.
- 24 September 2021
- Published by Center for Open Science in MetaArXiv
Open AccessCluster Wild Bootstrapping to Handle Dependent Effect Sizes in Meta-Analysis with a Small Number of Studies
The most common and well-known meta-regression models work under the assumption that there is only one effect size estimate per study and that the estimates are independent. However, meta-analytic reviews of social science research often include multiple effect size estimates per primary study, leading to dependence in the estimates. Some meta-analyses also include multiple studies conducted by the same lab or investigator, creating another potential source of dependence. An increasingly popular method to handle dependence is robust variance estimation (RVE), but this method can result in inflated Type I error rates when the number of studies is small. Small-sample correction methods for RVE have been shown to control Type I error rates adequately but may be overly conservative, especially for tests of multiple-contrast hypotheses. We evaluated an alternative method for handling dependence, cluster wild bootstrapping, which has been examined in the econometrics literature but not in the context of meta-analysis. Results from two simulation studies indicate that cluster wild bootstrapping maintains adequate Type I error rates and provides more power than extant small sample correction methods, particularly for multiple-contrast hypothesis tests. We recommend using cluster wild bootstrapping to conduct hypothesis tests for meta-analyses with a small number of studies. We have also created an R package that implements such tests.
- Open Access
- 30 September 2020
- Published by Taylor & Francis in Multivariate Behavioral Research
- Vol. 57 (2-3) , 298-317
Multilevel Meta-Analysis of Individual Participant Data of Single-Case Experimental Designs: One-Stage versus Two-Stage Methods
To conduct a multilevel meta-analysis of multiple single-case experimental design (SCED) studies, the individual participant data (IPD) can be analyzed in one or two stages. In the one-stage approach, a multilevel model is estimated based on the raw data. In the two-stage approach, an effect size is calculated for each participant and these effect sizes and their sampling variances are subsequently combined to estimate a meta-analytic multilevel model. The multilevel model in the two-stage approach has fewer parameters to estimate, in exchange for the reduction of information of the raw data to effect sizes. In this paper we explore how the one-stage and two-stage IPD approaches can be applied in the context of meta-analysis of single-case designs. Both approaches are compared for several single-case designs of increasing complexity. Through a simulation study we show that the two-stage approach obtains better convergence rates for more complex models, but that model estimation does not necessarily converge at a faster speed. The point estimates of the fixed effects are unbiased for both approaches across all models, as such confirming results from methodological research on IPD meta-analysis of group-comparison designs. In light of these results, we discuss the implementation of both methods in R.
- 30 September 2020
- Published by Taylor & Francis in Multivariate Behavioral Research
- Vol. 57 (2-3) , 298-317
Open AccessMultilevel Meta-Analysis of Individual Participant Data of Single-Case Experimental Designs: One-Stage versus Two-Stage Methods
To conduct a multilevel meta-analysis of multiple single-case experimental design (SCED) studies, the individual participant data (IPD) can be analyzed in one or two stages. In the one-stage approach, a multilevel model is estimated based on the raw data. In the two-stage approach, an effect size is calculated for each participant and these effect sizes and their sampling variances are subsequently combined to estimate a meta-analytic multilevel model. The multilevel model in the two-stage approach has fewer parameters to estimate, in exchange for the reduction of information of the raw data to effect sizes. In this paper we explore how the one-stage and two-stage IPD approaches can be applied in the context of meta-analysis of single-case designs. Both approaches are compared for several single-case designs of increasing complexity. Through a simulation study we show that the two-stage approach obtains better convergence rates for more complex models, but that model estimation does not necessarily converge at a faster speed. The point estimates of the fixed effects are unbiased for both approaches across all models, as such confirming results from methodological research on IPD meta-analysis of group-comparison designs. In light of these results, we discuss the implementation of both methods in R.
- Open Access
- 4 March 2019
- Published by Taylor & Francis in The Journal of Experimental Education
- Vol. 88 (4) , 698-710
Brief Research Report: Bayesian Versus REML Estimations With Noninformative Priors in Multilevel Single-Case Data
Recently, researchers have used multilevel models for estimating intervention effects in single-case experiments that include replications across participants (e.g., multiple baseline designs) or for combining results across multiple single-case studies. Researchers estimating these multilevel models have primarily relied on restricted maximum likelihood (REML) techniques, but Bayesian approaches have also been suggested. The purpose of this Monte Carlo simulation study was to examine the impact of estimation method (REML versus Bayesian with noninformative priors) on the estimation of treatment effects (relative bias, root mean square error) and on the inferences about those effects (interval coverage) for autocorrelated multiple-baseline data. Simulated conditions varied with regard to the number of participants, series length, and distribution of the variance within and across participants. REML and Bayesian estimation led to estimates of the fixed effects that showed little to no bias but that differentially impacted the inferences about the fixed effects and the estimates of the variances. Implications for applied researchers and methodologists are discussed.
- 4 March 2019
- Published by Taylor & Francis in The Journal of Experimental Education
- Vol. 88 (4) , 698-710
Open AccessBrief Research Report: Bayesian Versus REML Estimations With Noninformative Priors in Multilevel Single-Case Data
Recently, researchers have used multilevel models for estimating intervention effects in single-case experiments that include replications across participants (e.g., multiple baseline designs) or for combining results across multiple single-case studies. Researchers estimating these multilevel models have primarily relied on restricted maximum likelihood (REML) techniques, but Bayesian approaches have also been suggested. The purpose of this Monte Carlo simulation study was to examine the impact of estimation method (REML versus Bayesian with noninformative priors) on the estimation of treatment effects (relative bias, root mean square error) and on the inferences about those effects (interval coverage) for autocorrelated multiple-baseline data. Simulated conditions varied with regard to the number of participants, series length, and distribution of the variance within and across participants. REML and Bayesian estimation led to estimates of the fixed effects that showed little to no bias but that differentially impacted the inferences about the fixed effects and the estimates of the variances. Implications for applied researchers and methodologists are discussed.
- Open Access
- 3 March 2019
- Published by Center for Open Science in PsyArXiv
The Application of Meta-Analytic Models with Multiple Random Effects: A Systematic Review
In meta-analysis, study participants are nested within studies, leading to a multilevel data structure. The traditional random effects model can be considered as a model with a random study effect, but additional random effects can be added in order to account for dependent effects sizes within or across studies. The goal of this systematic review is three-fold. First, we will describe how multilevel models with multiple random effects (i.e., hierarchical three-, four-, five-level models or cross-classified random effects models) are applied in meta-analysis. Second, we will illustrate how in some specific three-level meta-analyses, a more sophisticated model could have been used to deal with additional dependencies in the data. Third and last, we will describe the distribution of the characteristics of three-level meta-analyses (e.g., distribution of the number of outcomes across studies or which dependencies are typically modeled) so that future simulation studies can simulate more realistic conditions. Results showed that four- or five-level or cross-classified random effects models are not often used although they might account better for the meta-analytic data structure of the analyzed datasets. Also, we have found that the simulation studies done on multilevel meta-analysis with multiple random factors could have used more realistic simulation factor conditions. The implications of these results are discussed and further suggestions are given.
- 3 March 2019
- Published by Center for Open Science in PsyArXiv
Open AccessThe Application of Meta-Analytic Models with Multiple Random Effects: A Systematic Review
In meta-analysis, study participants are nested within studies, leading to a multilevel data structure. The traditional random effects model can be considered as a model with a random study effect, but additional random effects can be added in order to account for dependent effects sizes within or across studies. The goal of this systematic review is three-fold. First, we will describe how multilevel models with multiple random effects (i.e., hierarchical three-, four-, five-level models or cross-classified random effects models) are applied in meta-analysis. Second, we will illustrate how in some specific three-level meta-analyses, a more sophisticated model could have been used to deal with additional dependencies in the data. Third and last, we will describe the distribution of the characteristics of three-level meta-analyses (e.g., distribution of the number of outcomes across studies or which dependencies are typically modeled) so that future simulation studies can simulate more realistic conditions. Results showed that four- or five-level or cross-classified random effects models are not often used although they might account better for the meta-analytic data structure of the analyzed datasets. Also, we have found that the simulation studies done on multilevel meta-analysis with multiple random factors could have used more realistic simulation factor conditions. The implications of these results are discussed and further suggestions are given.
- 5 December 2018
- Published by Taylor & Francis in Leukemia & Lymphoma
- Vol. 60 (6) , 1548-1556
Quantifying the association between acute leukemia and serum zinc, copper, and selenium: a meta-analysis
Acute leukemia is a clonal malignant disorder that occurs when immature blast cells accumulate in bone marrow. Zinc (Zn) and copper (Cu) are related to normal lymphocyte maturation and immune function regulation. Selenium (Se) is protective against oxidative damage. The aim of this meta-analysis is to statistically synthesize results from studies that have investigated the levels of Zn, Cu, and Se in acute leukemia patients. The effect size, delta, was used to standardize the raw data. The robust variance estimation (RVE) method was performed to measure the pooled effect size and variance. Results suggest significant negative differences for levels of serum Zn (p < .05, delta = −1.21; 95% CI, −2.13-−0.28) and Se (p < .05, delta = −1.84; 95% CI, −3.39-−0.29) and significantly positive differences between serum Cu levels (p < .01, delta = 1.94; 95% CI, 1.02–2.87) in acute leukemia, as compared to the controls.
- 5 December 2018
- Published by Taylor & Francis in Leukemia & Lymphoma
- Vol. 60 (6) , 1548-1556
Quantifying the association between acute leukemia and serum zinc, copper, and selenium: a meta-analysis
Acute leukemia is a clonal malignant disorder that occurs when immature blast cells accumulate in bone marrow. Zinc (Zn) and copper (Cu) are related to normal lymphocyte maturation and immune function regulation. Selenium (Se) is protective against oxidative damage. The aim of this meta-analysis is to statistically synthesize results from studies that have investigated the levels of Zn, Cu, and Se in acute leukemia patients. The effect size, delta, was used to standardize the raw data. The robust variance estimation (RVE) method was performed to measure the pooled effect size and variance. Results suggest significant negative differences for levels of serum Zn (p < .05, delta = −1.21; 95% CI, −2.13-−0.28) and Se (p < .05, delta = −1.84; 95% CI, −3.39-−0.29) and significantly positive differences between serum Cu levels (p < .01, delta = 1.94; 95% CI, 1.02–2.87) in acute leukemia, as compared to the controls.
- Open Access
- 26 June 2018
- Published by Wiley in International Journal of Dentistry
- Vol. 2018, 1-14
Zinc, Copper, and Iron in Oral Submucous Fibrosis: A Meta-Analysis
Oral submucous fibrosis (OSF) is a potentially malignant disorder which causes fibrosis and inflammation of the oral mucosa. Studies have reported altered levels of trace elements in oral submucous fibrosis subjects, but findings have been inconsistent. The objective of this research is to perform a meta-analysis to summarize studies that report zinc (Zn), copper (Cu), and iron (Fe) in patients, with and without OSF. A literature search of Embase, PubMed, Cochrane Library, and Web of Science electronic databases was conducted for studies up to January 2017. A total of 34 reports met the inclusion criteria. The standardized mean difference was utilized as the effect size. The robust variance estimation method was chosen to handle dependency of multiple related outcomes in meta-analysis. There was a significant increase in the levels of Cu (effect size = 1.17,value < 0.05, 95% confidence interval (CI): 0.164–2.171) and a significant decrease in levels of Zn (effect size = −1.95,value < 0.05, 95% CI: −3.524 to −0.367) and Fe (effect size = −2.77,value < 0.01, 95% CI: −4.126 to −1.406) in OSF patients. The estimation of Zn, Cu, and Fe levels may serve as additional biomarkers in the diagnosis and prognosis of OSF along with the clinical features.- 26 June 2018
- Published by Wiley in International Journal of Dentistry
- Vol. 2018, 1-14
Open AccessZinc, Copper, and Iron in Oral Submucous Fibrosis: A Meta-Analysis
Oral submucous fibrosis (OSF) is a potentially malignant disorder which causes fibrosis and inflammation of the oral mucosa. Studies have reported altered levels of trace elements in oral submucous fibrosis subjects, but findings have been inconsistent. The objective of this research is to perform a meta-analysis to summarize studies that report zinc (Zn), copper (Cu), and iron (Fe) in patients, with and without OSF. A literature search of Embase, PubMed, Cochrane Library, and Web of Science electronic databases was conducted for studies up to January 2017. A total of 34 reports met the inclusion criteria. The standardized mean difference was utilized as the effect size. The robust variance estimation method was chosen to handle dependency of multiple related outcomes in meta-analysis. There was a significant increase in the levels of Cu (effect size = 1.17,value < 0.05, 95% confidence interval (CI): 0.164–2.171) and a significant decrease in levels of Zn (effect size = −1.95,value < 0.05, 95% CI: −3.524 to −0.367) and Fe (effect size = −2.77,value < 0.01, 95% CI: −4.126 to −1.406) in OSF patients. The estimation of Zn, Cu, and Fe levels may serve as additional biomarkers in the diagnosis and prognosis of OSF along with the clinical features.