Tom D. Stanley
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Tom D. Stanley has a total of 68 co-authors. Here are the top co-authors by number of shared publications:
- Hristos Doucouliagos
Deakin University
13 shared publications Hristos Doucouliagos
Department of Economics, Deakin University, Victoria, Australia
11 shared publications- František Bartoš
University of Amsterdam
10 shared publications - Eric-Jan Wagenmakers
Department of Psychology, University of Amsterdam, Amsterdam, Netherlands
9 shared publications Maximilian Maier
7 shared publicationsChris Doucouliagos
Department of Economics Deakin University Melbourne Victoria Australia
6 shared publications
Latest Publications
- Open Access
- 12 May 2026
- Published by Wiley in Journal of Economic Surveys
Reporting Guidelines for Meta‐Analysis in Economics—Updated for AI
Meta‐analysis is how science takes stock of its vast research output. The advent of increasingly capable artificial intelligence (AI) promises an unprecedented ability to identify and synthesize relevant research and its findings. In this document, the meta‐analysis of economics research network (MAER‐Net) updates existing Reporting Guidelines to be consistent with community‐driven best practices for the responsible use and disclosure of AI‐assistance in meta‐analysis research. This update is meant to further improve the transparency, replicability, and quality of meta‐analyses by building upon the 2020 and 2013 Reporting Guidelines published by this Journal. Cook et al. (2026) describe the guiding principles behind the update. Future meta‐analyses, whether or not they use AI, are expected to follow these updated guidelines or to be prepared to give reasons if they deviate from them.
- 12 May 2026
- Published by Wiley in Journal of Economic Surveys
Open AccessReporting Guidelines for Meta‐Analysis in Economics—Updated for AI
Meta‐analysis is how science takes stock of its vast research output. The advent of increasingly capable artificial intelligence (AI) promises an unprecedented ability to identify and synthesize relevant research and its findings. In this document, the meta‐analysis of economics research network (MAER‐Net) updates existing Reporting Guidelines to be consistent with community‐driven best practices for the responsible use and disclosure of AI‐assistance in meta‐analysis research. This update is meant to further improve the transparency, replicability, and quality of meta‐analyses by building upon the 2020 and 2013 Reporting Guidelines published by this Journal. Cook et al. (2026) describe the guiding principles behind the update. Future meta‐analyses, whether or not they use AI, are expected to follow these updated guidelines or to be prepared to give reasons if they deviate from them.
- Open Access
- 21 April 2026
- Published by Wiley in Journal of Economic Surveys
Guidance for the Use of AI in the Meta‐Analysis of Economics Research
Meta‐analysis is widely accepted to be the most rigorous and objective approach to the synthesis, interpretation, and understanding of findings from specific areas of empirical economics research. With the advent of increasingly capable generative artificial intelligence and AI's potential to transform the practice of meta‐analysis, the Meta‐Analysis of Economics Research Network (MAER‐Net) has adopted this set of principles. These principles are meant to provide guidance to meta‐researchers, as well as editors and reviewers, in the use of AI in meta‐analysis of economics research. Future meta‐analyses that employ AI are expected to embody these guiding principles and to follow the associated Reporting Guidelines for Meta‐Analysis in Economics ‐ Updated for AI (Cook et al. 2026).
- 21 April 2026
- Published by Wiley in Journal of Economic Surveys
Open AccessGuidance for the Use of AI in the Meta‐Analysis of Economics Research
Meta‐analysis is widely accepted to be the most rigorous and objective approach to the synthesis, interpretation, and understanding of findings from specific areas of empirical economics research. With the advent of increasingly capable generative artificial intelligence and AI's potential to transform the practice of meta‐analysis, the Meta‐Analysis of Economics Research Network (MAER‐Net) has adopted this set of principles. These principles are meant to provide guidance to meta‐researchers, as well as editors and reviewers, in the use of AI in meta‐analysis of economics research. Future meta‐analyses that employ AI are expected to embody these guiding principles and to follow the associated Reporting Guidelines for Meta‐Analysis in Economics ‐ Updated for AI (Cook et al. 2026).
- Open Access
- 12 March 2025
- Published by Center for Open Science in PsyArXiv
Robust Bayesian Meta-Regression: Model-Averaged Moderation Analysis in the Presence of Publication Bias
Meta-regression is an essential meta-analytic tool for investigating sources of heterogeneity and assessing the impact of moderators. However, existing methods for meta-regression have limitations, such as inadequate consideration of model uncertainty and poor performance under publication bias. To overcome these limitations, we extend robustBayesian meta-analysis (RoBMA) to meta-regression (RoBMA-regression). RoBMA-regression allows for moderator analyses while simultaneously taking into account the uncertainty about the presence and impact of other factors (i.e., the main effect, heterogeneity, publication bias, and other potential moderators). The methodology presents a coherent way of assessing the evidence for and against the presence of both continuous and categorical moderators. We further employ a Savage-Dickey density ratio test to quantify the evidence for and against the presence of the effect at different levels of categorical moderators. We illustrate RoBMA-regression in an empirical example and demonstrate its performance in a simulation study. We implemented the methodology in the RoBMA R package. Overall, RoBMA-regression presents researchers with a powerful and flexible tool for conducting robust and informative meta-regression analyses.
- 12 March 2025
- Published by Center for Open Science in PsyArXiv
Open AccessRobust Bayesian Meta-Regression: Model-Averaged Moderation Analysis in the Presence of Publication Bias
Meta-regression is an essential meta-analytic tool for investigating sources of heterogeneity and assessing the impact of moderators. However, existing methods for meta-regression have limitations, such as inadequate consideration of model uncertainty and poor performance under publication bias. To overcome these limitations, we extend robustBayesian meta-analysis (RoBMA) to meta-regression (RoBMA-regression). RoBMA-regression allows for moderator analyses while simultaneously taking into account the uncertainty about the presence and impact of other factors (i.e., the main effect, heterogeneity, publication bias, and other potential moderators). The methodology presents a coherent way of assessing the evidence for and against the presence of both continuous and categorical moderators. We further employ a Savage-Dickey density ratio test to quantify the evidence for and against the presence of the effect at different levels of categorical moderators. We illustrate RoBMA-regression in an empirical example and demonstrate its performance in a simulation study. We implemented the methodology in the RoBMA R package. Overall, RoBMA-regression presents researchers with a powerful and flexible tool for conducting robust and informative meta-regression analyses.
- Open Access
- 17 February 2025
- Published by American Psychological Association (APA) in Psychological Methods
Robust Bayesian meta-regression: Model-averaged moderation analysis in the presence of publication bias.
Meta-regression is an essential meta-analytic tool for investigating sources of heterogeneity and assessing the impact of moderators. However, existing methods for meta-regression have limitations, such as inadequate consideration of model uncertainty and poor performance under publication bias. To overcome these limitations, we extend robust Bayesian meta-analysis (RoBMA) to meta-regression (RoBMA-regression). RoBMA-regression allows for moderator analyses while simultaneously taking into account the uncertainty about the presence and impact of other factors (i.e., the main effect, heterogeneity, publication bias, and other potential moderators). The methodology presents a coherent way of assessing the evidence for and against the presence of both continuous and categorical moderators. We further employ a Savage-Dickey density ratio test to quantify the evidence for and against the presence of the effect at different levels of categorical moderators. We illustrate RoBMA-regression in an empirical example and demonstrate its performance in a simulation study. We implemented the methodology in the RoBMA R package. Overall, RoBMA-regression presents researchers with a powerful and flexible tool for conducting robust and informative meta-regression analyses. (PsycInfo Database Record (c) 2025 APA, all rights reserved).
- 17 February 2025
- Published by American Psychological Association (APA) in Psychological Methods
Open AccessRobust Bayesian meta-regression: Model-averaged moderation analysis in the presence of publication bias.
Meta-regression is an essential meta-analytic tool for investigating sources of heterogeneity and assessing the impact of moderators. However, existing methods for meta-regression have limitations, such as inadequate consideration of model uncertainty and poor performance under publication bias. To overcome these limitations, we extend robust Bayesian meta-analysis (RoBMA) to meta-regression (RoBMA-regression). RoBMA-regression allows for moderator analyses while simultaneously taking into account the uncertainty about the presence and impact of other factors (i.e., the main effect, heterogeneity, publication bias, and other potential moderators). The methodology presents a coherent way of assessing the evidence for and against the presence of both continuous and categorical moderators. We further employ a Savage-Dickey density ratio test to quantify the evidence for and against the presence of the effect at different levels of categorical moderators. We illustrate RoBMA-regression in an empirical example and demonstrate its performance in a simulation study. We implemented the methodology in the RoBMA R package. Overall, RoBMA-regression presents researchers with a powerful and flexible tool for conducting robust and informative meta-regression analyses. (PsycInfo Database Record (c) 2025 APA, all rights reserved).
- Open Access
- 1 January 2025
- Published by Cambridge University Press (CUP) in Research Synthesis Methods
- Vol. 16 (1) , 42-59
Reducing the biases of the conventional meta-analysis of correlations
Conventional meta-analyses (both fixed and random effects) of correlations are biased due to the mechanical relationship between the estimated correlation and its standard error. Simulations that are closely calibrated to match actual research conditions widely seen across correlational studies in psychology corroborate these biases and suggest two solutions: UWLS+3 and HS. UWLS+3 is a simple inverse-variance weighted average (the unrestricted weighted least squares) that adjusts the degrees of freedom and thereby reduces small-sample bias to scientific negligibility. UWLS+3 as well as the Hunter and Schmidt approach (HS) are less biased than conventional random-effects estimates of correlations and Fisher’s z, whether or not there is publication selection bias. However, publication selection bias remains a ubiquitous source of bias and false-positive findings. Despite the relationship between the estimated correlation and its standard error in the absence of selective reporting, the precision-effect test/precision-effect estimate with standard error (PET-PEESE) nearly eradicates publication selection bias. Surprisingly, PET-PEESE keeps the rate of false positives (i.e., type I errors) within their nominal levels under the typical conditions widely seen across psychological research whether there is publication selection bias, or not.
- 1 January 2025
- Published by Cambridge University Press (CUP) in Research Synthesis Methods
- Vol. 16 (1) , 42-59
Open AccessReducing the biases of the conventional meta-analysis of correlations
Conventional meta-analyses (both fixed and random effects) of correlations are biased due to the mechanical relationship between the estimated correlation and its standard error. Simulations that are closely calibrated to match actual research conditions widely seen across correlational studies in psychology corroborate these biases and suggest two solutions: UWLS+3 and HS. UWLS+3 is a simple inverse-variance weighted average (the unrestricted weighted least squares) that adjusts the degrees of freedom and thereby reduces small-sample bias to scientific negligibility. UWLS+3 as well as the Hunter and Schmidt approach (HS) are less biased than conventional random-effects estimates of correlations and Fisher’s z, whether or not there is publication selection bias. However, publication selection bias remains a ubiquitous source of bias and false-positive findings. Despite the relationship between the estimated correlation and its standard error in the absence of selective reporting, the precision-effect test/precision-effect estimate with standard error (PET-PEESE) nearly eradicates publication selection bias. Surprisingly, PET-PEESE keeps the rate of false positives (i.e., type I errors) within their nominal levels under the typical conditions widely seen across psychological research whether there is publication selection bias, or not.
- Open Access
- 1 January 2025
- Published by Elsevier in SSRN Electronic Journal
The Delayed Acceptance of Female Research in Economics
We investigate gender differences in the time taken to accept empirical articles. On average, female-authored economics articles take notably longer to accept. Acceptance delay is nine weeks longer when solo-authored and five weeks longer for all female teams. This gender gap cannot be attributed to differences in author affiliation, research productivity, research quality and novelty. Female-authored articles are of higher quality, as measured by citations, reflecting higher research thresholds for female-authored work. The gender composition of editorial boards does not affect acceptance time for female authors. Nevertheless, this gender gap narrows as female representation in an area of research deepens.
- 1 January 2025
- Published by Elsevier in SSRN Electronic Journal
Open AccessThe Delayed Acceptance of Female Research in Economics
We investigate gender differences in the time taken to accept empirical articles. On average, female-authored economics articles take notably longer to accept. Acceptance delay is nine weeks longer when solo-authored and five weeks longer for all female teams. This gender gap cannot be attributed to differences in author affiliation, research productivity, research quality and novelty. Female-authored articles are of higher quality, as measured by citations, reflecting higher research thresholds for female-authored work. The gender composition of editorial boards does not affect acceptance time for female authors. Nevertheless, this gender gap narrows as female representation in an area of research deepens.
- Open Access
- 20 May 2024
- Published by Wiley in Journal of Economic Surveys
- Vol. 39 (3) , 980-999
Conventional wisdom, meta‐analysis, and research revision in economics
Over the past several decades, meta‐analysis has emerged as a widely accepted tool to understand economics research. Meta‐analyses often challenge the established conventional wisdom of their respective fields. We systematically review a wide range of influential meta‐analyses in economics and compare them to “conventional wisdom.” After correcting for observable biases, the empirical economic effects are typically much closer to zero and sometimes switch signs. Typically, the relative reduction in effect sizes is 45%–60%.
- 20 May 2024
- Published by Wiley in Journal of Economic Surveys
- Vol. 39 (3) , 980-999
Open AccessConventional wisdom, meta‐analysis, and research revision in economics
Over the past several decades, meta‐analysis has emerged as a widely accepted tool to understand economics research. Meta‐analyses often challenge the established conventional wisdom of their respective fields. We systematically review a wide range of influential meta‐analyses in economics and compare them to “conventional wisdom.” After correcting for observable biases, the empirical economic effects are typically much closer to zero and sometimes switch signs. Typically, the relative reduction in effect sizes is 45%–60%.
- 1 April 2024
- Published by American Psychological Association (APA) in Psychological Methods
- Vol. 29 (2) , 407-420
Harnessing the power of excess statistical significance: Weighted and iterative least squares.
We introduce a new meta-analysis estimator, the weighted and iterated least squares (WILS), that greatly reduces publication selection bias (PSB) when selective reporting for statistical significance (SSS) is present. WILS is the simple weighted average that has smaller bias and rates of false positives than conventional meta-analysis estimators, the unrestricted weighted least squares (UWLS), and the weighted average of the adequately powered (WAAP) when there is SSS. As a simple weighted average, it is not vulnerable to violations in publication bias corrections models' assumptions too often seen in application. WILS is based on the novel idea of allowing excess statistical significance (ESS), which is a necessary condition of SSS, to identify when and how to reduce PSB. We show in comparisons with large-scale preregistered replications and in evidence-based simulations that the remaining bias is small. The routine application of WILS in the place of random effects would do much to reduce conventional meta-analysis's notable biases and high rates of false positives. Translational Abstract The credibility of conventional meta-analysis has been a casualty of the 'replication crisis.' Selective publication bias (i.e., reporting bias, publication bias and QRP) is often present in the reported research record and is the reason that preregistered replications find much smaller effects. Several methods are available to correct or reduce these potential biases, all have limitations, and none outperform the others widely across all likely circumstances found in psychological research. We present a new method, the weighted and iterated least squares (WILS), that greatly reduces selective publication bias when present at surprisingly little statistical cost. WILS is a simple weighted average that makes no assumption about the nature of the selection process or how publication selection can best be modelled. It works by identifying excess statistical significance (ESS) observed in meta-analyses and then removing those studies most responsible. Thus, WILS is widely applicable, often reducing publication selection bias to scientific insignificance.
- 1 April 2024
- Published by American Psychological Association (APA) in Psychological Methods
- Vol. 29 (2) , 407-420
Harnessing the power of excess statistical significance: Weighted and iterative least squares.
We introduce a new meta-analysis estimator, the weighted and iterated least squares (WILS), that greatly reduces publication selection bias (PSB) when selective reporting for statistical significance (SSS) is present. WILS is the simple weighted average that has smaller bias and rates of false positives than conventional meta-analysis estimators, the unrestricted weighted least squares (UWLS), and the weighted average of the adequately powered (WAAP) when there is SSS. As a simple weighted average, it is not vulnerable to violations in publication bias corrections models' assumptions too often seen in application. WILS is based on the novel idea of allowing excess statistical significance (ESS), which is a necessary condition of SSS, to identify when and how to reduce PSB. We show in comparisons with large-scale preregistered replications and in evidence-based simulations that the remaining bias is small. The routine application of WILS in the place of random effects would do much to reduce conventional meta-analysis's notable biases and high rates of false positives. Translational Abstract The credibility of conventional meta-analysis has been a casualty of the 'replication crisis.' Selective publication bias (i.e., reporting bias, publication bias and QRP) is often present in the reported research record and is the reason that preregistered replications find much smaller effects. Several methods are available to correct or reduce these potential biases, all have limitations, and none outperform the others widely across all likely circumstances found in psychological research. We present a new method, the weighted and iterated least squares (WILS), that greatly reduces selective publication bias when present at surprisingly little statistical cost. WILS is a simple weighted average that makes no assumption about the nature of the selection process or how publication selection can best be modelled. It works by identifying excess statistical significance (ESS) observed in meta-analyses and then removing those studies most responsible. Thus, WILS is widely applicable, often reducing publication selection bias to scientific insignificance.
- Open Access
- 21 February 2024
- Published by Wiley in Research Synthesis Methods
- Vol. 15 (4) , 590-602
Estimating the extent of selective reporting: An application to economics
Using a sample of 70,399 published p-values from 192 meta-analyses, we empirically estimate the counterfactual distribution of p-values in the absence of any biases. Comparing observed p-values with counterfactually expected p-values allows us to estimate how many p-values are published as being statistically significant when they should have been published as non-significant. We estimate the extent of selectively reported p-values to range between 57.7% and 71.9% of the significant p-values. The counterfactual p-value distribution also allows us to assess shifts of p-values along the entire distribution of published p-values, revealing that particularly very small p-values (p < 0.001) are unexpectedly abundant in the published literature. Subsample analysis suggests that the extent of selective reporting is reduced in research fields that use experimental designs, analyze microeconomics research questions, and have at least some adequately powered studies.
- 21 February 2024
- Published by Wiley in Research Synthesis Methods
- Vol. 15 (4) , 590-602
Open AccessEstimating the extent of selective reporting: An application to economics
Using a sample of 70,399 published p-values from 192 meta-analyses, we empirically estimate the counterfactual distribution of p-values in the absence of any biases. Comparing observed p-values with counterfactually expected p-values allows us to estimate how many p-values are published as being statistically significant when they should have been published as non-significant. We estimate the extent of selectively reported p-values to range between 57.7% and 71.9% of the significant p-values. The counterfactual p-value distribution also allows us to assess shifts of p-values along the entire distribution of published p-values, revealing that particularly very small p-values (p < 0.001) are unexpectedly abundant in the published literature. Subsample analysis suggests that the extent of selective reporting is reduced in research fields that use experimental designs, analyze microeconomics research questions, and have at least some adequately powered studies.
- Open Access
- 21 February 2024
- Published by The Royal Society in Royal Society Open Science
- Vol. 11 (2) , 231486
Exploring open science practices in behavioural public policy research
In their book ‘Nudge: Improving Decisions About Health, Wealth and Happiness’, Thaler & Sunstein (2009) argue that choice architectures are promising public policy interventions. This research programme motivated the creation of ‘nudge units’, government agencies which aim to apply insights from behavioural science to improve public policy. We closely examine a meta-analysis of the evidence gathered by two of the largest and most influential nudge units (DellaVigna & Linos (2022 Econometrica 90 , 81–116 ( doi:10.3982/ECTA18709 ))) and use statistical techniques to detect reporting biases. Our analysis shows evidence suggestive of selective reporting. We additionally evaluate the public pre-analysis plans from one of the two nudge units (Office of Evaluation Sciences). We identify several instances of excellent practice; however, we also find that the analysis plans and reporting often lack sufficient detail to evaluate (unintentional) reporting biases. We highlight several improvements that would enhance the effectiveness of the pre-analysis plans and reports as a means to combat reporting biases. Our findings and suggestions can further improve the evidence base for policy decisions.
- 21 February 2024
- Published by The Royal Society in Royal Society Open Science
- Vol. 11 (2) , 231486
Open AccessExploring open science practices in behavioural public policy research
In their book ‘Nudge: Improving Decisions About Health, Wealth and Happiness’, Thaler & Sunstein (2009) argue that choice architectures are promising public policy interventions. This research programme motivated the creation of ‘nudge units’, government agencies which aim to apply insights from behavioural science to improve public policy. We closely examine a meta-analysis of the evidence gathered by two of the largest and most influential nudge units (DellaVigna & Linos (2022 Econometrica 90 , 81–116 ( doi:10.3982/ECTA18709 ))) and use statistical techniques to detect reporting biases. Our analysis shows evidence suggestive of selective reporting. We additionally evaluate the public pre-analysis plans from one of the two nudge units (Office of Evaluation Sciences). We identify several instances of excellent practice; however, we also find that the analysis plans and reporting often lack sufficient detail to evaluate (unintentional) reporting biases. We highlight several improvements that would enhance the effectiveness of the pre-analysis plans and reports as a means to combat reporting biases. Our findings and suggestions can further improve the evidence base for policy decisions.