Marta Pellegrini

This meta-review explored the current practices used in education intervention meta-analyses in terms of systematic review procedures and meta-analysis methods. We reviewed 247 meta-analyses on the effects of K–12 school-based interventions on student academic achievement published after 2011. We found that many reviews were mostly consistent with several best practice recommendations for the review stage, including problem formulation, selection, and coding procedures. Reviews rarely preregistered their protocol or shared data, which reduces the transparency and reproducibility of the process. Best practice meta-analysis methods with robust consensus among methodologists were seldom used in our review sample. Recommendations for generating more credible and reproducible findings are provided. We also identify areas in need of more research and guidance, including how to conduct critical appraisal, how to deal with missing covariate data, and disciplined strategies to build meta-regression models.

One of the key goals of meta-analyses is to provide educators with relevant evidence to guide their decisions in practice. This meta-review examined the extent to which education meta-analysts have used strategies that likely increase the relevance, applicability, and accessibility of research to practitioners. We reviewed 103 meta-analyses of school-based academic interventions, coding for: (a) stakeholder engagement in review phases; (b) reporting of study population, setting, and intervention characteristics and testing them as moderators; and (c) accessibility of the findings to a wide audience using effect size metrics and visualizations. We found limited stakeholder involvement. Certain characteristics such as grade level were commonly reported and used to explored heterogeneity, while others, like material and training costs, were rarely considered. Effect size transformations were not common, and traditional forest plots were the most prevalent visualization method. We propose future research directions to improve the relevance, applicability, and accessibility of meta-analysis findings for educational practice.

Randomized controlled trials (RCTs) are increasingly used in sociology to generate causal evidence. While the uptake of RCTs in sociology has lagged behind other disciplines, recent growth signals a shift toward what some now call “experimental sociology.” However, the methodological promise of RCTs depends not only on random assignment but also on transparent reporting practices that allow readers to assess internal validity, implementation quality, and the generalizability of findings. In this article, we conduct a systematic scoping review of 91 field experiments published since 2000 in high-ranking sociological journals. Using an adapted version of the CONSORT-SPI checklist, we evaluate reporting accuracy across a range of design and implementation indicators. We find substantial variability and frequent omissions in how sociological RCTs are reported, with implications for replicability and the cumulative advancement of causal knowledge. Our findings suggest that sociologists have yet to fully adopt reporting standards developed in other disciplines, and that greater attention to documentation could strengthen the credibility and utility of experimental evidence. We offer concrete recommendations for researchers, reviewers, and journal editors to improve transparency and comparability in sociological field experiments.

The aim of this study is to provide a synthesis of the most recent empirical research examining the relationship between economic, social and cultural status (ESCS) and school education in Italy, in relation to learning outcomes, career guidance, and dropout rates. To this end, we conducted a scoping review. Analysis of the 26 studies reveals that the most investigated indicators are the parental education and poverty status. The relationship between ESCS and education has been predominantly studied in terms of learning outcomes, followed by its impact on school dropout. Results show a positive association between the indicators of ESCS and academic performance, with familial background influencing the choice of educational pathway and the level of education attained. Status socioeconomico-culturale e istruzione scolastica in Italia. Risultati di una scoping review. L’obiettivo di questo studio è offrire una rassegna degli studi empirici più recenti sulla relazione tra status socioeconomico-culturale (SSEC) e istruzione scolastica in Italia, in merito agli apprendimenti, all’orientamento e ai tassi di dispersione e abbandono. A questo scopo è stata condotta una scoping review che ha incluso 26 studi. I risultati mostrano come gli indicatori più indagati dello SSEC siano l’istruzione dei genitori e lo stato di povertà. La relazione tra SSEC e istruzione è stata studiata soprattutto in riferimento ai risultati di apprendimento e alla sua incidenza sulla dispersione scolastica. I risultati mostrano una relazione positiva tra gli indicatori dello SSEC e il rendimento, con le origini familiari che influenzano la scelta del percorso scolastico e il livello di istruzione dei figli.

The issue of students dropping out before completing secondary education is a global concern with significant individual and societal consequences. Various terms, such as Early School Leaving (ESL), Early Leaving from Education and Training (ELET), and school dropout, reflect different policy perspectives on this phenomenon. Despite international efforts to address school dropout, a comprehensive review identifying areas with strong evidence and research gaps is lacking. This Evidence and Gap Map (EGM) systematically reviews the evidence on programs aimed at reducing school dropout and improving intermediate outcomes (e.g., educational performance, attendance). Following the 2011 European Council Recommendations, interventions are categorized into three domains: prevention, intervention, and compensation. The EGM will provide school and policy decision-makers, as well as program developers, with an overview of research evidence useful for guiding efforts to reduce school dropout and design effective programs. By pinpointing research gaps, the EGM will help identify critical areas where further investigation is needed to better understand and address the impact of school dropout programs.

The Every Student Succeeds Act has made the use of evidence even more relevant to policymakers and practitioners. The What Works Clearinghouse (WWC) serves a role as reviewer of evidence as policymakers and practitioners attempt to navigate the broad expanse of educational research. For this reason, it is of vital interest to understand whether WWC policies lead to fair estimates of effect sizes. Previous research (e.g. Cheung & Slavin, 2016) has indicated that small studies are associated with greatly inflated effect sizes. This study examined whether this could be explained by methodological and program factors of studies in the WWC database. Using the outcomes from all accepted studies in the reading and mathematics areas, we found a non-linear trend of effect size decreasing as the sample size increases. Even controlling for research design, level of assignment, type of intervention, and type of outcome, there was still an effect of sample size. While publication bias, intervention strength, methodological rigor, and superrealization may each contribute partially to this sample size effect, neither individually nor together do they explain the full impact of sample size on effect size. We suggest the WWC institute a policy of weighting studies by inverse variance before averaging effect sizes, and setting minimum sample criteria to reduce the inflationary effect of small study bias.

This protocol presents the planned methods and procedures for a meta-review that explores the methodological quality of education meta-analyses and their potential significance for research use. An ever-increasing number of meta-analyses are published every year while at the same time new approaches to enhance the validity and reproducibility of systematic reviews are in constant development. We aim to conduct a meta-review to examine the current practices of education meta-analyses in terms of review procedures, meta-analysis methods, and strategies for making meta-analyses useful for non-research audiences. We will focus on meta-analyses including randomized controlled trials and quasi-experimental designs on the effects of K-12 school-based academic interventions on student academic achievement. A comprehensive search will be conducted to retrieve all studies that potentially meet our criteria. Study features will be coded to evaluate three main dimensions: the quality of the review process, which includes the practices used at each systematic review stage (e.g., search procedure, selection, and critical appraisal); the quality of the meta-analysis methods, including the methods used for synthesizing the results, exploring heterogeneity and additional analyses, such as publication bias assessment; the significance for research use, which includes evaluating the ways authors present findings to increase accessibility and relevance for practitioners (e.g., stakeholder engagement in the process, reporting of results). We will identify strengths and weaknesses of current meta-analysis practices and compare them across time, publication outlet, and funding status.