The Computational Reproducibility of Psychology Learning and Teaching Research
Open Access
- 14 January 2026
- journal article
- Published by SAGE Publications in Teaching of Psychology
Abstract
Background: Research is computationally reproducible when independent analysts can use the underlying data to reproduce the original results. Some research in psychology is computationally reproducible, although much is not. Objective: To assess the computational reproducibility of research published in Teaching of Psychology (ToP), Psychology Learning and Teaching (PLaT), and Scholarship of Teaching and Learning in Psychology (SoTL-P). Method: We identified key claims in 60 papers published with open data in ToP, PLaT, and SoTL-P between 2017 and 2025. We then sought to reproduce 101 results supporting these claims. Results: We exactly reproduced 73 of the 101 results. For a further 23 results, the substantive interpretations of our results matched those of the original researchers. The majority of the reproductions were performed relatively quickly by a psychology undergraduate with 2 years of relevant experience. Conclusion: Psychology learning and teaching research appears more computationally reproducible than research in other psychology sub-fields. However, reproducibility is undermined by several factors, including a lack of open data and code. Teaching Implications: Computational reproducibility activities can be incorporated into undergraduate research methods classes, effectively address many research methods learning outcomes, and help students develop both subject-specific and transferable employability skills.Keywords
Funding Information
- British Psychological Society (Undergraduate Research Assistantship Scheme)
This publication has 14 references indexed in Scilit:
- Who gets remembered? Disparities in recognition of early eminent psychologists and implications for belonging in psychology.Scholarship of Teaching and Learning in Psychology, 2025
- The reliability of replications: a study in computational reproductionsRoyal Society Open Science, 2025
- Teaching Constructive Replications in the Behavioral and Social Sciences Using Quantitative DataTeaching of Psychology, 2023
- Many researchers were not compliant with their published data sharing statement: a mixed-methods studyJournal of Clinical Epidemiology, 2022
- Emphasizing Transferable Skills in Undergraduate Cognitive Psychology is Associated With Higher GradesTeaching of Psychology, 2022
- The reproducibility of statistical results in psychological research: An investigation using unpublished raw data.Psychological Methods, 2021
- Analytic reproducibility in articles receiving open data badges at the journal Psychological Science : an observational studyRoyal Society Open Science, 2021
- Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015Nature Human Behaviour, 2018
- Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal CognitionRoyal Society Open Science, 2018
- The FAIR Guiding Principles for scientific data management and stewardshipScientific Data, 2016