Reproducibility debt: insights from practitioners
Open Access
- 3 November 2026
- journal article
- research article
- Published by Springer Nature in Empirical Software Engineering
- Vol. 32 (2) , 54
Abstract
Context: Reproducibility Debt (RpD) refers to the accumulative issues and challenges that hinder the ability to reproduce scientific outcomes, stemming from challenges inherent in scientific software code and data, including development, organisation, dissemination, and documentation. RpD may accumulate when researchers and scientific software practitioners engage in sub-optimal activities, often for short-term benefits, which ultimately compromise the reproducibility of research outcomes. Hence, it is essential to understand the factors that could lead scientific software teams to incur RpD and the strategies they adopt to prevent RpD in their projects. Objective: This research aims to understand practitioners’ perspectives on RpD, the factors they perceive as contributing to it, their consequences, and the practices they adopt to address it. Methodology: We present a mixed-methods study combining exploratory interviews with scientific software practitioners and a complementary survey conducted using our InsightRpD survey tool. Results: In total, 23 practitioners participated in interviews and 59 completed the survey. We identified 69 practitioner-reported causes and 80 practitioner-reported effects associated with RpD, confirming and extending current knowledge. Of the 38 causes reported in our previous Systematic Literature Review (SLR), 31 were supported by evidence from this study. The empirical data also highlights 38 additional causes and 47 additional effects not captured in prior literature, which are associated with evolving software practices, such as automation, decentralised teams, and the use of generative AI. Additionally, we explore the mitigation strategies employed by practitioners to address and reduce the impact of RpD. Conclusion: By identifying factors contributing to RpD, its associated consequences, and mitigation strategies from practitioner insights, this study provides evidence-based diagrams of practitioner-reported cause-effect relationships and practical approaches that advance the understanding and management of RpD, while informing future research and practice in Research Software Engineering (RSE) and software sustainability.Keywords
This publication has 53 references indexed in Scilit:
- Building cyberinfrastructure for the reuse and reproducibility of complex hydrologic modeling studiesEnvironmental Modelling & Software, 2023
- Reproducibility in Neuroimaging Analysis: Challenges and SolutionsBiological Psychiatry: Cognitive Neuroscience and Neuroimaging, 2022
- A taxonomy for reproducible and replicable research in environmental modellingEnvironmental Modelling & Software, 2020
- The practitioners’ point of view on the concept of technical debt and its causes and consequences: a design for a global family of industrial surveys and its first results from BrazilEmpirical Software Engineering, 2020
- Hearing the Voice of Software Practitioners on Causes, Effects, and Practices to Deal with Documentation Debt2020
- The open science initiative of the Empirical Software Engineering journalEmpirical Software Engineering, 2019
- Computing environments for reproducibility: Capturing the “Whole Tale”Future Generation Computer Systems, 2019
- Architectural Debt Management in Value-Oriented Architecting2014
- Comparing four approaches for technical debt identificationSoftware Quality Journal, 2013
- Experimentation in Software Engineering2012