CODECHECK: an Open Science initiative for the independent execution of computations underlying research articles during peer review to improve reproducibility
Open Access
- 20 July 2021
- journal article
- research article
- Published by F1000 Research Ltd in F1000Research
- Vol. 10, 253
Abstract
The traditional scientific paper falls short of effectively communicating computational research. To help improve this situation, we propose a system by which the computational workflows underlying research articles are checked. The CODECHECK system uses open infrastructure and tools and can be integrated into review and publication processes in multiple ways. We describe these integrations along multiple dimensions (importance, who, openness, when). In collaboration with academic publishers and conferences, we demonstrate CODECHECK with 25 reproductions of diverse scientific publications. These CODECHECKs show that asking for reproducible workflows during a collaborative review can effectively improve executability. While CODECHECK has clear limitations, it may represent a building block in Open Science and publishing ecosystems for improving the reproducibility, appreciation, and, potentially, the quality of non-textual research artefacts. The CODECHECK website can be accessed here: https://codecheck.org.uk/.Keywords
Funding Information
- Mozilla Foundation
- Sustainable Software Institute
- German Research Foundation (PE1632/17-1)
This publication has 70 references indexed in Scilit:
- The Availability of Research Data Declines Rapidly with Article AgeCurrent Biology, 2014
- Value all research productsNature, 2013
- Scientific UtopiaPerspectives on Psychological Science, 2012
- The Effects of FreeSurfer Version, Workstation Type, and Macintosh Operating System Version on Anatomical Volume and Cortical Thickness MeasurementsPLOS ONE, 2012
- A Universal Identifier for Computational ResultsProcedia Computer Science, 2011
- Publish your computer code: it is good enoughNature, 2010
- The role of statistical reviewer in biomedical scientific journalBiochemia Medica, 2009
- Spike Timing Dependent Plasticity Finds the Start of Repeating Patterns in Continuous Spike TrainsPLOS ONE, 2008
- WaveLab and Reproducible Research1995
- Neural networks and physical systems with emergent collective computational abilities.Proceedings of the National Academy of Sciences, 1982