Preprint
Abstract
The reproducibility crisis has emerged as an important concern across many fields of science including life science, since many published results failed to reproduce. Systems biology modelling, which involves mathematical representation of biological processes to study complex system behaviour, was expected to be least affected by this crisis. While lack of reproducibility of experimental results and computational analysis could be a repercussion of several compounded factors, it was not fully understood why systems biology models with well-defined mathematical expressions fail to reproduce and how prevalent it is. Hence, we systematically attempted to reproduce 455 kinetic models of biological processes published in peer-reviewed research articles from 152 journals; which is collectively a work of about 1400 scientists from 49 countries. Our investigation revealed that about half ( 49% ) of the models could not be reproduced using the information provided in the published manuscripts. With further effort, an additional 12% of the models could be reproduced either by empirical correction or support from authors. The other 37% remained non-reproducible models due to missing parameter values, missing initial concentration, inconsistent model structure, or a combination of these factors. Among the corresponding authors of the non-reproducible model we contacted, less than 30% responded. Our analysis revealed that models published in journals across several fields of life science failed to reproduce, revealing a common problem in the peer-review process. Hence, we propose an 8-point reproducibility scorecard that can be used by authors, reviewers and journal editors to assess each model and address the reproducibility crisis.Keywords
All Related Versions
- Published version: Molecular Systems Biology, 17 (2), e9982.
This publication has 40 references indexed in Scilit:
- Open is not enoughNature Physics, 2018
- Reproducibility failures are essential to scientific inquiryProceedings of the National Academy of Sciences, 2018
- Is science really facing a reproducibility crisis, and do we need it to?Proceedings of the National Academy of Sciences, 2018
- Reproducibility of science: Fraud, impact factors and carelessnessJournal of Molecular and Cellular Cardiology, 2018
- The reproducibility of research and the misinterpretation of p -valuesRoyal Society Open Science, 2017
- 1,500 scientists lift the lid on reproducibilityNature, 2016
- Contextual sensitivity in scientific reproducibilityProceedings of the National Academy of Sciences, 2016
- Reproducibility in ScienceCirculation Research, 2015
- An open investigation of the reproducibility of cancer biology researcheLife, 2014
- Reproducibility of research and preclinical validation: problems and solutionsNature Reviews Clinical Oncology, 2013