Testing the reproducibility of ecological studies on insect behavior in a multi-laboratory setting identifies opportunities for improving experimental rigor
Open Access
- 22 April 2025
- journal article
- research article
- Published by Public Library of Science (PLoS) in PLOS Biology
- Vol. 23 (4) , e3003019
Abstract
The reproducibility of studies involving insect species is an underexplored area in the broader discussion about poor reproducibility in science. Our study addresses this gap by conducting a systematic multi-laboratory investigation into the reproducibility of ecological studies on insect behavior. We implemented a 3 × 3 experimental design, incorporating three study sites, and three independent experiments on three insect species from different orders: the turnip sawfly (Athalia rosae, Hymenoptera), the meadow grasshopper (Pseudochorthippus parallelus, Orthoptera), and the red flour beetle (Tribolium castaneum, Coleoptera). Using random-effect meta-analysis, we compared the consistency and accuracy of treatment effects on insect behavioral traits across replicate experiments. We successfully reproduced the overall statistical treatment effect in 83% of the replicate experiments, but overall effect size replication was achieved in only 66% of the replicates. Thus, though demonstrating sufficient reproducibility in some measures, this study also provides the first experimental evidence for cases of poor reproducibility in insect experiments. Our findings further show that reasons causing poor reproducibility established in rodent research also hold for other study organisms and research questions. We believe that a rethinking of current best practices is required to face reproducibility issues in insect studies but also across disciplines. Specifically, we advocate for adopting open research practices and the implementation of methodological strategies that reduce bias and problems arising from over-standardization. With respect to the latter, the introduction of systematic variation through multi-laboratory or heterogenized designs may contribute to improved reproducibility in studies involving any living organisms.Keywords
Funding Information
- Deutsche Forschungsgemeinschaft (316099922)
- Deutsche Forschungsgemeinschaft (396776123)
- Deutsche Forschungsgemeinschaft (396777467)
- Deutsche Forschungsgemeinschaft (396780003)
- Deutsche Forschungsgemeinschaft (396776775)
This publication has 102 references indexed in Scilit:
- Electrophysiological and Behavioral Activity of Secondary Metabolites in the Confused Flour Beetle, Tribolium confusumJournal of Chemical Ecology, 2007
- Why Most Published Research Findings Are FalsePLOS Medicine, 2005
- New replicable anxiety-related measures of wall vs. center behavior of mice in the open fieldJournal of Applied Physiology, 2004
- An introduction to power and sample size estimationEmergency Medicine Journal, 2003
- Different data from different labs: Lessons from studies of gene–environment interactionJournal of Neurobiology, 2002
- Identification and ranking of genetic and laboratory environment factors influencing a behavioral trait, thermal nociception, via computational analysis of a large data archiveNeuroscience & Biobehavioral Reviews, 2002
- Influences of laboratory environment on behaviorNature Neuroscience, 2002
- Microbe Inhibition by Tribolium Flour Beetles Varies with Beetle Species, Strain, Sex, and Microbe GroupJournal of Chemical Ecology, 2002
- Behaviour and the standardization fallacyNature Genetics, 2000
- Genetics of Mouse Behavior: Interactions with Laboratory EnvironmentScience, 1999