What is our power to detect device effects in animal tracking studies?
- 25 March 2021
- journal article
- research article
- Published by Wiley in Methods in Ecology and Evolution
- Vol. 12 (7) , 1174-1185
Abstract
The use of bio‐logging devices to track animal movement continues to grow as technological advances and device miniaturisation allow researchers to study animal behaviour in unprecedented detail. Balanced against the remarkable data that bio‐loggers can provide is a need to understand the impact of devices on animal behaviour and welfare. Recent meta‐analyses have demonstrated the impacts of device attachment on animal behaviour, but there is a concern about the frequency and clarity with which device effects are reported. One aspect lacking in many studies is assessment of the statistical power of tests of device effects, yet such information would assist the interpretation of results. We address this issue by providing an overview of the statistical power, as well as the Type M (magnitude) and Type S (sign) error rate, of tests of device effects within the avian tracking literature across a range of assumed effect sizes. The median power of statistical tests ranged from 9% to 65% across a range of assumed effect sizes corresponding to benchmark values for small, moderate and large effects (d = 0.2, 0.5, 0.8, respectively). Moreover, when using effect sizes derived from previous a meta‐analysis (d = 0.1), median power was only 6%. When assuming smaller effect sizes, statistical tests were characterised by high Type M and Type S error rates, suggesting that statistically significant results of device effects will tend to exaggerate the size of such effects and may estimate the sign of an effect in the wrong direction. Well‐designed tracking studies will reduce device effects to low levels and consequently issues associated with low power will be commonplace. Nevertheless, assessment of device effects remains important, particularly when embarking on novel tracking studies. We recommend that statistical tests of device effects are reported clearly and are routinely accompanied by assessment of statistical power, including Type M and Type S errors, based upon realistic external estimates of effect size. Reporting the statistical power can help avoid the pitfalls of overstating results from individual studies, shift the emphasis to accurate reporting of effect sizes and guide decisions about the ethical impacts of device attachment.Keywords
This publication has 68 references indexed in Scilit:
- Reliability of flipper-banded penguins as indicators of climate changeNature, 2011
- How Many Studies Do You Need?Journal of Educational and Behavioral Statistics, 2010
- Effects of geolocator attachments on breeding parameters of Lesser KestrelsJournal of Field Ornithology, 2009
- Standardized or simple effect size: What should be reported?British Journal of Psychology, 2009
- Effect size, confidence interval and statistical significance: a practical guide for biologistsBiological Reviews, 2007
- Life of the mind: The interface of psychopharmaceuticals, domestic economies, and social abandonmentAmerican Ethnologist, 2004
- The case against retrospective statistical power analyses with an introduction to power analysisacta ethologica, 2004
- How meta-analysis increases statistical power.Psychological Methods, 2003
- Quantifying heterogeneity in a meta‐analysisStatistics in Medicine, 2002
- Remote-sensing systems and seabirds: their use, abuse and potential for measuring marine environmental variablesMarine Ecology Progress Series, 2002