Statistical Methods For Assessing Measurement Error (Reliability) in Variables Relevant to Sports Medicine
- 1 January 1998
- journal article
- review article
- Published by Springer Nature in Sports Medicine
- Vol. 26 (4) , 217-238
Abstract
Minimal measurement error (reliability) during the collection of interval- and ratio-type data is critically important to sports medicine research. The main components of measurement error are systematic bias (e.g. general learning or fatigue effects on the tests) and random error due to biological or mechanical variation. Both error components should be meaningfully quantified for the sports physician to relate the described error to judgements regarding ‘analytical goals’ (the requirements of the measurement tool for effective practical use) rather than the statistical significance of any reliability indicators. Methods based on correlation coefficients and regression provide an indication of ‘relative reliability’. Since these methods are highly influenced by the range of measured values, researchers should be cautious in: (i) concluding acceptable relative reliability even if a correlation is above 0.9; (ii) extrapolating the results of a test-retest correlation to a new sample of individuals involved in an experiment; and (iii) comparing test-retest correlations between different reliability studies. Methods used to describe ‘absolute reliability’ include the standard error of measurements (SEM), coefficient of variation (CV) and limits of agreement (LOA). These statistics are more appropriate for comparing reliability between different measurement tools in different studies. They can be used in multiple retest studies from ANOVA procedures, help predict the magnitude of a ‘real’ change in individual athletes and be employed to estimate statistical power for a repeated-measures experiment. These methods vary considerably in the way they are calculated and their use also assumes the presence (CV) or absence (SEM) of heteroscedasticity. Most methods of calculating SEM and CV represent approximately 68% of the error that is actually present in the repeated measurements for the ‘average’ individual in the sample. LOA represent the test-retest differences for 95% of a population. The associated Bland-Altman plot shows the measurement error schematically and helps to identify the presence of heteroscedasticity. If there is evidence of heteroscedasticity or non-normality, one should logarithmically transform the data and quote the bias and random error as ratios. This allows simple comparisons of reliability across different measurement tools. It is recommended that sports clinicians and researchers should cite and interpret a number of statistical methods for assessing reliability. We encourage the inclusion of the LOA method, especially the exploration of heteroscedasticity that is inherent in this analysis. We also stress the importance of relating the results of any reliability statistic to ‘analytical goals’ in sports medicine.Keywords
This publication has 57 references indexed in Scilit:
- Methodological commentary: Effect of measurement error on tests of statistical significanceJournal of Clinical and Experimental Neuropsychology, 1997
- Circadian Variation in Sports PerformanceSports Medicine, 1996
- A comparison of skinfold thickness, body mass index, bioelectrical impedance analysis and dual-energy X-ray absorptiometry in assessing body composition in obese subjects before and after weight lossClinical Nutrition, 1994
- Current issues and future directions for performance‐related research in sport psychologyJournal of Sports Sciences, 1994
- The future of performance‐related sports biomechanics researchJournal of Sports Sciences, 1994
- A review of research in sports physiologyJournal of Sports Sciences, 1994
- Sources of variation in back and leg dynamometryErgonomics, 1994
- How “Significant” is Your Reliability?Research Quarterly for Exercise and Sport, 1993
- Analysis of serial measurements in medical research.BMJ, 1990
- A comparison of three methods of assessing inter‐observer variation applied to ultrasonic fetal measurement in the third trimesterBJOG: An International Journal of Obstetrics & Gynaecology, 1989