Interrater Agreement and Reliability
Author:
- Laura D. GoodwinLGLaura D. Goodwin
- 1 March 2001
- journal article
- Published by Taylor & Francis in Measurement in Physical Education and Exercise Science
- Vol. 5 (1) , 13-34
Abstract
The distinction between interrater (or interobserver, interjudge, interscorer) "agreement" and "reliability" is discussed. A total of 3 approaches or techniques for the estimation of interrater agreement and reliability are illustrated and compared, using data from a hypothetical study. The 3 approaches are (a) simple percentage of agreement and kappa, (b) simple correlational techniques, and (c) generalizability (G) theory techniques. In discussing the relative advantages and disadvantages of the various approaches, the G theory techniques are emphasized-because they are the most comprehensive and flexible, allowing the researcher to isolate multiple sources of measurement error in one study. Some recommendations regarding the "method of choice" for various situations are offered.Keywords
This publication has 51 references indexed in Scilit:
- MINI-LOGGER® Reliability and Validity for Estimating Energy Expenditure and Heart Rate in AdolescentsResearch Quarterly for Exercise and Sport, 1999
- Observational Learning and the Fearful Child: Influence of Peer Models on Swimming Skill Performance and Psychological ResponsesResearch Quarterly for Exercise and Sport, 1998
- The Reliability and Validity of a Chair Sit-and-Reach Test as a Measure of Hamstring Flexibility in Older AdultsResearch Quarterly for Exercise and Sport, 1998
- Application of Generalizability Theory to Measurement of Activity in Males Who are Not Regularly Active: A Preliminary ReportResearch Quarterly for Exercise and Sport, 1998
- Physical Activity Levels and Prompts in Young Children at Recess: A Two-Year Study of a Bi-Ethnic SampleResearch Quarterly for Exercise and Sport, 1997
- How “Significant” is Your Reliability?Research Quarterly for Exercise and Sport, 1993
- Efficacy of Skinfold Training Clinics: A Comparison between Clinic Trained and Experienced TestersResearch Quarterly for Exercise and Sport, 1992
- Validity and Generalizability of Social Dance Performance RatingsResearch Quarterly for Exercise and Sport, 1991
- Measures of response agreement for qualitative data: Some generalizations and alternatives.Psychological Bulletin, 1971
- A Coefficient of Agreement for Nominal ScalesEducational and Psychological Measurement, 1960