Kappa and Its Dependence on Marginal Rates
Author:
- 15 February 2005
- book chapter
- Published by Wiley
Abstract
The kappa statistic is a widely used measure for quantifying agreement between raters using nominal scales. It is advocated on the grounds that it “corrects” for a level of agreement that would be expected by chance alone based on the marginal rates. The operating characteristics of kappa, however, show that it responds in a complicated and unintuitive way to subject‐specific and marginal agreement. Modifications to kappa and supplementary statistics, which have been proposed to help interpret kappa are discussed. Remarks are made as to why the kappa statistic has such problematic features.Keywords
This publication has 14 references indexed in Scilit:
- Conditional inference for predictive agreementStatistics in Medicine, 1999
- Behavior and interpretation of the κ statistic: Resolution of the two paradoxesJournal of Clinical Epidemiology, 1996
- Conditional inference for subject‐specific and marginal agreement: Two families of agreement measuresCanadian Journal of Statistics, 1995
- Bias, prevalence and kappaJournal of Clinical Epidemiology, 1993
- On a discrete distribution associated with the statistical assessment of nominal scale agreementDiscrete Mathematics, 1991
- High agreement but low kappa: II. Resolving the paradoxesJournal of Clinical Epidemiology, 1990
- High agreement but low Kappa: I. the problems of two paradoxesJournal of Clinical Epidemiology, 1990
- Kappa coefficients in epidemiology: An appraisal of a reappraisalJournal of Clinical Epidemiology, 1988
- Ramifications of a Population Model for κ as a Coefficient of ReliabilityPsychometrika, 1979
- A Coefficient of Agreement for Nominal ScalesEducational and Psychological Measurement, 1960