Kappa and Its Dependence on Marginal Rates

Author:

Abstract
The kappa statistic is a widely used measure for quantifying agreement between raters using nominal scales. It is advocated on the grounds that it “corrects” for a level of agreement that would be expected by chance alone based on the marginal rates. The operating characteristics of kappa, however, show that it responds in a complicated and unintuitive way to subject‐specific and marginal agreement. Modifications to kappa and supplementary statistics, which have been proposed to help interpret kappa are discussed. Remarks are made as to why the kappa statistic has such problematic features.

This publication has 14 references indexed in Scilit: