A Coefficient of Agreement Adjusted for Bias in Paired Ordered Categorical Data
- 18 January 1997
- journal article
- research article
- Published by Wiley in Biometrical Journal
- Vol. 39 (6) , 643-657
Abstract
The quality of ordered categorical recordings is determined from repeated measurements on the same subject in order to assess the level of agreement between raters, scales or occasions. The presented rating‐invariant method for ordered categorical data provides means of analysing the quality of single‐item rating scales, irrespective of the number of possible response values and the marginal distributions. Marginal heterogeneity implies systematic disagreement, so‐called bias. An augmented ranking approach is the basis for the separation of inter‐rater disagreement into systematic and random components. Correlation between pairs of augmented rank values provides a measure of agreement to the best common ordering of paired classifications, given inter‐rater bias. The essential differences in interpretation and applicability of the proposed coefficient of agreement and the Spearman rank‐order correlation for ordered categorical data are discussed.Keywords
This publication has 11 references indexed in Scilit:
- Separation of systematic and random differences in ordinal rating scalesStatistics in Medicine, 1994
- Modelling patterns of agreement and disagreementStatistical Methods in Medical Research, 1992
- Statistical methods for assessing observer variability in clinical measures.BMJ, 1992
- Quality-of-Life Assessment: Can We Keep It Simple?Journal of the Royal Statistical Society Series A: Statistics in Society, 1992
- Regression Models for Ordinal DataJournal of the Royal Statistical Society Series B: Statistical Methodology, 1980
- A review of statistical methods in the analysis of data arising from observer reliability studies (Part I)*Statistica Neerlandica, 1975
- Ordinal Measures of AssociationJournal of the American Statistical Association, 1958
- On a Test of Whether one of Two Random Variables is Stochastically Larger than the OtherThe Annals of Mathematical Statistics, 1947
- THE TREATMENT OF TIES IN RANKING PROBLEMSBiometrika, 1945
- The Relation Between Measures of Correlation in the Universe of Sample PermutationsBiometrika, 1944