Reliability and Consensus of Experienced Wine Judges: Expertise Within and Between?
Author:
- 1 May 2012
- journal article
- research article
- Published by Cambridge University Press (CUP) in Journal of Wine Economics
- Vol. 7 (1) , 70-87
Abstract
This paper considers the levels of reliability and consensus of wine quality judgments found in studies of experienced wine judges. Both reliability, which concerns the similarity of repeat judgments of a particular wine by the same judge, and consensus, which concerns the similarity of judgments of a particular wine across judges, are necessary requirements for expertise in wine judging. Reliability and consensus levels found in wine judging are compared to those documented by a large body of research in six other fields: medicine, clinical psychology, business, auditing, personnel management, and meteorology. In all fields, including wine judging, reliability is greater than consensus. Both reliability and consensus are, on average, substantially lower in wine judging than in other fields, although tremendous variability exists across judges in every field. Overall, little support is found for the idea that experienced wine judges should be regarded as experts. (JEL Classification: C91)Keywords
This publication has 34 references indexed in Scilit:
- The Judgment of Paris According to BordaJournal of Wine Research, 2009
- Evaluation of the consistency of wine quality assessments from expert wine tastersAustralian Journal of Grape and Wine Research, 2008
- The evaluation of judges at wine competitions: the application of Eggshell plotsJournal of Wine Research, 1997
- The biological basis of food perception and acceptanceFood Quality and Preference, 1993
- How quantity judgment changes as the number of cues increases: An analytical framework and review.Psychological Bulletin, 1992
- A general model of consensus and accuracy in interpersonal perception.Psychological Review, 1991
- Rationality and the sanctity of competenceBehavioral and Brain Sciences, 1981
- Observer agreement in evaluating coronary angiograms.Circulation, 1975
- Man versus model of man: A rationale, plus some evidence, for a method of improving on clinical inferences.Psychological Bulletin, 1970
- ORGANOLEPTIC RATINGS OF WINES ESTIMATED FROM ANALYTICAL DATAJournal of Food Science, 1953