Statistical Analysis of Receiver Operating Characteristic (ROC) Curves for the Ratings of theA‐NotAand the Same–Different Methods
Authors:
- 17 December 2012
- journal article
- Published by Wiley in Journal of Sensory Studies
- Vol. 28 (1) , 34-46
Abstract
Ratings are widely and frequently used in sensory analysis. However, ratings from different methods cannot be treated equivalently because they represent different types of data generated from different cognitive mechanisms. Ignoring this fact may lead to misleading conclusions about true sensory difference or treatment effects obtained from the ratings data analysis. The receiver operating characteristic (ROC) curve in signal detection theory is used in this paper to model the ratings of theA‐NotAand the same–different methods.This paper comprehensively discusses statistical analysis of theROCcurves for the methods, including presentations of theROCcurve functions, estimations ofd′s and their variances, goodness‐of‐fit tests, difference and similarity tests, testing powers and sample sizes, etc. The paper also discusses alternative indices of sensory difference derived from theROCcurve, including theR‐index andGini index. It is appealing to rethink the significance of theR‐index, not only as a powerful test statistic but also as a useful sensory measure.Rfunctions are provided for implementing all the analyses discussed in the paper.Practical Applications: This paper provides a variety of approaches and R functions for statistical analysis for the ratings of theA–NotA, and the ratings of the same–different methods. The measures ofd′s and other indices, which are method‐independent measures of sensory differences and treatment effects, can be estimated from different types of ratings data.Keywords
This publication has 52 references indexed in Scilit:
- A Transfer of Technology from Engineering: Use of ROC Curves from Signal Detection Theory to Investigate Information Processing in the Brain during Sensory Difference TestingJournal of Food Science, 2010
- d′ AND VARIANCE OF d′ FOR FOUR-ALTERNATIVE FORCED CHOICE (4-AFC)Journal of Sensory Studies, 2010
- Comparison of d′ estimates produced by three versions of a duo-trio test for discriminating tomato juices with varying salt concentrations: The effects of the number and position of the reference stimulusFood Quality and Preference, 2010
- Estimation and inference in the same–different testFood Quality and Preference, 2009
- Variants of A Not-A and 2AFC tests: Signal Detection Theory modelsFood Quality and Preference, 2009
- Variance ofd′ for thesame-different methodBehavior Research Methods, Instruments, & Computers, 2002
- Statistical models for the degree of difference testFood Quality and Preference, 2002
- Comparison of d′ values for the 2-AFC (paired comparison) and 3-AFC discrimination methods: Thurstonian models, sequential sensitivity analysis and powerFood Quality and Preference, 1998
- A comparison of the Two One-Sided Tests Procedure and the Power Approach for assessing the equivalence of average bioavailabilityJournal of Pharmacokinetics and Biopharmaceutics, 1987
- The area above the ordinal dominance graph and the area below the receiver operating characteristic graphJournal of Mathematical Psychology, 1975