A Potential Bias in Subjective Ratings of Mental Effort
- 19 September 2018
- journal article
- research article
- Published by American Speech Language Hearing Association in Journal of Speech, Language, and Hearing Research
- Vol. 61 (9) , 2405-2421
Abstract
Purpose: Subjective reports of listening effort are frequently inconsistent with behavioral and physiological findings. A potential explanation is that participants unwittingly substitute an easier question when faced with a judgment that requires computationally expensive analysis (i.e., heuristic response strategies). The purpose of this study was to investigate whether participants substitute the question “How did I perform?” when asked “How much effort did that take?”. Method: Participants completed 2 sets of online surveys containing a text-based, multiple-choice synonym task. Expected performance and mental effort were manipulated across sets in 4 experiments, using a visual masking technique shown to correlate with speech-reception-testing in noise. Experiment 1 was designed to yield stable accuracy and differing effort across sets. Experiment 2 elicited differing accuracy and stable effort. Experiments 3 and 4 manipulated accuracy and performance in opposite directions. Participants included 273 adults (aged 19–68 years, M = 38.4 years). Results: Experiment 1 revealed no influence of perceived performance on ratings of effort when accuracy was stable. Experiment 2 showed that ratings of effort differed inversely with ratings of performance (lower performance and increased effort). Experiments 3 and 4 also demonstrated that participants rated effort in a manner inversely related to performance, regardless of the effort inherent in the condition. Conclusions: Participants likely substitute an easier question when asked to rate the multidimensional construct of mental effort. The results presented here suggest that perceived performance can serve as a ready heuristic and may explain the dissociation between subjective measures of listening effort and behavioral and physiological measures.Keywords
This publication has 62 references indexed in Scilit:
- Neural mechanisms of mental fatigue elicited by sustained auditory processingNeuropsychologia, 2017
- Multiple brain signatures of integration in the comprehension of degraded speechNeuroImage, 2011
- Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics supportJournal of Biomedical Informatics, 2009
- Heuristics made easy: An effort-reduction framework.Psychological Bulletin, 2008
- The affect heuristicEuropean Journal of Operational Research, 2007
- Representativeness Revisited: Attribute Substitution in Intuitive Judgment2002
- Multivariate workload evaluation combining physiological and subjective measuresInternational Journal of Psychophysiology, 2001
- Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research1988
- The robust beauty of improper linear models in decision making.American Psychologist, 1979
- Final Report of Experimental Psychology Group1979