The validity of the stimulated retrospective think-aloud method as measured by eye tracking
Authors:
Open Access
- 22 April 2006
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1253-1262
Abstract
Retrospective Think aloud (RTA) is a usability method that collects the verbalization of a user's performance after the performance is over. There has been little work done to investigate the validity and reliability of RTA. This paper reports on an experiment investigating these issues with a form of the method called stimulated RTA. By comparing subjects' verbalizations with their eye movements, we support the validity and reliability of stimulated RTA: the method provides a valid account of what people attended to in completing tasks, it has a low risk of introducing fabrications, and its validity isn't affected by task complexity. More detailed analysis of RTA shows that it also provides additional information about user's inferences and strategies in completing tasks. The findings of this study provide valuable support for usability practitioners to use RTA and to trust the users' performance information collected by this method in a usability study.Keywords
This publication has 21 references indexed in Scilit:
- Retrospective vs. concurrent think-aloud protocols: Testing the usability of an online library catalogueBehaviour & Information Technology, 2003
- Interaction designUbiquity, 2002
- A perception/action substrate for cognitive modeling in HCIInternational Journal of Human-Computer Studies, 2001
- Damaged Merchandise? A Review of Experiments That Compare Usability Evaluation MethodsHuman–Computer Interaction, 1998
- Prompted reflectionsInteractions, 1998
- Concurrent versus Retrospective Verbal Protocol for Comparing Window UsabilityProceedings of the Human Factors Society Annual Meeting, 1990
- The validity of verbal protocolsMemory & Cognition, 1989
- Task Complexity: A Review and AnalysisAcademy of Management Review, 1988
- Task complexity: Definition of the constructOrganizational Behavior and Human Decision Processes, 1986
- Eye movements and overt rehearsal in word recall.Journal of Experimental Psychology: Human Learning and Memory, 1977