Evaluating the Downstream Utility of User Tests and Examining the Developer Effect: A Case Study
Author:
- Effie Lai-Chong LawELEffie Lai-Chong Law
- 1 November 2006
- journal article
- Published by Taylor & Francis in International Journal of Human–Computer Interaction
- Vol. 21 (2) , 147-172
Abstract
In this article, a case study on evaluating the downstream utility of user tests performed on a digital library with reference to the developer effect is reported. Downstream utility is defined as the effectiveness with which the resolution to a usability problem (UP) is implemented, and developer effect is defined as developers' bias toward fixing UPs with particular characteristics. To measure the effectiveness of the user tests, the actual impacts of fixing or not fixing the UPs identified were analyzed. To address a theoretical void in studying the persuasive power of usability evaluation results, Information Integration Theory was employed. Six research questions that predict the persuasiveness of different qualities of usability problems to induce fixes and the effectiveness of such fixes were investigated. Multiperspective data have been collected from usability specialists, the development team of the digital library, and its old as well as new users. Implications for reporting UPs and future rese...Keywords
This publication has 20 references indexed in Scilit:
- Evaluating usability methodsInteractions, 2003
- The Evaluator Effect: A Chilling Fact About Usability Evaluation MethodsInternational Journal of Human–Computer Interaction, 2001
- Criteria For Evaluating Usability Evaluation MethodsInternational Journal of Human–Computer Interaction, 2001
- Damaged Merchandise? A Review of Experiments That Compare Usability Evaluation MethodsHuman–Computer Interaction, 1998
- On our case study of claims analysis and other usability evaluation methodsBehaviour & Information Technology, 1998
- Heuristic Walkthroughs: Finding the Problems Without the NoiseInternational Journal of Human–Computer Interaction, 1997
- Tracking the effectiveness of usability evaluation methodsBehaviour & Information Technology, 1997
- Book Review: Cost Justifying UsabilityACM SIGCHI Bulletin, 1995
- Contributions to Information Integration Theory: Vol. II. SocialThe American Journal of Psychology, 1994
- Combining of stimulus dimensions in children's and adults' judgments of area: An information integration analysis.Developmental Psychology, 1979