Evaluation of Prototypes and the Problem of Possible Futures
Authors:
Open Access
- 2 May 2017
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 2064-2077
Abstract
There is a blind spot in HCI's evaluation methodology: we rarely consider the implications of the fact that a prototype can never be fully evaluated in a study. A prototype under study exists firmly in the present world, in the circumstances created in the study, but its real context of use is a partially unknown future state of affairs. This present-future gap is implicit in any evaluation of prototypes, be they usability tests, controlled experiments, or field trials. A carelessly designed evaluation may inadvertently evaluate the wrong futures, contexts, or user groups, thereby leading to false conclusions and expensive design failures. The essay analyses evaluation methodology from this perspective, illuminating how to mitigate the present-future gap.Keywords
Funding Information
- H2020 European Research Council (637991)
- Suomen Akatemia (259281,298879)
This publication has 50 references indexed in Scilit:
- Some Whys and Hows of Experiments in Human—Computer InteractionFoundations and Trends® in Human-Computer Interaction, 2013
- Concept-Driven Interaction Design ResearchHuman–Computer Interaction, 2010
- Embodied interaction with a 3D versus 2D mobile mapPersonal and Ubiquitous Computing, 2008
- Interacting with user data – Theory and examples of drama and dramaturgy as methods of exploration and evaluation in user-centered designInteracting with Computers, 2006
- Permanently Beta: Responsive Organization in the Internet Era2004
- Understanding contexts by being there: case studies in bodystormingPersonal and Ubiquitous Computing, 2003
- Everyday Life as a Stage in Creating and Performing Scenarios for Wireless DevicesPersonal and Ubiquitous Computing, 2002
- Design: Cultural probesInteractions, 1999
- When and how to use scenario planning: A heuristic approach with illustrationJournal of Forecasting, 1991
- The Computer for the 21st CenturyScientific American, 1991