A Work-Centered Approach to System User-Evaluation
- 25 July 2021
- journal article
- research article
- Published by SAGE Publications in Journal of Cognitive Engineering and Decision Making
- Vol. 15 (4) , 155-174
Abstract
New systems are often based on optimistic assumptions of how they will improve human performance. In the cognitive engineering tradition, these assumed benefits are regarded as hypotheses that need to be tested. An important element of a system user evaluation is to determine whether the hypothesized benefits are realized. Evaluation may also uncover unsupported aspects of performance or unanticipated side-effects of introducing the new technology that need to be addressed. We present a work-centered approach to user evaluation intended to meet these objectives, focusing specifically on design of tailored user-feedback questionnaires (work-centered questionnaires) that are intended to be diagnostic of how specific system elements do, or do not, support work. We summarize two recent evaluation studies we have conducted that illustrate our approach and the diagnostic power of work-centered questionnaires. We discuss how the goals and approach of a work-centered evaluation differ from more traditional approaches to usability evaluation that emphasize the use of standardized questionnaires and broad assessments of usability.Keywords
Funding Information
- Agency for Healthcare Research and Quality (R01 grant (R01HS022542) and R18 grant (R18HS02043)
This publication has 21 references indexed in Scilit:
- Psychometric Evaluation of the USE (Usefulness, Satisfaction, and Ease of use) Questionnaire for Reliability and ValidityProceedings of the Human Factors and Ergonomics Society Annual Meeting, 2018
- The System Usability Scale: Past, Present, and FutureInternational Journal of Human–Computer Interaction, 2018
- Measuring Perceived Usability: The CSUQ, SUS, and UMUXInternational Journal of Human–Computer Interaction, 2017
- Usability evaluation of an emergency department information system prototype designed using cognitive systems engineering techniquesApplied Ergonomics, 2017
- Cognitive engineering and health informatics: Applications and intersectionsJournal of Biomedical Informatics, 2017
- Assessment of Innovative Emergency Department Information Displays in a Clinical Simulation CenterJournal of Cognitive Engineering and Decision Making, 2015
- Analysis of Cognitive WorkReviews of Human Factors and Ergonomics, 2007
- Work-Centered Design: A Cognitiive Engineering Approach to System DesignProceedings of the Human Factors and Ergonomics Society Annual Meeting, 2003
- A framework for work-centered product evaluation2003
- Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information TechnologyMIS Quarterly, 1989