Evaluation Techniques for Chatbot Usability: A Systematic Mapping Study
Authors:
- 1 November 2019
- journal article
- research article
- Published by World Scientific Pub Co Pte Ltd in International Journal of Software Engineering and Knowledge Engineering
- Vol. 29 (11n12) , 1673-1702
Abstract
Background: The use of chatbots has increased considerably in recent years. These are used in different areas and by a wide variety of users. Due to this fact, it is essential to incorporate usability in their development. Aim: Our objective is to identify the state-of-the-art in chatbot usability and applied human–computer interaction techniques, to analyze how to evaluate chatbot usability. Method: We have conducted a systematic mapping study, by searching the main scientific databases. The search retrieved 170 references and 21 articles were retained as primary studies. Results: The works were categorized according to four criteria: usability techniques, usability characteristics, research methods and type of chatbots. Conclusions: Chatbot usability is still a very incipient field of research where the published studies are mainly surveys, usability tests, and rather informal experimental studies. Hence, it becomes necessary to perform more formal experiments to measure user experience, and exploit these results to provide usability-aware design guidelines.Keywords
This publication has 26 references indexed in Scilit:
- Pilot testing a digital pet avatar for older adultsGeriatric Nursing, 2017
- Sherlock: Experimental Evaluation of a Conversational Agent for Mobile Information TasksIEEE Transactions on Human-Machine Systems, 2017
- Acceptability of Embodied Conversational Agent in a Health Care Context2016
- A Conversational Agent for an Online Mental Health Intervention2016
- Natural Language Interface to Relational Database (NLI-RDB) Through Object Relational Mapping (ORM)2016
- Extending Empirical Analysis of Usability and Playability to Multimodal Computer Games2016
- Adaptive Grounding and Dialogue Management for Autonomous Conversational Assistants for Elderly Users2015
- Design and evaluation of a conversational agent for the touristic domain2014
- A Configurable Conversational Agent to Trigger Students’ Productive Dialogue: A Pilot Study in the CALL DomainInternational Journal of Artificial Intelligence in Education, 2014
- Current practice in measuring usability: Challenges to usability studies and researchInternational Journal of Human-Computer Studies, 2006