Assessment of Sign Language-Based versus Touch-Based Input for Deaf Users Interacting with Intelligent Personal Assistants
Open Access
- 11 May 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1-15
Abstract
With the recent advancements in intelligent personal assistants (IPAs), their popularity is rapidly increasing when it comes to utilizing Automatic Speech Recognition within households. In this study, we used a Wizard-of-Oz methodology to evaluate and compare the usability of American Sign Language (ASL), Tap to Alexa, and smart home apps among 23 deaf participants within a limited-domain smart home environment. Results indicate a slight usability preference for ASL. Linguistic analysis of the participants' signing reveals a diverse range of expressions and vocabulary as they interacted with IPAs in the context of a restricted-domain application. On average, deaf participants exhibited a vocabulary of 47 +/- 17 signs with an additional 10 +/- 7 fingerspelled words, for a total of 246 different signs and 93 different fingerspelled words across all participants. We discuss the implications for the design of limited-vocabulary applications as a stepping-stone toward general-purpose ASL recognition in the future.Keywords
All Related Versions
Funding Information
- National Institute on Disability, Independent Living, and Rehabilitation Research (90REGE0013)
This publication has 26 references indexed in Scilit:
- Usability Evaluation of Artificial Intelligence-Based Voice Assistants: The Case of Amazon AlexaSN Computer Science, 2021
- Deaf, Hard of Hearing, and Hearing Perspectives on Using Automatic Speech Recognition in Conversation2017
- Learnability through Adaptive Discovery Tools in Voice User Interfaces2017
- Comparing Social Robot, Screen and Voice Interfaces for Smart-Home Control2017
- What can I say?2016
- On the Future of Personal Assistants2016
- Combining body pose, gaze, and gesture to determine intention to interact in vision-based interfaces2014
- Multimodal interaction: A reviewPattern Recognition Letters, 2014
- When designing usability questionnaires, does it hurt to be positive?2011
- Difficulties in Automatic Speech Recognition of Dysarthric Speakers and Implications for Speech-Based Applications Used by the Elderly: A Literature ReviewAssistive Technology, 2010