Raja Kushalnagar

Intelligent Personal Assistants (IPAs) are currently limited to mostly voice input by users, which often does not work for Deaf and Hard of Hearing (DHH) users’ accessibility. Touch interfaces are an accessible alternative in principle, and these recently have been combined with large language models (LLMs) for usability enhancements. However, these are not hands-free, and it is not always feasible to walk up to a device and interact with its touchscreen, such as in the kitchen with dirty hands. This paper situates an LLM-powered touch interface against hands-free options. We present a study with 23 DHH participants who tested three potential input methods for interacting with IPAs: American Sign Language (ASL) in a Wizard-of-Oz setting, LLM-assisted touch through a touchscreen, and LLM-assisted touch through hands-free head pointing. ASL and LLM-assisted touch had comparable usability scores, while headpointing scored much worse. Despite comparable usability between ASL and touch, participants were much more enthusiastic about ASL input. This suggests ASL recognition should be the ultimate goal, but because such technology is not yet commercially viable, further research is needed for identifying practical hands-free alternatives to voice interaction with IPAs.

Hybrid events have become increasingly common, yet supporting accessible participation for deaf and hard of hearing (DHH) audiences remains challenging. Although accessibility practices for in-person and virtual settings are relatively well established, hybrid environments introduce additional coordination demands, particularly in aligning sign language interpretation, captioning, and audiovisual (AV) workflows across modalities. We present a case study of a large DHH-focused hybrid conference with over 300 virtual and approximately 90 in-person and hybrid attendees. Drawing on planning materials, live workflow observations, and post-event reflections, we examine how accessibility was implemented across in-person and remote contexts. Our analysis identifies recurring challenges in interpreter configuration, Q&A management, and AV coordination. We further identify strategies that supported equitable participation, including the use of separate interpreter teams, structured Q&A workflows, and coordinated AV control across environments. Our findings highlight key trade-offs in supporting visual communication across modalities.