Using Large Language Models for Robot-Assisted Therapeutic Role-Play: Factuality is not enough!
Open Access
- 8 July 2024
- conference paper
- Published by Association for Computing Machinery (ACM)
- p. 1-6
Abstract
Robot-assisted social role-play can help neurodivergent individuals practice social skills in a safe environment. Large language models (LLM) facilitate the implementation of such agents. However, high quality standards must be ensured in this sensitive setting. This article argues that current evaluation methods of generated language are not sufficient because they are grounded in beliefs about language as an external code to describe the world (referential functions of language). We argue that non-referential functions of language must be part of the evaluation of LLM-generated language when LLMs engage in social interactions with users. We test the feasibility of our approach in a pilot implementation of a platform for robot-assisted social role-play. Out proposed evaluation framework helps to assess systematically referential and non-referential functions of LLM-generated language. We argue that the evaluation framework can be also applied to multimodal interaction.Keywords
Funding Information
- FNR Luxembourg (IF 17762538-CAIDA)
This publication has 34 references indexed in Scilit:
- Prevalence of bias against neurodivergence‐related terms in artificial intelligence language modelsAutism Research, 2024
- Large Language Model (LLM)-Powered Chatbots Fail to Generate Guideline-Consistent Content on Resuscitation and May Provide Potentially Harmful AdvicePrehospital and Disaster Medicine, 2023
- Non-Standard Vietnamese Word Detection and Normalization for Text–to–Speech2022
- Are social robots ready yet to be used in care and therapy of autism spectrum disorder: A systematic review of randomized controlled trialsNeuroscience & Biobehavioral Reviews, 2021
- “I Feel More Luxembourgish, but Portuguese Too” Cultural Identities in a Multicultural SocietyIntegrative Psychological and Behavioral Science, 2019
- Using Linguistic Alignment to Enhance Learning Experience with Pedagogical Agents: The Special Case of Dialect2013
- The Effects of Culturally Congruent Educational Technologies on Student Achievement2013
- How Language Shapes ThoughtScientific American, 2011
- The linguistic construction of expert identity in professor–student discussions of scienceCultural Studies of Science Education, 2006
- Constructing languages and publics1995