Ali Paikan
Ali Paikan has a total of 5 co-authors. Here are the top co-authors by number of shared publications:
- Elisabeth André
Universitat Augsburg, Augsburg, Germany
2 shared publications - Jauwairia Nasir
University of Augsburg, Germany
2 shared publications - Pouyan Ziafati
LuxAI S. A., Luxembourg
2 shared publications - Sviatlana Höhn
SnT, University of Luxembourg, Luxembourg, Luxembourg
2 shared publications - Daniel Carnieto Tozadore
University College London, London, United Kingdom
1 shared publication
Latest Publications
- Open Access
- 24 November 2024
- Published by Association for Computing Machinery (ACM)
- p. 393-395
Beyond Pretend-Reality Dualism: Frame Analysis of LLM-powered Role Play with Social Agents
Role-playing activities offer opportunities for developing individuals’ creativity, communication, and problem-solving skills. Recent advances in large language models (LLM) facilitate fluent conversations with machines. To investigate benefits and pitfalls of LLMs in a relatively unexplored context of human-agent role-play as a culturally contextualised activity, a dataset of twelve human-agent interactions produced by two researchers with two state-of-the-art LLMs was annotated based on a frame analysis scheme from literature. The pilot study shows that human-agent play has a similar complexity as human-human play in which players maintain identities of themselves, external observers and play characters simultaneously going beyond the pretend-reality dualism. Results suggest that, while the LLMs can maintain and shift between roles, they play some roles better than others, and display cultural and gender stereotypes. Additionally, the coding scheme shows potential to help identify LLM outputs that require embodied enactment, and to be used for LLM bench-marking for role-play.
- 24 November 2024
- Published by Association for Computing Machinery (ACM)
- p. 393-395
Open AccessBeyond Pretend-Reality Dualism: Frame Analysis of LLM-powered Role Play with Social Agents
Role-playing activities offer opportunities for developing individuals’ creativity, communication, and problem-solving skills. Recent advances in large language models (LLM) facilitate fluent conversations with machines. To investigate benefits and pitfalls of LLMs in a relatively unexplored context of human-agent role-play as a culturally contextualised activity, a dataset of twelve human-agent interactions produced by two researchers with two state-of-the-art LLMs was annotated based on a frame analysis scheme from literature. The pilot study shows that human-agent play has a similar complexity as human-human play in which players maintain identities of themselves, external observers and play characters simultaneously going beyond the pretend-reality dualism. Results suggest that, while the LLMs can maintain and shift between roles, they play some roles better than others, and display cultural and gender stereotypes. Additionally, the coding scheme shows potential to help identify LLM outputs that require embodied enactment, and to be used for LLM bench-marking for role-play.
- Open Access
- 8 July 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-6
Using Large Language Models for Robot-Assisted Therapeutic Role-Play: Factuality is not enough!
Robot-assisted social role-play can help neurodivergent individuals practice social skills in a safe environment. Large language models (LLM) facilitate the implementation of such agents. However, high quality standards must be ensured in this sensitive setting. This article argues that current evaluation methods of generated language are not sufficient because they are grounded in beliefs about language as an external code to describe the world (referential functions of language). We argue that non-referential functions of language must be part of the evaluation of LLM-generated language when LLMs engage in social interactions with users. We test the feasibility of our approach in a pilot implementation of a platform for robot-assisted social role-play. Out proposed evaluation framework helps to assess systematically referential and non-referential functions of LLM-generated language. We argue that the evaluation framework can be also applied to multimodal interaction.
- 8 July 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-6
Open AccessUsing Large Language Models for Robot-Assisted Therapeutic Role-Play: Factuality is not enough!
Robot-assisted social role-play can help neurodivergent individuals practice social skills in a safe environment. Large language models (LLM) facilitate the implementation of such agents. However, high quality standards must be ensured in this sensitive setting. This article argues that current evaluation methods of generated language are not sufficient because they are grounded in beliefs about language as an external code to describe the world (referential functions of language). We argue that non-referential functions of language must be part of the evaluation of LLM-generated language when LLMs engage in social interactions with users. We test the feasibility of our approach in a pilot implementation of a platform for robot-assisted social role-play. Out proposed evaluation framework helps to assess systematically referential and non-referential functions of LLM-generated language. We argue that the evaluation framework can be also applied to multimodal interaction.