Daksitha Senel Withanage Don
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Daksitha Senel Withanage Don has a total of 38 co-authors. Here are the top co-authors by number of shared publications:
- Elisabeth André
Universitat Augsburg, Augsburg, Germany
10 shared publications - Silvan Mertes
Technical University of Applied Sciences Augsburg, Augsburg, Germany
4 shared publications - Johanna Kuch
University of Augsburg, Augsburg, Germany
3 shared publications - Philipp Müller
Max-Planck-Institute for Intelligent Systems, Stuttgart, Germany
3 shared publications - Andreas Bulling
University of Stuttgart, Stuttgart, Germany
2 shared publications - Corinna Reck
Department of Psychology, Ludwig Maximilian University of Munich, Munich, Germany
2 shared publications
Latest Publications
- Open Access
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 17836-17841
Engagement Estimation in Human-Robot and Human-Human Interactions across Domain, Age and Culture: The MultiMediate '26 Challenge
- 9 November 2026
- Published by Association for Computing Machinery (ACM)
- p. 17836-17841
Open AccessEngagement Estimation in Human-Robot and Human-Human Interactions across Domain, Age and Culture: The MultiMediate '26 Challenge
- Open Access
- 4 October 2026
- Published by Association for Computing Machinery (ACM)
- p. 477-485
Towards a Voice Design Tool to Support Game Masters of Tabletop Role-Playing Games
- 4 October 2026
- Published by Association for Computing Machinery (ACM)
- p. 477-485
Open AccessTowards a Voice Design Tool to Support Game Masters of Tabletop Role-Playing Games
- Open Access
- 4 October 2026
- Published by Association for Computing Machinery (ACM)
- p. 1029-1038
Automated Behavioural Analysis in Parent-Infant Interactions using Multimodal-Large-Language Models
- 4 October 2026
- Published by Association for Computing Machinery (ACM)
- p. 1029-1038
Open AccessAutomated Behavioural Analysis in Parent-Infant Interactions using Multimodal-Large-Language Models
- Open Access
- 10 August 2026
- Published by Springer Nature in KI - Künstliche Intelligenz
- p. 1-20
Agents Through the Ages
Over several decades, research on socially interactive agents has transformed human–machine interaction, evolving from early script-based systems to today’s generative, multimodal agents. Tracing this development through the ages, this article reflects on major conceptual and technical shifts that shaped the field, drawing on advances across multiple areas of AI, and discusses how progress in natural language processing and multimodal behavior analysis and generation has redefined AI as a social interface. Rather than presenting this trajectory as a narrative of continuous progress, we also examine persistent limitations, including weak grounding, difficult evaluation, opaque decision processes, safety risks in sensitive contexts, and the gap between convincing social simulation and genuine social understanding.
- 10 August 2026
- Published by Springer Nature in KI - Künstliche Intelligenz
- p. 1-20
Open AccessAgents Through the Ages
Over several decades, research on socially interactive agents has transformed human–machine interaction, evolving from early script-based systems to today’s generative, multimodal agents. Tracing this development through the ages, this article reflects on major conceptual and technical shifts that shaped the field, drawing on advances across multiple areas of AI, and discusses how progress in natural language processing and multimodal behavior analysis and generation has redefined AI as a social interface. Rather than presenting this trajectory as a narrative of continuous progress, we also examine persistent limitations, including weak grounding, difficult evaluation, opaque decision processes, safety risks in sensitive contexts, and the gap between convincing social simulation and genuine social understanding.
- Open Access
- 13 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-5
What Happens When a Virtual Character Speaks With the Cloned Voice of a Person You Know?
What happens when a virtual character speaks with the cloned voice of a person you know? Our results suggest that such a known-voice clone can subtly shape social perception, but does not reliably transfer the perceived personality of the original speaker to the virtual character. We investigate whether voice cloning can shape the attribution of personality and social traits to a virtual character when that character speaks with a cloned version of a known voice. Across two studies, we compared ratings of an original voice speaker with ratings of a virtual character speaking with either a known-voice clone or an unknown voice. The first study was a pilot in which students were exposed to the voice of their lecturer. The second study was a preregistered follow-up in which content creators’ voices were presented to their respective communities. Participants evaluated personality traits and social characteristics, and we measured how similar these ratings were to those of the original speaker. Quantitative results revealed small and selective effects. In the pilot study, differences emerged for likability and agreeableness, while in the follow-up, effects were observed for extraversion. Qualitative analyses showed a shift in focus away from perceptions of the salience of artificiality and unnaturalness toward the incongruence between voice and visual appearance. Overall, the findings indicate that known-voice cloning is not a general mechanism for personality transfer. Instead, it functions as a weak and context-dependent cue whose influence is constrained by multimodal coherence and users’ expectations. These results provide practical guidance on when known-voice cloning may be beneficial, when it may backfire, and which aspects should be prioritized in the design of embodied voice interfaces.
- 13 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 1-5
Open AccessWhat Happens When a Virtual Character Speaks With the Cloned Voice of a Person You Know?
What happens when a virtual character speaks with the cloned voice of a person you know? Our results suggest that such a known-voice clone can subtly shape social perception, but does not reliably transfer the perceived personality of the original speaker to the virtual character. We investigate whether voice cloning can shape the attribution of personality and social traits to a virtual character when that character speaks with a cloned version of a known voice. Across two studies, we compared ratings of an original voice speaker with ratings of a virtual character speaking with either a known-voice clone or an unknown voice. The first study was a pilot in which students were exposed to the voice of their lecturer. The second study was a preregistered follow-up in which content creators’ voices were presented to their respective communities. Participants evaluated personality traits and social characteristics, and we measured how similar these ratings were to those of the original speaker. Quantitative results revealed small and selective effects. In the pilot study, differences emerged for likability and agreeableness, while in the follow-up, effects were observed for extraversion. Qualitative analyses showed a shift in focus away from perceptions of the salience of artificiality and unnaturalness toward the incongruence between voice and visual appearance. Overall, the findings indicate that known-voice cloning is not a general mechanism for personality transfer. Instead, it functions as a weak and context-dependent cue whose influence is constrained by multimodal coherence and users’ expectations. These results provide practical guidance on when known-voice cloning may be beneficial, when it may backfire, and which aspects should be prioritized in the design of embodied voice interfaces.
- Open Access
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 14150-14155
MultiMediate '25: Cross-cultural Multi-domain Engagement Estimation
- 27 October 2025
- Published by Association for Computing Machinery (ACM)
- p. 14150-14155
Open AccessMultiMediate '25: Cross-cultural Multi-domain Engagement Estimation
- Open Access
- 16 September 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-4
VoiceX as a Design Tool for Virtual Agents' Voices
- 16 September 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-4
Open AccessVoiceX as a Design Tool for Virtual Agents' Voices
- Open Access
- 24 March 2025
- Published by Association for Computing Machinery (ACM)
- p. 163-166
MeLaX: Conversations with Generative AI in Socially Interactive Agents
- 24 March 2025
- Published by Association for Computing Machinery (ACM)
- p. 163-166
Open AccessMeLaX: Conversations with Generative AI in Socially Interactive Agents
- Open Access
- 4 November 2024
- Published by Association for Computing Machinery (ACM)
- p. 428-438
Towards Automated Annotation of Infant-Caregiver Engagement Phases with Multimodal Foundation Models
Caregiver mental health disorders increase the risk of insecure infant attachment and can negatively impact multiple aspects of child development, including cognitive, emotional, and social growth. Infant-caregiver interactions contain subtle psychological and behavioral cues that reveal these adverse effects, underscoring the need for analytical methods to assess them effectively. The Face-to-Face-Still-Face (FFSF) paradigm is a key approach in psychological research for investigating these dynamics, and the Infant and Caregiver Engagement Phases revised German edition (ICEP-R) annotation scheme provides a structured framework for evaluating FFSF interactions. However, manual annotation is labor-intensive and limits scalability, thus hindering a deeper understanding of early developmental impairments. To address this, we developed a computational method that automates the annotation of caregiver-infant interactions using features extracted from audio-visual foundational models. Our approach was tested on 92 FFSF video sessions. Findings demonstrate that models based on bidirectional LSTM and linear classifiers show varying effectiveness depending on the role and feature modality. Specifically, bidirectional LSTM models generally perform better in predicting complex infant engagement phases across multimodal features, while linear models show competitive performance, particularly with unimodal feature encodings like Wav2Vec2-BERT. To support further research, we share our raw feature dataset annotated with ICEP-R labels, enabling broader refinement of computational methods in this area.
- 4 November 2024
- Published by Association for Computing Machinery (ACM)
- p. 428-438
Open AccessTowards Automated Annotation of Infant-Caregiver Engagement Phases with Multimodal Foundation Models
Caregiver mental health disorders increase the risk of insecure infant attachment and can negatively impact multiple aspects of child development, including cognitive, emotional, and social growth. Infant-caregiver interactions contain subtle psychological and behavioral cues that reveal these adverse effects, underscoring the need for analytical methods to assess them effectively. The Face-to-Face-Still-Face (FFSF) paradigm is a key approach in psychological research for investigating these dynamics, and the Infant and Caregiver Engagement Phases revised German edition (ICEP-R) annotation scheme provides a structured framework for evaluating FFSF interactions. However, manual annotation is labor-intensive and limits scalability, thus hindering a deeper understanding of early developmental impairments. To address this, we developed a computational method that automates the annotation of caregiver-infant interactions using features extracted from audio-visual foundational models. Our approach was tested on 92 FFSF video sessions. Findings demonstrate that models based on bidirectional LSTM and linear classifiers show varying effectiveness depending on the role and feature modality. Specifically, bidirectional LSTM models generally perform better in predicting complex infant engagement phases across multimodal features, while linear models show competitive performance, particularly with unimodal feature encodings like Wav2Vec2-BERT. To support further research, we share our raw feature dataset annotated with ICEP-R labels, enabling broader refinement of computational methods in this area.
- Open Access
- 9 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 507-516
ReNeLiB: Real-time Neural Listening Behavior Generation for Socially Interactive Agents
Flexible and natural nonverbal reactions to human behavior remain a challenge for socially interactive agents (SIAs) that are predominantly animated using hand-crafted rules. While recently proposed machine learning based approaches to conversational behavior generation are a promising way to address this challenge, they have not yet been employed in SIAs. The primary reason for this is the lack of a software toolkit integrating such approaches with SIA frameworks that conforms to the challenging real-time requirements of human-agent interaction scenarios. In our work, we for the first time present such a toolkit consisting of three main components: (1) real-time feature extraction capturing multi-modal social cues from the user; (2) behavior generation based on a recent state-of-the-art neural network approach; (3) visualization of the generated behavior supporting both FLAME-based and Apple ARKit-based interactive agents. We comprehensively evaluate the real-time performance of the whole framework and its components. In addition, we introduce pre-trained behavioral generation models derived from psychotherapy sessions for domain-specific listening behaviors. Our software toolkit, pivotal for deploying and assessing SIAs’ listening behavior in real-time, is publicly available. Resources, including code, behavioural multi-modal features extracted from therapeutic interactions, are hosted at https://daksitha.github.io/ReNeLib
- 9 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 507-516
Open AccessReNeLiB: Real-time Neural Listening Behavior Generation for Socially Interactive Agents
Flexible and natural nonverbal reactions to human behavior remain a challenge for socially interactive agents (SIAs) that are predominantly animated using hand-crafted rules. While recently proposed machine learning based approaches to conversational behavior generation are a promising way to address this challenge, they have not yet been employed in SIAs. The primary reason for this is the lack of a software toolkit integrating such approaches with SIA frameworks that conforms to the challenging real-time requirements of human-agent interaction scenarios. In our work, we for the first time present such a toolkit consisting of three main components: (1) real-time feature extraction capturing multi-modal social cues from the user; (2) behavior generation based on a recent state-of-the-art neural network approach; (3) visualization of the generated behavior supporting both FLAME-based and Apple ARKit-based interactive agents. We comprehensively evaluate the real-time performance of the whole framework and its components. In addition, we introduce pre-trained behavioral generation models derived from psychotherapy sessions for domain-specific listening behaviors. Our software toolkit, pivotal for deploying and assessing SIAs’ listening behavior in real-time, is publicly available. Resources, including code, behavioural multi-modal features extracted from therapeutic interactions, are hosted at https://daksitha.github.io/ReNeLib