Iain Matthews
Iain Matthews has a total of 9 co-authors. Here are the top co-authors by number of shared publications:
- Jonathan Windle
Epic Games, London, United Kingdom
3 shared publications - Sarah Taylor
School of Computing Sciences University of East Anglia Norwich UK
3 shared publications A. Bangham
School of Computing Sciences
1 shared publicationB.-J. Theobald
School of Computing Sciences
1 shared publication- Ben Milner
University of East Anglia, England
1 shared publication - Erica Alcusa Sáez
Epic Games, Zurich, Switzerland
1 shared publication
Latest Publications
- Open Access
- 27 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-11
xADA: Controllable and Expressive Audio-Driven Animation
- 27 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-11
Open AccessxADA: Controllable and Expressive Audio-Driven Animation
- Open Access
- 17 October 2024
- Published by Wiley in Computer Graphics Forum
- Vol. 43 (8)
Llanimation: Llama Driven Gesture Animation
Co‐speech gesturing is an important modality in conversation, providing context and social cues. In character animation, appropriate and synchronised gestures add realism, and can make interactive agents more engaging. Historically, methods for automatically generating gestures were predominantly audio‐driven, exploiting the prosodic and speech‐related content that is encoded in the audio signal. In this paper we instead experiment with using Large‐Language Model (LLM) features for gesture generation that are extracted from text using Llama2. We compare against audio features, and explore combining the two modalities in both objective tests and a user study. Surprisingly, our results show that Llama2 features on their own perform significantly better than audio features and that including both modalities yields no significant difference to using Llama2 features in isolation. We demonstrate that the Llama2 based model can generate both beat and semantic gestures without any audio input, suggesting LLMs can provide rich encodings that are well suited for gesture generation.
- 17 October 2024
- Published by Wiley in Computer Graphics Forum
- Vol. 43 (8)
Open AccessLlanimation: Llama Driven Gesture Animation
Co‐speech gesturing is an important modality in conversation, providing context and social cues. In character animation, appropriate and synchronised gestures add realism, and can make interactive agents more engaging. Historically, methods for automatically generating gestures were predominantly audio‐driven, exploiting the prosodic and speech‐related content that is encoded in the audio signal. In this paper we instead experiment with using Large‐Language Model (LLM) features for gesture generation that are extracted from text using Llama2. We compare against audio features, and explore combining the two modalities in both objective tests and a user study. Surprisingly, our results show that Llama2 features on their own perform significantly better than audio features and that including both modalities yields no significant difference to using Llama2 features in isolation. We demonstrate that the Llama2 based model can generate both beat and semantic gestures without any audio input, suggesting LLMs can provide rich encodings that are well suited for gesture generation.
- Open Access
- 9 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 802-810
The UEA Digital Humans entry to the GENEA Challenge 2023
This paper describes our entry to the GENEA (Generation and Evaluation of Non-verbal Behaviour for Embodied Agents) Challenge 2023. This year’s challenge focuses on generating gestures in a dyadic setting – predicting a main-agent’s motion from the speech of both the main-agent and an interlocutor. We adapt a Transformer-XL architecture for this task by adding a cross-attention module that integrates the interlocutor’s speech with that of the main-agent. Our model is conditioned on speech audio (encoded using PASE+), text (encoded using FastText) and a speaker identity label, and is able to generate smooth and speech appropriate gestures for a given identity. We consider the GENEA Challenge user study results and present a discussion of our model strengths and where improvements can be made.
- 9 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 802-810
Open AccessThe UEA Digital Humans entry to the GENEA Challenge 2023
This paper describes our entry to the GENEA (Generation and Evaluation of Non-verbal Behaviour for Embodied Agents) Challenge 2023. This year’s challenge focuses on generating gestures in a dyadic setting – predicting a main-agent’s motion from the speech of both the main-agent and an interlocutor. We adapt a Transformer-XL architecture for this task by adding a cross-attention module that integrates the interlocutor’s speech with that of the main-agent. Our model is conditioned on speech audio (encoded using PASE+), text (encoded using FastText) and a speaker identity label, and is able to generate smooth and speech appropriate gestures for a given identity. We consider the GENEA Challenge user study results and present a discussion of our model strengths and where improvements can be made.
- 1 January 2003
- Published by British Machine Vision Association and Society for Pattern Recognition
- p. 3.1-3.10
2.5D Visual Speech Synthesis Using Appearance Models
- 1 January 2003
- Published by British Machine Vision Association and Society for Pattern Recognition
- p. 3.1-3.10
2.5D Visual Speech Synthesis Using Appearance Models