Jonathan Windle
Jonathan Windle has a total of 8 co-authors. Here are the top co-authors by number of shared publications:
- Iain Matthews
Epic Games, Pittsburgh, USA
3 shared publications - Sarah Taylor
School of Computing Sciences University of East Anglia Norwich UK
3 shared publications - Ben Milner
University of East Anglia, England
1 shared publication David Greenwood
Univ East Anglia, Norwich, Norfolk, England
1 shared publication- Erica Alcusa Sáez
Epic Games, Zurich, Switzerland
1 shared publication Iain Matthews
Univ East Anglia, Norwich, Norfolk, England
1 shared publication
Latest Publications
- Open Access
- 27 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-11
xADA: Controllable and Expressive Audio-Driven Animation
- 27 July 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-11
Open AccessxADA: Controllable and Expressive Audio-Driven Animation
- Open Access
- 17 October 2024
- Published by Wiley in Computer Graphics Forum
- Vol. 43 (8)
Llanimation: Llama Driven Gesture Animation
Co‐speech gesturing is an important modality in conversation, providing context and social cues. In character animation, appropriate and synchronised gestures add realism, and can make interactive agents more engaging. Historically, methods for automatically generating gestures were predominantly audio‐driven, exploiting the prosodic and speech‐related content that is encoded in the audio signal. In this paper we instead experiment with using Large‐Language Model (LLM) features for gesture generation that are extracted from text using Llama2. We compare against audio features, and explore combining the two modalities in both objective tests and a user study. Surprisingly, our results show that Llama2 features on their own perform significantly better than audio features and that including both modalities yields no significant difference to using Llama2 features in isolation. We demonstrate that the Llama2 based model can generate both beat and semantic gestures without any audio input, suggesting LLMs can provide rich encodings that are well suited for gesture generation.
- 17 October 2024
- Published by Wiley in Computer Graphics Forum
- Vol. 43 (8)
Open AccessLlanimation: Llama Driven Gesture Animation
Co‐speech gesturing is an important modality in conversation, providing context and social cues. In character animation, appropriate and synchronised gestures add realism, and can make interactive agents more engaging. Historically, methods for automatically generating gestures were predominantly audio‐driven, exploiting the prosodic and speech‐related content that is encoded in the audio signal. In this paper we instead experiment with using Large‐Language Model (LLM) features for gesture generation that are extracted from text using Llama2. We compare against audio features, and explore combining the two modalities in both objective tests and a user study. Surprisingly, our results show that Llama2 features on their own perform significantly better than audio features and that including both modalities yields no significant difference to using Llama2 features in isolation. We demonstrate that the Llama2 based model can generate both beat and semantic gestures without any audio input, suggesting LLMs can provide rich encodings that are well suited for gesture generation.
- Open Access
- 9 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 802-810
The UEA Digital Humans entry to the GENEA Challenge 2023
This paper describes our entry to the GENEA (Generation and Evaluation of Non-verbal Behaviour for Embodied Agents) Challenge 2023. This year’s challenge focuses on generating gestures in a dyadic setting – predicting a main-agent’s motion from the speech of both the main-agent and an interlocutor. We adapt a Transformer-XL architecture for this task by adding a cross-attention module that integrates the interlocutor’s speech with that of the main-agent. Our model is conditioned on speech audio (encoded using PASE+), text (encoded using FastText) and a speaker identity label, and is able to generate smooth and speech appropriate gestures for a given identity. We consider the GENEA Challenge user study results and present a discussion of our model strengths and where improvements can be made.
- 9 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 802-810
Open AccessThe UEA Digital Humans entry to the GENEA Challenge 2023
This paper describes our entry to the GENEA (Generation and Evaluation of Non-verbal Behaviour for Embodied Agents) Challenge 2023. This year’s challenge focuses on generating gestures in a dyadic setting – predicting a main-agent’s motion from the speech of both the main-agent and an interlocutor. We adapt a Transformer-XL architecture for this task by adding a cross-attention module that integrates the interlocutor’s speech with that of the main-agent. Our model is conditioned on speech audio (encoded using PASE+), text (encoded using FastText) and a speaker identity label, and is able to generate smooth and speech appropriate gestures for a given identity. We consider the GENEA Challenge user study results and present a discussion of our model strengths and where improvements can be made.
- Open Access
- 29 August 2022
- Published by Elsevier in Speech Communication
- Vol. 144, 75-88
Arm motion symmetry in conversation
- 29 August 2022
- Published by Elsevier in Speech Communication
- Vol. 144, 75-88
Open AccessArm motion symmetry in conversation