InterAct: A Large-Scale Dataset of Dynamic, Expressive and Interactive Activities between Two People in Daily Scenarios 53
Open Access
- 8 August 2025
- journal article
- Published by Association for Computing Machinery (ACM) in Proceedings of the ACM on Computer Graphics and Interactive Techniques
- Vol. 8 (4) , 1-27
Abstract
We address the problem of accurate capture of interactive behaviors between two people in daily scenarios. Most previous works either only consider one person or solely focus on conversational gestures of two people, assuming the body orientation and/or position of each actor are constant or barely change over each interaction. In contrast, we propose to simultaneously model two people’s activities, and target objective-driven, dynamic, and semantically consistent interactions which often span longer duration and cover bigger space. To this end, we capture a new multi-modal dataset dubbed InterAct, which is composed of 241 motion sequences where two people perform a realistic and coherent scenario for one minute or longer over a complete interaction. For each sequence, two actors are assigned different roles and emotion labels, and collaborate to finish one task or conduct a common interaction activity. The audios, body motions, and facial expressions of both persons are captured. InterAct contains diverse and complex motions of individuals and interesting and relatively long-term interaction patterns barely seen before. We also demonstrate a simple yet effective diffusion-based method that estimates interactive face expressions and body motions of two people from speech inputs. Our method regresses the body motions in a hierarchical manner, and we also propose a novel fine-tuning mechanism to improve the lip accuracy of facial expressions. To facilitate further research, the data and code will be made public.Keywords
All Related Versions
Funding Information
- Innovation and Technology Fund, HKSAR (ITS/319/21FP)
This publication has 62 references indexed in Scilit:
- InterGen: Diffusion-Based Multi-human Motion Generation Under Complex InteractionsInternational Journal of Computer Vision, 2024
- BEAT: A Large-Scale Semantic and Emotional Multi-modal Dataset for Conversational Gestures Synthesis2022
- MEAD: A Large-Scale Audio-Visual Dataset for Emotional Talking-Face Generation2020
- Learning Individual Styles of Conversational Gesture2019
- Capture, Learning, and Synthesis of 3D Speaking Styles2019
- Self-report captures 27 distinct categories of emotion bridged by continuous gradientsProceedings of the National Academy of Sciences of the United States of America, 2017
- Facial animation based on context-dependent visemesComputers & Graphics, 2006
- MikeTalk: a talking facial display based on morphing visemes2002
- Face animation based on observed 3D speech dynamics2002
- Practical Parameterization of Rotations Using the Exponential MapJournal of Graphics Tools, 1998