Morpheus: A Neural-Driven Animatronic Face with Hybrid Actuation and Diverse Emotion Control
- 1 January 2025
- journal article
- Published by World Scientific Pub Co Pte Ltd in International Journal of Artificial Intelligence and Robotics Research
- Vol. 02
Abstract
Previous animatronic faces struggle to express emotions effectively due to hardware and software limitations. On the hardware side, earlier approaches either used rigid-driven mechanisms, which provided precise control but were difficult to design within constrained spaces, or tendon-driven mechanisms, which are more space-efficient but challenging to control. In contrast, we propose a hybrid actuation approach that combines the best of both worlds. The eyes and mouth– key areas for emotional expression — are controlled using rigid mechanisms for precise movement, while the nose and cheek, which convey subtle facial microexpressions, are driven by strings. This design allows us to build a compact yet versatile hardware platform capable of expressing a wide range of emotions. On the algorithmic side, our method introduces a self-modeling network that maps motor actions to facial landmarks, allowing us to automatically establish the relationship between blendshape coefficients for different facial expressions and the corresponding motor control signals through gradient backpropagation. We then train a neural network to map speech input to corresponding blendshape controls. With our method, we can generate distinct emotional expressions such as happiness, fear, disgust, and anger, from any given sentence, each with nuanced, emotion-specific control signals — a feature that has not been demonstrated in earlier systems. We release the hardware design and code at https://github.com/ZZongzheng0918/Morpheus-Hardware and https://github.com/ZZongzheng0918/Morpheus-Software .Keywords
This publication has 45 references indexed in Scilit:
- Expression snippet transformer for robust video-based facial expression recognitionPattern Recognition, 2023
- FaceFormer: Speech-Driven 3D Facial Animation with Transformers2022
- Audio-Driven Emotional Video Portraits2021
- Facially expressive humanoid robotic faceHardwareX, 2020
- Capture, Learning, and Synthesis of 3D Speaking Styles2019
- Learning human-like facial expressions for Android Phillip K. Dick2014
- A Survey on Perception Methods for Human–Robot Interaction in Social RobotsInternational Journal of Social Robotics, 2013
- HEFES: An Hybrid Engine for Facial Expressions Synthesis to control human-like androids and avatars2012
- Mechanical Design of Emotion Expression Humanoid Robot WE-4RII2008
- Emotion and sociable humanoid robotsInternational Journal of Human-Computer Studies, 2003