PrimateFace: A Machine Learning Resource for Automated Face Analysis in Human and Non-human Primates
Authors:
- Felipe ParodiFPFelipe Parodi
- Jordan K. MatelskyJMJordan K. Matelsky
- Alessandro P. LamacchiaALAlessandro P. Lamacchia
- Melanie SegadoMSMelanie Segado
- Yaoguang JiangYJYaoguang Jiang
- Alejandra Regla-VargasARAlejandra Regla-Vargas
- Liala SofiLSLiala Sofi
- Clare KimockCKClare Kimock
- Bridget M. WallerBWBridget M. Waller
- Michael L. PlattMPMichael L. Platt
- Konrad P. KordingKKKonrad P. Kording
Preprint
Abstract
Machine learning has revolutionized human face analysis, but equivalent tools for non-human primates remain limited and species-specific, hindering progress in neuroscience, anthropology, and conservation. Here, we present PrimateFace, a comprehensive, cross-species platform for primate facial analysis comprising a systematically curated dataset of 260,000+ images spanning over 60 genera, including a genus-balanced subset of 60,000 images, annotated with bounding boxes and facial landmark configurations. Face detection and facial landmark estimation models trained on PrimateFace achieve high cross-species performance, from tarsiers to gorillas, achieving performance comparable to baseline models trained exclusively on human data (0.34 vs. 0.39 mAP for face detection; 0.061 vs. 0.053 normalized landmark error), demonstrating the generalization benefits of cross-species training. PrimateFace enables diverse downstream applications including individual recognition, gaze analysis, and automated extraction of stereotyped (e.g., lip-smacking) and subtle (e.g., soft left turn) facial movements. PrimateFace provides a standardized platform for facial phenotyping across the primate order, empowering data-driven studies that advance the health and well-being of human and non-human primates. All models, notebooks, and data can be found at github.com/KordingLab/PrimateFace .Keywords
This publication has 42 references indexed in Scilit:
- Grounding DINO: Marrying DINO with Grounded Pre-training for Open-Set Object Detection2024
- Deep learning for automatic facial detection and recognition in Japanese macaques: illuminating social networksPrimates, 2024
- Biomechanical and Cortical Control of Tongue Movements During Chewing and SwallowingDysphagia, 2023
- Self-supervised Secondary Landmark Detection via 3D Representation LearningInternational Journal of Computer Vision, 2023
- OpenMonkeyChallenge: Dataset and Benchmark Challenges for Pose Estimation of Non-human PrimatesInternational Journal of Computer Vision, 2022
- Whole-Body Human Pose Estimation in the Wild2020
- The Open Images Dataset V4International Journal of Computer Vision, 2020
- Chimpanzee Faces in the Wild: Log-Euclidean CNNs for Predicting Identities and Attributes of Primates2016
- Brief communication: MaqFACS: A muscle‐based facial movement coding system for the rhesus macaqueAmerican Journal of Physical Anthropology, 2010
- A Cross-species Comparison of Facial Morphology and Movement in Humans and Chimpanzees Using the Facial Action Coding System (FACS)Journal of Nonverbal Behavior, 2006