Multi-Order Networks for Action Unit Detection
- 2 June 2022
- journal article
- research article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 14 (4) , 2876-2888
Abstract
Action Units (AU) are muscular activations used to describe facial expressions. Therefore accurate AU recognition unlocks unbiaised face representation which can improve face-based affective computing applications. From a learning standpoint AU detection is a multi-task problem with strong inter-task dependencies. To solve such problem, most approaches either rely on weight sharing, or add explicit dependency modelling by decomposing the joint task distribution using Bayes chain rule. If the latter strategy yields comprehensive inter-task relationships modelling, it requires imposing an arbitrary order into an unordered task set. Crucially, this ordering choice has been identified as a source of performance variations. In this paper, we present Multi-Order Network (MONET), a multi-task method with joint task order optimization. MONET uses a differentiable order selection to jointly learn task-wise modules with their optimal chaining order. Furthermore, we introduce warmup and order dropout to enhance order selection by encouraging order exploration. Experimentally, we first demonstrate MONET capacity to retrieve the optimal order in a toy environment. Second, we validate MONET architecture by showing that MONET outperforms existing multi-task baselines on multiple attribute detection problems chosen for their wide range of dependency settings. More importantly, we demonstrate that MONET significantly extends state-of-the-art performance in AU detection.Keywords
All Related Versions
Funding Information
- French National Agency (ANR-17-CE33-0002, ANR-11-IDEX-0004-02)
- IDRIS (AD011013183)
This publication has 28 references indexed in Scilit:
- HyperFace: A Deep Multi-Task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender RecognitionIEEE Transactions on Pattern Analysis and Machine Intelligence, 2017
- UberNet: Training a Universal Convolutional Neural Network for Low-, Mid-, and High-Level Vision Using Diverse Datasets and Limited Memory2017
- EAC-Net: A Region-Based Deep Enhancing and Cropping Approach for Facial Action Unit Detection2017
- Expression-assisted facial action unit recognition under incomplete AU annotationPattern Recognition, 2017
- Deep Region and Multi-label Learning for Facial Action Unit Detection2016
- Cross-Stitch Networks for Multi-task Learning2016
- Deep multi-task learning with low level tasks supervised at lower layers2016
- Deep neural networks employing Multi-Task Learning and stacked bottleneck features for speech synthesis2015
- Multi-Task Learning for Multiple Language Translation2015
- Facial Landmark Detection by Deep Multi-task Learning2014