MTMani Kumar Tellamekala

Mani Kumar Tellamekala

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Mani Kumar Tellamekala has a total of 6 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Top Cited Papers

    COLD Fusion: Calibrated and Ordinal Latent Distribution Fusion for Uncertainty-Aware Multimodal Emotion Recognition

    by Mani Kumar Tellamekala, Shahin Amiriparian, Björn W. Schuller, Elisabeth André, Timo Giesbrecht, Michel Valstar

    Automatically recognising apparent emotions from face and voice is hard, in part because of various sources of uncertainty, including in the input data and the labels used in a machine learning framework. This paper introduces an uncertainty-aware multimodal fusion approach that quantifies modality-wise aleatoric or data uncertainty towards emotion prediction. We propose a novel fusion framework, in which latent distributions over unimodal temporal context are learned by constraining their variance. These variance constraints, Calibration and Ordinal Ranking, are designed such that the variance estimated for a modality can represent how informative the temporal context of that modality is w.r.t. emotion recognition. When well-calibrated, modality-wise uncertainty scores indicate how much their corresponding predictions are likely to differ from the ground truth labels. Well-ranked uncertainty scores allow the ordinal ranking of different frames across different modalities. To jointly impose both these constraints, we propose a softmax distributional matching loss. Our evaluation on AVEC 2019 CES, CMU-MOSEI, and IEMOCAP datasets shows that the proposed multimodal fusion method not only improves the generalisation performance of emotion recognition models and their predictive uncertainty estimates, but also makes the models robust to novel noise patterns encountered at test time.


  • Are 3D Face Shapes Expressive Enough for Recognising Continuous Emotions and Action Unit Intensities?

    by Mani Kumar Tellamekala, Ömer Sümer, Björn W. Schuller, Elisabeth André, Timo Giesbrecht, Michel Valstar

    Recognising continuous emotions and action unit (AU) intensities from face videos, requires a spatial and temporal understanding of expression dynamics. Existing works primarily rely on 2D face appearance features to extract such dynamics. This work focuses on a promising alternative based on parametric 3D face alignment models, which disentangle different factors of variation, including expression-induced shape variations. We aim to understand how expressive 3D face shapes are in estimating valence-arousal and AU intensities compared to the state-of-the-art 2D appearance-based models. We benchmark five recent 3D face models: ExpNet, 3DDFA-V2, RingNet, DECA, and EMOCA. In valence-arousal estimation, expression features of 3D face models consistently surpassed previous works and yielded an average concordance correlation of. 745 and. 574 on SEWA and AVEC 2019 CES corpora, respectively. We also study how 3D face shapes performed on AU intensity estimation on BP4D and DISFA datasets, and report that 3D face features were on par with 2D appearance features in recognising AUs 4, 6, 10, 12, and 25, but not the entire set of AUs. To understand this discrepancy, we conduct a correspondence analysis between valence-arousal and AUs, which points out that accurate prediction of valence-arousal may require the knowledge of only a few AUs.


  • Dimensional Affect Uncertainty Modelling for Apparent Personality Recognition

    by Mani Kumar Tellamekala, Timo Giesbrecht, Michel Valstar

    Despite achieving impressive performance, dimensional affect or emotion recognition from faces is largely based on uncertainty-unaware models that predict only point estimates. Modelling uncertainty is important to learn reliable facial emotion recognition models with the abilities to (a). holistically quantify predictive uncertainty estimates and (b). propagate those estimates to the benefit of downstream behavioural analysis tasks. In this work, we first quantify uncertainties in dimensional emotion recognition by adopting the framework of epistemic (model) and aleatoric (data) uncertainty categorisation. Then for evaluating the practical utility of uncertainty-aware emotion predictions, we introduce them in learning an important downstream task, apparent personality recognition. To this end, we ask two questions: how to effectively (a). use already known behavioural attributes (emotions) in a downstream task (personality recognition) and (b). summarise global temporal context from uncertainty-aware emotion predictions fused with image embeddings. Answering these questions, we learn a conditional latent variable model building on recently proposed neural latent variable models. Our experiments on two in-the-wild datasets, SEWA for emotion recognition and ChaLearn for personality recognition, demonstrate that fusion of epistemic and aleatoric emotion uncertainties significantly improves personality recognition performance, with $\sim$42% relative improvement in Pearson correlation coefficient, leading to a new state-of-the-art.


  • Modelling Stochastic Context of Audio-Visual Expressive Behaviour With Affective Processes

    by Mani Kumar Tellamekala, Timo Giesbrecht, Michel Valstar

    Recognising apparent emotion from audio-visual signals in naturalistic conditions remains an open problem. Existing methods that build on recurrent models, or in the modelling of contextual dependencies at the feature level using self-attention fail to model the long-term dependencies that subtly occur at different levels of abstraction. Affective Processes have emerged as a novel paradigm to the modelling of temporal dynamics through a probabilistic global latent variable that captures context and induces dependencies in the outputs, showing superior performance with little complexity. Despite its impressive results on visual data, Affective Processes remain unexplored in the domain of audio data, known to crucially influence the perception of emotions. In this paper, we first revisit and extend Affective Processes to the speech domain, identifying the key components and learning procedures for their efficient training. We then extend Affective Processes to audio-visual affect recognition, using modality-specific context encoders. Finally, we propose a novel application of Affective Processes in the domain of Cooperative Machine Learning for propagating affect labels in videos using sparse human supervision. We conduct extensive ablation studies, identifying the main components behind the success of Affective Processes, as well as comparisons against existing works in a variety of datasets.