Alexander Heimerl
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Research Interests
Alexander Heimerl has a total of 36 co-authors. Here are the top co-authors by number of shared publications:
- Elisabeth André
Universitat Augsburg, Augsburg, Germany
8 shared publications - Tobias Baur
University of Augsburg, Augsburg, Germany
4 shared publications - Andreas Bulling
University of Stuttgart, Stuttgart, Germany
2 shared publications - Dominik Schiller
University of Augsburg, Augsburg, Germany
2 shared publications - Francois Bremond
INRIA Sophia Antipolis, Sophia Antipolis, France
2 shared publications - Jan Alexandersson
DFKI, Saarbrücken, Germany
2 shared publications
Latest Publications
- 1 November 2025
- Published by IATED Academy in ICERI Proceedings
- Vol. 1, 1193-1202
INTERACTIVE MACHINE LEARNING TO ENHANCE STUDENTS' UNDERSTANDING AND SATISFACTION APPLYING THE LEARNING BY DOING METHODOLOGY
- 1 November 2025
- Published by IATED Academy in ICERI Proceedings
- Vol. 1, 1193-1202
INTERACTIVE MACHINE LEARNING TO ENHANCE STUDENTS' UNDERSTANDING AND SATISFACTION APPLYING THE LEARNING BY DOING METHODOLOGY
- Open Access
- 18 November 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 16 (2) , 1219-1234
The ForDigitStress Dataset: A Multi-Modal Dataset for Automatic Stress Recognition
We present a multi-modal stress dataset that uses digital job interviews to induce stress. The dataset provides multi-modal data of 40 participants including audio, video (motion capturing, facial landmarks, eye tracking) as well as physiological information (photoplethysmography, electrodermal activity). In addition to that, the dataset contains time-continuous annotations for stress and occurred emotions (e.g., shame, anger, anxiety, and surprise). In order to establish a baseline, five different machine learning classifiers (Support Vector Machine, K-Nearest Neighbors, Random Forest, Feed-forward Neural Network, and Long-Short-Term Memory Network) have been trained and evaluated on the presented dataset for a binary stress classification task. The best-performing classifier has been a Long-Short-Term Memory Network, which achieved an accuracy of 91.7% and an F1-score of 90.2%. The ForDigitStress dataset is freely available to other researchers.
- 18 November 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 16 (2) , 1219-1234
Open AccessThe ForDigitStress Dataset: A Multi-Modal Dataset for Automatic Stress Recognition
We present a multi-modal stress dataset that uses digital job interviews to induce stress. The dataset provides multi-modal data of 40 participants including audio, video (motion capturing, facial landmarks, eye tracking) as well as physiological information (photoplethysmography, electrodermal activity). In addition to that, the dataset contains time-continuous annotations for stress and occurred emotions (e.g., shame, anger, anxiety, and surprise). In order to establish a baseline, five different machine learning classifiers (Support Vector Machine, K-Nearest Neighbors, Random Forest, Feed-forward Neural Network, and Long-Short-Term Memory Network) have been trained and evaluated on the presented dataset for a binary stress classification task. The best-performing classifier has been a Long-Short-Term Memory Network, which achieved an accuracy of 91.7% and an F1-score of 90.2%. The ForDigitStress dataset is freely available to other researchers.
- Open Access
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 11377-11382
MultiMediate'24: Multi-Domain Engagement Estimation
Estimating the momentary level of participant's engagement is an important prerequisite for assistive systems that support human interactions. Previous work has addressed this task in within-domain evaluation scenarios, i.e. training and testing on the same dataset. This is in contrast to real-life scenarios where domain shifts between training and testing data frequently occur. With MultiMediate'24, we present the first challenge addressing multi-domain engagement estimation. As training data, we utilise the NOXI database of dyadic novice-expert interactions. In addition to within-domain test data, we add two new test domains. First, we introduce recordings following the NOXI protocol but covering languages that are not present in the NOXI training data. Second, we collected novel engagement annotations on the MPIIGroupInteraction dataset which consists of group discussions between three to four people. In this way, MultiMediate'24 evaluates the ability of approaches to generalise across factors such as language and cultural background, group size, task, and screen-mediated vs. face-to-face interaction. This paper describes the MultiMediate'24 challenge and presents baseline results. In addition, we discuss selected challenge solutions.
- 28 October 2024
- Published by Association for Computing Machinery (ACM)
- p. 11377-11382
Open AccessMultiMediate'24: Multi-Domain Engagement Estimation
Estimating the momentary level of participant's engagement is an important prerequisite for assistive systems that support human interactions. Previous work has addressed this task in within-domain evaluation scenarios, i.e. training and testing on the same dataset. This is in contrast to real-life scenarios where domain shifts between training and testing data frequently occur. With MultiMediate'24, we present the first challenge addressing multi-domain engagement estimation. As training data, we utilise the NOXI database of dyadic novice-expert interactions. In addition to within-domain test data, we add two new test domains. First, we introduce recordings following the NOXI protocol but covering languages that are not present in the NOXI training data. Second, we collected novel engagement annotations on the MPIIGroupInteraction dataset which consists of group discussions between three to four people. In this way, MultiMediate'24 evaluates the ability of approaches to generalise across factors such as language and cultural background, group size, task, and screen-mediated vs. face-to-face interaction. This paper describes the MultiMediate'24 challenge and presents baseline results. In addition, we discuss selected challenge solutions.
- Open Access
- 18 September 2024
- Published by Association for Computing Machinery (ACM)
- p. 348-354
A Machine Learning-Driven Interactive Training System for Extreme Vocal Techniques
The scarcity of vocal instructors proficient in extreme vocal techniques and the lack of individualized feedback present challenges for novices learning these techniques. Therefore, this work explores the use of neural networks to provide real-time feedback for extreme vocal techniques within an interactive training system. An Extreme-Vocal dataset created for this purpose served as the basis for training a model capable of classifying False-Cord screams, Fry screams, and a residual class. The neural network achieved an overall accuracy of 0.83. We integrated the model into a user application to enable real-time visualization of classification results. By conducting a first qualitative user study involving 12 participants, we investigated whether interacting with the training system could enhance self-efficacy regarding the correct application of extreme vocal techniques. Our study participants indicated that they found the training system helpful for learning and categorizing extreme vocal techniques.
- 18 September 2024
- Published by Association for Computing Machinery (ACM)
- p. 348-354
Open AccessA Machine Learning-Driven Interactive Training System for Extreme Vocal Techniques
The scarcity of vocal instructors proficient in extreme vocal techniques and the lack of individualized feedback present challenges for novices learning these techniques. Therefore, this work explores the use of neural networks to provide real-time feedback for extreme vocal techniques within an interactive training system. An Extreme-Vocal dataset created for this purpose served as the basis for training a model capable of classifying False-Cord screams, Fry screams, and a residual class. The neural network achieved an overall accuracy of 0.83. We integrated the model into a user application to enable real-time visualization of classification results. By conducting a first qualitative user study involving 12 participants, we investigated whether interacting with the training system could enhance self-efficacy regarding the correct application of extreme vocal techniques. Our study participants indicated that they found the training system helpful for learning and categorizing extreme vocal techniques.
- Open Access
- 26 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 9640-9645
MultiMediate '23: Engagement Estimation and Bodily Behaviour Recognition in Social Interactions
Automatic analysis of human behaviour is a fundamental prerequisite for the creation of machines that can effectively interact with- and support humans in social interactions. In MultiMediate'23, we address two key human social behaviour analysis tasks for the first time in a controlled challenge: engagement estimation and bodily behaviour recognition in social interactions. This paper describes the MultiMediate'23 challenge and presents novel sets of annotations for both tasks. For engagement estimation we collected novel annotations on the NOvice eXpert Interaction (NOXI) database. For bodily behaviour recognition, we annotated test recordings of the MPIIGroupInteraction corpus with the BBSI annotation scheme. In addition, we present baseline results for both challenge tasks.
- 26 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 9640-9645
Open AccessMultiMediate '23: Engagement Estimation and Bodily Behaviour Recognition in Social Interactions
Automatic analysis of human behaviour is a fundamental prerequisite for the creation of machines that can effectively interact with- and support humans in social interactions. In MultiMediate'23, we address two key human social behaviour analysis tasks for the first time in a controlled challenge: engagement estimation and bodily behaviour recognition in social interactions. This paper describes the MultiMediate'23 challenge and presents novel sets of annotations for both tasks. For engagement estimation we collected novel annotations on the NOvice eXpert Interaction (NOXI) database. For bodily behaviour recognition, we annotated test recordings of the MPIIGroupInteraction corpus with the BBSI annotation scheme. In addition, we present baseline results for both challenge tasks.
- 24 July 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 15 (2) , 417-432
The Deep Method: Towards Computational Modeling of the Social Emotion Shame Driven by Theory, Introspection, and Social Signals
Understanding emotions is key to Affective Computing. Emotion recognition focuses on the communicative component of emotions encoded in social signals. This view alone is insufficient for a deeper understanding and computational representation of the internal, subjectively experienced component of emotions. This paper presents a cognition-based method called Deep as a starting point for deeper computational modeling of the internal component of emotions. Deep incorporates an approach to query individual internal emotional experiences and to represent such information computationally. It combines social signals, verbalized introspection information, context information, and theory-driven knowledge. We apply the Deep method to the emotion of shame as an example and compare it to a typical emotion recognition model, highlighting the differences and advantages.
- 24 July 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 15 (2) , 417-432
The Deep Method: Towards Computational Modeling of the Social Emotion Shame Driven by Theory, Introspection, and Social Signals
Understanding emotions is key to Affective Computing. Emotion recognition focuses on the communicative component of emotions encoded in social signals. This view alone is insufficient for a deeper understanding and computational representation of the internal, subjectively experienced component of emotions. This paper presents a cognition-based method called Deep as a starting point for deeper computational modeling of the internal component of emotions. Deep incorporates an approach to query individual internal emotional experiences and to represent such information computationally. It combines social signals, verbalized introspection information, context information, and theory-driven knowledge. We apply the Deep method to the emotion of shame as an example and compare it to a typical emotion recognition model, highlighting the differences and advantages.
- Open Access
- 9 December 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 13 (3) , 1155-1167
Unraveling ML Models of Emotion With NOVA: Multi-Level Explainable AI for Non-Experts
In this article, we introduce a next-generation annotation tool called NOVA for emotional behaviour analysis, which implements a workflow that interactively incorporates the ‘human in the loop’. A main aspect of NOVA is the possibility of applying semi-supervised active learning where Machine Learning techniques are used already during the annotation process by giving the possibility to pre-label data automatically. Furthermore, NOVA implements recent eXplainable AI (XAI) techniques to provide users with both, a confidence value of the automatically predicted annotations, as well as visual explanations. We investigate how such techniques can assist non-experts in terms of trust, perceived self-efficacy, cognitive workload as well as creating correct mental models about the system by conducting a user study with 53 participants. The results show that NOVA can easily be used by non-experts and lead to a high computer self-efficacy. Furthermore, the results indicate that XAI visualisations help users to create more correct mental models about the machine learning system compared to the baseline condition. Nevertheless, we suggest that explanations in the field of AI have to be more focused on user-needs as well as on the classification task and the model they want to explain.
- 9 December 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 13 (3) , 1155-1167
Open AccessUnraveling ML Models of Emotion With NOVA: Multi-Level Explainable AI for Non-Experts
In this article, we introduce a next-generation annotation tool called NOVA for emotional behaviour analysis, which implements a workflow that interactively incorporates the ‘human in the loop’. A main aspect of NOVA is the possibility of applying semi-supervised active learning where Machine Learning techniques are used already during the annotation process by giving the possibility to pre-label data automatically. Furthermore, NOVA implements recent eXplainable AI (XAI) techniques to provide users with both, a confidence value of the automatically predicted annotations, as well as visual explanations. We investigate how such techniques can assist non-experts in terms of trust, perceived self-efficacy, cognitive workload as well as creating correct mental models about the system by conducting a user study with 53 participants. The results show that NOVA can easily be used by non-experts and lead to a high computer self-efficacy. Furthermore, the results indicate that XAI visualisations help users to create more correct mental models about the machine learning system compared to the baseline condition. Nevertheless, we suggest that explanations in the field of AI have to be more focused on user-needs as well as on the classification task and the model they want to explain.
- Open Access
- 7 July 2020
- Published by Springer Nature
- Vol. 12175, 113-128
Towards Demystifying Subliminal Persuasiveness: Using XAI-Techniques to Highlight Persuasive Markers of Public Speeches
The literature provides evidence for the importance of non-verbal cues when it comes to persuading other people and developing persuasive robots. Mostly, people use these non-verbal cues subconsciously and, more importantly, are not aware of the subliminal impact of them. To raise awareness of subliminal persuasion and to explore a way for investigating persuasive cues for the development of persuasive robots and agents, we have analyzed videos of political public speeches and trained a neural network capable of predicting the degree of perceived convincingness based on visual input only. We then created visualizations of the predictions by making use of the explainable artificial intelligence methods Grad-CAM and layer-wise relevance propagation that highlight the most relevant image sections and markers. Our results show that the neural network learned to focus on the person, more specifically their posture and contours, as well as on their hands and face. These results are in line with existing literature and, thus, show the practical potential of our approach.
- 7 July 2020
- Published by Springer Nature
- Vol. 12175, 113-128
Open AccessTowards Demystifying Subliminal Persuasiveness: Using XAI-Techniques to Highlight Persuasive Markers of Public Speeches
The literature provides evidence for the importance of non-verbal cues when it comes to persuading other people and developing persuasive robots. Mostly, people use these non-verbal cues subconsciously and, more importantly, are not aware of the subliminal impact of them. To raise awareness of subliminal persuasion and to explore a way for investigating persuasive cues for the development of persuasive robots and agents, we have analyzed videos of political public speeches and trained a neural network capable of predicting the degree of perceived convincingness based on visual input only. We then created visualizations of the predictions by making use of the explainable artificial intelligence methods Grad-CAM and layer-wise relevance propagation that highlight the most relevant image sections and markers. Our results show that the neural network learned to focus on the person, more specifically their posture and contours, as well as on their hands and face. These results are in line with existing literature and, thus, show the practical potential of our approach.