Sascha Gruss
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Sascha Gruss has a total of 46 co-authors. Here are the top co-authors by number of shared publications:
- Steffen Walter
University Hospital Ulm, Ulm, Germany
7 shared publications Steffen Walter
Medical Psychology Group, University Clinic, Ulm, Germany
4 shared publications- Ayoub Al-Hamadi
Neuro-Information Technology, Otto-von-Guericke-University Magdeburg, 39106 Magdeburg, Germany
3 shared publications - Friedhelm Schwenker
Institute of Neural Information Processing, Ulm University, James-Franck-Ring, 89081 Ulm, Germany
3 shared publications - Philipp Werner
University of Fribourg
3 shared publications Eberhard Barth
Department of Psychosomatic Medicine and Psychotherapy, University Hospital of Ulm, Ulm, Germany
2 shared publications
Latest Publications
- Open Access
Randomized Modality Mixing with Patchwise RBF Networks for Robust Multimodal Pain Recognition
Pain recognition based on multimodal physiological signals remains a challenge, not only because of the limited training data, but also due to the varying responses of individuals. In this article, we present a randomized modality mixing technique (Modmix) for multimodal data augmentation and a patchwise radial basis function (RBF) network designed to improve robustness in limited and highly heterogeneous data. Modmix generates new samples by randomly swapping modalities between existing data points, creating new data in a very simple but effective way. The RBF patch network divides the input into randomly selected, overlapping patches that capture local similarities between modalities. Each patch network is trained end-to-end using stochastic gradient descent. Moreover, the model’s performance is further improved by using multiple independently trained networks and combining them into a single decision. Experiments with the two different pain datasets X-ITE and BioVid were performed under limited training data conditions, where only approximately 30% of the original datasets were used for training. With both datasets the RBF patch network achieved significant improvements for a subset of subjects, resulting in a similar or even slightly better mean accuracy compared to competing related models such as random forest and support vector machine.
Open AccessRandomized Modality Mixing with Patchwise RBF Networks for Robust Multimodal Pain Recognition
Pain recognition based on multimodal physiological signals remains a challenge, not only because of the limited training data, but also due to the varying responses of individuals. In this article, we present a randomized modality mixing technique (Modmix) for multimodal data augmentation and a patchwise radial basis function (RBF) network designed to improve robustness in limited and highly heterogeneous data. Modmix generates new samples by randomly swapping modalities between existing data points, creating new data in a very simple but effective way. The RBF patch network divides the input into randomly selected, overlapping patches that capture local similarities between modalities. Each patch network is trained end-to-end using stochastic gradient descent. Moreover, the model’s performance is further improved by using multiple independently trained networks and combining them into a single decision. Experiments with the two different pain datasets X-ITE and BioVid were performed under limited training data conditions, where only approximately 30% of the original datasets were used for training. With both datasets the RBF patch network achieved significant improvements for a subset of subjects, resulting in a similar or even slightly better mean accuracy compared to competing related models such as random forest and support vector machine.
- Open Access
- 23 April 2025
- Published by Frontiers Media SA in Frontiers in Pain Research
- Vol. 6, 1562099
Pseudo-labeling based adaptations of pain domain classifiers
Introduction: Each human being experiences pain differently. In addition to the highly subjective phenomenon, only limited labeled data, mostly based on short-term pain sequences recorded in a lab setting, is available. However, human beings in a clinic might suffer from long painful time periods for which even a smaller amount of data, in comparison to the short-term pain sequences, is available. The characteristics of short-term and long-term pain sequences are different with respect to the reactions of the human body. However, for an accurate pain assessment, representative data is necessary. Although pain recognition techniques, reported in the literature, perform well on short-term pain sequences. The collection of labeled long-term pain sequences is challenging and techniques for the assessment of long-term pain episodes are still rare. To create accurate pain assessment systems for the long-term pain domain a knowledge transfer from the short-term pain domain is inevitable.Methods: In this study, we adapt classifiers for the short-term pain domain to the long-term pain domain using pseudo-labeling techniques. We analyze the short-term and long-term pain recordings of physiological signals in combination with electric and thermal pain stimulation.Results and conclusions: The results of the study show that it is beneficial to augment the training set with the pseudo labeled long-term domain samples. For the electric pain domain in combination with the early fusion approach, we improved the classification performance by 2.4% to 80.4% in comparison to the basic approach. For the thermal pain domain in combination with the early fusion approach, we improved the classification performance by 2.8% to 70.0% in comparison to the basic approach.
- 23 April 2025
- Published by Frontiers Media SA in Frontiers in Pain Research
- Vol. 6, 1562099
Open AccessPseudo-labeling based adaptations of pain domain classifiers
Introduction: Each human being experiences pain differently. In addition to the highly subjective phenomenon, only limited labeled data, mostly based on short-term pain sequences recorded in a lab setting, is available. However, human beings in a clinic might suffer from long painful time periods for which even a smaller amount of data, in comparison to the short-term pain sequences, is available. The characteristics of short-term and long-term pain sequences are different with respect to the reactions of the human body. However, for an accurate pain assessment, representative data is necessary. Although pain recognition techniques, reported in the literature, perform well on short-term pain sequences. The collection of labeled long-term pain sequences is challenging and techniques for the assessment of long-term pain episodes are still rare. To create accurate pain assessment systems for the long-term pain domain a knowledge transfer from the short-term pain domain is inevitable.Methods: In this study, we adapt classifiers for the short-term pain domain to the long-term pain domain using pseudo-labeling techniques. We analyze the short-term and long-term pain recordings of physiological signals in combination with electric and thermal pain stimulation.Results and conclusions: The results of the study show that it is beneficial to augment the training set with the pseudo labeled long-term domain samples. For the electric pain domain in combination with the early fusion approach, we improved the classification performance by 2.4% to 80.4% in comparison to the basic approach. For the thermal pain domain in combination with the early fusion approach, we improved the classification performance by 2.8% to 70.0% in comparison to the basic approach.
- Open Access
- 9 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 121-130
Pain Recognition Differences between Female and Male Subjects: An Analysis based on the Physiological Signals of the X-ITE Pain Database
- 9 October 2023
- Published by Association for Computing Machinery (ACM)
- p. 121-130
Open AccessPain Recognition Differences between Female and Male Subjects: An Analysis based on the Physiological Signals of the X-ITE Pain Database
- Open Access
Automated Electrodermal Activity and Facial Expression Analysis for Continuous Pain Intensity Monitoring on the X-ITE Pain Database
This study focuses on improving healthcare quality by introducing an automated system that continuously monitors patient pain intensity. The system analyzes the Electrodermal Activity (EDA) sensor modality modality, compares the results obtained from both EDA and facial expressions modalities, and late fuses EDA and facial expressions modalities. This work extends our previous studies of pain intensity monitoring via an expanded analysis of the two informative methods. The EDA sensor modality and facial expression analysis play a prominent role in pain recognition; the extracted features reflect the patient’s responses to different pain levels. Three different approaches were applied: Random Forest (RF) baseline methods, Long-Short Term Memory Network (LSTM), and LSTM with the sample-weighting method (LSTM-SW). Evaluation metrics included Micro average F1-score for classification and Mean Squared Error (MSE) and intraclass correlation coefficient (ICC [3, 1]) for both classification and regression. The results highlight the effectiveness of late fusion for EDA and facial expressions, particularly in almost balanced datasets (Micro average F1-score around 61%, ICC about 0.35). EDA regression models, particularly LSTM and LSTM-SW, showed superiority in imbalanced datasets and outperformed guessing (where the majority of votes indicate no pain) and baseline methods (RF indicates Random Forest classifier (RFc) and Random Forest regression (RFr)). In conclusion, by integrating both modalities or utilizing EDA, they can provide medical centers with reliable and valuable insights into patients’ pain experiences and responses.
Open AccessAutomated Electrodermal Activity and Facial Expression Analysis for Continuous Pain Intensity Monitoring on the X-ITE Pain Database
This study focuses on improving healthcare quality by introducing an automated system that continuously monitors patient pain intensity. The system analyzes the Electrodermal Activity (EDA) sensor modality modality, compares the results obtained from both EDA and facial expressions modalities, and late fuses EDA and facial expressions modalities. This work extends our previous studies of pain intensity monitoring via an expanded analysis of the two informative methods. The EDA sensor modality and facial expression analysis play a prominent role in pain recognition; the extracted features reflect the patient’s responses to different pain levels. Three different approaches were applied: Random Forest (RF) baseline methods, Long-Short Term Memory Network (LSTM), and LSTM with the sample-weighting method (LSTM-SW). Evaluation metrics included Micro average F1-score for classification and Mean Squared Error (MSE) and intraclass correlation coefficient (ICC [3, 1]) for both classification and regression. The results highlight the effectiveness of late fusion for EDA and facial expressions, particularly in almost balanced datasets (Micro average F1-score around 61%, ICC about 0.35). EDA regression models, particularly LSTM and LSTM-SW, showed superiority in imbalanced datasets and outperformed guessing (where the majority of votes indicate no pain) and baseline methods (RF indicates Random Forest classifier (RFc) and Random Forest regression (RFr)). In conclusion, by integrating both modalities or utilizing EDA, they can provide medical centers with reliable and valuable insights into patients’ pain experiences and responses.
- Open Access
- 15 March 2023
- Published by Frontiers Media SA in Frontiers in Medicine
- Vol. 10, 1038154
The impact of analgesic on EMG and other biosignals in a postoperative setting
Background: In the clinical context, the assessment of pain in patients with inadequate communication skills is standardly performed externally by trained medical staff. Automated pain recognition (APR) could make a significant contribution here. Hereby, pain responses are captured using mainly video cams and biosignal sensors. Primary, the automated monitoring of pain during the onset of analgesic sedation has the highest relevance in intensive care medicine. In this context, facial electromyography (EMG) represents an alternative to recording facial expressions via video in terms of data security. In the present study, specific physiological signals were analyzed to determine, whether a distinction can be made between pre-and post-analgesic administration in a postoperative setting. Explicitly, the significance of the facial EMG regarding the operationalization of the effect of analgesia was tested.Methods: N = 38 patients scheduled for surgical intervention where prospectively recruited. After the procedure the patients were transferred to intermediate care. Biosignals were recorded and all doses of analgesic sedations were carefully documented until they were transferred back to the general ward.Results: Almost every biosignal feature is able to distinguish significantly between ‘before’ and ‘after’ pain medication. We found the highest effect sizes (r = 0.56) for the facial EMG.Conclusion: The results of the present study, findings from research based on the BioVid and X-ITE pain datasets, staff and patient acceptance indicate that it would now be appropriate to develop an APR prototype.
- 15 March 2023
- Published by Frontiers Media SA in Frontiers in Medicine
- Vol. 10, 1038154
Open AccessThe impact of analgesic on EMG and other biosignals in a postoperative setting
Background: In the clinical context, the assessment of pain in patients with inadequate communication skills is standardly performed externally by trained medical staff. Automated pain recognition (APR) could make a significant contribution here. Hereby, pain responses are captured using mainly video cams and biosignal sensors. Primary, the automated monitoring of pain during the onset of analgesic sedation has the highest relevance in intensive care medicine. In this context, facial electromyography (EMG) represents an alternative to recording facial expressions via video in terms of data security. In the present study, specific physiological signals were analyzed to determine, whether a distinction can be made between pre-and post-analgesic administration in a postoperative setting. Explicitly, the significance of the facial EMG regarding the operationalization of the effect of analgesia was tested.Methods: N = 38 patients scheduled for surgical intervention where prospectively recruited. After the procedure the patients were transferred to intermediate care. Biosignals were recorded and all doses of analgesic sedations were carefully documented until they were transferred back to the general ward.Results: Almost every biosignal feature is able to distinguish significantly between ‘before’ and ‘after’ pain medication. We found the highest effect sizes (r = 0.56) for the facial EMG.Conclusion: The results of the present study, findings from research based on the BioVid and X-ITE pain datasets, staff and patient acceptance indicate that it would now be appropriate to develop an APR prototype.
- Open Access
- 5 November 2021
- Published by Springer Nature in BMC Anesthesiology
- Vol. 21 (1) , 1-13
Measurement of the nociceptive flexion reflex threshold in critically ill patients – a randomized observational pilot study
Background: Pain detection and treatment is a major challenge in the care of critically ill patients, rendered more complex by the need to take into consideration the risk of insufficient or excessive analgesia. The nociceptive flexion reflex threshold (NFRT) has become the established basis for measuring the level of analgesia in the perioperative context. However, it remains unclear whether NFRT measurement can be usefully applied to mechanically ventilated, analgosedated critically ill patients who are unable to communicate. Therefore, the aim of the present study was to investigate whether there is an association between the NFRT measurement and the Behavioral Pain Scale (BPS) in critically ill, analgosedated, and mechanically ventilated patients and whether the NFRT measurement can also detect potential excessive analgesia. Methods: This prospective, observational, randomized single-center pilot study included patients admitted to the surgical Intensive Care Unit of University Hospital Ulm, Germany, all of whom were analgosedated and intubated. Major exclusion criteria were defined as the need for the administration of neuromuscular blocking agents or neurological diseases associated with peripheral nerve conduction restriction. Initial NFRT and BPS measurements were conducted within 12 h after admission. A structured pain assessment was performed at least twice daily until extubation throughout the observation period thereafter (Group A: BPS + NFRT, Group B: BPS). Results: 114 patients were included in the study. NFRT is associated negatively with BPS. NFRT was almost twice as high in patients with a Richmond Agitation Sedation Scale (RASS) score of -5 than in patients with a RASS score ≥ -4 (RASS -5 – NFRT: 59.40 vs. RASS -4 – NFRT: 29.00, p < 0.001). Conclusions: NFRT measurement is associated negatively with the BPS in critically ill patients. NFRT measurement provides guidance for the evaluation of nociceptive processes in patients with RASS scores ≤ −4, in whom analgesia level is often difficult to assess. However, in order to identify excessive analgesia and derive therapeutic consequences, it is necessary to gradually decrease analgesics and sedatives until a stimulus threshold is reached at which the patient does not feel pain. Trial Registration: Retrospectively registered in the German Clinical Trials Register, registration number DRKS00021149, date of registration: March 26, 2020. https://www.drks.de/drks_web/navigate.do?navigationId=trial.HTML&TRIAL_ID=DRKS00021149.
- 5 November 2021
- Published by Springer Nature in BMC Anesthesiology
- Vol. 21 (1) , 1-13
Open AccessMeasurement of the nociceptive flexion reflex threshold in critically ill patients – a randomized observational pilot study
Background: Pain detection and treatment is a major challenge in the care of critically ill patients, rendered more complex by the need to take into consideration the risk of insufficient or excessive analgesia. The nociceptive flexion reflex threshold (NFRT) has become the established basis for measuring the level of analgesia in the perioperative context. However, it remains unclear whether NFRT measurement can be usefully applied to mechanically ventilated, analgosedated critically ill patients who are unable to communicate. Therefore, the aim of the present study was to investigate whether there is an association between the NFRT measurement and the Behavioral Pain Scale (BPS) in critically ill, analgosedated, and mechanically ventilated patients and whether the NFRT measurement can also detect potential excessive analgesia. Methods: This prospective, observational, randomized single-center pilot study included patients admitted to the surgical Intensive Care Unit of University Hospital Ulm, Germany, all of whom were analgosedated and intubated. Major exclusion criteria were defined as the need for the administration of neuromuscular blocking agents or neurological diseases associated with peripheral nerve conduction restriction. Initial NFRT and BPS measurements were conducted within 12 h after admission. A structured pain assessment was performed at least twice daily until extubation throughout the observation period thereafter (Group A: BPS + NFRT, Group B: BPS). Results: 114 patients were included in the study. NFRT is associated negatively with BPS. NFRT was almost twice as high in patients with a Richmond Agitation Sedation Scale (RASS) score of -5 than in patients with a RASS score ≥ -4 (RASS -5 – NFRT: 59.40 vs. RASS -4 – NFRT: 29.00, p < 0.001). Conclusions: NFRT measurement is associated negatively with the BPS in critically ill patients. NFRT measurement provides guidance for the evaluation of nociceptive processes in patients with RASS scores ≤ −4, in whom analgesia level is often difficult to assess. However, in order to identify excessive analgesia and derive therapeutic consequences, it is necessary to gradually decrease analgesics and sedatives until a stimulus threshold is reached at which the patient does not feel pain. Trial Registration: Retrospectively registered in the German Clinical Trials Register, registration number DRKS00021149, date of registration: March 26, 2020. https://www.drks.de/drks_web/navigate.do?navigationId=trial.HTML&TRIAL_ID=DRKS00021149.
- Open Access
Automatic vs. Human Recognition of Pain Intensity from Facial Expression on the X-ITE Pain Database
Prior work on automated methods demonstrated that it is possible to recognize pain intensity from frontal faces in videos, while there is an assumption that humans are very adept at this task compared to machines. In this paper, we investigate whether such an assumption is correct by comparing the results achieved by two human observers with the results achieved by a Random Forest classifier (RFc) baseline model (called RFc-BL) and by three proposed automated models. The first proposed model is a Random Forest classifying descriptors of Action Unit (AU) time series; the second is a modified MobileNetV2 CNN classifying face images that combine three points in time; and the third is a custom deep network combining two CNN branches using the same input as for MobileNetV2 plus knowledge of the RFc. We conduct experiments with X-ITE phasic pain database, which comprises videotaped responses to heat and electrical pain stimuli, each of three intensities. Distinguishing these six stimulation types plus no stimulation was the main 7-class classification task for the human observers and automated approaches. Further, we conducted reduced 5-class and 3-class classification experiments, applied Multi-task learning, and a newly suggested sample weighting method. Experimental results show that the pain assessments of the human observers are significantly better than guessing and perform better than the automatic baseline approach (RFc-BL) by about 1%; however, the human performance is quite poor due to the challenge that pain that is ethically allowed to be induced in experimental studies often does not show up in facial reaction. We discovered that downweighting those samples during training improves the performance for all samples. The proposed RFc and two-CNNs models (using the proposed sample weighting) significantly outperformed the human observer by about 6% and 7%, respectively.
Open AccessAutomatic vs. Human Recognition of Pain Intensity from Facial Expression on the X-ITE Pain Database
Prior work on automated methods demonstrated that it is possible to recognize pain intensity from frontal faces in videos, while there is an assumption that humans are very adept at this task compared to machines. In this paper, we investigate whether such an assumption is correct by comparing the results achieved by two human observers with the results achieved by a Random Forest classifier (RFc) baseline model (called RFc-BL) and by three proposed automated models. The first proposed model is a Random Forest classifying descriptors of Action Unit (AU) time series; the second is a modified MobileNetV2 CNN classifying face images that combine three points in time; and the third is a custom deep network combining two CNN branches using the same input as for MobileNetV2 plus knowledge of the RFc. We conduct experiments with X-ITE phasic pain database, which comprises videotaped responses to heat and electrical pain stimuli, each of three intensities. Distinguishing these six stimulation types plus no stimulation was the main 7-class classification task for the human observers and automated approaches. Further, we conducted reduced 5-class and 3-class classification experiments, applied Multi-task learning, and a newly suggested sample weighting method. Experimental results show that the pain assessments of the human observers are significantly better than guessing and perform better than the automatic baseline approach (RFc-BL) by about 1%; however, the human performance is quite poor due to the challenge that pain that is ethically allowed to be induced in experimental studies often does not show up in facial reaction. We discovered that downweighting those samples during training improves the performance for all samples. The proposed RFc and two-CNNs models (using the proposed sample weighting) significantly outperformed the human observer by about 6% and 7%, respectively.
- Top Cited Papers
- 11 October 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 13 (1) , 530-552
Automatic Recognition Methods Supporting Pain Assessment: A Survey
- 11 October 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 13 (1) , 530-552
Top Cited PapersAutomatic Recognition Methods Supporting Pain Assessment: A Survey
- 5 April 2019
- Published by MyJove Corporation in Journal of Visualized Experiments
- No. 146,p. e59057
Multi-Modal Signals for Analyzing Pain Responses to Thermal and Electrical Stimuli
The assessment of pain relies mostly on methods that require a person to communicate. However, for people with cognitive and verbal impairments, existing methods are not sufficient as they lack reliability and validity. To approach this problem, recent research focuses on an objective pain assessment facilitated by parameters of responses derived from physiology, and video and audio signals. To develop reliable automated pain recognition systems, efforts have been made in creating multimodal databases in order to analyze pain and detect valid pain patterns. While the results are promising, they only focus on discriminating pain or pain intensities versus no pain. In order to advance this, research should also consider the quality and duration of pain as they provide additional valuable information for more advanced pain management. To complement existing databases and the analysis of pain regarding quality and length, this paper proposes a psychophysiological experiment to elicit, measure, and collect valid pain reactions. Participants are subjected to painful stimuli that differ in intensity (low, medium, and high), duration (5 s / 1 min), and modality (heat / electric pain) while audio, video (e.g., facial expressions, body gestures, facial skin temperature), and physiological signals (e.g., electrocardiogram [ECG], skin conductance level [SCL], facial electromyography [EMG], and EMG of M. trapezius) are being recorded. The study consists of a calibration phase to determine a subject’s individual pain range (from low to intolerable pain) and a stimulation phase in which pain stimuli, depending on the calibrated range, are applied. The obtained data may allow refining, improving, and evaluating automated recognition systems in terms of an objective pain assessment. For further development of such systems and to investigate pain reactions in more detail, additional pain modalities such as pressure, chemical, or cold pain should be included in future studies. Recorded data of this study will be released as the “X-ITE Pain Database”.
- 5 April 2019
- Published by MyJove Corporation in Journal of Visualized Experiments
- No. 146,p. e59057
Multi-Modal Signals for Analyzing Pain Responses to Thermal and Electrical Stimuli
The assessment of pain relies mostly on methods that require a person to communicate. However, for people with cognitive and verbal impairments, existing methods are not sufficient as they lack reliability and validity. To approach this problem, recent research focuses on an objective pain assessment facilitated by parameters of responses derived from physiology, and video and audio signals. To develop reliable automated pain recognition systems, efforts have been made in creating multimodal databases in order to analyze pain and detect valid pain patterns. While the results are promising, they only focus on discriminating pain or pain intensities versus no pain. In order to advance this, research should also consider the quality and duration of pain as they provide additional valuable information for more advanced pain management. To complement existing databases and the analysis of pain regarding quality and length, this paper proposes a psychophysiological experiment to elicit, measure, and collect valid pain reactions. Participants are subjected to painful stimuli that differ in intensity (low, medium, and high), duration (5 s / 1 min), and modality (heat / electric pain) while audio, video (e.g., facial expressions, body gestures, facial skin temperature), and physiological signals (e.g., electrocardiogram [ECG], skin conductance level [SCL], facial electromyography [EMG], and EMG of M. trapezius) are being recorded. The study consists of a calibration phase to determine a subject’s individual pain range (from low to intolerable pain) and a stimulation phase in which pain stimuli, depending on the calibrated range, are applied. The obtained data may allow refining, improving, and evaluating automated recognition systems in terms of an objective pain assessment. For further development of such systems and to investigate pain reactions in more detail, additional pain modalities such as pressure, chemical, or cold pain should be included in future studies. Recorded data of this study will be released as the “X-ITE Pain Database”.
- 10 January 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 12 (3) , 743-760
Multi-Modal Pain Intensity Recognition Based on the SenseEmotion Database
- 10 January 2019
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Affective Computing
- Vol. 12 (3) , 743-760
Multi-Modal Pain Intensity Recognition Based on the SenseEmotion Database