Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Sophia C. Steinhaeusser has a total of 35 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Effects of a robotic storytelling intervention integrating music and sound effects on prejudice toward mental illness

    by Sophia C. Steinhaeusser, Miriam Semineth, Birgit Lugrin

    Aims: Mental illnesses affect millions worldwide. Despite greater public awareness, significant stigma remains. Anti-stigmatization interventions can help reduce prejudice by promoting education and meaningful contact with people with mental illness. Technology-based interventions may further support these efforts by simulating such contact through approaches such as storytelling. Social robots may offer additional advantages for storytelling-based interventions, especially through the integration of music and sound effects. Methods: In multimodal robotic storytelling, non-speech sounds remain largely overlooked, despite their importance in related media. To address this gap, we compared a robotic storyteller narrating a story about a person experiencing a panic attack using only voice and bodily expression with versions integrating sound effects, background music, or both. Results: While the addition of sounds did not affect recipients’ prejudice differently than storytelling without them, adding both sound effects and music increased transportation and improved associative empathy. In contrast, adding only music decreased associative empathy. Although mediation was not indicated, the results revealed transportation as a predictor for empathy and prejudice, while being only minimally influenced by sound integration itself. Conclusion: The effects of sound integration in a robotic storytelling intervention on recipients’ prejudice were mixed, recommending either the combination of both sound types or complete omission. Transportation was indicated as a key lever for increasing empathy and decreasing prejudice that warrants further investigation. Future work is needed to gain deeper insights into robotic storytelling as an intervention tool for reducing prejudice, including work on transportation as a modifiable factor and the integration of pre-post-measurements.

  • Development of the Fully Automatic Robotic Storyteller – Comparing Manual, Semi-Automatic, and LLM-Based Emotional Story Annotation for Multimodal Robotic Storytelling

    by Sophia C. Steinhaeusser, Albin Zehe, Sophia Maier, Andreas Hotho, Birgit Lugrin

    Storytelling has accompanied mankind since its beginnings and receiving stories is still a popular leisure activity. Today, new storytelling media are emerging. For example, social robots can act as storytellers, using different modalities such as facial expressions or body postures. With the goal of automating robotic storytelling, we annotated stories with emotion labels which a robot can use to automatically adapt its behavior to. Doing so, three different approaches and a baseline are compared in two studies in this paper: 1) manual labels by human annotators (MA), 2) software-based word-sensitive annotation using the Linguistic Inquiry and Word Count program (LIWC), 3) a LLM approach using ChatGPT (GPT), and 4) random annotations as baseline (RBL). In an online study showing videos of a storytelling robot, the annotations were validated using a backwards annotation task, with GPT closely followed by MA achieving the best, and the RBL the worst results. In a subsequent lab user study, all versions of the story were compared regarding storytelling experience, emotions, and robot perception, revealing no significant differences for any of these variables based on the annotation approach, but a significant decrease of negative affect and fatigue over the storytelling in all conditions. Taken together, the results indicate suitability of both the manual and GPT approaches. On this empirical basis, the Automatic Robotic Storyteller was implemented using GPT-based annotations and bodily expressions. The reduced programming effort of our system makes robotic storytelling accessible to a broader audience, strengthening its deployment in the field of affective computing.


  • Message for You: Observing the Effects of a Social Robot’s Interruptions During an Office Task

    by Elisabeth Ganal, Sophia C. Steinhaeusser, Florian Niebling, Birgit Lugrin


  • Social Robots Against Bullying – Effects of Embodiment and Interactivity on Social Story Experience and Efficiency

    by Sophia C. Steinhaeusser, Ohenewa Bediako Akuffo, Hanna-Finja Weichert, Gerhild Nieding, Birgit Lugrin


  • Open Access

    Integrating colored lights into multimodal robotic storytelling

    by Sophia C. Steinhaeusser, Sophia Maier, Birgit Lugrin

    Aims: Storytelling has evolved alongside human culture, giving rise to new media such as social robots. While these robots employ modalities similar to those used by humans, they can also utilize non-biomimetic modalities, such as color, which are commonly associated with emotions. As research on the use of colored light in robotic storytelling remains limited, this study investigates its integration through three empirical studies. Methods: We conducted three studies to explore the impact of colored light in robotic storytelling. The first study examined the effect of emotion-inducing colored lighting in romantic storytelling. The second study employed an online survey to determine appropriate light colors for specific emotions, based on images of the robot’s emotional expressions. The third study compared four lighting conditions in storytelling: emotion-driven colored lights, context-based colored lights, constant white light, and no additional lighting. Results: The first study found that while colored lighting did not significantly influence storytelling experience or perception of the robot, it made recipients felt more serene. The second study showed improved recognition of amazement, rage, and neutral emotional states when colored light accompanied body language. The third study revealed no significant differences across lighting conditions in terms of storytelling experience, emotions, or robot perception; however, participants generally appreciated the use of colored lights. Emotion-driven lighting received slightly more favorable subjective evaluations. Conclusion: Colored lighting can enhance the emotional expressiveness of robots. Both emotion- driven and context-based lighting strategies are appropriate for robotic storytelling. Through this series of studies, we contribute to the understanding of how colored lights are perceived in robotic communication, particularly within storytelling contexts.


  • Open Access

    Pushing Yourself to the Limit - Influence of Emotional Virtual Environment Design on Physical Training in VR

    by Sebastian Oberdörfer, Sophia C. Steinhaeusser, Amiin Najjar, Clemens Tümmers, Marc Erich Latoschik

    The design of virtual environments (VEs) can strongly influence users’ emotions. These VEs are also an important aspect of immersive Virtual Reality (VR) exergames – training system that can inspire athletes to train in a highly motivated way and achieve a higher training intensity. VR-based training and rehabilitation systems can increase a user’s motivation to train and to repeat physical exercises. The surrounding VE can potentially predominantly influence users’ motivation and hence potentially even physical performance. Besides providing potentially motivating environments, physical training can be enhanced by gamification. However, it is unclear whether the surrounding VE of a VR-based physical training system influences the effectiveness of gamification. We investigate whether an emotional positive or emotional negative design influences the sport performance and interacts with the positive effects of gamification. In a user study, we immerse participants in VEs following either an emotional positive, neutral, or negative design and measure the duration the participants can hold a static strength-endurance exercise. The study targeted the investigation of the effects of (1) emotional VE design as well as the (2) presence and absence of gamification. We did not observe significant differences in the performance of the participants independent of the conditions of VE design or gamification. Gamification caused a dominating effect on emotion and motivation over the emotional design of the VEs, thus indicating an overall positive impact. The emotional design influenced the participants’ intrinsic motivation but caused mixed results with respect to emotion. Overall, our results indicate the importance of using gamification and support the commonly used emotional positive VEs for physical training. They further indicate that the design space could also include other emotional directions of VE design.


  • Open Access

    Towards the development of an automated robotic storyteller: comparing approaches for emotional story annotation for non-verbal expression via body language

    by Sophia C. Steinhaeusser, Albin Zehe, Peggy Schnetter, Andreas Hotho, Birgit Lugrin

    Storytelling is a long-established tradition and listening to stories is still a popular leisure activity. Caused by technization, storytelling media expands, e.g., to social robots acting as multi-modal storytellers, using different multimodal behaviours such as facial expressions or body postures. With the overarching goal to automate robotic storytelling, we have been annotating stories with emotion labels which the robot can use to automatically adapt its behavior. With it, three different approaches are compared in two studies in this paper: 1) manual labels by human annotators (MA), 2) software-based word-sensitive annotation using the Linguistic Inquiry and Word Count program (LIWC), and 3) a machine learning based approach (ML). In an online study showing videos of a storytelling robot, the annotations were validated, with LIWC and MA achieving the best, and ML the worst results. In a laboratory user study, the three versions of the story were compared regarding transportation and cognitive absorption, revealing no significant differences but a positive trend towards MA. On this empirical basis, the Automated Robotic Storyteller was implemented using manual annotations. Future iterations should include other robots and modalities, fewer emotion labels and their probabilities.