Estêvão Bissoli Saleme
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Estêvão Bissoli Saleme has a total of 53 co-authors. Here are the top co-authors by number of shared publications:
- Celso Alberto Saibel Santos
Federal University of Espírito Santo, Vitória, Brazil
18 shared publications - Gheorghita Ghinea
Department of Computer Science, Brunel University London, London, UK
14 shared publications - Alexandra Covaci
University of Kent, Canterbury, United Kingdom
5 shared publications - Gebremariam Mesfin
Department of Computer Science, Brunel University, London, UK
5 shared publications Alexandra Covaci
University of Kent
4 shared publications- Elahe Kani-Zabihi
Computing and Engineering, University of West London, 7364 Ealing, London United Kingdom of Great Britain and Northern Ireland
3 shared publications
Latest Publications
- Open Access
- 1 January 2025
- Published by Sociedade Brasileira de Computação in Journal on Interactive Systems
- Vol. 16 (1) , 865-873
Prototyping and Evaluating TWIRL: A Temperature-Controlled DIY Airflow System for Enhancing Immersive Media
This study presents the design, prototyping, and user-centered evaluation of TWIRL (Thermal Wind Right and Left), a low-cost, temperature-controlled airflow system aimed at enhancing immersion in multisensory media experiences. TWIRL integrates hot and cold airflow generation, via Peltier-based cooling modules and modified hairdryer heating units, synchronized with audiovisual content using Sensory Effects Metadata (MPEG-V). A preliminary user study (N = 12) evaluated perceived realism, enjoyment, comfort, and engagement while experiencing thermally congruent video stimuli. The results indicated high participant acceptance, with thermal effects classified as realistic, pleasant, and nonintrusive, supported by strong internal consistency metrics. The participants expressed a willingness to use TWIRL in the future and recommend it, suggesting the potential of TWIRL to increase the presence and enjoyment of multisensory systems. The findings contribute to the understanding of thermal-wind integration in mulsemedia and offer design guidelines for future scalable, multisensory systems in the entertainment, education, and accessibility domains.
- 1 January 2025
- Published by Sociedade Brasileira de Computação in Journal on Interactive Systems
- Vol. 16 (1) , 865-873
Open AccessPrototyping and Evaluating TWIRL: A Temperature-Controlled DIY Airflow System for Enhancing Immersive Media
This study presents the design, prototyping, and user-centered evaluation of TWIRL (Thermal Wind Right and Left), a low-cost, temperature-controlled airflow system aimed at enhancing immersion in multisensory media experiences. TWIRL integrates hot and cold airflow generation, via Peltier-based cooling modules and modified hairdryer heating units, synchronized with audiovisual content using Sensory Effects Metadata (MPEG-V). A preliminary user study (N = 12) evaluated perceived realism, enjoyment, comfort, and engagement while experiencing thermally congruent video stimuli. The results indicated high participant acceptance, with thermal effects classified as realistic, pleasant, and nonintrusive, supported by strong internal consistency metrics. The participants expressed a willingness to use TWIRL in the future and recommend it, suggesting the potential of TWIRL to increase the presence and enjoyment of multisensory systems. The findings contribute to the understanding of thermal-wind integration in mulsemedia and offer design guidelines for future scalable, multisensory systems in the entertainment, education, and accessibility domains.
- Open Access
- 7 March 2023
- Published by Springer Nature in Multimedia Tools and Applications
- Vol. 82 (22) , 34485-34512
Thermal and wind devices for multisensory human-computer interaction: an overview
In order to create immersive experiences in virtual worlds, we need to explore different human senses (sight, hearing, smell, taste, and touch). Many different devices have been developed by both industry and academia towards this aim. In this paper, we focus our attention on the researched area of thermal and wind devices to deliver the sensations of heat and cold against people’s skin and their application to human-computer interaction (HCI). First, we present a review of devices and their features that were identified as relevant. Then, we highlight the users’ experience with thermal and wind devices, highlighting limitations either found or inferred by the authors and studies selected for this survey. Accordingly, from the current literature, we can infer that, in wind and temperature-based haptic systems (i) users experience wind effects produced by fans that move air molecules at room temperature, and (ii) there is no integration of thermal components to devices intended for the production of both cold or hot airflows. Subsequently, an analysis of why thermal wind devices have not been devised yet is undertaken, highlighting the challenges of creating such devices.
- 7 March 2023
- Published by Springer Nature in Multimedia Tools and Applications
- Vol. 82 (22) , 34485-34512
Open AccessThermal and wind devices for multisensory human-computer interaction: an overview
In order to create immersive experiences in virtual worlds, we need to explore different human senses (sight, hearing, smell, taste, and touch). Many different devices have been developed by both industry and academia towards this aim. In this paper, we focus our attention on the researched area of thermal and wind devices to deliver the sensations of heat and cold against people’s skin and their application to human-computer interaction (HCI). First, we present a review of devices and their features that were identified as relevant. Then, we highlight the users’ experience with thermal and wind devices, highlighting limitations either found or inferred by the authors and studies selected for this survey. Accordingly, from the current literature, we can infer that, in wind and temperature-based haptic systems (i) users experience wind effects produced by fans that move air molecules at room temperature, and (ii) there is no integration of thermal components to devices intended for the production of both cold or hot airflows. Subsequently, an analysis of why thermal wind devices have not been devised yet is undertaken, highlighting the challenges of creating such devices.
- 6 January 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. 29 (4) , 2093-2101
Multisensory 360° Videos Under Varying Resolution Levels Enhance Presence
Omnidirectional videos have become a leading multimedia format for Virtual Reality applications. While live 360 videos offer a unique immersive experience, streaming of omnidirectional content at high resolutions is not always feasible in bandwidth-limited networks. While in the case of flat videos, scaling to lower resolutions works well, 360 video quality is seriously degraded because of the viewing distances involved in head-mounted displays. Hence, in this paper, we investigate first how quality degradation impacts the sense of presence in immersive Virtual Reality applications. Then, we are pushing the boundaries of 360 technology through the enhancement with multisensory stimuli. 48 participants experimented both 360 scenarios (with and without multisensory content), while they were divided randomly between four conditions characterised by different encoding qualities (HD, FullHD, 2.5K, 4K). The results showed that presence is not mediated by streaming at a higher bitrate. The trend we identified revealed however that presence is positively and significantly impacted by the enhancement with multisensory content. This shows that multisensory technology is crucial in creating more immersive experiences.
- 6 January 2022
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. 29 (4) , 2093-2101
Multisensory 360° Videos Under Varying Resolution Levels Enhance Presence
Omnidirectional videos have become a leading multimedia format for Virtual Reality applications. While live 360 videos offer a unique immersive experience, streaming of omnidirectional content at high resolutions is not always feasible in bandwidth-limited networks. While in the case of flat videos, scaling to lower resolutions works well, 360 video quality is seriously degraded because of the viewing distances involved in head-mounted displays. Hence, in this paper, we investigate first how quality degradation impacts the sense of presence in immersive Virtual Reality applications. Then, we are pushing the boundaries of 360 technology through the enhancement with multisensory stimuli. 48 participants experimented both 360 scenarios (with and without multisensory content), while they were divided randomly between four conditions characterised by different encoding qualities (HD, FullHD, 2.5K, 4K). The results showed that presence is not mediated by streaming at a higher bitrate. The trend we identified revealed however that presence is positively and significantly impacted by the enhancement with multisensory content. This shows that multisensory technology is crucial in creating more immersive experiences.
- 21 June 2021
- Published by Sociedade Brasileira de Computação
SensoryX ’21 Workshop on Multisensory Experiences at ACM IMX ’21
Multisensory experiences have been increasingly undertaken in the digital world. With the emerging interest in immersive applications (i.e. 360 videos and virtual reality), more and more researchers and practitioners are in pursuit of ways to take these experiences to the next level, adding sensations that go beyond that of seeing and hearing. This one-day workshop aims at identifying the current practices, challenges, opportunities, and limitations to be overcome in the quest to transcend the overwhelmingly bisensorial nature of digital multimedia into a multisensory one by fostering discussions on the use of olfactory, gustatory and tactile effects in digital experiences.
- 21 June 2021
- Published by Sociedade Brasileira de Computação
SensoryX ’21 Workshop on Multisensory Experiences at ACM IMX ’21
Multisensory experiences have been increasingly undertaken in the digital world. With the emerging interest in immersive applications (i.e. 360 videos and virtual reality), more and more researchers and practitioners are in pursuit of ways to take these experiences to the next level, adding sensations that go beyond that of seeing and hearing. This one-day workshop aims at identifying the current practices, challenges, opportunities, and limitations to be overcome in the quest to transcend the overwhelmingly bisensorial nature of digital multimedia into a multisensory one by fostering discussions on the use of olfactory, gustatory and tactile effects in digital experiences.
- 21 June 2021
- Published by Sociedade Brasileira de Computação
Applying remote health monitoring to understand users’ QoE in multisensory applications in real-time
User’s Quality of Experience (QoE) understanding from objective metrics has been increasingly explored in multisensory research. However, capturing physiological data adds a degree of difculty to an already complex environment composed of software to reproduce content and actuators to deliver sensory effects. In this paper, we introduce the potential use of remote patient monitoring (RPM) systems to monitor users’ QoE through a specifc tool named HealthDash. We aim to raise discussion around them in digital multisensory experiences, their application, advantages and disadvantages, and challenges and opportunities.
- 21 June 2021
- Published by Sociedade Brasileira de Computação
Applying remote health monitoring to understand users’ QoE in multisensory applications in real-time
User’s Quality of Experience (QoE) understanding from objective metrics has been increasingly explored in multisensory research. However, capturing physiological data adds a degree of difculty to an already complex environment composed of software to reproduce content and actuators to deliver sensory effects. In this paper, we introduce the potential use of remote patient monitoring (RPM) systems to monitor users’ QoE through a specifc tool named HealthDash. We aim to raise discussion around them in digital multisensory experiences, their application, advantages and disadvantages, and challenges and opportunities.
- Open Access
- 30 June 2020
- Published by Sociedade Brasileira de Computação
- p. 13-18
A Conceptual Architecture and a Framework for Dealing with Variability in Mulsemedia Systems
The thesis is within the emerging topic of mulsemedia systems, which encompass audiovisual content associated with multisensory effects, users' quality of experience, and human-computer interaction. It presents a conceptual architecture and a framework that take into account the challenges and requirements for mulsemedia delivery systems identified from the gaps and shortcomings in related work. Furthermore, the work's outcome brings multiple and valuable contributions relying on a complex and solid experimental setup, which have been shared and published in relevant journals and conferences.
- 30 June 2020
- Published by Sociedade Brasileira de Computação
- p. 13-18
Open AccessA Conceptual Architecture and a Framework for Dealing with Variability in Mulsemedia Systems
The thesis is within the emerging topic of mulsemedia systems, which encompass audiovisual content associated with multisensory effects, users' quality of experience, and human-computer interaction. It presents a conceptual architecture and a framework that take into account the challenges and requirements for mulsemedia delivery systems identified from the gaps and shortcomings in related work. Furthermore, the work's outcome brings multiple and valuable contributions relying on a complex and solid experimental setup, which have been shared and published in relevant journals and conferences.
- 6 May 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 23 (15209210) , 1095-1105
Less is (Just as Good as) More - an Investigation of Odor Intensity and Hedonic Valence in Mulsemedia QoE using Heart Rate and Eye Tracking
Using olfactory media to enhance traditional multimedia content opens up novel opportunities for user interactions. Whilst the influence of olfaction on user Quality of Experience (QoE) in mulsemedia (multiple sensorial media) environments has been previously studied, the impact of the fundamental dimensions of scent intensity and valence (pleasantness) have been largely unexplored. This is precisely what we target in this paper, which reports the results of an empirical investigation examining how scent intensity and valence impact mulsemedia QoE. Accordingly, 54 participants were exposed to different odor valences and scent intensity levels when viewing three short multimedia clips. In particular, we examine both subjective (self-reported) as well as objective QoE metrics, as evidenced by user heart rates and eye gaze patterns. Results show that whilst eye gaze patterns are largely unaffected by the experimental conditions, valence does have a statistically significant impact upon user heart rates, as does intensity for two of the three clips employed in our study. In terms of subjective QoE, results indicate that valence impacts on the sense of reality and enjoyment; however varying intensity levels do not seem to differentially impact on QoE, bringing into question the need for strong scent intensities.
- 6 May 2020
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Multimedia
- Vol. 23 (15209210) , 1095-1105
Less is (Just as Good as) More - an Investigation of Odor Intensity and Hedonic Valence in Mulsemedia QoE using Heart Rate and Eye Tracking
Using olfactory media to enhance traditional multimedia content opens up novel opportunities for user interactions. Whilst the influence of olfaction on user Quality of Experience (QoE) in mulsemedia (multiple sensorial media) environments has been previously studied, the impact of the fundamental dimensions of scent intensity and valence (pleasantness) have been largely unexplored. This is precisely what we target in this paper, which reports the results of an empirical investigation examining how scent intensity and valence impact mulsemedia QoE. Accordingly, 54 participants were exposed to different odor valences and scent intensity levels when viewing three short multimedia clips. In particular, we examine both subjective (self-reported) as well as objective QoE metrics, as evidenced by user heart rates and eye gaze patterns. Results show that whilst eye gaze patterns are largely unaffected by the experimental conditions, valence does have a statistically significant impact upon user heart rates, as does intensity for two of the three clips employed in our study. In terms of subjective QoE, results indicate that valence impacts on the sense of reality and enjoyment; however varying intensity levels do not seem to differentially impact on QoE, bringing into question the need for strong scent intensities.
- Open Access
- 3 January 2020
- Published by Springer Nature in Multimedia Tools and Applications
- Vol. 79 (11-12) , 7987-8009
QoE of cross-modally mapped Mulsemedia: an assessment using eye gaze and heart rate
A great deal of research effort has been put in exploring crossmodal correspondences in the field of cognitive science which refer to the systematic associations frequently made between different sensory modalities (e.g. high pitch is matched with angular shapes). However, the possibilities cross-modality opens in the digital world have been relatively unexplored. Therefore, we consider that studying the plasticity and the effects of crossmodal correspondences in a mulsemedia setup can bring novel insights about improving the human-computer dialogue and experience. Mulsemedia refers to the combination of three or more senses to create immersive experiences. In our experiments, users were shown six video clips associated with certain visual features based on color, brightness, and shape. We examined if the pairing with crossmodal matching sound and the corresponding auto-generated haptic effect, and smell would lead to an enhanced user QoE. For this, we used an eye-tracking device as well as a heart rate monitor wristband to capture users’ eye gaze and heart rate whilst they were experiencing mulsemedia. After each video clip, we asked the users to complete an on-screen questionnaire with a set of questions related to smell, sound and haptic effects targeting their enjoyment and perception of the experiment. Accordingly, the eye gaze and heart rate results showed significant influence of the cross-modally mapped multisensorial effects on the users’ QoE. Our results highlight that when the olfactory content is crossmodally congruent with the visual content, the visual attention of the users seems shifted towards the correspondent visual feature. Crosmodally matched media is also shown to result in an enhanced QoE compared to a video only condition.
- 3 January 2020
- Published by Springer Nature in Multimedia Tools and Applications
- Vol. 79 (11-12) , 7987-8009
Open AccessQoE of cross-modally mapped Mulsemedia: an assessment using eye gaze and heart rate
A great deal of research effort has been put in exploring crossmodal correspondences in the field of cognitive science which refer to the systematic associations frequently made between different sensory modalities (e.g. high pitch is matched with angular shapes). However, the possibilities cross-modality opens in the digital world have been relatively unexplored. Therefore, we consider that studying the plasticity and the effects of crossmodal correspondences in a mulsemedia setup can bring novel insights about improving the human-computer dialogue and experience. Mulsemedia refers to the combination of three or more senses to create immersive experiences. In our experiments, users were shown six video clips associated with certain visual features based on color, brightness, and shape. We examined if the pairing with crossmodal matching sound and the corresponding auto-generated haptic effect, and smell would lead to an enhanced user QoE. For this, we used an eye-tracking device as well as a heart rate monitor wristband to capture users’ eye gaze and heart rate whilst they were experiencing mulsemedia. After each video clip, we asked the users to complete an on-screen questionnaire with a set of questions related to smell, sound and haptic effects targeting their enjoyment and perception of the experiment. Accordingly, the eye gaze and heart rate results showed significant influence of the cross-modally mapped multisensorial effects on the users’ QoE. Our results highlight that when the olfactory content is crossmodally congruent with the visual content, the visual attention of the users seems shifted towards the correspondent visual feature. Crosmodally matched media is also shown to result in an enhanced QoE compared to a video only condition.
- Open Access
- 1 December 2019
- Published by Association for Computing Machinery (ACM) in ACM SIGMultimedia Records
- Vol. 11 (4) , 1
Reports from ACM Multimedia 2019
The annual ACM Multimedia Conference was held in Nice, France during October 21st to 25th, 2019. Being the 27th of its series, it attracted approximately 800 participants from all over the World. Among them were the student volunteers who supported the smooth organization of the Conference.
- 1 December 2019
- Published by Association for Computing Machinery (ACM) in ACM SIGMultimedia Records
- Vol. 11 (4) , 1
Open AccessReports from ACM Multimedia 2019
The annual ACM Multimedia Conference was held in Nice, France during October 21st to 25th, 2019. Being the 27th of its series, it attracted approximately 800 participants from all over the World. Among them were the student volunteers who supported the smooth organization of the Conference.
- Open Access
- 1 December 2019
- Published by Association for Computing Machinery (ACM) in ACM SIGMultimedia Records
- Vol. 11 (4) , 1
A conceptual architecture and a framework for dealing with variability in mulsemedia systems
- 1 December 2019
- Published by Association for Computing Machinery (ACM) in ACM SIGMultimedia Records
- Vol. 11 (4) , 1
Open AccessA conceptual architecture and a framework for dealing with variability in mulsemedia systems