Bruno Herbelin
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Bruno Herbelin has a total of 167 co-authors. Here are the top co-authors by number of shared publications:
- Olaf Blanke
Laboratory of Cognitive Neuroscience, Neuro-X Institute, School of Life Sciences, École Polytechnique Fédérale de Lausanne, EPFL, Geneva, Switzerland
26 shared publications Olaf Blanke
21 shared publicationsRonan Boulic
Immersive Interaction Research Group, École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
10 shared publicationsAndrea Serino
Laboratory of Cognitive Neuroscience, Brain Mind Institute, Ecole Polytechnique Fédérale de Lausanne, Campus Biotech, Geneva, Switzerland; Center for Neuroprosthetics, School of Life Sciences, Ecole Polytechnique Fédérale de Lausanne, Campus Biotech, Geneva, Switzerland; MySpace Lab, Department of Clinical Neurosciences, University Hospital, Lausanne, Switzerland
9 shared publications- Roy Salomon
School of Psychological Sciences, University of Haifa
8 shared publications Javier Bello-Ruiz
Center for Neuroprosthetics, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
7 shared publications
Latest Publications
- Open Access
- 5 August 2026
- Published by Springer Nature in Mindfulness
- p. 1-16
Meditating with One’s Avatar in Virtual Reality Enhances Self-Compassion and Perspective-Taking
Objectives: Regular self-compassion practice is associated with improved mental health by fostering kindness toward one’s suffering and relying on self-related processes like perspective-taking. It is not known whether recently described self-related processes that rely on multisensory bodily signals, induced by exposing participants to an avatar and multisensory stimulation, also contribute to self-compassion meditation. We integrated virtual reality (VR)-based self-compassion meditation with multisensory stimulation promoting self-identification with the avatar. Method: We designed a 17-min-long practice that integrated multisensory stimulation, and audio-guided self-compassion meditation. Twenty-six participants completed an avatar-based self-compassion condition (ASC) and a standard self-compassion condition (SSC) in a within-subject design. Pre- and post-meditation assessments included scores of the State Self-Compassion Scale, Self-Other Four Immeasurables Scale, and an Own Body Transformation task to measure self-compassion, positive and negative qualities, and perspective-taking ability, respectively. Results: Participants in both ASC and SSC conditions demonstrated significantly higher self-compassion scores ( p < 0.001), increased positive qualities, and reduced negative valence (interaction effect: p < 0.001) versus baseline. Perspective-taking ability, as measured in the OBT task, improved in the ASC versus SSC condition. Conclusions: This study showed that (1) incorporating VR-based multisensory stimulation with an avatar into self-compassion practice (ASC) was feasible, (2) both ASC and SSC induced higher compassion ratings characterized by decreased negative and increased positive qualities compared to the baseline, and that (3) ASC meditation improved perspective-taking more than SSC. We proposed that multisensory perceptual cues provided by VR can support perspective-taking, potentially fostering empathetic engagement towards oneself, aiding self-compassion practices. Preregistration: This study is not pre-registered.
- 5 August 2026
- Published by Springer Nature in Mindfulness
- p. 1-16
Open AccessMeditating with One’s Avatar in Virtual Reality Enhances Self-Compassion and Perspective-Taking
Objectives: Regular self-compassion practice is associated with improved mental health by fostering kindness toward one’s suffering and relying on self-related processes like perspective-taking. It is not known whether recently described self-related processes that rely on multisensory bodily signals, induced by exposing participants to an avatar and multisensory stimulation, also contribute to self-compassion meditation. We integrated virtual reality (VR)-based self-compassion meditation with multisensory stimulation promoting self-identification with the avatar. Method: We designed a 17-min-long practice that integrated multisensory stimulation, and audio-guided self-compassion meditation. Twenty-six participants completed an avatar-based self-compassion condition (ASC) and a standard self-compassion condition (SSC) in a within-subject design. Pre- and post-meditation assessments included scores of the State Self-Compassion Scale, Self-Other Four Immeasurables Scale, and an Own Body Transformation task to measure self-compassion, positive and negative qualities, and perspective-taking ability, respectively. Results: Participants in both ASC and SSC conditions demonstrated significantly higher self-compassion scores ( p < 0.001), increased positive qualities, and reduced negative valence (interaction effect: p < 0.001) versus baseline. Perspective-taking ability, as measured in the OBT task, improved in the ASC versus SSC condition. Conclusions: This study showed that (1) incorporating VR-based multisensory stimulation with an avatar into self-compassion practice (ASC) was feasible, (2) both ASC and SSC induced higher compassion ratings characterized by decreased negative and increased positive qualities compared to the baseline, and that (3) ASC meditation improved perspective-taking more than SSC. We proposed that multisensory perceptual cues provided by VR can support perspective-taking, potentially fostering empathetic engagement towards oneself, aiding self-compassion practices. Preregistration: This study is not pre-registered.
- Open Access
Re-membering: spontaneous reactivation of motor cortex during memory re-experiencing
Episodic autobiographical memory (EAM) relies on reactivations of cortical regions, originally engaged during event encoding. While reinstatement of auditory and visual regions during retrieval of external signals is well documented, reinstatement of motor regions, reflecting internal self-related signals from the person’s body, remains unexplored. Here, we investigated motor reinstatement at both cortical and peripheral levels. Using mixed-reality, participants encoded lifelike episodes involving specific motor actions. The next day, they freely retrieved these episodes while undergoing fMRI (N=30) or EMG (N=23). fMRI results revealed lateralized, effector-specific reactivation in primary and supplementary motor cortices. Moreover, hippocampal–motor connectivity increased during retrieval, and activity in premotor and supplementary motor areas scaled with subjective re-experiencing ratings. EMG recordings further showed, during retrieval, sub-threshold activation of the muscle active during encoding. Together, these results provide evidence for cortical and peripheral motor reinstatement during the re-experiencing of past events, emphasizing the embodied nature of EAM.
Open AccessRe-membering: spontaneous reactivation of motor cortex during memory re-experiencing
Episodic autobiographical memory (EAM) relies on reactivations of cortical regions, originally engaged during event encoding. While reinstatement of auditory and visual regions during retrieval of external signals is well documented, reinstatement of motor regions, reflecting internal self-related signals from the person’s body, remains unexplored. Here, we investigated motor reinstatement at both cortical and peripheral levels. Using mixed-reality, participants encoded lifelike episodes involving specific motor actions. The next day, they freely retrieved these episodes while undergoing fMRI (N=30) or EMG (N=23). fMRI results revealed lateralized, effector-specific reactivation in primary and supplementary motor cortices. Moreover, hippocampal–motor connectivity increased during retrieval, and activity in premotor and supplementary motor areas scaled with subjective re-experiencing ratings. EMG recordings further showed, during retrieval, sub-threshold activation of the muscle active during encoding. Together, these results provide evidence for cortical and peripheral motor reinstatement during the re-experiencing of past events, emphasizing the embodied nature of EAM.
- Open Access
- 23 August 2025
- Published by Center for Open Science in PsyArXiv
Viewpoint Retrieval as a Behavioral Proxy for Episodic Re-experiencing
Episodic autobiographical memory retrieval (EAM) is characterized by the vivid sense of re-experiencing past events. While doing so, different viewpoints over the scene can be taken, often referred to as the field or observer perspectives. However, the precision with which individuals can accurately reconstruct their original viewpoint over an event, within the field perspective, has not yet been characterized. Here we developed a task and continuous implicit measure to quantify this ability (viewpoint retrieval accuracy, VRA) and study its association with explicit measures of re-experienced vividness. Twenty-five participants experienced lifelike events in mixed reality, with five controlled observer–object angles at encoding. After a 24 h delay, they retrieved each event, rated re-experiencing vividness, and indicated their original viewpoint (VRA). Mean VRA decreased with increasing encoding eccentricity (observer–object angles) and was consistently biased toward more central viewpoints. Moreover, VRA significantly explained re-experiencing vividness, with higher re-experiencing ratings being associated with smaller errors. These findings establish VRA as an implicit and continuous proxy for re-experiencing vividness, and a new tool for investigating the spatial and bodily foundations of EAM retrieval.
- 23 August 2025
- Published by Center for Open Science in PsyArXiv
Open AccessViewpoint Retrieval as a Behavioral Proxy for Episodic Re-experiencing
Episodic autobiographical memory retrieval (EAM) is characterized by the vivid sense of re-experiencing past events. While doing so, different viewpoints over the scene can be taken, often referred to as the field or observer perspectives. However, the precision with which individuals can accurately reconstruct their original viewpoint over an event, within the field perspective, has not yet been characterized. Here we developed a task and continuous implicit measure to quantify this ability (viewpoint retrieval accuracy, VRA) and study its association with explicit measures of re-experienced vividness. Twenty-five participants experienced lifelike events in mixed reality, with five controlled observer–object angles at encoding. After a 24 h delay, they retrieved each event, rated re-experiencing vividness, and indicated their original viewpoint (VRA). Mean VRA decreased with increasing encoding eccentricity (observer–object angles) and was consistently biased toward more central viewpoints. Moreover, VRA significantly explained re-experiencing vividness, with higher re-experiencing ratings being associated with smaller errors. These findings establish VRA as an implicit and continuous proxy for re-experiencing vividness, and a new tool for investigating the spatial and bodily foundations of EAM retrieval.
- Open Access
- 12 May 2025
- Published by Elsevier in NeuroImage
- Vol. 314, 121265
Meditation in the third-person perspective modulates minimal self and heartbeat-evoked potentials
- 12 May 2025
- Published by Elsevier in NeuroImage
- Vol. 314, 121265
Open AccessMeditation in the third-person perspective modulates minimal self and heartbeat-evoked potentials
- Open Access
- 18 March 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. 31 (5) , 3077-3086
An embodied body morphology task for investigating self-avatar proportions perception in Virtual Reality
The perception of one's own body is subject to systematic distortions and can be influenced by exposure to visual stimuli showing distorted bodies. In Virtual Reality (VR), echoing such body judgment inaccuracies, avatars with strong appearance dissimilarities with respect to users' bodies can be successfully embodied. The present experimental work investigates, in the healthy population, the perception of the own body in immersive and embodied VR, as well as the impact of being co-present with virtual humans on such self-perception. Participants were successively presented with different avatars, corresponding to various upper- and lower-body proportions, and were asked to compare them with their perceived own body morphology. To investigate the influence of co-present virtual humans on this judgment, the task was performed in co-presence with virtual agents corresponding to various body appearances. Results show an overall overestimation of one's leg length and no influence of the co-present agent's appearance. Importantly, the embodiment scores reflect such body morphology judgment inaccuracy, with participants reporting lower levels of embodiment for avatars with very short legs than for avatars with very long legs. Our findings suggest specifics of embodied body judgment methods, likely resulting from the experience of embodying the avatar as compared to visual appreciation only.
- 18 March 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Visualization and Computer Graphics
- Vol. 31 (5) , 3077-3086
Open AccessAn embodied body morphology task for investigating self-avatar proportions perception in Virtual Reality
The perception of one's own body is subject to systematic distortions and can be influenced by exposure to visual stimuli showing distorted bodies. In Virtual Reality (VR), echoing such body judgment inaccuracies, avatars with strong appearance dissimilarities with respect to users' bodies can be successfully embodied. The present experimental work investigates, in the healthy population, the perception of the own body in immersive and embodied VR, as well as the impact of being co-present with virtual humans on such self-perception. Participants were successively presented with different avatars, corresponding to various upper- and lower-body proportions, and were asked to compare them with their perceived own body morphology. To investigate the influence of co-present virtual humans on this judgment, the task was performed in co-presence with virtual agents corresponding to various body appearances. Results show an overall overestimation of one's leg length and no influence of the co-present agent's appearance. Importantly, the embodiment scores reflect such body morphology judgment inaccuracy, with participants reporting lower levels of embodiment for avatars with very short legs than for avatars with very long legs. Our findings suggest specifics of embodied body judgment methods, likely resulting from the experience of embodying the avatar as compared to visual appreciation only.
- Open Access
- 13 March 2025
- Published by Center for Open Science in OSF Preprints
Meditating with one’s avatar in virtual reality enhances self-compassion and perspective-taking
Objectives:Regular self-compassion practice is associated with improved mental health by fostering kindness toward one’s suffering and relying on self-related processes like perspective-taking. It is not known whether recently described self-related processes of self-identification, that rely on multisensory own body signals (tested in virtual reality (VR) by exposing participants to an avatar and multisensory stimulation) also contribute to self-compassion meditation. We integrated virtual reality (VR)-based self-compassion meditation with multisensory stimulation promoting self-identification with the avatar. Methods:We designed a practice that integrated VR, multisensory stimulation, and audio-guided self-compassion meditation. Participants carried out a VR-based avatar self-compassion condition (ASC), and a standard self-compassion condition (SSC). Pre- and post-meditation assessments included scores of the State Self-Compassion Scale, Self-Other Four Immeasurables (SOFI) Scale, and an Own Body Transformation task to measure self-compassion, positive/negative qualities, perspective-taking ability, and electrocardiogram (ECG), respectively.Results:Participants in both ASC and SSC condition, demonstrated significantly higher self-compassion scores, increased positive qualities, and reduced negative valence (SOFI) versus baseline. Perspective-taking ability, as measured in the OBT task, improved in the ASC versus SSC condition, without any ECG differences. Conclusions:This study shows that (1) incorporating VR-based multisensory stimulation with an avatar into self-compassion practice (ASC) is feasible, (2) both ASC and SSC induced higher compassion ratings characterized by decreased negative and increased positive qualities comparable to the baseline, and that (3) ASC meditation improved perspective-taking more than SSC. We propose that multisensory perceptual cues provided by VR can support perspective-taking, potentially fostering empathetic engagement towards oneself, aiding self-compassion practices.
- 13 March 2025
- Published by Center for Open Science in OSF Preprints
Open AccessMeditating with one’s avatar in virtual reality enhances self-compassion and perspective-taking
Objectives:Regular self-compassion practice is associated with improved mental health by fostering kindness toward one’s suffering and relying on self-related processes like perspective-taking. It is not known whether recently described self-related processes of self-identification, that rely on multisensory own body signals (tested in virtual reality (VR) by exposing participants to an avatar and multisensory stimulation) also contribute to self-compassion meditation. We integrated virtual reality (VR)-based self-compassion meditation with multisensory stimulation promoting self-identification with the avatar. Methods:We designed a practice that integrated VR, multisensory stimulation, and audio-guided self-compassion meditation. Participants carried out a VR-based avatar self-compassion condition (ASC), and a standard self-compassion condition (SSC). Pre- and post-meditation assessments included scores of the State Self-Compassion Scale, Self-Other Four Immeasurables (SOFI) Scale, and an Own Body Transformation task to measure self-compassion, positive/negative qualities, perspective-taking ability, and electrocardiogram (ECG), respectively.Results:Participants in both ASC and SSC condition, demonstrated significantly higher self-compassion scores, increased positive qualities, and reduced negative valence (SOFI) versus baseline. Perspective-taking ability, as measured in the OBT task, improved in the ASC versus SSC condition, without any ECG differences. Conclusions:This study shows that (1) incorporating VR-based multisensory stimulation with an avatar into self-compassion practice (ASC) is feasible, (2) both ASC and SSC induced higher compassion ratings characterized by decreased negative and increased positive qualities comparable to the baseline, and that (3) ASC meditation improved perspective-taking more than SSC. We propose that multisensory perceptual cues provided by VR can support perspective-taking, potentially fostering empathetic engagement towards oneself, aiding self-compassion practices.
- Open Access
Bodily perception links memory and self: a case study of an amnesic patient
Episodic autobiographical memory (EAM) is a building block of self-consciousness, involving recollection and subjective re-experiencing of personal past experiences. Any life episode is originally encoded by a subject within a body. This raises the possibility that memory encoding is shaped by bodily self-consciousness (BSC), a basic form of self-consciousness arising from the multisensory and sensorimotor perceptual signals from the body. Recent studies in healthy subjects showed that embodied encoding improves EAM, with the involvement of the hippocampus. However, there are only few imaging studies to date, hippocampal data are not consistent, and the role of hippocampal damage is not understood. We investigated how different BSC states during encoding, modulate later EAM retrieval, in a patient with severe amnesia caused by rare bilateral hippocampal damage. We performed three separate behavioral experiments using immersive virtual reality. The patient showed consistently more difficulties recollecting information encoded in embodied vs. disembodied states, particularly when asked to recall her perspective experienced at encoding. These results contrasted with the usual beneficial effect of BSC on EAM, and significantly differed from controls. These data provide consistent evidence that BSC impacts encoding and later reliving, and shows that the hippocampus is not just a critical structure for EAM, but also for effects of embodiment on memory. Additional fMRI data extend these findings by revealing that hippocampal-parietal connectivity mediates BSC-EAM coupling. Our findings plead for an important role of BSC in EAM, mediated by the hippocampus and its connectivity, leading to embodied memories that are experienced as belonging to the self. Highlights: We studied a patient with memory deficits caused by bilateral hippocampal atrophy Multisensory stimulation modulated embodiment during episode encoding Across three studies the patient showed stronger memory deficits for embodied episodes Multisensory bodily perception impacts bodily self-consciousness and episodic memory Our findings link the bodily self to episodic memory and the hippocampus
Open AccessBodily perception links memory and self: a case study of an amnesic patient
Episodic autobiographical memory (EAM) is a building block of self-consciousness, involving recollection and subjective re-experiencing of personal past experiences. Any life episode is originally encoded by a subject within a body. This raises the possibility that memory encoding is shaped by bodily self-consciousness (BSC), a basic form of self-consciousness arising from the multisensory and sensorimotor perceptual signals from the body. Recent studies in healthy subjects showed that embodied encoding improves EAM, with the involvement of the hippocampus. However, there are only few imaging studies to date, hippocampal data are not consistent, and the role of hippocampal damage is not understood. We investigated how different BSC states during encoding, modulate later EAM retrieval, in a patient with severe amnesia caused by rare bilateral hippocampal damage. We performed three separate behavioral experiments using immersive virtual reality. The patient showed consistently more difficulties recollecting information encoded in embodied vs. disembodied states, particularly when asked to recall her perspective experienced at encoding. These results contrasted with the usual beneficial effect of BSC on EAM, and significantly differed from controls. These data provide consistent evidence that BSC impacts encoding and later reliving, and shows that the hippocampus is not just a critical structure for EAM, but also for effects of embodiment on memory. Additional fMRI data extend these findings by revealing that hippocampal-parietal connectivity mediates BSC-EAM coupling. Our findings plead for an important role of BSC in EAM, mediated by the hippocampus and its connectivity, leading to embodied memories that are experienced as belonging to the self. Highlights: We studied a patient with memory deficits caused by bilateral hippocampal atrophy Multisensory stimulation modulated embodiment during episode encoding Across three studies the patient showed stronger memory deficits for embodied episodes Multisensory bodily perception impacts bodily self-consciousness and episodic memory Our findings link the bodily self to episodic memory and the hippocampus
- Open Access
Meditation in the third-person perspective modulates minimal self and heartbeat-evoked potentials
Experienced meditation practitioners often report altered states of their sense of self, including decentering and distancing the self from the body and one’s current concerns. Altered states of the sense of self, such as disembodiment and distancing of the self from the body, have also been induced experimentally using virtual reality (VR) and linked neurally to heartbeat evoked potentials (HEPs). Whereas many studies investigated the related neural correlates of such decentering during meditation, none experimentally modulated the sense of self during meditation practice using VR nor determined the potentially associated behavioral changes of the sense of self. Here we determined HEPs and behavioral measures in 23 participants who performed a guided meditation practice in VR, either from a third-person (3PP) or first-person perspective (1PP) to modulate the sense of self. In the 3PP versus 1PP meditation condition, we report stronger sensations of detachment and disconnection, reduced salience of the perceived body boundary, and reduced self-identification with the body. HEP analysis revealed differential neural responses between conditions, characterized by a more negative HEP amplitude in the 3PP condition, associated with activation of the posterior cingulate cortex and medial prefrontal cortex. Leveraging a new VR-supported meditation platform and methods, these data link the sense of self in meditation practice to the neuroscience of the bodily self, based on subjective, behavioral, and neural measures.
Open AccessMeditation in the third-person perspective modulates minimal self and heartbeat-evoked potentials
Experienced meditation practitioners often report altered states of their sense of self, including decentering and distancing the self from the body and one’s current concerns. Altered states of the sense of self, such as disembodiment and distancing of the self from the body, have also been induced experimentally using virtual reality (VR) and linked neurally to heartbeat evoked potentials (HEPs). Whereas many studies investigated the related neural correlates of such decentering during meditation, none experimentally modulated the sense of self during meditation practice using VR nor determined the potentially associated behavioral changes of the sense of self. Here we determined HEPs and behavioral measures in 23 participants who performed a guided meditation practice in VR, either from a third-person (3PP) or first-person perspective (1PP) to modulate the sense of self. In the 3PP versus 1PP meditation condition, we report stronger sensations of detachment and disconnection, reduced salience of the perceived body boundary, and reduced self-identification with the body. HEP analysis revealed differential neural responses between conditions, characterized by a more negative HEP amplitude in the 3PP condition, associated with activation of the posterior cingulate cortex and medial prefrontal cortex. Leveraging a new VR-supported meditation platform and methods, these data link the sense of self in meditation practice to the neuroscience of the bodily self, based on subjective, behavioral, and neural measures.
- Open Access
- 18 July 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-2
Virtual Reality for inward contemplation
When performed at a high level, meditation and contemplative practices can give rise to altered states of consciousness, the study of which is key for cognitive neurosciences. Those effects indeed can help to better understand the underlying mechanisms of the self and its relation with the body. But reaching those states is rare and thus hard to observe. They require intense practice and dedication, even among regular meditators. To make those specific states more accessible, we propose a new approach of a “neuro-engineered meditation”, bringing technologies such as virtual reality (VR) and biofeedback to give practitioners new perspectives over their practice, by targeting their own body and internal sensations. We expect this approach to help participants better understand complex concepts and develop deeper results, in comparison with a regular practice.
- 18 July 2024
- Published by Association for Computing Machinery (ACM)
- p. 1-2
Open AccessVirtual Reality for inward contemplation
When performed at a high level, meditation and contemplative practices can give rise to altered states of consciousness, the study of which is key for cognitive neurosciences. Those effects indeed can help to better understand the underlying mechanisms of the self and its relation with the body. But reaching those states is rare and thus hard to observe. They require intense practice and dedication, even among regular meditators. To make those specific states more accessible, we propose a new approach of a “neuro-engineered meditation”, bringing technologies such as virtual reality (VR) and biofeedback to give practitioners new perspectives over their practice, by targeting their own body and internal sensations. We expect this approach to help participants better understand complex concepts and develop deeper results, in comparison with a regular practice.
- Open Access
- 27 March 2024
- Published by IOP Publishing in Journal of Neural Engineering
- Vol. 21 (2) , 026016
Customizing the human-avatar mapping based on EEG error related potentials
Objective. A key challenge of virtual reality (VR) applications is to maintain a reliable human-avatar mapping. Users may lose the sense of controlling (sense of agency), owning (sense of body ownership), or being located (sense of self-location) inside the virtual body when they perceive erroneous interaction, i.e. a break-in-embodiment (BiE). However, the way to detect such an inadequate event is currently limited to questionnaires or spontaneous reports from users. The ability to implicitly detect BiE in real-time enables us to adjust human-avatar mapping without interruption. Approach. We propose and empirically demonstrate a novel brain computer interface (BCI) approach that monitors the occurrence of BiE based on the users’ brain oscillatory activity in real-time to adjust the human-avatar mapping in VR. We collected EEG activity of 37 participants while they performed reaching movements with their avatar with different magnitude of distortion. Main results. Our BCI approach seamlessly predicts occurrence of BiE in varying magnitude of erroneous interaction. The mapping has been customized by BCI-reinforcement learning (RL) closed-loop system to prevent BiE from occurring. Furthermore, a non-personalized BCI decoder generalizes to new users, enabling ‘Plug-and-Play’ ErrP-based non-invasive BCI. The proposed VR system allows customization of human-avatar mapping without personalized BCI decoders or spontaneous reports. Significance. We anticipate that our newly developed VR-BCI can be useful to maintain an engaging avatar-based interaction and a compelling immersive experience while detecting when users notice a problem and seamlessly correcting it.
- 27 March 2024
- Published by IOP Publishing in Journal of Neural Engineering
- Vol. 21 (2) , 026016
Open AccessCustomizing the human-avatar mapping based on EEG error related potentials
Objective. A key challenge of virtual reality (VR) applications is to maintain a reliable human-avatar mapping. Users may lose the sense of controlling (sense of agency), owning (sense of body ownership), or being located (sense of self-location) inside the virtual body when they perceive erroneous interaction, i.e. a break-in-embodiment (BiE). However, the way to detect such an inadequate event is currently limited to questionnaires or spontaneous reports from users. The ability to implicitly detect BiE in real-time enables us to adjust human-avatar mapping without interruption. Approach. We propose and empirically demonstrate a novel brain computer interface (BCI) approach that monitors the occurrence of BiE based on the users’ brain oscillatory activity in real-time to adjust the human-avatar mapping in VR. We collected EEG activity of 37 participants while they performed reaching movements with their avatar with different magnitude of distortion. Main results. Our BCI approach seamlessly predicts occurrence of BiE in varying magnitude of erroneous interaction. The mapping has been customized by BCI-reinforcement learning (RL) closed-loop system to prevent BiE from occurring. Furthermore, a non-personalized BCI decoder generalizes to new users, enabling ‘Plug-and-Play’ ErrP-based non-invasive BCI. The proposed VR system allows customization of human-avatar mapping without personalized BCI decoders or spontaneous reports. Significance. We anticipate that our newly developed VR-BCI can be useful to maintain an engaging avatar-based interaction and a compelling immersive experience while detecting when users notice a problem and seamlessly correcting it.