SBSean Banerjee

Sean Banerjee

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Sean Banerjee has a total of 60 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Understanding Influence of Self-identified Gender and Race on Social Norm Violations in an Immersive Doctor’s Office

    by Elza Ibragimov, Natasha Kholgade Banerjee, Sean Banerjee, Ashutosh Shivakumar

  • Open Access

    Understanding how loneliness relates to user behavior in an interactive virtual reality cafe

    by Elza Ibragimov, Natasha Kholgade Banerjee, Sean Banerjee, Ashutosh Shivakumar

    Loneliness is a growing concern among young adults. Though loneliness is distinct from social isolation, studies indicate that causes can stem from diminishment in the parameters of social contact that underlie isolation. In this work, we analyze how pre-existing levels of loneliness for Generation Z adults relate to behavior, choices made, and frustration levels in response to wait times in a visually realistic virtual reality (VR)-based representation of a cafe, an environment involving social interactions and providing services to meet basic needs such as getting food. We conduct an experiment in which participants experience a VR cafe environment where they interact with a non-playable character representing a maître d’ and go through a selection of dining preferences and a selection on a digital menu. We introduce delay times to our virtual interaction as conditions in four randomized trials. We analyze data from a pilot study with 35 participants who completed all four trials of our experiment, providing raw positional data, as well as standardized surveys for measuring loneliness and frustration. Our results show participants self-identifying as female or Asian report ‘Moderate’ or higher categories of loneliness on the Revised UCLA-20 scale. Participants in the ‘Moderate’ or higher category had higher average frustration on the Frustration Discomfort Scale. Participants in the ‘Low’ category for loneliness were more likely to opt for a dine-in experience when there were no delays with the maître d’, while a higher percentage of participants in the ‘Moderate’ or higher category opted to exit when there were delays. In delayed scenarios, a higher percentage of participants in the ‘Moderate’ or higher category chose a non-table option after choosing to dine in. Our results indicate that loneliness may be associated with variations in individual behavior in social scenarios. Our findings enable informing future work on creating environments to assist individuals with overcoming the adverse effects of loneliness and frustration.

  • Open Access

    Dine in or take out dataset: user behavior in an interactive virtual reality café

    by Elza Ibragimov, Natasha Kholgade Banerjee, Sean Banerjee, Ashutosh Shivakumar


  • Open Access

    Understanding User Perceptions Toward Queue Cutting in Immersive VR Environments

    by Elza Ibragimov, Natasha Kholgade Banerjee, Sean Banerjee, Ashutosh Shivakumar

    Queue cutting is a frustrating experience in the real‐world when a person is waiting to receive service as it violates the normative behavior of first‐in‐first‐out ordering of queues. As social experiences move to virtual worlds users may experience human and non‐playable characters (NPCs) attempting to cut the queue, thereby adding to the negative experiences of waiting. While normative behavior for activities such as joining an existing conversational group or walking between agents engaged in conversation have been studied in virtual reality, perceptions toward queue cutters have not been studied. We conduct a study on understanding how users perceive a queue‐cutting NPC in a virtual doctor's office reception area. During the wait, a queue‐cutting NPC requests an in‐queue NPC to cut in ahead of them and is either allowed or denied entry into the queue. Using data on subjective responses from 45 participants we find significant differences in perception of time, frustration, and likelihood of exit when the queue‐cutting NPC is allowed in. Our work enables future research on detecting situations capable of generating user frustration, and providing appropriate intervention via the VR environment, mitigating negative experiences, and ensuring timely service.


  • Open Access

    Real or fake motion: protecting virtual reality (VR) behavioral authentication systems against motion forecasting attacks

    by Mingjun Li, Natasha Kholgade Banerjee, Sean Banerjee

    As virtual reality (VR) systems are used in serious applications, such as physical therapy for rehabilitation, workforce training in military scenarios, or in the classroom, protecting user data from malicious agents is critical in ensuring trust in the VR system. Over the past half decade approaches in behavioral biometrics have explored the use of user motion as a mechanism for authentication and identification in VR. Approaches in using motion forecasting for behavioral biometrics in VR enable the use of limited user data by forecasting future behavior for user authentication. Such techniques enhance security by limiting the amount of user data exposed to a malicious agent. However, recent techniques have exposed an emergent vulnerability, wherein a malicious agent can use a motion forecasting model to generate plausible behavior of the target user to attack a virtual reality system protected using behavioral biometrics. In this paper, we present the first approach for classifying whether a user trajectory presented to a VR authentication system is an authentic or a deepfake trajectory. In this case, an authentic, or real, trajectory would be the motion data generated from the VR device through the actions of the intended user. On the other hand, a deepfake, or fake, trajectory would be motion synthetically generated using a deep learning-based motion forecasting approach that is used to masquerade as the motion of the intended user. Our approach uses a fully convolutional network (FCN) to determine whether a trajectory is real or fake. Using the 3D position data for the right-hand controller from the 41-subject Miller et al. ball throwing dataset collected using 3 VR systems, namely a Meta Quest, HTC Vive, and HTC Vive Cosmos, we show that our approach achieves accuracies up to 0.997, 0.998, and 0.992 when discriminating real and fake trajectories. Our approach provides the first protection against motion forecasting-based attacks against VR authentication systems. Our code is available at: https://github.com/Terascale-All-sensing-Research-Studio/Real_or_Fake_VR_Motion .

  • Open Access

    Descriptor: Line Cutting While Waiting in Line at a Doctor’s Office in Virtual Reality Dataset (LC-WAIT)

    by Elza Ibragimov, Natasha Kholgade Banerjee, Sean Banerjee, Ashutosh Shivakumar

    Waiting in line for service provision is ubiquitous to any customer-facing service environment such as a doctor’s office, bank, or coffee shop. In this paper, we describe the LC-WAIT dataset consisting of 45 participants waiting in line at a virtual doctor’s office receptionist area. In our dataset, participants experience two scenarios, or treatments—line-cutter allowed and denied, where the line-cutter is a non-playable character (NPC) that is allowed to join the front of the queue by an in-queue NPC. The LC-WAIT dataset contains participant demographics as well as responses to the Frustration Discomfort Scale (FDS), NASA Task Load Index (TLX) and System Usability Scale (SUS), and per treatment participant responses to perceived frustration due to line-cutter’s intrusive behavior, likelihood to allow the line-cutter to cut in line, and likelihood to exit due to the line-cutter’s actions. Our dataset also contains head, left, and right-hand position and orientation, and the names and hit positions of objects the participant visually interacts with. The LC-WAIT dataset can be used by interdisciplinary researchers in computer science, psychology, and human factors to study how participant behavior changes when a line-cutter is allowed to cut into a queue. These findings can be used to design social VR applications with responsive NPCs that modulate behavior based on participant states. Researchers may also use pre-trained algorithms to predict participant responses to the post-treatment questions on perception of time, frustration, and likelihood to exit. Using these models, empathetic agents or gamifications can be designed to reduce negative states due to the line-cutter and enable the participant to complete the task.

  • Open Access

    Descriptor: Waiting in Line in Virtual Reality Dataset (WAIT)

    by Elza Ibragimov, Ava Megyeri, Natasha Kholgade Banerjee, Sean Banerjee, Ashutosh Shivakumar

    As immersive social virtual reality (VR) applications emerge, users may expect to wait in virtual queues as part of the social experience or due to delays with the service provider. This paper describes the WAIT dataset containing data from 36 human participants who partook in a study on waiting in line in VR at a virtual doctor’s office receptionist area to check in for an appointment. Participants took part in three treatments using the Meta Quest Pro head mounted device (HMD)—no waiting in line, waiting for 3 minutes, and waiting for 6 minutes, with non-playable characters (NPCs) introduced to explain the reason for wait in the 3- and 6-minute treatments. The dataset contains demographics data, pre- and post-study subject reports of frustration using the Frustration Discomfort Scale (FDS), subject ratings of perceived frustration, likelihood to exit, and perception of elapsed time after each treatment, and post-study responses to the NASA Task Load Index, the System Usability Scale, and the Virtual Reality Sickness Questionnaire. The dataset also contains quantitative data on the position and orientation of the headset and the left and right hands, and names and hit positions of objects interacted with. The WAIT dataset has value for research in computer science, human factors and psychology by enabling behavior understanding in stressful scenarios, design of interruption management systems to engage participants during delays based on frustration states, and development of behavior-based security measures for critical applications in healthcare. The dataset is shared on IEEE DataPort at: https://dx.doi.org/10.21227/0jq7-nm37.


  • Open Access

    Understanding User Perceptions Toward Waiting For Service Provision in Virtual Reality

    by Elza Ibragimov, Natasha Kholgade Banerjee, Sean Banerjee, Ashutosh Shivakumar

    We analyze user perceptions of queue waits prior to receiving service in a virtual reality (VR) service provision environment. Waiting is a natural part of service receipt in real-world and online environments. As service providers such as clinics, banks, airlines, and retailers transition to VR, the process of serving multiple clients’ needs will require clients to wait in VR as well. Though users are likely to wait for a valuable service, prolonged waits have the potential to generate negative emotions such as anxiety, boredom, and frustration, and leave them facing the difficult decision of waiting versus abandoning service. To evaluate user perceptions toward wait in VR, we provide a VR environment representing the receptionist’s desk of a doctor’s office, together with non-playable character (NPC) avatars representing the receptionist and clients standing in a queue. We evaluate perceptions toward three conditions—no wait, 3-minute wait and 6-minute wait—with client NPCs in the latter two conditions. Using data on subjective responses from 33 participants interacting with our environment, we find significant differences in frustration level, exit likelihood, and absolute deviation of perceived wait time from actual time between no wait versus 3-minute wait and no wait versus 6-minute wait conditions, with higher values for longer wait times. Using quantitative data tracked from the VR system, we find significant differences between the same two condition sets for variance in eye gaze hit locations and count of object viewed, quantities that represent user motivation to occupy time through viewing-based engagement with the scene.

  • Open Access

    Descriptor: Real-World 3-D Object, Human Giver, Human Receiver, and Sampled Robot Grasps Dataset (HOH-Grasps)

    by Xinchao Song, James Cox, Sean Banerjee, Natasha K. Banerjee

    With the spread of robotic systems that perform close-proximity interactions with humans, it is important to have datasets that enable creation of robot object grasp and manipulation algorithms that show awareness of how humans interact with objects. In this paper, we describe HOH-Grasps, a large-scale dataset containing data on robot grasps synthetically sampled from real-world 3D object point clouds combined with point clouds of human giver and receiver hands acquired from real-world captures of human-human object handover interactions. HOH-Grasps contains object, giver hand, and receiver hand point clouds from 2,720 scenes, spanning 40 giver-receiver participant pairs and 136 objects. The robotic grasp data consists of 2,047,492,800 antipodal, parallel-jaw grasps over all 136 objects, of which 73,682,572 are valid, i.e., have no object collisions, and 25,432,094 total grasps are high-quality, i.e., have a friction coefficient ≤ 0.5 to achieve force closure. When object grasps are transformed to all 2,720 scenes, the dataset provides 2,316,434,197 valid grasps, i.e., grasps without collision with the object or tabletop plane, and 802,228,990 high-quality grasps. The dataset provides value in enabling training and evaluation of human- and object-aware robot grasp and manipulation algorithms, as well as broad-range tasks of interest in robotics and vision, such as object classification and pose estimation, to enable robots to engage in physical human-robot interaction tasks such as handover and collaborative assembly. The dataset is shared on HuggingFace at: https://huggingface.co/datasets/tars-home/HOH-Grasps.