DGDebasmita Ghose

Debasmita Ghose

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Debasmita Ghose has a total of 20 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Open-Ended Goal Inference through Actions and Language for Human-Robot Collaboration

    by Debasmita Ghose, Oz Gitelson, Marynel Vázquez, Brian Scassellati

  • Open Access

    A Replicable, Autonomous System for In-the-Wild Mental Health Applications with the Ommie Robot

    by Kayla Matheus, Debasmita Ghose, Jirachaya (Fern) Limprayoon, Michal A. Lewkowicz, Brian Scassellati


  • I’ve Changed My Mind: Robots Adapting to Changing Human Goals During Collaboration

    by Debasmita Ghose, Oz Gitelson, Ryan Jin, Grace Abawe, Marynel Vázquez, Brian Scassellati

    For effective human-robot collaboration, a robot must align its actions with human goals, even as they change mid-task. Prior approaches often assume fixed goals, reducing goal prediction to a one-time inference. However, in real-world scenarios, humans frequently shift goals, making it challenging for robots to adapt without explicit communication. We propose a method for detecting goal changes by tracking multiple candidate action sequences and verifying their plausibility against a policy bank. Upon detecting a change, the robot refines its belief in relevant past actions and constructs Receding Horizon Planning (RHP) trees to actively select actions that assist the human while encouraging Differentiating Actions to reveal their updated goal. We evaluate our approach in a collaborative cooking environment with up to 30 unique recipes and compare it to three comparable human goal prediction algorithms. Our method outperforms all baselines, quickly converging to the correct goal after a switch, reducing task completion time and improving collaboration efficiency.


  • Open Access

    Integrating Multimodal Affective Signals for Stress Detection from Audio-Visual Data

    by Debasmita Ghose, Oz Gitelson, Brian Scassellati

    Stress detection in real-world settings presents significant challenges due to the complexity of human emotional expression influenced by biological, psychological, and social factors. While traditional methods like EEG, ECG, and EDA sensors provide direct measures of physiological responses, they are unsuitable for everyday environments due to their intrusive nature. Therefore, using non-contact, commonly available sensors like cameras and microphones to detect stress would be helpful. In this work, we use stress indicators from four key affective modalities extracted from audio-visual data: facial expressions, vocal prosody, textual sentiment, and physical fidgeting. To achieve this, we first labeled 353 video clips featuring individuals in monologue scenarios discussing personal experiences, indicating whether or not the individual is stressed based on our four modalities. Then, to effectively integrate signals from the four modalities, we extract stress signals from our audio-visual data using unimodal classifiers. Finally, to explore how the different modalities would interact to predict if a person is stressed, we compare the performance of three multimodal fusion methods: intermediate fusion, voting-based late fusion, and learning-based late fusion. Results indicate that combining multiple modes of information can effectively leverage the strengths of different modalities and achieve an F1 score of 0.85 for binary stress detection. Moreover, an ablation study shows that the more modalities are integrated, the higher the F1 score for detecting stress across all fusion techniques, demonstrating that our selected modalities possess complementary stress indicators.


  • Open Access

    Interactive Policy Shaping for Human-Robot Collaboration with Transparent Matrix Overlays

    by Jake Brawer, Debasmita Ghose, Kate Candon, Meiying Qin, Alessandro Roncone, Marynel Vázquez, Brian Scassellati

    One important aspect of effective human--robot collaborations is the ability for robots to adapt quickly to the needs of humans. While techniques like deep reinforcement learning have demonstrated success as sophisticated tools for learning robot policies, the fluency of human-robot collaborations is often limited by these policies' inability to integrate changes to a user's preferences for the task. To address these shortcomings, we propose a novel approach that can modify learned policies at execution time via symbolic if-this-then-that rules corresponding to a modular and superimposable set of low-level constraints on the robot's policy. These rules, which we call Transparent Matrix Overlays, function not only as succinct and explainable descriptions of the robot's current strategy but also as an interface by which a human collaborator can easily alter a robot's policy via verbal commands. We demonstrate the efficacy of this approach on a series of proof-of-concept cooking tasks performed in simulation and on a physical robot.


  • Open Access

    Why We Should Build Robots That Both Teach and Learn

    by Timothy Adamson, Debasmita Ghose, Shannon C. Yasuda, Lucas Jehu Silva Shepard, Michal A. Lewkowicz, Joyce Duan, Brian Scassellati

    In this paper, we argue in favor of creating robots that both teach and learn. We propose a methodology for building robots that can learn a skill from an expert, perform the skill independently or collaboratively with the expert, and then teach the same skill to a novice. This requires combining insights from learning from demonstration, human-robot collaboration, and intelligent tutoring systems to develop knowledge representations that can be shared across all three components. As a case study for our methodology, we developed a glockenspiel-playing robot. The robot begins as a novice, learns how to play musical harmonies from an expert, collaborates with the expert to complete harmonies, and then teaches the harmonies to novice users. This methodology allows for new evaluation metrics that provide a thorough understanding of how well the robot has learned and enables a robot to act as an efficient facilitator for teaching across temporal and geographic separation.


  • Cost effective road accident prevention system

    by J Mohan Kumar, Rishabh Mahajan, Deeksha Prabhu, Debasmita Ghose

    In the current scenario one of the major causes of road accidents in the world is the fact that after driving hundreds of kilometers at a stretch, the driver feels drowsy and eventually dozes off while driving or they are not aware of the environment they might just be entering. They might also be unaware of any defect in their cars. There is an alarming increase in hit and run cases caused by drivers driving under the influence of alcohol. So, there is a pressing need of a device which prevents this, so that a lot of lives can be saved every day. Safety is the primary concern for humans. When it comes to driving, we need to be extra cautious. So, this project aims at developing an affordable solution to these safety concerns. The proposed system will be mounted on a car, in front of the driver's seat, which will continuously focus on the person driving the vehicle thereby monitoring the person's actions and give voice messages to the driver, instructing him about the safety procedures every time he comes inside the car. If he found to be drowsy, he is alerted by a voice message telling him that he is drowsy and he should stop the car. If he is found to be under the influence of alcohol or tailgating while driving, the GPS coordinates of the car along with the details of the car, are sent to the nearest control room, so that appropriate action can be taken by the police. So, this system has the capability of ensuring the driver's safety along with the co-passengers and can easily be integrated with the existing safety systems in the car. This will also enhance overall road safety, by reporting any kind of reckless driving to the nearest control room. In future, it can be enforced by law for every car to have such a safety system installed for driver and passenger protection. Hence the system is a complete solution which prevents accidents, so that a lot of lives can be saved every day.