Marynel Vázquez
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Research Interests
Marynel Vázquez has a total of 113 co-authors. Here are the top co-authors by number of shared publications:
- Brian Scassellati
Department of Computer Science, Yale University, New Haven, USA
9 shared publications - Kate Candon
Yale University, New Haven, USA
9 shared publications - Nathan Tsoi
Yale University, New Haven, CT, USA
7 shared publications - Qiping Zhang
Yale University, New Haven, CT, USA
5 shared publications - Alexander Lew
Department of Computer Science, Yale University, New Haven, CT, USA
4 shared publications - Austin Narcomey
Yale University, New Haven, CT, USA
3 shared publications
Latest Publications
- 1 December 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 11 (12) , 14716-14723
A Causal Approach to Predicting and Improving Human Perceptions of Social Navigation Robots
- 1 December 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 11 (12) , 14716-14723
A Causal Approach to Predicting and Improving Human Perceptions of Social Navigation Robots
- 3 September 2026
- Published by Springer Nature in Autonomous Robots
- Vol. 50 (3) , 38
Robots influencing humans to reveal their goals during collaboration and competition
- 3 September 2026
- Published by Springer Nature in Autonomous Robots
- Vol. 50 (3) , 38
Robots influencing humans to reveal their goals during collaboration and competition
- 14 April 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 11 (6) , 6991-6998
From Tool to Teammate: Balancing Task-Oriented and Social Talk During Human–Robot Collaboration
As robots become more integrated into shared work environments, effective communication must balance clear task directives with social engagement to support both efficiency and user satisfaction. This letter examines how people perceive task-oriented versus social dialogue during human–robot collaboration and how they judge the appropriateness of switching between them. We discuss a pilot, in-person data collection and two follow-up online studies with an autonomous robot in a collaborative pizza-making task. The pilot assesses the technical feasibility of an autonomous robot generating distinct task and social dialogue styles. Next, with a larger participant sample, we evaluate the salience of the two talk types and their impact on perceptions of the robot as a collaborative partner. The second study explores judgments of talk type switching using video clips from the pilot, revealing how timing, task state, and user behavior influence perceived appropriateness. Together, these findings highlight nuanced human expectations for robot communication and inform the design of collaborative robots that can adaptively balance task-oriented and social talk in contextually sensitive ways.
- 14 April 2026
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 11 (6) , 6991-6998
From Tool to Teammate: Balancing Task-Oriented and Social Talk During Human–Robot Collaboration
As robots become more integrated into shared work environments, effective communication must balance clear task directives with social engagement to support both efficiency and user satisfaction. This letter examines how people perceive task-oriented versus social dialogue during human–robot collaboration and how they judge the appropriateness of switching between them. We discuss a pilot, in-person data collection and two follow-up online studies with an autonomous robot in a collaborative pizza-making task. The pilot assesses the technical feasibility of an autonomous robot generating distinct task and social dialogue styles. Next, with a larger participant sample, we evaluate the salience of the two talk types and their impact on perceptions of the robot as a collaborative partner. The second study explores judgments of talk type switching using video clips from the pilot, revealing how timing, task state, and user behavior influence perceived appropriateness. Together, these findings highlight nuanced human expectations for robot communication and inform the design of collaborative robots that can adaptively balance task-oriented and social talk in contextually sensitive ways.
- Open Access
- 16 March 2026
- Published by Association for Computing Machinery (ACM)
- p. 68-77
Open-Ended Goal Inference through Actions and Language for Human-Robot Collaboration
- 16 March 2026
- Published by Association for Computing Machinery (ACM)
- p. 68-77
Open AccessOpen-Ended Goal Inference through Actions and Language for Human-Robot Collaboration
- Open Access
- 16 March 2026
- Published by Association for Computing Machinery (ACM)
- p. 98-107
The Dynamics of Human Fairness Judgments towards a Robot
Fairness is critical for collaboration between humans, and recent research has shown its importance in human–robot collaboration. However, most human-robot interaction (HRI) studies probe fairness judgments toward a robot only at the conclusion of an interaction, overlooking the fact that perceptions of fairness can evolve over time. We present two studies of dynamic fairness that both leverage a Multiplayer Space Invaders game, where a robot controls a spaceship and distributes support across players' sides of the screen. The robot's support is at times biased in favor of one player or the other. In the first study, we examine how fairness perceptions are influenced by the timing of a robot's biased support (early vs. late in the interaction) and the beneficiary of this support (the participant vs. another agent). In the second study, we investigate how expectations of a robot's support behavior (biased vs. unbiased) interact with its actual behavior (biased vs. unbiased) in a setting where two participants each worked to score an individual score threshold. We find that fairness judgments are dynamic: fairness falls after the robot's allocation of support becomes biased but is slower to recover once support becomes unbiased, and participants expecting unbiased behavior judge fairness more harshly when these expectations are violated. Our findings advance understanding of fairness in HRI by presenting it as a dynamic construct shaped not only by the actual behavior of the robot but also by the timing of robot actions and expectations of robot behavior.
- 16 March 2026
- Published by Association for Computing Machinery (ACM)
- p. 98-107
Open AccessThe Dynamics of Human Fairness Judgments towards a Robot
Fairness is critical for collaboration between humans, and recent research has shown its importance in human–robot collaboration. However, most human-robot interaction (HRI) studies probe fairness judgments toward a robot only at the conclusion of an interaction, overlooking the fact that perceptions of fairness can evolve over time. We present two studies of dynamic fairness that both leverage a Multiplayer Space Invaders game, where a robot controls a spaceship and distributes support across players' sides of the screen. The robot's support is at times biased in favor of one player or the other. In the first study, we examine how fairness perceptions are influenced by the timing of a robot's biased support (early vs. late in the interaction) and the beneficiary of this support (the participant vs. another agent). In the second study, we investigate how expectations of a robot's support behavior (biased vs. unbiased) interact with its actual behavior (biased vs. unbiased) in a setting where two participants each worked to score an individual score threshold. We find that fairness judgments are dynamic: fairness falls after the robot's allocation of support becomes biased but is slower to recover once support becomes unbiased, and participants expecting unbiased behavior judge fairness more harshly when these expectations are violated. Our findings advance understanding of fairness in HRI by presenting it as a dynamic construct shaped not only by the actual behavior of the robot but also by the timing of robot actions and expectations of robot behavior.
- Open Access
- 16 March 2026
- Published by Association for Computing Machinery (ACM)
- p. 1040-1049
Learning Human Preferences over a Human-Robot Collaboration Based on Explicit and Implicit Human Feedback
There is significant interest in enabling robots to learn to perform tasks directly from interactions with non-expert users. Typically, a human serves as a teacher whose only task is to provide feedback to a robot learner. However, in real-world human-robot collaborations, the human often assists with the task while also offering feedback. Our key insight is that we can extract additional, implicit feedback from the human’s actions during the collaboration to augment the robot learning process. Under the assumption of fixed-role assignments, we first propose to formalize human preferences over a human-robot collaboration as a shared set of parameters encoding alignment between two reward functions: one that drives human behavior, and another that should direct robot behavior. This allows us to extract implicit feedback from an interaction by reasoning about the human’s actions in the task as actions that reveal the human’s preferences. Then, we combine this implicit feedback with traditional explicit human feedback to facilitate estimating the human’s preferences. We evaluated our proposed approach for Preference learning from Implicit and Explicit feedback (PIE) in simulations and with real users in a cooking scenario. Our simulation results indicate that combining multiple modalities of human feedback improves a robot’s ability to estimate human preferences over the collaboration, with a similar trend observed in real-world evaluations. These findings highlight a promising direction for enabling robots to adapt to a user’s preference model more quickly, thereby reducing the amount of time a person must spend teaching a robot.
- 16 March 2026
- Published by Association for Computing Machinery (ACM)
- p. 1040-1049
Open AccessLearning Human Preferences over a Human-Robot Collaboration Based on Explicit and Implicit Human Feedback
There is significant interest in enabling robots to learn to perform tasks directly from interactions with non-expert users. Typically, a human serves as a teacher whose only task is to provide feedback to a robot learner. However, in real-world human-robot collaborations, the human often assists with the task while also offering feedback. Our key insight is that we can extract additional, implicit feedback from the human’s actions during the collaboration to augment the robot learning process. Under the assumption of fixed-role assignments, we first propose to formalize human preferences over a human-robot collaboration as a shared set of parameters encoding alignment between two reward functions: one that drives human behavior, and another that should direct robot behavior. This allows us to extract implicit feedback from an interaction by reasoning about the human’s actions in the task as actions that reveal the human’s preferences. Then, we combine this implicit feedback with traditional explicit human feedback to facilitate estimating the human’s preferences. We evaluated our proposed approach for Preference learning from Implicit and Explicit feedback (PIE) in simulations and with real users in a cooking scenario. Our simulation results indicate that combining multiple modalities of human feedback improves a robot’s ability to estimate human preferences over the collaboration, with a similar trend observed in real-world evaluations. These findings highlight a promising direction for enabling robots to adapt to a user’s preference model more quickly, thereby reducing the amount of time a person must spend teaching a robot.
- Open Access
Few-Shot Inference of Human Perceptions of Robot Performance in Social Navigation Scenarios
Understanding how humans evaluate robot behavior during human-robot interactions is crucial for developing socially aware robots that behave according to human expectations. While the traditional approach to capturing these evaluations is to conduct a user study, recent work has proposed utilizing machine learning instead. However, existing data-driven methods require large amounts of labeled data, which limits their use in practice. To address this gap, we propose leveraging the few-shot learning capabilities of Large Language Models (LLMs) to improve how well a robot can predict a user's perception of its performance, and study this idea experimentally in social navigation tasks. To this end, we extend the SEAN TOGETHER dataset with additional real-world human-robot navigation episodes and participant feedback. Using this augmented dataset, we evaluate the ability of several LLMs to predict human perceptions of robot performance from a small number of in-context examples, based on observed spatio-temporal cues of the robot and surrounding human motion. Our results demonstrate that LLMs can match or exceed the performance of traditional supervised learning models while requiring an order of magnitude fewer labeled instances. We further show that prediction performance can improve with more in-context examples, confirming the scalability of our approach. Additionally, we investigate what kind of sensor-based information an LLM relies on to make these inferences by conducting an ablation study on the input features considered for performance prediction. Finally, we explore the novel application of personalized examples for in-context learning, i.e., drawn from the same user being evaluated, finding that they further enhance prediction accuracy. This work paves the path to improving robot behavior in a scalable manner through user-centered feedback.
Open AccessFew-Shot Inference of Human Perceptions of Robot Performance in Social Navigation Scenarios
Understanding how humans evaluate robot behavior during human-robot interactions is crucial for developing socially aware robots that behave according to human expectations. While the traditional approach to capturing these evaluations is to conduct a user study, recent work has proposed utilizing machine learning instead. However, existing data-driven methods require large amounts of labeled data, which limits their use in practice. To address this gap, we propose leveraging the few-shot learning capabilities of Large Language Models (LLMs) to improve how well a robot can predict a user's perception of its performance, and study this idea experimentally in social navigation tasks. To this end, we extend the SEAN TOGETHER dataset with additional real-world human-robot navigation episodes and participant feedback. Using this augmented dataset, we evaluate the ability of several LLMs to predict human perceptions of robot performance from a small number of in-context examples, based on observed spatio-temporal cues of the robot and surrounding human motion. Our results demonstrate that LLMs can match or exceed the performance of traditional supervised learning models while requiring an order of magnitude fewer labeled instances. We further show that prediction performance can improve with more in-context examples, confirming the scalability of our approach. Additionally, we investigate what kind of sensor-based information an LLM relies on to make these inferences by conducting an ablation study on the input features considered for performance prediction. Finally, we explore the novel application of personalized examples for in-context learning, i.e., drawn from the same user being evaluated, finding that they further enhance prediction accuracy. This work paves the path to improving robot behavior in a scalable manner through user-centered feedback.
- 12 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 11 (2) , 1490-1497
I’ve Changed My Mind: Robots Adapting to Changing Human Goals During Collaboration
For effective human-robot collaboration, a robot must align its actions with human goals, even as they change mid-task. Prior approaches often assume fixed goals, reducing goal prediction to a one-time inference. However, in real-world scenarios, humans frequently shift goals, making it challenging for robots to adapt without explicit communication. We propose a method for detecting goal changes by tracking multiple candidate action sequences and verifying their plausibility against a policy bank. Upon detecting a change, the robot refines its belief in relevant past actions and constructs Receding Horizon Planning (RHP) trees to actively select actions that assist the human while encouraging Differentiating Actions to reveal their updated goal. We evaluate our approach in a collaborative cooking environment with up to 30 unique recipes and compare it to three comparable human goal prediction algorithms. Our method outperforms all baselines, quickly converging to the correct goal after a switch, reducing task completion time and improving collaboration efficiency.
- 12 December 2025
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 11 (2) , 1490-1497
I’ve Changed My Mind: Robots Adapting to Changing Human Goals During Collaboration
For effective human-robot collaboration, a robot must align its actions with human goals, even as they change mid-task. Prior approaches often assume fixed goals, reducing goal prediction to a one-time inference. However, in real-world scenarios, humans frequently shift goals, making it challenging for robots to adapt without explicit communication. We propose a method for detecting goal changes by tracking multiple candidate action sequences and verifying their plausibility against a policy bank. Upon detecting a change, the robot refines its belief in relevant past actions and constructs Receding Horizon Planning (RHP) trees to actively select actions that assist the human while encouraging Differentiating Actions to reveal their updated goal. We evaluate our approach in a collaborative cooking environment with up to 30 unique recipes and compare it to three comparable human goal prediction algorithms. Our method outperforms all baselines, quickly converging to the correct goal after a switch, reducing task completion time and improving collaboration efficiency.
- Open Access
The Social Context of Human-Robot Interactions
The Human-Robot Interaction (HRI) community often highlights the social context of an interaction as a key consideration when designing, implementing, and evaluating robot behavior. Unfortunately, researchers use the term "social context" in varied ways. This can lead to miscommunication, making it challenging to draw connections between related work on understanding and modeling the social contexts of human-robot interactions. To address this gap, we survey the HRI literature for existing definitions and uses of the term "social context". Then, we propose a conceptual model for describing the social context of a human-robot interaction. We apply this model to existing work, and we discuss a range of attributes of social contexts that can help researchers plan for interactions, develop behavior models for robots, and gain insights after interactions have taken place. We conclude with a discussion of open research questions in relation to understanding and modeling the social contexts of human-robot interactions.
Open AccessThe Social Context of Human-Robot Interactions
The Human-Robot Interaction (HRI) community often highlights the social context of an interaction as a key consideration when designing, implementing, and evaluating robot behavior. Unfortunately, researchers use the term "social context" in varied ways. This can lead to miscommunication, making it challenging to draw connections between related work on understanding and modeling the social contexts of human-robot interactions. To address this gap, we survey the HRI literature for existing definitions and uses of the term "social context". Then, we propose a conceptual model for describing the social context of a human-robot interaction. We apply this model to existing work, and we discuss a range of attributes of social contexts that can help researchers plan for interactions, develop behavior models for robots, and gain insights after interactions have taken place. We conclude with a discussion of open research questions in relation to understanding and modeling the social contexts of human-robot interactions.
- Open Access
- 18 April 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Human-Robot Interaction
- Vol. 14 (3) , 1-27
Predicting Human Perceptions of Robot Performance during Navigation Tasks
Understanding human perceptions of robot performance is crucial for designing socially intelligent robots that can adapt to human expectations. Current approaches often rely on surveys, which can disrupt ongoing human–robot interactions. As an alternative, we explore predicting people’s perceptions of robot performance using non-verbal behavioral cues and machine learning techniques. We contribute the SEAN TOGETHER Dataset consisting of observations of an interaction between a person and a mobile robot in Virtual Reality, together with perceptions of robot performance provided by users on a 5-point scale. We then analyze how well humans and supervised learning techniques can predict perceived robot performance based on different observation types (like facial expression and spatial behavior features). Our results suggest that facial expressions alone provide useful information, but in the navigation scenarios that we considered, reasoning about spatial features in context is critical for the prediction task. Also, supervised learning techniques outperformed humans’ predictions in most cases. Further, when predicting robot performance as a binary classification task on unseen users’ data, the \(F_{1}\) -Score of machine learning models more than doubled that of predictions on a 5-point scale. This suggested good generalization capabilities, particularly in identifying performance directionality over exact ratings. Based on these findings, we conducted a real-world demonstration where a mobile robot uses a machine learning model to predict how a human who follows it perceives it. Finally, we discuss the implications of our results for implementing these supervised learning models in real-world navigation. Our work paves the path to automatically enhancing robot behavior based on observations of users and inferences about their perceptions of a robot.- 18 April 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Human-Robot Interaction
- Vol. 14 (3) , 1-27
Open AccessPredicting Human Perceptions of Robot Performance during Navigation Tasks
Understanding human perceptions of robot performance is crucial for designing socially intelligent robots that can adapt to human expectations. Current approaches often rely on surveys, which can disrupt ongoing human–robot interactions. As an alternative, we explore predicting people’s perceptions of robot performance using non-verbal behavioral cues and machine learning techniques. We contribute the SEAN TOGETHER Dataset consisting of observations of an interaction between a person and a mobile robot in Virtual Reality, together with perceptions of robot performance provided by users on a 5-point scale. We then analyze how well humans and supervised learning techniques can predict perceived robot performance based on different observation types (like facial expression and spatial behavior features). Our results suggest that facial expressions alone provide useful information, but in the navigation scenarios that we considered, reasoning about spatial features in context is critical for the prediction task. Also, supervised learning techniques outperformed humans’ predictions in most cases. Further, when predicting robot performance as a binary classification task on unseen users’ data, the \(F_{1}\) -Score of machine learning models more than doubled that of predictions on a 5-point scale. This suggested good generalization capabilities, particularly in identifying performance directionality over exact ratings. Based on these findings, we conducted a real-world demonstration where a mobile robot uses a machine learning model to predict how a human who follows it perceives it. Finally, we discuss the implications of our results for implementing these supervised learning models in real-world navigation. Our work paves the path to automatically enhancing robot behavior based on observations of users and inferences about their perceptions of a robot.