HWHaochen Wu

Haochen Wu

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Haochen Wu has a total of 19 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    BOSA: Bayesian Online Strategy Adaptation for Unexpected Events in Multi-Agent Teams

    by Haochen Wu, Amin Ghadami, Alparslan Emrah Bayrak, Jonathon M. Smereka, Bogdan I. Epureanu

    Reinforcement learning provides multi-agent systems with team strategies to effectively operate in dynamic yet enclosed environments, but the learned expertise does not generalize to novelty, i.e., unexpected scenarios diverging from the known properties of an initial environment. Adaptation is particularly crucial for multi-agent teams in collaborative tasks because even minor disturbances can cascade into failures. Continual learning or online expertise adjustment methods require re-training and lack flexibility, making them inefficient and unrealistic to deploy in urgent and fluctuating situations. To enable efficient adaptation, we propose a Bayesian Online Strategy Adaptation (BOSA). Our approach achieves instant online adaptation at the team strategy level by preparing a strategy pool offline, observing unusual task dynamics caused by novelty, and inferring the most suitable strategy to adapt through Bayesian reasoning and strategy selection heuristics. Experiments are designed to validate the strategy inference mechanism and the adaptation to unseen agent capability degradation scenarios. We investigate how BOSA can scale for large operations and demonstrate that BOSA can effectively address sudden capability degradation events within one decision step with a strategy selection heuristic on belief inclination

  • Open Access

    SafeAR: Safe Algorithmic Recourse by Risk-Aware Policies

    by Haochen Wu, Shubham Sharma, Sunandita Patra, Sriram Gopalakrishnan

    With the growing use of machine learning (ML) models in critical domains such as finance and healthcare, the need to offer recourse for those adversely affected by the decisions of ML models has become more important; individuals ought to be provided with recommendations on actions to take for improving their situation and thus receiving a favorable decision. Prior work on sequential algorithmic recourse---which recommends a series of changes---focuses on action feasibility and uses the proximity of feature changes to determine action costs. However, the uncertainties of feature changes and the risk of higher than average costs in recourse have not been considered. It is undesirable if a recourse could (with some probability) result in a worse situation from which recovery requires an extremely high cost. It is essential to incorporate risks when computing and evaluating recourse. We call the recourse computed with such risk considerations as Safe Algorithmic Recourse (SafeAR). The objective is to empower people to choose a recourse based on their risk tolerance. In this work, we discuss and show how existing recourse desiderata can fail to capture the risk of higher costs. We present a method to compute recourse policies that consider variability in cost and connect algorithmic recourse literature with risk-sensitive reinforcement learning. We also adopt measures "Value at Risk" and "Conditional Value at Risk" from the financial literature to summarize risk concisely. We apply our method to two real-world datasets and compare policies with different risk-aversion levels using risk measures and recourse desiderata (sparsity and proximity).


  • Open Access

    SafeAR: Safe Algorithmic Recourse by Risk-Aware Policies

    by Haochen Wu, Shubham Sharma, Sunandita Patra, Sriram Gopalakrishnan

    With the growing use of machine learning (ML) models in critical domains such as finance and healthcare, the need to offer recourse for those adversely affected by the decisions of ML models has become more important; individuals ought to be provided with recommendations on actions to take for improving their situation and thus receiving a favorable decision. Prior work on sequential algorithmic recourse -- which recommends a series of changes -- focuses on action feasibility and uses the proximity of feature changes to determine action costs. However, the uncertainties of feature changes and the risk of higher than average costs in recourse have not been considered. It is undesirable if a recourse could (with some probability) result in a worse situation from which recovery requires an extremely high cost. It is essential to incorporate risks when computing and evaluating recourse. We call the recourse computed with such risk considerations as Safe Algorithmic Recourse (SafeAR). The objective is to empower people to choose a recourse based on their risk tolerance. In this work, we discuss and show how existing recourse desiderata can fail to capture the risk of higher costs. We present a method to compute recourse policies that consider variability in cost and connect algorithmic recourse literature with risk-sensitive reinforcement learning. We also adopt measures "Value at Risk" and "Conditional Value at Risk" from the financial literature to summarize risk concisely. We apply our method to two real-world datasets and compare policies with different risk-aversion levels using risk measures and recourse desiderata (sparsity and proximity).

  • Open Access

    Multiagent Inverse Reinforcement Learning via Theory of Mind Reasoning

    by Haochen Wu, Pedro Sequeira, David V. Pynadath

    We approach the problem of understanding how people interact with each other in collaborative settings, especially when individuals know little about their teammates, via Multiagent Inverse Reinforcement Learning (MIRL), where the goal is to infer the reward functions guiding the behavior of each individual given trajectories of a team's behavior during some task. Unlike current MIRL approaches, we do not assume that team members know each other's goals a priori; rather, that they collaborate by adapting to the goals of others perceived by observing their behavior, all while jointly performing a task. To address this problem, we propose a novel approach to MIRL via Theory of Mind (MIRL-ToM). For each agent, we first use ToM reasoning to estimate a posterior distribution over baseline reward profiles given their demonstrated behavior. We then perform MIRL via decentralized equilibrium by employing single-agent Maximum Entropy IRL to infer a reward function for each agent, where we simulate the behavior of other teammates according to the time-varying distribution over profiles. We evaluate our approach in a simulated 2-player search-and-rescue operation where the goal of the agents, playing different roles, is to search for and evacuate victims in the environment. Our results show that the choice of baseline profiles is paramount to the recovery of the ground-truth rewards, and that MIRL-ToM is able to recover the rewards used by agents interacting both with known and unknown teammates.


  • Open Access

    Evaluating Emergent Coordination in Multi-Agent Task Allocation Through Causal Inference and Sub-Team Identification

    by Haochen Wu, Amin Ghadami, Alparslan Emrah Bayrak, Jonathon M. Smereka, Bogdan I. Epureanu

    Coordination in multi-agent systems is a vital component in teaming effectiveness. In dynamically changing situations, agent decisions depict emergent coordination strategies from following pre-defined rules to exploiting incentive-driven policies. While multi-agent reinforcement learning shapes team behaviors from experience, interpreting learned coordination strategies offers benefits in understanding complex agent dynamics and further improvement in developing adaptive strategies for evolving and unexpected situations. In this work, we develop an approach to quantitatively measure team coordination by collecting decision time series data, detecting causality between agents, and identifying statistically high coordinated sub-teams in decentralized multi-agent task allocation operations. We focus on multi-agent systems with homogeneous agents and homogeneous tasks as the strategy formation is more ambiguous and challenging than heterogeneous teams with specialized capabilities. Emergent team coordination is then analyzed using rule-based and reinforcement learning-based strategies for task allocation in operations at different demand stages (stress) levels. We also investigate correlation vs. causation and agent over- or under-estimating demand levels.


  • Investigating the Potential of Truck Platooning for Energy Savings: Empirical Study of the U.S. National Highway Freight Network

    by Xiaotong Sun, Haochen Wu, Mojtaba Abdolmaleki, Yafeng Yin, Bo Zou

    Truck platooning enabled by connected automated vehicle (CAV) technology has been demonstrated to effectively reduce fuel consumption for trucks in a platoon. However, given the limited number of trucks in the traffic stream, it remains questionable how great an energy saving it may yield for a practical freight system if we only rely on ad-hoc platooning. Assuming the presence of a central platooning coordinator, this paper is offered to substantiate truck platooning benefits in fuel economy produced by exploiting platooning opportunities arising from the United States’ domestic truck demands on its highway freight network. An integer programming model is utilized to schedule trucks’ itineraries to facilitate the formation of platoons at platoonable locations to maximize energy savings. A simplification of the real freight network and an approximation algorithm are used to solve the model efficiently. By analyzing the numerical results obtained, this study quantifies the importance of scheduled platooning in improving trucks’ fuel economy. Furthermore, the allowable platoon size, schedule flexibility, and fuel efficiency all play a crucial role in energy savings. Specifically, by assuming that following vehicles in a platoon obtain a 10% energy reduction, an average energy reduction of 8.48% per truck can be achieved for the overall network if the maximum platoon size is seven, and the schedule flexibility is 30 min. The cost–benefit analysis provided at the end suggests that the energy-saving benefits can offset the investment cost in truck platooning technology.


  • Open Access

    Impact of Heterogeneity and Risk Aversion on Task Allocation in Multi-Agent Teams

    by Haochen Wu, Amin Ghadami, Alparslan Emrah Bayrak, Jonathon M. Smereka, Bogdan I. Epureanu

    Cooperative multi-agent decision-making is a ubiquitous problem with many real-world applications. In many practical applications, it is desirable to design a multi-agent team with a heterogeneous composition where the agents can have different capabilities and levels of risk tolerance to address diverse requirements. While heterogeneity in multi-agent teams offers benefits, new challenges arise including how to find optimal heterogeneous team compositions and how to dynamically distribute tasks among agents in complex operations. In this work, we develop an artificial intelligence framework for multi-agent heterogeneous teams to dynamically learn task distributions among agents through reinforcement learning. The framework extends Decentralized Partially Observable Markov Decision Processes (Dec-POMDP) to be compatible to model various types of heterogeneity. We demonstrate our approach with a benchmark problem on a disaster relief scenario. The effect of heterogeneity and risk aversion in agent capabilities and decision-making strategies on the performance of multi-agent teams in uncertain environments is analyzed. Results show that a well-designed heterogeneous team outperforms its homogeneous counterpart and possesses higher adaptivity in uncertain environments.


  • Open Access

    Case Study 1

    by Haochen Wu, Cindy L. Bethel, Yue Wang, Huanfei Zheng, Bogdan Epureanu, Amin Ghadami

    There is no I in team: optimal task allocation in human-autonomy teams Bogdan I. Epureanu, Tulga Ersal, Haochen Wu†, Connor Esterwood†, Timothy Ohtake† (UM), Emrah Bayrak (SIT), Mert Egilmez*(Veoneer-Nissin), Cindy L. Bethel, Jessie E. Cossitt† (MSU), Yue Wang, Huanfei Zheng†, Ashit Mohanty† (CU), Victor Paul§, John Brabbs§, Jillyn Alban§, Jonathon Smereka§(GVSC) (†Student, *

  • Open Access

    Project 2.13 Optimal Distribution of Tasks in Human-Autonomy Teams

    by Haochen Wu, Amin Ghadami, Bogdan Epureanu

    Project 2.13 Optimal Distribution of Tasks in Human-Autonomy Teams