Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Yajie Bao has a total of 32 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Learning-Based Resilience Optimization of Human-on-the-Loop Multiple-Unmanned-Aerial-System Positioning Using Mean Field Games

    by Yajie Bao, Dan Shen, Genshe Chen, Khanh Pham, Erik Blasch

    This article presents a learning-based game-theoretic approach for resilience optimization of human-on-the-loop (HotL) positioning for multiple unmanned aerial systems (UASs). In particular, mean field games (MFGs) are used to obtain the [ ]-Nash equilibrium policies for a large population of non-cooperative UASs that compete for limited human resources to maintain high positioning accuracy. The resilience dynamics model is learned from the HotL multi-UAS positioning system data. Furthermore, an improved actor–critic–mass algorithm is proposed to enhance the learning efficiency of the near-optimal policies. Then, the policies are exploited to guide human–machine interactions. Simulations of a HotL multi-UAS positioning system demonstrate that the proposed MFG-resilient optimization of the multi-UAS approach enhances the system’s resilience and increases the success rate of navigating to the desired positions compared to the approaches without using the MFG policies.


  • Feedback for Improved Hierarchical Reinforcement Learning With Timed Subgoals

    by Yajie Bao, Dan Shen, Genshe Chen, Hao Xu, Samson Badlia, Simon Khan, Erik Blasch, Khanh Pham

    Hierarchical reinforcement learning (RL) aims to improve sample efficiency by decomposing complex long-horizon tasks into fast low-level myopic and slower high-level non-myopic subtasks. However, the unilateral nested policy structure in current goal-conditioned hierarchical RL (HRL) methods sets subgoals at the high level without considering feedback from the low level, which significantly degrades the performance of high-level subgoal generation and sampling efficiency. Hindsight action relabeling further weakens subgoal settings by submitting to low-level reachability. Inspired by feedback control of dynamic systems, we present Feedback for Improved HRL with Timed Subgoals (FIHTS), a mechanism allowing feedback control of subgoal generation for improved HRL. Unlike current HRL, FIHTS enables both the high level to set subgoals and the low level to receive rewards based on subgoal achievement. Our experiments in various challenging dynamic RL environments show that our FIHTS method achieves higher success rates with higher sample efficiency than existing subgoal-based HRL methods.


  • Adaptive Multi-stage Sensor Fusion Under Neuro-Symbolic Framework for The Multi-modal Ranging System in Adverse Weather Conditions

    by Yajie Bao, Peng Cheng, Ping Zhuang, Yunqi Zhang, Zhengyang Fan, Genshe Chen, Erik Blasch, Khanh Pham


  • Open Access

    Real-Time Proprioceptive Sensing Enhanced Switching Model Predictive Control for Quadruped Robot Under Uncertain Environment

    by Sanket Lokhande, Yajie Bao, Peng Cheng, Dan Shen, Genshe Chen, Hao Xu

    Quadruped robots have shown significant potential in disaster relief applications, where they have to navigate complex terrains for search and rescue or reconnaissance operations. However, their deployment is hindered by limited adaptability in highly uncertain environments, especially when relying solely on vision-based sensors like cameras or LiDAR, which are susceptible to occlusions, poor lighting, and environmental interference. To address these limitations, this paper proposes a novel sensor-enhanced hierarchical switching model predictive control (MPC) framework that integrates proprioceptive sensing with a bi-level hybrid dynamic model. Unlike existing methods that either rely on handcrafted controllers or deep learning-based control pipelines, our approach introduces three core innovations: (1) a situation-aware, bi-level hybrid dynamic modeling strategy that hierarchically combines single-body rigid dynamics with distributed multi-body dynamics for modeling agility and scalability; (2) a three-layer hybrid control framework, including a terrain-aware switching MPC layer, a distributed torque controller, and a fast PD control loop for enhanced robustness during contact transitions; and (3) a multi-IMU-based proprioceptive feedback mechanism for terrain classification and adaptive gait control under sensor-occluded or GPS-denied environments. Together, these components form a unified and computationally efficient control scheme that addresses practical challenges such as limited onboard processing, unstructured terrain, and environmental uncertainty. A series of experimental results demonstrate that the proposed method outperforms existing vision- and learning-based controllers in terms of stability, adaptability, and control efficiency during high-speed locomotion over irregular terrain.


  • Open Access

    Learning-Assisted Multi-IMU Proprioceptive State Estimation for Quadruped Robots

    by Xuanning Liu, Yajie Bao, Peng Cheng, Dan Shen, Zhengyang Fan, Hao Xu, Genshe Chen

    This paper presents a learning-assisted approach for state estimation of quadruped robots using observations of proprioceptive sensors, including multiple inertial measurement units (IMUs). Specifically, one body IMU and four additional IMUs attached to each calf link of the robot are used for sensing the dynamics of the body and legs, in addition to joint encoders. The extended Kalman filter (KF) is employed to fuse sensor data to estimate the robot’s states in the world frame and enhance the convergence of the extended KF (EKF). To circumvent the requirements for the measurements from the motion capture (mocap) system or other vision systems, the right-invariant EKF (RI-EKF) is extended to employ the foot IMU measurements for enhanced state estimation, and a learning-based approach is presented to estimate the vision system measurements for the EKF. One-dimensional convolutional neural networks (CNN) are leveraged to estimate required measurements using only the available proprioception data. Experiments on real data from a quadruped robot demonstrate that proprioception can be sufficient for state estimation. The proposed learning-assisted approach, which does not rely on data from vision systems, achieves competitive accuracy compared to EKF using mocap measurements and lower estimation errors than RI-EKF using multi-IMU measurements.


  • Open Access

    Learning-Assisted Multi-IMU Proprioceptive State Estimation for Quadruped Robots

    by Xuanning Liu, Yajie Bao, Peng Cheng, Dan Shen, Zhengyang Fan, Hao Xu, Genshe Chen

    This paper presents a learning-assisted approach for state estimation of quadruped robots using observations of proprioceptive sensors including multiple inertial measurement units (IMUs). Specifically, one body IMU and four additional IMUs attached to each calf link of the robot are used for sensing the dynamics of the body and legs, in addition to joint encoders. The extended Kalman filter (EKF) is employed to fuse sensor data to estimate the robot’s states in the world frame and enhance the convergence of EKF. To circumvent the requirements for the measurements from the motion capture (mocap) system or other vision systems, the right-invariant EKF (RI-EKF) is extended to employ the foot IMU measurements for enhanced state estimation, and a learning-based approach is presented to estimate the vision system measurements for the EKF. 1D convolutional neural networks (CNN) are leveraged to estimate required measurements using only the available proprioception data. Experiments on the real data of a quadruped robot demonstrate that proprioception can be sufficient for state estimation, and the proposed learning-assisted approach without using data from vision systems can achieve competitive accuracy compared to EKF using the mocap measurements and smaller estimation errors than RI-EKF using multi-IMU measurements.


  • Open Access

    Adaptive Uncertainty Quantification for Scenario-based Control Using Meta-learning of Bayesian Neural Networks

    by Yajie Bao, Javad Mohammadpour Velni

    Scenario-based optimization and control has proven to be an efficient approach to account for system uncertainty. In particular, the performance of scenario-based model predictive control (MPC) schemes depends on the accuracy of uncertainty quantification. However, current learning- and scenario-based MPC (sMPC) approaches employ a single timeinvariant probabilistic model (learned offline), which may not accurately describe time-varying uncertainties. Instead, this paper presents a model-agnostic meta-learning (MAML) of Bayesian neural networks (BNN) for adaptive uncertainty quantification that would be subsequently used for adaptive-scenario-tree model predictive control design of nonlinear systems with unknown dynamics to enhance control performance. In particular, the proposed approach learns both a global BNN model and an updating law to refine the BNN model. At each time step, the updating law transforms the global BNN model into more precise local BNN models in real time. The adapted local model is then used to generate scenarios for sMPC design at each time step. A probabilistic safety certificate is incorporated in the scenario generation to ensure that the trajectories of the generated scenarios contain the real trajectory of the system and that all the scenarios adhere to the constraints with a high probability. Experiments using closed-loop simulations of a numerical example demonstrate that the proposed approach can improve the performance of scenario-based MPC compared to using only one BNN model learned offline for all time steps.

  • Scenario-Based Hybrid Model Predictive Design for Cooperative Adaptive Cruise Control in Mixed-Autonomy Environments

    by Sahand Mosharafian, Yajie Bao, Javad Mohammadpour Velni

    This paper presents a scenario-based hybrid model predictive control (MPC) design approach for cooperative adaptive cruise control (CACC) in mixed-autonomy traffic environments with uncertainties stemming from unexpected maneuvers of human-driven vehicles. Different from the past works that consider one possible realization of uncertainty, the proposed approach here considers multiple scenarios based on the uncertainty description that varies with the relative location of the human-driven vehicles (HVs) and the connected and automated vehicles (CAVs), For each scenario, a mixed integer quadratic programming problem is formulated for the control of CAVs with four operating modes, namely free following, braking, danger, and lane change. Each CAV's operating mode is determined based on the predictive information it receives from its predecessors and the anticipated behaviors of surrounding HVs. All the scenarios are handled simultaneously using the scenario-based MPC approach for a robust CACC. Simulations in a mixed-autonomy traffic system including two lanes demonstrate that the proposed scenario-based hybrid MPC approach significantly reduces deviations from the desired spacing policy and the desired velocity in the platoon during unexpected human-driven vehicle maneuvers, compared with the past work, particularly, a discrete hybrid stochastic (DHSA) MPC approach.