MHMuhan Hou

Muhan Hou

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Muhan Hou has a total of 7 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Active Robot Learning from Demonstrations with Human Teachers

    by Muhan Hou

    Learning from Demonstrations (LfD) enables robots to acquire new skills by imitating a domain expert, most commonly a human. In practice, however, human demonstrations are typically available only under a limited budget and rarely cover the full range of situations a robot may encounter. This limitation often induces a distribution shift between the states represented in offline demonstrations and those visited by the robot during online exploration, which can substantially degrade performance. Interactive Imitation Learning (IIL) mitigates this shift by keeping the human teachers in the learning loop, allowing them to provide online input such as feedback and corrections. Yet these benefits come at the cost of sustained human supervision and can be undermined by noise, inconsistency, and errors in human teaching. This thesis investigates Active Learning from Demonstrations (Active LfD) as a principled approach to reduce distribution shift while accounting for human factors in realistic human-in-the-loop settings. In Active LfD, the robot learner does not passively consume demonstrations. Instead, it optimizes its query decisions to selectively request demonstrations from a human teacher. By determining when and what to query, the learner aims to reduce reliance on continuous monitoring, focus human effort where it is most beneficial, and limit the impact of suboptimal human teaching decisions. The thesis is organized into three parts. In Part I, I lay the foundation for how a robot learner can actively shape human teaching by revisiting conventional offline LfD through the lens of demonstration distributions. After identifying biased human teaching strategies under unguided conditions, I develop an Active LfD algorithm that operates with offline demonstrations and guides the resulting demonstration distribution toward a more balanced one, or more generally toward any specified target distribution, so as to improve robot learning. Importantly, however, a ``balanced'' distribution (e.g., a uniform distribution used as a default) is not necessarily optimal for learning. The most beneficial demonstration distribution depends on the robot’s evolving policy and its online exploration, and is therefore difficult to specify a priori. To address this coupling, I present a second Active LfD algorithm that leverages online demonstrations, iteratively deciding both the timing and content of queries as learning progresses in order to optimize the demonstration distribution for the learner. In Part II, I explore how Active LfD extends to more realistic and complex scenarios of learning from human teachers. I first extend the paradigm to transfer learning for complex manipulation tasks, spanning a range of policy transfer scenarios. I then consider the practical case of non-expert human teachers, who may provide imperfect demonstrations relative to the task objective. To accommodate imperfect teaching, I develop an Active LfD framework that optimizes the query sequence by jointly accounting for both the expected improvement in the robot’s policy induced by a query and the teacher’s capability to provide informative guidance in the queried region. Part III closes the loop of human-robot interactive learning by examining how robot query design influences human teaching beyond user experience alone. I leverage Curriculum Learning to design an Active LfD algorithm that benefits both robot learning and human teaching, encouraging a reciprocal loop between the robot learner and the human teacher.

  • Open Access

    Static Is Not Enough: A Comparative Study of VR and SpaceMouse in Static and Dynamic Teleoperation Tasks

    by Yijun Zhou, Muhan Hou, Kim Baraka

    Imitation learning relies on high-quality demonstrations, and teleoperation is a primary way to collect them, making teleoperation interface choice crucial for the data. Prior work mainly focused on static tasks, i.e., discrete, segmented motions, yet demonstrations also include dynamic tasks requiring reactive control. As dynamic tasks impose fundamentally different interface demands, insights from static-task evaluations cannot generalize. To address this gap, we conduct a within-subjects study comparing a VR controller and a SpaceMouse across two static and two dynamic tasks ($N=25$). We assess success rate, task duration, cumulative success, alongside NASA-TLX, SUS, and open-ended feedback. Results show statistically significant advantages for VR: higher success rates, particularly on dynamic tasks, shorter successful execution times across tasks, and earlier successes across attempts, with significantly lower workload and higher usability. As existing VR teleoperation systems are rarely open-source or suited for dynamic tasks, we release our VR interface to fill this gap.

  • Open Access

    Robot Policy Transfer with Online Demonstrations: An Active Reinforcement Learning Approach

    by Muhan Hou, Koen Hindriks, A. E. Eiben, Kim Baraka

    Transfer Learning (TL) is a powerful tool that enables robots to transfer learned policies across different environments, tasks, or embodiments. To further facilitate this process, efforts have been made to combine it with Learning from Demonstrations (LfD) for more flexible and efficient policy transfer. However, these approaches are almost exclusively limited to offline demonstrations collected before policy transfer starts, which may suffer from the intrinsic issue of covariance shift brought by LfD and harm the performance of policy transfer. Meanwhile, extensive work in the learning-from-scratch setting has shown that online demonstrations can effectively alleviate covariance shift and lead to better policy performance with improved sample efficiency. This work combines these insights to introduce online demonstrations into a policy transfer setting. We present Policy Transfer with Online Demonstrations, an active LfD algorithm for policy transfer that can optimize the timing and content of queries for online episodic expert demonstrations under a limited demonstration budget. We evaluate our method in eight robotic scenarios, involving policy transfer across diverse environment characteristics, task objectives, and robotic embodiments, with the aim to transfer a trained policy from a source task to a related but different target task. The results show that our method significantly outperforms all baselines in terms of average success rate and sample efficiency, compared to two canonical LfD methods with offline demonstrations and one active LfD method with online demonstrations. Additionally, we conduct preliminary sim-to-real tests of the transferred policy on three transfer scenarios in the real-world environment, demonstrating the policy effectiveness on a real robot manipulator.

  • Open Access

    Active Robot Curriculum Learning from Online Human Demonstrations

    by Muhan Hou, Koen Hindriks, A. E. Eiben, Kim Baraka

    Learning from Demonstrations (LfD) allows robots to learn skills from human users, but its effectiveness can suffer due to sub-optimal teaching, especially from untrained demonstrators. Active LfD aims to improve this by letting robots actively request demonstrations to enhance learning. However, this may lead to frequent context switches between various task situations, increasing the human cognitive load and introducing errors to demonstrations. Moreover, few prior studies in active LfD have examined how these active query strategies may impact human teaching in aspects beyond user experience, which can be crucial for developing algorithms that benefit both robot learning and human teaching. To tackle these challenges, we propose an active LfD method that optimizes the query sequence of online human demonstrations via Curriculum Learning (CL), where demonstrators are guided to provide demonstrations in situations of gradually increasing difficulty. We evaluate our method across four simulated robotic tasks with sparse rewards and conduct a user study (N=26) to investigate the influence of active LfD methods on human teaching regarding teaching performance, post-guidance teaching adaptivity, and teaching transferability. Our results show that our method significantly improves learning performance compared to three other LfD baselines in terms of the final success rate of the converged policy and sample efficiency. Additionally, results from our user study indicate that our method significantly reduces the time required from human demonstrators and decreases failed demonstration attempts. It also enhances post-guidance human teaching in both seen and unseen scenarios compared to another active LfD baseline, indicating enhanced teaching performance, greater post-guidance teaching adaptivity, and better teaching transferability achieved by our method.