Distance-Aware World Model-Based Reinforcement Learning for Mobile Manipulation Behaviors
Open Access
- 21 May 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Access
- Vol. 14, 78614-78629
Abstract
Separate control of mobile manipulation offers a promising paradigm for managing its complexity by planning for these modalities independently. However, end-to-end learning of such behaviors necessitates effective mechanisms for both embodiment selection and motion planning. For low-level motion planning, prior world model-based reinforcement learning (RL) approaches have employed the similarity of latent states as a dense reward proxy. However, using latent similarity as an implicit distance metric yields inaccurate representations and depends heavily on predefined goal states. In this work, we address the limitations of similarity-based rewards by explicitly integrating 3D spatial information into the world model and deriving more accurate rewards from that distance-aware model for policy training. For high-level embodiment selection, prior model-based methods have either utilized reachability rewards inferred from predicted task rewards or relied on heuristic composite rewards. In this study, we retain the reachability reward and demonstrate that it yields superior performance in embodiment selection compared with composite reward formulations. By combining reachability-based selection and distance-guided planning, the model improves the performance of end-to-end learning for separate mobile manipulation behaviors. Extensive evaluations were conducted across two simulation environments with both fixed and randomized target settings. For motion planning, the distance reward achieves near 100% success rates across all configurations, significantly outperforming the baseline similarity reward, which achieves only 60% success in fixed-target scenarios and drops below 20% when targets are randomized. Regarding embodiment selection, the reachability-based separate control rapidly achieves a stable result with over 85% of arm activations occurring rationally within the near-goal region. Conversely, the heuristic composite reward slowly increases to a maximum of 60% for near-goal arm selections. These findings validate that the reachability reward provides a more effective and efficient mechanism for embodiment selection.Keywords
Funding Information
- Japan Agency for Medical Research and Development (JPMJMS2011)
This publication has 14 references indexed in Scilit:
- Mastering diverse control tasks through world modelsNature, 2025
- Predictive Reachability for Embodiment Selection in Mobile Manipulation BehaviorsIEEE Robotics and Automation Letters, 2025
- N$^{2}$M$^{2}$: Learning Navigation for Arbitrary Mobile Manipulation Motions in Unseen and Dynamic EnvironmentsIEEE Transactions on Robotics, 2023
- Robot Learning of Mobile Manipulation With Reachability Behavior PriorsIEEE Robotics and Automation Letters, 2022
- A survey of the development of quadruped robots: Joint configuration, dynamic locomotion control method and mobile manipulation approachBiomimetic Intelligence and Robotics, 2022
- Learning Kinematic Feasibility for Mobile Manipulation Through Deep Reinforcement LearningIEEE Robotics and Automation Letters, 2021
- ReLMoGen: Integrating Motion Generation in Reinforcement Learning for Mobile Manipulation2021
- Stance selection for humanoid grasping tasks by inverse reachability maps2015
- Robot placement based on reachability inversion2013
- Capturing robot workspace structure: representing robot capabilities2007