ManiVID-3D: Generalizable View-Invariant Reinforcement Learning for Robotic Manipulation via Disentangled 3D Representations
- 9 February 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Robotics and Automation Letters
- Vol. 11 (4) , 4235-4242
Abstract
Deploying visual reinforcement learning (RL) policies in real-world manipulation is often hindered by camera viewpoint changes. A policy trained from a fixed front-facing camera may fail when the camera is shifted—an unavoidable situation in real-world settings where sensor placement is hard to manage appropriately. Existing methods often rely on precise camera calibration or struggle with large perspective changes. To address these limitations, we propose ManiVID-3D, a novel 3D RL architecture designed for robotic manipulation, which learns view-invariant representations through self-supervised disentangled feature learning. The framework incorporates ViewNet, a lightweight yet effective module that automatically aligns point cloud observations from arbitrary viewpoints into a unified spatial coordinate system without the need for extrinsic calibration. Additionally, we develop an efficient GPU-accelerated batch rendering module capable of processing over 5000 frames per second, enabling large-scale training for 3D visual RL at unprecedented speeds. Extensive evaluation across 10 simulated and 5 real-world tasks demonstrates that our approach achieves a 40.6% higher success rate than state-of-the-art methods under viewpoint variations while using 80% fewer parameters. The system’s robustness to severe perspective changes and strong sim-to-real performance highlight the effectiveness of learning geometrically consistent representations for scalable robotic manipulation in unstructured environments.Keywords
Funding Information
- Guangdong Provincial Educational Science Planning Project (2025GXJK0629)
- Guangzhou Higher Education Teaching Research and Reform Project (2024YBJG094)
- Guangzhou Municipal Educational Science Planning Project (202316577)
This publication has 22 references indexed in Scilit:
- Generalizable Humanoid Manipulation with 3D Diffusion Policies2025
- A Comprehensive Survey of Data Augmentation in Visual Reinforcement LearningInternational Journal of Computer Vision, 2025
- L-VAE: variational auto-encoder with learnable beta for disentangled representationMachine Vision and Applications, 2025
- RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins2025
- G3Flow: Generative 3D Semantic Flow for Pose-aware and Generalizable Object Manipulation2025
- Point Cloud Models Improve Visual Robustness in Robotic Learners2024
- Masked Autoencoders Are Scalable Vision Learners2022
- Deep reinforcement learning based moving object graspingInformation Sciences, 2021
- SAPIEN: A SimulAted Part-Based Interactive ENvironment2020
- MuJoCo: A physics engine for model-based control2012