Canonical Policy: Learning Canonical 3-D Representation for $\text{SE}(3)$-Equivariant Policy
- 26 May 2026
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Robotics
- Vol. 42 (15523098) , 2325-2344
Abstract
Visual imitation learning has achieved remarkable progress in robotic manipulation, yet generalization to unseen objects, scene layouts, and camera viewpoints remains a key challenge. Recent advances address this by using 3D point clouds, which provide geometry-aware, appearance-invariant representations, and by incorporating equivariance into policy architectures to exploit spatial symmetries. However, existing equivariant approaches often lack interpretability and rigor due to unstructured integration of equivariant components. We introduce canonical policy, a principled framework for 3D equivariant imitation learning that unifies 3D point cloud observations under a canonical representation. We first establish a theory of 3D canonical representations, enabling equivariant observation-to-action mappings by grouping both seen and novel point clouds to a canonical representation. We then propose a flexible policy learning pipeline that leverages geometric symmetries from canonical representation and the expressiveness of modern generative models. We validate canonical policy on 12 diverse simulated tasks and 4 real-world manipulation tasks across 16 configurations, involving variations in object color, shape, camera viewpoint, and robot platform. Compared to state-of-the-art imitation learning policies, canonical policy achieves an average improvement of 18.0% in simulation and 39.7% in real-world experiments, demonstrating superior generalization capability and sample efficiency.Keywords
All Related Versions
Funding Information
- National Science Foundation (2322056)
- National Science Foundation (2423068)
- National Science Foundation (2520136)
- U.S. Department of Agriculture (2023-67021-39072)
- U.S. Department of Agriculture (2024-67021-42878)
This publication has 30 references indexed in Scilit:
- AnySkin: Plug-and-Play Skin Sensing for Robotic Touch2025
- Learning Visuotactile Skills With Two Multifingered Hands2025
- Symmetric Models for Visual Force Policy Learning2024
- RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunking2024
- Symmetry-aware Reinforcement Learning for Robotic Assembly under Partial Observability with a Soft Wrist2024
- SEIL: Simulation-augmented Equivariant Imitation Learning2023
- Vector Neurons: A General Framework for SO(3)-Equivariant Networks2021
- Point Transformer2021
- Highly accurate protein structure prediction with AlphaFoldNature, 2021
- On the Continuity of Rotation Representations in Neural Networks2019