MORE: Simultaneous Multi-View 3D Object Recognition and Pose Estimation
Preprint
Abstract
Simultaneous object recognition and pose estimation are two key functionalities for robots to safely interact with humans as well as environments. Although both object recognition and pose estimation use visual input, most state-of-the-art tackles them as two separate problems since the former needs a view-invariant representation while object pose estimation necessitates a view-dependent description. Nowadays, multi-view Convolutional Neural Network (MVCNN) approaches show state-of-the-art classification performance. Although MVCNN object recognition has been widely explored, there has been very little research on multi-view object pose estimation methods, and even less on addressing these two problems simultaneously. The pose of virtual cameras in MVCNN methods is often predefined in advance, leading to bound the application of such approaches. In this paper, we propose an approach capable of handling object recognition and pose estimation simultaneously. In particular, we develop a deep object-agnostic entropy estimation model, capable of predicting the best viewpoints of a given 3D object. The obtained views of the object are then fed to the network to simultaneously predict the pose and category label of the target object. Experimental results showed that the views obtained from such positions are descriptive enough to achieve a good accuracy score. Furthermore, we designed a real-life serve drink scenario to demonstrate how well the proposed approach worked in real robot tasks. Code is available online at: github.com/SubhadityaMukherjee/more_mvcnnKeywords
All Related Versions
This publication has 10 references indexed in Scilit:
- MVGrasp: Real-time multi-view 3D object grasping in highly cluttered environmentsRobotics and Autonomous Systems, 2022
- RotationNet for Joint Object Categorization and Unsupervised Pose Estimation from Multi-View ImagesIEEE Transactions on Pattern Analysis and Machine Intelligence, 2019
- MV-C3D: A Spatial Correlated Multi-View 3D Convolutional Neural NetworksIEEE Access, 2019
- Unsupervised Primitive Discovery for Improved 3D Generative Modeling2019
- Relation-Shape Convolutional Neural Network for Point Cloud Analysis2019
- RGB-D-Based Pose Estimation of Workpieces with Semantic Segmentation and Point Cloud RegistrationSensors, 2019
- Learning Multi-View Representation With LSTM for 3-D Shape Recognition and RetrievalIEEE Transactions on Multimedia, 2018
- Inductive Multi-Hypergraph Learning and Its Application on View-Based 3D Object ClassificationIEEE Transactions on Image Processing, 2018
- A Survey of Viewpoint Selection Methods for Polygonal ModelsEntropy, 2018
- Probabilistic roboticsCommunications of the ACM, 2002