MORE: simultaneous multi-view 3D object recognition and pose estimation
Open Access
- 20 June 2023
- journal article
- research article
- Published by Springer Nature in Intelligent Service Robotics
- Vol. 16 (4) , 497-508
Abstract
Simultaneous object recognition and pose estimation are two key functionalities for robots to safely interact with humans as well as environments. Although both object recognition and pose estimation use visual input, most state of the art tackles them as two separate problems since the former needs a view-invariant representation, while object pose estimation necessitates a view-dependent description. Nowadays, multi-view convolutional neural network (MVCNN) approaches show state-of-the-art classification performance. Although MVCNN object recognition has been widely explored, there has been very little research on multi-view object pose estimation methods, and even less on addressing these two problems simultaneously. The pose of virtual cameras in MVCNN methods is often pre-defined in advance, leading to bound the application of such approaches. In this paper, we propose an approach capable of handling object recognition and pose estimation simultaneously. In particular, we develop a deep object-agnostic entropy estimation model, capable of predicting the best viewpoints of a given 3D object. The obtained views of the object are then fed to the network to simultaneously predict the pose and category label of the target object. Experimental results showed that the views obtained from such positions are descriptive enough to achieve a good accuracy score. Furthermore, we designed a real-life serve drink scenario to demonstrate how well the proposed approach worked in real robot tasks. Code is available online at: https://github.com/SubhadityaMukherjee/more_mvcnn.Keywords
All Related Versions
This publication has 26 references indexed in Scilit:
- Multi-view 6D Object Pose Estimation and Camera Motion Planning Using RGBD Images2017
- SegICP: Integrated deep semantic segmentation and pose estimation2017
- Shape completion enabled robotic grasping2017
- Volumetric and Multi-view CNNs for Object Classification on 3D Data2016
- Multi-view Convolutional Neural Networks for 3D Shape Recognition2015
- VoxNet: A 3D Convolutional Neural Network for real-time object recognition2015
- A benchmark for best view selection of 3D objects2010
- ImageNet: A large-scale hierarchical image database2009
- Probabilistic roboticsCommunications of the ACM, 2002
- A mathematical theory of communicationACM SIGMOBILE Mobile Computing and Communications Review, 2001