Deep metric learning for visual servoing: when pose and image meet in latent space
Authors:
- 29 May 2023
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 741-747
Abstract
We propose a new visual servoing method that controls a robot's motion in a latent space. We aim to extract the best properties of two previously proposed servoing methods: we seek to obtain the accuracy of photometric methods such as Direct Visual Servoing (DVS), as well as the behavior and convergence of pose-based visual servoing (PBVS). Photometric methods suffer from limited convergence area due to a highly non-linear cost function, while PBVS requires estimating the pose of the camera which may introduce some noise and incurs a loss of accuracy. Our approach relies on shaping (with metric learning) a latent space, in which the representations of camera poses and the embeddings of their respective images are tied together. By leveraging the multimodal aspect of this shared space, our control law minimizes the difference between latent image representations thanks to information obtained from a set of pose embeddings. Experiments in simulation and on a robot validate the strength of our approach, showing that the sought out benefits are effectively found.Keywords
This publication has 29 references indexed in Scilit:
- Learning Deep Structure-Preserving Image-Text Embeddings2016
- Deep Residual Learning for Image Recognition2016
- Learning Image Representations Tied to Ego-Motion2015
- PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization2015
- Deep Metric Learning Using Triplet Network2015
- Control Camera and Light Source Positions using Image Gradient Information2007
- Image Moments: A General and Useful Set of Features for Visual ServoingIEEE Transactions on Robotics, 2004
- Visual Servoing from LinesThe International Journal of Robotics Research, 2002
- Relative end-effector control using Cartesian position based visual servoingIEEE Transactions on Robotics and Automation, 1996
- Multidimensional binary search trees used for associative searchingCommunications of the ACM, 1975