Few-shot Keypoint Detection with Uncertainty Learning for Unseen Species
Authors:
- 1 June 2022
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 19394-19404
Abstract
Current non-rigid object keypoint detectors perform well on a chosen kind of species and body parts, and require a large amount of labelled keypoints for training. Moreover, their heatmaps, tailored to specific body parts, cannot rec-ognize novel keypoints (keypoints not labelled for training) on unseen species. We raise an interesting yet challenging question: how to detect both base (annotated for training) and novel keypoints for unseen species given a few an-notated samples? Thus, we propose a versatile Few-shot Keypoint Detection (FSKD) pipeline, which can detect a varying number of keypoints of different kinds. Our FSKD provides the uncertainty estimation of predicted keypoints. Specifically, FSKD involves main and auxiliary keypoint representation learning, similarity learning, and keypoint localization with uncertainty modeling to tackle the local-ization noise. Moreover, we model the uncertainty across groups of keypoints by multivariate Gaussian distribution to exploit implicit correlations between neighboring keypoints. We show the effectiveness of our FSKD on (i) novel keypoint detection for unseen species, (ii) few-shot Fine-Grained Vi-sual Recognition (FGVR) and (iii) Semantic Alignment (SA) downstream tasks. For FGVR, detected keypoints improve the classification accuracy. For SA, we showcase a novel thin-plate-spline warping that uses estimated keypoint un-certainty under imperfect keypoint co respondences.Keywords
All Related Versions
This publication has 33 references indexed in Scilit:
- Mask R-CNN2017
- Deep Alignment Network: A Convolutional Neural Network for Robust Face Alignment2017
- Deep Residual Learning for Image Recognition2016
- Bilinear CNN Models for Fine-Grained Visual Recognition2015
- Building a bird recognition app and large scale dataset with citizen scientists: The fine print in fine-grained dataset collection2015
- DeepPose: Human Pose Estimation via Deep Neural Networks2014
- Deep Convolutional Network Cascade for Facial Point Detection2013
- Articulated Human Detection with Flexible Mixtures of PartsIEEE Transactions on Pattern Analysis and Machine Intelligence, 2012
- One-shot learning of object categoriesIEEE Transactions on Pattern Analysis and Machine Intelligence, 2006
- Principal warps: thin-plate splines and the decomposition of deformationsIEEE Transactions on Pattern Analysis and Machine Intelligence, 1989