Top-Down Saliency Detection Driven by Visual Classification
Preprint
Abstract
This paper presents an approach for top-down saliency detection guided by visual classification tasks. We first learn how to compute visual saliency when a specific visual task has to be accomplished, as opposed to most state-of-the-art methods which assess saliency merely through bottom-up principles. Afterwards, we investigate if and to what extent visual saliency can support visual classification in nontrivial cases. To achieve this, we propose SalClassNet, a CNN framework consisting of two networks jointly trained: a) the first one computing top-down saliency maps from input images, and b) the second one exploiting the computed saliency maps for visual classification. To test our approach, we collected a dataset of eye-gaze maps, using a Tobii T60 eye tracker, by asking several subjects to look at images from the Stanford Dogs dataset, with the objective of distinguishing dog breeds. Performance analysis on our dataset and other saliency bench-marking datasets, such as POET, showed that SalClassNet out-performs state-of-the-art saliency detectors, such as SalNet and SALICON. Finally, we analyzed the performance of SalClassNet in a fine-grained recognition task and found out that it generalizes better than existing visual classifiers. The achieved results, thus, demonstrate that 1) conditioning saliency detectors with object classes reaches state-of-the-art performance, and 2) providing explicitly top-down saliency maps to visual classifiers enhances classification accuracy.Keywords
All Related Versions
This publication has 35 references indexed in Scilit:
- Multi-level Net: A Visual Saliency Prediction Model2016
- Top-Down Neural Attention by Excitation Backprop2016
- Saliency Detection via Combining Region-Level and Pixel-Level Predictions with CNNs2016
- Intrinsic and extrinsic effects on image memorabilityVision Research, 2015
- SuperCNN: A Superpixelwise Convolutional Neural Network for Salient Object DetectionInternational Journal of Computer Vision, 2015
- Learning to predict eye fixations for semantic contents using multi-layer sparse networkNeurocomputing, 2014
- Training Object Class Detectors from Eye Tracking Data2014
- Saliency in Crowd2014
- Tag-Saliency: Combining bottom-up and top-down information for saliency detectionComputer Vision and Image Understanding, 2014
- Selective visual attention enables learning and recognition of multiple objects in cluttered scenesComputer Vision and Image Understanding, 2005